After the sandbox, Jensen Huang's GTC Berlin keynote turned to the thing that makes an agent specific to one company: its knowledge. The slide behind him was the Agent Knowledge Foundation. Unstructured documents take one path. Structured databases take another. This article follows that slide, then the partner examples he used for post-training Nemotron, without treating any of them as a Workstation case study.
Why a base model is not the agent's memory
Huang's order of parts was deliberate. First the harness, running in an OpenShell sandbox. Second, Nemotron as the language model. Those two, he said, are only the start. What turns an agent into something he called a super agent is access to the company's own knowledge. The libraries and models for that job are what NVIDIA put on one slide and named the Agent Knowledge Foundation.
Company data arrives in two shapes. Documents, PDFs and other files an AI has to read. And rows in a database. The foundation is the set of libraries that extract meaning from both.
The document path
Left to right on the unstructured row of the slide:
- Nemotron Parse reads the documents. Huang called it world-class and "top of leaderboard." He did not name the board. We will not invent one.
- Nemotron Embed turns a query and the parsed knowledge into vectors so they can be associated. He used the same "top of leaderboard" phrase.
- cuVS is the vector search. The slide puts it between Embed and Rerank. cuVS is NVIDIA's library for similarity search on the GPU.
- Nemotron Rerank takes the many passages the search returns and orders them by relevance to the question.
That is a retrieval pipeline, drawn as product names rather than as a generic "RAG" box. Parse, embed, search, rerank. The agent still has to be allowed to call it. In the five-element picture, this whole row is the knowledge element, reached through a tool the sandbox policy permits.
The database path
The structured row starts at Auto-Ontology, then splits.
- Analytical work goes to cuDF, the GPU dataframe library.
- Predictive work goes to Kumo. The keynote transcript renders this name as "QMO" and "CUMA." The slide in the hall spells it Kumo. We follow the slide.
Huang's example is the one to remember, because it uses both rows. A note that a supplier will be short, or late, arrives as a document. The orders that supplier feeds are in SQL. Kumo is how he said you predict which orders slip, and by how much. Finding a replacement component is a vector search back into engineering documents. One decision, two data shapes, two tools.
Open weights, then your data
Before the slide, Huang argued for open models you can inspect. Nemotron for language. Cosmos as the world foundation model. GR00T for humanoid robotics. Alpamayo for autonomous driving. Earth-2 for physics. He said NVIDIA publishes the weights, the data used to build the model, and the scripts to reproduce it, so a lab can see the provenance. "Providence" in the raw transcript is provenance.
He was also blunt about rank. These language models are not automatically the frontier of frontiers. Customisation, with partners, is how he said you turn them into a frontier model for one job, and turn proprietary data into what he called sovereign intelligence. "Super intelligence" later in the same passage is his phrase for the result of that post-training. It is a claim about fitness for a task, not a measured IQ.
The examples he named from the stage, in the published transcript, were these. We have not verified the customer engagements beyond the fact that he said them:
- A21 helping Alice, turning Nemotron into a model he said exceeds frontier capability on their task.
- Base10 helping CodeRabbit, fine-tuning Nemotron on that customer's data.
- Trajectory and Harvey, post-training Nemotron.
- Scale working with Skydance, the same pattern.
- Palantir with Tampa General Hospital.
The neocloud wheel later in the keynote spells a company Baseten. The transcript of this passage says Base10. We are not merging those into one firm. If you are CodeRabbit, Harvey, Skydance or Tampa General, the sentence above is Huang's, not a result we reproduced.
Tools are the last element
Knowledge without tools still leaves the agent as a reader. Huang's last beat was CUDA-X: domain libraries used in science, engineering, research and creative work. He called them the crown jewels, because an assistant that designs or calculates has to call an algorithm, not only a chat API. The RTX Spark demo later in the keynote put that in one breath: Hermes as the harness, Nemotron and a model the transcript calls "OpenAI's GPT-6 Astra," and CUDA-X tools including FreeCAD, Blender, Isaac Sim, the Newton physics engine and Isaac Lab. That demo was NVIDIA's, on stage, for a Hugging Face robot called Microduck. It is not a Workstation benchmark.
What to build first
If you are arming an agent with your own files, the Berlin slide is a checklist rather than a purchase order.
- Put the harness in a sandbox before you connect it to the file store. The OpenShell article is that half.
- Decide which knowledge is a document and which is a table. Do not send both through the same pipeline and hope.
- Parse and embed the documents. Search with a vector library. Rerank before the model sees the pile.
- Keep SQL as SQL. Use a dataframe library for analysis and a predictive model only where you have a question like Huang's supplier example.
- Post-train an open model only when the generic model is wrong in a way your data can fix. He said the technology exists. He did not say it is cheap or that every fine-tune wins.
Walk the supplier example
Put Huang's story in the order a system would actually run it.
- A PDF or email says a supplier will be late. That file hits Nemotron Parse.
- Nemotron Embed turns the delay note, and the query "which orders does this hit?", into vectors.
- cuVS searches the vector store. Nemotron Rerank keeps the passages that are actually about that supplier.
- The order book is not in that store. It is SQL. Auto-Ontology is the slide's name for making that structured world legible to the agent. cuDF is the analytical step over those rows. Kumo is the predictive step: who is short, and by how long.
- A replacement part is unstructured again: a drawing or a spec. Search returns it. The harness proposes an action. The sandbox decides whether that action is allowed.
Skip step 5 and you have a well-read model with a shell. That is the failure OpenShell is for.
When not to post-train
Huang's partner list is a list of teams who already had a task and data. It is not a reason to fine-tune on day one. If the generic model is wrong because it cannot see your files, fix retrieval first. If it can see the files and still uses the wrong terms, the wrong units, or the wrong policy, then post-training is the step he described: start from Nemotron, keep the recipe so you know what you changed, and test on your own traces. He said the result can reach the frontier for that task. He did not publish the scores. Neither will we.
CUDA-X is the tool shelf under that. He said one computer runs fluid dynamics, particle physics, quantum chemistry, data processing and ray tracing, and that the GPU path is on the order of 10, 50 or 100 times a native CPU run. That range was spoken on stage. It is not a single benchmark with a named dataset. Use it as his argument for why tools belong on a GPU, then measure the library you actually call.
Sources
- Keynote transcript, including the knowledge-foundation passage and the partner names: SB61112
- Slide photographed at the keynote: unstructured path Nemotron Parse, Nemotron Embed, cuVS, Nemotron Rerank; structured path Auto-Ontology, cuDF, Kumo.
- Companion: the keynote overview
