Nvidia releases open-source stack to ground enterprise AI agents in context

Nvidia has introduced the Agent Knowledge Foundation, a new open-source collection designed to help enterprise AI agents make sense of complex internal data. Announced at the GTC Berlin conference on October 21, 2026, this toolkit aims to solve the context gap that often plagues large language models when they interact with structured business information. The move positions Nvidia as a provider of foundational intelligence layers rather than a direct competitor to existing data platforms.

What happened

The core challenge for enterprises today is not just connecting AI to data sources, but ensuring the AI understands the relationships within that data. While vendors like Snowflake, Databricks, and Palantir manage storage and access controls, they do not always provide the semantic understanding agents need to perform complex tasks. Nvidia’s new Agent Knowledge Foundation, added to its Agent Toolkit, bundles several key components: Nemotron Retriever models, an Auto-Ontology library, the Kumo family of models for structured data, and CUDA-X data processing libraries. This approach allows companies to keep their data in place while enhancing how agents interpret it.

Adel El Hallak, Nvidia’s vice president of product for agentic AI, highlighted the limitations of current setups. He noted that simply connecting an agent to systems like Snowflake or email exchanges does not magically grant the AI an understanding of the enterprise context. Large language models excel at predicting next tokens but struggle with specific queries across multi-tab spreadsheets or disparate databases. The new foundation seeks to bridge this gap by improving retrieval accuracy and mapping raw data to meaningful business concepts.

How it works

The foundation relies on three main technological pillars. First, the Nemotron Retriever embedding model improves search relevance by focusing on accuracy over speed for individual lookups. Nvidia claims this reduces the total number of search steps required, ultimately lowering overall latency. In tests on an open-source OpenSearch setup, the model improved search relevance by 65%. Partners like SAP and ServiceNow have reported significant performance gains, with SAP’s fine-tuned reranker running nearly three times faster and ServiceNow achieving better quality with 25% less latency.

Second, the Auto-Ontology library moves beyond simple retrieval-augmented generation (RAG) by building knowledge graphs. These graphs map relationships between concepts, such as linking "tier" to "plan" in mobile carrier data, which LLMs often confuse. This library was bolstered by Nvidia’s acquisition of Illumex, an Israeli startup specializing in automated semantic layers, for approximately $75 million in February. The library exposes a REST API and a Model Context Protocol server, allowing agents to query these structured relationships directly.

Third, the Kumo family of models handles predictions on relational data. Acquired by Nvidia in June in a deal valued at over $400 million, Kumo provides foundation models for tabular data. These models replace custom pipelines previously built for specific tasks like churn prediction or sentiment analysis. By harmonizing data through ontologies, Kumo models can drive consistent predictive analytics across different tables.

Key details

• Component bundle: The Agent Knowledge Foundation includes Nemotron Retriever models, Auto-Ontology library, Kumo models, and CUDA-X libraries.
• Performance gains: Nemotron Retriever raised search relevance by 65% in OpenSearch tests; SAP’s reranker ran three times faster.
• Ontology automation: The Auto-Ontology library builds knowledge graphs from company tables, mapping raw data to business concepts like customers and orders.
• Strategic acquisitions: Nvidia acquired Illumex for ~$75 million in February and Kumo for >$400 million in June to support this stack.
• Partner integration: The stack integrates with Palantir Foundry, Databricks Genie Ontology, and Snowflake Notebooks, leaving access control to partners.
• Distribution: Models are distributed via Hugging Face, following Nvidia’s $12.9 billion agreement to acquire the platform in September.

Why it matters

For software engineers and technical leads, this release signals a shift toward modular, open-source components for building agentic workflows. Instead of relying on monolithic vendor solutions for every layer of the AI stack, teams can now integrate specialized models for retrieval, semantic mapping, and prediction. This modularity allows for finer control over performance and cost, particularly when dealing with high-volume enterprise data where latency and accuracy are critical trade-offs.

Furthermore, the emphasis on ontologies addresses a persistent pain point in RAG implementations: the lack of structural understanding. By automating the creation of semantic layers, Nvidia reduces the manual effort required to align database schemas with natural language queries. This harmonization enables more reliable predictive analytics and reduces the hallucination risks associated with unstructured text retrieval. It also clarifies the division of labor in the enterprise AI ecosystem, with Nvidia providing the intelligence layers and partners like Snowflake managing security and data governance.

What you can do

• Evaluate the Nemotron Retriever models on your current search infrastructure to measure potential relevance improvements.
• Explore the Auto-Ontology library to map your existing database tables to business concepts using the provided REST API.
• Test Kumo models for tabular data tasks such as churn prediction or anomaly detection to replace custom-built pipelines.
• Review integration options with your current data platform, such as Snowflake Notebooks or Databricks Genie Ontology.
• Monitor the Hugging Face repository for updates to the open-weight Kumo and Nemotron models.
• Assess whether your team needs to build custom semantic layers or if the automated tools in the foundation suffice.