🔍 Read the full analysis: Why NVIDIA Nemotron 3.5 Puts Retrieval At The Center Of AI Agents on ThorstenMeyerAI.com
TL;DR
NVIDIA has expanded its Nemotron Retriever collection with five open models for multimodal embedding, reranking and document parsing. The company reports improved retrieval scores and 13% to 14% fewer agent inference tokens per query in selected tests, but the results have not been independently validated in the supplied material.
NVIDIA has expanded its Nemotron Retriever collection with five open models for embedding, reranking and document parsing, targeting AI agents that need to find evidence across text and image-rich documents, as detailed in the original analysis. The company says selected evaluations showed better retrieval accuracy alongside 13% to 14% fewer agent inference tokens per query, though the supplied announcement does not include independent validation of those results.
The new collection includes Nemotron 3.5 Embed 8B BF16, Embed 1B BF16 and Embed 1B NVFP4, plus Rerank 8B BF16 and Parse, a 631-million-parameter model. NVIDIA describes the embedding models as searching text and visual material, including tables, charts, infographics, datasheets and schematics. The reranker scores query-document pairs to help an agent put the most relevant results first, a capability relevant to AI agents handling tasks beyond code.
In tests involving ViDoRe v3, BRIGHT and BrowseComp-Plus, NVIDIA says it held the language model, tools, tasks, corpus and execution policy fixed while comparing retrieval setups. It reports that adding Nemotron 3.5 Rerank reduced agent inference tokens per query by about 14% with Embed 8B and 13% with Embed 1B, while improving retrieval accuracy. The source does not provide full methodology or results tables for those comparisons, including the evaluation details in the source report.
For a separate ViDoRe v3.1 comparison, NVIDIA reports an nDCG@10 score of 61.17 for Embed 8B, versus 52.23 for Llama Nemotron Embed VL 1B. Embed 1B BF16 scored 58.38, which the company says is 95.4% of the 8B model’s score. These are reported benchmark outcomes; they do not establish how the models will perform on other collections or agent workloads.
Retrieval Can Change Agent Costs
For agents working through large document collections, retrieval quality can affect both the evidence they use and the amount of repeated searching needed to reach an answer. If an agent repeatedly surfaces irrelevant material, it may spend additional inference tokens inspecting and refining results. NVIDIA’s reported evaluations connect retrieval accuracy with token use, making retrieval part of the agent’s operating cost rather than only a search-system concern.
The multimodal focus addresses another practical issue: relevant information can sit inside a chart, table or schematic instead of ordinary searchable text. Models that can retrieve such material may help agents handle document-heavy tasks where answers depend on evidence in several formats. The actual effect will depend on the document collection, hardware, workload and integration; the company’s benchmark figures are not a guarantee of savings or accuracy in production.
As an affiliate, we earn on qualifying purchases.
From Text Search to Visual Documents
NVIDIA presents Nemotron 3.5 Embed as an extension of its earlier Nemotron 3 Embed text embedding models. An embedding model represents queries and documents so a search system can find semantically related material. A reranker then compares candidate documents with the query to refine their order. In the workflow described by NVIDIA, an agent searches, inspects evidence and may search again based on what it finds.
The collection offers different model sizes and formats. NVIDIA positions Embed 8B BF16 for retrieval quality and Embed 1B BF16 as a smaller multimodal option. Embed 1B NVFP4 uses quantization on text and image encoders for inference efficiency on supported hardware. NVIDIA also describes fine-tuning recipes for adapting embeddings to custom collections. In chart and infographic experiments, it reports relative nDCG@10 gains of 18% for Embed 1B BF16 and 3% for Embed 8B BF16; those figures are company-reported results from the stated experiments.
NVIDIA says text retrieval was evaluated across 34 languages. For multilingual visual retrieval, it cites MIRACL Vision across 18 languages and KoViDoRe. The announcement says support may extend beyond the languages tested, but the supplied information does not establish benchmark results for additional languages.
“Adding Nemotron 3.5 Rerank reduces agentic token cost per query by approximately 14% with Nemotron 3.5 Embed 8B, and 13% with Nemotron 3.5 Embed 1B, while improving retrieval accuracy in both configurations.”
— NVIDIA, in its Hugging Face announcement
As an affiliate, we earn on qualifying purchases.
Benchmark and Release Details Missing
The supplied announcement does not state a publication date, provide full benchmark methodology or results tables, or cite independent replication of the reported scores and token reductions. Performance may vary across corpora, languages and agent tasks, and the source does not establish that the reported gains will carry over to enterprise deployments.
Details on Nemotron 3.5 Parse are especially limited: the material gives its parameter count but little information about capabilities, supported formats or evaluation. The supplied source also does not specify pricing, licensing terms, hardware needs beyond the NVFP4 note, release links or a schedule for broader availability. NVIDIA describes the models as open and commercially available, but those particulars are not included here.
As an affiliate, we earn on qualifying purchases.
Documentation and Independent Tests
Model documentation and release artifacts will be needed to check licensing, access, deployment requirements and the evaluation setup behind NVIDIA’s claims. Independent tests on enterprise document collections could show whether the reported retrieval improvements and token reductions hold across different workloads. More technical information about Parse would clarify how it fits into the collection alongside embedding and reranking.
Until those details are available in the source material, the measured results should be read as NVIDIA-reported outcomes from selected evaluations. The practical next step for prospective users is to compare the models on their own documents and agent tasks rather than assume the benchmark results transfer directly.
As an affiliate, we earn on qualifying purchases.
Key Questions
What did NVIDIA add to the Nemotron Retriever collection?
NVIDIA lists five open models: Embed 8B BF16, Embed 1B BF16, Embed 1B NVFP4, Rerank 8B BF16 and Parse, a 631-million-parameter model.
What token savings does NVIDIA report?
NVIDIA reports about 14% fewer agent inference tokens per query when using Rerank with Embed 8B, and about 13% fewer with Embed 1B, alongside improved retrieval accuracy in the selected evaluations.
Are the benchmark results independently verified?
The supplied announcement does not include independent replication. The scores and token reductions should be treated as NVIDIA-reported results from its stated evaluation setups.
What does the Parse model do?
The source identifies Parse as a 631-million-parameter model but provides little detail about its capabilities, supported document formats or evaluation. Those specifics remain unclear from the supplied material.
When and under what terms can the models be used?
NVIDIA describes the collection as open and commercially available, but the supplied source gives no release links, licensing details, pricing or broader availability schedule. Users would need the model documentation or release artifacts to confirm those terms.
Primary source: Hugging Face · via ThorstenMeyerAI.com
