NVIDIA Nemotron Bolt E4B: 4GB Hardware and Limits
NVIDIA's new local agent model is not a 4 GB download. Compare actual VRAM tests, model size, text-only limits, licensing and setup.
NVIDIA released Nemotron 3.5 Bolt E4B 4GB on October 23, 2026, targeting fast, locally deployed conversational agents rather than giant general-purpose chatbots. The practical question for developers is whether its compact inference footprint, tool-use features and open weights make it suitable for a real device. The name needs care: the official Q4_K_M model download is 6.18 GB, and NVIDIA's approximately 4 GB memory target is not a universal hardware requirement.
What NVIDIA actually released
Bolt E4B is post-trained from Gemma 4 E4B-it and has 8 billion total parameters, of which NVIDIA describes 4 billion as effective. Its Q4_K_M GGUF checkpoint accepts text and produces text, supports a context window up to 64,000 tokens, and provides chat-template controls for reasoning and concise conversational replies. NVIDIA also offers a separate BF16 checkpoint for customization and a DFlash drafter for optional speculative decoding. The release targets device assistants, game characters, robotics interfaces and other agents that must respond quickly while using tools.
The important numbers
Does the 4GB label match real hardware?
Not for every prompt or device. NVIDIA says the model can fit into roughly 4 GB for its intended short-context workloads, but its own llama.cpp measurements vary with hardware and context. With about 3,000 input tokens and 50 output tokens at batch size one, the company reports 3.5 GB VRAM on an RTX 5070 and 4.2 GB on a Jetson Orin Nano 8GB. Longer cached-context tests consume more: the reported Orin Nano figure rises to 4.9 GB. These are vendor-run benchmarks, not independent compatibility guarantees.
Selected NVIDIA-measured Q4_K_M inference results
| Hardware / workload | Peak VRAM | Generation |
|---|---|---|
| RTX 5070; 3K input, 50 output | 3.5 GB | 133.4 tokens/s |
| Jetson Orin Nano 8GB; 3K input, 50 output | 4.2 GB | 15.5 tokens/s |
| Jetson Orin Nano 8GB; 32K cached, 2K input, 256 output | 4.9 GB | 13.0 tokens/s |
Where the model helps—and where it does not
A small on-device assistant is useful when latency, privacy architecture, offline operation or per-request cloud fees matter. Bolt E4B's text interface and tool-call support make it a candidate for a voice assistant's language layer or a game's interactive dialogue engine. But audio recognition, vision, action permissions and reliable external tool execution remain separate application components. A 64K advertised context does not mean a low-memory device can serve the entire window at the same footprint.
NVIDIA's own comparison is mixed rather than a universal win. It reports stronger instruction-hierarchy results for Bolt E4B than Qwen 3.5 4B, but Qwen leads on several tool-use and long-context benchmarks, including Tau2-Bench and RULER. The figures are vendor measurements using specified evaluation settings. Teams should run their own task-based tests, especially for actions with real-world consequences, instead of treating a single score or the model's small memory target as a production readiness certificate.
How to evaluate it safely
- Download the official NVIDIA Q4_K_M GGUF from the model card and verify the runtime supports the Gemma 4 chat template.
- Start with NVIDIA's documented llama.cpp example, a short context and one controlled text-only conversation.
- Test real prompts and tool calls while measuring peak RAM/VRAM, first-token latency and successful task completion on your actual device.
- Add approval, validation and permissions around tool actions; test failure and recovery paths before production use.
- Review the OpenMDW-1.1 model license and the separate NVIDIA API trial terms if using hosted inference.
The model weights are downloadable and NVIDIA lists commercial and non-commercial use under OpenMDW-1.1. A hosted NVIDIA playground is also listed, but trial access and data-handling terms are distinct from self-hosting rights. Neither route establishes a guaranteed free production API, a universal 4 GB GPU requirement, or independent benchmark validation.
Questions people ask
Is NVIDIA Nemotron 3.5 Bolt E4B really a 4 GB download?
No. The official NVIDIA Q4_K_M GGUF checkpoint is listed as a 6.18 GB download on Hugging Face. The “4GB” name refers to NVIDIA's approximate memory target for certain inference workloads, not the file size. The full-precision BF16 checkpoint is a separate, larger release. Budget disk space, runtime memory and context-cache memory independently when planning a local installation.
Can I run Bolt E4B on a GPU with exactly 4 GB of VRAM?
Not reliably on that specification alone. NVIDIA's Q4_K_M tests report about 3.1–5.0 GB of VRAM depending on device and context, and a Jetson Orin Nano 8GB example uses 4.2 GB for a short prompt. Memory needs also change with runtime settings and application components. Test your actual context length and hardware before assuming a 4 GB card is sufficient.
Does Nemotron Bolt E4B process images, speech or video directly?
No. NVIDIA describes the released Q4_K_M Bolt E4B checkpoint as text input and text output. It can serve as the language and tool-use component of a voice or robotics system, but speech recognition, speech synthesis and visual perception require other components. NVIDIA's robot demonstration pairs Bolt with a separate vision model, so the combined system should not be mistaken for native multimodal support in Bolt.
Can I use the open weights commercially or access a hosted API?
NVIDIA says Bolt E4B is available for commercial and non-commercial use under the OpenMDW-1.1 license, which developers should review before redistribution or deployment. NVIDIA also provides a hosted model playground and API documentation under separate trial terms. Those terms can include input/output logging and access conditions; a hosted demo does not guarantee unrestricted or free production API service.
Bolt E4B is a practical candidate for responsive, text-based local agents with tool calls. Its open GGUF release and measured small-device results are useful, but “4GB” is not a file size or universal runtime promise. Choose it after checking the actual device footprint, task accuracy, application safety controls and license.
Explore related Make Better resources
- NVIDIA Open Agent Safety: controls outside the model— Related governance and permissions for tool-using agents
Sources & useful resources
- NVIDIA: Nemotron Bolt E4B technical launch— Official Oct 23 release, workload benchmarks and device VRAM measurements
- NVIDIA official Hugging Face model card— Official checkpoint, model capabilities, 64K context, deployment instructions and license
- NVIDIA official Q4_K_M GGUF file— 6.18 GB download size verified from official file listing
- NVIDIA NIM model card and API terms— Hosted access, terms and model architecture