NVIDIA Nemotron 3.5 Super VL arrives in AWS SageMaker JumpStart

AWS has made NVIDIA Nemotron 3.5 Super VL available in Amazon SageMaker JumpStart, giving users deployment options for a multimodal model that accepts images, video and text and generates text. The service offers an NVFP4 variant for optimized inference and BF16 reference weights for customization.
- AWS added NVIDIA Nemotron 3.5 Super VL to SageMaker JumpStart.
- The model accepts image, video and text inputs and generates text.
- NVFP4 targets optimized inference, while BF16 supports customization.
- The model supports context lengths of up to 1 million tokens.
- Deployment is available through SageMaker Studio and the SageMaker Python SDK.
A 120B-parameter vision-language model
Nemotron 3.5 Super VL is the vision-language model in NVIDIA's Nemotron 3.5 family. It combines a 120B-parameter hybrid Mixture-of-Experts language backbone, with 12B active parameters per token, and a C-RADIO vision encoder.
The model supports context lengths of up to 1 million tokens. Its language backbone has 88 layers that interleave Mamba-2 and Mixture-of-Experts layers with eight attention layers, while the vision encoder processes images and sampled video frames before sending them into the language model through a vision-language connector.
Reasoning can be configured for each request. The model also supports tool calling and JSON schema output, and includes a Multi-Token Prediction head that supports speculative decoding. However, AWS says speculative decoding is not enabled in the JumpStart serving configuration.
Two variants and multimodal use cases
JumpStart provides two versions. NVFP4 is quantized from BF16 using quantization-aware distillation and is intended for optimized inference. It uses 4-bit weights and activations on Blackwell GPUs, has a checkpoint size of about 75 GB, and runs by default on the ml.g7.24xlarge instance. The BF16 version has a checkpoint size of about 249 GB and provides full-precision reference weights for supervised fine-tuning, reinforcement learning, distillation, domain adaptation and custom quantization.
Both variants accept image, video and text inputs and produce text. They support English, French, German, Italian, Japanese, Spanish and Chinese. NVIDIA designed Super VL for document intelligence, graphical user interface understanding, video question answering and multimodal agentic applications.
Potential workloads include optical character recognition, chart and table analysis, multipage document understanding, screen grounding for computer-use agents, and analysis of short MP4 clips. AWS says the JumpStart serving configuration accepts one image and one video per request. The endpoint serves base weights only and does not load LoRA adapters.
Performance and deployment options
AWS measured the model using a workload with 250 input tokens, 1,500 output tokens and five concurrent streaming requests. NVFP4 reached mean output throughput of 63.07 tokens per second per request on ml.g7.24xlarge and 64.01 tokens per second on ml.g7.48xlarge. BF16 reached 140.85 tokens per second on ml.p5en.48xlarge. Mean time to first token was 0.45 seconds, 0.51 seconds and 0.53 seconds respectively.
AWS cautions that the measurements compare different GPU systems and configurations, so they are not an isolated BF16 versus NVFP4 precision comparison. With a 10,000-token input under the summarize profile, time to first token on ml.g7.24xlarge was 12.62 seconds.
Users can deploy the model through SageMaker Studio, the SageMaker Python SDK or the Hugging Face model page. NVFP4 is supported on G7 instances with NVIDIA RTX PRO 4500 Blackwell Server Edition GPUs, while BF16 runs on P5en instances with NVIDIA H200 GPUs. Deployment requires an AWS account with permission to create Amazon SageMaker AI endpoints, an appropriate IAM role and service quota for the selected instance type.


