How to Run Nex-N2.5 Mini Locally on RunPod (Dual H100 Setup)
A practical guide to deploying Nex-N2.5 Mini on RunPod using dual H100 GPUs, an SGLang Docker template, and correct VRAM sizing.
How do you run Nex-N2.5 Mini on RunPod?
You run Nex-N2.5 Mini on RunPod by provisioning a pod with two H100 GPUs (or any 80GB-class equivalent), building a custom template around the SGLang Docker image that Nex AGI publishes on the model card, attaching enough persistent storage for the weights, and launching the model with tensor parallelism set to 2. Once served, you get an OpenAI-compatible endpoint you can hit over SSH or the web terminal, with inference speeds that reportedly hit over 200 tokens per second on this hardware.
TL;DR
- Nex-N2.5 Mini is the smaller model in Nex AGI’s N2.5 agentic family, built for long-horizon tasks like operating computers, browsing the web, and self-correcting with visual feedback rather than just answering a single prompt.
- The model card specifies two GPUs at 80GB VRAM each (H100 or equivalent) as the minimum viable setup, which rules out most single-GPU consumer rigs.
- On RunPod, deployment centers on creating a custom template using Nex AGI’s own SGLang Docker container rather than relying on a generic vLLM or Ollama template.
- Actual VRAM consumption in testing landed around 66GB per GPU across two H100s, leaving some headroom for context and batching.
- The model launches with tensor parallelism (TP) set to 2 to split the workload across both GPUs evenly.
- In hands-on testing, the model handled multilingual translation, HTML/canvas generation, and vision-based role-play prompts with results described as competitive but not frontier-leading against models like Claude Opus and Gemini 3.
- Benchmark positioning puts N2.5 Mini’s larger sibling, N2.5 Pro, in the middle of the pack against GLM 5.3, DeepSeek V4, and GPT 5.6, rather than at the top.
Plans first. Then code.
Remy writes the spec, manages the build, and ships the app.
What is Nex-N2.5 Mini and why does it need this much hardware?
Nex-N2.5 Mini comes from Nex AGI’s N2.5 family, a line of models designed around agentic workflows instead of single-turn chat. That means the model is built to take actions, browse, operate interfaces, and iterate on its own output using feedback loops, including visual feedback when working with images or screenshots. This kind of looped, multi-step reasoning is heavier on compute and context than a typical chatbot interaction, which is part of why even the “mini” variant in the lineup still asks for two 80GB-class GPUs rather than something you’d run on a single consumer card.
The model is also multimodal and multilingual by design. Hands-on testing that pushed it through vision prompts, dozens of languages, and generative coding tasks showed it producing coherent, structured output across all three domains, which suggests the parameter count and architecture are doing meaningful work rather than padding a spec sheet.
What GPU and VRAM setup does Nex-N2.5 Mini actually require?
The official guidance is two GPUs with 80GB of VRAM each, satisfied by H100s or comparable cards. In practice, running the model with SGLang and tensor parallelism split across two H100s produced VRAM usage of roughly 66GB per card. That leaves a reasonable buffer above the raw model footprint for KV cache, longer context windows, and concurrent requests, but it also means you shouldn’t try to squeeze this onto anything smaller than the recommended 80GB-per-GPU baseline. Attempting a single-GPU deployment, or substituting cards with less memory, isn’t something the model card supports and isn’t likely to work without aggressive quantization that would fall outside the officially tested configuration.
For storage, 150GB of persistent disk was sufficient to hold the downloaded weights and container overhead in testing, though your own number can flex depending on whether you’re layering additional tools or checkpoints.
How do you set up the RunPod template for this model?
The key step in a RunPod deployment isn’t just picking the GPU, it’s building the right template. Here’s the general flow:
- Open the Pods section on the RunPod dashboard and use the GPU filter slider to select 80GB-class cards.
- Choose H100 as the GPU type and set the GPU count to two.
- Instead of picking a pre-built community template (RunPod ships plenty for ComfyUI, Ollama, vLLM, and others), click “New Template” and point it at the SGLang Docker image that Nex AGI provides on the model’s card page. This is the detail that makes the deployment work correctly, since Nex-N2.5 Mini is packaged around SGLang rather than a generic inference server.
- Set the compute type to Nvidia GPU and the template type to Pod if you want an always-on, persistently controlled machine. RunPod also offers a serverless template type that autoscales per request, which suits bursty or infrequent workloads better than a pod you’re paying for continuously.
- Allocate persistent storage (150GB worked in testing) and save the template.
- Go back to Deploy, confirm the two-GPU, 80GB configuration and your new template are selected, and launch the pod.
Remy doesn't write the code. It manages the agents who do.
Remy runs the project. The specialists do the work. You work with the PM, not the implementers.
Provisioning is reportedly fast, and once the pod is live you can connect either through SSH from your own terminal or through RunPod’s built-in web terminal, whichever you prefer for managing the container.
How do you actually launch and serve the model once the pod is running?
After SSH-ing into the container, the workflow is straightforward: download the model weights (which came down quickly in testing, suggesting solid network throughput on RunPod’s infrastructure), then launch SGLang with the GPU count and tensor parallelism explicitly set to match your two-GPU setup. In the demonstrated configuration, the model was served with Hermes-style agent formatting enabled, which matters if you’re planning to use it for tool calling or agentic workflows rather than plain chat completion.
Once serving starts, you can confirm everything is working correctly by checking GPU memory allocation (expect roughly 66GB per card on two H100s) and by sending a test prompt to verify throughput. Reported speeds reached over 200 tokens per second in this setup, which is fast enough for interactive use and for agent loops that require multiple back-and-forth reasoning steps.
Is Nex-N2.5 Mini worth self-hosting compared to frontier models?
It depends on what you need. Benchmark comparisons put the N2.5 family in the middle of the pack, trailing frontier models like Claude Opus and Gemini 3 on coding and agentic benchmarks, and behind Claude on multimodal tasks specifically. N2.5 Pro (the larger sibling) reportedly holds its own against GLM 5.3, DeepSeek V4, and GPT 5.6 on several tests without leading any of them outright.
That positioning matters for a self-hosting decision. If you need the single best model regardless of cost, an API call to a frontier provider will likely outperform Nex-N2.5 Mini on raw capability. But if you want a competitive, genuinely agentic, multilingual, multimodal model that you control end to end, that runs on hardware you can provision on demand rather than own outright, and that you can fine-tune or integrate into a custom pipeline, this is a reasonable mid-tier option. Hands-on testing showed it handling nuanced multilingual translation requests (correctly flagging languages it wasn’t confident about rather than fabricating answers), generating a moderately complex self-contained HTML/canvas simulation, and giving a decisive, well-reasoned response to an ambiguous vision-based role-play prompt.
Frequently Asked Questions
How much VRAM does Nex-N2.5 Mini need?
The model card calls for two GPUs with 80GB of VRAM each. In actual deployment on two H100s, usage came out to roughly 66GB per GPU, leaving headroom for context and inference overhead.
Can I run Nex-N2.5 Mini on a single GPU?
Not with the officially supported configuration. The model card specifies two 80GB-class GPUs, and testing was done using tensor parallelism across two H100s rather than a single-card setup.
Why does the RunPod deployment use SGLang instead of vLLM or Ollama?
Nex AGI packages and publishes its own SGLang Docker container specifically for this model family, so building the RunPod template around that image rather than a generic inference server ensures compatibility with the model’s expected serving configuration.
How fast is inference once the model is deployed?
One coffee. One working app.
You bring the idea. Remy manages the project.
Testing on a two-H100 RunPod pod showed throughput above 200 tokens per second, which is fast enough for interactive chat and for multi-step agentic workflows that require several reasoning passes.
How does Nex-N2.5 Mini compare to frontier models like Claude Opus or Gemini?
It trails frontier models on most coding and agentic benchmarks and on multimodal tasks specifically. It’s better understood as a competitive mid-tier agentic model, landing in similar territory as GLM 5.3, DeepSeek V4, and GPT 5.6 rather than leading the pack outright.
