Possible NVFP4 Loading Issue with Qwen3.6-35B-A3B-NVFP4

[YuDsheng]

Qwen3.6-35B-A3B-NVFP4 Vision model loads with MISSING MoE weights and generates corrupted multilingual output

1. Did you update? pip install --upgrade unsloth unsloth_zoo

Yes.

Studio version:

v0.1.45-beta

Package version:

2026.6.2

---

2. Colab or Kaggle or local / cloud

Local Linux server.

---

3. Number GPUs used, use nvidia-smi

1 GPU

Hardware detected:

NVIDIA GB10
Max memory: 121.69 GB

During generation:

GPU Memory Usage: ~65-80GB
GPU Utilization: ~90-95%

---

4. Which notebook? Please link!

Unsloth Studio local deployment.

Not using a public notebook.

---

5. Which Unsloth version, TRL version, transformers version, PyTorch version?

From logs:

Unsloth 2026.6.2
Transformers 5.5.0
Torch 2.10.0+cu130
CUDA Toolkit 13.0
Triton 3.6.0

Additional package versions if needed:

pip show unsloth
pip show unsloth_zoo
pip show transformers
pip show trl
pip show compressed-tensors
python -c "import torch; print(torch.__version__)"

---

6. Which trainer? SFTTrainer, GRPOTrainer etc

No training.

Inference only.

---

Model

model_name = "unsloth/Qwen3.6-35B-A3B-NVFP4"

---

Environment Notes

During startup I see:

Your Flash Attention 2 installation seems to be broken.
Using Xformers instead.

and

The fast path is not available because one of the required library is not installed.
Falling back to torch implementation.

I understand this may affect performance, but I do not think it explains the corrupted outputs described below.

---

Model Detection

The model is detected as:

model_type=qwen3_5_moe
architectures=['Qwen3_5MoeForConditionalGeneration']
is_vision=True

Therefore Studio correctly identifies it as a Vision model.

---

Loading Report

After loading weights successfully, I get the following report:

Qwen3_5MoeForConditionalGeneration LOAD REPORT from: unsloth/Qwen3.6-35B-A3B-NVFP4

Key                                                                      | Status
-------------------------------------------------------------------------+-----------
model.layers.{0...39}.mlp.experts.{0...255}.down_proj.input_global_scale | UNEXPECTED
model.layers.{0...39}.mlp.experts.{0...255}.up_proj.input_global_scale   | UNEXPECTED
model.layers.{0...39}.mlp.experts.{0...255}.gate_proj.input_global_scale | UNEXPECTED
model.layers.{0...39}.mlp.experts.down_proj_scale                        | UNEXPECTED
model.layers.{0...39}.mlp.experts.down_proj_global_scale                 | UNEXPECTED
model.layers.{0...39}.mlp.experts.down_proj_packed                       | UNEXPECTED
model.layers.{0...39}.mlp.experts.gate_up_proj_packed                    | UNEXPECTED
model.layers.{0...39}.mlp.experts.gate_up_proj_scale                     | UNEXPECTED
model.layers.{0...39}.mlp.experts.gate_up_proj_global_scale              | UNEXPECTED

model.language_model.layers.{0...39}.mlp.experts.gate_up_proj            | MISSING
model.language_model.layers.{0...39}.mlp.experts.down_proj               | MISSING

The loader also reports:

MISSING: those params were newly initialized because missing from the checkpoint.
Consider training on your downstream task.

---

Observed Behavior

The model loads successfully:

Successfully loaded model: unsloth/Qwen3.6-35B-A3B-NVFP4

and memory usage appears reasonable:

GPU Memory [After loading]
64.53GB / 121.69GB

However, generation behavior is abnormal.

---

Reproduction

Prompt:

你是什么模型？

No images attached.

Generation starts normally:

Starting text generation

and eventually finishes:

Finished text generation

Generation duration:

324.47 seconds

---

Actual Output

The model produces corrupted multilingual text consisting of random fragments from multiple languages:

对她只开始...
tail突然还是个...
stro Collaboration...
organis橱窗...
总书记不愿...
double bouncing...
[blocked]

The output is completely unrelated to the prompt.

The response contains:

• Chinese fragments
• English fragments
• Arabic fragments
• Japanese fragments
• Random tokens
• Broken words
• Repeated token patterns

It appears similar to a corrupted decode or partially initialized model rather than a normal language model response.

---

Why I Suspect a Loading Issue

The model:

• Loads successfully
• Consumes expected GPU memory
• Completes generation

However:

• Generation takes over 5 minutes for a trivial prompt
• Output is nonsensical
• Output is unrelated to the prompt
• Load report shows many MoE expert weights as MISSING

The suspicious part is:

UNEXPECTED:
*_packed
*_scale
*_global_scale

MISSING:
gate_up_proj
down_proj

which appear to be core MoE expert projection layers.

This makes me wonder whether:

1. NVFP4 packed expert weights are not being correctly mapped.

2. Some expert layers are being randomly initialized.

3. There is a compatibility issue between:

   • Qwen3.6-35B-A3B-NVFP4
   • Transformers 5.5.0
   • Unsloth Studio 2026.6.2

4. The checkpoint format is not being fully restored during loading.

---

Questions

1. Are these UNEXPECTED and MISSING entries expected for this checkpoint?

2. Should the following be present after loading?

gate_up_proj
down_proj

3. Are NVFP4 packed tensors:

*_packed
*_scale
*_global_scale

supposed to be automatically converted into standard expert projection layers?

4. Could these missing expert projections explain the corrupted multilingual output?

5. Is there a known issue with:

unsloth/Qwen3.6-35B-A3B-NVFP4

under:

Unsloth Studio v0.1.45-beta
Unsloth 2026.6.2
Transformers 5.5.0

6. Is this load report expected, or does it indicate an architecture / checkpoint compatibility problem?

Thanks for your help.

[Shashank200345]

Qwen3.6-35B-A3B-NVFP4 Vision model loads with MISSING MoE weights and generates corrupted multilingual output

1. Did you update? pip install --upgrade unsloth unsloth_zoo

Yes.

Studio version:

v0.1.45-beta

Package version:

2026.6.2

2. Colab or Kaggle or local / cloud

Local Linux server.

3. Number GPUs used, use nvidia-smi

1 GPU

Hardware detected:

NVIDIA GB10
Max memory: 121.69 GB

During generation:

GPU Memory Usage: ~65-80GB
GPU Utilization: ~90-95%

4. Which notebook? Please link!

Unsloth Studio local deployment.

Not using a public notebook.

5. Which Unsloth version, TRL version, transformers version, PyTorch version?

From logs:

Unsloth 2026.6.2
Transformers 5.5.0
Torch 2.10.0+cu130
CUDA Toolkit 13.0
Triton 3.6.0

Additional package versions if needed:

pip show unsloth
pip show unsloth_zoo
pip show transformers
pip show trl
pip show compressed-tensors
python -c "import torch; print(torch.version)"

6. Which trainer? SFTTrainer, GRPOTrainer etc

No training.

Inference only.

Model

model_name = "unsloth/Qwen3.6-35B-A3B-NVFP4"

Environment Notes

During startup I see:

Your Flash Attention 2 installation seems to be broken.
Using Xformers instead.

and

The fast path is not available because one of the required library is not installed.
Falling back to torch implementation.

I understand this may affect performance, but I do not think it explains the corrupted outputs described below.

Model Detection

The model is detected as:

model_type=qwen3_5_moe
architectures=['Qwen3_5MoeForConditionalGeneration']
is_vision=True

Therefore Studio correctly identifies it as a Vision model.

Loading Report

After loading weights successfully, I get the following report:

Qwen3_5MoeForConditionalGeneration LOAD REPORT from: unsloth/Qwen3.6-35B-A3B-NVFP4

Key                                                                      | Status
-------------------------------------------------------------------------+-----------
model.layers.{0...39}.mlp.experts.{0...255}.down_proj.input_global_scale | UNEXPECTED
model.layers.{0...39}.mlp.experts.{0...255}.up_proj.input_global_scale   | UNEXPECTED
model.layers.{0...39}.mlp.experts.{0...255}.gate_proj.input_global_scale | UNEXPECTED
model.layers.{0...39}.mlp.experts.down_proj_scale                        | UNEXPECTED
model.layers.{0...39}.mlp.experts.down_proj_global_scale                 | UNEXPECTED
model.layers.{0...39}.mlp.experts.down_proj_packed                       | UNEXPECTED
model.layers.{0...39}.mlp.experts.gate_up_proj_packed                    | UNEXPECTED
model.layers.{0...39}.mlp.experts.gate_up_proj_scale                     | UNEXPECTED
model.layers.{0...39}.mlp.experts.gate_up_proj_global_scale              | UNEXPECTED

model.language_model.layers.{0...39}.mlp.experts.gate_up_proj            | MISSING
model.language_model.layers.{0...39}.mlp.experts.down_proj               | MISSING

The loader also reports:

MISSING: those params were newly initialized because missing from the checkpoint.
Consider training on your downstream task.

Observed Behavior

The model loads successfully:

Successfully loaded model: unsloth/Qwen3.6-35B-A3B-NVFP4

and memory usage appears reasonable:

GPU Memory [After loading]
64.53GB / 121.69GB

However, generation behavior is abnormal.

Reproduction

Prompt:

你是什么模型？

No images attached.

Generation starts normally:

Starting text generation

and eventually finishes:

Finished text generation

Generation duration:

324.47 seconds

Actual Output

The model produces corrupted multilingual text consisting of random fragments from multiple languages:

对她只开始...
tail突然还是个...
stro Collaboration...
organis橱窗...
总书记不愿...
double bouncing...
[blocked]

The output is completely unrelated to the prompt.

The response contains:

• Chinese fragments
• English fragments
• Arabic fragments
• Japanese fragments
• Random tokens
• Broken words
• Repeated token patterns

It appears similar to a corrupted decode or partially initialized model rather than a normal language model response.

Why I Suspect a Loading Issue

The model:

• Loads successfully
• Consumes expected GPU memory
• Completes generation

However:

• Generation takes over 5 minutes for a trivial prompt
• Output is nonsensical
• Output is unrelated to the prompt
• Load report shows many MoE expert weights as MISSING

The suspicious part is:

UNEXPECTED:
*_packed
*_scale
*_global_scale

MISSING:
gate_up_proj
down_proj

which appear to be core MoE expert projection layers.

This makes me wonder whether:

1. NVFP4 packed expert weights are not being correctly mapped.

2. Some expert layers are being randomly initialized.

3. There is a compatibility issue between:

   • Qwen3.6-35B-A3B-NVFP4
   • Transformers 5.5.0
   • Unsloth Studio 2026.6.2

4. The checkpoint format is not being fully restored during loading.

Questions

1. Are these UNEXPECTED and MISSING entries expected for this checkpoint?
2. Should the following be present after loading?

gate_up_proj
down_proj

3. Are NVFP4 packed tensors:

*_packed
*_scale
*_global_scale

supposed to be automatically converted into standard expert projection layers?

4. Could these missing expert projections explain the corrupted multilingual output?
5. Is there a known issue with:

unsloth/Qwen3.6-35B-A3B-NVFP4

under:

Unsloth Studio v0.1.45-beta
Unsloth 2026.6.2
Transformers 5.5.0

6. Is this load report expected, or does it indicate an architecture / checkpoint compatibility problem?

Thanks for your help.

Thanks for the detailed report — this looks like a checkpoint-loading/key-mapping issue rather than a hardware or generation-config problem.
Looking at the load report, the checkpoint stores MoE expert weights in NVFP4-packed form (gate_up_proj_packed, down_proj_packed, plus *_scale / *_global_scale tensors), but Qwen3_5MoeForConditionalGeneration expects dense experts.gate_up_proj / experts.down_proj tensors. Since the packed/scale keys aren't being consumed, they're flagged UNEXPECTED, and since the dense keys never appear, they get MISSING and are randomly initialized.
That would fully explain the symptoms:

All other weights (attention, embeddings, etc.) load fine → reasonable memory usage and "successful" load
All 40 layers of MoE expert FFNs are random noise → garbage multilingual output regardless of prompt
Generation time is high and unrelated to prompt content, consistent with the FFN output being essentially noise

I checked model.safetensors.index.json for this repo and confirmed the checkpoint only contains the _packed/_scale/_global_scale naming for mlp.experts. — no plain gate_up_proj/down_proj tensors exist, so there's nothing for the current loader to pick up for these layers.
This suggests the NVFP4 dequant/unpack + key-remap step that exists for other architectures hasn't been wired up for qwen3_5_moe's expert layers yet (or there's a dispatch issue if compressed-tensors already supports MoE NVFP4 decompression).
I'd like to dig into this, happy to put together a minimal repro (synthetic checkpoint with the same key structure, on a tiny config) and look at where the remapping should plug in. If a maintainer can point me to the relevant loading code for qwen3_5_moe / NVFP4, that'd help me get oriented faster, but I'll start tracing it either way.