Nanbeige 4.2-3B Review: A 3B Agentic Model That Punches Above Its Weight

Nanbeige 4.2-3B is a compact, open-weight language model built by the Nanbeige LLM Lab. It has only 3 billion non-embedding parameters—about 4 billion total—and yet it is being positioned as a fully agentic model capable of multi-step tool use, office workflows, coding agents and local personal-assistant tasks. The interesting claim is not just that it is small; it is that the architecture makes it competitive with much larger models.

The model is built on a Looped Transformer design that reuses the same transformer layers multiple times during a single forward pass. This increases effective depth without adding parameters. The result is a model that fits on consumer GPUs and, according to the Nanbeige team, beats systems like Qwen 3.5-9B and Gemma 4-12B on several agent and code benchmarks.

This review breaks down what Nanbeige 4.2-3B actually is, how the looped architecture works, what the published benchmarks claim, how to run it locally, and where it still needs caution.

What Nanbeige 4.2-3B is

Nanbeige 4.2-3B is a small decoder-only language model released as a full chat model on Hugging Face. Its base is Nanbeige 4.2-3B-Base, which was trained from scratch on 28 trillion tokens. The chat version then went through supervised fine-tuning across real and synthesized agent environments, followed by reinforcement learning that combined outcome and process rewards.

The model is explicitly designed for agentic workflows. The chat template exposes two flags:

• enable_thinking: turns internal reasoning on or off per response. It is on by default.
• preserve_thinking: keeps reasoning from previous assistant turns in multi-turn conversation. The team recommends False for general chat and True for multi-turn tool use, office tasks and code agents.

For the best tool-calling results, the Nanbeige team recommends tool_call_format="xml". JSON is available for compatibility.

How the Looped Transformer changes the math

A normal transformer stacks N distinct layers. A Looped Transformer takes the same layer block and runs it multiple times, routing the hidden state back through it. The idea is that a model can get deeper reasoning capacity without paying the full memory and parameter cost of training a larger stack.

Nanbeige 4.2-3B adds several specific architectural twists:

• LoopSplit: splits the loop into multiple sub-loops or branches so the model can refine different aspects of a representation in parallel.
• mHC with depth attention: a modified attention mechanism that conditions on the loop depth, so the model can treat earlier and later passes differently.
• Concatenated n-gram embeddings: mixes token embeddings with n-gram embeddings to give the model a richer local context signal.

These details are in the released modeling_nanbeige.py file on Hugging Face. The practical takeaway is that Nanbeige is trying to squeeze more effective capacity out of a small footprint by making the same parameters work harder, rather than simply scaling width and depth.

That matters because small models are easier to run locally, easier to quantize, cheaper to host and simpler to keep private. The trade-off is usually capability. If the Looped Transformer closes that capability gap even partially, it becomes a meaningful alternative architecture for edge and local AI.

Reported benchmark performance

The Nanbeige team publishes a wide comparison against Qwen 3.5 and Gemma 4 models. The headline is that Nanbeige 4.2-3B wins on many agent and reasoning tasks despite being far smaller.

General and office agent benchmarks

These numbers are vendor-reported and use the Nanbeige in-house scaffold for office tasks. They are not an independent audit, but they are the best public evidence currently available. Across every listed general-agent benchmark, Nanbeige 4.2-3B is ahead of Qwen 3.5-9B and Gemma 4-12B, even though it has roughly one-third the non-embedding parameters of Qwen 3.5-9B and one-quarter of Gemma 4-12B.

Code agent benchmarks

Software-engineering agent benchmarks are notoriously sensitive to scaffold, tool access and evaluation harness. Still, the gap here is large enough that Nanbeige 4.2-3B clearly deserves attention from anyone building coding agents on small models.

Reasoning benchmarks

The model is not always first: Gemma 4-12B is ahead on SciCode, and it ties on LiveCodeBench-V6. But on most of the listed math, science and competition-style tasks, Nanbeige 4.2-3B leads. The GPQA-Diamond score of 87.4 is especially high for a 3B model.

Alignment and safety benchmarks

Alignment is more mixed. Nanbeige 4.2-3B is competitive, but Gemma 4-12B leads on IF-Bench and Recruit-Bench. This is a reminder that benchmark wins do not mean the model is uniformly better across every dimension.

Important caveat: All benchmark values above are from the Nanbeige Hugging Face model card and associated technical report. They were not independently reproduced for this review. Evaluation settings, scaffolds and prompt templates matter enormously for agent and code benchmarks. Treat these as a strong signal, not a final verdict.

Local personal assistant: the OpenClaw angle

Nanbeige 4.2-3B is compact enough to run locally, and the team specifically tested it as a local personal assistant inside the OpenClaw scaffold. The idea is that the model can handle multi-step daily tasks, office work and research without sending everything to a cloud API.

Again, these are vendor-reported with the OpenClaw framework. But if they hold up under independent testing, the implication is interesting: a local 3B model could outperform a 9B cloud-dependent alternative on tasks where privacy and latency matter. That is exactly the use case small open models have been chasing for years.

How to run Nanbeige 4.2-3B

The model card gives clear instructions for four inference engines. The important detail is that each engine currently requires a Nanbeige-specific fork or branch for full support, because the Looped Transformer, XML tool parser and reasoning parser need code that is not yet in the main releases.

Hugging Face Transformers

The simplest path for experimentation:

1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model_id = "Nanbeige/Nanbeige4.2-3B"
4
5tokenizer = AutoTokenizer.from_pretrained(
6    model_id,
7    use_fast=False,
8    trust_remote_code=True
9)
10model = AutoModelForCausalLM.from_pretrained(
11    model_id,
12    torch_dtype="auto",
13    device_map="auto",
14    trust_remote_code=True
15)
16
17messages = [
18    {"role": "user", "content": "Which number is bigger, 9.11 or 9.8?"}
19]
20prompt = tokenizer.apply_chat_template(
21    messages,
22    add_generation_prompt=True,
23    tokenize=False
24)
25input_ids = tokenizer(
26    prompt,
27    add_special_tokens=False,
28    return_tensors="pt"
29).input_ids
30output_ids = model.generate(
31    input_ids.to("cuda"),
32    max_new_tokens=131072,
33    temperature=0.6,
34    top_p=0.95,
35    top_k=20,
36    eos_token_id=166101
37)
38response = tokenizer.decode(
39    output_ids[0][len(input_ids[0]):],
40    skip_special_tokens=True
41)
42print(response)

The model uses trust_remote_code=True because the modeling file is in the repository. This is normal for newer architectures, but it means you should review the code before running it in sensitive environments.

SGLang

1git clone -b nbg42 https://github.com/Nanbeige/sglang.git
2cd sglang
3pip install -e "python"
4
5MODEL_PATH=/path/to/Nanbeige4.2-3B
6python -m sglang.launch_server \
7  --model-path ${MODEL_PATH} \
8  --host 0.0.0.0 \
9  --port 8000 \
10  --tp-size 1 \
11  --mem-fraction-static 0.8 \
12  --reasoning-parser nanbeige \
13  --tool-call-parser nanbeige

SGLang is a good choice for high-throughput serving once the branch is installed.

vLLM

1git clone -b nanbeige42 https://github.com/Nanbeige/vllm.git
2cd vllm
3pip install -e .
4
5MODEL_PATH=/path/to/Nanbeige4.2-3B
6vllm serve ${MODEL_PATH} \
7  --host 0.0.0.0 \
8  --port 8000 \
9  --tensor-parallel-size 1 \
10  --gpu-memory-utilization 0.8 \
11  --enable-auto-tool-choice \
12  --tool-call-parser nanbeige \
13  --reasoning-parser nanbeige

vLLM is the standard production serving engine for many teams. The Nanbeige fork adds the tool-call and reasoning parsers.

llama.cpp

For pure CPU or quantized GPU inference, convert the Hugging Face model to GGUF:

1git clone -b nanbeige42 https://github.com/Nanbeige/llama.cpp.git
2cd llama.cpp
3cmake -B build -DGGML_CUDA=ON
4cmake --build build --config Release -j
5
6MODEL_PATH_HF=/path/to/Nanbeige4.2-3B
7MODEL_PATH_BF16_GGUF=/path/to/Nanbeige4.2-3B-BF16.gguf
8MODEL_PATH_GGUF_Q4_K_M=/path/to/Nanbeige4.2-3B-Q4_K_M.gguf
9
10python3 convert_hf_to_gguf.py ${MODEL_PATH_HF} \
11  --outfile ${MODEL_PATH_BF16_GGUF} \
12  --outtype bf16
13
14./build/bin/llama-quantize \
15  ${MODEL_PATH_BF16_GGUF} \
16  ${MODEL_PATH_GGUF_Q4_K_M} \
17  Q4_K_M
18
19./build/bin/llama-cli \
20  -m ${MODEL_PATH_GGUF_Q4_K_M} \
21  -ngl 99

Ollama

Ollama users can run a GGUF model or an MLX safetensors model on macOS arm64:

1git clone -b nanbeige42 https://github.com/Nanbeige/ollama.git
2cd ollama
3cmake -B build .
4cmake --build build --parallel $(nproc)
5
6cp -r /path/to/llama.cpp/build/bin/* build/lib/ollama/
7go build .
8
9./ollama serve
10./ollama run nanbeige/nanbeige4.2:3b-Q4_K_M
11
12cat > Modelfile <<EOF
13FROM /path/to/Nanbeige4.2-3B-Q4_K_M.gguf
14PARAMETER temperature 0.6
15PARAMETER top_p 0.95
16PARAMETER top_k 20
17EOF
18
19./ollama create nanbeige42-local -f Modelfile
20./ollama run nanbeige42-local

For MLX on Apple Silicon, the Modelfile uses FROM pointing to the Hugging Face directory with safetensors, plus RENDERER nanbeige and PARSER nanbeige.

Recommended inference settings

The model card suggests different temperature and token limits depending on the scenario:

For XML tool calling, the team recommends a top-p of 0.95 and top-k of 20. The context window is 256K tokens, which is generous for a 3B-class model and useful for long documents or extended agent state.

Limitations and caveats

Small models are improving rapidly, but the same rules still apply:

1. Vendor-reported benchmarks. The headline numbers come from the Nanbeige team. Independent reproduction is the next step before trusting them for production decisions.
2. Trust-remote-code. The Hugging Face implementation requires trust_remote_code=True. Audit the modeling file before using it in security-sensitive environments.
3. Fork dependency. SGLang, vLLM, llama.cpp and Ollama all need the Nanbeige branch right now. Mainline support is not guaranteed yet, and the branches may lag behind upstream releases.
4. Tool and agent scaffolds. The best results rely on specific tool parsers, XML tool formats and scaffolds. Swapping in an arbitrary JSON tool caller may not give the same scores.
5. Safety and alignment. The model card explicitly warns that the model may still produce inaccurate, biased or harmful outputs. Alignment benchmarks are good but not perfect.
6. Hardware assumptions. The published benchmarks are run on capable GPUs. A quantized GGUF on CPU will be much slower and may behave differently from the BF16 evaluation.

Who should care about Nanbeige 4.2-3B

This model is most interesting for three groups:

• Local AI builders. If you want a private, offline assistant or coding agent, a 3B model that runs on a single consumer GPU or even a high-end laptop is a very different proposition from a 70B+ model that needs multiple accelerators.
• Edge and embedded developers. Smaller models reduce hosting cost and latency. A looped architecture that improves capability without growing the footprint is directly relevant to edge deployment.
• Architecture researchers. If Looped Transformers with depth-attention and LoopSplit can consistently beat dense scaling at small sizes, the field has a new design space to explore beyond the default "bigger is better" path.

Final verdict

Nanbeige 4.2-3B is a notable release because it attacks the right problem: can a small model do real agent work? The published numbers suggest the answer is closer to yes than many people expected. It leads the displayed benchmarks in general-agent, code-agent and many reasoning tasks against much larger competitors, and it is explicitly designed to run locally with OpenClaw-style personal assistants.

The caveats are standard but real. Independent verification is needed, the inference stack requires project-specific forks, and alignment is not uniformly dominant. Still, if the results reproduce, Nanbeige 4.2-3B is one of the most credible small models yet for developers who want agentic capability without cloud dependency or massive hardware.

The real test is whether the architecture generalizes. If Nanbeige 4.5 and later models scale the same looped ideas upward while keeping inference efficiency, this could be more than a small-model trick. It could be a different route to useful AI.

Sources and further reading

• Nanbeige 4.2-3B on Hugging Face
• Nanbeige 4.2-3B-Base on Hugging Face
• Nanbeige 4.1-3B on Hugging Face
• Nanbeige 4.2-3B technical report on arXiv
• Reddit discussion on LocalLLM
• There’s An AI For That: Nanbeige 4.2-3B
• AI Adoption Agency overview

Editorial note: Benchmark scores are vendor-reported from the Nanbeige model card and technical report, unless otherwise noted. Recheck them before making hardware or commercial decisions.