navid¶

NaVid (Zhang et al., RSS 2024) and Uni-NaVid (Zhang et al., RSS 2025) behind the arena_planners bridge. Both are video-based vision-language models (Vicuna-7B with an EVA-ViT-g encoder, LLaMA-VID lineage) that follow a natural-language instruction from egocentric RGB alone, without a goal pose or depth. One planner hosts both released checkpoints, config.variant picks one.

arena feature planners add navid
arena launch sim:=gazebo robot:=jackal[camera=vln_navid] robot.planner:=navid
arena launch sim:=gazebo robot:=jackal[camera=vln_uninavid] robot.planner:=navid robot.mobile.config.variant:=uninavid

robot.planner:=navid selects robot.mobile:=vla. The planner accepts the instruction only (goal_inputs: [instruction]), so robot.mobile.goal:=instruction is also the default.

Variants¶

`variant`	Checkpoint	Camera	Model call	Answer	Executed
`navid` (default)	NaVid R2R+RxR training split	`vln_navid`, 640x480, 90 degrees HFOV	every frame of the episode, re-encoded each call	one move, e.g. `move forward 75 cm`	up to 3 primitives
`uninavid`	Uni-NaVid	`vln_uninavid`, 640x480, 120 degrees HFOV	only the frames since the last call, online token merging	four words from `forward left right stop`	the first 2

The cameras are the VLN-CE ones of each model's evaluation, 1.25 m above the floor in Habitat.

NaVid's answer is parsed like upstream's extract_result: stop anywhere wins, otherwise the first of forward, left, right with the first integer, expanded to min(3, n // 25) forward or min(3, n // 30) turn primitives. Uni-NaVid's words are taken in order up to the first word outside the action set. An answer that yields no primitive (no number, a move under one primitive, unknown words) moves one primitive forward and logs a warning, where upstream draws a random action.

Every observation the bridge sends is recorded, including those after queued primitives, so the model sees the frame after every move like the upstream agents. Sampling is upstream's (do_sample=True, temperature 0.2, the checkpoint's top_p=0.6), seeded with config.seed at every reset, so an episode replays identically.

Inputs¶

Datasource	Model input
`rgb` (`image` sensor) at 640x480	CLIP preprocessing of upstream: shortest edge to 224 (bicubic), center crop 224x224, CLIP mean and std
instruction	vicuna_v1 prompt with upstream's navigation template and token splicing

Primitives¶

The planner returns discrete commands (action_type: discrete) and the bridge executes each as a motion primitive:

Model action	Command	Primitive
forward	`forward`	0.25 m straight ahead
left	`left`	30 degrees counterclockwise
right	`right`	30 degrees clockwise
stop	none	ends the episode for the planner

The primitive sizes are the FORWARD_STEP_SIZE and TURN_ANGLE of both models' VLN-CE configs.

On stop the planner latches: it stops calling the model and, for a goto phase with signal: arrived, keeps sending arrived until the next reset. Without an expected signal it stands still. Before an instruction arrives, or with an empty one, the planner stands still without running the model. Upstream's evaluation harness also forced a stop after 25 steps without progress or 400 steps, using the privileged distance to the goal. Arena's episode limits replace it.

Instructions come from the scenario, per goto phase:

robots:
  - start: [1.0, 1.0, 0.0]
    phases:
      - goto: [8.0, 8.0, 0.0]
        instruction: walk past the sofa and stop at the door to the kitchen
        signal: arrived

Config¶

Set in planner.yaml > config, override per launch with robot.mobile.config.<key>:=<value>.

Key	Default	Effect
`variant`	`navid`	`navid` or `uninavid`, the checkpoint and inference recipe.
`device`	`auto`	Torch device, `auto` picks CUDA when available. CUDA runs in fp16, CPU in fp32.
`seed`	`0`	Torch seed applied at every reset.

Either variant needs approx. 15 GB of GPU memory (14.8 GiB peak over a 12-step episode on an RTX PRO 6000). NaVid's call time grows with the episode since every frame is re-encoded, Uni-NaVid's stays flat.

Weights and license¶

arena feature planners add navid fetches both checkpoints (each two pickle .bin shards of 9.98 and 5.52 GB, plus tokenizer and config files) and model/eva_vit_g.pth (2.0 GB, the EVA-ViT-g encoder as published by Salesforce LAVIS), 33 GB in all. Every file is pinned by sha256. The shards and the encoder load with torch.load(weights_only=True), the planner forces it for the whole process.

NaVid: https://huggingface.co/Jzzhang/NaVid, CC BY-NC-4.0, non-commercial use only.
Uni-NaVid: https://huggingface.co/Jzzhang/Uni-NaVid, no license stated on the model repository.
EVA-ViT-g: https://storage.googleapis.com/sfr-vision-language-research/LAVIS/models/BLIP2/eva_vit_g.pth.

The checkpoints carry a copy of the encoder that upstream ignores, the planner does the same and loads eva_vit_g.pth. The dependencies are pinned to the upstream stack where it matters (transformers==4.31.0, whose tokenizers wheels stop at Python 3.11, accelerate==0.21.0, timm==0.6.13), on current torch.

The code is MIT, the vendored navid/ and uninavid/ packages keep their upstream headers, see LICENSE and ATTRIBUTION.md. Cite both papers when using this planner:

@article{zhang2024navid,
  title={NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation},
  author={Zhang, Jiazhao and Wang, Kunyu and Xu, Rongtao and Zhou, Gengze and Hong, Yicong and Fang, Xiaomeng and Wu, Qi and Zhang, Zhizheng and Wang, He},
  journal={Robotics: Science and Systems},
  year={2024}
}

@article{zhang2024uni,
  title={Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Tasks},
  author={Zhang, Jiazhao and Wang, Kunyu and Wang, Shaoan and Li, Minghan and Liu, Haoran and Wei, Songlin and Wang, Zhongyuan and Zhang, Zhizheng and Wang, He},
  journal={Robotics: Science and Systems},
  year={2025}
}