omnivla

OmniVLA (Hirose et al., ICRA 2026, arXiv 2509.19480) behind the arena_planners bridge: the full 7.5B vision-language-action model (OpenVLA, Llama 2 7B with DINOv2 and SigLIP encoders), not omnivla-edge. It drives from the front camera toward a goal given as a 2D pose, a language instruction, or both, and returns an 8-point waypoint path at 3 Hz, the rate the model was trained at.

arena feature planners add omnivla
arena launch sim:=gazebo robot:=jackal[camera=vln_navid] robot.planner:=omnivla robot.mobile.goal:=instruction

robot.planner:=omnivla selects robot.mobile:=vla. The robot needs an image sensor in model_params.yaml, the camera frame is resized to 224x224 on the bridge side. The weights take about 15 GB on disk and 15 GB of GPU memory.

Goal inputs

robot.mobile.goal picks what the planner is told, the model's goal mask follows from it.

robot.mobile.goal	Model input
pose+instruction (default)	pose and instruction, pose only for goals without an instruction
pose	pose only
instruction	instruction only, the goal pose never reaches the planner

robot.mobile.instruction:=<text> stands in for goals that carry none. Instructions are lowercased and lose their closing punctuation, the form of the training prompts (move toward blue trash bin). The model's image-goal modes are not exposed.

The model has no arrival output, so the planner lists no signals and goto phases with signal: arrived refuse it. The goal pose decides success, and robot.mobile.goal:=instruction runs score how well the instruction alone leads there.

Actions

Each tick the model predicts 8 cumulative waypoints about 0.1 to 0.2 m apart in the robot frame of the observation. The bridge follows the path with pure pursuit (lookahead 0.5 m, 0.3 m/s, the speed cap of upstream's controller) for one tick, then asks again. Upstream steers at the fifth waypoint with the same caps.

Config

Set in planner.yaml > config, override per launch with robot.mobile.config.<key>:=<value>.

Key	Default	Effect
device	cuda	Torch device. The model runs in bfloat16, CPU inference is impractical.

Notes

• Measured on an RTX PRO 6000 Blackwell with torch 2.14.1+cu130: 14.3 GiB allocated after load, 14.9 GiB peak, 45 to 47 ms per step after a first step of about 0.8 s.
• transformers comes from the OpenVLA-OFT fork (4.40.1 with bidirectional attention), pinned in pyproject.toml. It pins tokenizers 0.19.1 and timm 0.9.16, which the model code checks at load.
• tests/test_model.py reproduces upstream's inference/1_ex.jpg image-goal trajectory on the sample images and needs the weights and a CUDA device, the other tests run on CPU without them.