Project Page, 2026

A 3D-Centric World-Spatial-Action Model for Generalizable Robot Control

Jiahao Jiang Jianing Zhang Zhenhan Yin Ruidong Chen Sen Wang Zhaoshu Yu Pengpeng Zeng

XiaoFeng Cao Xuanhan Wang^†⋄ Jingkuan Song^†⋄ Heng Tao Shen^†⋄

^†Project Leader ^⋄Corresponding Author

School of Computer Science and Technology, Tongji University Shanghai Innovation Institute

Paper Code Models Citation

Overview

Framework

WSA is a World-Spatial-Action embodied foundation model for generalizable robot control across simulation and real-world manipulation tasks.

Overview of WSA pretraining data, WSA model components, and downstream performance.

Unified WSA Modeling WSA modeling unifies semantic understanding, 3D world modeling, and physical execution.

Bidirectional 3D Causality Learns both action-conditioned scene dynamics and 3D inverse dynamics.

Mixture-of-Transformers Coordinates 2D planning, 3D prediction, and 3D action generation with shared dependency rules.

Data-Efficient Pretraining Pretraining on 6,000 demonstration hours yields strong simulation and real-world manipulation performance.

Superior Performance State-of-the-art results across simulation and real-world robot manipulation tasks, achieved by our open-source model.

World-Spatial-Action modeling

WSA learns robot control in a shared 2D-3D latent space, unifying instruction-aligned visual planning, action-conditioned 3D world modeling, and 3D-aware action generation.

WSA architecture with 2D Spatial Expert, 3D Spatial Expert, and 3D Action Expert. — The framework couples 2D Spatial, 3D Spatial, and 3D Action experts with 3D-causal bidirectional attention, so visual subgoals, future 3D states, and action chunks are optimized together.

Existing VLA and WAM policies mainly model 2D semantic-action or 2D video-action causality. World-Spatial-Action (WSA) lifts this process into 3D by jointly learning subgoal visual planning, action-conditioned 3D scene prediction, and 3D inverse dynamics.

This builds a bidirectional world-action loop: robot actions explain how the 3D scene evolves, while the desired 3D scene evolution guides continuous action generation for closed-loop execution.

Performance

Real-World Foundational Experiments

In real-world experiments of basic capacity, WSA demonstrates strong performance on a series of complex tabletop manipulation tasks, covering single-arm AgileX PiPER and dual-arm ARX Lift2 platforms.

Object Organization

Tidy up the table by sorting the toys, stationery, and trash into their designated containers.

Trash Cleaning

Pick up the trash, hand it over to the other gripper, and place it into the trash bin.

Sweep Trash

Sweep the trash into the dustpan using a broom.

RGB Block Sorting

Position red, green, and blue blocks from left to right in the specified sequence.

Unzip Pencil Bag

Grasp the pencil bag, unzip it, and place it back onto the tabletop.

Toy Box Organization

Pick up toys from the desktop, place them into the toy box, and arrange them neatly inside.

RoboChallenge V2.0 Real-World-Based Benchmarking

Leaderboard

1x speed

Stack bowls

1x speed

RoboTwin2.0 Performance

RoboTwin2.0 benchmark evaluates multi-task generalization in simulation. WSA reaches 93.5% on the clean (easy) setting, while WSA-L achieves 93.14% on the randomized (hard) setting, showing stable and robust performance across diverse ALOHA manipulation tasks.

Easy SR 93.5%

Average success under the clean (easy) setting.

Hard SR 93.1%

WSA-L average success under the randomized (hard) setting.

Task coverage 50

Manipulation tasks.

Metric	π₀	π_0.5	ABot-M0	Motus	InternVLA-A1	LingBot-VA	Fast-WAM	WSA-B	WSA-L
Avg. Success (Hard)	58.40%	76.76%	85.08%	87.02%	89.64%	91.50%	91.78%	92.70%	93.14%

Place A to B

Placement

Analysis

What makes WSA work

Ablation study comparing action-only, visual planning, 3D world prediction, and full WSA modeling. — WSA ablation: visual planning and 3D world prediction each improve over the action-only baseline, while full WSA modeling achieves the best average performance, showing that 2D semantic cues and 3D spatial reasoning are complementary.

Pretraining study comparing from-scratch, post-train only, pre-train only, and pre/post-train settings. — Pretraining study: WSA pretraining builds stronger general manipulation priors than post-training alone, and applying WSA in both pretraining and post-training yields the strongest downstream performance.

Qualitative visualization of current observation, planned subgoal images, predicted 3D world states, and 3D action execution. — Qualitative WSA visualization: WSA plans instruction-aligned subgoal images, predicts the corresponding 3D goal world, and executes continuous robot actions toward that spatial goal.

Paper

Citation

WSA₁: a 3D-Centric World-Spatial-Action Model for Generalizable Robot Control

Jiahao Jiang, Jianing Zhang, Zhenhan Yin, Ruidong Chen, Sen Wang, Zhaoshu Yu, Pengpeng Zeng, Xiaofeng Cao, Xuanhan Wang, Jingkuan Song, and Heng Tao Shen. arXiv:2607.03941, 2026.

@misc{jiang2026wsa,
  title         = {WSA$_1$: a 3D-Centric World-Spatial-Action Model for Generalizable Robot Control},
  author        = {Jiahao Jiang and Jianing Zhang and Zhenhan Yin and Ruidong Chen and Sen Wang and Zhaoshu Yu and Pengpeng Zeng and Xiaofeng Cao and Xuanhan Wang and Jingkuan Song and Heng Tao Shen},
  year          = {2026},
  eprint        = {2607.03941},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url           = {https://arxiv.org/abs/2607.03941}
}