Reason about the world before acting
Jointly model future world dynamics and action generation, using predicted 2D observations and 3D spatial states as training signals for current decisions.
- 3D WORLD
- LATENT FORESIGHT
I am currently a master's student at Tongji University, advised by Prof. Jingkuan Song, and expect to graduate in March 2027. I am also a research intern at Shanghai AI Laboratory, under the mentorship of Haoxiang Ma.
My research focuses on vision-language-action models, World-Action Models, and training infrastructure for large-scale foundation models. I am particularly interested in jointly modeling world dynamics and action generation, as well as improving generalization across tasks, environments, and robot embodiments. My work spans Mixture-of-Transformers architectures, large-scale pre-training, embodied-data pipelines, distributed training, and evaluation and deployment in simulation and on real robots.
Jointly model future world dynamics and action generation, using predicted 2D observations and 3D spatial states as training signals for current decisions.
My research asks how robots can reason about spatial and physical change, then transfer prior experience to new tasks, scenes, and embodiments. I work on VLA and WAM foundation models, human-robot mutual imitation, latent foresight, and efficient model deployment, building on earlier work in visual learning and semi-supervised semantic segmentation.
01Most robot foundation models predict actions directly from 2D observations and language instructions, without explicitly modeling how those actions will change the 3D world. WSA₁ asks whether a robot can anticipate future spatial states before acting, so that predicted world changes and action generation constrain one another.
WSA₁ introduces 3D-centric World-Spatial-Action modeling, jointly connecting future 2D visual observations, future 3D world states, and continuous actions. The framework learns spatial regularities from large-scale, heterogeneous demonstrations and transfers them across tasks and robot embodiments.
WSA₁ reaches 93.1% success on RoboTwin 2.0 and 98.2% on LIBERO, while improving real-world robot-control performance by 20% on average over representative baselines. Built around WSA₁, our team placed 4th overall—and 1st among university teams—in the CVPR 2026 RoboChallenge.
02Human videos and simulated robot data offer scalable sources for VLA pre-training, yet differences in viewpoint, appearance, and embodiment make this experience difficult to transfer directly to real robots. MiVLA begins with the behavioral structure shared by human hands and robot arms, bringing human and robot demonstrations into a unified pre-training framework.
Through human-robot mutual imitation and bidirectional alignment between their action spaces, MiVLA learns transferable behavioral representations across embodiments. The behavioral fidelity of real human video and the task diversity of simulation therefore complement one another, improving generalization to new viewpoints, visual domains, and robot platforms.
03A unified VLA model must preserve the semantic priors of a pretrained vision-language model while also learning physical dynamics and continuous control. Pixel-level future prediction can introduce objective interference and substantial inference cost, so InternVLA-A1.5 organizes understanding, foresight, and action as complementary capabilities with clearly separated roles.
Built on a native VLM backbone, the model continues learning visual question answering and subtask prediction while a lightweight unified expert generates continuous actions. A small set of latent queries distills task-relevant future information during training; the video-generation branch is removed at inference, preserving complex-instruction understanding, long-horizon execution, and real-time control.
04The long-tailed Softmax and GeLU activations in Transformers are difficult to represent at low bit widths with uniformly spaced or Log2-constrained quantization levels, making clipping and rounding errors hard to balance. CFQuant removes structural constraints on activation quantization intervals and uses Lloyd-Max iterations to learn quantization values and decision boundaries from a small calibration set, without model retraining.
CFQuant further introduces an efficient scale-shift algorithm (ESA) to reduce the distribution shift between calibration and inference, together with lookup-table matrix multiplication (MM-LUT) for efficient low-bit computation. At W4/A4, Qwen3-8B achieves 67.21% average zero-shot accuracy across six tasks, 25.49 percentage points above PrefixQuant; on DeiT-S, deployment experiments deliver about 8× model compression and 3.7× end-to-end speedup.
05Wheat-head segmentation is challenged by limited annotations, shifts in acquisition conditions, and accumulated noise in pseudo-labels. Developed for the Global Wheat competition, this work studies how unlabeled data can be used more effectively to build semantic-segmentation models that remain robust in complex field imagery.
The method combines a SegFormer backbone with two-stage hybrid training, extensive augmentation, and an iterative teacher-student self-training loop that progressively refines pseudo-labels and expands effective supervision. Its emphasis on systematic data use and training design also offers a transferable recipe for other vision tasks with scarce annotations.

Research Intern · Physical Intelligence Center
Contribute to the next generation of VLA/WAM foundation models, including model architecture and training-method design, implementation, and experimental validation across pre-training, fine-tuning, and evaluation.
Evaluate Muon for VLA and Mixture-of-Transformers training through controlled AdamW/Muon parameter grouping and ablations, measuring convergence, throughput, GPU-memory use, and downstream performance.
Build a VLM-based pipeline for subtask segmentation and hierarchical semantic annotation of embodied pre-training data, with data cleaning, quality assurance, and training-format conversion.
Contribute to deployment, testing, and iterative task optimization on real robot-manipulation experiments, including result verification.
Design and build an InternVLA-A1.5 physical demo that turns model capabilities into a reproducible real-robot presentation.
Algorithm Engineer · Embodied AI
Built a multimodal dual-arm system for intelligent service scenarios, coupling LLM/VLM reasoning with VLA control. The LLM/VLM stack interprets visual and natural-language inputs and decomposes complex instructions into executable atomic tasks; the VLA maps those tasks to robot-arm action sequences. I fine-tuned PaliGemma2-3B with LLaMA-Factory for task decomposition, integrated Whisper and CosyVoice for speech interaction, and connected model services, dual-arm control, and execution feedback through ROS, HTTP, and multiprocessing communication for latency-controlled human-robot collaboration.
Outputs: one first-author software copyright registration and one first-author patent application accepted for examination.
CVPR 2026 Workshop · GigaBrain Challenge · Real-Robot Track
RoboChallenge is the real-robot evaluation track of the CVPR 2026 GigaBrain Challenge. Conducted through fully online evaluation on Table30 V2, it requires a single model to complete more than 20 diverse manipulation tasks, jointly testing perception, decision-making, control, and cross-task generalization.
The MagicBot team competed with WSA₁ as its core model, placing 4th overall among more than 100 teams and 1st among university teams.
CVPR 2026 Workshop · GigaBrain Challenge · Simulation Track
The RoboTwin Track evaluates the generalization of VLA models in simulation across more than 50 bimanual manipulation tasks and five robot platforms, emphasizing complex multi-task control and cross-embodiment generalization.
The MagicBot team competed with WSA₁ and placed 3rd in the track with an official score of 89.3.
Global Wheat 2025 · Full Semantic Segmentation
The competition focused on pixel-level semantic segmentation of wheat organs under challenging field conditions. With 1,096 annotated and 52,078 unlabeled images, it emphasized fine-grained recognition and cross-environment generalization under limited supervision.
Our solution placed 2nd among 142 participants with a test mIoU of 0.7099. The work was subsequently published as a first-author extended abstract at the ICCV 2025 CVPPA Workshop.
VLA & World-Action Models · Training Infrastructure · Robot Learning