In the robotics lab, this scene is a recurring one: the simulated robotic arm on the screen moves along a perfect trajectory, but in the real world, the glass is still on the table, or has even been knocked over by the gripper.
Today's robotic world models face challenges such as grippers failing to grasp objects, robotic arms unable to interact with their environment, and predictions becoming completely wrong after occlusion. AI-generated future videos may appear clear and natural frame by frame, but the content is utterly unreliable.
Behind this phenomenon lies a major paradox in the field of embodied intelligence: everyone is part of a world model, yet physical logic simply cannot keep up.
Muka Robotics, a Chinese startup founded less than four months ago, has proposed a new solution.LJM (Latent Joint-conditional Model) World ModelThe entire training process was completed using only 32 cards. Under the anonymous code name SisyphusWorld, it ranked second on the world's most authoritative embodied world model evaluation WorldArena, setting a new state-of-the-art record in all four visual metrics on the LIBERO robot operation benchmark.
In the sub-results, LJM achieved 89.17 Motion Quality, 92.60 3D Accuracy, and 85.93 Controllability, demonstrating that the model not only focuses on the appearance of a single frame but also maintains good consistency between motion, spatial structure, and motion control.
LJM redefined the first principles of the robot world model: to predict the future, one must first understand in the hidden space, "what will happen in the world after this action is performed."
Understanding the interactions first, and then generating predictions, may be the turning point for world models to become practical.
Why is 99% of the generated results...
It's useless for robots?
What is the direction of exploration for the next generation of general-purpose robots? Nine out of ten experts in embodied intelligence will tell you it's the world model.
The current industry consensus is that VLA (Vision-Language-Motion Model) excels at establishing reactive mappings from observed actions, but lacks an intrinsic understanding of physical laws (gravity, friction, inertia). Introducing a World Model is equivalent to giving a robot an "internal simulator," allowing it to predict how the environment will change before performing actions.
As embodied intelligence moves towards large-scale deployment, its ability to operate stably and sustainably has become a key concern. In less than five months since the WorldArena leaderboard was launched, the top spot has changed hands more than a dozen times, with over a hundred teams vying for higher scores, creating a seemingly vibrant and competitive landscape. However, the results generated by the vast majority of these world models are hardly practical for robots intended for actual deployment.
The most obvious trap is that everyone is becoming a "visual freak".From its inception, the video diffusion model has been optimized to measure the difference between the generated image and the real image at the pixel level. This fundamentally determines that it will prioritize fitting the content that accounts for the vast majority of pixels in the image: the fineness of object texture, the reflection of ambient light, the smoothness of robot movement, etc. Embodied intelligence, on the other hand, needs to focus on the physical interactions in the real world: the dozen or so pixels when the gripper contacts the object, the tiny deformation when the object is picked up, and the state changes at the moment of collision. This content may account for less than 1% of the pixels in the entire video.
To minimize overall loss, the world model will unhesitatingly sacrifice 1% of physical accuracy for 99% visual appeal. This deviates from the intended optimization goal from the outset. The most direct evidence is the publicly available scores on WorldArena: Wan2.2, a general-purpose video model with cinematic image generation capabilities, scored 62.35, and CogVideoX scored 62.71, a significant gap compared to the top-ranked robot-specific models around 73 points.
A deeper misconception lies in "action injection".The prevailing approach is to use a general video model and then input the robot's motion signals. Some people use linear projection to feed the motion vectors into the input layer, use FiLM and AdaLN to modulate the motion into modulation parameters, and others use cross-attention to make the model "pay attention" to the motion. The essence of this approach is to treat the motion as a string of meaningless low-dimensional numerical signals and let the model figure it out on its own.
However, the physical meaning of an action is never written into the numerical values. Discussing an action without considering the scene, the contact state, and the task objective is like discussing the meaning of a word without context; the model cannot truly understand it, so the actions it conceives will naturally be difficult to make reasonable.
Finally, there's the memory problem that no world model can escape.If the generated world model is "forgotten as soon as it's generated," it could have a huge impact on the robot's operational planning.
These three problems cannot be solved by brute-force scaling laws, nor can engineering fill the gaps. While the entire industry is fretting about "how to modify a generic video model to be usable by robots," few people stop to ask the most fundamental question: What kind of world model should robots truly need?
LJM's approach
First, figure out the "interaction".
LJM's biggest improvement lies in its restructuring of the world model workflow, assigning the tasks of "thinking out what will happen" and "drawing out the results" to two specialized experts, each with their own responsibilities.
The research team proposed a "dual-brain collaboration" framework, which is similar to the working mode of people driving: your prefrontal cortex is responsible for inference and prediction. If the car in front signals to change lanes, I have to release the accelerator and slow down; while your visual cortex is responsible for processing the scene in front of me, recognizing lane lines and traffic signs. The division of labor within the brain is clear.
LJM also divided the model into two cooperating experts. One is...Latent reasoning expertBased on the modified Qwen3-VL-2B, its function is to deduce the script in a continuous latent space—given human language instructions, past visual history, and the entire series of actions to be performed, it first deduces the entire interaction process in the latent space—which coordinate the robotic arm will move to next, in which frame it will make contact with the object, whether the object will be picked up after the gripper closes, and where it will be moved to afterward; the second is...World modeling expertBased on the Wan2.2-TI2V-5B video diffusion model, it needs to render the interactive "script" that the reasoning experts have figured out into smooth and realistic future videos frame by frame.
The shift from "generating first and then understanding" to "understanding first and then generating" resolves the underlying contradictions of the previous world model.
The LJM overall architecture and CoT latent construction are shown on the left. The left side shows the joint modeling by two experts through shared attention, and the right side shows the interleaved supervision of future ontology state, optical flow, and visual latent.
However, a two-stage architecture alone is not enough.LJM also assigned a clear, computable physical target to each inference token in the latent space.Specifically, each token output by the reasoning expert must be responsible for the future in three dimensions simultaneously: accurately predicting the three-dimensional position of the robotic arm's end effector at every step in the future (body state), the visual state changes of objects in the future scene (visual dynamics), and the motion direction of each pixel in the future image (pre-calculated optical flow).
All three objectives can be directly extracted from real training data, down to the frame number, specific position, direction of motion, amplitude, and specific speed. This eliminates the model's "assumptions" from the training mechanism, and each step of reasoning is responsible for the real physical results.
The communication method between the two experts is another core design element of LJM that is easily overlooked. Previous dual-module schemes mostly involved the inference module calculating the result and then sending it to the generation module all at once. LJM, however, uses a different method...Mixture-of-Transformers (MoT) Shared Attention MechanismThis allows for bidirectional dialogue between the two brains at each layer of the Transformer: the inference token and the video token are mapped to the same attention space. With each layer of attention computation, the video token receives the latest inference constraints, while the inference token sees the half-generated video state and adjusts its inference results in real time, ensuring continuous information exchange. Simultaneously, the original low-dimensional action signals are not discarded but are preserved through two direct pathways controlling the generative model: one as the context for cross-attention and the other as the AdaLN modulation parameters.
This forms a complete dual-path control system: the original motion provides precise numerical constraints at the millimeter level (how many centimeters to move, how many millimeters the gripper opens), and the inference latent variables provide high-level physical interpretation (whether it is currently moving freely, aligning, or performing a grasp). The combination of the two opens up a complete pathway from control values to the physical semantics of the motion, and then to the real visual result, thus avoiding the problem of "the motion values are correct, but the physical result is wrong" in terms of mechanism.
There are also solutions to memory problems.Value-Driven Temporal Condition (VTC) MechanismLJM uses algorithms to select the most valuable historical frames, improving the coverage of scene states and thus increasing the accuracy of robot evaluation and prediction.
The impact of VTC on long-term state consistency. The top, middle, and bottom rows show the generation results for the true future (GT), without VTC, and with VTC, respectively. The numbers at the top represent the relative frame count. When key historical information is lacking, the model struggles to stably maintain the object state after interactions, easily leading to object disappearances or duplicate generation. After introducing VTC, the number and spatial relationships of objects are closer to the true future.
From the division of responsibilities in the dual-expert architecture, to equipping reasoning with physical supervision, layer-by-layer bidirectional communication, and then to the memory mechanism, the world model ultimately gained the ability to correctly understand physical interactions and predict future scenes, laying the foundation for robot motion planning.
In addition to WorldArena, this work further evaluated the well-known robot operation benchmark LIBERO. LJM achieved the best results in all four metrics: PSNR, SSIM, FVD, and LPIPS, comprehensively outperforming the comparison methods.
The experimental results further illustrate that a strong video prior does not equate to a robot world model. Only when actions are interpreted as interactive variables jointly defined by language, scene, and future state can the video generation backbone truly be transformed into a controllable robot dynamic model.
Established less than four months
Aiming for second place globally
Moke Robotics, which ranked second on the WorldArena list, just 0.58 points behind the leader, is a startup company that was founded just three months ago.
This month, Chi Xiaowei, co-founder and CEO of Moke Robotics and a PhD from the Hong Kong University of Science and Technology, discussed "How World Models Empower Embodied Intelligence" at WAIC. He stated that world models can empower embodied policy learning through three paths: generating training data, acting as a reinforcement learning environment, and learning physical representations. These three paths require the world model's predictions of states to have physical realism.
This also determined that when Moke Robotics was first established, its primary goal was to solve the bottleneck problem of enabling robots to understand the world.
It is worth mentioning that,LJM's entire process, from pre-training to fine-tuning, runs on Alibaba Cloud Zhenwu 810E.From the very first line of code, the team simultaneously optimized distributed training and adapted to domestic computing power, which laid a solid foundation for future engineering and technology implementation.
The emergence of LJM may mark a critical tipping point for the embodied intelligence industry: we are finally beginning to "understand true physical causality," and AI models are beginning to learn the operating logic of the physical world.
When world models can accurately predict the consequences of every action, it is expected to provide a new path to bridge the long-standing sim-real gap in the field of robotics. In the future, robots can rehearse millions of times in world models, making all possible mistakes, before being safely deployed to real machines.
Only then will the prepared robots be deployed to every real-world scenario where they are needed.