MoDex: planning high-dimensional dexterous control via learning neural internal models
1 Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
2 School of Mechanical and Aerospace Engineering, Nanyang Technology University, Singapore
3 Department of Electrical and Computer Engineering, Carnegie Mellon University, Pennsylvania, USA
  • Volume
  • Citation
    Wu T, Li S, Lyu C, Sou K, Chan W, et al. MoDex: planning high-dimensional dexterous control via learning neural internal models. Robot Learn. 2026(1):0008, https://doi.org/10.55092/rl20260008. 
  • DOI
    10.55092/rl20260008
  • Copyright
    Copyright2026 by the authors. Published by ELSP.
Abstract

Controlling dexterous hands in high-dimensional action spaces remains challenging, whereas humans naturally achieve such control through internal models that predict and adapt body dynamics.  Inspired by this concept, we present MoDex, a neural internal model framework that learns the intrinsic dynamics of dexterous hands via coupled forward and inverse networks. MoDex enables efficient bidirectional planning through integration with a Cross-Entropy Method (CEM) optimizer, achieving superior data efficiency and faster decision-making compared to model-free and model-based baselines. Furthermore, the pretrained internal model serves as a transferable module: when combined with an external dynamics model, it improves data efficiency in in-hand object manipulation, and when coupled with a large language model (LLM), it enables few-shot gesture generation in both simulation and the real world. Extensive experiments across multiple robotic hands demonstrate MoDex’s versatility and effectiveness in high-dimensional control.

Keywords

model-based learning; dexterous manipulation; high-dimensional control

Preview