MIT News June 3, 2024
Existing robotic datasets vary in different modalities such as color, depth, tactile, and proprioceptive information, and collected in different domains. Current methods usually collect and pool all data from one domain to train a single policy to handle such heterogeneity in tasks and domains, which is prohibitively expensive and difficult. Researchers at MIT presented a flexible approach, called Policy Composition, to combine information across such diverse modalities and domains for learning scene-level and task-level generalized manipulation skills, by composing different data distributions represented with diffusion models. Their method could use task-level composition for multi-task manipulation and composed with analytic cost functions to adapt policy behaviors at inference time. They trained their method on simulation, human, and real robot data and evaluated in tool-use tasks. The composed policy achieved robust and dexterous performance under varying scenes and tasks and outperformed baselines in both simulation and real-world experiments. Video …read more. Open Access TECHNICAL ARTICLEÂ
Three different data domains… allow a robot to learn to use different tools. Credit: MIT News, June 3, 2024Â