NVIDIA Isaac GR00T Accelerates Synthetic Data Collection Solutions
AgiBot accelerates the collection of embodied intelligent synthetic data using NVIDIA Isaac GR00T, launches Genie Sim (a comprehensive robot simulation framework), and the "AgiBot Digital World" open-source synthetic dataset to assist end-to-end large model development.
Case Introduction
In this case, utilizing the remote simulation operation functionality provided by the NVIDIA Isaac GR00T blueprint for synthetic manipulation motion generation AgiBot has developed a large-scale robotic simulation framework, Genie Sim, which supports physically accurate, high-fidelity simulation and multi-dimensional generalization. Additionally, AgiBot has launched the "AgiBot Digital World" (https://agibot-digitalworld.cn), a massive synthetic dataset for embodied intelligence.
Imitation learning plays a crucial role in advancing development of robotics. Robots can quickly acquire new skills by observing and imitating the demonstrations of human experts. However, collecting high-quality demonstration data is often time-consuming, labor-intensive, and costly, significantly limiting the efficiency of robot learning.
To address this challenge, AgiBot has launched a large-scale synthetic data collection solution based on the Isaac GR00T blueprint for synthetic manipulation motion generation, along with a substantial open-source synthetic dataset, AgiBot Digital World, to tackle the challenge of scarcity in robotics more effectively.
AgiBot reconstructs the training environment by leveraging NVIDIA Isaac Sim's highly realistic visual rendering and precise physics engine. It also integrates GR00T-Teleop remote operation with GR00T-Mimic's data augmentation technology powered by NVIDIA Cosmos World Foundation Models (WFM) to rapidly generate high-quality and diverse expertized trajectory data.
This approach significantly reduces the cost and time of data collection and provides substantial synthetic data resources for robot model training, enabling a more effective integration of robots into human society and encourages the intelligent transformation across various industries.
Using GR00T-Teleop to Accelerate Simulation Robot Data Collection
AgiBot employs GR00T-Teleop to establish a more efficient and user-friendly simulation teleoperation method. GR00T-Teleop is a core module of the blueprint, which enables real-time remote control using VR/AR devices and transmits remote operation data streams to Isaac Sim. This allows operators to precisely control robots for executing complex actions and tasks within the simulation environment, supplying high-quality simulation training data for imitation learning and significantly lowering the barriers to robot skill development.
GR00T-Teleop supports remote operation based on Apple Vision Pro. In light of this, AgiBot and NVIDIA have collaboratively developed a cost-effective and efficient remote operation solution using PICO. This innovative solution has been successfully integrated into the Genie Sim simulation framework, offering robust data acquisition support for large-scale synthetic data production and open-source initiatives. It further expands application scenarios of teleoperation technology and enhances the efficiency and convenience of robot data collection.
By wearing the PICO VR device, users can manipulate the arm, waist, and chassis movements of AgiBot G1 robot within Isaac Sim in real time. Among these movements, arm control is the most intricate, requiring the system to monitor the robot's current joint angles continuously. A control command is immediately sent to the simulation system as the user moves the controllers. The system employs an inverse kinematics algorithm to combine the current joint angles with the command data, calculating the target joint angles in real time. This process enables precise control over the robot's joint postures in the simulation environment.
This process establishes an efficient closed loop from instruction input to action execution, fully demonstrating the central role and powerful capability of teleoperation technology in robot simulation control.
Transition from Human Demonstration to Extensive Generalization via GR00T-Mimic
Based on GR00T-Teleop's efficient and human-like synthetic data collection capabilities for training large embodied models, AgiBot utilizes GR00T-Mimic to generate a substantial amount of synthetic motion trajectory data from a limited number of human demonstrations, resulting in an exponential increase in the scale of data collection.
AgiBot has seamlessly integrated the GR00T-Mimic framework into Genie Sim platform. The AgiBot simulation team has intelligently generalized initial trajectories collected through GR00T-Teleop, scaling human demonstrations into thousands of high-fidelity motion trajectories. After automated validation in Isaac Sim, the refined trajectories are systematically converted into training datasets for embodied AI models.
This innovative workflow enhances the efficiency of synthetic data collection and delivers a high-quality dataset for training embodied models, establishing a solid foundation for research in robot algorithms.
Genie Sim: A Simulation Framework to Accelerate Robot Development
Genie Sim, the simulation framework from AgiBot, provides developers efficient data generation capabilities through a deep integration of GR00T-Teleop and GR00T-Mimic technologies, along with a large-scale and high-precision 3D asset library.
Leveraging NVIDIA's robust computing platform and AgiBot's cutting-edge innovative applications, developers can accumulate massive robot synthetic data rapidly, significantly reduce the cost of data collection, and improve both the diversity and quality of the data.
Furthermore, Genie Sim has established a comprehensive closed loop pipeline, encompassing trajectory generation, model training, benchmarking, and deployment validation. Users can quickly validate algorithm performance and optimize models through this efficient simulation toolchain. Whether for simple grasping tasks or complex long-range operations, Genie Sim can provide a highly realistic simulation environment and precise evaluation metrics, empowering developers to efficiently complete the development and iteration of robotic technologies.
Through deep collaboration with NVIDIA, AgiBot has significantly enhanced the performance and scalability of the simulation framework, offering robust support for the innovation and application of robotics technology.
By integrating NVIDIA’s Isaac Sim, AgiBot has significantly enhanced the performance and scalability of the simulation framework, offering robust support for the innovation and application of robotics technology.
