(Live demonstration of LimX VGM. Image source: LimX Dynamics)
LimX Dynamics has found an easier and cheaper way to teach robots a task just by watching a human do it in a video.
Introducing LimX VGM (VideoGenMotion), the company’s first-ever embodied manipulation framework that has some promise to change how automation can be engineered.
LimX VGM involves a combination of instructions (via AI prompts) and some visual demonstrations to instruct robots how to do certain tasks. This system will serve as the “bridge” to comprehend them all together for the robots to execute the tasks accurately.
Basically, it has three steps:
• Training: Robots watch real videos of humans doing the task, which will be used to improve existing generation models for them to understand how the task is done.
• Inference: Uses prompts or instructions to picture the scene of the task. Then the trained models will infer from the prompts and generate own videos of humans doing the task, gathering depth information (how far things are). These will be translated into robot actions.
• Execution: The system calculates exactly how the robot should move to imitate the tasks based on the following actions and supported models, performing the task in real life.
Think of it like teaching a child how to do a task through mimicking behavior, but more refined. And definitely less clumsy.
This technology can help cut down costs and time needed to acquire a lot of real-world data. And because it works with various kinds of robots, LimX VGM is also very flexible and efficient.
Currently, LimX Dynamics is forming collaborations across the world to make LimX VGM work in various industries.