AI EMPLOYEE
Industries
INDUSTRIES
Restaurants Cleaning Services Home Service Companies Dental & Orthodontics Fitness & Wellness Hospitality HVAC/Plumbing
Pricing
Partners
PARTNERS
Become a partner Partners Listing
Resources
DEVELOPERS
Agent Creator NEWO BUILDER API & DOCUMENTATION ACADEMY VIDEO TUTORIALS COMMUNITY HELP How to choose AI automation
COMPANY
ABOUT CONTACT BLOG INSIGHTS
Call 24/7: +1 (888) 639-6939 free consultations
Book a Demo
Home / Blog / Google’s DeepMind Revolutionizes Robot Training: Unleashing the Power of Intelligent Agents
1 years ago 3 minutes

Google’s DeepMind Revolutionizes Robot Training: Unleashing the Power of Intelligent Agents

img

In a groundbreaking move, Google's DeepMind Robotics researchers have unveiled innovative methods for training robots that could reshape the landscape of generative AI and robotics in 2024. Traditionally, robots have been confined to repetitive tasks, limiting their adaptability to changes or errors. The team's newly introduced AutoRT is a game-changer, harnessing large foundational models for better situational awareness and the ability to manage a fleet of robots working collaboratively.

AutoRT utilizes a Visual Language Model (VLM) for enhanced environmental understanding and employs a Large Language Model (LLM) to suggest tasks, reducing the necessity for hard-coded skills. Tested rigorously over the past seven months, AutoRT demonstrated its prowess by orchestrating up to 20 robots simultaneously, handling a total of 52 devices and collecting an impressive dataset of 77,000 trials, including more than 6,000 tasks.

In addition to AutoRT, the team introduced RT-Trajectory, a revolutionary approach leveraging video input for robotic learning. Unlike conventional methods using YouTube videos, RT-Trajectory overlays a two-dimensional sketch of the robot's arm over the video, significantly enhancing the learning process. This method, showcasing double the success rate of previous training at 63%, not only advances the efficiency of robot-control policies but also taps into previously underutilized rich robotic-motion information present in datasets.

The significance of these advancements lies in the potential to move beyond the limitations of single-purpose robots, opening doors to more adaptive and intelligent agents. These developments mark a crucial step towards robots comprehending natural language commands, reducing the need for rigid programming and unlocking knowledge from existing datasets. As we step into 2024, the intersection of generative AI and robotics is set to redefine the capabilities of non-human workers, paving the way for a new era of digital employees. 

Key Highlights:

  • AutoRT Revolutionizes Robotic Understanding: Google's DeepMind Robotics researchers introduce AutoRT, a groundbreaking system that harnesses large foundational models, including Visual Language Models (VLM) and Large Language Models (LLM), to enhance robotic situational awareness. This innovation enables robots to manage complex tasks collaboratively, marking a departure from the traditional single-purpose robotic approach.
  • Adaptive Robot Orchestration: AutoRT has been rigorously tested over seven months, demonstrating its ability to orchestrate up to 20 robots simultaneously, handling 52 different devices. The system collected a vast dataset of 77,000 trials, including over 6,000 tasks, showcasing its adaptability and efficiency in varied scenarios.
  • RT-Trajectory Enhances Robotic Learning: DeepMind introduces RT-Trajectory, leveraging video input for robotic learning. This method, with a 63% success rate compared to 29% in previous training, overlays a two-dimensional sketch of the robot's arm on video, offering a novel approach to training and tapping into underutilized rich robotic-motion information in datasets.
  • Unlocking Natural Language Understanding: The combination of VLM and LLM in AutoRT signifies a move towards robots understanding more natural language commands, reducing the reliance on hard-coded skills. These advancements mark a pivotal step in developing intelligent agents capable of comprehending and executing tasks based on human-like communication.
  • 2024: A Year of AI and Robotics Fusion: As we enter 2024, the intersection of generative AI and robotics is poised to redefine the capabilities of non-human workers. AutoRT and RT-Trajectory represent a significant leap towards creating digital employees that can adapt, learn, and perform tasks efficiently in diverse and dynamic environments.

Reference:

https://techcrunch.com/2024/01/04/google-outlines-new-methods-for-training-robots-with-video-and-large-language-models/?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&guce_referrer_sig=AQAAAAZsT5bJqOhvXZ-8KJ5JO9k5jvUKLhtqyvMFOVAXuETTn5UpO3xfEdpu7ubzzs6A_myvSb68HkmGHrWIGx2KRm21MRhiSd_k6h77dNyUy4Z7yTqH_Z7MdJxyZ3nV52uOA6i9bMIN7hU-9wpVkTL47SOcGsQFe7Zg7YZbWXI1arfS

Recent Posts See all
The Rise of Robotics in Automotive Manufacturing: A Game-Changer for the Industry
The Future of Manufacturing: How Humanoid Robots are Transforming China’s Industrial Landscape
Intelligent Agents Transform Marine Operations: A New Era of Autonomous Ocean Exploration
Industries
  • Restaurants
  • Fitness & Wellness
  • Home Services
  • Cleaning Services
  • Dental & Orthodontics
Company
  • Digital Employee
  • About Us
Resources
  • Pricing
  • Documentation
  • Academy
  • Community
  • Partner Program
Social
  • Linkedin
  • Instagram
  • © 2025 Newo.ai
  • Terms of Use
  • Privacy Policy
  • Data Processing Addendum
  • Trust Center