Google DeepMind on Thursday debuted SIMA 2 – its reasoning AI agent that the firm claims behaves like a human inside virtual worlds. The tech company said SIMA 2 helps DeepMind advance beyond simple on-screen actions and move toward AI that plans and explains itself, as well as learn through experience.

The firm said the launch marked a significant step toward Artificial General Intelligence (AGI). DeepMind also warned that SIMA 2 has important general implications for the future of robotics and AI-embodiment.

SIMA 2 thinks for itself and takes actions in interactive environments

The tech company released the first version of SIMA (Scalable Instructable MultiWorld Agent) in March. Google said the AI agent learned hundreds of basic skills by watching the screen and using virtual keyboard and mouse controls. The firm also acknowledged that the latest version of the AI agent takes things a step further by allowing the AI to think for itself.

Google DeepMind also revealed that Gemini powers the AI agent. The tech company stated that integrating SIMA 2 and Gemini helps the AI agent understand a user’s high-level goal, perform complex reasoning, and skillfully execute goal-oriented actions in games. 

The firm said SIMA 2 is the company’s most capable AI agent for virtual 3D worlds. DeepMind found that interacting with the agent felt less like giving it commands and more like collaborating with a reasoning companion about the task at hand.

According to the announcement, SIMA 2 goes beyond following basic instructions to thinking, understanding, and taking actions in interactive environments. The AI agent will allow users to interact with it through text, voice, or even images.

Google said its Gemini AI model helps SIMA 2 interpret high-level goals and talk through the steps it intends to take. The firm added that Gemini helps the new human-centered agent collaborate within games with a level of reasoning the original system could not achieve.

The tech company also reported stronger generalization across virtual environments. DeepMind confirmed that SIMA 2 completed longer, more complex tasks, including logic prompts, screen-drawn sketches, and emojis. Google said the ability gets SIMA 2’s performance closer to that of a human player on a wide range of tasks. The firm also noted that the AI agent had a 65% task completion rate, compared to 31% by SIMA 1.

DeepMind found that SIMA 2 interpreted instructions and acted inside entirely new 3D worlds generated by Genie 3. The project was released last year, which creates interactive environments from a single image or text prompt. The tech company said SIMA 2 could orient itself, understand goals, and take meaningful action in worlds it had never encountered until before testing.

Google argued that the human-centered agent is now far better at carrying out detailed instructions, even in worlds it’s never experienced before. The firm said SIMA 2 can transfer learned concepts from one game to another, connecting the dots between similar tasks.

DeepMind finds gaps in SIMA 2 that need to be addressed

Researchers noted that the agent switched into self-directed play after learning from human demonstrations. The agent used trial and error, along with feedback generated by Gemini, to create new experience data. The new experience data includes a training loop where SIMA 2 attempted the tasks it generated and fed its own trajectory data back into the next version of the model.

Although DeepMind hailed SIMA 2 as an advancement in artificial intelligence, the research also found gaps that need to be addressed. Google identified gaps, including working within a limited memory window, struggling with very long, multi-step tasks, and facing visual-interpretation challenges seen in 3D AI systems.

DeepMind revealed that SIMA 2 served as a testbed for skills that could be used in robotics and navigation in the future. The firm said its SIMA 2 research offers a strong path towards applications in robotics and also AGI in the real world.

Get seen where it counts. Advertise in Cryptopolitan Research and reach crypto’s sharpest investors and builders.