Google’s SIMA AI: Your new gaming sidekick revolutionising virtual adventures

While much of Google’s AI research focuses on solving complex tasks, Scalable, Instructable, Multiworld Agent (SIMA) is here to add a touch of fun to your gaming experience. Designed by DeepMind, SIMA is not your average NPC. This AI learns to play alongside you, responding to natural language instructions and image recognition cues to navigate various game environments.

Unlike traditional gaming AI, SIMA isn’t limited to mastering one game. With 600 basic skills in its arsenal, it’s trained to adapt to a range of games, from strategy classics to open-world adventures. Its abilities span navigation, object interaction, and menu use, enabling it to tackle tasks just like a human player would.

To train SIMA, Google exposed it to nine different video games, ranging from sandbox adventures like No Man’s Sky to quirky simulations like Goat Simulator 3. By immersing SIMA in diverse gaming contexts, Google equipped it with the knowledge to accomplish in-game tasks, such as gathering resources or piloting spaceships.

The training process was no small feat. Google collaborated with eight-game studios and utilized four research environments, including a custom-built Unity engine environment called the Construction Lab. Videos of humans playing the games and providing instructions formed a crucial part of SIMA’s training data, allowing it to understand and execute commands with ease.

So, what does SIMA bring to the gaming table? Unlike conventional AI, it doesn’t require access to game source code or complex APIs. Instead, it simply needs on-screen images and straightforward instructions to spring into action, controlling the game via keyboard and mouse inputs. While currently adept at following simple commands, Google envisions SIMA mastering multi-level prompts, opening the door to more complex gaming experiences.

Beyond gaming, Google sees broader applications for AI like SIMA. By training agents on multiple games, they can generalize their skills beyond specific contexts, potentially leading to advancements in real-world tasks. From driving cars to assisting in everyday activities, the possibilities are limitless with AI like SIMA leading the way.