In February 2025, Microsoft introduced a revolutionary AI model named ‘Muse’ designed to generate gameplay visuals and controller actions in real-time as players engage with the game. According to Microsoft, Muse operates as a World and Human Action Model (WHAM) utilizing cutting-edge AI technology to create entirely new gameplay sequences. It is based on a Transformer architecture.
After training the Muse model through 1 million updates, it mastered fundamental movements, geometry, proper interactions, and even flying mechanics. Microsoft has now launched Copilot Gaming Experiences as part of the Copilot Labs initiative, showcasing experimental AI demonstrations and engagement ideas.
In a fascinating twist, Microsoft has reimagined the classic game Quake II using the Muse AI model to develop gameplay dynamically in real-time. The system creates new gameplay sequences immediately following each user input. The best part? It’s available for anyone to try in their web browser for free by clicking this link.
I had the chance to experience Microsoft’s AI-enhanced Quake II, and it served as an interesting proof of concept. Players can move around using the WASD keyboard controls and perform actions. However, a significant drawback is the lack of mouse support. Currently, the graphics aren’t high-quality, but it’s an early demo, and with enhancements moving forward, Generative AI could truly transform gaming, much like Nvidia’s multi-frame generation in DLSS 4.
That said, there’s a short playtime limit of about two minutes, which restricts players from fully exploring the AI-generated environment. During my session, I noticed the absence of clear goals or objectives, and intriguingly, objects and enemies would often vanish unexpectedly. This inconsistency represents a challenge for the AI version of Quake II.
Google’s AI Model for Gaming
In another development, Google DeepMind exhibited a similar AI gaming technology in 2024. Their game engine, GameNGen, operates on an AI neural model (Diffusion-based) that enables real-time interaction in complex gaming environments. Google demonstrated that GameNGen could run the classic DOOM game at 20 frames per second on a single TPU.
Moreover, with the launch of Genie 2, a large-scale foundation world model (Diffusion-based), Google can now create a wide range of gaming environments that respond to keyboard inputs. This model is capable of simulating essential physics features such as gravity, reflections, smoke, and lighting. As AI continues to make inroads into gaming, it will be exciting to see how it shapes the future of the industry.
