Generative AI Meets Real-Time Gaming in a Browser-Based Demo
- Microsoft’s WHAMM model introduces generative AI to real-time gaming, demonstrated through a playable Quake II demo.
- The WHAMM model, an evolution of WHAM-1.6B, uses a MaskGIT-style setup for faster, parallel token generation.
- While the demo showcases impressive AI capabilities, it also highlights areas for improvement, such as input lag and graphical anomalies.
In a groundbreaking move, Microsoft has unveiled WHAMM, a generative AI model that brings real-time AI rendering to the classic game Quake II. This interactive demo, accessible through a web browser, allows users to experience the power of AI in gaming firsthand. While the frame rate may not be ideal, hovering in the low to mid-teens, Microsoft emphasizes that the focus should be on analyzing the model’s quirks rather than judging it as a complete gaming experience.
WHAMM, an acronym for World and Human Action MaskGIT Model, represents a significant update to the original WHAM-1.6B model launched in February. By transitioning to a MaskGIT-style setup, WHAMM enables the parallel generation of all tokens for an image, reducing dependency and the number of forward passes required. This shift allows for a more seamless and real-time experience compared to the autoregressive model used in its predecessor.
The training process for WHAMM is equally impressive, requiring just over a week of data on Quake II, a dramatic reduction from the seven years needed for WHAM-1.6B. Additionally, the resolution has been improved from a pixel-like 300 x 180 to a slightly less pixel-like 640 x 360. Users can try out the demo themselves at Copilot Labs, where they can shoot, move, jump, crouch, look around, and even engage with enemies. However, it’s important to note that the demo is primarily a showcase and cannot fully replicate the original Quake II experience.
While the WHAMM demo demonstrates the model’s ability to keep track of the existing environment and adapt to user inputs, it is not without its flaws. Enemy interactions are described as fuzzy, the context length is limited, and the game struggles to accurately store vital stats like health and damage. Furthermore, the demo is confined to a single level, limiting the overall experience.
The announcement of WHAMM comes on the heels of OpenAI’s latest Ghibli trend, which has faced significant backlash. As AI continues to advance at a rapid pace, there is a growing concern about its potential to replace human creativity in fields like gaming and filmmaking. However, the sweet spot lies in AI enhancing, rather than replacing, creative works. Nvidia’s ACE technology, which powers lifelike NPCs and is already integrated into the life simulation game inZOI, serves as a prime example of this approach.
From a technological standpoint, WHAMM represents a significant step forward compared to previous attempts at AI-generated gaming. While earlier efforts were often chaotic, incoherent, and plagued by hallucinations, WHAMM showcases a more refined and controlled use of AI in real-time gaming. As the technology continues to evolve, we can expect to see even more impressive applications of AI in the gaming industry, potentially leading to fully AI-generated games and movies in the near future.