The Hidden UX of Super Mario Bros. and How It's Teaching AI
- Kevin Not-A-Robot
- Mar 14
- 3 min read
Updated: Mar 19
033 - A PERSPECTIVE on the benchmarking of AI models with Super Mario Bros.
The OPINION
Despite what the die-hard AI community thinks about the benchmark’s validity, I am happy to see Nintendo’s Super Mario Bros. play a role in this transformational time of AI.
I love video games.
I grew up with them, starting with a Nintendo Entertainment System and the pack-in game Super Mario Bros. (remember when games used to come with consoles?). They've shaped my life and inspired me to enter the digital technology and entertainment space, specifically the world of user experience.
Now, we are in a world of AI generation. The video game that inspired me and many others is inspiring engineers to benchmark popular AI models. Researchers at Hao AI Lab (University of California) have recently put a handful of AI models to task with completing levels of Super Mario Bros, a revolutionary game released in 1985. The game needs no introduction.
The current discourse around using Super Mario Bros. to benchmark AI models has been around whether it is the correct way to measure and compare the different models' performance. I want to highlight how Super Mario Bros. is an excellent case study for game design, teaching people how to play and why I think it can make sense for AI to learn as well.
First, you must put yourself in a customer's mindset when the game debuted in 1985. The game was revolutionary because it was the first of its kind for home videogame consoles. Audiences, myself included, had never played anything like it before.
The game had to introduce its gameplay mechanics in a way that wasn't complicated, easy enough to understand, and fun. Shigeru Miyamoto—Mario's game creator and designer—admits to creating the famous first level, World 1-1, last in the design process as it introduced how to play the game.
The level was carefully crafted to teach you everything about the game in one go. Its design showed the player how to control and where to move, the dangers that threatened your character, and the bonuses that gave you power and increased your score. In the video below, Shigeru explains how they crafted the first level, which is cleverly a tutorial for the whole game. The beauty is that it's never advertised as such and is seamlessly blended into the gameplay.
With AI models constantly iterating and the race to achieve artificial general intelligence (AGI), I understand why engineers might want to use this particular game as a benchmark. It is a perfect example of teaching one how to interface with a digital product without overtly stating it through traditional text instructions or illustrations.
If you've ever played Super Mario Bros., you've grasped the game's core concepts and mechanics in that first level and can power through the rest. Each subsequent level challenged you by raising the difficulty. This challenge took applying and adapting what you learned to beat the game. I don't know about you, but any AI model that can do the same independently would impress me.
Just as we players learned and adapted to something new when playing Super Mario Bros., I can understand why AI doing the same can be valuable in determining how well and fast the models can process learning and adapting.
Despite what the die-hard AI community thinks about the benchmark's validity, I am happy to see Nintendo's Super Mario Bros. play a role in this transformational time of AI. When Shigeru Miyamoto and their team first created the game, I know they could have never imagined their creation would be at the forefront of a new generation of players, AI.
Watch the video below to learn more about the clever UX of World 1-1 in Super Mario Bros.
Listen to this article discussion on TheGoodBadUgly.ai: Deep Dives Podcast