Pokémon plays Twitch: How a robot got IRC running on an unmodified SNES

The highlight of last year's Awesome Games Done Quick (AGDQ), the annual weeklong marathon of video game speedrunning, was easily the block run by TASBot (short for tool-assisted speedrun robot). The moment when that emulator-fueled robot hacked into an unmodified Super Mario World cartridge and hard-coded simple versions of Pong and Snake on the fly, using nothing but standard button inputs, was truly surprising and bewildering to behold.

This year, though, the team behind TASBot outdid themselves, showing a new level of mastery over decades-old gaming hardware. As organizer Allan Cecil (a.k.a. DwangoAC) put it during the livestreamed demonstration Sunday night, "We took over Pokémon Red. Then we took over the Super Game Boy. Then we took over the entire Super NES."

"Then we started streaming [IRC-based] Twitch chat directly through the controller cable."

You can watch that entire incredible demonstration in action using this Twitch archive link. If you're hoping to fully follow how the TASBot team used a Game Boy cartridge to get arbitrary code running on a Super NES, though, we should probably back up a bit.

Super Mario Inception

While TASBot is mounted to a classic NES R.O.B. unit, the key to the system is a specially designed "replay board" that can enter inputs into a console at frame-level speed (i.e. 30 or 60 times per second). The inputs being fed into the system are first developed, tested, recorded, and re-recorded on emulators, then played back to the board through a Linux box (in this case a Macbook). While TASBot can enter inputs more quickly and precisely than any human, it's only sending standard controller signals to the console; there isn't any hardware-level hacking or memory alteration beyond what is possible through the games and hardware as they originally existed.

As a preamble during its AGDQ block Sunday night, the TASBot team used its godlike input powers to set up a particularly hazard-filled Choco Mountain course in Mario Kart 64. After that, the team staged a modified repetition of last year's Super Mario World demonstration, taking "total control" of the game using a series of exploits that got the system to treat the controller input buffer as raw assembly code.

This year, once total control was achieved, the team decided to code in a fully functional copy of Super Mario Bros. onto the Super NES through faster-than-human button presses, essentially writing the game to the system's 128KB of RAM in real time at a rate of 384 bytes per frame (23 KB/s).

Adapting the original Super Mario Bros. into a ROM that would run on an SNES wasn't a simple task. TASBot team members going by p4plus2 and Masterjun didn't directly emulate the raw NES game, which may well have been difficult on hardware as underpowered as the SNES, but instead ported the raw engine from one platform to another line by line. In doing so, the team had to get around a number of system-level challenges, including differences in sprite tile sets that had the entire game looking puke green at one point.

There was at least one happy, effort-saving coincidence during the conversion, however. "When Masterjun later took p4plus2's standalone .smc ROM file of SMB on an SNES and ported it to execute inside of SMW, he initially did not include the SPC initialization to write the audio data, [which] thus pointed to whatever SMW left there," TASBot organizer Allan Cecil explained to Ars in an e-mailed interview. "The result was extremely surprising—nearly every single sound effect from SMW matched up to SMB perfectly, with some minor exceptions such as the flagpole. ... it's possible to run SMB on SNES standalone, but we did discover by pure chance that the SMW developers apparently re-used the same general [sound] mappings from SMB."

The team also ran into some real-world problems during its live demonstration of the Super Mario Bros. takeover during AGDQ Sunday night. TASBot's first attempt failed seemingly thanks to physical input wires that crossed over each other, messing with the precise electrical timing necessary for the in-game exploits to work. "When you're running on actual hardware things can get a little interesting because you're dealing with real physics in terms of electrical interference and the like," TASBot team member micro500 explained during the demonstration. "Refrigerators mess with it." As Cecil put it to Ars later, "The metal rolling cart acted like a giant antenna funneling EMI right at our unshielded cables."

The second attempt left the timing for the hard-coded speedrun of Super Mario Bros. failing, and graphical glitches plaguing the game-within-a-game. Hey, even robots can make mistakes...