Steam blocks another weird trick for estimating sales

Valve have stifled another statistical technique people used to estimate Steam sales figures, one which temporarily revived the popular estimatron SteamSpy. This latest weird trick, based on Steam achievement numbers, has been cut off less than a week after game dev Tyler Glaiel--of Closure and The End Is Night fame--brought it to public attention. While such numbers are mostly useful to Jo Public for funsies and conspiracy theories, many developers had previously said they used SteamSpy estimates in their market research. Valve do claim they're working on better tools for developers to get data out of Steam, but for now they're back in the lurch.

The technique Glaiel popularised used Steam achievements to estimate sales, building on a trick he picked up from a key trading site. He explained in a blog post:

"They were looking at achievement data and extrapolating how many users would be necessary to get the percents shown. For example, if an achievement had '50%' of players achieving it, that would imply at least 2 players. '33%' implies at least 3. If a game has both of those, it implies at least 6 players (3/6 for the 50%, 2/6 for the 33%)."

With sufficiently precise percentages, which Valve's Steam API offered, and some serious Numberwang (I won't get into it but Glaiel's post does), he could calculate how many people had any game with Steam achievements in their libraries. This technique, Glaiel said, gave numbers far more precise than from SteamSpy's method based on sampling random public user profiles. He shared his technique with SteamSpy operator Sergey Galyonkin, who used it to get the site back up and running.

After revealing this technique, Glaiel tweeted on June 29th that they had "emailed Steam about it a week ago and got no response, so I'm assuming they just don't care." Ha ha. Come July 4th, he noted the technique no longer worked. Glaiel said the API had started giving out rounded achievement numbers, which are far too imprecise for this mathematical trickery to work. Once again, everyone's in the dark.

SteamSpy died in April after Valve changed Steam's default privacy settings, hiding the game list on user profiles that SteamSpy relied upon. Valve said that the European Union's new General Data Protection Regulation demanded the change, though the company have made no secret of their reservations about the site.

"We had two problems with SteamSpy," Valve's Jan-Peter Ewert said on June 28th at White Nights, a games industry conference in Russia, when asked by dev Michael Kuzmin about blocking the tool many had found useful. Oleg Chumakov filmed the exchange.

"One is GDPR came along. The other thing is, SteamSpy had a lot of variance in how accurate it was. It was very accurate for some games, it was very inaccurate for a few others, and so a couple of developers-" at which point Kuzmin interrupted saying he knew SteamSpy only had predictions but still found those useful.

"So, to be clear, we don't have a business selling iPhones," Ewert resumed. "The only way that we can make money is if you make good decisions in bringing the right games to the platform and finding your audience. And so yes, we are very much working on new tools and new ways of getting data out of Steam, and we hope that data will be more accurate and more useful than what SteamSpy previously offered."

That's a vague plan at best. Promising, but vague - and with no timeline.

Some have taken Valve blocking Glaiel's one weird trick as proof that the Steamers lied about the GDPR's requirements and used it as a smokescreen to kill SteamSpy, seeing as Glaiel didn't go anywhere near any user's profile. But why not both? It could be true that Valve needed to hide certain information for GDPR reasons, then realised they were glad to have that 'hole' filled and decided to fill this hole too while working on installing a proper window themselves. It's not like people have a right to this data, even if it can be helpful.

SteamSpy's data could be less helpful in the hands of some players and fans. Sure, it can be fun to coo over big numbers ("oooh lookit that one with all those 0s like a digital ghost's wail"), but a little knowledge can be a dangerous thing - and the Internet is already plenty dangerous. Often when I saw players refer to SteamSpy numbers, they were filtered through shoddy analysis and a poor understanding of how games are made then wielded as weapons to chastise "greedy" developers. I have not missed that at all during SteamSpy's downtime.

For the curious, SteamSpy overlord Galyonkin shared a snapshot of figures calculated by Glaiel's technique with Ars Technica. Team Fortress 2 is the most-'owned' game on Steam, according to that, though of course it is free-to-play now. Counter-Strike: Global Offensive is top of the paid games, then Playerunknown's Battlegrounds isn't far behind. And I'm sure our Alec will be relieved to hear that an estimated 3,759,377 other lost souls have clicked a portion of their life away in AdVenture Capitalist. You're not alone there, pal.