The quest to solve VR's biggest problem: walking around

Jason Rubin is thinking a lot about whether or not we’ll feel comfortable while taking a walk. The scope of the former THQ president’s job is as big as it ever was, but as head of Oculus’ game development group, the problems he’s talking about seem quaint in comparison to that struggle: exciting blue sky futuristic stuff that it’s still a bit hard to believe is real. For instance, how are we going to move around virtual reality worlds while sat on our butts? Will it make us lose our lunch?

During a recent visit to Oculus HQ, we spoke to Rubin at length about his development teams and the third-party Rift devs who are solving entirely new problems in the medium. In this primordial stage, new guidelines are being introduced and struck down regularly. Everything is new, even things we take for granted in non-VR games. For instance, before we even get to movement: How do you represent the player’s body, or another player’s body? There’s no single correct solution, and when what you’re making is the first of its kind, there’s no way to know for sure if your chosen method will work.

“It’s going to take a long time for us to get to the point where we’re iterative as opposed to revolutionary,” says Rubin. “So we have these hand tracked devices now, right, all of the VR headsets. I’m looking at you, you’re in VR, I’m in VR, I’ve got three points of information about what you’re doing. I have head rotation and position, hand rotations and positions. I want your whole body. I don’t know anything about your feet. How do I make that look like a human and not like some weird marionette that’s kind of stretched amongst it?”

Likewise, when you look down in VR, you’ll often see your hands floating in the ether. Game developers could model your arms, sure, but they don’t know where your elbow is and using guesswork is jarring. “If you look down and you know your elbow’s here, but the game elbow’s there, you get this weird out of body experience,” says Rubin. So, for now, we’re seeing a lot of those ghost hands in VR games. Eventually someone is going to make a new rule though. Whether it requires the addition of full-body tracking or some clever software trick is unclear, but no rule will last forever.

Walk it off

Rubin isn’t afraid to break the ‘rules’ of what’s currently considered good VR either. When he arrived at Oculus, he was confronted with this law: “Don’t control things with your head.” And yet when tasked with getting a game out to coincide with the release of the GearVR Innovator’s Edition, he suggested Anshar, which has players steer a spaceship with their heads. No one wanted to try it.

“When I came to the company it was a lot of scientists, not game designers,” says Rubin. “So people that had worked in VR, vision or whatever for a long time. But they weren’t used to trial and error to get around things.”

They said it would be uncomfortable—even if not for them, for someone—because steering with your head just doesn’t work. “No! We don’t do those games,” they said, according to Rubin. But he persisted, sending Anshar to John Carmack. Carmack liked it, which convinced others to give it a go, and no one at Oculus found it uncomfortable. A rule was struck down and Anshar Wars 2 was released.

I fundamentally believe we’re going to get to a position where we have a way of locomoting that doesn’t make you uncomfortable.

More vital to Rubin, and I think a lot of people who are curious about VR, is how to move around in first-person without physically walking around. The HTC Vive’s sensors allow players to be slightly mobile within the room they’ve designated as their VR play area, but the Rift is a seated or standing experience. Unfortunately, our brains don’t seem to love it when we use an analog stick to propel invisible legs beneath us.

“I fundamentally believe we’re going to get to a position where we have a way of locomoting that doesn’t make you uncomfortable,” says Rubin. “Probably has to do a little with moving in and out your viewport and making sure that your periphery doesn’t see a lot of pixels moving when you’re moving. Nobody’s cracked that one. I’ve seen a lot of demos that are getting there.”

A big part of the discomfort, according to Rubin, comes from movement in our peripheral vision. He pointed to Ubisoft’s Eagle Flight as a game that’s solving it by blacking out the periphery, which he calls ‘tunneling.’ You can see what that looks like in the video below from UploadVR.

“Most of the discomfort comes from your periphery and a lot of the way that your eyes deal with things when it thinks it’s real,” says Rubin. “As soon as you put it in a television you lose that. So some people get sick playing first person shooters but most people don’t because of the fact that the room is not moving, so I accept the TV is an alternate universe. It’s not even a conscious thing, it’s just the way your brain deals with it.”

So, if we get rid of peripheral vision when moving, we get some of the TV effect. We’re seeing movement in front of us, not moving ourselves quite so much. The idea behind tunneling is simple: when you’re standing still, your vision is full, and when you’re moving, it’s restricted to reduce discomfort. In flight sims, the cockpit may do the job of creating tunnel vision, while other games may have to invent reasons to put blinders on you.

“I think over the next year, [we’ll see] the tunneling approach where you restrict the viewpoint to get rid of the discomfort during movement but then broaden out to get all the benefits of VR when you’re still,” says Rubin. “I think that’s going to be the next stage. Post-Bullet Train, if you will.”

Rubin is referring to Epic’s Bullet Train demo, which solves the movement problem by teleporting the player between stationary points. Other games, including Oculus’ own Damaged Core, use a similar idea. It’s a compromise that’s inspiring some interesting design decisions: Damaged Core, for instance, has the player blinking between robot bodies. It could be a case of limitations engendering creativity. But it’s not something I want from every game, and the same goes for ‘tunneling.’ That’s a limitation, too, because shouldn’t I want to feel the full force of my movement and not get sick? The good news is that Rubin doesn’t think these compromises will be around forever.

We’re going to end up with something that’s not like a first-person shooter of today, but not that dissimilar, and gives you 100 percent of what you want.

“The hardware is going to get better, making discomfort less likely over time. The user is going to get a acclimatized, so first-person shooter players, the first time they play a first person shooter they’re limited, and after they’ve played a bunch of Call of Duty you can throw anything at them and they’re fine. And then the software is going to get better and figure out these tricks to play and a fiction that makes sense in the game to use it, and we’re going to end up with something that’s not like a first-person shooter of today, but not that dissimilar, and gives you 100 percent of what you want.”

As for what Rubin himself, as a player, wants most from VR, the answer is probably on a lot of people’s minds. “I put 132 hours into Skyrim before my wife said, ‘Really? Come on.’ I mean literally New Year’s Eve that year, I didn’t go out.” 

It’s not necessarily Skyrim itself that he wants in VR, but giant open worlds explored on foot, such as the world of Far Cry Primal. Like Rubin, I think just being in a place and exploring it will be a big draw of VR. A flat game can show you beautiful scenery, but a VR game can make you feel like you’re there. Just as long as you can walk around without feeling sick. 

The kind of big game Rubin’s talking about is already expensive and takes a long time to make, and Rubin knows it will be the same deal in VR—plus, there’s the need to figure out locomotion. But he doesn’t think that’ll stop anyone. “People will love it. It’s absolutely going to happen.”

Story by Tyler Wilde; Interview by Wes Fenlon.