Eye tracking for gamers: seeing is believing

Screen shot from Hitman video game

Gaming with an eye tracker feels like virtual reality-lite. It’s cheaper, easier to set up, and is already available in some of your favorite games. It offers some of the same features as VR, too: you can swivel the camera by moving your gaze or turning your head, for example.

But can it fundamentally change the way you play? And what’s next for the tech? Tobii is the largest company in the space, so TechRadar tried out its flagship eye tracker – the 4C – and spoke to its head of gaming to find out.

The Tobii 4C

The 4C is much larger than I expected. It’s a black bar that stretches 13 inches across the bottom of your monitor, which is most of my laptop screen. It’s chunky enough to stop you closing your laptop when you’re finished playing, but it does at least have a discrete, sleek finish.

It plugs into your laptop or PC via a USB cable, and you attach it to the bottom of your monitor with a sturdy stick-on strip. It’s easy to set up, and after some quick calibration you’re ready to jump into one of the 120+ games that support it. The list includes everything from sprawling RPG Kingdom Come: Deliverance to space sim Elite Dangerous. Tobii has especially close ties with both Square Enix and Ubisoft, which means a lot of the games those studios publish or develop are compatible, including Assassin’s Creed Origins, Final Fantasy XV and Deus Ex: Mankind Divided.

The Tobii 4C eye tracker

The Tobii 4C eye tracker

The tracker has two main components: cameras and illuminators. The illuminators shine infra-red light into your eyes, while the camera take pictures. Among other things, the camera is looking for the 'twinkle' of the screen’s reflection in your eye, and measures the angle between that spot and your pupil to work out where you’re looking; it’s taking 90 pictures a second to make sure it stays responsive.

The results are reassuringly accurate. It picks up my eyes from a variety of angles and distances away from the screen, and it works even if I sit off-center. When I turn on ‘gaze trace’, which traces where the system thinks your eyes are looking on screen, my point of focus is always within the boundaries of the white circle.

If you turn on 'gaze trace', the Tobii 4C will display where it thinks you're looking, shown by the white shape on the right of the screen here

If you turn on 'gaze trace', the Tobii 4C will display where it thinks you're looking, shown by the white shape on the right of the screen here

It’s never bang in the centre of the circle, however, which means that it sometimes makes mistakes. When I try to select an option in the Tobii desktop menu, for example, it occasionally thinks I’m looking at the option below or above. Tobii says it expects slight variations in accuracy because “everyone’s eyes are just a little bit different”, which might explain the discrepancy.

Within 10 minutes of playing Hitman I’m glancing up at the sky and through windows without even thinking about it.

But it’s more than good enough for gaming, which is what I’m really interested in. Supported games will each offer different features, but the one that’s most common is ‘extended view’. Essentially, this lets you move the camera with your eyes: if you look at the edge of your screen, the camera will move that way. It won’t fully turn – it’s more of a gentle pan in that direction – but it’s a useful way of taking a glance at anything that’s just out of shot.

I expected it to be a gimmick, but it’s natural to control, and makes me feel more present in the game worlds. Within 10 minutes of playing Hitman I’m glancing up at the sky and through windows without even thinking about it. 

Some games also let you turn your head to move the camera even further. It works, but it feels a lot less natural, like I’m turning my head further than I ever naturally would when sat in front of a monitor.

You can see both features at work in the video below, recorded in Hitman. After walking down the steps, I don’t touch the controls at all. The first pans – to the four corners – are controlled by my glancing at the edge of the screen, and then I turn my head to left and right for bigger swings.

That clip also shows off perhaps my favorite eye-tracking trick: the game will hide bits of the UI when you’re not looking at them. That means the mini-map in the bottom left is transparent during normal gameplay, and only fills in when you glance in its direction. I love minimal UIs, so I’ll take anything that makes my screen less cluttered.

But the best features come when eye tracking is more integral to a game’s controls. The best example I’ve found is in Deus Ex: Mankind Divided, where you can move from cover to cover using your eyes. You simply look at the spot that you want to go to next and press a button to send Adam Jensen scurrying off, as you can see in the clip below. 

It feels smooth and intuitive. As with a lot of eye-tracking features, it’s especially useful if you want to play with a controller rather than mouse and keyboard. A mouse can give you the fine control that a joystick often can’t, so being able to point with your eyes is a good compensator. 

I’m still not sold on the act of aiming a weapon with a glance, though. That’s another common feature, which I try in Human Revolution, Hitman, and co-op hack-and-slash game Vermintide 2. In the first two, your weapon will snap to wherever you’re looking if you press the aim-down-sights button. In Vermintide 2, you can aim the wizard class’s fireballs with your eyes: you just click the fire button and he’ll shoot them wherever you’re looking.

It sounds cool, but it feels alien and awkward. In Vermintide 2 my fireballs are accurate, but aiming away from the center of the screen doesn’t look right, and I much prefer using a mouse or game pad, while aiming down sights in Hitman and Human Revolution is jarring because the camera instantly snaps to the spot you’re looking at. Take a look in the clip below.

'The sky's the limit'

Tobii’s head of gaming, Martin Lindgren, tells me the company is constantly trying to improve both the tech and its integration with games. The earlier Tobii gets on board in a game’s development, the deeper it can embed eye-tracking mechanics, and so it’s constantly building up relationships with developers. 

Lindgren says “the sky’s the limit” when it comes to the number of games that could implement eye-tracking, and he expects more and more developers to adopt it in the coming years. “The more games we do, the more integrated we get into the core of the games, the more we can affect the gameplay,” he adds.

“The more games we do, the more integrated we get into the core of the games, the more we can affect the gameplay.”

Martin Lindgren, Head of Gaming, Tobii

As I’ve found, some of the current uses feel lightweight, but the camera panning is particularly impressive. I try it out in racing sim F1 2017 too, and it works well, letting you look into the corners as you round them. Lindgren says it’s his favorite eye-tracking feature too, specifically in space sim Elite Dangerous.

“When you play space simulators, which I play a little bit of, with joysticks, you have no way of controlling the camera, it just looks in the direction you steer,” he says. “But with our eye tracker, you can decouple the camera from the movement of the ship. So while you’re flying around you can look around freely while still piloting it.”

If Tobii can work with developers to add more tricks that feel as natural as extended view, then it will make an eye-tracking kit much more desirable.

Refining the tech

Away from speaking to more developers, the next step for Tobii is to make the trackers smaller and faster, Lindgren says. It’s also working with eSports pros and broadcasters to track where players eyes are looking during a match, which could improve our understanding of what sets the best players apart.

Foveated rendering is another innovation Lindgren is excited about. Because an eye tracker knows where you’re looking, it’s possible to reduce the resolution of the rest of the screen, where your eyes won’t notice the difference. Then, when your eyes move, the image could shift so that wherever you look is always displaying in high resolution. 

Like a lot of the innovation in eye tracking, it’s probably coming to VR headsets first. Oculus plans to have moving lenses within its next VR headset, and the suspicion is that will involve some sort of eye-tracking camera. For Tobii’s part, Lindgren will only say that the company has “discussions with everyone in the VR ecosystem”. Watch this space.

For now, much like VR, eye tracking feels like a luxury item, and that’s reflected in the 4C’s price tag of around $150 / £125 / AU$200. And just like VR, for now its lack of killer applications means it’s not an essential piece of kit for PC gamers. But it’s certainly worth keeping an eye on. 

TechRadar's Next Up series is brought to you in association with Honor