The Twitch Streamer Using Face Recognition to Make Video Games More Accessible

Two years ago, developer Valentin Squirelo went on a mission. With Twitch fans at his side, he and his viewers began work on a face-recognition program that allows disabled individuals to control game inputs through their expressions. For Squirelo and his community, the program—dubbed PlayAbility—was the answer to the limitations of accessible hardware.

“We are huge believers of connecting everything together,” Squirelo says. “We have created PlayAbility not to replace everything but to connect with everything. So that's why we choose to focus on face gestures, being able to move analog triggers, analog sticks, stuff that you can't really do with other software.”

Having software that can enable a player to use facial movements rather than a game controller is huge. Despite being designed for disabled gamers, Microsoft's Xbox Adaptive Controller and PlayStation's Access Controller don’t work for everyone. They're also cost-prohibitive for some players.

Other adaptive third-party controllers exist but aren't compatible with all systems. Combine that with the fact that each piece of accessible hardware primarily addresses only a certain set of needs and you’re left with a scenario where there’s no universal device that can serve the more than 450 million disabled players who exist worldwide. And it's all but impossible to create one.

“When you’re looking at accessible hardware, the number one thing that you need to do is look at your client first,” says Erik Johnson, an occupational therapist and the chief medical officer of Warfighter Engaged, a nonprofit that makes modified game controllers for veterans. “Time and time again, what I will find is that people will try and match the hardware with the client rather than the client with the hardware. In the community of people with disabilities, there’s always going to be something they can’t do.”

PlayAbility, Squirelo hopes, can bridge the gap between accessible hardware and disabled individuals who need it.