Recently, we’ve been doing a good amount of focus testing at the office with external testers. It’s a great opportunity to expose exploits in the design, find out what people like and don’t like, discover if mission pacing is working, etc. We tape each playtest live (screen and tester’s face) with the tester in a room with just a designer or two. The rest of the team stays in our theater across the hall watching live. This is similar to a one sided mirror, basically we don’t want 15 people watching the playtest to intimidate the tester.
Anyways, while this has been useful and will continue to be so as the project runs its course, recently I had the opportunity to hear a presentation from a company who was selling a service where they can supposedly rate the interest of users in your game as they play it. They test this biometrically via a headband the player wears that supposedly measures “adrenaline” by doing EKG readings, pulse, blink rate, and whatever other hueristics they did not mention. They record this data continuously while the tester plays the game and then use some patended algorithms to turn the biometric data into what they are calling “adrenaline” as a way to measure the player’s interest in your game.
They then sell a service to further compare the data for your unreleased game to the data they’ve recorded for games in similar genres. For example, if you wanted to see how your FPS was stacking up to Gears of War they could compare their measurements between the two games to try to show where you could make improvements.
It all sounds very cool on paper but I can’t help but to think it might be snake oil. I’m no scientist but the sheer number of variables that go into this kind of reading concerns me. What if the tester had a really bad day (break up with girlfriend, pet died, etc.)? Does that significantly affect your result? Is it valid to aggregate data from wildly different groups of testers? There’s just a TON of data being recorded with many external variables that can affect the data.
Anyone else out there know more about the science of these types of readings? Can we accurately measure interest level in this day and age through technology?
The highlight of the presentation was personally getting to try out the device. I put on the headband and watched a Budweiser beer commercial as a short test case. They showed the graph of my interest level afterwards. They seemed surprised since my interest level dipped when the beer was finally shown at the end, when typical male interest spikes. Heh, I’m not surprised though… had they shown a Bass Ale or Pyramid beer and logo I would have perked up immediately, I can’t say that I’m a huge fan of Bud.