The sense and nonsense of haptic technology in games
Jan Jacob Mekes

It seems a beautiful day. Suddenly the sky clouds over, and out of a lake of lava, and enormous monster rises, bent on devouring you. This spectacle is accompanied by a heavy vibration in your hands. This event would very probably not take place in real life, but in computer games, situations like these are commonplace.

Computer games or video games, usually referred to as just 'games' - as in this essay - offer a splendid opportunity to get away from reality for a while. Razor-sharp graphics, crystal clear sound, and a vague vibration in your hands that, when hitting back a tennis ball, hardly feels different than with the appearance of the monster referred to earlier. Exactly this superficial use of the sense of touch when playing games can, in my opinion, do with some improvement. How should this haptic technology - which works behind the scenes of these vibrations - be improved then? And what do other gamers say of the use of haptic technology in games?

Haptic technology - what is it?
It would be good to, before answering these questions, understand what haptic technology actually does, and what it is used for in games. Where traditional interfaces only make use of visual and audial signals, haptic interfaces also generate mechanical signals that stimulate the user's sense of touch.¹ In practice, with games this means that the sense of touch in the hands is stimulated, mainly because those are used both in sensing input and in acting upon that input.²

This is, in games, achieved by equipping input devices such as joysticks and gamepads, with a mechanism that creates vibrations (this is known as 'rumble') or a feeling that evokes a sense of resistance (this artificial resistance is also called 'force feedback').³ Since the mid-1990s, haptic technology has been used to simulate, as Immersion, the leading haptic technology producer, states, 'textures, bouncing and hitting a ball, and vibrations from fun fire'⁴ - vibrations that, I feel, hardly differ from each other.

What do gamers think?
Perhaps though, my opinion is far from representative of what gamers think, so I have asked others what they think of the way haptic technology is used in games. To that end I have placed a questionnaire on a popular web site, asking the visitors whether they would say rumble or force feedback adds anything to the games they play. Out of the 354 people who responded, 64 percent agreed with this notion.

However, a problem with this site is that it is not just visited by gamers, which means the results could be skewed - perhaps the enthusiasm about haptic technology is even greater among people who regularly play games. It would also be interesting to find out the reasons why haptic technology is or isn't appreciated. To be able to do this, I have asked 9 people who regularly play games, through a discussion forum, some questions concerning their experiences with haptic technology in games.

One of them puts into words my feelings on the way haptic technology is now usually used in games: 'It felt gimmicky to me in the beginning, and now it just annoys me to the extent that I turn it off in games that have it as a possibility. It feels too artificial to me to convey any sort of feeling or build up a certain atmosphere.' He does, however, name examples of situations in games where rumble can be used in a way that makes the gamer a part of the game instead of pushing him away - according to him, simulating a heartbeat is such a way.

I encountered this line of thought with other respondents as well. In certain circumstances, according to them, haptic technology can contribute to immersion, more than just images and sound would be able to. For instance, one of them names a game where the controller shakes and vibrates when the main character is about to die, which should better convey the feelings of the main character to the player. Not only seeing and hearing, but also feeling an explosion or gunshot, was also named by one of the respondents as something through which he feels more involved in the game he's playing.

Haptic technology seems to be able to add something to games, but at the same time, the way that this technology is currently applied, is seen as superfluous in certain games, and sometimes even as outright annoying. Are there perhaps other ways in which the gamer's sense of touch can be involved in the game more - ways that garner mostly positive reactions? To be able to form an answer to this question, I have looked at some recent developments concerning haptic technology, and asked the previously named 9 gamers about them, in order to better understand how haptic technology could best be developed in the future.

The future
The first example of those developments is a system by Philips called amBX, which, among other things, uses light, but also haptic technology - such as rumble in the armrests of a couch, and fans that simulate wind - to involve the gamer more with the game. This system was launched by Philips in 2006, with the expectation that PC gamers in particular would jump on this technology from the get-go.⁵ What do the gamers themselves think?

The consensus among the 9 respondents is that this system would probably better contribute to immersion than rumble alone would be able to, but at the same time, concerns are raised about the price. Rightly so, because buying the entire system would set one back about 300 Euros.⁶ One person also wondered whether this system wouldn't distract from the gaming experience more than it would add anything.

Another development in the area of haptic technology is less costly, but the question is if the resulting product doesn't distract from the game as much, if not more so. Namely because the product in question is a vest that should be worn by the gamer. Once connected to the PC, the impact of virtual bullets and bombs are thus simulated as if it would seem that the player himself is under fire, and that for less than 200 dollars.⁷

The opinions on this application of haptic technology are divided. Some of the respondents are not interested, the main reason being that wearing such a vest could be a hindrance, and that directly feeling the impact of a bullet would make the game a little too real. Others are enthusiastic, naming the greater immersion caused by this vest, as something positive. All in all, this product seems to mainly target the serious gamer who plays a lot of shooting games - a puzzle game clearly wouldn't benefit from such a vest in any way.

It is interesting to see that not everyone is opposed to haptic technology that stimulates not just the hands to make a gamer feel more involved in the game. Some people are, as said, open to using a vest to stimulate several parts of the body. But where are the boundaries of what is experienced as a pleasant simulation - does it stop at pain?

In the beginning of the 21st century, researchers of Harvard Medical School discovered that right before people feel pain, the area in the brain is activated that is used in enjoying pleasant things, such as money, food, and sex.⁸ Apparently, there is a certain connection in the brain between pain and pleasure. Would this be a reason to also use pain when simulating sensations during a game?

Such an application of inducing pain has really taken place, in the form of an art project named 'PainStation'. After having been in development for a year, the PainStation was released in 2001 by two students of the Kunsthochschule für Medien in Cologne. The use of pain in games was apparently so successful, that not long afterwards, a second version of the machine was released.⁹

The machine, that offers as its only game an adapted version of the two-dimensional table tennis game Pong, operates as follows. The player places his hand on a button, that must be held at all times, or the game is over. When he is losing, his hand is burnt with a heat lamp, or he receives an electric shock, or his hand is hit with a small plastic whip - and sometimes all at the same time. This sounds rather uncomfortable, but, according to the makers, more people love the PainStation than hate it.¹⁰

Still, the PainStation is a separate case, meant more as an art project than as a regular computer game. The question is, then, if inducing pain could have a place in normal computer games. I have also asked this question to the panel of gamers, and the reaction is an almost unanimous 'no'. One of them expresses himself very strongly: 'who the hell thought that was a good idea', is his only reaction to the PainStation. And yet, two of the respondents would not find it a bad idea to include pain in the panoply of game-induced sensations, providing it wouldn't leave any lasting injuries. So the radical way in which the PainStation induces pain, is a bridge too far.

The question remains in which way haptic technology should go. Should we start redecorating our living rooms, change clothes, or even endure pain, simply to play a game? To get an idea of what gamers themselves expect from haptic technology in the future, I have asked the 9 respondents about their expectations. This brought out several different ideas.

One respondent brings up the use of temperature as a possible way of involving the sense of touch in a game more - most of the others aren't so radical in their predictions. One possible reason for this is named by one of them: 'To be honest, the idea of full immersion sort of scares me, things can become a little too real [...] I like that there is still a division, that what you see if [sic] just in front of you.' Consequently, most of the suggestions involve improvements on the way haptic technology is currently used in games: the wish to be able to feel textures is uttered, and to improve vibration technology in such a way that it is less irritating after a long gaming session.

Perhaps an upgrade of the current technology is he best way to ensure a gamer-approved future of rumble and force feedback. And perhaps there is not a whole lot to improve about the current concept, in which the hands are the main receptors. In any case, the full potential of current haptic technology has not been reached yet, but perhaps this will change when not only the producer of the technology, but also the makers of games give more attention to haptic technology. For instance, this could be by appointing a so-called 'haptic artist', who is responsible for the use of haptic technology in games.¹¹ There is, in any case, still room for improvement.

Blind and deaf gamers
To be able to form a complete judgement about what the future of haptic technology should look like, all sorts of gamers need to be taken into consideration. So far in this discussion, a group of gamers has not been mentioned - a group that may find haptic technology most useful. I am talking here about gamers who miss one of the senses that are normally used by gamers - sight and hearing. Does haptic technology answer to their wishes, and do they profit from it at all? To find out, I have asked several blind and deaf gamers, through Internet forums, about their opinion.

Although the blind have lost their sense of vision, they are still able to observe the contours of the world around them through their sense of touch - it is the sense of touch, not the sense of vision, that places us in a solid world.¹² As such, research has made clear that blind people are able to, by touching statues, place in its temporal context and, perhaps even more important, appreciate a work of art.¹³ Other research made clear that blind people are, to a degree, even able to correctly identify a picture, by feeling with their fingers a relief of it.¹⁴ The question is, then, if the sense of touch can also be used effectively in games for the blind, which normally only use sound.

The very first reaction that I get, however, points out a practical problem: to be able to use haptic technology, a gamepad or joystick has to be connected. The problem with this is that not many blind people purchase such gaming devices, because they are supported by only a few games that are specifically made for the blind. As a result of this scarce distribution of gamepads and joysticks, most games developed for the blind do not take into account their use - in other words, there appears to be a vicious circle.

That is a shame, because other reactions show that haptic technology can add something to a game for the blind. One blind gamer, who writes under the name 'dark empathy', gives the example of using vibration in stead of sound when a character sustains damage. However, he adds that he would only purchase a gamepad if more games would support its use.

Another blind gamer, 'cx2', who indicated earlier that gamepads and joysticks are not used very often, gives another reason for this. The software that is used to configure these devices, is often hard to use with a screen reader. And yet he does share the feelings of 'dark empathy', referred to earlier, and of 'zhtfreak', a third blind gamer who is enthusiastic about the possible use of haptic technology. Thus, the problem here is entirely practical - the customer demand is there, but the producer is often not able to answer this demand, for several different reasons.

Then what about deaf gamers? The fact that they miss their sense of hearing, does not mean that they are altogether unable to experience the things that sound has to offer. For instance, in listening to music, not only the sense of hearing is used, but also the somatic senses.¹⁵ It might be expected, then, that somehow those somatic senses can be used to let deaf gamers enjoy sound.

From reactions from deaf gamers it appears, as expected, how much they appreciate the use of haptic technology. One of the deaf gamer respondents relates the following about his first experience with rumble in a video game: 'I play and I hit the throttle and I FELT it for the VERY FIRST time in my gaming life. I don't know what to call it, but if I remeber [sic] it this vividly, it must have been quite profound.' He goes on to say that his friends, whom he let play his rumble-enabled games, were also enthusiastic, and felt more involved in the game.

Haptic technology not only offers pleasure to deaf gamers, sometimes it is necessary to be able to play a game at all. A deaf gamer calling himself 'Deaf Smith', who told about his first experiences with haptic technology, explains. 'I would say I really need it. I need it big time in a FPS multiplayer game so I can be aware of when I am being shot at, or a shot has just registered on me,' he says about the usefulness of haptic technology in shooting games.

Other deaf gamers confirm the necessity of making games more accessible to them through haptic technology. 'Vanquishhh' says the following about this in relation to Halo, a shooting game: 'In Halo, using a rumble helps me to have a sense of what my surroundings is like whether there was a grenade exploding behind me or of course, bullets hitting me.' Another deaf gamer, 'Audist', also emphasizes how haptic technology helps him: 'The rumble feature in Halo 3 helps my map awareness. I know if grenades are being blown up behind me, or where I am being shot at. It's quite a helpful feature.'

So, deaf gamers gain a lot from using haptic technology. At the same time, 'Vanquishhh' expresses the concern that the game industry pays little attention to deaf gamers, because they are a minority compared to the gamers who can use their sense of hearing. If game makers would give more attention to deaf gamers' needs, this could result in them perfecting haptic technology at a higher rate - and other gamers would likely benefit from this as well.

Conclusion
In summary, it can be said that haptic technology can definitely add something to a lot of games, although not everyone appreciates this on the same level. That haptic technology isn't for everybody, is especially apparent in applications of this technology for which the living room (as with amBX) or clothing (as with the haptic vest) has to be adapted. Yet stimulating just the hands can make a gamer feel more part of the game, especially, and that is key, if haptic technology will continue to be developed further.

Special attention goes out to blind and deaf gamers, of which the latter group is currently best able to use haptic technology. Producers of both input devices and games will have to take into consideration the needs of both these groups. Then not only blind and deaf gamers, but all gamers would get more pleasure from their games. In short, the sense of touch is instrumental to experiencing games - a fact that is sadly overlooked all too often.

---

Notes

1. Vincent Hayward, Oliver R. Astley, Manuel Cruz-Hernandez, Danny Grant and Gabriel Robles-De-La-Torre, ‘Haptic interfaces and devices’, in Sensor Review 24 (2004) 16.
2. Hayward, ‘Haptic interfaces’, 16-17.
3. Hayward, ‘Haptic interfaces’, 25.
4. History of Haptics Technology (January 2, 2008).
5. amBX: Rumble in the Sofa (January 2, 2008).
6. Philips amBX Premium Kit (January 2, 2008).
7. FPS Vest/Game Bundle (black) (January 3, 2008).
8. Pleasure, pain activate same part of brain (January 3, 2008).
9. From art to arcade (January 3, 2008).
10. PainStation: Gaming Till It Hurts (January 3, 2008).
11. Does Rumble Matter? (January 4, 2008).
12. F. David Martin, ‘The Autonomy of Sculpture’, in The Journal of Aesthetics and Art Criticism 34 (1976) 280.
13. Frances W. Herring, ‘Touch: The Neglected Sense’, in The Journal of Aesthetics and Art Criticism 7 (1949) 205.
14. Dominic M. M. Lopes, ‘Art Media and the Sense Modalities: Tactile Pictures’, in The Philosophical Quarterly 47 (1997) 429.
15. F. David Martin, ‘The Autonomy of Sculpture’, 281.