Last week I was at the Game Developers Conference and heard many complaints about lack of innovation. With this in mind I gave a lecture on “Using Touch, Sight, and Gesture in Mobile Games”. Whilst I covered using RFID/NFC and the Camera on the mobile phone as a novel interface, the aspect that seems that drew most interest from the audience, and later on the web,  were some demonstrations (available on Youtube: here and here) for games we (myself, Tamas Vajk and Will Bamford) created using the Nokia 5500 in-built accelerometers  to control games running on a PC via Bluetooth . The original game was based on the old tron classice light riders is shown below but the videos show the newer Tilt Racer.

My proposition in the whole talk was that using Nintendo and the DS and Wii as an inspiration for attracting a non-traditional gamer audience (although some comments on web relating reports of my talk wrongly assumed I was suggesting phones as a replacement). I believe that accelerometers have an enormous potential for attracting non-gamers as Andreas Jakl already indicated and indeed atracting this audience was highlighted as the inspiration for the DS and Wii by shigeru Miyamoto in his keynote.To highlight this in a none-game setting I thought I would also add some plots of the sensor outputs to illustrate my point.

In the following we see the three accelerometer outputs when the phone was simply placed upon the desk. It can be seen that outputs x and y are approximately zero although some sensor noise is and output z is showing a positive 1 g force which is the effect of gravity on the device.  Note: that this noise could be reduced by the application of a digital filer on the output values but in this case we are showing the raw data. Overall this Figure illustrates that the effects gravity provide a means of deducing the orientation of the phone which can then be utilized to provide a ‘tilt’ based controller.

In following we see the accelerometer outputs resulting from a rapid sideways movement of the phone across a desk. At the start we see outputs in the same state as in first Figure and then very large acceleration forces on y produced by the rapid sideways movement. There are some smaller residual artefacts from this movement seen in both x and z but this because the test was performed by hand rather than using a fixed jig and its difficult to isolate a movement completely under such conditions.  We clearly show the potential for utilizing ‘hand gestures’ for the control of events within. However, gesture recognition requires careful study as the variation in how the user holds the phone could produce anomalous outputs and there is no method of obtaining the phones physical position within actual space.

The final figure shows the output after the phone has been dropped. The trace shows that initially the phone is held upright with the screen held at the top and gravity is acting on x. When the phone is dropped the effect of gravity is overcome and all three accelerometers are zero before the phone hits the floor with a jolt. The aim of the test was to show that the phone could be used to measure activity of a player as per the snowboarder application developed by Andreas and his students.

Hopefully the games and applications already developed highlight the potential for developers but I would also urge Nokia to add these sensors to as many of there devices as possible as having a critical mass of devices in the market will be essential for getting operators to take this seriously.