You Are Here:

Community: Blogs

Arto Holopainen's Forum Nokia Blog

Symbian device control using muscle activity - a case study

ahoccc | 13 September, 2006 12:02

My previous blog “Symbian and smart monitoring of human motion” seemed to inspire a lot of people, so I decided to go deeper into this subject.

 

In this blog I present a real world working case for controlling your mobile using your muscles.

 

Case setup:

  • Series60/Series80 mobile phone
  • Small battery-operated Bluetooth enabled two-channel EMG monitoring device (MultiTrainer by Mega Electronics Ltd)
  • MuscleMonitor application. This application was built specifically for this case and it monitors muscle activity (EMG) with two channels, so two different muscles (biceps in this case) can be monitored, one channel for each muscle.

Controlling Symbian device with biceps MuscleControl case setupThe application shows muscle activity as graphical trend and is capable of basic operations such as muscle balance calculations, muscle activity counters and min/max follow-up.

 

I think this is quite intriguing as it is but then I added possibility to map certain muscle activity level to Symbian commands (key press, key press sequences etc.). So, basically using muscle by contracting and relaxing I can execute certain commands for Symbian device. And these commands are sent to the foreground (active) application. In essence, the application monitors muscle activity in the background and sends commands to the active application based on muscle activity.

 

As a proof-of-concept, I configured muscle activity command sequence for the Nokia SnakeEx game controls. You know the Snake game, where you control snake in a maze trying to collect food and avoid crashing to the maze borders. Following picture shows the command sequence configured for this case.

MuscleControl SnakeEx commands 

All this means simply that when right bicep is contracted, snake turns clockwise and when left bicep is contracted, snake turns counter-clockwise.

 

However as you can see from above picture, it is not enough just to execute one command e.g. when turning clockwise but several depending on the current snake direction. Also when contracting both biceps at once, left softkey is executed. This allows you to initiate new game without touching the device.

This application actually works as a general muscle-controller for Symbian device. With a few adjustments, application could follow the active foreground application name and based on that change command set on-the-fly. So when you are in messaging application, you could e.g. use muscles to write messages with T9. And when some game is active, command set is related to that game. Also different muscle contract-relax sequences could be mapped to different actions as well as different muscle activity levels. As I see it, possibilities as enormous with this kind of “bio-control”. Even more, in this case only two-channels were utilised, but when adding more channels you could use more muscles and more controls.

 

I must say that when you have played the game for some time, you feel like you have been doing real workout.


[added later]

Please review the addition "Symbian device control using muscle activity - a case study part 2" to this case as well.

 Find more related topics in my blogs here.

RSSComments

Re: Symbian device control using muscle activity - a case study

coultonp | 13/09/2006, 13:06

coultonp Arto

This is very cool and a really innovative use of technology. I think this whole area of using mixed reality gaming has enormous potential for public health and its something I have been trying to pursue in my own research. Interestingly in the UK this week we report about the dangers of the sedentary lifestyle invoked by the computer screen http://news.bbc.co.uk/1/hi/uk/5338572.stm which provoked much discussion in the papers and on TV. I think the mobile development community has a lot to offer in this area and is perhaps something we should be promoting with greater vigour particularly for games.

Re: Symbian device control using muscle activity - a case study

arturo_duro | 14/09/2006, 12:40

How about a bluetooth heart monitor belt + bluetooth GPS antenna to monitor sport fans performances? I solicited Polar some months ago. They seemed interested. In the meanwhile, though, Garmin went to market with this brand new Watch/wireless HR monito/GPS: Garmin ForeRunner 205 GPS Trainer.

cYa

Re: Symbian device control using muscle activity - a case study

ahoccc | 14/09/2006, 14:52

ahoccc Yes, heartbeat and GPS together are quite a powerful combination. There is also company called FRWD (www.frwd.fi) that manufactures device like that. Their device is a small "black-box" that has heartbeat monitoring, GPS as well as Bluetooth for on-line monitoring e.g. with Series60 devices.

This kind of devices with mobile phones does not only allow very powerful personal training monitoring but also on-line data transmission e.g. to personal trainer who can monitor training in real time regardless of the location. Actually this kind of experiments have been conducted with certain sport events, where online measurement data from the contestant is transferred via mobile network to server that feeds it to the TV transmission.

Re: Symbian device control using muscle activity - a case study

arturo_duro | 21/09/2006, 10:54

Thank you Arto that's a valuable piece of info although the FRWD solution is a closed box one. I managed to install their demo app on my phone and it seems to be device dependant. You must have purchased their HRM/GPS unit in order to make the GPS BT connection work properly.

Furthermore I would like to still use my favourite GPS navigation app while HR-recording thus an independent HRM unit would have been more appealing for me even if it raises some question about which BT profile will use since, for instance, my GPS unit already occupies the BT serial profile. So a new BT standard profile for health monitoring pourposes would be required and it seems to be on its way to mass market. Have a look here:

http://www.electronicsweekly.com/ARTICLES/2006/05/24/38777/Intel+plans+wireless+blood+pressure+monitors.HTM

cYa

Re: Symbian device control using muscle activity - a case study

ahoccc | 21/09/2006, 19:14

ahoccc You are quite correct about FRWD device. It is closed-box but it effectively avoids the multiple Bluetooth connection issue. But of course it comes with the down-side that you cannot use the device with separate applications. Would it be nice if there would be some kind of server API available that would route Bluetooth connections to clients that have requested it? This way several applications could listen incoming data flow from same Bluetooth device at the same time...

About the Bluetooth profiles: actually there is no reason (from the Bluetooth point of view) why you could not use several SPP connections at the same time. It is the device (or the Bluetooth stack) that limits the amount of simultaneous connections. It is quite limiting feature of current Symbian devices for certain type of applications. For that reason I have used Windows Mobile devices that support several simultaneous SPP connections e.g. for applications that monitor heart rate, oxygen saturation and muscle activity at the same time (three separate Bluetooth connections with different measurement devices).

But in the future I strongly believe that also Symbian devices will support more simultaneous Bluetooth connections than now. There was time when you could not use GPS and Bluetooth headset at the same time with Symbian...

Re: Symbian device control using muscle activity - a case study

mayankkedia | 14/12/2006, 16:52

Indeed awesome use of technology and the immense possibilites of the same.

Great work..keep enlightening..

Cheers
mayank

FRWD Bluetooth format

Friendly Reader | 22/12/2007, 04:34

Does anybody have any information about the FRWD Bluetooth protocol?

I would like to know how to read strings like:

FRWD01FB0102```bhfbnhbrj```````````````````````````Z```lnZ```lnZ```ll````bb`dbhhVdfnrrrrrrrrrrrrrrrr*0C

I have a 1 GB memory chip on my Nokia 6682 and I'd like to record
performance data to that chip, and then parse the data on the phone and
schedule more exercise time in my calendar when I'm being lazy.

I have an FRWD F500.

Thank you.

Undefined symbol: 'class CRDAccelerometer * CRDAccelerometer::NewL

kulvijay | 13/02/2008, 13:06

Hi i'm developing a small accelerometer based app for N95. I downloaded the RDAccelerometer plugin. I could not build the Moving ball app on carbide. Its throwing an error "Undefined symbol: 'class CRDAccelerometer * CRDAccelerometer::NewL(class MRDAccelerometerObserver &) (?NewL@CRDAccelerometer@@SAPAV1@AAVMRDAccelerometerObserver@@@Z)"...

i hv checked the mmp file..I did add the RDAccelerometer.lib.. i dnt hv any clue why its not working.. Plz help..
wat do i do????

test with NOKIA N80 IE

mdrahman.nokia770 | 13/07/2008, 23:33

hi ahoccc,
I don't know how busy you are in these days. Can I request you to help me in one of my projects? I believe you are very capable of doing that. I am eagerly waiting to hear from you.

You must login to post comments. Login
 

Rate This

 
 
Bookmark this page: DeliciousDiggFacebookGoogleYahooStumbleUponRedditDiigoTechnocratiTwitter  Share this page Share this page Print this Page Print this page Invite a friend Invite a friend
京ICP备05048969号    Email Newsletters Press Terms & Conditions Privacy Policy Sitemap Contact Us © 2009 Nokia 
RDF Facets: qdcZidentifierQSxhttpE3aE2fE2fblogsE2eforumE2enokiaE2ecomhttpE3aE2fE2fblogsE2eforumE2enokiaE2ecomE2findeE78E2ephpE3fopE3dViewArticleE26blogIdE3d38462E26articleIdE3d198X qdcZtypeQUqfnZE45E78cludedFromGeneralE4cistingsQ qdcZtypeQUqfntypeZBlogContentQ qdcZtypeQUqfntypeZBlogE45ntryQ qdcZtypeQUqfntypeZCommunityContentQ qdcZtypeQUqfntypeZE52esourceQ qdcZtypeQUqfntypeZWebpageQ qdcZtypeQUqmarsZManagedE52esourceQ qdcZtypeQUqwebZInformationE52esourceQ qdcZtypeQUqwebZPageQ qdcZtypeQUqwebZE52esourceQ qdcZtypeQUqrdfsZE52esourceQ qfnZtopicQUqfnTopicZbusinessE5fopportunitiesE5fservicesQ qfnZtopicQUqfnTopicZbusinessE5ftechnicalE5fservicesQ qfnZtopicQUqfnTopicZentertainmentQ qfnZtopicQUqfnTopicZgamesQ qfnZtypeQUqfntypeZBlogContentQ qfnZtypeQUqfntypeZBlogE45ntryQ qfnZtypeQUqfntypeZCommunityContentQ qfnZtypeQUqfntypeZE52esourceQ qfnZtypeQUqfntypeZWebpageQ qfnZuserE5ftagQSxentertainmentX qmarsZlanguageQUxhttpE3aE2fE2fswE2enokiaE2ecomE2flanguageE2d1E2fenX qrdfZtypeQUqfnZE45E78cludedFromGeneralE4cistingsQ qrdfZtypeQUqfntypeZBlogContentQ qrdfZtypeQUqfntypeZBlogE45ntryQ qrdfZtypeQUqfntypeZCommunityContentQ qrdfZtypeQUqfntypeZE52esourceQ qrdfZtypeQUqfntypeZWebpageQ qrdfZtypeQUqmarsZManagedE52esourceQ qrdfZtypeQUqwebZInformationE52esourceQ qrdfZtypeQUqwebZPageQ qrdfZtypeQUqwebZE52esourceQ qrdfZtypeQUqrdfsZE52esourceQ