Realeyes3D – finalist of the 2007 Forum Nokia Developer Innovation event with motion tracking technology

Can you tell us a little about your company?

Realeyes3D is a privately owned company specialized in imaging technologies, applications and services for camera phones. Realeyes3D headquarters are in Paris, France and has offices in Tokyo, Hong Kong and San Francisco.

The company’s mission statement is to “take camera phones beyond taking pictures” by enabling innovative camera-based applications and services such as handwritten messaging, mobile scanning and motion detection.

What background do you have, previous software projects, success cases?

Realeyes3D core competences cover three areas: Imaging (several innovation prizes since 2001 for our image processing technologies), Mobile terminals (We are development experts in all major platform environments: Symbian S60, BREW, Windows Mobile, Java and proprietary platforms) and Server technology (Qipit.com consumer service for mobile scanning was launched in September 2006).

Realeyes3D is the world leader in Mobile Handwritten Messaging with its products Digitizer (enabling the capture, processing and sending of handwritten notes, drawings, doodles) and w-Postcard (picture decoration and personalization).
Since 2005 over 50 millions handsets have been shipped with Realeyes3D applications factory pre-installed.

Was there something in particular that sparked off the interest in camera based motion tracking technology?

The idea came from the explosion of content on the handset and more specifically dense and high resolution content (megapixel pictures, large size web sites, large maps, scanned documents, email attachments, etc.). On the other hand the physical size of the handset displays can’t grow much larger; today, a 3.2 Mpix camera phone image is often up to 40 times the size of a handset screen. Traditional navigation based on menus (Zoom In / Out) and keys became more and more tedious as content get larger. There had to be a solution to navigate inside this content easily and intuitively.

In parallel the motion detection paradigm started to get momentum with the launch of the Wii console from Nintendo and in other industries.

As for the use of the camera to detect motion, the idea came from analysts’ prediction that while camera phone resolution will continue increasing, hardware accelerometer solutions will remain confidential in volumes for the next 3-4 years: there is plenty of room for a software based solution if you can make use of the existing sensor - the camera module.

What challenges did you face in developing theis technology?

Our target was to enable motion navigation without extra hardware cost. And being experts in imaging for camera phone, we naturally focused on the use of the camera preview to calculate the motion.

Some of the challenges we faced are related to the accessibility of the camera APIs on some platforms.
Other challenges were related to the user experience and how we can make the navigation paradigm intuitive and easy knowing that two main actions are possible: Zooming in and out (2 way decision), and panning around (in all directions).

Finally we had to solve the challenge of minimizing battery consumption while the camera is being used for navigation.
To overcome these last two challenges, we implemented a two-key navigation, while the user is pressing the key, the navigation mode is activated (zooming or panning). When the user releases the key, the camera is released; preserving the battery moreover the image stabilizes and is no longer sensitive to ancillary movements.

And lastly, we have developed mechanisms for memory management that not only take advantage of the increasing memory available in recent handsets, but also work perfectly on handsets with more limited amount of memory.

.

What assistance did you get from Forum Nokia in helping overcome those hurdles?

Forum Nokia is the greatest source of developer information when it comes to software development in the mobile device space. And that, combined with the fact that the S60 platform is the best one in opening its features to third party developers, make the S60 platform the reference platform of choice for developing and maturing new technologies in the mobile imaging field. 

In your view - what are the main development opportunities offered by your technology?

Integrating the Motionized® technology as a core feature of the platform enables motion-based, innovative user interaction in many applications such as web browsers, picture, map and document viewers, games, etc.
What were the main difficulties you faced in getting your concept to market?

The challenge of introducing innovation and facing skepticism that is not uncommon when a whole paradigm is changed. However, motion navigation is getting more and more momentum today and our technology has been recognized through industry awards.

What lessons have been learned from recent projects?

We confirmed our view that the handset user interface is going through a major revolution in 2007 and 2008.

What do you see as being important for your company in 2008?

Our goal is to establish motion detection as a key solution for browsing large content on handset and to make camera-based motion detection the technology driving this change across the range of handsets, alongside hardware motion detection that will be available on selected high end devices.

The MobileMonday Expert Jury Award we received last year for Motionized® confirmed the tremendous potential of the technology.