Daniel Rocha's Forum Nokia Blog

Commercial S60 applications in Python

2008-11-28T18:10:33-01:00

Why aren't there (m)any commercial apps written in Python for S60? This is a question I have been asking myself since I started working with this technology back in late 2006.

Mikko Ohtamaa from Red Innovation wrote a very interesting piece on the topic back in February, and from his post plus practical experience plus I have put together a list of the main issues (not in any particular order):

Symbian Signed compatibility: Python apps do not have sophisticated memory control, therefore they may fail some SS tests related to performing well under scarce memory conditions. The new Symbian Signed "Express Signed" feature partially solves this issue (?) since it does not involve sending an app for testing by a test house - so the standard test criteria is not used.
Packaging and distribution: Packaging Python applications in .sis files is hard. If a developer does it without the Python runtime (distributed as a separate file), users have to be instructed to download the runtime from another place, which is less than ideal since we want our applications to be installed in a single shot. Also if two different applications are packaged with the different versions of the runtime, problems will arise as Symbian prevents installing conflicting binaries with the same UIDs. Python for S60 Community Edition presents a valid solution for this problem, since it allows to package a Python app, customized runtime, C++ extensions and everything else into a single .sis file that can be installed as if it were a native application.
API set: I would say that Python API set is situated between C++ and Java sets. It has more features than Java (in general) but less than C++. This can generate high expectations on developers, who think of Python as an easier way of writing native apps. But when they realize the API set is greatly reduced from what's available in C++, they may get frustrated on having to write their own extensions. There's a healthy amount of activity happening around creating C++ extensions for PyS60, so that helps a bit as well.
Culture: Python developers are hackers, used to a lot of freedom in other platforms. They may get quickly annoyed by the constraints of mobile development, specially PlatSec and Symbian Signed, and give up trying to get their apps to commercial quality. I see no solution for this problem as of now.

That being said, I wish to say I am a supporter of the idea that Python for S60 can be used for commercial apps, not only for rapid prototyping, and that's exactly why I am puzzled by the lack of commercial apps in this technology.

What do you think is the reason for that?

[]s
Daniel

Mobile Drummer with N95

2008-11-28T17:23:10-01:00

Fun moment of the Friday. FN Champion Jackson Feijo and his mobile drum kit, composed of three Nokia N95s plus some Symbian C++ code. I would call it a "poor man's drum kit" but as real drums cost less than three N95s I will just call it "mobile man's drum kit".

Enjoy:

Mobile Drummer with N95

Shake your music with Python

2008-06-11T17:52:55+00:00

I was preparing demos for the next Forum Nokia Tech Days, to be held in São Paulo, Brazil next week, and it came to mind I should do something with the sensors using Python.

Here is the result: Sensor-powered music player for Nokia N95. Nothing too sophisticated, but it goes to show how you can showcase your ideas pretty quickly using PyS60: it took me about 2 hours to develop.

Which technology should I use? Round 3: Python

2008-06-03T18:49:17+00:00

The comic above pretty much sums it up: Python is a refreshing look at the world of programming. Being so easy to learn and use, it has gained lots of attention from the developer community and it's used for several different purposes such as: web development, desktop apps, utilities, scientific computing, scripting language for games and special effects software, flying and of course mobile development. In fact, I am very surprised that it took so long before any mobile platform offered Python as a viable software creation vehicle.

In our platforms, Python is offered as a runtime option for S60 devices, ranging from 2nd. to 3rd Edition and all their feature packs. That makes it pretty wide reaching platform, since we have some 130 million+ devices in the market for which you can write Python applications.

The core technologies for mobile development have been analyzed here before: C++ is the raw power of Symbian OS available to developers, with its immense API and functionality sets but steep learning curve; Java is easy to learn and use and it's present in more than a billion devices. Its functionality is good but restricted by the Java Community Process since all APIs have to be agreeded upon by participants of each JSR spec.

Python comes in the middle, and that's in my view its main advantage: it is as easy as Java but (almost) as powerful as C++, and not limited by any industry standard processes: you want a Flying API which is not available? Just create an extension and you're good to go. Our good friend and Forum Nokia Champion Pankaj Nathani (a.k.a. croozeus) has in his website a great list of resources for Python development, including how-to's on writing extensions and more: http://croozeus.googlepages.com/py60extensions. Also, if you are the kind of developer who likes to hack around, Python for S60 runtime is open source, so you can spend your days and nights hunting bugs, adding new features and creating your own version of it!

First part of this article: Which technology should I use for development? Round 1: Java
Second part of this article: Which technology should I use for development? Round 2: C++

Technology: Python for S60

Good for:

Internet mashups - You can easily mix the Location API with urlib and Google Maps and create a powerful mobile/web hybrid application which can leverage both platforms strengths. For example, take a look at MobyMaps, a geocoding application in just 104 lines of code, or Fupper, pyAmazon, Mobile Google Maps and many others.
Productivity applications - Access to the filesystem, contacts database, local connectivity, camera, gallery with incredible simple APIs lets you quickly write applications through which you can interact with your device and other people. MobileGTD, PyS60RemoteControl, PySpy are some examples.
Rapid prototyping - This is probably one of the most popular uses of Python for S60, and you should consider it very seriously. It seems that RP is still not very popular, as people usually choose a technology and develop an app with it; if something goes wrong or doesn't work, they will only find it out at the end of the project, when it's too late. I have seen countless cases like this, specially with Java development: "What, I cannot intercept a phone call with Java? But the product is almost ready!". Python helps you tackle this issue in a very elegant and effective way: you have an idea, want to check whether it works or not, just write a PyS60 script to test it. If you're satisfied, just tweak it a bit more and make it your final product, otherwise implement the final version in C++ or Java if you need more power or portability. I cannot stress enough the importance of showing your customer something working from Day One; this is much more effective than showing slides or specification documents.
Utilities - If you have a special event or need and you need to cook up a quick solution for a simple problem, use PyS60. To give you an example, I needed some sample apps for a talk I gave back in March. I was able to develop 10 utilities (photo takers, Bluetooth FTP client, video recorder, etc.) in less than a week.
Games - Given Python's powerful API set that can be used in games, such as: Sensor API, Open GL ES access, graphics and image manipulation, it is a good tool for churning out casual games meant to have a short lifecycle. It can compete well with Flash Lite for this purpose; even though its graphics facilities are not as nice, the powerful APIs will help you create the "wow" factor in your mini-games.

Why?

API/functionality availability - The S60-only list of APIs is reasonably extensive, and you can always add new ones by programming extensions using Python/C API. Many times modules can be added from other platforms without modification (for example, Python debugger - pdb).
Ease of use - It's by far one of the easiest languages to learn and work with. In a matter of weeks you can be highly productive with it.
Community support - There's a lot of Python code written by the open source community out there that can be used in your application.

Bad for:

High performance apps - Python's performance is reasonable, but it is slower than C++, Java and Flash. So if you need to develop a scientific application which requires a lot of number crunching, or a video streaming application, this is probably not the best solution.
Commercial apps, destinated to operator channels - Please take this one with a grain of salt. Applications for sale at operator channels usually are required to go through Symbian Signed process, and Python apps don't play very well with it.
Portability - Although the core Python language is the same, there are several APIs which are S60-exclusive so they won't work in any other platforms without several modifications.

Why?

Symbian Signed - In theory Python is compatible with the program, but in reality Symbian Signed was designed for C++ applications, so in some situations PyS60 apps may fail the tests so you have either to get a waiver for the failure or use the Express Signed option. For lack of a better option, I put Python on the "bad for" side of this article, but in the real world you may have different options that make it suitable for operator channels.
Platform security - Although it's very noble on its purpose (securing users' devices against malware), platform security can be a pain to live with during the development cycle. The whole Symbian Signed environment and platform security API restrictions slow down development time and can increase development costs, because of test fees, certificate purchasing, re-test, etc. This also affects Python for S60 so I left it here from the previous article.
Performance - Don't forget Python runs on an interpreter, so it has some performance disavantages when compared to native applications.
Tools - As it's a very new platform on S60, its tools are not as mature as other technolgies'. You can use the C++ SDK with the Python plug-in for testing your scripts, Python Shell for running scripts on your device, and py2sis or Ensymble to package them in an installable .sis file. There's still no integration with Carbide, and the emulator runs only on Windows. This is a pretty basic selection when compared to NetBeans or EclipseME for Java, Adobe CS3 for Flash Lite or Carbide.c++ for, hm, C++.

Python brings back the fun of hacking your device the way you want, with just a few lines of code. It's also a powerful tool for fast development of applications and utilities, besides being ideal for rapid prototyping of more complex software. The learning curve is smooth, the API set is big, and the support of open source community gives it a large set of libraries you can use in your application. As it's still in an incipient phase in the mobile software development, you can use it and contribute for its evolution in the next releases, making it a sweet platform for innovation on S60.

If you are interested in learning more, there's a ton of resources on PyS60 in Forum Nokia web site and also in its wiki page. For impatient people who can't wait to try it, I have posted a few months back a complete presentation on how to get started with Python, along with a collection of sample source code; both can be found here.

Happy hacking!

<a href="http://technorati.com/claim/ymg32hejh6" rel="me">Technorati Profile</a>

Which technology should I use for development? Round 2: C++

2008-05-30T15:39:41+00:00

The Holy Grail, according to Monty Python

So let us talk now about the Holy Grail of mobile development in Nokia platforms: Symbian/S60 C++. It is the language in which the operating system is developed (about 95%), it is the language that allows developing all sorts of applications, from simple expense trackers to VoIP and geotagging and video-sharing applications. It is the language about which odes are written, flowers strewn at is feet, and virgins sacrificed to its altar.

It is also one of the hardest development environments there is to learn: it's not standard C++, you have to instrument a lot of the memory management yourself, the learning curve is very steep, and the amount of code it takes to write even a simple "Hello World" GUI application is larger than any other technology we'll talk about. However, if you have the skills to learn it, the patience to develop apps with it, and want to power your application with the most sophisticated features available, it can be a very rewarding experience using it after all.

Obs.: I know I promised Python this time but I felt like covering the two core technologies first.

First part of this article: Which technology should I use for development? Round 1: Java

Technology: Symbian/S60 C++

Good for:

Games - Full-fledged commercial complex games, such as N-Gage's, as the APIs for direct screen access, 3D graphics, audio and video and camera manipulations are very powerful and enrich and gaming experience.
Multimedia applications - You can write very complex applications which integrate a handful of different powerful features such as: location, geo-tagging, video and audio low-level manipulation, VoIP, camera access and so on. As examples, check Shozu, Fring, Windows Live, Google Search, and many others.
Mapping and location apps - These leverage the powerful Location APIs along with the OS-provided Assisted GPS and the Map and Navigation framework (3.2 only) to give you a very fine-grained control over obtaining the location of your device and integrating it with other features or APIs. For example, Shozu integrates GPS location with EXIF JPEG data manipulation to upload pictures which are geotagged to known services such as Flickr, where they are placed automatically on a map. Very creative. Check also WayFinder, Google Maps, Nokia Maps, Route 66.
VoIP Applications - There are simply too many to list, just check for yourself.
E-mail applications - Your e-mail apps can talk to Microsoft Exchange using ActiveSync protocol, and also be integrated into the standard Messaging application so users can access it without having to go to the menu.
Many others - Enablers, virtual machines, services, device drivers, file managers, are all examples of sophisticated applications which can be written in C++.

Why?

API/functionality availability - Programming with Symbian C++ is almost like programming desktop applications: you will get a stagerring number of APIs, powerful features, sophisticated integration between applications and services; those allow you to do more with your device than any other technology.
Performance - Native C++ applications have the best performance of all analyzed technologies (unless you're a really really bad programmer).
Porting from other platforms - This used to be a nightmare, but with the introduction of Open C and Open C++ porting layers, it became much easier to port from other POSIX platforms.
Tools - Although the building tools themselves are not standard, Carbide.c++ does a fine job of hiding the complexity from you the programmer, allowing for faster application development. Its debugging tools are specially worth mentioning.

Bad for:

Simple applications - If you need to develop an expense or sports tracker, a blogging tool, casual game or such, using C++ is probably an overkill. You can achieve the same in much less time using Java, Flash Lite or Python.
Rapid prototyping - Symbian C++'s complexity will add much more time to your development projects, so it's not well suited for protopying. Do it in Python or Java and if needed, write the final version in C++.
Casual games - That would be another overkill. You will end up writing 7000 lines of code or so per game, and your productivity when compared to your competitors using lighter technologies will suffer. A lot.
Web applications - At risk of souding repetitive, another case where you're better off with technologies whose focus is Web services. I'm talking Widgets, Widsets and Python here.

Why?

Harder to learn than any other technology - Even an experienced programmer, coming from other C/C++ languages, will have a hard time learning Symbian C++, using the non-standard tools and proprietary features, such as resource and registration files. It can take a long time to get a decent working knowledge of how everything works and even then you will find yourself.
Slower development time - Due to the steep learning curver and overall complexity, native development in S60 is generally though to take 3 to 10 times more time than Java or Flash Lite projects.
Portability - First off, there's a binary compatibility break between S60 3rd. Edition and its previous editions. Secondly, even within the same versions there some API incompatibilities/deprecations you'll only find out about when you're deep into development. In this case you'll have to produce a workaround or ask technical support from Forum Nokia to fix you an update, workaround or private API that does the job.
Platform security - Although it's very noble on its purpose (securing users' devices against malware), platform security can be a pain to live with during the development cycle. The whole Symbian Signed environment and platform security API restrictions slow down development time and can increase development costs, because of test fees, certificate purchasing, re-test, etc.

"With great power, comes great responsibility", once said Spider-Man's uncle. In this case, however, with great power you'll get great complexity. If after careful evaluation of requirements you decide you should go with C++, commit yourself and your development team to learn the plaform right, and well. Otherwise it will cost you time and money. We know that time = money, so it will end up costing you money².

Symbian/S60 C++ offers you the opportunity of leveraging as much functionality as the device can provide. This can be a real differentiation factor for your applications and can result in more users, thus more money. Just make sure it's the right technology for you before you start up your project.

OK, now I'm not lying: next round will be about Python.

See you!

Which technology should I use for development? Round 1: Java

2008-05-27T18:29:05+00:00

If I had just made a dollar each time I heard this question... *sigh*.

With the availability of so many different development technologies in Nokia's platforms today, sometimes is hard to decide which of them we should use to build our application, isn't it? There's Java, Symbian C++, Open C, Flash Lite, WidSets, WRT Widgets, Python, and the list is still growing.

This is good news for developers, since not everybody has the same technical background nor is developing the same kinds of applications; this means at least one of the technologies will fit our experience and purpose like a glove, and we are going to use it. On the other side, this may confuse newcomers to the wonderful but fragmented mobile software world. Usually it goes like this:

Question: "I want to write an application that opens a file and sends it out to a web server. What should I use to create it?"
Answer 1: "Java ME, FileConnection API"
Answer 2: "Don't bother, use C++ or you will get a lot of security prompts."
Answer 3: "Why use C++ when you can do the same with Python? C++ is way too hard!"
Answer 4: "Python does not run on S40, use Java."
Answer 5: "Do you really need all that jazz? Why don't you go with Flash Lite and use server-based persistence? Your application will look great!"
Answer 6: "Flash Lite is too fragmented. Use Java, but beware of the MIDlet signing issues or your app's user experience will be crap!"

Go check the Discussion Boards. It is crowded with examples of conversations like the one above. Having answered this question so many times before, I decided to summarize my answers in this blog, in the hopes of helping developers decide what use from the growing technology options pool offered in mobile devices these days. Please keep in mind that the following are my opinions on the subject; yours may differ and even prove mine wrong, but that's the beauty of interacting with people within a community. As the number of technologies it too big for a single post, I will continue my analysis in the upcoming posts. I will start off with the most popular mobile developmentp platform, Java ME.

Technology: Java

Good for:

Productivity applications - Examples: sports trackers, expense control apps, Personal Information Management (calendar, phonebook, etc.).
Games - Both casual and complex.
Enterprise applications - These don't require fancy user interfaces. Using LCDUI and the standard packages can go a long way pretty fast.
Rapid prototyping and proof-of-concept applications.
Network applications - RSS readers, weather services, blogging applications, mobile versions of Web applications.
Creating truly portable apps

Why?

Java and its mobile version are very simple to use and learn - An experienced programmer, even coming from other C/C++-based languages, can get a pretty good working knowledge of how everything works in a matter of weeks and start producing applications quickly. This leads to faster time-to-market for Java ME apps.
Applications are generally portable between S60 and S40 platforms - Although there are some differences in implementation, supported packages, audio/video formats and so on, those are few in number and thus easier to work around. You don't need to entirely rewrite your application so it will run on N95 instead of 6131: a few runtime checks and versioning tools will be enough.
Game API combined with optional packages makes it a killer for game development
Straightforward network API makes it easy to communicate with web services - The GCF (Generic Connection Framework) is a simple and unified way of using the network on Java ME applications. Once you learn how to do one kind of connection, you will have learned them all.

Bad for:

Accessing device resources - Examples: SMS Inbox, file system, gallery, hardware.
Multimedia applications - Examples: Video streaming players, audio and video recorders, camera applications.
Performance-intensive applications - Scientific computing, image processors, video decoders, sound processors.

Why?

Java's functionalities depend on Java Community Process process: if there's no official, industry-standard API for what you want to do, you are out of luck. Example: accessing SMS inbox.
Mobile Media API implementation is (too) highly modular, a lot of runtime checks are needed. Also, most functionality is implemented on the native side and then made available to Java's virtual machine, such as the camera access. This may have a serious impact on performance.
Fine-grained control over video, audio and camera is generally not available. Although JSR 234 (Advanced Multimedia Supplements) tackles some of these issues, it still can't be compared to what's available on other technologies, and its device availability is still low.
Access to the file system is provided by FileConnection API, but it's somewhat restricted and requires the midlet to be signed, otherwise multiple security prompts will be displayed, taking your user to tears (believe me). The signing process itself costs money and has some bureaucracy, so be prepared.

As I said before, these are my opinions on the technologies. It does not mean that X is better than Z, and Z should be thrown into the dirt and kicked in the head, like XML in the funny The Parable of the Languages. If you get your requirements right, it's very likely that your question of "what to use" will answer itself. Need to access the Inbox? Down with Java. Need portability? Down with Python and C++. Fancy UI? Flash. Requirements management is a science, not an art, so when you get it right from the start, the development technology to use will pop up naturally.

Next time I will cover the darling of all Internet programmers, now getting much of the attention on S60 platform: Python.

See you!

Want a sexy UI in your Java MIDlet? It's now possible with LWTUI! (with video)

2008-05-14T22:23:39+00:00

Sexy UIs are all the rage today, right? FlashLite apps, WRT Widgets, WidSets, all of them have distinctive user experiences which differ greatly from the way we interact with native S40 and S60 applications. More colors, more animations, better usability, all making up a great way for developers to differentiate their applications and make them more attractive to the end user.

In Java MIDlets however, you are stuck with a very poor UI toolkit, the LCDUI, which has very few components and few options on how to alter their appearance; most of the time you're giving hints to the underlying implementation regarding the way you want the components to be displayed. Even if you do so, however, there's still a big chance your hints will be ignored and the UI will be rendered differently on each device. This can be a pain if you are developing for many different phones, with different screen sizes, input methods and form factors. Actually, to be fair, this can be a pain even if your target is a single device.

To circumvent these challenges, developers had either to developer their own UI toolkits, based on raw Canvasses, or use a 3rd. party toolkit such as J2ME Polish, which is great but requires a paid license to develop commercial applications, and this can be an unbearable cost to a small developer.

Searching the web and some mailing lists, I found this very interesting project, called LTWUI (Lightweight UI Toolkit), which is a cross-device UI toolkit created to provide developers with the possibility of developing compelling user interfaces without writing painful low-level Canvas code. It's based on Swing programming paradigm and it's key functionalities are:

LWTUI demo midlet running on E61i

Swing Like MVC
Layouts
PLAF & Themes
Fonts
Touch Screen
Animations & Transitions
Rich widgets
3D Integration
Painters
Modal Dialogs
External Tools
I18N/L10N

It's still in Early Access release but you can go to the project page and download it so you can play around with the toolkit and run its demos on emulators and real devices. The great thing is that the library is released under the Sun Licensing Agreement (SLA) which makes it usable by commercial applications without having to pay any fee.

I ran the LWTUIDemo midlet on my Nokia E61i and N95 and it worked beautifully. It's a bit slow though, but nothing that compromises the user experience. I recorded a video of LWTUI running on my N95, so you can see what's up for use in your Java apps; I guarantee you will enjoy it.

Last but not least, the project is open source, so you can contribute to it, just go to the project page for more details on how to participate.

Here's the video, enjoy it:

[]s
Daniel

Follow-up on the Symbian 9.2 Platform Security hack

2008-04-02T14:18:23+00:00

I have tested it as promised in my last post. After some initial trouble, I managed to get the Symbian 9.2 Platform Security hack working on my N95 with 20.0.015 firmware. It does work, as long as you don’t reboot the phone, as you can see in the screenshots I have posted on my personal blog, rawsocket dot org, where you can also find more details on this issue.

The trick is still a bit cumbersome to perform, but it works. I'll keep my eye on the latest developments from both sides, hackers' and Nokia's, to see where this is going. Are we seeing the beginning of a catch-up game like the one Sony and Apple have been playing against hackers, with all the jailbreaks and cracked firmwares?

Symbian 9.2 Platform Security hacked (or jailbroken)

2008-03-27T16:49:37-01:00

At least that's what says this story run by Engadget Mobile:

App certificates have long been a bane to S60 users and developers alike, causing pain, frustration, and an almost obligatory cash outlay to get your hard work certified to run on the very platform Nokia is so quick to call "open." Finally, it truly is, thanks to the hard work of the Symbian hacking community that has developed an easy (or easy sounding, anyway) method of "jailbreaking" the Symbian 9.2 device in your life (S60 3rd Edition FP1 users, that's you). After that, installed apps won't need a certificate at all -- let alone an invalid one -- to do their dirty work. Open, indeed.

It points you to this site where instructios are given to unlock a single phone using AppTRK and a couple of Python scripts. The hack isn't permanent, which means after a reboot your device gets back to normal. They say they're working on a permanent hack, which would allow iPhone-style jailbroken firmwares running unsigned applications at full capabilities.

I haven't tested it myself, but it seems it works at least on Feature Pack 1 devices. I will test with my N95 with an application which requires all capabilities (including manufacturer-given) to see what happens.

Anyway, I would advise nobody to do it to their own devices, since this opens up the possibility of you installing apps whose authors you don't know and can do real damage to your phone if they are making use of sensitive capabilities whose use is protected by the whole signing scheme.

Make your widgets and Flash Lite applications fly!

2008-03-24T19:09:47-01:00

As I mentioned in my last post, I have been thinking a lot lately about how to extend applications that run on sandboxed environments, such as Java ME, Flash Lite and Widgets, using some powerful native technologies such as C++, Python and Mobile Web Server.

I did my research and turns out some people have already been thinking (and doing) this for a while now, so I wanted to share this with you, for a reason I will explain later:

- Our very own Sr. Technology Expert Petro Soininen has created a widget that's able to send text messages and read the SMS inbox of an S60 device, using a creative combination of Widgets, Python, PHP and the Mobile Web Server. Here's the link for the article with complete source code: http://wiki.forum.nokia.com/index.php/How_to_utilize_device_resources_from_a_WRT_widget

- Forum Nokia Champion Felipe Andrade and his company have been working for quite some time in the Flyer Framework, a server application written in Python for S60 whose goal is to provide enhanced native functions to Flash Lite applications. It looks pretty mature, and I wonder if it could be used with Widgets as well with some adjusts in connectivity (full HTTP vs XML sockets).

I mentioned these two ways of extending widgets and Flash Lite apps because of the Global Code Camp Contest. I figured that an application/widget with enhanced functionality has more room for being innovative and delight users in many different ways.

As nothing was mentioned in the contest rules, I asked the Forum Nokia team responsible for the competition whether applications produced with these extension methods would be valid or not. An the answer is that these turbo web widgets and Flash apps are allowed, so you have a lot more options to introduce cool features in your apps via sweet Python and PHP scripts.

So take a look at them both and make your widgets and Flash apps fly higher than ever!