Gabor Torok's Forum Nokia Blog

Mobile software development - Functional testing

Thu, 08 May 2008 01:36:47 +0200

Simon Judge's blog made me think, again. He wrote about Mob4Hire, a company offering people for mobile application testing. Testers get paid via PayPal after bidding on projects (i.e. mobile applications/solutions) and developers get testers at a (hopefully) reasonable price. Finding testers might be especially useful if your geographic area is not the one you'd like your software to be tested in.

You know, lots of developers (dare to say: most?) do not recognize the importance of testing. This is the least pleasant part of development, I must admit, yet one of the most important. There are various kinds of testing including (but not limited to) unit-, regression, load, "smoke", etc. testing. The one, Mob4Hire provides solution to is called functional testing, where one can test the whole solution end-to-end. You know, mobile handsets are very-very fragmented in terms of platforms: applications can be developed in many programming languages like Java, Symbian C++, Windows Mobile Win32/C#, iPhone Obj-C, Linux C/C++, etc. And even when sticking to the same platform and programming environment, JME for example, the supported features vary very much from device to device. This, along with the complexity of what operators allow 3rd-party programs to do, makes it very difficult for a new application to be thoroughly tested.

Nokia already provides a free service for developers wishing to test their mobile applications written for Nokia S60 platform: it's called Nokia RDA (short for Remote Device Access). It is an Internet-based solution, where you can remote control a real mobile phone. You can request, for example, that SIM- and/or memory card be inserted in the test device as well as more than one phone be reserved for your test session to test peer-to-peer communications.

DeviceAnywhere provides a similar solution to Nokia RDA, however, it's not limited to a particular platform, nor to only 1-2 network operators. According to their web site, their service is "a revolutionary online service that provides access to hundreds of real handsets, on live worldwide networks, remotely over the Internet". Unfortunately, it is not free of charge.

Note that you cannot test everything with these solutions. For example, applications that use camera, GPS, accelerometer are basically out of question as well as ones using external accessories.

Another option for functional testing is making use of the services of Test Houses. Professional testers verify the quality of your software (compared to Mob4Hire, for example, see Simon's opinion) so that you can be sure you get the most what you paid for. Sometimes it's even required for your application to pass certain tests in order to get certified by some authorities. However, you may need to pay a lot for this service, see the list & pricing of Test Houses that Symbian Signed accepts, for example.

Finally, let's talk about community-driven testing. Once your application is in such a shape that it is ready for external people to play with it, you can ask them to go and use it extensively. You can offer free copies of the software to them, for example, or they may do it just for their own gratification - it's the same. This way of testing works extremely well in solutions based on client-server architecture with a mobile front-end and a server back-end. It's quite common in these scenarios that the mobile-end is just a light-weight client software that can be freely distributed, thus it doesn't cause any inconvenience if software distrubition is not strictly controlled. The point is that you may get lots of people playing with your software, because it's their passion. And passion drives people to do their job well, simply because they enjoy it, they love your program and they'd like it to be even better. I'm really a great supporter of this kind of testing. :)

Can you recommend any other way for performing functional mobile software testing? Please let me (us) know!

Tote

True emulation for Symbian development - when?

Fri, 11 Apr 2008 16:24:17 +0200

Though I've already heard that Windows Mobile SDK offers true phone emulation, however, only now have I got to the point that I ask for your opinion: why cannot we do the same during Symbian software development?

Note that you might have also heard that iPhone developers can also rely on this useful service. Nevertheless, don't forget to bear in mind that they must be working on the same platform, i.e. MacIntosh OSX. That is, there's not too much to emulate there.

But Windows Mobile is different: you develop on Windows presumably on an x86 architecture and produce binary code for another processor architecture (ARM) that you can even debug on. How? Why on Earth cannot we do the same?

Tote

Symbian Signed is not an anti-virus software

Wed, 23 May 2007 16:35:10 +0200

The Register reported today that a new spyware for mobile phones had appeared on the horizon. It's harmful for S60 phones, too, 3^rd Edition devices included. And what causes the stir in the water is that it's a Symbian Signed application.

There's a general misconception here, I'm afraid. I think the biggest problem most people don't understand that signing has not much to do with protection against malicious programs. These people don't understand that the process is about signing (surprisingly), i.e. certifying that the application comes from a well-known source. Additionally, in order for an application to be Symbian Signed it must undergo thorough testing being done by independent test houses. Since this application is Symbian Signed, it must have passed those tests.

The problem is that it's impossible to test everything an application can do. It's even possible to acquire for a capability (and get it!) just by saying that the application needs it for a different purpose. As this example shows: I can ask for e.g. NetworkServices capability and say that I need it for remote backup. And then make no mention on the fact that I will use it for other reasons, too. You know, it can be done since no-one checks the source code, it's not part of the approval process for Symbian Signed certification. And it will never be, I suppose, as no-one will ever share their best kept secret (i.e. the source code) with outsiders.

What Symbian (Signed) could do better, though, is that they shouldn't advertise these signed applications as "trusted". Because they aren't. What you can trust, though, is that the author of a Symbian Signed application is accountable. If he/she/they produce a software that proves to contain some malicious code, then they can be "caught" and counter-measures can be taken. What counter-measures? For example, the author's certificate can be revoked and added to a list, called Certificate Revocation List or CRL for short. This list can be always checked upon on-line. For example, when a user is just about to install a 3^rd party software whose author is not known (or at least not trusted), the Application Installer can do this cross-verification as part of the installation process. Pretty useful info, isn't it? Worth noting that most users are not aware of this and they have this feature disabled on their phones. Including me, but that's on purpose. :-

Original from mobile-thoughts.blogspot.com.

Just my two cents,

Tote