Join Now

Random thoughts about mobile (enterprise) application development.

Prompt user at uninstall, ok or not?

widianuser | 14 February, 2007 09:04

We have successfully got an application Symbian Signed and we had to to make a minor change to the application (add one setting field, to be exact). OK, no problem. We change the application, prepare it for Symbian Signing, get the quotations and select the test house. Because application was already signed by another test house, I really wasn't expecting any problems. But...


Application was rejected because of a fatal problem, said the test report, application cannot be uninstalled. This was because our application has a couple of background servers running all the time. When such an application is to be uninstalled, terminal application manager will prompt user for closing down the background servers. User selects yes, servers are closed and all files are removed. However, the test house was feeling that this is not according to the test specification and they decided to reject the application. No matter how hard I try to read the test case PKG-04 I can't see where it is said that users must not be prompted during uninstall. A quick search to Forum Nokia discussion board shows that we are not alone with this problem, please see this thread. Yes, surely it would be possible to add "close down all the background servers" feature to our application but what's the point? The organizations that decide to use our solution want applications to be running all the time at the background and test criteria doesn't force us to add the unnecessary feature either.


I feel the test plan must be clarified regarding uninstallation. Can users be prompted during uninstall or not? Now that is not explicitly stated and test houses interpret the test plans differently.


So, how did the story end? I submitted the unchanged application to the test house we used for first signing, got package signed and customer installations have begun.

Enterprise, Symbian C++, Testing | Next | Previous | Comments (8) | Trackbacks (0)

Comments

Re: Prompt user at uninstall, ok or not?

Darlehen | 01/06/2008, 02:46

Well interesting discussing. The waiver I agree ond and the other improvements also good :) not that dramatic and very useful.

Re: Prompt user at uninstall, ok or not?

widianuser | 20/03/2007, 10:58

Update to this issue: I complained to Symbian Signed team about this uninstallation "problem" and I recently got a response from them: test houses have received new guidelines that applications prompting user during uninstallation, like described above, will be passed.

Re: Prompt user at uninstall, ok or not?

ptrmn | 15/02/2007, 22:25

I've had similar experiences. I've had 20-30 SIS files go through Symbian Signed testing, and it's quite frustrating how loose the criteria are. It's not just what's considered a problem, but the severity of it that's open to interpretation. Sometimes an issue will be considered critical, at other times the same issue will be considered a minor problem.

I'm not sure how detailed the test criteria could be, though. It's not possible to foresee every possible situation, and specify the severity of every potential issue. However, it'd be good if these common problems, like prompting on uninstall, could be considered by some sort of "supreme court", as suggested by Harri.

There are also cases that don't seem to have been taken into consideration at all. The criteria are very focused on standalone applications. At one point I had to consider the problem of getting a virtual machine to pass the test process. Fortunately we didn't have to go through with it, but I did inquire about it, and there didn't seem to be any established process to deal with interpreters/VM:s. It's not a common case anyway, but the Symbian Signed process has to be flexible enough to allow different kinds of content to pass.

Re: Prompt user at uninstall, ok or not?

rihoe | 14/02/2007, 13:13

Our application acts exatly the same way. Luckily (knock-knock) it hasn't been an issue yet. But the annoying thing is that each time when we submit the app for signing (after some minor changes) they find different problems which were not raised last time. It looks like they will keep some problems is reserve in order to get more testing rounds = more money.

Re: Prompt user at uninstall, ok or not?

widianuser | 15/02/2007, 10:21

That's scary. I wish there were some kind of a "supreme court for Symbian Signed Process" that would comment on problems and instruct test houses and ISV's how to handle situations that are not explicitly written to test plan.

Re: Prompt user at uninstall, ok or not?

mgroeber9110 | 16/02/2007, 11:07

We are currently experiencing exactly the same thing, even including the "Close down applications" warning as one of the newly raised issues, as well as issues raised for a 3rd party component that has passed Symbian Signed in other applications already.

On paper the concept you describe already exists, through the Waiver process: Symbian Signed (or the manufacturer, in case of having their own testing with additional criteria, like Symbian Signed for Nokia) is the ultimate "Supreme Court". and no matter what issues a specfic test house raises, the approval for Waivers should be granted on equal terms by Symbian or the vendor.

From my experience, there are two issues with this at the moment:

- Feedback from Waivers into new test rounds is fairly slow - issues may need to be waived again for each application (officially, this is even part of the specs: Waivers are meant as one-shot exceptions), and updates to the Symbian Signed specs to include "Frequently Waived Issues" seem to be pretty rare. The reason may be that no central list of granted waivers is maintained...

- The process of getting a Waiver is in itself a bit rough sometimes, because it involves three parties, exchange of word documents and PDFs, and often happens in the critical phase just before a release, so people sometimes don't quite know who has to wait for whom.

My suggestion would be making Waivers part of the official workflow on Symbian Signed (did I mention today that I think the site is badly in need for an overhaul? ;-)), so that they can essentially be handled like bug reports in, say, bugzilla, with a clear concpt of "ownership" (test house, developer, Symbian/Vendor), and the ability to re-open previous ones, mark duplicates, treat as general issues etc.

Re: Prompt user at uninstall, ok or not?

widianuser | 16/02/2007, 16:11

I agree with you about the waiver process and suggested improvements. Now I feel a little bit silly about raising a waiver when application "fails" a test case that doesn't exist in a test plan.

Re: Prompt user at uninstall, ok or not?

njzk2 | 14/02/2007, 12:44

what you say raise a _very_ big issue : it suggests that the validation depends strongly on the subjective interpretation by the test house of the test specification, and that this interpretation may lead to huge differences between applications validated by different test houses !
It first means that the symbian signed test specs are not specific enought, and can lead to interpretation.
Therefore, it means that the risk exists, in a close future to see that, not only people will have a natural tendancy to search for the easiest test houses - which will be known at some point - , at the detriment of the quality and relyability of the symbian signed process, but also that this subjectivity may conduct to different levels of symbian signed : " - hey, i got my application tested by ****** ! - That's nothing ! Mine is ******** tested ! - Well done, i failed with this one, so i only got **** to validate it "
I there at Symbian Signed any entity in charge of deciding for this kind of case ? in order to establish a future policy and add to the process the specifications for this particular case ?
You must login to post comments. Login
 
 
Powered by LifeType
RDF Facets: qfnZtopicQUqfnTopicZcppQ qfnZtopicQUqfnTopicZenterpriseQ qfnZtopicQUqfnTopicZtestingQ qfnZtypeQUqfnTypeZBlogContentQ qfnZtypeQUqfnTypeZBlogE45ntryQ qfnZtypeQUqfnTypeZCommunityContentQ qfnZtypeQUqfnTypeZWebpageQ qmarsZlanguageQUxhttpE3aE2fE2fswE2enokiaE2ecomE2flanguageE2d1E2fenX