I'm all for using static analysis tools, even if it can be a bit annoying to use tools like leavescan, because they're quite stupid. And lint is a real pain to run, as it gives you so many warnings. However, these tools will find stuff that you'd miss otherwise, so it's worth running them once in a while.

The Symbian Signed criteria is a good collection of things that should be tested, not just if you're trying to get your app Symbian Signed. Things like checking so that your code behaves well on incoming calls or messages are very important, but might be ignored because you can't test it on the emulator, you need to have SIM cards in your test devices etc.

When I designed a Symbian specific test suite for our gaming middleware at my last job, I got a lot of inspiration from the Symbian Signed criteria. However, as many have pointed out before, they're not good as actual tests, as they're not nearly as well-specified as they ought to be.

BTW, what do you think of the S60 QA book? I felt that it could have been good, if it had been finished before being printed.

Well, you're right - I've got the same feeling. I suppose book has been written mainly before S60 3rd edition and was quickly adopted to it (1 - author mentions about MMC certificates needed for a phone program (page 51), but no word about mini/microSD, wifi which are almost a standard in 3rd edition; 2 - author writes (page 3) that S60 devices have 12 keys; not true - there were available devices with QWERTY support like E61, E70 before the book was finished, e.g. check appendinx A, where N80, N73, N93 are mentioned - these devices were developed after E61, E70). I also haven't found an answer for a simple question: how long does it take from a product idea to the day it is available for the end user? (I can only guess) As for a book about QA, publisher of the book should improve its own QA process ;-) (a lot of small spelling errors like: majr -> major, test -> text, ebing -> being, has -> his, te -> the, fix at -> fix it, to have be -> to be, fom -> from to mention only a few) - I wrote an erratum to the book, but I haven't found (so far) a way to deliver it to the publisher/author (maybe they will use it in next reedition). I'm not complaining about the book - I understand that's not easy to write a book by one person having many other things to do in life.

Generally, it's a good book - I recommend it for people who would like to understand a process of making S60. An additional advantage IMHO is a very good organisation of paragraphs (there're short and it's very comfortable to read/pause/continue).

Yes, the QA on the book is really bad. There are complete misspellings of UIKON as UiCon (where did that case come from), and use of the term DFRD which is a long defunct Symbian term.
In the most part I found that the book was a copy-and-paste job which assembles a pretty good range of areas which need to be addressed on a S60 based phone programme. So I welcome the book, but think there could be much more depth/walkthoroughs (especially outside of the testing sections which are clearly in the author's comfort zone).

I think it's also worth mentioning Continuous Integration (CI), too. With such a tool capable of integrating code continuously we're able to detect build errors as soon as new code has been checked in to our version control system. And not only can we test the build process, but we can run automated tests upon each check-in and generate statistics from the result. Not to mention SymScan, code coverage, etc. integration. This is a very agile way of testing and integrating.

Tote