We have all seen it and most likely all inadvertently done it: introduced a software regression in our product or project. It happens to the best of us. A software regression is never good and I’d like to think that most try to fix regressions as quickly as possible, especially if they are very noticeable. I’ve seen a few recently with Android that have me disappointed and concerned, and it has me thinking about how development teams can avoid or minimize our chances of creating a software regression.
Before I start railing on the good folks at Google and AOSP, let me say this: the Android platform has advanced in terms of features, robustness and development tools/docs since it was first made available to the public. I get it. Sometimes things go sideways when you are running at full throttle on a rocket ship.
Now, back to the software regression examples. The first one is with the Android emulator. When Android 1.0 then 1.5 (anyone else remember Cupcake?) hit the scene it had an excellent feature set, was stable and workable. This new kid on the block came with good documentation and integration with Eclipse, a popular IDE in the Linux and open source world. It also included a device emulator for ARM which was surprisingly snappy. Having worked on WindowsCE, Windows Mobile and Symbian, I had seen other device emulators and the Android emulator blew them out of the water. It rocked. It was responsive, available on Windows, Linux and Mac, and supported emulated hardware keys, sound, etc. Fast forward to today. Frankly, the Android emulator for 4.2 is abysmal. The default arm-eabi-v7a (that’s version 7 of the ARM architecture) is slooooowwwww. Painfully slow, even on a quad core host machine running Linux or Windows. I find it next to useless. I was giving a demo at AnDevCon back in May which required an emulator and it was difficult at best. This type of software regression is very noticeable to those of us who have been working with Android for years but it also has an impact on newer Android developers who get a bad taste in their mouth for the tools. The other complaint I have is the emulator images are somewhat crippled. For example: Android 4.2 introduced multi-user support (see my blog here for speculations on the future of it), but the emulator images ship with this completely disabled. Admittedly, this is probably more of an omission than a software regression, but it is still strange to disable a key bit of new functionality.
The other software regression with Android which has surfaced recently is with the app development toolchain. Near the end of May a new SDK and SDK tools package was published. If you are just getting started and pulled down the ADT bundle, no problem (other than quirky Eclipse behavior). But, if you are working with the SDK and an existing install of Eclipse, it fails. Big time. The SDK manager will happily update the components and report all is well. Eclipse’s Android support on the other hand goes completely off the rails. Some of your existing projects will not build, especially if they use RenderScript. Creating a new project can crash Eclipse completely and leave you with a partially created project tree. How in the world did this make it out the door? After Googling around a bit, you have to go through this dance of updating the manager, restarting it and installing a new “SDK build tools” package. Even after that it fails. Oddly enough, IntelliJ IDEA was able to work with the updated SDK (after the dance) without issue. Perhaps this is why Google is creating the new Android Studio based on IntelliJ rather than Eclipse. This concerns me because there is such a large developer base using Eclipse (Android Studio is in pre-release right now) and this made it into the wild.
Now that I have drug Android through the mud, back to general discussion of software regression. For development teams, what are some best practices to prevent a software regression, catch it early as well as deal with one effectively post-release? In short: good development process and testing. Here’s a quick punch list:
- Create and maintain test plans. Unit tests. Functional tests. Black box tests.
- Developers actually run through the test plans before QA
- QA independently tests the product and follows the test plans
- QA and development communicate. If you are an Agile shop, this should be happening in stand-ups. If you are using waterfall, you should have frequent bug scrubs.
- Peer code reviews before changes are allowed into the release line
- Use a clear-cut process to get changes from development into release
- Ensure there is a feedback loop from end users back to QA and development
It is sometimes hard to ensure you do all of these, but the potential fallout of introducing a regression is not worth short-circuiting a good process to get things out the door.
