I’m a fan of services neatly packaged as products that answer a very well defined need. A good example is psd2html: you know what you put in the box, and you know what you’re going to get at the other end. Great way to complement your own resources. We’re coming to a point where our quite informal internal QA is showing its limits, and we need to reinforce our development methodology to include proper regression tests and some level of automation. What should we use, JMeter? Zend? Empirix?(Yeah I know those are apples and oranges to some degree.) Application health monitoring (especially for transactions) is also something we need to investigate as low level monitors such as CPU or RAM consumption don’t give the whole picture. Many of the tools in this space are too expensive, and more importantly, too cumbersome for our organization size and project scope. I need to research leads such as this QaTraq lite approach. Byrne Reese‘s Test Run seems targeted for people like us.
Now, tools is only part of the answer. Even though we’re growing our in-house development resources, we’re still a publisher at heart and at our scale, I can’t justify a full-time QA person. What I’d like to find is an external resource we can use when we need it, with a very predictable output and cost. Poking around the web, there’s uTest, but they’re working off their own environment. I need someone to work within our Trac system so that we don’t multiply systems and tools with overlapping purposes.
Please email any advice or suggestion as my comments are still broken (sigh). I really need to overhaul my whole blog.
Update: there doesn’t seem to be much that integrates with Trac (Test Run and QaTraq don’t), except for a beta plugin that’s not very user friendly.
02/03/08 update: TestLink 1.7.3 was released last month with Trac integration. Now I need to look how well this integration is done. I love how Trac tickets and Subversion changesets can very easily be linked together (you’ll have to scroll down to Vidar Hokstad’s entry since LinkedIn hasn’t implemented comment permalinks yet — sheesh why does every site that rolls out community features have to make the same design mistakes again and again?) thanks to the common wiki syntax.
03/28/08 update: Oracle buying Web app test tools from Empirix (another successful software company coming from outside the silicon valley).