Agile is a step forward for implementing software systems. But, I have seen teams struggle with several quality challenges when they move to Agile. While Agile provides more freedom in the implementation of stories, it can be more limiting in the amount of time spent during the specifying and test planning phase. The methodology rewards lean specification and favors implementation. Unfortunately, this can result in a few casualties in the quality war.
A casualty I have seen in Agile development is the non-functional requirements. Examples of this are response times, volume requirements, background performance, memory footprint, and database size, to name a few. Under waterfall, you might complete the specification of the entire system, then have a section of your design devoted to these requirements.
When you are doing progressive design and specifying in increments, the impact that an individual story or feature has on non-functional requirements may not be obvious. There are a few ways to combat this problem.
First, educate the team on the importance of these requirements. Since most analysts come from a business background, technical requirements may not be as apparent to them. They may also be assuming the developers are creating non-functional requirements as they implement.
Second, be sure that your specification and design artifacts, no matter how lean they are, contain a section for describing these non-functional requirements.
Finally, identify technical resources at the team and enterprise level that are responsible for ensuring that these requirements are understood, implemented, and tested. This is usually the role of an architect at the enterprise level, and an architect or lead developer at the team level. Any code reviews should also include these topics.
The irony in many organizations is that they put all of their energy into business requirements and then wonder why they have performance or reliability issues. Sometimes the response is “isn’t that the developers job to figure out”? So you give them a great deal of specification on functional requirements, absolutely nothing on non-functional, and expect them to figure it out. If that logic is correct, why wouldn’t you ask them to figure out business requirements also without upfront analysis?
Testing outside of story and epic acceptance tests
Test Driven Development, Test First Development, and Test Driven Design are all breakthroughs in improving the quality of an implementation. These are not so much breakthroughs in testing as they are breakthroughs in specification. My reason to advocate writing tests first is that the specification for the story is improved when you have to describe how it will be validated. The specification is stronger and it may lead to modifications to the implementation to assist validation.
One area where teams sometimes struggle is in identifying when they need to step out of the defined acceptance test paradigm in order to fully test a complex story or feature with unique testing requirements.
An example of this may be a mission-critical, high transaction rate data interface. For this type of feature, the ability to generate many transactions with a variety of good and bad data is necessary in order to move this to production quality code. Now you can argue that this would be covered in system testing, but that is too late in the development cycle and the system testers may not understand the inner workings of the code in order to properly test. Also, system testers may not have the tools necessary to create all of the volume and data conditions needed to properly test.
To address this testing gap, I have seen teams create backlog items for the work necessary to create these special test tools. The test tool, or “test harness” will go through its own specifying, development, and testing phases. The work to execute this tool with the code under test will be covered in the acceptance of the main feature.
Agile is better than waterfall in so many ways. If you keep an eye on your technical requirements and testing you can avoid any regression in these critical areas.