Second Functional Test Workshop Results

Posted by Mark Levison on Aug 05, 2008 10:20 PM

Community

Agile

Topics

Agile Techniques ,

Software Testing ,

Research

Tags

agile2008 ,

Acceptance Testing

The second Agile Alliance Functional Test Workshop was held as a pre-conference session before Agile 2008. It was run as a series of open space sessions facilitated by Jeff Paton. The primary purpose of this workshop was to discuss cutting-edge advancements in and envision possibilities for the future of automated functional testing tools.

In response to the purpose the group created a diverse list of open space sessions:

• Narrative Testing – what is and how can it help.
• Why hasn’t Acceptance first development taken off.
• Tests vs. Specifications – what is we’re writing?
• Tool Clearing house? Why so many tools?
• User Understandable Tests vs. the Power of Programming
• HTML: ideal testing DSL or an abomination?

After a lunch the group conducted a “Futurespective” – a retrospective looking a year into the future. The goal was to identify things that we wanted to happen in the next year. The group discovered several big needs, among them articles that explain current best practices in functional testing and the distinction between test frameworks (tools that run and report tests) and drivers (tools responsible for translating the tests into the language of the system under test).

On the subject “Why hasn’t Acceptance Test Driven Development Taken Off”:

Programmers

• miss the fun / aha! moments / benefits that they get
  from UTDD (Unit Test Driven Development).
• there is a difference in scope and cadence of tests.
  ("Cadence" became a key word people kept coming back to.)
• laborious writing of fixture, which doesn't feel as valuable as "real
  programming".

Business People

• don't see the value (or ROI) from ATDD (Acceptance Test Driven Development)
• they are not used to working at that level of precision
• no time
• they prefer rules to examples

The session on Tools resulted in agreement to build a clearing house for Agile Functional Test Tools. Among many attributes the group decided to characterize tools by certain attributes:

• Test Input Format (XML, HTML, Java, Groovy, Vendor Script)
• Support for Writing a Domain Specific Language
• Test Target: Windows Application, Browser based, WinForms, Eclipse Rich Client
• …

Work is just beginning on classifying the tools.

Additional session notes: Tests vs. Specifications/Requirements, Tests vs. Examples,  Narrative Testing and pictures are available. Finally Mike Debbo has published his as AA-FTT 2008 workshop redux, part 1 and part 2.

Previous InfoQ articles: Workshop Announcement and Last years workshop: “Next-Generation Functional Testing”

1. Back to top

   Level of Precision

   Aug 6, 2008 12:54 PM by Curtis Olson

   <p>I had something of a Eureka moment when I read that. I've noticed that everyone but programmers and testers get annoyed when asked to focus on low-level details (including not just customers, but managers and even requirements analysts). I think this "impedence mismatch" of expectations between business and technical people is the primary cause of "scope-creep" - and we all know what that leads to. Managing expectations is difficult at best, and I have yet to see a business person get excited about co-authoring acceptance tests. Typically, management will attempt to separate techies and business folks when they notice any annoyance on the customer's part, so co-authoring falls to the wayside.</p>
   
   <p>I'm interested to hear more about this, particulary discussions regarding business people deriving more benefit from acceptance testing.</p>
   
   Curtis Olson

   Reply

2. Back to top

   Re: Level of Precision

   Aug 7, 2008 12:50 PM by Mark Levison

   I'm glad the discussion helped. I had a whole bunch of ahha moments on Monday. Along with one like you had, I also understood that some tools are drivers and others are frameworks.
   
   I hope you're able to join us on aa-ftt mailing list (yahoo groups).
   
   Cheers
   Mark Levison

   Reply

• Older >