Jump to content

Agile software development

From Wikipedia, the free encyclopedia

This is an old revision of this page, as edited by 12.10.219.37 (talk) at 20:29, 9 December 2008 (Criticism). The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.

Agile software development refers to a group of software development methodologies that are based on similar principles. Agile methodologies generally promote: A project management process that encourages frequent inspection and adaptation; a leadership philosophy that encourages team work, self-organization and accountability; a set of engineering best practices that allow for rapid delivery of high-quality software; and a business approach that aligns development with customer needs and company goals.

Introduction

There are many specific agile development methods. Most promote development iterations, teamwork, collaboration, and process adaptability throughout the life-cycle of the project.

Agile chooses to do things in small increments with minimal planning, rather than long-term planning. Iterations are short time frames (known as 'timeboxes') which typically last from one to four weeks. Each iteration is worked on by a team through a full software development cycle, including planning, requirements analysis, design, coding, unit testing, and acceptance testing when a working product is demonstrated to stakeholders. This helps to minimize the overall risk, and allows the project to adapt to changes more quickly. Documentation is produced as required by stakeholders. An iteration may not add enough functionality to warrant releasing the product to market, but the goal is to have an available release (with minimal bugs) at the end of each iteration. Multiple iterations may be required to release a product or new features.

Team composition in an agile project is usually cross-functional and self-organizing without consideration for any existing corporate hierarchy or the corporate roles of team members. Team members normally take responsibility for tasks that deliver the functionality of an iteration. They decide for themselves how they will execute during an iteration.

Agile methods emphasize face-to-face communication over written documents. Most agile teams are located in a single open office to facilitate such communication. Team size is typically small (5-9 people) to help make team communication and team collaboration easier. Larger development efforts may be delivered by multiple teams working toward a common goal or different parts of an effort. This may also require a coordination of priorities across teams.

No matter what development disciplines are required, each agile team will contain a customer representative. This person is appointed by stakeholders to act on their behalf and makes a personal commitment to being available for developers to answer mid-iteration problem-domain questions. At the end of each iteration, stakeholders and the customer representative review progress and re-evaluate priorities with a view to optimizing the return on investment and ensuring alignment with customer needs and company goals.

Most agile methodologies include a routine and formal daily face-to-face communication among team members. This specifically includes the customer representative and any interested stakeholders as observers. In a brief session, team members report to each other what they did yesterday, what they intend to do today, and what their roadblocks are. This standing face-to-face communication prevents problems being hidden.

Agile methods emphasize working software as the primary measure of progress. Combined with the preference for face-to-face communication, agile methods usually produce less written documentation than other methods. In an agile project, documentation and other project artifacts all rank equally with working product. Stakeholders are encouraged to prioritize them with other iteration outcomes based exclusively on business value perceived at the beginning of the iteration.

Specific tools and techniques such as continuous integration, automated or xUnit test, pair programming, test driven development, design patterns, domain-driven design, code refactoring and other techniques are often used to improve quality and enhance project agility.

History

The modern definition of agile software development evolved in the mid-1990s as part of a reaction against "heavyweight" methods, perceived to be typified by a heavily regulated, regimented, micro-managed use of the waterfall model of development. The processes originating from this use of the waterfall model were seen as bureaucratic, slow, demeaning, and inconsistent with the ways that software developers actually perform effective work. A case can be made that agile and iterative development methods are a return to development practice seen early in the history of software development.[1] Initially, agile methods were called "lightweight methods." In 2001, prominent members of the community met at Snowbird, Utah, and adopted the name "agile methods." Later, some of these people formed The Agile Alliance[2], a non-profit organization that promotes agile development.

An adaptive software development process was introduced in a paper by Edmonds (1974)[3]. Notable early Agile methods include Scrum (1986), Crystal Clear, Extreme Programming (1996), Adaptive Software Development, Feature Driven Development, and Dynamic Systems Development Method (DSDM) (1995). These are typically referred to as Agile Methodologies since the Agile Manifesto was published in 2001.

Principles behind agile methods

Agile methods are a family of development processes, not a single approach to software development. In 2001, 17 prominent figures[4] in the field of agile development (then called "light-weight methods") came together at the Snowbird ski resort in Utah to discuss ways of creating software in a lighter, faster, more people-centric way. They created the Agile Manifesto, widely regarded as the canonical definition of agile development and accompanying agile principles. The Agile Manifesto states:

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

  • Individuals and interactions over processes and tools
  • Working software over comprehensive documentation
  • Customer collaboration over contract negotiation
  • Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

Some of the principles behind the Agile Manifesto[5] are:

  • Customer satisfaction by rapid, continuous delivery of useful software
  • Working software is delivered frequently (weeks rather than months)
  • Working software is the principal measure of progress
  • Even late changes in requirements are welcomed
  • Close, daily cooperation between business people and developers
  • Face-to-face conversation is the best form of communication (Co-location)
  • Projects are built around motivated individuals, who should be trusted
  • Continuous attention to technical excellence and good design
  • Simplicity
  • Self-organizing teams
  • Regular adaptation to changing circumstances

The manifesto spawned a movement in the software industry known as agile software development.

In 2005, Alistair Cockburn and Jim Highsmith gathered another group of people — management experts, this time — and wrote an addendum, known as the PM Declaration of Interdependence.

The functioning principles of Agile can be found in lean manufacturing and six sigma. These concepts include error proofing, eliminating waste, creating flow, adding customer value, and empowering workers. The concepts were first formally espoused in the 14 principles of the Toyota Way, the two pillars of the Toyota Production System (Just-in-time and smart automation), the 5S methodology, and Deming’s 14 points. These have been summarized in the 7 points of lean software development.

Comparison with other methods

Agile methods are sometimes characterized as being at the opposite end of the spectrum from "plan-driven" or "disciplined" methods. This distinction is misleading, as it implies that agile methods are "unplanned" or "undisciplined". A more accurate distinction is that methods exist on a continuum from "adaptive" to "predictive".[6] Agile methods lie on the "adaptive" side of this continuum.

Adaptive methods focus on adapting quickly to changing realities. When the needs of a project change, an adaptive team changes as well. An adaptive team will have difficulty describing exactly what will happen in the future. The further away a date is, the more vague an adaptive method will be about what will happen on that date. An adaptive team can report exactly what tasks are being done next week, but only which features are planned for next month. When asked about a release six months from now, an adaptive team may only be able to report the mission statement for the release, or a statement of expected value vs. cost.

Predictive methods, in contrast, focus on planning the future in detail. A predictive team can report exactly what features and tasks are planned for the entire length of the development process. Predictive teams have difficulty changing direction. The plan is typically optimized for the original destination and changing direction can cause completed work to be thrown away and done over differently. Predictive teams will often institute a change control board to ensure that only the most valuable changes are considered.

Agile methods have much in common with the "Rapid Application Development" techniques from the 1980/90s as espoused by James Martin and others.

Contrasted with other iterative development methods

Most agile methods share other iterative and incremental development methods' emphasis on building releasable software in short time periods. Agile development differs from other development models: in this model time periods are measured in weeks rather than months and work is performed in a highly collaborative manner. Most agile methods also differ by treating their time period as a strict timebox.

Contrasted with the waterfall model

Agile development has little in common with the waterfall model. As of 2008, the waterfall model is still in common use.[7] The waterfall model is the most structured of the methods, stepping through requirements-capture, analysis, design, coding, and testing in a strict, pre-planned sequence. Progress is generally measured in terms of deliverable artifacts: requirement specifications, design documents, test plans, code reviews and the like.

The main problem with the waterfall model is the inflexible division of a project into separate stages, so that commitments are made early on, and it is difficult to react to changes in requirements. Iterations are expensive. This means that the waterfall model is likely to be unsuitable if requirements are not well understood or are likely to change in the course of the project.[8]

Agile methods, in contrast, produce completely developed and tested features (but a very small subset of the whole) every few weeks. The emphasis is on obtaining the smallest workable piece of functionality to deliver business value early, and continually improving it/adding further functionality throughout the life of the project.

In this respect, agile critics may assert that these features are not placed in context of the overall project, concluding that, if the sponsors of the project are concerned about completing certain goals with a defined timeline or budget, agile may not be appropriate. Proponents of Agile development counter that adaptations of Scrum[9] show how agile methods are augmented to produce and continuously improve a strategic plan.

Some agile teams use the waterfall model on a small scale, repeating the entire waterfall cycle in every iteration.[10] Other teams, most notably Extreme Programming teams, work on activities simultaneously.

Contrasted with "cowboy coding"

Cowboy coding is the absence of a defined method: team members do whatever they feel is right. Agile development's frequent re-evaluation of plans, emphasis on face-to-face communication, and relatively sparse use of documents sometimes causes people to confuse it with cowboy coding. Agile teams, however, do follow defined (and often very disciplined and rigorous) processes.

As with all development methods, the skill and experience of the users determine the degree of success and/or abuse of such activity. The more rigid controls systematically embedded within a process offer stronger levels of accountability of the users. The degradation of well-intended procedures can lead to activities often categorized as cowboy coding.

Suitability of agile methods

There is little if any consensus on what types of software projects are best suited for agile methodologies. Many large organizations have difficulty bridging the gap between a more traditional waterfall method and an agile one.

Large scale agile software development remains an active research area.[11][12]

Agile development has been widely documented (see Experience Reports, below, as well as Beck[13] pg. 157, and Boehm and Turner[14] pg. 55-57) as working well for small (<10 developers) co-located teams.

Some things that can negatively impact the success of an agile project are:

  • Large scale development efforts (>20 developers), though scaling strategies[15] and evidence to the contrary[16] have been described.
  • Distributed development efforts (non-co-located teams). Strategies have been described in Bridging the Distance[17]and Using an Agile Software Process with Offshore Development[18]
  • Command-and-control company cultures
  • Forcing an agile process on a development team

Several successful large scale agile projects have been documented. BT has had several hundred developers situated in the UK, Ireland and India working collaboratively on projects and using Agile methods. While questions undoubtedly still arise about the suitability of some Agile methods to certain project types, it would appear that scale or geography, by themselves, are not necessarily barriers to success.

Barry Boehm and Richard Turner suggest that risk analysis be used to choose between adaptive ("agile") and predictive ("plan-driven") methods.[14] The authors suggest that each side of the continuum has its own home ground as follows:

Agile home ground:

  • Low criticality
  • Senior developers
  • Requirements change very often
  • Small number of developers
  • Culture that thrives on chaos

Plan-driven home ground:

  • High criticality
  • Junior developers
  • Requirements don't change too often
  • Large number of developers
  • Culture that demands order

Agile methods and method tailoring

In the literature, different terms refer to the notion of method adaptation, including ‘method tailoring’, ‘method fragment adaptation’ and ‘situational method engineering’. Method tailoring is defined as:

A process or capability in which human agents through responsive changes in, and dynamic interplays between contexts, intentions, and method fragments determine a system development approach for a specific project situation.[19]

Potentially, almost all agile methods are suitable for method tailoring. Even the DSDM method is being used for this purpose and has been successfully tailored in a CMM context.[20] Situation-appropriateness can be considered as a distinguishing characteristic between agile methods and traditional software development methods, with the latter being relatively much more rigid and prescriptive. The practical implication is that agile methods allow project teams to adapt working practices according to the needs of individual projects. Practices are concrete activities and products that are part of a method framework. At a more extreme level, the philosophy behind the method, consisting of a number of principles, could be adapted (Aydin, 2004).[19]

Extreme Programming (XP) makes the need for method adaptation explicit. One of the fundamental ideas of XP is that not one process fits every project, but rather that practices should be tailored to the needs of individual projects. Although partial adoption of XP practices, as suggested by Beck, has been reported on several occasions.[21] A tailoring practice is proposed by Mehdi Mirakhorli which provides sufficient roadmap and guideline for adapting all the practices. RDP Practice is designed for customizing XP. This practice first time proposed as a long research paper in APSO workshop at ICSE 2008 conference and yet it is the only proposed and applicable method for customizing XP. Although it is specifically a solution for XP, this practice has the capability of extending to other methodologies. At first glance, this practice seems to be in the category of static method adaptation but experiences with RDP Practice says that it can be treated like dynamic method adaptation. The distinction between static method adaptation and dynamic method adaptation is subtle.[22] The key assumption behind static method adaptation is that the project context is given at the start of a project and remains fixed during project execution. The result is a static definition of the project context. Given such a definition, route maps can be used in order to determine which structured method fragments should be used for that particular project, based on predefined sets of criteria. Dynamic method adaptation, in contrast, assumes that projects are situated in an emergent context. An emergent context implies that a project has to deal with emergent factors that affect relevant conditions but are not predictable. This also means that a project context is not fixed, but changing during project execution. In such a case prescriptive route maps are not appropriate. The practical implication of dynamic method adaptation is that project managers often have to modify structured fragments or even innovate new fragments, during the execution of a project (Aydin et al, 2005).[22]

Agile methods and project management

Agile methods differ to a large degree in the way they cover project management. Some methods are supplemented with guidelines on project management, but there is generally no comprehensive support.[20]

PRINCE2 has been suggested as a suitable, complementary project management system.[23]

Project management tools for agile development teams

A number of project management tools are specifically aimed at agile development. They are designed to help plan, track, analyse and integrate work. These tools play an important role in agile development, as a means of Knowledge Management.

Common features include: Version control integration, progress tracking, easy work allocation, integrated release and iteration planning, discussion forums, and reporting and tracking of software defects.

Earned Value Management is a PMI (Project Management Institute) approved project management technique to objectively measure the success of a project.

Agile methods

Some of the well-known agile software development methods:

Agile Practices

Note: Although these are often considered methodologies in and of themselves, they are simply practices used in different methodologies.

Agile beyond software development

Agile software development depends on some special characteristics possessed only by software, such as object technologies and the ability to automate testing. However, related techniques have been created for developing non-software products, such as semiconductors, motor vehicles, or chemicals. For more on them, see Flexible product development.

Measuring agility

While many see agility as a means to an end, a number of approaches have been proposed to quantify agility. Agility Index Measurements (AIM)[1] score projects against a number of agility factors to achieve a total. The similarly-named Agility Measurement Index [2], scores developments against five dimensions of a software project (duration, risk, novelty, effort, and interaction). Other techniques are based on measurable goals [3]. Another study using fuzzy mathematics[24] has suggested that project velocity can be used as a metric of agility.

While such approaches have been proposed to measure agility, the practical application of such metrics has yet to be seen.

Criticism

Extreme Programming's initial buzz and controversial tenets, such as pair programming and continuous design, have attracted particular criticism, such as McBreen[25] and Boehm and Turner.[14] Many of the criticisms, however, are believed by Agile practitioners to be misunderstandings of agile development.[26]

In particular, Extreme Programming is reviewed and critiqued by Matt Stephens's and Doug Rosenberg's Extreme Programming Refactored.[27]

Criticisms include:

  • Lack of structure and necessary documentation
  • Only works with senior-level developers
  • Incorporates insufficient software design
  • Requires too much cultural change to adopt
  • Can lead to more difficult contractual negotiations
  • Can be very inefficient — if the requirements for one area of code change through various iterations, the same programming may need to be done several times over. Whereas if a plan were there to be followed, a single area of code is expected to be written once.
  • Impossible to develop realistic estimates of work effort needed to provide a quote, because at the beginning of the project no one knows the entire scope/requirements
  • Can increase the risk of scope creep due to the lack of detailed requirements documentation
  • Agile is feature driven, non-functional quality attributes are hard to be placed as user stories

The criticisms regarding insufficient software design and lack of documentation are addressed by the Agile Modeling method, which can easily be tailored into agile processes.

Agile software development has been criticized because it may not bring about all of the claimed benefits when programmers of average ability use this method.[28] However, when paired programming is practiced within the Agile methodology, average programmers find themselves much closer to being senior.

Post-Agilism

In software engineering, post-Agilism is an informal movement of practitioners who prefer to avoid being constrained by what they consider to be "Agile Dogma" (or "Agile with a capital 'A'")

Some have argued that the meaning of Agile is ambiguous and has been applied inappropriately to a very wide range of approaches like Six Sigma and CMMi (though proponents of some agile methods, such as Scrum, show that the methods can exist comfortably within a CMMi environment.)[29]Some have argued also that "Agile", "evolutionary", and "lean" (as in Lean software development) do not mean the same thing in practice, even though they are all lumped under the banner of "Agile" and are often used interchangeably by practitioners.

Proponents also argue that process-oriented methods, especially methods that rely on repeatable results and that incrementally reduce waste and process variation like Six Sigma, have a tendency to limit an organisation's adaptive capacity (their "slack"), making them less able to respond to discontinuous change - i.e., less agile. It is proposed that "agile", "lean" and "evolutionary" are strategies that need to be properly understood and appropriately applied to any specific context. That is, there is a time to be "agile", a time to be "lean" and a time to be "evolutionary." Some agilists agree with this position, promoting the concept of agile methods as one set of tools that should be available to managers for use in appropriate situations, not as one-size-fits-all methods that should be forced onto all organizations.[30]

Much of post-Agile thinking centers around Nonlinear Management, a superset of management techniques that include many Agile practices.

Some commentators propose a model of post-Agilism that is effectively constructive anarchy, in that teams should be self-organising to the point where even the core values of the Agile movement are considered too prescriptive, and that teams should simply "do whatever works for them." This position is controversial in the agile community and breaks from the trend of later agile methods being more structured (fitting better within CMMi environments) than earlier agile methods.

Experience reports

Agile development has been the subject of several conferences. Some of these conferences have had academic backing and included peer-reviewed papers, including a peer-reviewed experience report track. The experience reports share industry experiences with agile software development.

As of 2006, experience reports have been or will be presented at the following conferences:

  • XP (2000[31], 2001, 2002, 2003, 2004, 2005, 2006[32])
  • XP Universe (2001[33])
  • XP/Agile Universe (2002,[34]2003,[35] 2004[36])
  • Agile Development Conference (2003,[37] 2004,[38]) (peer-reviewed; proceedings published by IEEE?)
  • Agile (2005,[39] 2006) (peer-reviewed; proceedings published by IEEE)

See also

References

  1. ^ Gerald M. Weinberg: We were doing incremental development as early as 1957, in Los Angeles, under the direction of Bernie Dimsdale [at IBM’s ServiceBureau Corporation]. He was a colleague of John von Neumann, so perhaps he learned it there, or assumed it as totally natural. I do remember Herb Jacobs (primarily, though we all participated) developing a large simulation for Motorola, where the technique used was, as far as I can tell, indistinguishable from XP. [. . .] All of us, as far as I can remember, thought waterfalling of a huge project was rather stupid, or at least ignorant of the realities. I think what the waterfall description did for us was make us realize that we were doing something else, something unnamed except for “software development. quoted in Larman, Craig (2003). "Iterative and Incremental Development: A Brief History" (pdf). Computer. 36 (No. 6): pp 47–56. doi:10.1109/MC.2003.1204375. Retrieved 2007-02-22. {{cite journal}}: |issue= has extra text (help); |pages= has extra text (help); Unknown parameter |coauthors= ignored (|author= suggested) (help); Unknown parameter |month= ignored (help) (Permission note)
  2. ^ Agile Alliance
  3. ^ Edmonds, E. A. (1974), "A process for the development of software for non-technical users as an adaptive system", General Systems, XIX: 215–218
  4. ^ Kent Beck, Mike Beedle, Arie van Bennekum, Alistair Cockburn, Ward Cunningham, Martin Fowler, James Grenning, Jim Highsmith, Andrew Hunt, Ron Jeffries, Jon Kern, Brian Marick, Robert C. Martin, Steve Mellor, Ken Schwaber, Jeff Sutherland and Dave Thomas
  5. ^ Agile Manifesto principles
  6. ^ Boehm, B. (2004). Balancing Agility and Discipline: A Guide for the Perplexed. Boston, MA: Addison-Wesley. ISBN 0-321-18612-5. {{cite book}}: Unknown parameter |coauthors= ignored (|author= suggested) (help) Appendix A, pages 165-194
  7. ^ Laplante, P.A. (2004). ""The Demise of the Waterfall Model Is Imminent" and Other Urban Myths". ACM Queue. 1 (10): 10. doi:10.1145/971564.971573. Retrieved 2006-05-13. {{cite journal}}: Unknown parameter |coauthors= ignored (|author= suggested) (help); Unknown parameter |month= ignored (help)
  8. ^ Sommerville, Ian (2007) [1982]. "4.1.1. The waterfall model". Software engineering (8th edition ed.). Harlow: Addison Wesley. pp. pp 66f. ISBN 0-321-31379-8. {{cite book}}: |edition= has extra text (help); |pages= has extra text (help)
  9. ^ Ambler, S. (2008). ""Scaling Scrum - Meeting Real World Development Needs". Dr. Dobbs. Retrieved 2008-04-23. {{cite journal}}: Unknown parameter |month= ignored (help)
  10. ^ As reported by HeavyLogic
  11. ^ Agile Processes Workshop II Managing Multiple Concurrent Agile Projects. Washington: OOPSLA 2002
  12. ^ "Supersize Me" in Dr. Dobb's Journal, February 15, 2006.
  13. ^ Beck, K. (1999). Extreme Programming Explained: Embrace Change. Boston, MA: Addison-Wesley. ISBN 0-321-27865-8.
  14. ^ a b c Boehm, B. (2004). Balancing Agility and Discipline: A Guide for the Perplexed. Boston, MA: Addison-Wesley. ISBN 0-321-18612-5. {{cite book}}: Unknown parameter |coauthors= ignored (|author= suggested) (help)
  15. ^ Supersize Me
  16. ^ Schaaf, R.J. (2007). "Agility XL", Systems and Software Technology Conference 2007, Tampa, FL
  17. ^ Bridging the Distance
  18. ^ Using an Agile Software Process with Offshore Development
  19. ^ a b Aydin, M.N., Harmsen, F., Slooten, K. v., & Stagwee, R. A. (2004). An Agile Information Systems Development Method in use. Turk J Elec Engin, 12(2), 127-138
  20. ^ a b Abrahamsson, P., Warsta, J., Siponen, M.T., & Ronkainen, J. (2003). New Directions on Agile Methods: A Comparative Analysis. Proceedings of ICSE'03, 244-254
  21. ^ Abrahamsson, P., Salo, O., Ronkainen, J., & Warsta, J. (2002). Agile Software Development Methods: Review and Analysis. VTT Publications 478
  22. ^ a b Aydin, M.N., Harmsen, F., Slooten van K., & Stegwee, R.A. (2005). On the Adaptation of An Agile Information Systems Development Method. Journal of Database Management Special issue on Agile Analysis, Design, and Implementation, 16(4), 20-24
  23. ^ Agile Alliance at http://agilealliance.org/system/article/file/904/file.pdf :
    PRINCE2 (Projects in Controlled Environments) . . . is a project management method that was specifically designed to be generic and independent of any particular project type or development method. As with DSDM, its use is dramatically on the increase in both the public and private sectors. As a development method and a project management method, the two should be complementary. Some have perceived the dynamic emphasis of DSDM and the control emphasis of PRINCE2 to be in conflict. However, this is not the case. When DSDM was being developed, those involved had PRINCE firmly in mind. This is reflected in a number of the DSDM principles and techniques – for example, product-based planning, the involved partnership of users and developers, and the strong emphasis on the underlying business case.
  24. ^ Kurian, Tisni (2006). "Agility Metrics: A Quantitative Fuzzy Based Approach for Measuring Agility of a Software Process" ISAM-Proceedings of International Conference on Agile Manufacturing'06(ICAM-2006), Norfolk, U.S.
  25. ^ McBreen, P. (2003). Questioning Extreme Programming. Boston, MA: Addison-Wesley. ISBN 0-201-84457-5.
  26. ^ sdmagazine
  27. ^ Extreme Programming Refactored
  28. ^ The Great Pyramid of Agile
  29. ^ Sutherland, Jeff. "Scrum and CMMI Level 5: The Magic Potion for Code Warriors." Proc. European Systems and Software Engineering Process Group Conference (EUROPEAN SEPG). June 2007.
  30. ^ See "Managing Agile Projects." Ed. Kevin Aguanno. Oshawa, Ontario: Multi-Media Publications Inc., 2005. ISBN-10: 1895186110.
  31. ^ 2000
  32. ^ 2006
  33. ^ 2001
  34. ^ 2002
  35. ^ 2003
  36. ^ 2004
  37. ^ 2003
  38. ^ 2004
  39. ^ 2005

Further reading