Agile software development
|Software development process|
|Paradigms and models|
|Methodologies and frameworks|
|Standards and BOKs|
Agile software development is a group of software development methods in which solutions evolve through collaboration between self-organizing, cross-functional teams. It promotes adaptive planning, evolutionary development, early delivery, continuous improvement, and encourages rapid and flexible response to change.
The Manifesto for Agile Software Development, also known as the Agile Manifesto, was first proclaimed in 2001, six years after "Agile Methodology" was originally introduced by the preeminent software engineers of the late 80's and early 90's and came out of the DSDM Consortium in 1994 although its roots go back to the mid 80's at DuPont and works by James Martin and James Kerr et al.
- 1 History
- 2 Overview
- 3 Philosophy
- 4 Agile methods
- 5 Experience and adoption
- 5.1 Common agile pitfalls
- 5.1.1 Lack of overall project design
- 5.1.2 Adding stories to a sprint in progress
- 5.1.3 Lack of sponsor support
- 5.1.4 Insufficient training
- 5.1.5 Product owner role is not properly filled
- 5.1.6 Teams are not focused
- 5.1.7 Excessive preparation/planning
- 5.1.8 Problem-solving in the daily standup
- 5.1.9 Assigning Tasks
- 5.1.10 Scrum master as a contributor
- 5.1.11 Lacking test automation
- 5.1.12 Allowing technical debt to build up
- 5.1.13 Attempting to take on too much in a sprint
- 5.1.14 Fixed time, resources, scope, and quality
- 5.2 Measuring agility
- 5.1 Common agile pitfalls
- 6 Criticism
- 7 Applications outside software development
- 8 References
- 9 Further reading
- 10 External links
Incremental software development methods trace back to 1957. In 1974, E. A. Edmonds wrote a paper that introduced an adaptive software development process. Concurrently and independently, the same methods were developed and deployed by the New York Telephone Company's Systems Development Center under the direction of Dan Gielan. In the early 1970s, Tom Gilb started publishing the concepts of evolutionary project management (EVO), which has evolved into competitive engineering. During the mid- to late 1970s, Gielan lectured extensively throughout the U.S. on this methodology, its practices, and its benefits.
A collection of lightweight software development methods evolved in the mid-1990s in reaction to the perceived heavyweight waterfall-oriented methods, which critics called heavily regulated, regimented, and micro-managed; although some proponents of these lightweight methods contended that they were simply returning to earlier software practices. These lightweight methods included: from 1994, unified process and dynamic systems development method (DSDM); from 1995, scrum; from 1996, crystal clear and extreme programming (aka "XP"); and from 1997, adaptive software development and feature-driven development. Although these originated before the publication of the Agile Manifesto in 2001, they are now collectively referred to as agile methods; and often abbreviated loosely as Agile, with a capital A, although this is progressively becoming deprecated.
The Agile Manifesto
We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
Individuals and interactions over Processes and tools
Working software over Comprehensive documentation
Customer collaboration over Contract negotiation
Responding to change over Following a planThat is, while there is value in the items on the right, we value the items on the left more.
Kent Beck James Grenning Robert C. Martin Mike Beedle Jim Highsmith Steve Mellor Arie van Bennekum Andrew Hunt Ken Schwaber Alistair Cockburn Ron Jeffries Jeff Sutherland Ward Cunningham Jon Kern Dave Thomas Martin Fowler Brian Marick
© 2001, the above authors. This declaration may be freely copied in any form, but only in its entirety through this notice.
The meanings of the manifesto items on the left are:
- Individuals and interactions: self-organization and motivation are important, as are interactions like co-location and pair programming.
- Working software: working software is more useful and welcome than just presenting documents to clients in meetings.
- Customer collaboration: requirements cannot be fully collected at the beginning of the software development cycle, therefore continuous customer or stakeholder involvement is very important.
- Responding to change: agile methods are focused on quick responses to change and continuous development.
Some of the authors formed the Agile Alliance, a non-profit organization that promotes software development according to the manifesto's values and principles. Introducing the manifesto on behalf of the Agile Alliance, Jim Highsmith said,
The Agile movement is not anti-methodology, in fact many of us want to restore credibility to the word methodology. We want to restore a balance. We embrace modeling, but not in order to file some diagram in a dusty corporate repository. We embrace documentation, but not hundreds of pages of never-maintained and rarely-used tomes. We plan, but recognize the limits of planning in a turbulent environment. Those who would brand proponents of XP or SCRUM or any of the other Agile Methodologies as "hackers" are ignorant of both the methodologies and the original definition of the term hacker.— Jim Highsmith, History: The Agile Manifesto
The Agile Manifesto is based on 12 principles:
- Customer satisfaction by early and continuous delivery of useful software
- Welcome changing requirements, even late in development
- Working software is delivered frequently (weeks rather than months)
- Close, daily cooperation between business people and developers
- Projects are built around motivated individuals, who should be trusted
- Face-to-face conversation is the best form of communication (co-location)
- Working software is the principal measure of progress
- Sustainable development, able to maintain a constant pace
- Continuous attention to technical excellence and good design
- Simplicity—the art of maximizing the amount of work not done—is essential
- Self-organizing teams
- Regular adaptation to changing circumstance
Later, Ken Schwaber with others founded the Scrum Alliance and created the Certified Scrum Master programs and its derivatives. Schwaber left the Scrum Alliance in the fall of 2009, and founded Scrum.org.
In 2005, a group headed by Alistair Cockburn and Jim Highsmith wrote an addendum of project management principles, the Declaration of Interdependence, to guide software project management according to agile software development methods.
In 2009, a movement spearheaded by Robert C Martin wrote an extension of software development principles, the Software Craftsmanship Manifesto, to guide agile software development according to professional conduct and mastery.
In 2011 the original Agile Alliance created the Guide to Agile Practices, an evolving open-source compendium of the working definitions of agile practices, terms, and elements, along with interpretations and experience guidelines from the world-wide community of agile practitioners.
There are many specific agile development methods. Most promote development, teamwork, collaboration, and process adaptability throughout the life-cycle of the project.
Iterative, incremental and evolutionary
Most agile development methods break the tasks into small increments with minimal planning and do not directly involve long-term planning. Iterations are short time frames (timeboxes) that typically last from one to four weeks. Each iteration involves a cross-functional team working in all functions: planning, requirements analysis, design, coding, unit testing, and acceptance testing. At the end of the iteration a working product is demonstrated to stakeholders. This minimizes overall risk and allows the project to adapt to changes quickly. An iteration might not add enough functionality to warrant a market release, but the goal is to have an available release (with minimal bugs) at the end of each iteration. Multiple iterations might be required to release a product or new features.
Working software is the primary measure of progress.
Efficient and face-to-face communication
No matter what development disciplines are required, each agile team should include a customer representative (product owner in scrum). This person is appointed by stakeholders to act on their behalf and makes a personal commitment to being available for developers to answer mid-iteration questions. At the end of each iteration, stakeholders and the customer representative review progress and re-evaluate priorities with a view to optimizing the return on investment (ROI) and ensuring alignment with customer needs and company goals.
In agile software development, an information radiator is a (normally large) physical display located prominently in an office, where passers-by can see it. It presents an up-to-date summary of the status of a software project or other product. The name was coined by Alistair Cockburn, and described in his 2002 book Agile Software Development. A build light indicator may be used to inform a team about the current status of their project.
Very short feedback loop and adaptation cycle
A common characteristic in agile is the daily "stand-up", also known as the daily scrum. In a brief session, team members report to each other what they did the previous day toward their team's sprint goal, what they intend to do today toward their team's sprint goal, and any roadblocks or impediments they can see to their team's sprint goal.
Specific tools and techniques, such as continuous integration, automated unit testing, pair programming, test-driven development, design patterns, domain-driven design, code refactoring and other techniques are often used to improve quality and enhance project agility.
Compared to traditional software engineering, agile software development mainly targets complex systems and projects with dynamic, non-deterministic and non-linear characteristics, where accurate estimates, stable plans, and predictions are often hard to get in early stages—and big up-front designs and arrangements would probably cause a lot of waste, i.e., are not economically sound. These basic arguments and previous industry experiences, learned from years of successes and failures, have helped shape agile development's favor of adaptive, iterative and evolutionary development.
Adaptive vs. predictive
Development methods exist on a continuum from adaptive to predictive. Agile methods lie on the adaptive side of this continuum. One key of adaptive development methods is a "Rolling Wave" approach to schedule planning, which identifies milestones but leaves flexibility in the path to reach them, and also allows for the milestones themselves to change. Adaptive methods focus on adapting quickly to changing realities. When the needs of a project change, an adaptive team changes as well. An adaptive team has difficulty describing exactly what will happen in the future. The further away a date is, the more vague an adaptive method is about what will happen on that date. An adaptive team cannot report exactly what tasks they will do next week, but only which features they plan for next month. When asked about a release six months from now, an adaptive team might be able to report only the mission statement for the release, or a statement of expected value vs. cost.
Predictive method, in contrast, focus on analysing and planning the future in detail and cater for known risks. In the extremes, a predictive team can report exactly what features and tasks are planned for the entire length of the development process. Predictive methods rely on effective early phase analysis and if this goes very wrong, the project may have difficulty changing direction. Predictive teams often institute a change control board to ensure they consider only the most valuable changes.
Risk analysis can be used to choose between adaptive (agile or value-driven) and predictive (plan-driven) methods. Barry Boehm and Richard Turner suggest that each side of the continuum has its own home ground, as follows:
|Agile methods||Plan-driven methods||Formal methods|
|Low criticality||High criticality||Extreme criticality|
|Senior developers||Junior developers(?)||Senior developers|
|Requirements change often||Requirements do not change often||Limited requirements, limited features see Wirth's law|
|Small number of developers||Large number of developers||Requirements that can be modeled|
|Culture that responds to change||Culture that demands order||Extreme quality|
Iterative vs. waterfall
One of the differences between agile and waterfall is the approach to quality and testing. In the waterfall model, there is always a separate testing phase after a build phase; however, in agile development testing is usually done concurrently with, or at least in the same iteration as, programming.
Because testing is done in every iteration—which develops a small piece of the software—users can frequently use those new pieces of software and validate the value.
After the users know the real value of the updated piece of software, they can make better decisions about the software's future. Having a value retrospective and software re-planning session in each iteration—scrum typically has iterations of just two weeks—helps the team continuously adapt its plans so as to maximize the value it delivers.
This iterative practice also introduces a product mindset rather than the waterfall model's project mindset. Software can be seen as a living organism, which actively changes due to environmental change. As long as the software is being used, especially when it has competition, iterations in agile software development drive the change.
Because of the short iteration style of agile software development, it also has strong connections with the lean startup concept.
Code vs. documentation
In a letter to IEEE Computer, Steven Rakitin expressed cynicism about agile development, calling it "yet another attempt to undermine the discipline of software engineering" and translating "Working software over comprehensive documentation" as "We want to spend all our time coding. Remember, real programmers don't write documentation."
This is disputed by proponents of agile software development, who state that developers should write documentation if that's the best way to achieve the relevant goals, but that there are often better ways to achieve those goals than writing static documentation. Scott Ambler states that documentation should be "Just Barely Good Enough" (JBGE), that too much or comprehensive documentation would usually cause waste, and developers rarely trust detailed documentation because it's usually out of sync with code, while too little documentation may also cause problems for maintenance, communication, learning and knowledge sharing. Alistair Cockburn wrote of the Crystal Clear method:
Crystal considers development a series of co-operative games, and intends that the documentation is enough to help the next win at the next game. The work products for Crystal include use cases, risk list, iteration plan, core domain models, and design notes to inform on choices...however there are no templates for these documents and descriptions are necessarily vague, but the objective is clear, just enough documentation for the next game. I always tend to characterize this to my team as: what would you want to know if you joined the team tomorrow.— Alistair Cockburn.
Popular agile software development methods and/or process frameworks include (but are not limited to):
- Adaptive software development (ASD)
- Agile modeling
- Agile Unified Process (AUP)
- Business analyst designer method (BADM)
- Crystal Clear Methods
- Disciplined agile delivery
- Dynamic systems development method (DSDM)
- Extreme programming (XP)
- Feature-driven development (FDD)
- Lean software development
- Kanban (development)
Agile methods are focused on different aspects of the software development life cycle. Some focus on the practices (e.g. XP, pragmatic programming, agile modeling), while others focus on managing the software projects (e.g. scrum). Yet, there are approaches providing full coverage over the development life cycle (e.g. DSDM, RUP), while most of them are suitable from the requirements specification phase on (FDD, for example). Thus, there is a clear difference between the various agile methods in this regard.
Agile development is supported by a bundle of concrete practices, covering areas like requirements, design, modelling, coding, testing, project management, process, quality, etc. Some notable agile practices include:
- Acceptance test-driven development (ATDD)
- Agile modeling
- Backlogs (Product and Sprint)
- Behavior-driven development (BDD)
- Cross-functional team
- Continuous integration (CI)
- Domain-driven design (DDD)
- Information radiators (scrum board, task board, visual management board, burndown chart)
- Iterative and incremental development (IID)
- Pair programming
- Planning poker
- Scrum events (sprint planning, daily scrum, sprint review and retrospective)
- Test-driven development (TDD)
- Agile testing
- Use case
- User story
- Story-driven modeling
- Velocity tracking
- User Story Mapping
The Agile Alliance has provided a comprehensive online collection with a map guide to the applying agile practices.
In the literature, different terms refer to the notion of method adaptation, including 'method tailoring', 'method fragment adaptation' and 'situational method engineering'. Method tailoring is defined as:
A process or capability in which human agents determine a system development approach for a specific project situation through responsive changes in, and dynamic interplays between contexts, intentions, and method fragments.
Potentially, almost all agile methods are suitable for method tailoring. Even the DSDM method is being used for this purpose and has been successfully tailored in a CMM context. Situation-appropriateness can be considered as a distinguishing characteristic between agile methods and traditional software development methods, with the latter being relatively much more rigid and prescriptive. The practical implication is that agile methods allow project teams to adapt working practices according to the needs of individual projects. Practices are concrete activities and products that are part of a method framework. At a more extreme level, the philosophy behind the method, consisting of a number of principles, could be adapted (Aydin, 2004).
Some approaches, such as Extreme programming (XP) make the need for method adaptation explicit. With these less-prescriptive frameworks, one of the fundamental ideas is that no one process fits every project, but rather that practices should be tailored to the needs of individual projects (partial adoption of XP practices, as suggested by Beck, has been reported on several occasions ). Mehdi Mirakhorli proposes a tailoring practice that provides a sufficient road-map and guidelines for adapting all the practices. RDP Practice is designed for customizing XP. This practice, first proposed as a long research paper in the APSO workshop at the ICSE 2008 conference, is currently the only proposed and applicable method for customizing XP. Although it is specifically a solution for XP, this practice has the capability of extending to other methodologies. At first glance, this practice seems to be in the category of static method adaptation but experiences with RDP Practice says that it can be treated like dynamic method adaptation. The distinction between static method adaptation and dynamic method adaptation is subtle.
Comparison with other methods
Agile methods have much in common with the Rapid Application Development techniques from the 1980/90s as espoused by James Martin and others. In addition to technology-focused methods, customer-and-design-centered methods, such as Visualization-Driven Rapid Prototyping developed by Brian Willison, work to engage customers and end users to facilitate agile software development.
Further, James M. Kerr and Richard Hunter wrote a book on the subject that presented a day-by-day diary of an actual RAD project, It covered the project from inception to production and contains many of the techniques that forge the backbone of, and are very much present in, today's Agile approaches 
In 2008 the Software Engineering Institute (SEI) published the technical report "CMMI or Agile: Why Not Embrace Both" to make clear that the Capability Maturity Model Integration and Agile can co-exist. Modern CMMI-compatible development processes are also iterative. The CMMI Version 1.3 includes tips for implementing CMMI and agile process improvement together.
Large-scale, offshore and distributed
Agile development has been widely seen as highly suited to certain types of environments, including small teams of experts working on greenfield projects,:157 and the challenges and limitations encountered in the adoption of agile methods in a large organization with legacy infrastructure are well-documented and understood.
In response, a range of strategies and patterns has evolved for overcoming challenges with large-scale development efforts (>20 developers) or distributed (non-colocated) development teams, amongst other challenges; and there are now several recognised frameworks that seek to mitigate or avoid these challenges, including:
- Scaled agile framework (SAFe), Dean Leffingwell inter alia
- Disciplined agile delivery (DAD), Scott Ambler inter alia
- Large-scale scrum (LeSS), Craig Larman and Bas Vodde
- Nexus (scaled professional Scrum), Ken Schwaber
- Scrum at Scale  Jeff Sutherland, Alex Brown
- Enterprise Scrum, Mike Beedle
- Setchu (Scrum-based lightweight framework)  Michael Ebbage
When agile software development is applied in a distributed setting (with teams dispersed across multiple business locations), it is commonly referred to as Distributed Agile Development. The goal is to leverage the unique benefits offered by each approach. Distributed development allow organizations to build software by strategically setting up teams in different parts of the globe, virtually building software round-the-clock (more commonly referred to as follow-the-sun model). On the other hand, agile development provides increased transparency, continuous feedback and more flexibility when responding to changes.
Agile methods were initially seen as best suitable for non-critical software projects, thereby excluded from use in regulated domains such as medical devices, pharmaceutical, financial, nuclear systems, automotive, and avionics sectors, etc. However, in the last several years, there have been several initiatives for the adaptation of agile methods for these domains.
There are numerous standards that may apply in regulated domains, including ISO 26262, ISO 9000, ISO 9001, and ISO/IEC 15504. A number of key concerns are of particular importance in regulated domains which may conflict with the use of agile methods:
- Quality Assurance (QA): Systematic and inherent quality management underpinning a controlled professional process and reliability and correctness of product.
- Safety and Security: Formal planning and risk management to mitigate safety risks for users and securely protecting users from unintentional and malicious misuse.
- Traceability: Documentation providing auditable evidence of regulatory compliance and facilitating traceability and investigation of problems.
- Verification and Validation (V&V): Embedded throughout the software development process (e.g. user requirements specification, functional specification, design specification, code review, unit tests, integration tests, system tests).
Experience and adoption
Agile methods were first used by technology early adopters such as Tektronix. Although Agile can be used with any programming paradigm or language in practice Agile has usually been closely associated with object-oriented environments such as Smalltalk and Lisp and later Java. The initial adopters of Agile methods were usually small to medium sized teams working on unprecedented systems with requirements that were difficult to finalize and likely to change as the system was being developed. One of the first successful large scale uses of Agile was the Brooklyn Union Gas Smalltalk project. This section describes some common problems organizations encounter when they try to adopt Agile methods as well as various techniques to measure the quality and performance of Agile teams.
Common agile pitfalls
Organizations and teams implementing agile development often face difficulties transitioning from more traditional methods such as waterfall development, such as teams having an agile process forced on them. These are often termed agile anti-patterns or more commonly agile smells. Below are some common examples:
Lack of overall project design
A main goal of Agile is to reduce the amount of documentation created during a project and focus on the actual code. This is in contrast to waterfall models where the process is often highly controlled and minor changes to the system require significant revision of supporting documentation. However, this does not justify completely doing without any analysis or design at all. Failure to pay attention to design can cause a team to proceed rapidly at first but then to have significant rework required as they attempt to scale up the system. One of the key features of Agile is that it is an iterative process. When done correctly design emerges as the system is developed and commonalities and opportunities for re-use are discovered.
Adding stories to a sprint in progress
In Agile methods requirements are typically defined by stories. Stories are similar to use case descriptions in object-oriented development. In Agile a sprint is a short period of time during which the team has committed to specific goals. Adding stories to a sprint in progress is detrimental to the flow established by Agile. From Ilan Goldstein's Sprint issues – when sprints turn into crawls, "'Isn't the ability to change course on the fly what Scrum is all about?' Well not quite. Scrum certainly provides provision to change product backlog priorities mid-project however this needs to occur between sprints and not during them."
If an issue arises that requires additions to a sprint, Ilan recommends an abnormal sprint termination. This does not mean that a user story cannot expand. Teams must deal with new information, which may produce additional tasks for a user story. If the new information prevents the user story from being production-ready during the sprint, then it should carry over to the next sprint. However, during the next sprint planning, the user story should be prioritized over all remaining user stories. The information that requires new tasks may have altered the user story priority.
Lack of sponsor support
Agile is often implemented as a grassroots effort in organizations by software development teams trying to optimize their development processes and ensure consistency in the software development life cycle. By not having sponsor support, teams may face difficulties and resistance from business partners, other development teams and management. "Carrying on without an effective sponsor raises the probability of project failure..."
Additionally, the project sponsor is responsible for ensuring the team has appropriate funding and resources.
A survey performed by Version One found respondents cited insufficient training as the most significant cause for failed agile projects Teams have fallen into the trap of assuming the reduced processes of agile development compared to other methodologies such as waterfall means that there are no actual rules for agile development. Agile development is a set of prescribed methodologies, and training/practice is a requirement.
Product owner role is not properly filled
A common mistake is to have the product owner role filled by someone from the development team. According to Johanna Rothman this is a mistake, "When the business is unaccountable, the agile ecosystem breaks down." Having the development team fill this role results in the team making its own decisions on prioritization without real feedback from the business. Additionally, the team either tries to solve business issues internally or delay as they reach outside the core group for input. This can cause finger-pointing and divert from the collaborative process directed.
Teams are not focused
The agile process requires teams who focus on the project to meet project commitments. During a sprint, a resource who has the capacity is expected to take up tasks potentially outside their area of greatest expertise or comfort.
If team members have multiple projects, it is difficult to make spare capacity available to help complete the sprint. "While having information developers working on multiple scrum teams is not ideal, it can be done with some proper planning and judicious evaluation of which meetings you should attend."
Teams may fall into the trap of spending too much time preparing or planning. This is a common trap for teams less familiar with the agile process where the teams feel obligated to have a complete understanding of all user stories or a detailed design. Teams should leverage the ability for Sprints to act as a method discovery and moving forward with the information they do know. As more information is gained it should be applied to the next Sprint.
Problem-solving in the daily standup
A daily standup should be a focused, timely meeting where all team members disseminate information. If problem-solving occurs, it often can only involve certain team members and potentially is not the best use of the entire team's time. If during the daily standup the team starts diving into problem-solving, it should be tabled until a sub-team can discuss, usually immediately after the daily scrum completes.
One of the intended benefits of agile development is to empower the team to make choices, as they are closest to the problem. Additionally, they should make choices as close to implementation as possible—compared to a waterfall approach—to use more timely information in the decision. If team members are assigned tasks by others or too early in the process, the benefits of localized and timely decision making can be lost.
Another tendency is for assigners to box team members into certain roles (for example, team member A must always do the database work), which hinders cross-training. Team members themselves can choose to take on tasks that stretch their abilities and provide cross-training opportunities.
Scrum master as a contributor
Another common pitfall is for a scrum master to act as a contributor. While not prohibited by the Scrum methodology, the scrum master needs to ensure they have the capacity to act in the role of scrum master first and not working on tasks for the project. A scrum master's role is to facilitate the Scrum process. "Facilitating meetings such as a daily scrum, sprint planning, sprint reviews and sprint retrospectives is part of this. A technical contributor's role is to work with the other team members to figure out how to get the work done and to do it."
Having the scrum master also multitasking may result in too many context switches to be productive. Additionally, as a scrum master is responsible for ensuring roadblocks are removed so that the team can make forward progress, the benefit gained by individual tasks moving forward may not outweigh roadblocks that are deferred due to lack of capacity.
Lacking test automation
Due to the iterative nature of agile development, multiple rounds of testing for a project are often needed. "Having an automated testing framework, which takes care of both system and integration tests, adds a lot of firepower to such a team. It not only acts as a safety net against regressions caused by new development, but more importantly frees up a lot of precious developer and tester time - allowing them to focus on the things they do best."
Test automation also supports continued refactoring required by iterative software development. Allowing a developer to quickly run tests to confirm refactoring has not modified the functionality of the application may reduce the workload and increase confidence that cleanup efforts have not introduced new defects.
Allowing technical debt to build up
Focusing on delivering new functionality may result in increased technical debt. The team must allow themselves time for defect remediation and refactoring. Technical debt hinders planning abilities by increasing the amount of unscheduled work as production defects distract the team from further project progress.
Attempting to take on too much in a sprint
A common misconception is that agile development allows continuous change, however a sprint backlog is an agreement of what work can be completed during the sprint. Additionally having too much work-in-progress(WIP) can result in inefficiencies due "to avoid the penalties of wasted time, effort and resources."
A possible issue is the team being pressured into taking on additional work. "An important point to reiterate here is that it's the team that selects how much work they can do in the coming sprint. The product owner does not get to say, 'We have four sprints left so you need to do one-fourth of everything I need.' We can hope the team does that much (or more), but it's up to the team to determine how much they can do in the sprint."
Fixed time, resources, scope, and quality
Agile development fixes time (sprint duration), quality, and ideally resources in advance (though maintaining fixed resources may be difficult if developers are often pulled away from tasks to handle production incidents), while the scope remains variable. The customer or product owner often pushes for a fixed scope for a sprint. However, teams should be reluctant to commit to locked time, resources and scope (commonly known as the project management triangle). Efforts to add scope to the fixed time and resources of agile development may result in decreased quality.
The best Agile practitioners have always emphasized sound engineering principles. As a result there are a number of Agile best practices and tools for measuring the performance of projects and teams.
The Agility Index Measurements (AIM) scores projects against a number of agility factors to achieve a total. The similarly named Agility Measurement Index, scores developments against five dimensions of a software project (duration, risk, novelty, effort, and interaction). Other techniques are based on measurable goals. Another study using fuzzy mathematics has suggested that project velocity can be used as a metric of agility. There are agile self-assessments to determine whether a team is using agile practices (Nokia test, Karlskrona test, 42 points test).
While such approaches have been proposed to measure agility, the practical application of such metrics is still debated. There is agile software development ROI data available from the CSIAC ROI Dashboard.
One of the early studies reporting gains in quality, productivity, and business satisfaction by using Agile methods was a survey conducted by Shine Technologies from November 2002 to January 2003. A similar survey, the State of Agile, is conducted every year starting in 2006 with thousands of participants from around the software development community. The State of Agile survey tracks trends on the benefits of agile, trends, lessons learned, preferred practices and agile methodologies. From the 2013 results released in January 2014, the survey concludes that 73% of respondents say agile software helps them complete software projects faster; 92% say agile improves their ability to manage changing customer priorities; and 87% say agile improves their development team's productivity. In 2014 the survey "Status Quo Agile" showed better results of agile methods regarding all success criteria examined compared to classical project management  Yet another survey conducted in 2006 by Scott Ambler, the Practice Leader for Agile Development with IBM Rational's Methods Group reported similar benefits. Others claim that agile development methods are still too young to require extensive academic proof of their success.
Agile methodologies can be very difficult for large organizations such as governments and multinational banks to faithfully adopt, for reasons ranging from lack of sponsor buy-in to agile, to refusal to heed agile consultants' advice on co-located teams - and particularly in the case of governments, outdated procurement and project management policies that assume non-agile methodologies.
Agile methodologies can be inefficient in large organizations and certain types of projects. Agile methods seem best for developmental and non-sequential projects. Many organizations believe that agile methodologies are too extreme and adopt a hybrid approach that mixes elements of agile and plan-driven approaches. However, DSDM is an agile methodology that in fact mixes elements of agile and plan-driven approaches in a disciplined way, without sacrificing the fundamental principles that make agile work.
The term "agile" has also been criticized as being a management fad that simply describes existing good practices under new jargon, promotes a "one size fits all" mindset towards development strategies, and wrongly emphasizes method over results.
Alistair Cockburn organized a celebration of the 10th anniversary of the Agile Manifesto in Snowbird, Utah on February 12, 2011, gathering some 30+ people who had been involved at the original meeting and since. A list of about 20 elephants in the room ("undiscussable" agile topics/issues) were collected, including aspects: the alliances, failures and limitations of agile practices and context (possible causes: commercial interests, decontextualization, no obvious way to make progress based on failure, limited objective evidence, cognitive biases and reasoning fallacies), politics and culture. As Philippe Kruchten wrote:
The agile movement is in some ways a bit like a teenager: very self-conscious, checking constantly its appearance in a mirror, accepting few criticisms, only interested in being with its peers, rejecting en bloc all wisdom from the past, just because it is from the past, adopting fads and new jargon, at times cocky and arrogant. But I have no doubts that it will mature further, become more open to the outside world, more reflective, and also therefore more effective.
Applications outside software development
Agile methods have been extensively used for development of software products and some of them use certain characteristics of software, such as object technologies. However, these techniques can be applied to the development of non-software products, such as computers, motor vehicles, medical devices, food, clothing, and music; see Flexible product development. Some of the wider principles of agile have also found application in general management (e.g., strategy, governance, risk, finance) under the terms business agility or Agile Business Management.
Under an Agile Business Management model, agile techniques, practices, principles and values are expressed across four domains.
- Integrated customer engagement - to embed customers within any delivery process to share accountability for product/service delivery.
- Facilitation-based management - adopting agile management models, like the role of Scrum Master, to facilitate the day-to-day operation of teams.
- Agile work practices - adopting specific iterative and incremental work practices such as Scrum, Kanban, Test-Driven Development or Feature-Driven Development across all business functions (from Sales, Human Resources, Finance and Marketing).
- An enabling organisational structure - with a focus on staff engagement, personal autonomy and outcomes based governance.
Agile development paradigms can be used in other areas of life such as raising children. Its success in child development might be founded on some basic management principles; communication, adaptation and awareness. Bruce Feiler has claimed that the basic Agile Development paradigms can be applied to household management and raising children. In his TED Talk, "Agile programming -- for your family", these paradigms brought significant changes to his household environment, such as the kids doing dishes, taking out the trash, and decreasing his children's emotional outbreaks, which inadvertently increased their emotional stability.
- Collier, Ken W. (2011). Agile Analytics: A Value-Driven Approach to Business Intelligence and Data Warehousing. Pearson Education. pp. 121 ff. ISBN 9780321669544.
What is a self-organizing team?
- "What is Agile Software Development?". Agile Alliance. 8 June 2013. Retrieved 4 April 2015.
- Beck, Kent; et al. (2001). "Manifesto for Agile Software Development". Agile Alliance. Retrieved 14 June 2010.
- Martin, James (1991). Rapid Application Development. Macmillan. ISBN 0-02-376775-8.
- Kerr, James M.; Hunter, Richard (1993). Inside RAD: How to Build a Fully Functional System in 90 Days or Less. McGraw-Hill. ISBN 0-07-034223-7 p.3
- Gerald M. Weinberg, as quoted in Larman, Craig; Basili, Victor R. (June 2003). "Iterative and Incremental Development: A Brief History". Computer 36 (6): 47–56. doi:10.1109/MC.2003.1204375. ISSN 0018-9162.
We were doing incremental development as early as 1957, in Los Angeles, under the direction of Bernie Dimsdale at IBM's Service Bureau Corporation. He was a colleague of John von Neumann, so perhaps he learned it there, or assumed it as totally natural. I do remember Herb Jacobs (primarily, though we all participated) developing a large simulation for Motorola, where the technique used was, as far as I can tell ... All of us, as far as I can remember, thought waterfalling of a huge project was rather stupid, or at least ignorant of the realities. I think what the waterfall description did for us was make us realize that we were doing something else, something unnamed except for 'software development.'
- Edmonds, E. A. (1974). "A Process for the Development of Software for Nontechnical Users as an Adaptive System". General Systems 19: 215–18.
- Note by Edmonds: I presented these ideas in London in 1970 and first submitted the paper to the Journal Computer Aided Design. It was rejected with the comment "If you don't know what you are going to do before you start you shouldn't start"! Only then did I submit it to General Systems.
- "Evolutionary Project Management". Gilb.
- Larman, Craig (2004). Agile and Iterative Development: A Manager's Guide. Addison-Wesley. p. 27. ISBN 978-0-13-111155-4.
- "Agile With a Capital "A" Vs. agile With a Lowercase "a"". Rally. 2010. Retrieved 9 September 2015.
- Ambler, S.W. "Examining the Agile Manifesto". Retrieved 6 April 2011.
- Jim Highsmith (2001). "History: The Agile Manifesto". agilemanifesto.org.
- Beck, Kent; et al. (2001). "Principles behind the Agile Manifesto". Agile Alliance. Archived from the original on 14 June 2010. Retrieved 6 June 2010.
- Anderson, David (2005). "Declaration of Interdependence".
- "PRINCE2® Agile". www.axelos.com. Axelos Ltd. Retrieved 6 March 2015.
- Beck, Kent (1999). "Embracing Change with Extreme Programming". Computer 32 (10): 70–77. doi:10.1109/2.796139.
- Gauthier, Alexandre (17 August 2011). "What is scrum". Planbox.
- Cockburn, Alistair. "Information radiator".
- Ambler, Scott (12 April 2002). Agile Modeling: Effective Practices for EXtreme Programming and the Unified Process. John Wiley & Sons. pp. 12, 164, 363. ISBN 978-0-471-20282-0.
- Vasiliauskas, Vidas (2014). "Developing agile project task and team management practices". Eylean.
- Jeffries, Ron; Anderson, Ann; Hendrickson, Chet (2001). Extreme Programming installed. Addison-Weslsy. pp. 72–147. ISBN 0201-70842-6.
- Larman, Craig (2004). Agile and Iterative Development: A Manager's Guide. Addison-Wesley. p. 27. ISBN 978-0-13-111155-4.
- Boehm, B.; R. Turner (2004). Balancing Agility and Discipline: A Guide for the Perplexed. Boston, MA: Addison-Wesley. ISBN 0-321-18612-5. Appendix A, pages 165–194
- Larman, Craig (2004). "Chapter 11: Practice Tips". Agile and Iterative Development: A Manager's Guide. p. 253. ISBN 9780131111554. Retrieved October 14, 2013.
- Sliger, Michele; Broderick, Stacia (2008). The Software Project Manager's Bridge to Agility. Addison-Wesley. p. 46. ISBN 0-321-50275-2.
- Boehm, B.; R. Turner (2004). Balancing Agility and Discipline: A Guide for the Perplexed. Boston, MA: Addison-Wesley. pp. 55–57. ISBN 0-321-18612-5.
- Rakitin, Steven R. (2001). "Manifesto Elicits Cynicism: Reader's letter to the editor by Steven R. Rakitin". IEEE Computer 34: 4.
The article titled 'Agile Software Development: The Business of Innovation' . . . is yet another attempt to undermine the discipline of software engineering . . . We want to spend all our time coding. Remember, real programmers don’t write documentation.
- Scott Ambler. "Agile/Lean Documentation: Strategies for Agile Software Development".
- Scott Ambler. "Just Barely Good Enough Models and Documents: An Agile Best Practice".
- Geoffrey Wiseman (July 18, 2007). "Do Agile Methods Require Documentation?". InfoQ. quoting Cooper, Ian (6 July 2007). "Staccato Signals:Agile and Documentation". WordPress.com.
- Heap, Tony. "Business Analyst Designer Method". http://www.its-all-design.com/. Retrieved 2015-08-08.
- Abrahamson P, Salo O, Ronkainen J, Warsta J (2002). Agile software development methods: Review and analysis (PDF) (Technical report). VTT. 478.
- "Guide to Agile Practices". the Agile Alliance.
- Aydin, M.N., Harmsen, F., Slooten, K. v., & Stagwee, R. A. (2004). An Agile Information Systems Development Method in use. Turk J Elec Engin, 12(2), 127-138
- Abrahamsson, P., Warsta, J., Siponen, M.T., & Ronkainen, J. (2003). New Directions on Agile Methods: A Comparative Analysis. Proceedings of ICSE'03, 244-254
- Aydin, M.N., Harmsen, F., Slooten van K., & Stegwee, R.A. (2005). On the Adaptation of An Agile Information(Suren) Systems Development Method. Journal of Database Management Special issue on Agile Analysis, Design, and Implementation, 16(4), 20-24
- Lankhorst, Mark (2012). Agile Service Development. Springer-Verglag. pp. 17–19. ISBN 978-3-642-28187-7.
- Kerr, James M.; Hunter, Richard (1993). Inside RAD: How to Build a Fully Functional System in 90 Days or Less. McGraw-Hill. ISBN 0-07-034223-7.
- "TECHNICAL NOTE CMU/SEI-2008-TN-003 CMMI or Agile: Why Not Embrace Both".
- CMMI Product Team, ; CMMI for Development, Version 1.3 (CMU/SEI-2010-TR-033). Software Engineering Institute, Carnegie Mellon University, 2010. http://www.sei.cmu.edu/library/abstracts/reports/10tr033.cfm
- Beck, K. (1999). Extreme Programming Explained: Embrace Change. Boston, MA: Addison-Wesley. ISBN 0-321-27865-8.
- Evans, Ian. "Agile Delivery at British Telecom". Retrieved 21 February 2011.
- W. Scott Ambler (2006) Supersize Me in Dr. Dobb's Journal, 15 February 2006.
- Schaaf, R.J. (2007). Agility XL Systems and Software Technology Conference 2007, Tampa, FL
- "Bridging the Distance". Sdmagazine.com. Retrieved 1 February 2011.
- Martin Fowler. "Using an Agile Software Process with Offshore Development". Martinfowler.com. Retrieved 6 June 2010.
- leffingwell, dean. "scaled agile framework". scaled agile framework.
- Schwaber, Ken. scrum.org (PDF) https://www.scrum.org/Portals/0/NexusGuide%20v1.1.pdf. Retrieved 14 September 2015. Missing or empty
- Sutherland, Jeff; Brown, Alex. "Scrum At Scale: Part 1". Retrieved 14 September 2015.
- Beedle, Mike. "Enterprise Scrum". Retrieved 25 September 2015.
- Ebbage, Michael. "Setchu – Agile at Scale". Retrieved 30 September 2015.
- Agile Processes Workshop II Managing Multiple Concurrent Agile Projects. Washington: OOPSLA 2002
- Fitzgerald, B.; Stol, K.-J.; O'Sullivan, R.; O'Brien, D. (May 2013). "Scaling agile methods to regulated environments: An industry case study". 2013 35th International Conference on Software Engineering (ICSE): 863–872. doi:10.1109/ICSE.2013.6606635.
- Cawley, Oisín; Wang, Xiaofeng; Richardson, Ita (2010). Abrahamsson, Pekka; Oza, Nilay, eds. "Lean/Agile Software Development Methodologies in Regulated Environments – State of the Art". Lean Enterprise Software and Systems. Lecture Notes in Business Information Processing (Springer Berlin Heidelberg) 65: 31–36. ISBN 978-3-642-16415-6.
- McHugh, Martin; McCaffery, Fergal; Coady, Garret (2014-11-04). Mitasiunas, Antanas; Rout, Terry; O’Connor, Rory V.; et al., eds. "An Agile Implementation within a Medical Device Software Organisation". Software Process Improvement and Capability Determination. Communications in Computer and Information Science (Springer International Publishing) 477: 190–201. ISBN 978-3-319-13035-4.
- Thor Myklebust, Tor Stålhane, Geir Kjetil Hanssen, Tormod Wien and Børge Haugset: Scrum, documentation and the IEC 61508-3:2010 software standard, http://www.sintef.no/globalassets/ec-61508-documentation-and-safescrum-psam12.pdf
- Davis, John; Morgan, Tom (January 1993). "Object-Oriented Development at Brooklyn Union Gas". IEEE Software 10 (1): 67–74. Retrieved 2 October 2015.
- Beck, Kent (2000). Extreme Programming Explained. Addison-Wesley. pp. 1–24. ISBN 0201616416.
- Shore, James; Warden, Shane. Art of Agile Development. p. 47.
- Beck, Kent (2000). Extreme Programming Explained. Addison-Wesley. pp. 48–49. ISBN 0201616416.
- Rouse, Margaret. "Sprint (software development) definition". http://searchsoftwarequality.techtarget.com. Retrieved 2 October 2015.
- Goldstein, Ilan. "Sprint issues – when sprints turn into crawls". http://www.axisagile.com.au/. Retrieved 2014-06-08.
- Bourne, Lynda. "What Does a Project Sponsor Really Do?". blogs.pmi.org. Retrieved 2014-06-08.
- "Project Roles and Responsibility Distribution". agile-only.com. Retrieved 2014-06-15.
- "State of Agile Development Survey Results". http://www.versionone.com/. Version One. Retrieved 2014-06-08.
- Sims, Chris; Johnson, Hillary Louise (2011-02-15). The Elements of Scrum (Kindle ed.). Dymaxicon. p. 73.
- Rothman, Johanna Rothman. "When You Have No Product Owner At All". http://www.jrothman.com/. Retrieved 2014-06-08.
- Fox, Alyssa. "Working on Multiple Agile Teams". techwhirl.com/. Retrieved 2014-06-14.
- "Daily Scrum Meeting". http://www.mountaingoatsoftware.com/. Retrieved 2014-06-14.
- May, Robert. "Effective Sprint Planning". http://www.agileexecutives.org/. Retrieved 2014-06-14.
- Berczuk, Steve. "Mission Possible: ScrumMaster and Technical Contributor". http://www.agileconnection.com/. Retrieved 2014-06-14.
- Namta, Rajneesh. "Thoughts on Test Automation in Agile". http://www.infoq.com/. Retrieved 2014-06-14.
- Band, Zvi. "Technical Debt + Red October". Retrieved 8 June 2014.
- Shore, James. "The Art of Agile Development: Refactoring". www.jamesshore.com. Retrieved 2014-06-14.
- "Step 4: Sprint Planning (Tasks)". www.allaboutagile.com. Retrieved 2014-06-14.
- George, Claire. "Why Limiting Your Work-in-Progress Matters". leankit.com. Retrieved 2014-06-14.
- "Sprint Planning Meeting". www.mountaingoatsoftware.com. Retrieved 2014-06-14.
- McMillan, Keith. "Time, Resources, Scope… and Quality.". http://www.adeptechllc.com/. Retrieved 2014-06-15.
- "David Bock's Weblog : Weblog". Jroller.com. Retrieved 2 April 2010.
- "Agility measurement index". Doi.acm.org. Retrieved 2 April 2010.
- Peter Lappo; Henry C.T. Andrew. "Assessing Agility" (PDF). Retrieved 6 June 2010.
- Kurian, Tisni (2006). Agility Metrics: A Quantitative Fuzzy Based Approach for Measuring Agility of a Software Process, ISAM-Proceedings of International Conference on Agile Manufacturing'06(ICAM-2006), Norfolk, U.S.
- Joe Little (2 December 2007). "Nokia test, A scrum-specific test". Agileconsortium.blogspot.com. Retrieved 6 June 2010.
- Mark Seuffert, Mayberg, Sweden. "Karlskrona test, A generic agile adoption test". Mayberg.se. Retrieved 5 Apr 2014.
- "How Agile Are You? (Take This 42 Point Test)". allaboutagile.com/. Retrieved 3 April 2014.
- CSIAC ROI Dashboard Retrieved 11 November 2011.
- "Agile Methodologies Survey Results" (PDF). Shine Technologies. January 2003. Retrieved 3 June 2010.
95% stated that there was either no effect or a cost reduction ... 93% stated that productivity was better or significantly better ... 88% stated that quality was better or significantly better ... 83% stated that business satisfaction was better or significantly better
- "2013 State of Agile report: Why Agile?". stateofagile.com. 27 January 2014. Retrieved 13 August 2014.
- Status Quo Agile, Second study on success and forms of usage of agile methods. Retrieved 1 July 2015
- Ambler, Scott (3 August 2006). "Survey Says: Agile Works in Practice". Dr. Dobb's. Retrieved 3 June 2010.
Only 6% indicated that their productivity was lowered ... No change in productivity was reported by 34% of respondents and 60% reported increased productivity ... 66% [responded] that the quality is higher ... 58% of organizations report improved satisfaction, whereas only 3% report reduced satisfaction.
- "Answering the "Where is the Proof That Agile Methods Work" Question". Agilemodeling.com. 19 January 2007. Retrieved 2 April 2010.
- Larman, Craig; Bas Vodde (2009-08-13). "Top Ten Organizational Impediments to Large-Scale Agile Adoption". InformIT.
- Barlow, Jordan B.; Justin Scott Giboney; Mark Jeffery Keith; David W. Wilson; Ryan M. Schuetzler; Paul Benjamin Lowry; Anthony Vance (2011). "Overview and Guidance on Agile Development in Large Organizations". Communications of the Association for Information Systems 29 (1): 25–44.
- Kupersmith, Kupe. "Agile is a Fad".
- Kruchten, Philippe (2011-06-20). "Agile's Teenage Crisis?". InfoQ.
- Smith, Preston G (2007). Flexible Product Development. Jossey-Bass. p. 25. ISBN 978-0-7879-9584-3.
- "WIKISPEED – Applying Agile software principles and practices for fast automotive development". Agile Business Management Consortium. 2013-12-03. Retrieved 2015-09-11.
- Newton Lee (2014). "Getting on the Billboard Charts: Music Production as Agile Software Development," Digital Da Vinci: Computers in Music. Springer Science+Business Media. ISBN 978-1-4939-0535-5.
- Moran, Alan (2015). Managing Agile: Strategy, Implementation, Organisation and People. Springer Verlag. ISBN 978-3-319-16262-1.
- Leybourn, Evan (2013). Directing the Agile Organisation: A Lean Approach to Business Management. IT Governance Publishing. ISBN 978-1-849-28491-2.
- "Pair Trading: Collaboration in Finance". The Agile Director. 2015-03-11. Retrieved 2015-09-11.
- Abrahamsson, P., Salo, O., Ronkainen, J., & Warsta, J. (2002). Agile Software Development Methods: Review and Analysis. VTT Publications 478.
- Cohen, D., Lindvall, M., & Costa, P. (2004). An introduction to agile methods. In Advances in Computers (pp. 1–66). New York: Elsevier Science.
- Dingsøyr, Torgeir, Dybå, Tore and Moe, Nils Brede (ed.): Agile Software Development: Current Research and Future Directions, Springer, Berlin Heidelberg, 2010.
- Moran, Alan. Managing Agile: Strategy, Implementation, Organisation and People. Springer, Berlin Heidelberg, 2015. (ISBN 978-3-319-16262-1)
- Fowler, Martin. Is Design Dead?. Appeared in Extreme Programming Explained, G. Succi and M. Marchesi, ed., Addison-Wesley, Boston. 2001.
- Larman, Craig and Basili, Victor R. Iterative and Incremental Development: A Brief History IEEE Computer, June 2003
- Riehle, Dirk. A Comparison of the Value Systems of Adaptive Software Development and Extreme Programming: How Methodologies May Learn From Each Other. Appeared in Extreme Programming Explained, G. Succi and M. Marchesi, ed., Addison-Wesley, Boston. 2001.
- M. Stephens, D. Rosenberg. Extreme Programming Refactored: The Case Against XP. Apress L.P., Berkeley, California. 2003. (ISBN 1-59059-096-1)
- Shore, J., & Warden S. (2008). The Art of Agile Development. O'Reilly Media, Inc.
- Willison, Brian (2008). Iterative Milestone Engineering Model. New York, NY.
- Willison, Brian (2008). Visualization Driven Rapid Prototyping. Parsons Institute for Information Mapping.
- The New Methodology Martin Fowler's description of the background to agile methods
- Ten Authors of The Agile Manifesto Celebrate its Tenth Anniversary
- Agile Manifesto
- Agile Rapid Website Development