We have a major upcoming project here at Cocomore which is in the initial planning phase. It’s too early to provide the finer details of the project, but it involves creating a product database for a large publishing house and Drupal 7 has been chosen as the project framework. By mid-June, we will likely have four developers working on it, full-time. Of course, anyone who deals with software development almost certainly knows the problems that tend to occur during the planning of large-scale, long-term projects like this one: in the beginning, the client is often not yet 100% certain of their needs or desired end results, so new requirements and ideas arise in the middle of project development. This means that projects built to the initial specifications often fail to completely meet the client’s needs, which can be disappointing for everyone involved.
Hoping to avoid this scenario, we (our software development team, and project and senior management) proposed implementing the project using Scrum. This allows us to flexibly respond to changes in requirements and keeps the customer closely tied to the project development process so we can avoid unpleasant surprises at the end of development. At a recent Cocomore “KnowledgeLab”, I presented a brief overview of Scrum methodology, a topic which is certainly too broad to cover, in-depth, in an hour-long presentation. So I limited the scope of my presentation to the most essential elements of the Scrum process: roles, events, and artifacts; then used Lego Scrum to try to better illustrate the model. This article provides an overview of the same basic concepts covered in our workshop session and describes how I helped immerse team members who had not previously worked with Scrum in the concepts and processes. We have been using Scrum for other “ambitious” Drupal projects and plan to provide in-depth case studies for some of them, with details about more specifics related to Drupal; this article provides a general foundation for understanding these upcoming case studies.
Scrum: a process model for agile software development
Agile software development is characterized primarily by an iterative procedure with alternating planning and development phases. The advantage this provides is that parts of the system are developed early on and can be tested before implementation of other parts. This reduces the risk that project development heads in the wrong direction. Rather, responding quickly and flexibly to changes in the requirements, the components of a system can be redefined to best meet a client’s real needs.
The background of Scrum
The term “Scrum” originally comes from the sport of rugby and means “crowd”. Team-members work closely in a circular pack while their opponents try to prevent them from gaining ground. To the casual viewer, this may look merely like a disorganized shoving match, but it is based on a strict set of tactical rules and involves strategy and planning. The comparison is quite appropriate, because, as in rugby, in this form of development there is a team of players who stick together and adhere strictly to a few, well-defined rules. In general, the team plays the main role in Scrum. Unlike the classical approach to software development defined in the waterfall model, the team estimates how long it will take to implement the tasks requested by the product owner and decides which tasks can be accomplished during the timeframe of a sprint.
“Roles” in the Scrum process
Similar roles exist in Scrum. In addition to external roles such as “customer”, “manager”, and “user”, there are three primary roles involved in Scrum: the “Product Owner”, “ScrumMaster”, and “Development Team”:
- The “Product Owner” provides the team with a vision for the product that must be developed. She is in close contact with the customer. This role defines the product’s features and prioritizes them. After each sprint, she is responsible for accepting or rejecting the functionality delivered by the Team.
- The “Development Team” builds the product’s functionality, as requested by the Product Owner. It is responsible for compliance with an agreed set of quality standards for each component and jointly determines the required duration needed to implement each task. The Team also determines how many features it can deliver in a sprint.
- The “ScrumMaster” assists in the implementation of the Sprint’s goals. In order to do so, the ScrumMaster does not actually participate in the development process, nor does she influence the team’s decisions. Rather, she helps eliminate problems which hinder the team’s productivity and aids in guiding the sprint to ensure that the Scrum process is followed.
Meetings & “Artifacts” in the Scrum process
Before development begins, the Scrum Team commits the Definition of Done (DoD), in short, they decide, in advance, what will determine whether a feature has been completed, to specification. The Product Owner defines the tasks necessary to achieve the desired objective. These tasks are included in the “product backlog”. Ideally, these backlog items are defined with the help of “user stories”, with requirements which are are user-oriented rather than technology-oriented. For each user story, there should be at least one “user-acceptance test”. This not only helps during the development process, but also during the sprint review for accepting delivered functionality.
Backlog items in a Drupal-based project might typically involve tasks to assess which goals will entail new content types, whether existing “contrib” modules will satisfy elements of the use case, etc. The acceptance criteria might contain what fields these content types will need, how forms have to be validated, what happens after a submission, etc. The DOD might then involve adding modules to the project repository, configuring modules and content types with a Feature or installation profile, following the Drupal Coding standards, applying tests, etc. Of course some backlog items will often require custom modules, some of which might be ideal candidates for a Product Owner to sponsor as a new contribution to the Drupal community, the benefits to the Product Owner including likelihood of community involvement leading to functional improvements over time as well as building “good will” in the Drupal community—taking from it and giving something back.
In Sprint Planning Meeting 1, the Product Owner identifies individual backlog items and explains them. The Team is encouraged to question ambiguities and thus to get an accurate idea ofwhat the Product Owner expects of them. Subsequently, the Team estimates how long they will need for the implementation of an item in the backlog, a step which commonly involves the “Planning Poker” technique. According to these estimates, the Product Owner re-prioritizes the backlog items and determines the goals for the first sprint.
In the following Sprint Planning Meeting 2, items are again chosen for the sprint backlog. If the individual tasks are too large, the Team attempts to break them down into smaller sub-tasks, with the aim to ensure that all tasks on the backlog can be managed in a day or less. The Sprint Backlog is used during the Sprint to document which team member has been assigned to which tasks and what the status of each task is. For this purpose, a “Scrum Wall” or “task board” is often used, with columns for sprint members to quickly see the status of each task: columns or sections are created for “To-do”, “In progress”, and “Done”, allowing tasks to easily be sorted.
A sprint is always the same length, which should be between 14 and 30 days. Every day there is a “Daily Scrum”; this is a short meeting which attendees try to keep to a maximum of 15 minutes: each team member says what they accomplished the day before and what they plan to do during the current day; problems and obstacles are addressed. If determining a solution to an obstacle will take longer than the allotted time-frame, the ScrumMaster documents this in her “Impediment Backlog”—a list of obstacles impeding the accomplishment of the sprint goals—and works to find a way to resolve them (outside of the daily scrum meeting).
Each sprint ends with a “Sprint Review”. This is a meeting in which the team presents their work. The Product Owner verifies that the tasks fulfill her requirements. The review accepts only those tasks that are truly ready; unstable or untested items migrate back into the product backlog. Work progress is recorded in a “Burndown Chart” where time is on the x-axis and the y-axis includes outstanding tasks. The Burndown Chart helps to provide an overview of progress in working through the project backlog. But, as already mentioned, the issue is relatively complex and not really suited to a brief explanation. Now for the fun part…
As the acting Product Owner in our Scrum workshop, I established a project to build a “city”. Having dug out my old Legos from the cupboard (I hope nobody resented that they were a little dusty), I considered what I’d like to have built in my Lego town. I wanted: houses, a school, a bus stop, a Simpsons statue, and a few other things. I wrote these objectives on slips of paper and put them on our backlog “task board”. During the Team’s Planning Poker session, there was much debate about the size of each item, e.g. whether an elementary school is “bigger” than a church. The Team then considered what could be implemented in a sprint and the sprint backlog items were transferred to the backlog.
During our mini-model of the sprint process, each sprint was set at 3 minutes and we had three sprints to complete the project. After each sprint, I was allowed to mercilessly tear down built elements which did not meet my acceptance criteria. Of course there were heated discussions at these times about whether a building was not yet ready and there was haggling about windows, doors and church crosses. But in the end, the decision to accept or reject features was left to me, as the Product Owner, so some “completed” tasks migrated back into the product backlog. And so we continued to the next sprint planning, the next sprint, and so on. In the end, only the church and primary school were left with small defects, which I accepted; the rest of the city was just as I’d desired.
The whole process was certainly fun for everyone involved and I believe it contributed to an improved understanding of the steps in the Scrum development process, too.
And here’s the result, our beautiful, colorful Lego city!