All about Scrum, how it's gaps can be positive for building a project and the G.R.R. Martin effect.
Agile Frameworks II
As XP, Scrum is an iterative and incremental agile software development framework, which main focus is to aid in the development and sustainment of complex products.
Scrum was presented at the OOPSLA conference back in 1995 by Ken Schwaber and Jeff Sutherland (two of the agile manifesto authors), based on the learning both gained over the previous few years applying it.
Scrum borrowed some of the ideas described in a study called “New Product Development Game” published in 1986 in the Harvard Business Review by Hirotaka Takeuchi and Ikujiro Nonaka; where they described a new approach to commercial product development by using an analogy that compares a high-performing, cross-functional team to the scrum formation used by Rugby teams (the Scrum name was taken from this Rugby formation name).
Scrum is defined by a set of roles, events and artifacts and a set of rules that binds them together.
Product Owner: is a representative of the business (represents stakeholders and is the voice of the customer),the sole person responsible of managing the Product Backlog and ensuring the team delivers value to business.
Development Team: a cross-functional and self-organizing team of professionals which deliver potentially shippable or releasable increments, of the product being developed, at the end of each sprint. This team must be empowered by the organization to arrange and manage their own work.
Scrum Master: this person is a servant-leader for the Scrum Team responsible of ensuring Scrum is understood and enacted as it was intended, also helps in the interactions between the Scrum Team and people outside of it to maximize the value of such interactions.
The Sprint: each of the iterations in which the project gets divided; the idea is to split work into smaller more manageable amounts for the team to focus on, each of these blocks of work will be done during each sprint. So a Sprint is a time-box during which a usable and potentially shippable product increment gets created; the duration of the sprints should be kept constant during the entire project.
Sprint Planning: a time-boxed meeting (8 hours or less) at the beginning of each sprint to plan the work to be performed on it; this plan is created by a collaborative work of the entire Scrum Team. Sprint Planning answers the following:
Daily Scrum: It is a time boxed (fixed to 15 mins) daily meeting for the Development Team to synchronize activities and plan for the next 24 hours by inspecting the work done since the last Daily Scrum and forecasting the work that could be done before the next one. It is important to hold this meeting everyday at the same time on the same place. The Development Team is the only Role required to be present on this event.
Sprint Review: It is a time-boxed meeting (typically 4 hours for 1 month sprints) held at the end of each Sprint for the Scrum Team and Stakeholders to review the work done (inspect the increment) and adapt the Product BackLog if needed. Is important to notice this is an informal meeting, not a status meeting, is objective is to foster collaboration.
Sprint Retrospective: It is a time-boxed meeting (3 hours or less) that occurs after the Sprint Review and before the next Sprint Planning; its objective is to give an opportunity to the Scrum Team to inspect itself to identify improvements and to plan for them, so that they could be implemented on the next Sprint.
Product Backlog: is an ordered list of all the items (usually user stories and/or requirements) that might be implemented in the Product. It is the responsibility of the Product Owner to keep the Product Backlog available, updated and ordered (by priority).
Sprint Backlog: it is another ordered list that contains the set of items, from the Product Backlog, that are to be implemented in the current Sprint. It also contains a plan for delivering the next product increment.
Increment: It is the sum of all Product Backlog Items completed in the current Sprint plus all previous Sprint Increments.
Scrum also relies heavily on transparency; as much of the decisions made will be based on the perceived state of artifacts, thus their transparency is crucial. There are many techniques and practices toward transparency that I won’t discuss here, however there is one of them that is a key element towards transparency; and that is the “Definition of Done”.
A clear “Definition of Done” is important because each Product Backlog Item eventually will be set to “done”; it guides the development team in knowing how many Product Backlog Items can be selected into a Sprint during Sprint Planning; Also the progress of how much work has been done is ruled by this definition, so is the Increment, as the increment only contains items that are described as “done”.
The Scrum Team will set their own definition of “done”, stating what the entire team understand as “done”. As the Team matures their definitions of done will be better; they will expand to include other important characteristics like quality.
So a typical Scrum project goes like this:
A Product Backlog is created by the Product Owner
The Sprint starts.
Sprint Planning takes place and the Development Team creates a Sprint Backlog by selecting (prioritizing) the set of items from the Product Backlog that can be delivered in the upcoming Sprint; and they decide on how to implement them.
During the Sprint, a daily Scrum takes place every day at the beginning of the day to assess previous day’s work and to keep the team focused on the objective and the goal of the Sprint.
The Sprint Review and Retrospective takes place.
The Sprint ends; at this point the work done must be “potentially shippable”.
The next Sprint starts with the next Sprint Planning and the work continues following the same steps until all Sprints are completed.
So far we have covered the basics of Scrum, but before I finish this post I want to highlight some important facts that you must be aware of:
According to the Scrum Guide’s (http://www.scrumguides.org/scrum-guide.html) definition of Scrum; Scrum is:
Simple to understand
Difficult to master
At this point it is quite easy to understand why Scrum is considered “lightweight” and “simple to understand”; however why is it “Difficult to master”? It might not be so obvious...
Part of this difficulty has to do with the fact that Scrum is a framework, and as we have discussed earlier, frameworks are incomplete and are meant to be that way so that they remain highly customizable. This flexibility adds complexity to their adoption.
Most frameworks require a fill-in-the-blank and tailoring work to be done before they can be adopted; with Scrum is no different. Adopting it might require you to set new processes, rules and norms; and the use of practices and tools that might produce a deep change (depending on how agile friendly the organization is) in the organization’s way of developing software.
One of the principal things to notice is that Scrum focuses on the management part of the software development process and it leaves the technical aspects out; so you are in charge to fill those in.
A big problem is that not all organizations adopting Scrum are aware of it, and adopt Scrum without defining any of the technical aspects; thus leaving each Developing Team without any technical regulation and guide that would have helped them during software development.
This is highly risky as the success of the project will be left in the shoulders of the Development Team’s ability to keep things running smoothly each sprint (keep the adaptability and the quality of the code high enough to remain agile and prevent software decay).
When choosing Scrum, make sure the organization is prepared and aware of the need to define the technical aspects that will norm and guide - processes, rules, practices, tools and techniques - the development of software (you can also adopt additional frameworks that can help you define these aspects).
The G.R.R. Martin effect in scrum teams
There is a common effect that software development teams face when developing big and complex systems; and is more notorious when using Scrum. So, I’m going to use an analogy that involves George R. R. Martin’s work to exemplify it (as I’m a huge fan of his “A Song of Ice and Fire” series).
For people that might not know who George R. R. Martin is; he is the author of the acclaimed fantasy novel series “A Song of Ice and Fire” (a series of [5 published at the time of this writing] fantasy books that are the source material for HBO’s show Game of Thrones).
At the beginning of this year, Martin received heavy criticism because another announcement informed the delay of “Winds of Winter” the sixth book on his “A Song of Ice and Fire” series forecasted for the beginning of this year and before HBO’s GOT season six aired.
Most of the critics and fans blamed the attention Martin was giving to other projects and to the time he has invested during his participation on HBO’s GOT series as an advisor and also to the different events like interviews and conventions around the world to promote the show.
I agree that these might be part of the cause of such delays, however even if he was fully focused on solely writing the remaining books the time taken to publish each subsequent book would still have increased over time.
Let's review the book release dates:
Two years was the time Martin needed to write and publish each book at the beginning of the series; but as the story got bigger, more complicated and more plot lines and characters spawned in, the time needed to complete and publish the next book started to increase (five years between the third and fourth books and six years between the fourth and fifth book).
This is comprehensible and natural as there is too much backstory that needs to be handled; there are many more plots and characters to be managed for him to be sure he is not impacting or introducing inconsistencies in the story as he continues to write it; that, undoubtedly, slows him down.
Mmm… this sound familiar right? software developers face a similar problem:
As a software grows, harder will it be for developers to keep adding or modifying code without impacting something else. Big code bases are hard to maintain as there would be more dependencies to be handled, more lines of code that need to be managed; developers will take longer to code trying to prevent injecting undesired behaviours (bugs) as each new line of code will potentially impact more pieces, incompatibilities might be found and rearranges in code would need to be done to accommodate new pieces; testing the software will also take longer as the regression tests that must be executed will keep growing; all this complexities add up to the possibilities of introducing bad code further slowing down the team.
This effect is common to all software development, however agile development is more susceptible to it, as it deals with unstable sets of requirements while iterating in shorter periods of time often trying to avoid looking too far away into the future. Thi is why the Agile manifesto has one principle for continuous attention to technical excellence and good design.
Scrum teams are even more susceptible because Scrum focuses on the management side of agile development but “forgets” the technical aspects of it, or rather trusts in the organization's technical savviness to do it; sadly many organizations can’t do it because of limited resources or worst, because they aren’t aware of it.
This effect, and the pressure to keep a constant speed across sprints, can foster the rotting of code (bad technical decisions start to happen more and more often) leading to the proliferation of, as uncle Bob states it,“big messes”.
If you haven’t heard of Bob Martin (uncle Bob), he is an expert on Agile development, I highly recommend you to read his article “The land that Scrum Forgot” where he describes the ability of Scrum teams to create big messes; partly because human natrue, and also (as the title of the article implies) because of Scrum’s deliverate gaps towards technical aspects of agile development.
The good news is that there are many engineering practices, tools and techniques (some already adopted by other agile frameworks like XP) that are meant to aid agile developments to stay clean (avoid big messes). Organizations willing to use Scrum and their Scrum teams should evaluate and adopt some of them (as a way to keep work clean and as a guide to be used during agile development) to keep focused in their pursue of delivering value while keeping the code clean and adaptable.
In the next post I’ll name and describe some of the most commonly adopted practices and techniques that will help you prevent big Scrum messes.