Scaling Agile the Spotify way

The first book that I recommend anyone read about Agile is “Scrum and XP from the Trenches” by Henrik Kniberg.

Kniberg has a relaxed, informal and straight-forward way of explaining how to do Scrum, which leaves you in no doubt that the author has actually been a Scrum Master himself.

So naturally, when I heard that Kniberg was working at Spotify and was experimenting with some ideas to scale Agile beyond the team level, I was very interested in hearing what he had to say.

(Image courtesy of Spotify)

Squad

The squad is the central unit of Spotify’s scaled Agile approach. Squads are multi-functional, self-organising teams that focus on a particular aspect product.

Tribes

Tribes are collections of squads which share a focus on a certain product area. They meet regularly to showcase what each of the squads within the tribe is doing – both product work and the result of hack days (see below).

Chapters

Chapters are groups which are formed across squads made up of people with similar skillsets (e.g. web developers, testers, designers, UI developers). Each chapter has a senior member who, as well as being a member of one of the squads, acts as line manager to all the other members of that chapter.

Guilds

Guilds may have membership across more than one tribe and are not necessarily restricted to one subject area. Examples of guilds given by Kniberg are web guilds – which may include web developers, designers and testers and an Agile coaches guild.

The non-incidentals

Squads, tribes, chapters and guilds might be the most obvious features of the Spotify approach to scaling Agile. But some of the others aspects of the scaled Agile set up – which might seem incidental – may well be the most important.

Tribes sit together and have good facilities

In Spotify, squads all sit together, and wherever possible, squads in the same tribe are together in the same building. The squads have control over their surroundings. They have kitchens and breakout rooms and most of the walls have whiteboards on them.

Hack days

Each squad takes one day in ten as a “hack day”. The squads themselves can decide whether they take these singly or group them together to have a whole week. The purpose of this time is to give members of the squads the opportunity to learn about new technologies and communicate them to other members of the squad. It is also an opportunity for innovation.

The release team supports the Squads

Rather than having a separate release team which handles the release of software, the goal of the release team is to allow the development teams to release their own software.

What about architects?

The architecture of Spotify is service-oriented, with over a hundred systems making up the whole site. Each service has a system owner, and the most important systems have a developer/devops pair as system owner.

The System Owner is not a bottleneck or ivory tower architect. He does not personally have to make all decisions, or write all code, or do all releases. He is typically a squad member or chapter lead who has other day-to-day responsibilities in addition to the system ownership.

What about product owners?

There is a product owner associated with each squad. As Kniberg says:

“The PO is the ‘entrepreneur’ or ‘product champion’, focusing on delivering a great product.”

Although this isn’t discussed explicitly in the articles on Spotify that I’ve read, it can be imagined that the product owners themselves would be organised into a chapter – with their own chapter lead.

What is your experience in scaling Agile? Leave a reply below, or contact me by email.

How’s our driving? Software development using Empirical Process Control

Empirical [Adjective]

  1. Pertaining to or based on experience.
  2. Pertaining to, derived from, or testable by observations made using the physical senses or using instruments which extend the senses.
  3. (philosophy of science) Verifiable by means of scientific experimentation.

(definition taken from en.wiktionary.org)

In engineering there are two kinds of process control. Defined process control deals with processes which are repeatable and fully understood in detail. Empirical process control deals with processes which are either not fully understood or are naturally very variable.

One of the key insights of an Agile approach to software development is that building software is complex and unpredictable business and so it is best handled using empirical process control.

So what is Empirical Process Control?

To achieve empirical process control you need three things: transparency, inspection and adaptation.

Transparency

In order to be able to control a complex process, you need transparency – you need to be able to see what’s going on.

Driving a car is a complex control process. When you’re driving you need to be able to see through the windscreen.

Inspection

When you’re controlling a complex process, like driving a car, it’s not enough to be able see what’s going on.  You have to actually look.  You can have a nice clear windscreen but if you’re not looking out of it, if you’re not paying attention to what’s going on, then you’re still not going to have a very effective control mechanism in your car.

Adaptation

You need to able to see out of the windscreen (transparency), you need to actually look out of the windscreen, but then you also need to modify what you do in relation to what you see.  See an obstacle in front? Slow down.  See an empty road? Speed up.  See a sharp bend? Slow down and turn the steering wheel.

When you talk about driving in this way, it’s obvious how the pillars of empirical process control – transparency, inspection and adaptation – apply. But how are they achieved in software development?

Let’s take a look at how empirical process control is realised in one of the most popular Agile frameworks: Scrum.

Transparency and inspection: stand-ups, showcases and retrospectives

The basis of transparency in Scrum is the ‘stand-up’. This is short meeting of the team every day. People involved in the project say what they’re going to do today and what they did yesterday. They also highlight any problems which are preventing them from doing what they intend to do.

In Scrum there is a role of product owner.  The product owner acts as a single representative for the organisation that wants the software being developed by the development team. At the end of each sprint (a short, fixed period of time for which work is planned, typically two weeks) there’s a meeting called the ‘showcase’ entirely for the benefit of the product owner. In the showcase the team demonstrates working software that its developed in the recent sprint. The showcase makes transparent to the product owner, how the team is progressing.

At the end of each sprint the team runs a ‘retrospective’ where they ask themselves, what went well, what didn’t go so well. They come up with ideas for things they can try to improve the team’s performance. In terms of empirical process, the team is inspecting their performance and another way to think about this is that at the end of each sprint, the team members asks themselves and their passenger – “how’s our driving?”

Adaptation: making changes in response to what you see

Planning in Scrum happens not just once, but frequently: at the beginning of each sprint. This means the product owner can adapt what the team is going to do having looked at working software. If the software is right, the team can carry on going in that direction. If there’s a problem, the product owner can modify what the team should do next. By running retrospectives and acting on their findings, the team is also adapting.

What are your thoughts? Leave a reply below or contact me by email.

There’s no business like #snow business – the story of Project: Barry

This is the tale of a bunch of graduates, their first forays into projects and the trials and tribulations found within. This is the first of two blog posts – which can, and will be, officially regarded as a saga – so keep an eye out!

Outline

“Barry” is an application developed to track snow using Twitter, alerting users through phone notifications and mapping the information worldwide in real time. Barry has been used to track weather movements and is accurate when compared to traditional reports.

And some stats about the project:

  • Five graduates from Sopra Steria’s October 2015 intake
  • Three Java, plus GIS and Project Management streams
  • Seven weeks to compile the project as a side task
  • Two presentation dates with open invites to everyone in our Edinburgh office

Story Time

Outlook optimistic: mid October, end of week two with the company

Post-induction, post-presentations and post-welcomes, the Java graduates were looking for something with a little more bite. During one of our first meetings with our stream lead, it was tentatively proposed that we create a programme to grab information from Twitter and send a notification to your phone. This would become “Project: Barry”.

Naturally, the Java grads were keen for the opportunity to put our book knowledge to the test and stick a figurative toe into the sea of development. We decided to follow the general idea and the topic of snow was chosen; the temporary name of “SnowStalkers” was toyed with and we began putting our heads together.

The notification system came first, starting with the software Pushbullet, which is used for pushing notifications between devices. We developed a cheap and cheerful prototype and with that in place, we set our sights a little higher.

Clouds building: start of November, one week of work on project

We decided to open the doors of the project to other streams and in a quick series of conversations, we simultaneously increased and slimmed down our workload. We brought in a GIS graduate (Geographic Information Systems – it’s OK, I had to ask too), to expand into mapping the data we were gathering. Alongside this we picked up a Project Management graduate (yes we have those, and yes it’s viable), to whip us into shape and bring more structure to our project.

This was a big step towards making this idea into a serious project, as it was originally Java only – bringing in others allowed them to get more experience and working with other knowledge bases only improves your own learning. This is when Project Barry got its name; with a slip of the tongue, our GIS grad Brian was dubbed ‘Barry’ for the day and, as they say, the rest is history. We began structured meetings with agendas and began putting together our own scope – putting down features we must, should and wanted to have implemented.

First flakes: early November, under two weeks since the project expansion

Twitter integration working smoothly, mapping prototype running, and notifications flying – it was time for the first presentation, six weeks since starting the job. At the time Barry (v0.0) would grab one tweet based on snow and send a notification to the group. The presentation was to a few members of the Java team at Edinburgh, with all major points covered in under seven minutes (Pecha Kucha style).

With the first presentation completed, we made the change to Agile Sprints. We put together a trimmed feature list which at the time included:

  • Automation (continuously running without human input)
  • Twitter streaming API (similar to automation, but for Twitter)
  • Mapping (do you have to ask?)
  • Web crawling (grabbing information from websites linked in tweets)
  • Graphical User Interface (an interface to enter data)
  • Notification buffering (collecting tweets to send fewer notifications)

The aim was to implement one feature per week – taking us up to our apparent week 12 (since starting with the company) deadline with a week to spare. Java graduates brought automation and Twitter streaming into fruition soon after the presentation – Barry was continually running, pulling down tweets in real time and sending (far too many) notifications.

Next time on Project: Barry…

Read about the fate of Barry – its actionromance improvements, the twist in the tale and lessons learnt.


 

This is just one example of the innovative projects Sopra Steria graduates get involved with. If you are, or someone you know is, graduating in 2016 and looking for exciting opportunities, why not take a look at this year’s Graduate Recruitment programme.

Four reasons why Scrum is so popular

A 2013 survey by the company Version 1 showed that over 50 percent of all software development projects which are being managed using an Agile method are using Scrum. A further 15 percent or so are using either a “Scrum Hybrid” or “Scrumban” (a Scrum Hybrid).  This means that around two thirds (65 percent) of all the Agile projects that were taking place in 2013 were using Scrum. In the 2014 survey, this figure had crept up to 72 percent.

Why is Scrum so popular? Many of the other methods (DSMD, XP, Kanban) seem appealing and strong claims can be found for their success.  So why, in reality, is one method dominating? Here’s why I think Scrum is so popular.

Reason #1 – Scrum has a role dedicated to making Scrum work

The main purpose of the ScrumMaster is to facilitate the Scrum process. Actually, the ScrumMaster has two main purposes, the second being to remove impediments to progress that are identified by the team.  But the best way to identify those impediments is to facilitate the Scrum process.

Reason #2 – Scrum tells you what to do

Scrum isn’t an analysis, Scrum is a practice, or rather a set of practices. You too can start doing Scrum right now.

The most popular practice is the “daily Scrum” – or “stand-up”. This is a short meeting where everyone in the team says what they did yesterday, what they’re doing today, and, crucially, what’s stopping them – their impediments. This is one of the main ways the ScrumMaster identifies impediments. And then it’s his or her job to get them removed.

It really is that easy to get started, but to do Scrum more completely, the team needs to plan work in short, fixed periods of time (referred to as “Sprints”). They reflect at the end of each Sprint, how well they did, and what they would change (in a meeting called a ‘retrospective’). Finally, the team demonstrates working software to a representative of the customer.

Reason #3 – Scrum explicitly involves the customer

In Scrum the interests of the customer are represented by the Product Owner (PO). The PO agrees with the team at the beginning of each sprint which work should be done in that sprint. At the end of the sprint, the PO reviews the resulting working software and provides feedback. If things have changed since the beginning of the last sprint (priorities, understanding, technology, politics…) these can be incorporated into what’s planned for the next sprint.

Reason #4 – That’s it

There are lots of other techniques that you can use with Scrum – user stories, continuous integration, burn-down charts, burn-up charts, estimation of tasks in hours, complexity points, etc. But NONE of these are required.

Simplicity and clarity make it easy for teams to get started using Scrum. And, in my opinion, that’s why it’s the most popular Agile method, by far.

Mastering Scrum (rather than calling yourself a ScrumMaster) is a much more difficult task, but the only way to get there is to have that first stand-up.

What is your experience in Mastering Scrum? Leave a reply below, or contact me by email.

 

 

Five things I learned by teaching a training course “from the back of the room”

Our Agile Foundation Certificate training course is delivered using a “back of the room” approach. Almost all teaching points are taught using activities, discussions and self-authored presentations. We don’t use Powerpoint, we don’t use a screen projector. We do use internet searches, two-minute talks, Lego, balloons (with faces on them), Jenga blocks and origami.

Here’s a few things I’ve learned from delivering the course a dozen times:

  1. BOTR (“back of the room”) really works. Lugging a suitcase of Lego and Jenga up the stairs on the Tube I sometimes think it would be better to just talk to a slide deck. But the feedback that we get for our course is almost entirely positive. People seem to like it. And our pass rate is currently ninety-five percent, so they also remember and understand what they’ve learned.
  2. Showing is better than telling (especially for skeptics). We explore Agile techniques using physical activities and we encourage our delegates to research various Agile methodologies online and come back, not only with praise, but also with criticism (‘one star’ Amazon reviews of a method’s textbook are often a good place to look). This approach seems to work particularly well with people who are skeptical or unconvinced about whether an Agile approach will work for them, or for their project, or ever.
  3. People want to hear about real Agile projects. When we’re not delivering Agile training, our “day job” is working on Agile projects, maybe as ScrumMasters, maybe as Agile coaches. People really want to hear about this real-world experience. It’s a distinct advantage of using Agile practitioners to deliver the course.
  4. People take games very seriously. Even though the tasks that we set people in the training aren’t really important, people get totally absorbed. And that’s a good thing. We can use games in the training room to show important ideas in a safe (sometimes called a “safe-to-fail” environment). But it also means that if you’ve got any complex points to communicate – you should do them before you get out the Lego.
  5. Once you’ve tried BOTR – it’s hard to imagine doing it any other way. We’re now looking to develop other training courses, workshops and executive briefings using a back of the room approach. We know Agile and iterative approaches work. So when we do develop something new, we try it out at every step with real users, incorporating their feedback as we go.

If you’d like to be involved in that process – or you’d like to know more about attending our “Agile Foundation Course” please get in touch.

What IT service management can learn from CrossFit

Six months ago, when my children told me I was “great to bounce on”, I decided to sign up at the local CrossFit gym. I’d heard from friends that it was a good way to get in shape fast. CrossFit has a reputation for being somewhat tribal, with participants enthusiastically cheering, back-slapping and wearing garishly coloured clothes, so it was with some trepidation that I signed up for the induction to my local ‘box’ and set myself on a course to…

Forge. Elite. Fitness.

Two days later, it hit me hard. I couldn’t bend down far enough to put my socks on. Even brushing my teeth hurt! My head was full of acronyms such as WOD, AMRAP and EMOM and strange expressions such as “I totally RX’d Linda today, but I know I can lift more on the clean.” In short, I was a confused and broken mess. But something inside me clicked. I liked it and strangely, I embraced the pain. I was hooked.

I gave myself a week to recover and then bit the bullet and stepped back into the fray, partly because of what it could help me achieve over the longer term. I knew results wouldn’t come quickly, but with perseverance I was sure I could reach my objectives.

As weeks have turned to months, I’m still going, getting fitter, faster and starting to become a little less bouncy around the midriff. It still hurts, but I push past this and keep going because the will to succeed outweighs my desire to be a children’s trampoline.

For me, IT Service Management sometimes feels the same.

Bonding through adversity

It can get pretty intense in the box. Everybody is competing against themselves and the pain is very real even though the barriers to success are often more psychological than physical. A strong spirit of gym camaraderie is essential in getting the best out of everyone. Encouragement and praise is plentiful and audible; this helps maintain commitment and energy levels even when the going gets really tough.

It’s the same in an operations team that works in a fast paced, high pressure environment; a strong team spirit is essential in maintaining throughput of work and ensuring the team does not burn out. Give support to your colleagues, whether it’s a pat on the back, a well-timed joke or a few beers, it can make the difference between a team coping and falling apart. Get to know one another, trust each other, and enjoy working together.

Speaking a foreign language

“What on earth is an AMRAP?! You jerked what?! Your favourite Girl is Nicole?!” Such are the musings of a CrossFit newbie, as the acronyms and bizarre phraseology make CrossFit initially impenetrable.

IT Service Management (ITSM) can be just as bad. To the non-initiated, the fact that an issue is not the same as an incident which is not the same as a problem can be something of a conundrum. ITSM has an abundance of acronyms, abbreviations, and terminology that means nothing to your average developer, let alone the Chief Finance Officer (when you are trying to explain why you need £10K for that critical but under-performing SAN array).

I don’t think that’s a bad thing. Sometimes it helps to feel you belong to a private club that shares the same goal and interest. Having a common language helps us talk in a consistent and professional manner.

Scaling enables continuous output

At its most basic, scaling is the use of tools or alternative techniques to make CrossFit exercises easier and is particularly useful when you are starting out. Many routines (such as pull-ups) are beyond the reach of most mortals, so anything that makes the job a little bit easier is a godsend. It’s not cheating; it simply allows you to progress at your own pace and, most importantly, maintain it.

Going lighter isn’t the enemy to CrossFit. Stopping mid-workout is.

This is equally true of ITSM, where the ability to consistently deliver and maintain throughput is key to successful operations. Overburden a vital function and you might as well rip up that forward schedule of change.

We need to scale our operations. The use of tools to automate processes can pay massive dividends through the release of resource. If a process is slowing you down, make it more efficient. This doesn’t have to involve full blown Lean-ITSM. The removal of just one redundant step could mean the difference between getting that change deployed on time and missing the deadline. Consistent throughput of work though the removal of constraints is the key here.

Again, but faster

CrossFit is about getting stronger, faster, leaner and tougher. Athletes achieve this by performing an activity, analysing their performance, and then seeking to improve on this the next time round. CrossFit continuously encourages you to try to go that little bit harder, push that little bit more and find that last little pocket of energy you didn’t even realise you had.

ITSM should be like this. We face criticism for being overly bureaucratic, cumbersome, expensive or ineffective. These criticisms are not without merit and, like my ever-expanding waistline before CrossFit, are often the result of apathy and a reluctance to look introspectively for the source of the problem.

Continuous improvement processes should strive to first understand the baseline. Once this has been established, they must analyse the system to find out what can be added, removed, refined or improved to accelerate the flow of work. Then rinse and repeat. The team should always be challenging itself to go faster and improve performance but not at the expense of the overall machine. The value chain is only as efficient as its weakest link, so focus on the least efficient components first and make sure that when you cut out the fat, you are not losing some of the muscle with it.

“Having arms like Arnie won’t help you run any faster!”

Shout it out

“There’s only one rule of CrossFit and that’s that you never, ever, stop talking about CrossFit!”

CrossFit people can get a little obsessed and it does have a tendency to border on the intolerable for non-CrossFitters.

In my career I’ve come across a lot of ITIL* evangelists; hardcore supporters of the cause, unwilling to accept that there may be other ways of working or that, just perhaps, full blown change control isn’t required to reboot that printer in the lobby. But ITIL and ITSM is something to get passionate about. When I first found ITIL it was nothing short of a revelation. Here was an approach that allowed IT organisations to shine, add value and actively demonstrate, with meaningful metrics, how and why they were doing a good job and making a difference.

ITIL has come a long way since then, and now has global support in industry, but we shouldn’t be devout followers, deaf to all criticism. ITSM practices need to adapt if they are to keep pace with cloud, digital, and similar emerging technologies.

We have built ITIL and helped to promote the ITSM movement that has radically transformed the way that businesses have operated IT and it’s just as relevant now as it ever was, if we keep it fresh.

That is some achievement and, just like CrossFit, I think that is worth shouting about.

What do you think? Leave a reply below, or contact me by email.

(* ITIL – formerly known as the Information Technology Infrastructure Library)