Digital at scale: how digital can transform business

If you spend time at pretty much any tech company, from startups to big corporates, you’re likely to hear the word ‘digital’ a bit too much.  Some people are doing it, some are making their journey towards being more digital and others are still struggling to define what exactly it is, and in many ways, it’s that final category that have the most honest answer to the question – What is digital?  And this is what experts from the technology and financial services industry discussed during a recent seminar at London Technology Week.

It’s easy to define digital as being about technologies – that digital is at its core the binary ‘0’s and ‘1’s, on and off and all the brilliant devices and interfaces that have spawned out of it.  While that’s not entirely wrong, it paints a picture that everything digital is very clean cut, with a definite right and wrong answer that follows any question – but the truth is very different.  The technologies are far from a constant, and everything from the technology chosen to the implementation will change not only for different demographics but from person to person, and will adapt to their current situation, desires, needs and moods.  Technology then, is transient, and to be truly digital you must be open to constant and relentless change, throwing away technology, processes and ways of working constantly, and ensuring that the new tool adopted is chosen intelligently, to be the best tool for the job, and the most commercially viable solution.

This however all sounds like the territory of startup businesses.  Businesses that are new to the scene, or with very flexible business models are often far more adept to change as they do not have the long-standing commitments to clients, legacy platforms and some of the regulatory requirements of their big corporate counterparts.  Some may suggest that these big corporates should simply throw away the legacy platforms, circumvent the regulation and transform their clients, and noble though that may be, it’s a fool’s errand.  For these businesses, what they really need is to find a way to take advantage of new technology, whatever that may be, and develop systems that allow them to adapt to change which work alongside and complement their legacy ‘technological debt’ and support their regulatory requirements rather than dispose of them. This is digital at scale.

Put simply, digital at scale explores how businesses can leverage digital, be it technology, ways of working or any other idea that comes under the umbrella of digital to transform their business, supporting existing technologies, commitments and regulation where appropriate, and disposing of them where necessary.

Sopra Steria’s MiFID 2 project with the FCA is an example of where digital at scale has been implemented. For all the businesses that are wary of how technologies like cloud and open source could work in a highly regulated environment, there’s no better example than that of the regulator itself adopting these technologies.  The MiFID 2 regulatory support service is built for the cloud, ingesting, processing and persisting files on AWS, with innovative open source platforms like Cassandra and Spark ensuring that all submissions are processed quickly and with an extremely high degree of accuracy, with an architecture that supports changes should a specific client or geography require, like private vs public cloud or separate technology components.  What is particularly profound about this solution though is how it backs into and supports the legacy environment, through a simple FTP gateway, ensuring that the wealth of historical data is utilized and, as is so important in an environment like this, remembered with a system that can speak both the languages of the old and the new into the future, maintaining a stream of communication regardless of changes made on either end.

The MiFID 2 platform is only one example of these principles put to work, and though the distant future might see us living in a fully digital world we must be conscious today that whether we transition fast or slowly, we must do so safely too, and with a strong commercial focus to build not simply small digital players, but truly successful enterprises with digital at scale.

Find out more about our FCA Market Data Processing project and Sopra Steria’s #intelligentdigital campaign.

The Testing Pyramid

Agile has totally changed the focus of testing and introduced automation to support its development practices. I have been involved with software development teams for nearly thirty years. Over that time I have seen many different methods and practices come and go but testing has remained focused around manual testing. That is until Agile software development arrived.

An Agile iterative approach to software development means you have to test all your software all the time. Ensuring what you have just built has not broken a previous written piece of functionality. Agile automates testing at all layers of the application. This approach to testing is fast overtaking the traditional manual approach. Automated tests that previously existed were focused on testing the front end, the most volatile part of the application. They were also written after the application was built therefore not giving the short feedback loop we use in Agile software development. The Testing Pyramid puts a different testing emphasis on each of the application layers and focuses efforts at the beginning of the development cycle rather than at the end.

The Testing Pyramid

Looking around the web you will see various implementations of the Testing Pyramid with different names for the levels and implementing different types of tests. The one I use is the basic three level pyramid – Unit Tests, Service Tests and UI Tests. As with any pyramid, each level builds on the solidity of the level below it.

From technical view point one can look at these three levels as small, medium and large. Small tests are discreet unit tests that use mocks and stubs to collaborate with other objects and are typically written in one of the xUnit frameworks. Service level tests are the medium sized tests and interface with one other system, typically a database or a data bus. Large tests, UI tests collaborate with many sub-system parts they support the end-to-end scenarios.

Personally I look at the levels in a functional way:

  • UI Tests: Does the software work as expected by the end user?
  • Service Tests: Does the software meet the acceptance criteria on the user story?
  • Unit Tests: As a developer does the software work as I expect?

Unit Tests

Unit tests are the foundation of the Testing Pyramid. This is where the bulk of tests are written using one of the xUnit frameworks – for example, JUnit when developing in Java. They ask the question “Are we building the product right?”. When writing software I like to take a Test Driven Development (TDD) approach. TDD is a design approach to development where tests are written first and code written to support the test. There are a number of benefits to taking this approach:

  • High test coverage of the written code
  • Encourages the use of Object Orientated Analysis and Design (OOAD) techniques
  • Allows you to move forward with confidence knowing the functionality works
  • Debugging is minimised because a test takes you straight to the problem
  • Developers take more responsibility for the quality of their code
  • Because it is written to support specific tests, the code works by design not by coincidence or accident

Service Tests

I see Service Tests as supporting the acceptance criteria on the user story. They ask the question “Are we building the right product?”. When writing these tests, I like to take advantage of the Given/When/Then BDD format of the acceptance criteria and use one of the BDD test frameworks, typically Cucumber. I only adopt specification by example that is use real world examples in the acceptance criteria. This approach gives a number of benefits;

  • Assurance that all stakeholders and delivery team members understand what needs to be delivered through greater collaboration
  • The avoidance of re-work through precise specifications and therefore higher productivity

UI Tests

The User Interface is the most volatile part of the application and should not contain any business logic. For these reasons the least emphasis on automated testing should be here. That does not mean there is no testing, I like to automate key user journeys through the system using one of the UI testing frameworks e.g. WebDriver. UI testing demonstrates that all subsystems are talking to each other.

I use manual testing for the look and feel, checking the UI acts as expected and exploratory testing to find those hidden nuances.

Test Delivery

Unit tests and Service tests should be delivered in the iteration by the delivery team. As part of their development practices developers write Unit tests when building the functional code. Ideally a test first, TDD approach should be used.

My conclusion?

The Testing Pyramid inverts the traditional approach to testing. The focus is at the beginning of the development process with the developer taking responsibility for quality of their code. This is a very different way at looking at the problem from the traditional approach where code is handed over to the tester and they are assumed responsible for the code.

The early identification of defects gives two major business benefits;

  • Issues are discovered early in the development process reducing the cost of defect fixing associated with late discovery
  • Issues are identified and resolved early therefore negating the need to postpone the production release through late identification of issues

References

http://www.mountaingoatsoftware.com/blog/the-forgotten-layer-of-the-test-automation-pyramid

http://martinfowler.com/bliki/TestPyramid.html

http://www.agilecoachjournal.com/index.php/2014-01-28/testing-2/the-agile-testing-pyramid/

Modern Front-end Development Workflow

Over the past couple of decades, the internet as evolved beyond just a tool for search information on desktop computers at work, colleges and universities to a tool that is now integral to our everyday life. At its birth, the possibility of having the internet on mobile devices and watches would have been viewed as science fiction; this is now reality and the future is even more exciting.

At the centre of the continual growth of the internet are ever evolving protocols, standards and technologies. HTML, CSS and JavaScript in all their flavours are still core to the delivery of the internet, working together to present a familiar interface to the user. To a typical non-technical user, a website page is the internet so developing modern practices are essential to the growth of the internet.

Modern front-end development workflow covers the practices being utilised by developers today. It allows the developer to build User Interface (UI) based on the industry standards, with reduce time for delivery, high performance, reusable and easy to maintain code.

A typical front-end developer stack consists of HTML, CSS and JavaScript. Modern front-end development workflow enforces best practises, automation and consistency between build applications.

The workflow for application development can be categorised into six phases

  1. Scaffold is the setup of a web project – the files structure, download libraries and dependencies
  2. Develop is the phase where the application is actually coded. Code review among developers is good practice at this phase
  3. Test is where the developed code is tested. This phase will help identify bugs and fixes. It is common to bounce between the Test and Develop phases until all bugs are identified and fixed
  4. Integrate – at this phase the unit code is integrated with the rest of the application
  5. Optimise – at this phase, the code is debugged, integrated and ready for deployment to the production environment, additional tests are carried out to ensure the application performance is optimised, for example page load timing and object caching
  6. Deploy is where the code is pushed onto the production environment so it is available to be viewed by the users of the application

Each of these phases can be automated to improve consistency and efficiency. Suggestions on tools and practises to help automate these phases are described below –

  1. Automating Scaffold – There are a number of tools available to accomplish this. Dependency managers like NPM and Bower offer features that will help automate scaffolding. The main difference between both is that NPM is used for installing NodeJS modules but Bower is used for managing front-end components like HTML, CSS, JS etc
  2. Automating Development – Tools like Grunt and Gulp will help achieve automation in this phase. There is a command line dependency when using this tool
  3. Automating Testing –3 types of testing can be automated:
    – Code format tests. The code formatting is checked to see if they follow industry best practices.  We can use tools like JShint or SCSSLint
    – Unit tests. The code is checked against tests to make sure it works correctly and that new code doesn’t break the functionality of other parts of the site. These tests are usually JavaScript-oriented. Tools like Jasmine can be used. (Jasmine is a behaviour-driven development (BDD) framework for testing JavaScript.)
     – CSS Regression testing. The output of the browser is checked to see whether it breaks part of the page that was previously coded. It allows us to make visual comparisons between the correct (baseline) versions of our site, versions in development or just about to be deployed (new). Tools like PhantomCSS can help with this type of test
  4. Automating Integration – A version control tool like Git can help accomplish integration automation. It allows developers to push, pull and merge code from different sources into a single repository (repo). It is also possible to enforce standard before a code merge to ensure that the merged code conforms to set standards. To do this, continuous integration (CI) can be setup to automate testing after codebase integration
  5. Automating Optimization – The automation here is about getting the code ready for the production environment and ensuring the code is optimised for best performance. This may involve processes like minifying and concatenating CSS and JavaScript, optimizing images and SVGs. Build tools like Grunt or Gulp can be used in to automate the optimize phase
  6. Automating Deployment – The final phase is pushing the optimised code into the production environment to be viewed by the public. To achieve this, a post-receive hook may be created. This will make the server automatically pull new changes whenever there is an update.

In summary, modern front-end development workflows can be a very effective tools in the successful development of web applications if implemented and utilised properly in the development lifecycle. The tools described above are not the only ones available on the market to accomplish automation. Different companies and individual developers will have their preferences to suit the way they work. To fully take advantage of the benefits of modern front-end development workflows, it is essential that organisations invest in training their developers. If the developer’s skill set is kept up to date, the organisation will see a direct improvement in its productivity and the quality of the code.

What are your experiences in this area? Contact me by email or leave a reply below.

The #snow must go on – the story of Project: Barry

Recap

This is the tale of a bunch of graduates, their first forays into projects and the trials and tribulations found within. “Project: Barry” is a Graduate project created to map snow worldwide using data pulled from Twitter, and this is the end of it’s saga – catch up on the full story here.

When it snows it pours: late November, two weeks since presentation

During this time, GIS moved to complete the mapping section of the project. The server would comprehend the information being sent to it by Barry, store it in a database and then map it in real time. The viewpoint of the map would be updated to the location of the most recent tweet, creating an interactive experience.

Next we put together a buffer for notifications, making them more useful. This allowed us to stack up tweets over some time, then send one notification with a number of tweets and a selection of locations involved.

Things were moving smoothly and ahead of schedule when we received word that the final presentation would be on the upcoming Friday – not in several weeks like the original deadline.

Whiteout: beginning of December – presentation time

Due to several issues, the date had changed but thanks to the Agile methods, we had a working program ready to be put into production at the end of each week. This allowed us to simply cancel later features (sorry GUI and web crawling!) and tidy up.

This iteration of Barry (v0.1) was at the point of being run on a server, sending information to the database in real time to be mapped and sending buffered notifications to the group – and this was what was presented in the beginning of December. The presentation was open to the entire Edinburgh office and saw a full room ready to interact with Barry and the mapping system in real time.

Participants were able to tweet using the word ‘snow’ and have their name, (embarrassing) display picture and location appear on screen as the map bounced around showing the locations of tweets from around the world. See the example below for a first hand look at Barry in its natural habitat! (You may need to click on the image to start its animation)

Animated image showing snow-spots around the globe on a map of the world

Outcomes

It’s safe to say each Barrista (demonym for those of Project: Barry) was glad to have been a part of the project. Gaining project experience alongside book learning was a huge addition to being part of Sopra Steria’s Graduate programme. Working alongside other streams really makes you think for others and seriously improves your knowledge.

Expanding and developing the project has proven to be incredibly fruitful, both in terms of improving our standing within the company (recognised as being “Those guys who made that Twitter thing”!) and increasing our effectiveness when stepping into the Real World™. Experiencing real deadlines, shifts in scope and late notice changes really gave us a chance to prove our worth and flex our skills.

After the thaw

After the end of the project, the Barristas have since moved to other projects – but the legacy of Barry will live on…

Barry’s real time mapping component is to be utilised to provide visual representation of activity on a potentially company wide basis… so stay tuned!


This is just one example of the innovative projects Sopra Steria graduates get involved with. If you are, or someone you know is, graduating in 2016 and looking for exciting opportunities, why not take a look at this year’s Graduate Recruitment programme.

Scaling Agile the Spotify way

The first book that I recommend anyone read about Agile is “Scrum and XP from the Trenches” by Henrik Kniberg.

Kniberg has a relaxed, informal and straight-forward way of explaining how to do Scrum, which leaves you in no doubt that the author has actually been a Scrum Master himself.

So naturally, when I heard that Kniberg was working at Spotify and was experimenting with some ideas to scale Agile beyond the team level, I was very interested in hearing what he had to say.

(Image courtesy of Spotify)

Squad

The squad is the central unit of Spotify’s scaled Agile approach. Squads are multi-functional, self-organising teams that focus on a particular aspect product.

Tribes

Tribes are collections of squads which share a focus on a certain product area. They meet regularly to showcase what each of the squads within the tribe is doing – both product work and the result of hack days (see below).

Chapters

Chapters are groups which are formed across squads made up of people with similar skillsets (e.g. web developers, testers, designers, UI developers). Each chapter has a senior member who, as well as being a member of one of the squads, acts as line manager to all the other members of that chapter.

Guilds

Guilds may have membership across more than one tribe and are not necessarily restricted to one subject area. Examples of guilds given by Kniberg are web guilds – which may include web developers, designers and testers and an Agile coaches guild.

The non-incidentals

Squads, tribes, chapters and guilds might be the most obvious features of the Spotify approach to scaling Agile. But some of the others aspects of the scaled Agile set up – which might seem incidental – may well be the most important.

Tribes sit together and have good facilities

In Spotify, squads all sit together, and wherever possible, squads in the same tribe are together in the same building. The squads have control over their surroundings. They have kitchens and breakout rooms and most of the walls have whiteboards on them.

Hack days

Each squad takes one day in ten as a “hack day”. The squads themselves can decide whether they take these singly or group them together to have a whole week. The purpose of this time is to give members of the squads the opportunity to learn about new technologies and communicate them to other members of the squad. It is also an opportunity for innovation.

The release team supports the Squads

Rather than having a separate release team which handles the release of software, the goal of the release team is to allow the development teams to release their own software.

What about architects?

The architecture of Spotify is service-oriented, with over a hundred systems making up the whole site. Each service has a system owner, and the most important systems have a developer/devops pair as system owner.

The System Owner is not a bottleneck or ivory tower architect. He does not personally have to make all decisions, or write all code, or do all releases. He is typically a squad member or chapter lead who has other day-to-day responsibilities in addition to the system ownership.

What about product owners?

There is a product owner associated with each squad. As Kniberg says:

“The PO is the ‘entrepreneur’ or ‘product champion’, focusing on delivering a great product.”

Although this isn’t discussed explicitly in the articles on Spotify that I’ve read, it can be imagined that the product owners themselves would be organised into a chapter – with their own chapter lead.

What is your experience in scaling Agile? Leave a reply below, or contact me by email.

There’s no business like #snow business – the story of Project: Barry

This is the tale of a bunch of graduates, their first forays into projects and the trials and tribulations found within. This is the first of two blog posts – which can, and will be, officially regarded as a saga – so keep an eye out!

Outline

“Barry” is an application developed to track snow using Twitter, alerting users through phone notifications and mapping the information worldwide in real time. Barry has been used to track weather movements and is accurate when compared to traditional reports.

And some stats about the project:

  • Five graduates from Sopra Steria’s October 2015 intake
  • Three Java, plus GIS and Project Management streams
  • Seven weeks to compile the project as a side task
  • Two presentation dates with open invites to everyone in our Edinburgh office

Story Time

Outlook optimistic: mid October, end of week two with the company

Post-induction, post-presentations and post-welcomes, the Java graduates were looking for something with a little more bite. During one of our first meetings with our stream lead, it was tentatively proposed that we create a programme to grab information from Twitter and send a notification to your phone. This would become “Project: Barry”.

Naturally, the Java grads were keen for the opportunity to put our book knowledge to the test and stick a figurative toe into the sea of development. We decided to follow the general idea and the topic of snow was chosen; the temporary name of “SnowStalkers” was toyed with and we began putting our heads together.

The notification system came first, starting with the software Pushbullet, which is used for pushing notifications between devices. We developed a cheap and cheerful prototype and with that in place, we set our sights a little higher.

Clouds building: start of November, one week of work on project

We decided to open the doors of the project to other streams and in a quick series of conversations, we simultaneously increased and slimmed down our workload. We brought in a GIS graduate (Geographic Information Systems – it’s OK, I had to ask too), to expand into mapping the data we were gathering. Alongside this we picked up a Project Management graduate (yes we have those, and yes it’s viable), to whip us into shape and bring more structure to our project.

This was a big step towards making this idea into a serious project, as it was originally Java only – bringing in others allowed them to get more experience and working with other knowledge bases only improves your own learning. This is when Project Barry got its name; with a slip of the tongue, our GIS grad Brian was dubbed ‘Barry’ for the day and, as they say, the rest is history. We began structured meetings with agendas and began putting together our own scope – putting down features we must, should and wanted to have implemented.

First flakes: early November, under two weeks since the project expansion

Twitter integration working smoothly, mapping prototype running, and notifications flying – it was time for the first presentation, six weeks since starting the job. At the time Barry (v0.0) would grab one tweet based on snow and send a notification to the group. The presentation was to a few members of the Java team at Edinburgh, with all major points covered in under seven minutes (Pecha Kucha style).

With the first presentation completed, we made the change to Agile Sprints. We put together a trimmed feature list which at the time included:

  • Automation (continuously running without human input)
  • Twitter streaming API (similar to automation, but for Twitter)
  • Mapping (do you have to ask?)
  • Web crawling (grabbing information from websites linked in tweets)
  • Graphical User Interface (an interface to enter data)
  • Notification buffering (collecting tweets to send fewer notifications)

The aim was to implement one feature per week – taking us up to our apparent week 12 (since starting with the company) deadline with a week to spare. Java graduates brought automation and Twitter streaming into fruition soon after the presentation – Barry was continually running, pulling down tweets in real time and sending (far too many) notifications.

Next time on Project: Barry…

Read about the fate of Barry – its actionromance improvements, the twist in the tale and lessons learnt.


 

This is just one example of the innovative projects Sopra Steria graduates get involved with. If you are, or someone you know is, graduating in 2016 and looking for exciting opportunities, why not take a look at this year’s Graduate Recruitment programme.

Four reasons why Scrum is so popular

A 2013 survey by the company Version 1 showed that over 50 percent of all software development projects which are being managed using an Agile method are using Scrum. A further 15 percent or so are using either a “Scrum Hybrid” or “Scrumban” (a Scrum Hybrid).  This means that around two thirds (65 percent) of all the Agile projects that were taking place in 2013 were using Scrum. In the 2014 survey, this figure had crept up to 72 percent.

Why is Scrum so popular? Many of the other methods (DSMD, XP, Kanban) seem appealing and strong claims can be found for their success.  So why, in reality, is one method dominating? Here’s why I think Scrum is so popular.

Reason #1 – Scrum has a role dedicated to making Scrum work

The main purpose of the ScrumMaster is to facilitate the Scrum process. Actually, the ScrumMaster has two main purposes, the second being to remove impediments to progress that are identified by the team.  But the best way to identify those impediments is to facilitate the Scrum process.

Reason #2 – Scrum tells you what to do

Scrum isn’t an analysis, Scrum is a practice, or rather a set of practices. You too can start doing Scrum right now.

The most popular practice is the “daily Scrum” – or “stand-up”. This is a short meeting where everyone in the team says what they did yesterday, what they’re doing today, and, crucially, what’s stopping them – their impediments. This is one of the main ways the ScrumMaster identifies impediments. And then it’s his or her job to get them removed.

It really is that easy to get started, but to do Scrum more completely, the team needs to plan work in short, fixed periods of time (referred to as “Sprints”). They reflect at the end of each Sprint, how well they did, and what they would change (in a meeting called a ‘retrospective’). Finally, the team demonstrates working software to a representative of the customer.

Reason #3 – Scrum explicitly involves the customer

In Scrum the interests of the customer are represented by the Product Owner (PO). The PO agrees with the team at the beginning of each sprint which work should be done in that sprint. At the end of the sprint, the PO reviews the resulting working software and provides feedback. If things have changed since the beginning of the last sprint (priorities, understanding, technology, politics…) these can be incorporated into what’s planned for the next sprint.

Reason #4 – That’s it

There are lots of other techniques that you can use with Scrum – user stories, continuous integration, burn-down charts, burn-up charts, estimation of tasks in hours, complexity points, etc. But NONE of these are required.

Simplicity and clarity make it easy for teams to get started using Scrum. And, in my opinion, that’s why it’s the most popular Agile method, by far.

Mastering Scrum (rather than calling yourself a ScrumMaster) is a much more difficult task, but the only way to get there is to have that first stand-up.

What is your experience in Mastering Scrum? Leave a reply below, or contact me by email.