Sharing personal data – challenges and solutions

Some interesting recent strategy work on data sharing between Police, NHS agencies and schools has reminded me of the challenges that face any attempt to achieve significant sharing of personal data in the public sector.  By significant I mean anything involving multiple organisations, sensitive data, large data volumes and different content (or at least two of these factors!).

Our recent work was in response to a forthcoming legislative requirement for professionals in the Police and NHS to send child well-being concerns to teachers in state schools.  This is just the initial urgent requirement, later sharing could involve other agencies (e.g. Social Services, Fire and Rescue and the Third Sector), additional content (e.g. shared plans) and the outward sharing of collated data from schools to all of the above partners.  All the familiar challenges present themselves – for example, lack of a single secure communications infrastructure, the source data is in a large number of diverse systems, there is no standard for a well-being concern, governance arrangements are complex, there are many stakeholders, etc, etc.  Not to mention, very tight timescales and high expectations!

This work has led me to attempt to summarise the key factors that influence a data sharing solution, and the main ways that sharing can be achieved.  Understanding all the key factors that need to be considered and the technical options available can help avoid reinvention of the wheel.

I have also drawn on a previous study we undertook of all the main NHS and social care data sharing solutions in Scotland, ranging from the Orkneys (population about 20,000) to NHS Greater Glasgow and Clyde (serving 1.2 million people, and covering six complete local authorities and parts of two others).  This study revealed the diversity of requirements, constraints and solutions.

For example the factors can include:

  • What is the distribution mode?
  • Will data be pushed or pulled?
  • What is being shared?
  • What is the urgency?
  • How will data be matched?
  • What’s the transport mechanism?

And several others …

Although there’s a risk of over simplification I’ve found it helpful to categorise the solutions into five architectural models:

  • Single shared system
  • Stand-alone central store
  • Integrated central store
  • Data portal
  • Central messaging hub

It should be stressed that there are overlaps between the models, and one type of solution can evolve into another.

These solutions range in complexity from the simple (for example, a single shared system), to the very complex (for example, a sophisticated multi-hub messaging model with routing and protocol intelligence built into the hubs, linking to a variety of local data sharing solutions).

From this…


To this…


I have focused on the requirements that need a technical solution, and the forms that these solutions can take.  However it’s important to remember that any data sharing solution also needs to consider equally important factors such as the governance, security and benefits.

It’s a large and complicated subject for a blog, so if you are interested in a little more detail, have a look at my paper on data matching and routing.


ISPs and data sharing governance

As I near the end of another interesting and challenging data sharing consultancy exercise I thought it would be useful to take a step back and consider how public sector organisations can overcome some of the big challenges of sharing personal data.  I am thinking mainly of data sharing in the health, social services, education and justice domains, because these are the areas that have dominated my time recently, but the thoughts are equally applicable to other parts of the public sector.

I’m starting with the governance of data sharing – which can appear as a minefield of confusing terminology, guidelines and practices.  However taken one chunk at a time, and with some specialist advice – it’s not that hard!

At the core of data sharing governance is an agreement between two or more partners on how they will manage the sharing of data.  Linked to this agreement are areas that are specific to each organisation, i.e. the organisation’s management of its data security and general information governance.

The agreement

image of a scrabble board
Figure 1: Data sharing acronym “scrabble”

The standard way to document a data sharing agreement is by means of an Information Sharing Protocol or Agreement (ISP or ISA), which documents the who, why, where, when, what and how of the sharing.

There are a number of popular ISP frameworks available, for example, see the SASPI, WASPI and ICO web sites (links are given below).  The available templates and guidance gets the ISP process off to a quick start, and help partners develop a common understanding.  A recognised template should also help ensure that the legal aspects of the ISP are properly addressed.

When producing an ISP it’s important to remember to:

  • Keep it simple. Would a front-line practitioner understands it?  An ISP needs to clearly communicate the essential elements of the data sharing to all involved people, e.g. internal employees as well as external stakeholders (or a similar test – would my husband/wife/partner understand this?  – assuming you can persuade them to read it!)
  • Keep it standard. As detailed above, using an existing template helps to reach an agreement and avoids the pain of re-inventing the wheel
  • Start the process early. An ISP should not be a last minute afterthought, and there are dependencies with the parallel design of the technical solution for sharing and storage, and the assessment of information risks
  • Manage the process. One partner organisation should co-ordinate the development of the ISP, with designated ISP Coordinators appointed as primary points of contact in every organisation involved
  • Integrate with data security. The ISP must document how security controls are applied to the data that is being shared.  This should integrate with a Privacy Impact Assessment (PIA) that identifies the risks to the data, and how they will be managed
  • Make the sign-off easy. There’s always a risk that too many stakeholders in the sharing organisations become involved in the review and agreement process, which then grinds to a halt.  Numbers at the party should be kept to a minimum, and where several partners are involved, multiple bilateral flavours of the ISP should be avoided.  Variety will not spice up your life!
  • Communicate it. Publish it widely within your organisation (e.g. on intranets), with partners (consider a shared portal/intranet) and externally (on your web site, and on a national register, e.g. WASPI)
  • Keep it up-to-date. Factors such as business practices, technologies and the data shared will change over time so there needs to be a process in place to ensure the ISP reflects these changes, and does not get lonely on a shelf
  • Keep the benefits of sharing in mind. An ISP is a means to achieve benefits for the child, vulnerable adult etc

But don’t forget…

ISPs don’t exist in a vacuum and there are key areas where they integrate with the management of data in the rest of the organisation, in particular:

  • With the wider arrangements for information governance, for example Information Assurance Committees
  • With established data management roles, for example the Caldicott Guardian in the NHS or Data Protection Officers
  • With existing policies, for example an Information Governance Strategy, Data Security Policy, etc (but avoiding duplication with the ISP)

The sharing of personal data is a sensitive topic and some citizens will always be wary despite appreciating that it can simplify the experience of dealing with multiple public service organisations and support the delivery of integrated and enhanced standards of care.

Setting up a good ISP means that public bodies can confidently share data with successful outcomes.  An ISP provides a foundation that allows organisations to focus on the twin challenges of data security and the routing and matching of data between partners.

This has just been a quick overview – theses sites give some excellent and detailed guidance, templates and examples:

Next time – the ins and outs of data matching and routing.

If you have any comments, leave a reply below or contact me by email.

Young Scot Awards 2015: celebrating young people in Scotland

Last week I was privileged to attend the 2015 Young Scot Awards in the Usher Hall in Edinburgh. The night is a celebration of the success of young people in Scotland who have made various amazing contributions to the improve the lives of people in their communities.  A suite of celebrities were involved in the hosting and presentation of the awards, including Edith Bowman, the band Prides (definitely the loudest contributors, especially from my seat), Conor Maynard, Stevia McCrorie and Pudsey the Dog (the only one I recognised …). young-scot-performingFrom our table in the front row we got the full 360 degree sound experience – music to front and screaming to the rear. All the nominees and winners were very impressive, with the overall award going to Jak Truman for his inspirational fund raising efforts before his untimely death from cancer in February 2015.

The event made me think about the importance of young people to a company like Sopra Steria. Every year we recruit a significant number of graduates into all areas of the company (104 under 24s in 2014). Working with young people challenges us all to take a fresh approach to our work. Our graduates are invariably keen, work hard, liven things up, and bring a fresh perspective to digital technologies. Some of our projects may not involve the sort of systems they imagined they would work on while at university, e.g. paying farmer’s claims, court case management solutions and prison management systems but they always adapt quickly and successfully (although without the reward of meeting Pudsey).

All our graduates start with an induction programme and then move on to work on various projects, potentially involving a range of technologies and types of clients. We make sure our graduates have more experienced people to mentor them, as well as a buddy to help them settle in. See information about our Graduate opportunities.

In a similar way the Young Scot Awards show that with a little support and encouragement young people can achieve great things and make a real difference.

Many thanks to my hosts SOLACE (the UK representative body for Local Authority Chief Executives), Young Scot for organising a very inspiring and professional event, and above all to the many fantastic young people who were nominated for, and won, the awards.

On a personal note, my 16 year old daughter is part of a Young Scot focus group and was also enjoying the show. However no thanks for the text telling me I looked bald from her seat in the Grand Circle.