Sharing personal data – challenges and solutions

Some interesting recent strategy work on data sharing between Police, NHS agencies and schools has reminded me of the challenges that face any attempt to achieve significant sharing of personal data in the public sector.  By significant I mean anything involving multiple organisations, sensitive data, large data volumes and different content (or at least two of these factors!).

Our recent work was in response to a forthcoming legislative requirement for professionals in the Police and NHS to send child well-being concerns to teachers in state schools.  This is just the initial urgent requirement, later sharing could involve other agencies (e.g. Social Services, Fire and Rescue and the Third Sector), additional content (e.g. shared plans) and the outward sharing of collated data from schools to all of the above partners.  All the familiar challenges present themselves – for example, lack of a single secure communications infrastructure, the source data is in a large number of diverse systems, there is no standard for a well-being concern, governance arrangements are complex, there are many stakeholders, etc, etc.  Not to mention, very tight timescales and high expectations!

This work has led me to attempt to summarise the key factors that influence a data sharing solution, and the main ways that sharing can be achieved.  Understanding all the key factors that need to be considered and the technical options available can help avoid reinvention of the wheel.

I have also drawn on a previous study we undertook of all the main NHS and social care data sharing solutions in Scotland, ranging from the Orkneys (population about 20,000) to NHS Greater Glasgow and Clyde (serving 1.2 million people, and covering six complete local authorities and parts of two others).  This study revealed the diversity of requirements, constraints and solutions.

For example the factors can include:

  • What is the distribution mode?
  • Will data be pushed or pulled?
  • What is being shared?
  • What is the urgency?
  • How will data be matched?
  • What’s the transport mechanism?

And several others …

Although there’s a risk of over simplification I’ve found it helpful to categorise the solutions into five architectural models:

  • Single shared system
  • Stand-alone central store
  • Integrated central store
  • Data portal
  • Central messaging hub

It should be stressed that there are overlaps between the models, and one type of solution can evolve into another.

These solutions range in complexity from the simple (for example, a single shared system), to the very complex (for example, a sophisticated multi-hub messaging model with routing and protocol intelligence built into the hubs, linking to a variety of local data sharing solutions).

From this…


To this…


I have focused on the requirements that need a technical solution, and the forms that these solutions can take.  However it’s important to remember that any data sharing solution also needs to consider equally important factors such as the governance, security and benefits.

It’s a large and complicated subject for a blog, so if you are interested in a little more detail, have a look at my paper on data matching and routing.