putting data to good use

This post is part of innov-hate, a brief series of thoughts and reflections on innovation in a Government context.

credit: John Baker


In the previous posts in this series I have reframed innovation as speculation, spoken about the differences between the public and private sector, the need to prove concepts in context and the importance of the right aspirations, team, use cases and processes. This post gives some examples.

The Government Transformation Strategy better use of data priorities for 2020 has making better use of data as an enabler for public services, particularly where those services cross organisational boundaries right at the top. At the very top of the Department for Work and Pensions (DWP) Single Departmental Plan is supporting people into work.

A Hub approach

In the DWP Data Science Hub I lead we are bringing data and service design together. Our focus is working across boundaries to design and test re-usable components and services that make data more accessible for users to make decisions.

We use a combination of data science, user research and design. Exploring new data sources or taking a fresh look at data already in use to see whether it can support the decisions users need to make. We know that access ≠ accessible. We seek to understand users’ contexts to design the right solutions.

Supporting people

Helping people progress into and within work is an aim that cuts across many different organisations not just DWP. In central government Department for Digital, Culture, Media and Sport (DCMS), Department for Education (DFE) and Education and Skills Funding Agency have initiatives aimed at

  • helping local areas understand their skills and training needs
  • supporting people with their learning and training

Aligned to Local Industrial Strategies, Digital Skills Partnerships, Skills Advisory Panels, National Retraining Scheme and National Careers Service. The decisions made in developing strategies, policies and services, plus those by people looking for training or work should be informed by data.

User needs

Within our hub we are in conversation and collaboration with the above organisations on the above aims. This revolves around our work with data on skills, salaries and job titles from the Adzuna job advert aggregation website.

With tens of millions of adverts and many thousands of skills and job titles this is big text data. We have used techniques from natural language processing, big data-mining, cluster-computing, machine learning and market-basket algorithms. Users don’t care about this.

Place based strategists

Our sessions with local strategists have highlighted that they aren’t time rich, they have different levels of data confidence and they need information placed in context to build a rich picture. Things like

  • What skills or jobs are on the rise in Newcastle?
  • What skills does the Birmingham labour market demand most?
  • Are skills in my area more specialised than elsewhere?

To meet these needs we are developing a growing set of modular, repeatable and reusable logic. Stand alone code components that do one thing well.

  • Drawing data from APIs straight to the heart of the work (DWP’s Stat-Xplore , ONS’s beta and more mature GeoPortal plus Wikipedia APIs).
  • Standardising data and extracting measures and counts, for breakdowns by age, date/time and more.
  • Applying standard (portions, trends, ranks) grouped (spread, skew) and more advanced metrics (such as entropy: how focussed a measure is on a particular breakdown).
  • Identifying trends and outliers as insights.

These allow us to develop an intelligence layer to see beyond raw numbers, and explore data in context (time, geography, or subject context) without losing any of the nuance. Automatically highlighting what strategists care about — initially relating to salaries, skills and job titles — and surfacing with empathy and effectiveness through interfaces co-designed with real users.

Citizens (or advisors)

DataJam North East — a leap of faith event for Celine McLoughlin and I — included a hack for which Adzuna agreed to share some data. We took in problem statements important to the region, including on skills gaps.

  • What connected skills could I learn to boost my salary?
  • What jobs fit with my skill set?
  • How is demand for my skills changing — do I need to shift my focus to stay competitive?

The work before, during and after DataJam resulted in components and a prototype which uses similar algorithms to those used by online retailers. Using the relationships between skills, job titles, and salaries in adverts we can say “employers who asked for your skills also asked for…” to recommend to people which new skills might fill gaps, and what kinds of jobs (and salaries) these new skills would open up.

Doing it right

Assembling interconnected components, cross organisational collaboration and a foundation of data to offer more than just analysis but to tailor services fits with Government Digital Service hopes and expectations for 2030.

We’ve proven this in context — real data with real users — through speculation — exploring things that are relatively untested in public services — doing something different. We’ve done this whilst aligning to government design principles

  1. Start with user needs
  2. Do less
  3. Design with data
  4. Do the hard work to make it simple
  5. Iterate. Then iterate again
  6. This is for everyone
  7. Understand context
  8. Build digital services, not websites
  9. Be consistent, not uniform
  10. Make things open: it makes things better

This work is still in the early stages and the description above probably simplifies the data, design and cultural challenges we have and will face. But by avoiding the hype culture and focusing on the future of our public services instead we feel we can achieve genuine innovation.

Data Science Hub Lead @DWPDigital. These are my personal thoughts.