How would you describe the current state of data?
Data in a lot of organisations is unwieldy, sprawling, and out of control. Data teams, particularly data engineers are struggling with ever mounting data debt. It’s a real missed opportunity, born out of a lack of investment, and focus.
The thing hurting organisations the most is a lack of communication and governance around data – data producers, data teams, and data consumers all operate in silos. It reminds me of the days when QA, design, and IT Ops were operating as segregated teams, and data needs to go through the same type of change curve to progress.
Clients know they need to do something about their data issues but there’s a real lack of understanding about where and how to get started because data problems often feel so insurmountable and nebulous.
What is Data as a Product (DAAP) and why is it important?
Where data is concerned, most organisations still have an “if we build it, they will come” mentality; however, that doesn’t really consider the needs of users and that is fundamentally what data as a product is all about. For me personally, I think it’s about applying the same level of rigour to data as you would a software product.
If you change people’s mindsets about data to be closer to the mindsets typically adopted by product teams, seeing data through a product lens, then it has many advantages.
Can you tell us some experiences of where this approach has worked well and what the outcomes have been?
In my experience, maturity with data is rare; although there are many businesses that have seen success from focussing their efforts on data, such as FAANG, Spotify, Netflix, Go Compare, and Airbnb – all stick in the mind as being data savvy, product centric businesses.
Then in the public sector, ONS’ use of and presentation of data is impressive, particularly the emphasis on user needs and good design principles; and CDDO are pushing forward with an ambitious data strategy, which I was privileged enough to be involved in, which is about promoting data sharing between government departments – adopting common data standards, tools, and technologies to facilitate interoperability, and findability of data assets through data catalogues.
How do you get an organisation to start to work in this way?
How often do you see data roles (data engineers, data analysts, or data scientists) as part of multidisciplinary teams? Not very often, is the answer, and this is symptomatic of a wider problem. In contrast, user centred design specialists are in most multidisciplinary teams.
For me, it’s more about re-thinking organisational design and delivery teams to remove silos and barriers, and to encourage closer collaborations around client problems. And I think data teams would benefit from engaging with their users in a more tenacious, systematic way.
What sort of culture is needed to embed this way of working with data?
Organisations that don’t keep data teams locked away in a silo, in the outer reaches of the organisational structure are more likely to succeed with treating data as a product. The best cultures I’ve experienced are those where people enjoy trust and autonomy, but where there is a real, relentless focus on user needs and delighting customers through the early/consistent delivery of work.
In what ways is government set up well to embrace this way of working?
Government is already well versed with thinking about user needs and design, arguably that balance has gone too far because design in of itself is not a unit of delivering value or impact, but nonetheless there’s a level of maturity that is mostly positive. This foundation can be leveraged to embrace new ways of working with data.
The nature of a lot of government projects is not particularly data centric with some exceptions, and I think the struggle will be whether government can realistically treat data as a product in such a big, complex, and politically charged environment.
Who needs to be involved in making it happen?
Until the time when organisations realise data must sit at the top table, it is the technology leaders and business leaders that need to get involved in tipping the balance towards data being embraced as a product, but we know from history that businesses that embrace technology and data at their core are more likely to survive and thrive, especially in challenging economic environments.
Removing the layers of ‘management and bureaucracy’ between the subject matter experts and the business leadership is the fundamental shift that needs to happen.
What role does the rest of the data team come to play in this?
As with any successful strategy, to make it a reality, especially in larger enterprises, there must be top level representation and subject matter expertise, sufficient levels of investment, and a collateral of specialist expertise across the organisation. Strategy needs to be deeply understood and permeate at a cultural level; it doesn’t work in isolation, which is the space where most data teams operate in.
Assuming these pieces are in place, the data team has an important role to play. They need to promote what they do in the wider organisation, canvas for change, influence the right people, and start to drive solid action through operational changes.
Pivotl are one of the heading sponsors of the Think Data for Government conference on December 12th in Westminster. Register here to book your place.
