Using data effectively – five stumbling blocks on the road to becoming a data-driven company

• Most businesses have recognised the potential offered by strategic data use, but are still failing to execute data and AI projects successfully.

• Transitioning to a data-driven organization requires fundamental changes to data management, technology and corporate structure.

• In addition to the actual data, the success of a data project is dependent on scalability, acceptance and interdisciplinary collaboration.

The importance of data-driven processes is now widely recognised by business. In their study, “Data-driven Companies: On the path to strategic use of data,” Zühlke found that 85 percent of decision-makers think data and artificial intelligence (AI) offer major potential. Despite this, only 25% described their own business as data-driven. So why this discrepancy? What factors are preventing organizations from becoming data-driven, and how can a company successfully transition to a data-driven company?

New roles and structures enhance data access and quality

Zühlke’s experience from data projects, from its data-driven companies study, and from follow-up from that study show that it’s not the volume of data that’s the issue here. Most companies have enough data available to them. The biggest challenges are data access and data quality. To enhance data quality and data access and facilitate democratisation of your data, new roles and responsibilities – CDO, data steward, etc. – need to be created and integrated into company structures. This requires both structural changes and empowering staff, who are being asked to perform initially unfamiliar tasks, for their new roles.

The organisation as a whole often also has to adopt a previously unfamiliar role. We often find ourselves supporting companies that have recognised this, but that are still trying to identify the best course of action, are still learning how to define roles and specify job descriptions and are still working out what new structures are required and where and how they can integrate the new roles or people into the company.

Strategic data use is founded on scalable infrastructure

Businesses need an actively managed use case or innovation pipeline. A pipeline driven, not by technology, but by business considerations. A key point is that data and AI projects need to be planned and implemented holistically and continuously. Many of the businesses we surveyed for our study had already picked up some experience with data-driven use cases – and had grasped the fact that one flagship project does not a data-driven organization make. In many cases, companies had started out by implementing isolated proofs of concept in a range of different departments. What was often missing was an overall plan for how the company would bring these individual projects together to become a data-driven company – for how these siloed projects would grow together long-term. There therefore needs to be an overall concept in the form of an innovation pipeline, accompanied by related structural changes. Companies also need a managed pipeline to justify the investment needed to lay the foundations for bringing together these silos.

With data projects, success depends on acceptance

Other problem areas identified by our study included proofs of concept that are not taken forward, and perfectly trained models that are not used as designed. Companies report that they rarely or never succeed in integrating use cases into their existing business processes and tools. As a result, even technically outstanding solutions fail to deliver any value. Reasons for this failure include lack of user training and a strong mistrust of AI solutions, resulting in poor acceptance. Let’s take an example from our own practice. We worked with a bank that wanted to better target its sales activities and improve the specificity of its advice to customers. We were asked to implement a data and AI-based recommendation tool for use by customer service staff. The tool provides recommendations on what issues to discuss with customers and how to approach these discussions. Overcoming initial user scepticism required careful design of the user interface. The tool explicitly delivered suggestions only – customer service advisers retained ultimate decision-making authority. This played a key role in quickly establishing trust in the AI algorithms.

Data-driven organizations adopt an interdisciplinary approach

Another key factor when implementing data and AI projects is getting the composition of the interdisciplinary project team right. In our projects, we often witness poor integration of data scientists or poor cooperation between data scientists and subject experts. The key question facing companies is: how do I integrate the emerging discipline of data science? Another issue is that some companies are still unclear about the exact skills needed for successful data and AI project management. The objective therefore needs to be to facilitate collaboration between data scientists and subject experts, and to create a mindset and structure that promotes good interdisciplinary collaboration.

Invest 15 minutes for a status quo analysis

Our study found that, whether you’re a start-up, a modern business or a traditional company, strategic use of data is critical for success. This thesis was confirmed during follow-up work with the businesses surveyed. We conducted one to one interviews with survey participants in which we compared each company’s maturity, standout features and problems with the overall survey results. We carried out frank, in-depth interviews to explore the background to their responses and, both during the survey and in the follow-up interviews, were able to discuss initial recommendations derived using the Zühlke methodological framework.

Survey participants benefited in that, in return for an initial investment of less than 15 minutes to complete our online questionnaire, we held a one to one meeting in which we provided a detailed analysis of the status quo and the overall market to facilitate their strategy development. Survey participants were able to see how they compared to other businesses and re-evaluate their achievements to date. The insights gained can be applied in two ways. Firstly, for internal optimisation, i.e. optimising the business based on the results of the survey. Secondly, for external optimisation, i.e. looking at what they can offer their customers and how they can improve their portfolio and offerings to help their customers to become data-driven companies.

Are you looking to gain a better understanding of where you are in terms of strategic data use?

Talk to us today. We would love to work with you to analyse the status quo at your company.

You can download the full survey and explore the insights from it here. We’ll show you how far individual sectors and business models are along the road to becoming data-driven companies and the hurdles that await them along the way. Using cluster analysis, we highlight typical patterns, similarities and differences in how companies are going about this transition. Using our in-house methodological framework for supporting businesses on their path to becoming data-driven companies, we also offer recommendations for overcoming these hurdles.