Most companies continue to struggle in managing their data and putting it to work. They expend a lot of time and energy, but don’t get much for their efforts. Quality is low, people don’t trust the data, technical debt is out-of-control, and they miss opportunities to become data-driven, take advantage of advanced analytics and AI, and compete with data. Indeed, most organizations are simply not attuned to the rigors of working with data.
This, of course, is a problem. At this point, practically everyone’s job involves using, interpreting, and creating data. Yet somehow this seems to get lost at all levels of organizations — the structure, the culture, the people. It’s often unclear whose responsibility data is (the CDO? IT? Everyone? No one?), and because data tend to be hidden, in customer orders, logistics, and management reports, the power of the status quo prevails. Without clear expectations, chaos reigns. People don’t know what to do, basic tasks are left undone, and much of the work that is undertaken is done poorly. The unfortunate reality: more often than not, data is essentially unmanaged.
Businesses must craft better systems and approaches to working with data, and that starts with clarifying management responsibilities for everyone who touches data in any way, across the entire company. Here are five guidelines for deciding who should do what when it comes to data.
Most of the real data action involves “regular people,” who don’t have “data” in their titles. They create the stuff; interpret the stuff; use the stuff to satisfy customers and regulators, keep track of inventory and money, and make plans and decisions, and so forth. These people are effectively the front line of any larger data project, program, or strategy, and are essential to its function. Yet they’re almost always left out at the planning stage. Given the excitement about big data, artificial intelligence, and digital transformation, you might be surprised that including regular people is the single most important step companies can take to accelerate their data programs.
There’s huge potential here. To unleash it, companies need to clarify regular people’s roles and responsibilities, as data customers and creators when it comes to data quality, as small data scientists, as contributors to larger data projects, as better decision-makers, and as guardians of the company’s data. The first step leaders should take is putting regular people and these responsibilities front and center. They must also follow through, training and supporting people to help them become effective in their newly articulated responsibilities.
Build the infrastructure to work above, around, and through siloes. While companies reap the most value from data when it’s used across departments, siloes get in the way of the data sharing. Despite the fact that almost everyone depends on data created by other departments to do their jobs (e.g., sales uses lead data generated by marketing, and then passes sales data on to operations for fulfillment, and so on), departments are often unconcerned with the quality of data they pass on. Companies are gigantic daisy chains of data flows, and when bad data gets passed along, it mucks up everything.
For better or worse, silos probably aren’t going anywhere soon. Rather companies must build infrastructure that can transmit and coordinate the flow of a lot of data in an organized way — what I call “fat, organizational pipes” — to contend with them. First, companies need to define and manage data supply chains. Just as companies track the producers and raw materials, they rely on to manufacture and deliver physical products, they should define and track how data is created, how it moves from one place to another, and how it is analyzed and used along the way.
Second, they need to build a data science bridge, which supports communication between business teams and data science centers of excellence. These two teams often find themselves at cross purposes, as the former tries to build predictable processes and the latter tries to disrupt that stability to find improvements, update decision making, and develop new products. The bridge aims to ease this tension, by helping both teams understand the concerns and needs of the other.
Third, they need to create shared language between departments and across the company. As companies grow and departments specialize, the language used diverges. (E.g., the term “customer” comes to mean “prospect” to marketing, “person with sign-off authority” to sales, and “entity ultimately responsible for paying the bill” to risk management.) This makes it difficult for people to work across silos and for data scientists to make sense of the company’s data. Companies can make huge strides by identifying roughly 150 key concepts that bind the company together, and agreeing on common definitions.
Too many companies wrongly assign principal responsibility for data to their Information Technology departments. But most data is neither created nor used by IT — technology and data are different kinds of assets, in much the same way streaming services and movies are different. Companies should let IT do tech, building infrastructure capabilities, automating well-defined processes, and, in time, reducing technical debt.
Companies need small, professional data teams with deep expertise in a range of topics, including data quality, data science, metadata management, privacy, and security to drive home these responsibilities day-in and day-out. As explained in a previous article, perhaps half of these data teams’ effort should be aimed at training and helping regular people so they can step into the roles and discharge the responsibilities discussed above.
Professional assistance is also needed to help those charged with managing data supply chains and establishing shared language. A network of embedded data managers is essential to increase the reach of professional data teams in conducting this work. Professional data teams must also reserve some fraction of their time for specialist work — interpreting privacy regulations, developing data models, and leading especially tough or important data science initiatives (although these efforts must also involve regular people).
Over the past generation, terrific methods of data science and data quality have proven their mettle in countless circumstances, solving difficult problems, yielding new insights into customers, and driving costs down. Still, it has proven difficult to introduce these new ideas into companies and to extend successes in one part of a company to others; the failure rate of data science projects remains disturbingly high. Data programs are in dire need of senior leadership to help resolve these problems. Yet by and large, senior leaders have sat on the sidelines.
It appears to me that senior leaders want to do the right things, they just don’t know what the right things are. In their defense, they face a confused deluge of proposals, all of which promise dire consequences if ignored, but each of which offers different recommendations. Separating the signal from the noise is a tall order. With that in mind, leaders must initially focus on two things:
Companies are loaded with business problems/opportunities and with great ideas in the data space. But too often they fail to find one another — the business problems remain unsolved and those with great ideas grow frustrated. Senior leaders are singularly well-placed to connect the two.
If you’ve not done so already, hire a great Chief Data Officer, one with the courage of a lion to stand on the front lines of change everyday, and with the patience of a saint, to think long-term and not get distracted by petty sniping.
. . .
These guidelines represent a sea change in the way data is managed today. Some will say they are way too hard or not worth the trouble. But data management today is simply not getting the job done. It’s foolish to think that simply bolting on a data science center of excellence would lead to industry-changing insights, that a few raw PhDs, could change the minds of a generation used to managing by the seats of their pants and pantsuits, or that applying the latest whiz-bang technology would make up for the failure to manage data properly for a full generation.
Electrification provides a useful example. For electricity did not appear on the scene and magically make everything better. Leaders and technicians had to sort out how to make and deliver it safely, how to change the shop floor to accommodate it, how to deal with its fussy properties, and how to get everyone doing their parts! This took a full generation. We should expect no less for data.
Process Tempo is a Decision Intelligence Data Platform built on industry-leading graph technology. The no-code, collaborative data science, data engineering, and data analytics platform simplifies complex data environments, empowering people, processes, and technologies to work together harmoniously. The secure, governed, high-performance environment delivers actionable data and insight rapidly to all stakeholders, helping to accelerate the delivery of quality, data-driven decision-making and improve business outcomes at scale. Schedule a discovery session
Unlock insights and drive business growth with our platform
Discover the latest trends and insights in data analytics.
