The Dark Side of Shadow IT

Shadow IT is the term given to technology that is currently in use by the organization which is not managed or supported by the IT department. The use of Shadow IT technology can be both good and bad for an organization.

If done right, Shadow IT can provide a boost of productivity enabling a problem to be solved quickly because the deployment of the technology is achieved without the traditional, sometimes overly bureaucratic effort. A lenient IT department will allow this practice under certain circumstances and assuming if the value is there, will eventually reconcile the use of this technology into the organization's policies and best practices.

Shadow IT Pros:

• Can help address challenges in a timely fashion
• Allows the organization to test new technologies

The Dark Side of Shadow IT: Shadow Data

If not done right, Shadow IT can spiral out of control and can place the organization under considerable risk. The reliance on Shadow IT grows when exasperated employees looks outside of IT to find ways to solve problems.

Symptoms that your organization has a Shadow IT problem:

• An over reliance on spreadsheets
• An over reliance on databases such as Access, MySQL or Postgres
• An over reliance on freemium / open source tools
• An over reliance on web portals such as SharePoint

It is important to point out that it is not the tools that are the problem. The real problem is the data that these tools generate, consume or store. This data is called Shadow Data and decisions made using this data can create major problems for the organization.

Shadow Data Horror Stories

A large equipment manufacturer was building forecasts using spreadsheets. They found out too late that the data in one of the spreadsheets was incorrect. The cascading affect cost the organization millions in excess inventory carrying costs.

An IT team was planning a complex migration of their applications to the cloud. Their planning spreadsheets contained incorrect data which had been copied and distributed for months. The bad data went uncontested and had unknowingly spoiled much of their plans until it was uncovered way too late.

An infrastructure team had become overly reliant on a single person for guidance and support. This person had been manually collecting data in a spreadsheet for years and over time, this spreadsheet became the golden record for all things infrastructure related. This spreadsheet became the individuals job security mechanism and so he guarded it very carefully. This not only created bottlenecks it kept the organization from discovering ways to improve processes.

What is Shadow Data?

Shadow Data is any data that sits outside data pipelines that are managed by IT. In most organizations the main body of data flows from a variety of source systems to either a data warehouse or a data lake.

These data pipelines are historically inflexible and so if a user needs a different set of data than what is provided, they have to request a change. In some cases, these changes can take weeks to process. Complex changes take even longer and so work arounds are often developed. Shadow Data begins to appear in the form of spreadsheets, side databases and custom applications.

Issues with Shadow Data :

• Shadow IT produces or consumes Shadow Data which is data that may contain errors or data which has passed its expiration date
• The use of Shadow Data creates potential redundancy as work is not reused or repeated for similar requirements or for other departments
• A lack of insight and transparency. Spreadsheets or side databases do little to promote transparency and collaboration
• Creates a security risk as files such as spreadsheets can be easily emailed or downloaded outside of the working environment
• Decisions can be made using incorrect or older information

Addressing the Shadow IT / Shadow Data Problem

Chief Data Officers are very aware of the Shadow IT/ Data problem. It is what keeps them up at night. Understanding the inflexibility of their current data pipelines they look to alternative strategies that can help to coral the use of Shadow Data.

One strategy is to leverage technology that offers non-technical users a self-service capability. Without requiring IT's assistance, users can access a catalog of available data sources and get the data they need, when they need it. This level of flexibility can improve internal customer service, help the organization to spot issues sooner, and of course minimize Shadow Data.

Process Tempo represents a means in which IT can regain control of how data is used for decision making by offering an alternative to Shadow IT/Data dependencies. It fits the current architectural stack by offering a place for both data and analytics that sit outside the main data pipelines. The flexible graph-based data storage approach and an easy-to-use, self-service interface is purpose-built to tackle this problem.

Interested in learning more? Grab a spot on our calendar to chat!