Data Rules

Data is ubiquitous. It’s at our work, in our homes, on our phones.

Most businesses have figured out that the calculated use of corporate data is critical to surviving and maximizing revenue. For those companies that left their proverbial heads in the sand when it came to utilizing their data, well, unfortunately, many of them have closed down or are in the midst of a downward spiral.

The fact of the matter is this: Data rules. From the board room to the shipping dock, decisions are made every moment of the day using quantifiable, fact-based, trustworthy data. To build a modern data architecture that will efficiently store and manage data, and one that will scale into the future, companies are turning to data estates.

Data scientists are examining the information residing within the data estate on a daily basis.  They test, research, experiment and reason data to help users answer questions, solve problems, and identify trends by utilizing the storehouse of data captured by the business. They perform deep data analysis and exploration, utilize technologies such as artificial intelligence (AI) and rely on statistics, algorithms, programming and science. Their predictive models help them answer the questions posed and make data-based assumptions about where they believe the market is headed.

Reinforcing Competency Within Functional Departments

Data is also vital to the success of all departments within a corporation. Here’s how:

• Financial management uses data to get a top-level view of business units, divisions, product lines and key performance indicators so that it can make changes, such as pricing adjustments, to maximize margins.
• Human resources rely on data to determine the best sources to focus their recruiting efforts and budget. 
• Marketers work with data to pinpoint which email subject line performed best, which sales promotion garnered the most leads or how a specific campaign performed to enable them to invest their marketing spend wisely. 
• Sales teams work with marketing to acquire analytics data to better understand buyer demographics such as age, income, gender, location, interest, and past purchases. This allows them to formulate a data-driven sales plan to reach out and sell to various targets and defined niches. 
• Operations management uses data to reduce wasted expenses on inventory caused by spoilage, obsolescence, theft, or excessive purchasing, and to monitor and optimize vendor performance and the supply chain.
• Data also reveals external macroeconomic issues that can impact businesses such as emerging trends or environmental factors like regulations, legislation, or tariffs. 

Constructing an Intelligent Enterprise

One critical component for having all this data perform for us is data quality. Relying on poor-quality data can lead to faulty decision making. There are various data rules that go into ensuring that corporate data is trustworthy and relevant. 

I recommend a layered data architecture (a framework to help you collect, clean, organize, store and normalize data) in which raw data is gathered into a data lake. Data scientists like to start with raw data for data exploration and AI.

From the data lake, a modern data warehouse can be built. In it, data is cleansed and enriched, and historical records are stored. This more structured and governed data can form the basis of business intelligence.

Finally, data marts (we call them semantic models) can be defined to give business users access to relevant data. These views of the data provide targeted access to data that's important to a specific business function. If you build all these with a data management platform, it creates metadata about the layers, and provides automated documentation, data lineage and impact analysis.

I recommend also consolidating and integrating all data sources into a central repository to help ensure one version of the truth. Here are a few do's and don'ts to consider:

DO:

• Seek an integrated data management platform that provides multiple aspects of data management, including data integration, data extraction, transformation, loading, and data documentation.
• Think about how data security flows from source systems to analytics repositories and how to maintain security and ensure privacy while still providing access to important data.
• Consider how data will be documented for data analysts and data scientists.
• Make it easy for business users to find relevant data for analysis.
• Consider how much data you already manage and how fast it will grow in the future. Plan for an explosion in data usage where it’s properly managed and easily available to business users.
• Consider putting your data in the cloud for scalability and easy access.

DON’T:

• Try to implement a best-of-breed tool in each data management category.
• Make it hard for users to find or access business data.
• Hand-code unless absolutely necessary.
• Ignore privacy and compliance.

Data Accessibility

Business users need instant access to trustworthy data. They should be able to make decisions on the spot and often can’t afford to wait for approval. For this reason, the industry is moving to cloud-based solutions. 

The big three cloud data platforms that provide any sort of cloud infrastructure you might need are:  Microsoft (Azure Data Lake and other options), Amazon (Redshift), and Google (BigQuery). For each, evaluate how it fits with your chosen tools and other cloud infrastructure. There are also companies that provide data platforms that can run on-premises or in the cloud on the infrastructure of your choice (private or public cloud).    

Data Governance

Corporate management must ensure that safeguards are in place for privacy, security, access control, fraud prevention and regulations such as GDPR or HIPAA.

These data rules are the “tip of the iceberg” for having a high-powered, enterprise-wide data estate.

For more on how to rapidly build, deploy and manage a corporate data estate click here.

This story originally ran in Forbes on September 11, 2019.