Analytics in a Fragmented Data Landscape

The idea of a product recommendation engine is simple—count sales per customer and product and fill this in a matrix. Add correlated dimensions to the data, such as the customer’s industry sector, geography, and size. Train a model to learn different rules, like “customers buying three or more CCTV cameras are likely to subscribe to a security service contract.” Finally, predict the likelihood of blanks in the matrix to identify promising opportunities.

What if the data landscape in our company is fragmented? Products and services reside in different ERPs, separated from the CRM containing the customer contact details we need to run marketing campaigns. What if there are no join keys across sources and duplicates of customers and products within each system? A single customer and product will be spread across multiple rows and columns in our recommendation matrix. The model will learn a biased version of reality. Recommendations across fragments of our data landscape will not be possible, leaving us with a trivial subset of opportunities.