The data analytics consultant


Data mining demographic information and transaction data of a large retail company

Table of Contents

1.0 Introduction 5

1.1 Aims, objectives and possible outcomes 5

1.2 Background 5

1.3 Data analytics scenario and methodology 7

1.3.1 Methodology 7

1.3.2 Data collection 8

1.3.3 Deployment 10

1.4 Plan and time table 11

References 12

Appendix: Budget 13

List of figures

Figure 1: Customer analytics lifecycle 6

Figure 2: Visualization technique of demographic clusters 8

List of Tables

Table 1: Supermarket point-of-sale transactions 10

Table 2: co-occurrence of products 10

1.0 Introduction

This document is a proposal presentation of electronic business case using data mining techniques to obtain insights on how a company can identify and support loyal customers. From the loyalty card scheme, it is possible to collect large amounts of customer shopping history and data on sales of various products sold by the company. The source of data is demographic information and number and type of transactions undertaken by customers. In this proposal a suitable data mining technique and scheduling has been implemented to showcase use customer purchase trends and customer loyalty to build a need for funding of the program.

1.1 Aims, objectives and possible outcomes

The aim of this report is to data mine ways of attracting new customers and retaining existing customers at a retail supermarket.


  • To investigate changes in customer loyalty or consumption of a retail company

  • To suggest adjustments on variety of goods and pricing at the retail supermarket

Use of customer loyalty card information provides a way of observing the sequences of purchases from customers to analyze purchase trends and customer loyalty. Sequential pattern mining groups similar consumers into sequences at different periods based on goods they purchase.

1.2 Background

Large retail stores competing among small stores have literally forced the later to close down while gaining immense power within the supply chain. They have invented new ways to attract customers and dictate terms to the retailers on an increasing basis. Although it is an elusive dream to hold onto customer’s imagination for long, customer disloyalty has been blamed on demographic shifts, increasing competition and changing tastes and preferences among customers. This is the reason why retailers are going on an extra mile to understand and reach the customer. Business Intelligence (BI) gives retailers the chance to meet the ever-changing desires and needs of customers through use of tools like data mining and data warehousing.

Customer analytics helps to accurately predict customer behavior and unravel new trends and patterns in data. This in turn will help retain customers, churn prediction and identify valuable customers as shown in the customer analytics life cycle below.

	the	data	analytics	consultant

(Source: Isakka, 2006)

Figure 1: Customer analytics lifecycle

Isakka (2006) argues that different algorithms and data mining techniques have helped organizations to collect appropriate information on customers to develop new products, decide future plans and expand marketing. Data marketing techniques support marketing decisions and extracts marketing knowledge (Bose & Mahapatra, 2001). Sumathi and Sivanandam (2014) observe that analyzing the reasons for customer attrition is critical in effective customer retention programs. As one drills down to individual transactions that results in loyalty change, business intelligence provides a greater understanding of customer attrition and factor that influence customer behavior.

Although loyalty programs are needed by customers, their proliferation and customer willingness to join is diluting the actual intentions. For example, according to Maritz report, overall growth rate (27 percent) of membership showed active membership growth at 21 percent, while actual active membership reduced to 44 percent from 46 percent in two years (International Institute of Analytics, 2014). This implies that despite loyalty programs acquiring members, many are left unengaged and could be a result of loyalty card overload. There is a correlation between highly effective retention programs and analytics, especially those using customer-focused lens like segmentation and customer experience. High performing organizations create business impact and drive business strategy by employing customer analytics (Grabmeier & Rudolph, 2002). Moreover, many focus on customer needs, enable organizations offer personalized rewards to effectively target customers and develop long-term relationships across the customer experience.

1.3 Data analytics scenario and methodology

1.3.1 Methodology

Sequential Pattern Mining (SPM) is employed to extract sequences while maintaining order of transactions. SPM exploits a number of other techniques such as clusters and associative rule to show patterns (Bose & Mahapatra, 2001). This method is used for long-term data to identify regular occurrence of trends and similar events. Customer data can identify customers that purchase a given collection of products at various dates within a year. This information is used in a shopping basket application to suggest that some items be added to the basket based on past buying history and their frequency.

The large retail company with data on shopping history and sales of its customers would be interested to use cluster analysis to analyze demographic information of its loyal customers with cards and market basket case to analyze its transaction data. The retail supermarket has more than 7,000 loyal customers with cards and is interested to increase the number of transactions and then adjust the price of goods. To understand the customers, it has to cluster those who buy certain retail goods and those who do not. The data is gathered had the following fields; age, number of children, household income, marital status, average price of items, and amount of money to buy items, and total cost of items. The input values were clustered and mapped to lie between +1 and -1. The average value of each item is obtained in the clusters. For example, the proportion of males to females will have values of 0.0 mapped in a numerical range. The values when plotted in parallel coordinate as shown in the figure 2 below.

	the	data	analytics	consultant 1

Figure 2: Visualization technique of demographic clusters

From the graph above, it can be observed that consumers are segmented based on income, age, and marital status among other categories. It can be learned that high-income earners with children in their teens tend to buy above the average price and also buy diverse items in the shopping basket. However, for middle income households with children, they are at par with the average price but buy low proportion of diverse goods in the shopping basket.

1.3.2 Data collection

The transaction data fundamentally provides information about; customers, purchases and items. An order is a single buy event by the customer online or at a grocery store and it includes payment type, shipping charges, total number purchased and the total amount of purchase (Berry & Linoff, 2009). The individual items have data such as cost, tax charged, number of items and price paid for the item. Moreover, other information can include product reference and product hierarchy. On the customer table, an important aspect is customer ID which ties transactions together over a period of time.

The essence of tracking customers over time it to know the customers, for example, who ‘bake from scratch’ and is critical to prepackaged cake mixes or makers of flour. In this case, the customers are identified by proportion of total spend to the specific purchase, baking powder, and frequency of the purchase. Since they are purchased in different quantities and at different times, the following information will be data mined to observe relevance and sequence. First is to determine the average number of;

  • Items per order

  • Unique items per order

  • Orders per customer

For the given product, it will also be important to know;

  • Proportion of customers who have purchased the product to total customers

  • Mean value of order/customer including the item

  • Mean quantity bought in the order when the product was being bought

This scenario establishes that per given customer, the number of sales can be increased where proportion of orders/customer to repeat customers equals 1. Where number of products per order equals 1, cross-selling can be done while making the order (Berry & Linoff, 2009). Returning customers tend to purchase diverse goods meaning that the retailer can encourage customers who return to buy the same item bought the first time to consider other items (Rajagopal, 2011).

Similarly, the retail giant clusters the product by usage and recommend to customers with cards and who may have bought the products in the past. The association rule is used to understand the products that can be recommended to be purchased by customers at the same time (Kim et al. 2004). A transformation takes place at category level instead of individual product level so as to reduce the number of columns (Berry & Linoff, 2009). Since the retail supermarket sells a number of different items such as clothes, foods and other items, associative rule will consider the following;

  • Do customers purchase similar colors of washing and cooking items at the same time?

  • Do soft drinks and cakes sell at the same time?

  • Do customers who buy electronics in the supermarket also buy other household items?

The association rule will then lead to the following table below on point-of-sale transactions and co-occurrence of products.

Table 1: Supermarket point-of-sale transactions

	the	data	analytics	consultant 2

Table 2: co-occurrence of products

	the	data	analytics	consultant 3

From table 1 above, five grocery products can be combined as a basket and purchased at the same time. Besides, one set of products can be bought alongside other products as shown in table 2. For example, where ‘soda’ intersects with ‘detergent’ shows value ‘3’ which means that the set of items at the basket may contain 2 sodas and 1 detergent. The co-concurrent table shows simple patterns as;

  • Milk cannot be bought alongside detergent and soda

  • Soda and orange juice (OJ) can be bought at the same time

  • Window cleaner is never bought alongside detergents

This enables the customers to choose the right sets of items while allowing the retail supermarket to generate rules and overcome limits that emanates from a myriad of items.

1.3.3 Deployment

The program will be implemented in six months and will involve a number of activities as provided in the schedule below. The outcome of this data mining activity will enable the business use the data to run loyalty programs that involves promotions. Based on the transaction data and demographic data above, the business will target high-income earners with middle aged children as repeat buyers to the retail supermarket. Since there is enough information about them, they will be notified on emails and their internet banking platforms on the availability of certain products in the shopping list (Berry & Linoff, 2009). For example, high income earners with teenage children who love buying soda can be enticed with detergents and orange juice. This will serve to raise the sales of both detergents and orange juice since they trend shows customer preference. Similarly, posters or ads in the supermarket and television can show one cleaning a sitting room then later take a glass of orange juice or soda.

1.4 Plan and time table

The data mining project will be implemented by 10 staff in six months and is estimated to cost $623,000 (See appendix for budget). The plan is as shown in the table below.



Staff Involved

Business Brief/Case

Prototype of tools and techniques

Extrapolate performance constraints

Verify data quality

Sponsor/Project Manager

June 2016

Data preparation

Flattening and cleaning customer data

Data formatting e.g. neural networks

Project team

August 2016

Modeling (Cluster networks & Associative rules in Sequential Patterns)



Domain experts, data miners and project manager

October 2016


Meets business objectives/business requirements

Refining and revising the models

Domain experts, data miners and project manager

November 2016


Implement the system

Deploy into the retail system

Data miners and project manager


Berry, M.J. & Linoff, G.S. (2009) Data mining techniques (2nd Edition). John Wiley and Sons.

Bose, I. & Mahapatra, R.K. (2001) Business data mining-a machine learning perspective. Information and Management, Vol. 39, pp 211-225.

Grabmeier, J. & Rudolph, A. (2002) Techniques of cluster algorithms in data mining. Data Mining and Knowledge Discovery, Vol. 6, pp 303-360.

International Institute for Analytics (2014) Keeping customers: Successful loyalty through analytics. IIA.

Isakki, P. (2015) Purpose of data mining for analyzing customer data. International Journal of advance research in computer science and management studies, Vol. 3, No. 4, pp 199- 203.

Kim, Yong Seog, & Street, W. Nick (2004) An intelligent system for customer targeting: A data mining approach. Decision Support Systems, Vol. 37, pp 215–228.

Rajagopal, S. (2011) Customer data clustering using data mining technique. International Journal of Database Management Systems, Vol. 3, No. 4, pp 1-9.

Sumathi, S. & Sivanandam, S.N. (2006) Introduction to data mining and its applications. Springer science and Business media.

Appendix: Budget

Data rights and goodwill


Office lease and equipment

6 PCs+Firewall

Software (Preferably Statistica Data miner)


Office admin costs

Labour (Six months)

4 Data miners=$120,000

1 office admin=$30,000

1 Project Manager=$75,000

2 Marketing research [email protected],000