Recommender Systems for Product Assortment in Retail

January 16, 2023

By: Abhishek Pandey, Associate Consultant, Intelligent Bus Comm, LTIMindtree

What Are Recommender Systems?

We constantly seek recommendations from our friends and peers who have tastes like ours and an experience of buying a product or service that we desire. Recommender systems work on a similar logic, but on a higher scale.

A recommender system quantitatively combines the experience and the intuition of peers and historical interaction of the user, to derive the best selection from a given set of alternatives. A recommender system is defined as a subclass of information filtering system that provides suggestions for items that are most pertinent to a particular user. To put it simply, it is an AI algorithm based on machine learning, which process Big Data to predict what the users would prefer and make recommendations.

Such a system finds its application in the ecommerce and entertainment industries. Brands such as Amazon, Flipkart, Walmart, Airbnb, Netflix, YouTube, etc. leverage the learnings to provide meaningful suggestions and an enhanced customer experience. It is used extensively even in industries where its application is not apparent. For example, major players like Johnson & Johnson, HUL, Kellogg’s, etc. in the FMCG industry use recommender systems for assortment and planning.

Let’s see how recommender systems can be used in different industries and how they have helped top brands transform customer experiences.

Industry Use Cases:

Entertainment Industry: TikTok and Netflix use recommender systems suggest the most relevant content to its users. The Netflix recommender system influences more than 75% of viewer activity. They even went to the extent of announcing the “Netflix Prize” worth $1 million to improve their recommendations by 10 percent. The illustration below explains the use case in the entertainment industry.

Netflix's Use of Collaborative Filtering

Banking Industry: in the banking industry recommendation systems can be used to serve customers as well as improve internal performance.

Physical branches: physical branches and other areas of direct customer interaction can use parameterized recommendation systems to capture important variables, like types of services requested, time spent, products demonstrated, service staff engaged etc. against sales or services. These help the business in training staff as well as design tailored product and services.
Performance Management: a feature-based recommender system may help in identifying the performance of employees. Performance-based KPIs can help categorize the agents based on their skills, strengths, and weaknesses for example, sales or backend processing. The agents can be mapped to best incentive schemes or training programs using techniques like collaborative filtering. This can increase productivity and employee satisfaction.

E-Commerce: according to McKinsey, 35% of all Amazon purchases are drawn from recommendations given to the users. eBay and Moviefinder.com are among some of the popular organizations that use recommender systems. Here are some use cases:

Convert browsers into buyers: people often browse various websites to check features, prices, variants, etc. of products they learn about. A recommender system studies users’ past behavior and suggests relevant products/services. This reduces the users’ propensity to switch sites and may lead to a buy.
Cross-sell: recommender systems can boost cross sales by suggesting products or services that are typically bought along with the current selection of products or services. This is usually done by calculating the item-to-item similarity.
Loyalty: we are at a stage in technological advancement where competitors are just a click away, which makes retaining customers both important and challenging. A good recommendation system provides users with a relevant and exciting selection of products, thereby increasing their loyalty to the platform or the brand. Loyal users generate behavioral data, which helps make more relevant recommendations. This creates a positive cyclic effect and consequently establishes a robust relationship between the user and the brand.

Types of Recommender Systems:

Collaborative Recommender system: a collaborative recommender system tries to find the relationship between user-items, user-user, or item-item, and can use one or more of such combinations to find the desired selection. These are one of the most popular types of recommender systems, often used in the entertainment, e-commerce, medicine, and travel industries. The LTIMindtree APEX platform uses this system to recommend product selections to outlets in the CPG industry.

Collaborative filtering can be categorized into two types:

User Based:

Looks for users who share the same rating patterns with the user whom the prediction is for
Uses the ratings from like-minded users found in step 1 to calculate a prediction for the active user

Item Based: – (users who bought x also bought y), proceeds in an item-centric manner

Builds an item-item matrix, determining relationships between pairs of items
Determines the tastes of the current user by based on the matrix and matching that user’s data

Content-based recommender system: in this system the machine tries to match the describing keywords present in the current user selection to suggest relevant items to the user.

Ex: If a user reads a superhero, crime thriller, the system will read describing keywords in the description and pull-out selections matching author, genre, language, characters, etc. and recommend similar options.

Hybrid recommender system: it combines two or more recommender systems to enhance capability and eliminate weaknesses in individual implementations. The illustration below demonstrates one of the ways to build a hybrid recommender system.

Recommender Systems in CPG Industry: APEX LTIMindtree (Assortment, Planning, and Execution)

Several complex problems that are common in the CPG industry can be solved with a recommender system in place. The system will help with recommendations for must-sell (bestsellers), cross-sell (potential bestsellers), or outlet product line (distinct products sold at a particular outlet) categories. Here are a couple of scenarios, where recommender systems save time and effort.

Must-sell Scenario

A retail outlet has sold 30 distinct products in the last 6 months, but can display only 15 of them in the current month. An easy-to-implement and logical solution would be to display the top 15 products. However, if the retail outlet is a part of a chain that has 200,000 outlets and several hundred products, shortlisting bestsellers could be a daunting task. Even if the probability of a sale is known, scaling it to all the outlets and product combinations would still be difficult.

LTIMindtree has designed a Must-Sell solution within its APEX offering. It calculates the probability scores for each product and outlet combination, by studying the past sales pattern for that outlet. This solution is highly scalable and covers scenarios where a considerable number of outlets could be added in future.

Cross-sell Scenario

A brand which has several products in its portfolio, found that certain products were being sold repeatedly at one of their outlets. The brand, however, needs to increase their offering by introducing new products. Identifying products being sold in similar outlets, which the outlet in question isn’t selling yet, could help, but finding similar outlets and selecting the products to be sold in the current outlet is a challenge.

LTIMindtree has designed a Cross-Sell solution within its APEX offering, which identifies and groups the outlets that have a similar sales pattern. Then, it finds the most probable selection of products in that group, which is not present in a particular outlet, but may sell at the outlet in question.

Solution Development

The table below gives a basic idea on how this problem is approached.

Measuring Performance:

Accuracy, precision, and recall are commonly used metrics to measure performance of any Machine Learning (ML) implementation. They are not relevant to the on-ground sales team. The customer appreciates KPIs that give performance in terms of monetary value. We can use metrics that highlight the revenue increase or cost decrease or a combination of both.

Revenue-Side Approach: To evaluate the impact of a solution on sales, we first check if the revenue has increased after implementing it. This method fails in real-time scenarios because of:

1. Inconsistency in product prices
2. An offer in a particular month brings a significant increase in sales
3. A change in the external environment
4. Seasonal product sales.

There are more problems than one can think, and most of them, if not all, can be eliminated.

Cost-Side Approach: the cost side approach is not very popular but a prudent method to use. The idea here is to find the difference in the cost incurred in planning and sales before recommendation and compare it with the cost incurred in planning and sales after the implementing the recommendation system. This would involve measuring parameters like

1. The time taken by the on-ground sales staff to complete a sales beat
2. The number of out-of-stock situations before and after the recommender system deployment

These parameters could then be converted to a cost unit and evaluated.

Summary:
A good approach would involve considering both the cost and revenue side. For example, a month-on-month comparison of before and after the recommendation (compare January 2021 sales data with January 2022) and comparing based on lines sold (A line is combination of product code and outlet code). An analysis of each cost item relating to stocking and sales before and after the implementation will help understand the increase in the revenue.

Sources

recommender-systems-new-opportunities-in-financial-services.pdf (pwc.in)

Using Collaborative Filtering in E-Commerce: Advantages & Disadvantages (clerk.io)

Healthcare Recommendation System. Yes , you got it right. The above quote… | by Ashish Pal | Medium

Microsoft Word – Schafer.doc (acm.org)

J. Ben Schafer GroupLens Research Project Department of Computer Science and Engineering University of Minnesota Minneapolis, MN 55455 1-612-625-4002 schafer@cs.umn.edu

Joseph Konstan GroupLens Research Project Department of Computer Science and Engineering University of Minnesota Minneapolis, MN 55455 1-612-625-4002 konstan@cs.umn.edu

John Riedl GroupLens Research Project Department of Computer Science and Engineering University of Minnesota Minneapolis, MN 55455 1-612-625-4002 riedl@cs.umn.edu

https://medium.com/genifyai/recommender-systems-and-applications-in-banking-f0cef8f87fa6

https://medium.com/@itua/different-types-of-recommender-systems-a5545c8a074a

https://leuten-leiden.com/research/volume128/number13/tiwari-2015-ijca-906724p9hyb1344dej–.pdf

Types of Recommendation Systems & Their Use Cases | by Maruti Techlabs | MLearning.ai | Medium

How retailers can keep up with consumers | McKinsey

Collaborative filtering – Wikipedia