Tackling Product Matching for E-commerce using Automation

There is a vast number of products sold online through various outlets all over the world. Identifying, matching and cross-checking products for purposes such as price comparison becomes a challenge as there are no global unique identifiers.

Here's when product matching becomes extremely important. There are many situations where accurately identifying a product match is essential. For instance, stores may want to compare competitor prices for the same product they may offer. Similarly, customers may use comparison tools to get the best deals. 

Amazon allows different sellers to offer the same products only after ensuring that they are the same before listing the sellers in a single, unique product page.

Numerous products but no method to match them across different stores

Product titles/descriptions do not have a standardized format. Each store, as well as different sellers within a store, might have different titles and descriptions for the same products. Another challenge comes in with respect to attribute listings as different e-tailers follow different formats. The product images of the same product also differ across different e-tailers.

While there are standardized unique identifiers like UPC, MPN, GTIN, etc, they, however, may not be mentioned in the product page in all stores selling them. The attributes themselves may be described differently - for instance 9" and 9 inches. Images may be included but they can differ in perspective, clarity, tone, etc. The brand name may also be referred to in different ways like GE and General Electric.

It is an impossible task for a human to visit different product pages to ensure if they are matching the same products. Although, if the process is to be automated, how can it be ensured that the system makes sense of all the information. This is when AI and machine learning come into the picture. 

Machine Learning for Product Matching

In machine learning solutions for product matching, the solution provider must initially build a database with billions of products. This can be done by collecting information through web crawls and feeds. The system then has to come up with a universal taxonomy. This especially is a unique challenge as different retailers use different classifications for their products, and the same product might be listed in more than one category. For instance, a particular shoe model might be listed under casual shoes as well as dress shoes. The system first must design a standardized taxonomy, irrespective of how a particular store classifies its products.

There are standard classification models such as Google Taxonomy, GS1, and Amazon but a product match solution may devise its own taxonomy. The universal taxonomy is designed by identifying patterns and signals from titles, product descriptions and attributes, and from images.

Once a universal taxonomy is in place, the next step is making particular product matches. Here, there is a need for precise comparisons to ensure a particular product is indeed the same unique product, despite the differences in titles, images, descriptions, etc. First, there is a search for unique identifiers such as UPC or GTIN on the product page. Then, the product titles need to be compared. It needs to be noted that no two product titles are the same across different stores for the same product.

Neural networks play a key role

Neural networks and deep learning techniques are extensively used to identify and learn from similarities, to identify and learn from differences, and produce word-level embedding to create a system of representation for common words. This involves teaching the system to recognize different references to a unique entity such as 'GE' and General Electric or 7" or 7 inches, to come up with one unique representation for each entity.

A product can be identified using its title, description, images and attributes or its specifications list. In many cases, the product title itself will yield a lot of information and the system needs to be trained to differentiate the product name (for instance, brand model) from the attributes.

Product matching - Identify and Sort

The information then needs to be extracted and sorted into the appropriate slots - Phone model, version, memory size, etc. Different techniques might be used to help the system learn to parse and sort the different sets of information. 

The next comparison comes in the form of more information about the product such as the title, description containing additional information and a specs table. These help add more knowledge about the product, and the machine will be better able to identify an exact product match or mismatch in the following comparison.

The standard identifying signals are similar results or positive matches for unique identification numbers (UPC or MPN), classification, brand, title, attributes, and image. For each comparison, the system follows a long procedure of checks or safety valves. The checks pass through a search for the unique identification number, a test for keyword similarities, brand normalization and match (for example, HP is the same as Hewlett Packard), attribute normalization and match ( 9 inches is the same as 9in, 9"), image matching, etc. There is also a check for variation in attributes such as:

For the best product match result, there has to be at least 99% of positive results. It will be considered a mismatch, even if it is a variation of what is essentially the same product. Different product match solutions employ different techniques and training methods, and it is a complicated process. Although, there is an advantage that neural networks and machine learning learn over time, and get better with each use.

Related e-commerce articles:

While online retailers have the advantage of tracking cookies and web analytics tools to calibrate different aspects of an online shopping experience, brick and mortar retailers aren’t as lucky. They have had to depend on much erratic customer insights.
However,in today’s date, even physical retailers are required to hold up to some very high expectations in shopping experiences.

In fact, in order for physical stores to remain relevant, they have to focus on improving the quality of the experience they deliver. This has led to the creation of entire businesses in retail experience innovation.

One of the ways a brick and mortar retailer can provide a quality experience is through the employment of in-store analytics that provides insights into the behavior of the customers and uses that information to engage with their customers as they shop. The use of in-store analytics has revolutionized how retailers understand their customers and how they communicate with them.

How will in-store analytics build up communication between shoppers and retailers?

In order to answer that question it is important to know how In-store analytics works.
For example, when a furniture store to offers free Wi-Fi, it may seem a bit strange.
Actually when the Wi-Fi is enabled on one’s phone, the device sends out a connection request every few seconds on every Wi-Fi channel available. It updates the list of the available networks after listening for a fraction of a second for a response to come back,

Interestingly, when a device probes the Wi-Fi spectrum, it broadcasts its unique MAC address to any device that’s listening. So, as one walks around in that furniture store, every Wi-Fi probe then acts as a beacon for the location. With multiple Wi-Fi access points available inside a single store, it becomes possible to considerably precisely locate each address. As far as the owner of the device is concerned, this happens passively without having to actually join a Wi-Fi network.

Although nothing about a device’s owner is being shared, the retailer can build a picture of what individuals do as they walk around a store. Such as, the number of customers who went to the first floor, the time people tend to spend in a particular region, the waiting period of customers before they come back to the shop.

This aids in understanding the broader shopping habits and interceding with informed in-store customer communication. Instead of having communication with customer transpire at the convenience of the retailer, it can happen at the customer’s convenience. 

Sending an SMS to inform of a sale as an effective marketing tactic. Sending emails every month or even good old direct post may increase customer movement towards a local store. However, a more customer-centric communication of a timely WhatsApp message offering assistance when the furniture store operator gets to know that the customer has spent over 20 minutes in the dining table department.

MAC address tracking to deliver a more personalised Customer Experience

Anonymously tracking a MAC address results in a more personalized customer communication and in understanding individual behavior in the retail experience. As the data increases, the MAC addressing question ceases to just be a randomly generated number and instead represents the behaviors of a real person. At this stage, there’s nothing to identify the individual who owns the phone but it’s possible to build a picture of who they are.

Whether gathered in multiple locations or over a longer time period in just one location, as the data builds it becomes useful in crafting more personalized communication, which can help increase sales and enhance the customer experience.

Relying on anonymized data can deliver only so much, though. And that brings us back to why the furniture store offers free Wi-Fi. As soon as someone signs up for that Wi-Fi, the store can associate the MAC address with whatever data they capture in the sign-up process. At the very least, that’s likely to be a name, email address and cellphone number. Again, that person never has to use the Wi-Fi: as long as they keep the same device, their MAC address and identity are linked.

Other retailers might not rely just on free Wi-Fi. They might have a loyalty or coupon-based mobile app that requires users to provide some personal data. Depending on the phone’s operating system, that app might be able to access the MAC address itself and make the connection for the retailer. Either way, retailers can incentivize shoppers to make their MAC address personally identifiable. And when that happens, communication can truly become personalized.

Respecting the shopper’s personal data

Either through inertia or without realizing it, most people publicly surfing the web are constantly being tracked. Sure, there are some loud voices of complaint but the vast majority of people accept it or don’t care.

As the company behind smart recycling bin advertisements in London and Nordstrom in the US discovered, people are less keen to have their physical location tracked. Even if it’s only an anonymized MAC address, such tracking could feel intrusive.

A value exchange for a richer retail experience

The answer, perhaps, is to take a tip from the loyalty schemes of large retailers: provide a genuine benefit to customers in exchange for gathering valuable data on their habits. Just as loyalty schemes such as Air Miles and Tesco Clubcard offer coupons, cashback, and exclusive store events, retailers can build similar value into retail location tracking and analysis. Rather than silently track customers, they can volitionally opt in to a mobile-phone enabled rewards program when they enter the store-a loyalty scheme for the 21st century.

Location tracking has the potential to transform how retailers communicate with their customers. It will provide the insight to know precisely when to engage and when to leave someone alone. However, it will work only if customers can see a tangible benefit to giving up some of their privacy.

E-commerce is an intensely competitive market where businesses need to keep innovating and adopting new technologies to sustain. These technologies come in packages, large & small, to help optimize the systems and processes, ensuring seamless experiences for the customers.

One of these technologies - Chatbots - has been a popular topic of discussion in the retail and e-commerce industries. Some discussions focus on how the user experience is improved while the remaining provides a detailed view of implementing and adopting chatbots for businesses. In this blog, we try to provide a holistic view of how chatbots help scale e-commerce businesses and subsequently consumers by dividing the benefits into three categories - predictive recommendations, engaging consumer engagement, reduction in the purchase process

Predictive recommendations 

E-commerce industries are user heavy and need to cater to different segments of audiences and their requirements. Whenever there is a discussion about the role of chatbots, one quote stands out -

“Goal is to turn data into information and information into insights.” – Carly Fiorina, Former CEO of HP

Imagine chatbots to be the one that hoards information and turn information into insights. They help customers find the products they’re looking for without extensive browsing, thereby providing users an incentive to stay on the platform and reduce drop-offs.

For instance, Amazon provides best-in-class search for users to find the products they want. The search takes into consideration a number of factors including user dialects, ease in product categorization and user buying behavior & patterns.   

Simple & Quick Consumer Engagement 

The primary function of chatbots is to be conversational by nature using text, buttons, and images and understand typed natural language requests. This is the reason that the most famous chatbots run inside messaging applications such as Facebook Messenger or Skype.

Hence, chatbots provide a huge benefit to e-commerce businesses to connect with the users.  In a high volume business of e-commerce, chatbots provide a highly scalable system to manage individual user conversations simultaneously for millions of users while gaining user insights to improve the product flow and experience.  

Reducing Purchase Time 

Chatbots help consumers to interact with the products at critical stages of their journey, increasing customer satisfaction, loyalty, and engagement. Chatbots provide the assistance to access product information quickly and make informed decisions to purchase the products.

Moreover, customers do not need to look at any other source to gain information on the products. Chatbots process information in the form of notifications, reminders, product updates to fuel conversions and enhance social experiences.

Image credit: Photo by Matan Segev from Pexels

E-commerce requires awesome product data to support successful search and conversion. Product data for the online grocery market is currently being created manually. Retailers are struggling to acquire the rich product data necessary to support their online needs. Brands are struggling to generate good data.

IceCream Labs CatalogIQ is designed to automatically extract attributes from product images. Using either label mechanicals or actual product images of the packaging, CatalogIQ can extract text from the labels. From there, the artificial intelligence in CatalogIQ understands what the text is and inserts it into the appropriate product attribute. The AI can also determine which images are the hero image, and front, back and side images.

CatalogIQ identifies brand names, sub brands and variants, normalizing the brand to the appropriate text. titles are generated from various attributes to create an SEO-rich title to optimize search. Other key attributes include feature/benefits, ingredients and nutrition facts.

CatalogIQ can score the data to help the merchandising and ecommerce teams understand which product records have been enhanced.

• Missing attributes
• Accuracy of attributes (are all of the attributes congruent?)
• How unique are the attributes?
• Is your product record SEO optimized?
• Do you have relevant search keywords?
• How well does your product data match up to customer site searches?

Grocery merchandising teams have the chore of uploading new catalogs from suppliers and manufacturers. Often this data arrives in the form of a spreadsheet. CatalogIQ can easily upload a new catalog file (in spreadsheet form) to quickly and easily complete the ingestion process.

Grocery retailer and channel partners expect high quality product data to list and sell your products online. Can you deliver the content?

CatalogIQ allows brand and product managers to auto-generate high quality product data directly from product label mechanicals and/or product images. If you're currently using manual processes to create product content and to check the accuracy of product data, then let CatalogIQ help you automate the creation process. You'll be able to complete the data creation process much faster than manual methods. CatalogIQ can also validate the content and ensure that it matches what is contained on all of the product labeling.

As a merchandising manager with a large product catalog, you know the difficulties of reviewing your product data and ensuring that everything in the catalog is ready to publish live to customers. There is always the nagging concern that something is inaccurate or missing when you push the “publish” button. Every time that you receive new data from your suppliers, it’s a chore to process the data. You have a long checklist to complete before you can publish data to the live catalog. Processing this checklist can consume all of your time.

People have increasingly become comfortable using technologies such as AI and machine learning in their day-to-day lives. Various companies also have increased their use of AI and machine learning into their product offerings as well as their processes. With the computer processing technology advancing increasingly, companies, institutions and even governments are gathering massive amounts of data as more consumer interactions move towards digital. This type of technology is already transforming the payments landscape in the following aspects -

Improving Efficiency

AI and machine learning have the potential to revolutionize the way payments are processed by enhancing operational efficiency and decreasing costs involved. For instance, AI enabled chatbots are reducing the load for customer service representatives.

With machine learning being incorporated into payments, learning algorithms play an important in helping speed along authorization of transactions and monitoring.
Furthermore, AI helps reduce the processing time for payments. It also helps eliminate human error and save time for correcting those mistakes.

For a business, processing large amounts of data to generate financial reports to satisfy regulatory and compliance requirements would involve a team that would perform repetitive data processing tasks. Leveraging AI would involve training the models to do these tasks, and the model can ensure completing the tasks faster and more accurately than humans. These technologies can improve efficiency while gathering important user insights in real-time.

Machine learning has already proved to be an invaluable part of fraud detection, but there are many opportunities that lie in product sales, customer care, and retention. Machine learning can draw vast amounts of available data and utilize it to profile customers to guess their product needs while offering new opportunities for upselling.

This model can also help identify which customers that companies are at risk for losing as well as halt customer loss before it happens. Machine learning can help automate many customer service interactions. This model which uses deep insights, cognitive engines and natural language processing is already widely available and the usage will only grow with time.

Fraud Prevention

​There are various methods for customers to make payments today. They are no longer limited to paying with cash or even cards. There are new payment methods on the rise such as card-not-present (CNP) transactions, but as it gets popular come new opportunities for fraud. AI and machine learning are at the forefront of not only detecting fraud but also preventing it before it happens.

These technologies already have the capability to uncover patterns and drive hidden insights and are working towards fine-tuning these insights. Companies are choosing to move away from a static model where the reliance is on supervised learning with input towards unsupervised learning wherein the deep belief neural network does not require a labeled training set, but continuously updates the model as new patterns emerge, allowing for a more robust and flexible fraud prevention detection tool.

As more commerce and payments move online, more data is accessible. This new robust algorithm uses machine learning to decrease the false positives and more agile detection of the actual frauds.

AI and machine learning have come a long way in the past decade. These technologies have already been adopted by many sectors and have transformed many aspects of traditional processes. Though exciting new technologies have been adopted by businesses to improve and enhance the payment process and customer experiences, the scope for future implementation is endless.

While there is a lot of chatter around artificial intelligence and the potential it has to transform retail, it has the capability to impact the most fundamental shopping experiences: the grocery store. Most retailers are moving towards building a better grocery store experience, with the rise of subscription meal kits and competition from pure-play grocery delivery services. With the abundance of customer data and product information available, grocers today are in a special position to apply machine learning and AI in other areas of the business as well.

The grocery industry has a heavy reliance on the movement of perishable goods and with supermarkets struggling to plan, promote and sell goods in a short period of time, it is not efficient. Furthermore, there is a lot of food wastage that happens in this industry - while some of that waste happens in consumers’ homes, a good amount is also lost in the supply chain - anywhere between the farm and the store shelf. And with the various options for when, where and how to buy groceries, grocers compete on prices that are most likely to be profitable.

While grocers are faced with these challenges, they have a fairly good idea of their customer base, who they are and what they want to buy at what price point. With the data available, there is a lot of opportunities utilizing the data in the right manner. This is where AI comes in. Using machine learning capabilities and analytics, more grocers are leaning towards adopting this technology to strengthen the relationship with customers, as well as address some of the biggest challenges they are faced with today.

Leveraging the abundance of customer data

The grocery industry was one of the first industries to collect shopper buying data through programs such as loyalty programs and in-store promotional offers. These methods helped grocers gather information about their key shopper demographic as well as brand preferences. This information is already leveraged to provide discounts and special promotions, the new technology can help enhance the relationship between the grocer and shopper even further.

AI helps grocers to provide a deeper understanding of context and intent by answering the questions behind customers' shopping decisions. It also enables the grocers to parse the customer data and automate the ability to offer targeted promotions to each customer.

Enhanced inventory management

AI can change the entire way of managing inventory. AI can help stock shelves with the right mix of products and ensure that the supply chain is aligned to avoid out of stock products using point-of-sale information and inventory visibility.

Machine learning can build on grocers' rich customer data and combine that with contextual data such as weather, climate, holidays and events - providing a more accurate forecast compared to traditional methods.

Reducing waste

With better inventory management and data analytics, AI can provide better visibility on produce and perishables. Automation can help stores dynamically re-adjust orders based on demand or automate product promotions for the items that are not performing well or selling fast, and helping stores to protect margins, and reducing the amount of food that goes into landfills.

As AI and machine learning advances, grocers should begin to position their systems for a seamless transition towards a highly automated future.

AI must integrate into the commerce platforms and connect across systems to maximize its effectiveness throughout the business. While AI is a valuable tool for customer service, the impact of it will come through its ability to reward loyalty, understand consumer behavior, ensure reduced wastage and increase revenue for the retailers.