PIM, Part 2: Why do we need a Smart Product Catalog?

In the last part of the blog we have discussed what a Smart Product Catalog (or PIM – Product Information Management) is and what it is good for. Today’s question is:  who actually needs such a tool and why.

Before we get to the more complex scenarios such as banks, comparison engines, telco or on-line markets, let’s start with the simplest one – an e-shop. There are basically two options. An e-shop may either have its product offer
“hard wired” and hard coded on its website (a rather absurd idea) or the offer can change and this implies there must exist in the e-shop infrastructure a system maintaining the offer. And that is what we are talking about—a product catalog, also referred to as a PIM (Product Information Management) system.

What can such a thing look like? In its simplest form a product catalog can be implemented as a set of tables in a relation database. Upon each query of an
e-shop customer the database will either look up the relevant product list or detailed information about a specific item.

Where can one obtain such a system? The easiest way of course is to add a few new tables to an existing database and insert the data in them. If we put a new type of goods on offer next month, we just add a new table. And then another. And the programmers will be swamped with work and we will have to wait a few days for each new product (or a few weeks if the programmers happen to be occupied with some other project). This highlights the first important characteristic of a product catalog – flexibility. The product offer of (not only)
e-shops changes in time and a catalog must be sufficiently flexible for such changes. To call a software programmer for each such change is expensive and time consuming.

That is why we will ask the programmer to give us the opportunity to add new product information by ourselves without having to call him each time to give us a hand. The programmer will add another table for generic attributes which will allow us to add a new field with any content at any time to any item. Very nice. But only for a while. As the product offer grows, the system has to go through more and more information in these new fields. And gets slower and slower … This brings us to the second important parameter of a product catalog – speed. Customers shopping on a mobile phone will not want to wait several seconds for our system to gather the necessary information and display it to them.

Let’s get real. We have presented two examples of a bad solution. In reality no sane programmer (or most of those insane either) would try to design the system in a similar way. The above-given examples were intentionally far-fetched just to show the potential issues. Issues which are actually very real – and will in reality appear later and in a more complex way. A product catalog needs to be flexible and fast at the same time. It has to be flexible so that we can widen our offer and introduce new products. It has to be fast so that our customers do not have to wait long. These two requirements are to a large degree opposed to each other. This is not true for e-shops and product catalogs only- this applies to software in general (and not only to software).

A real solution to this issue is not that simple. It is hidden in the fact that speed and flexibility are needed in different situations. While flexibility is important for the preparation of product descriptions, speed is critical for their delivery to customers. Solution? A product catalog has two separate components – designer and runtime. The designer component is intended for product descriptions and what it provides in the first place is flexibility. It is used only by a few people and its speed is not such an issue. The runtime component obtains finished data from the designer and stores them in a structure optimal for fast provision to customers. Each component is built on totally different technologies optimal for their specific purpose.

By becoming aware that speed and flexibility are needed in different situations we have resolved a seemingly impossible problem. And in addition, we have acquired a bonus of scalability. Now that we have two components (designer and runtime) and an interface between them, we can relatively easily connect several runtime components to one designer. If our e-shop grows, we just launch another runtime component which will receive existing data from the designer via the existing interface. The government have locked the people in their homes and they rush into our e-shop? We just launch another runtime component and it’s done. We can focus on real problems such as storage and transport instead.

These are three of the four basic product catalog parameters – information delivery speed, data model flexibility and scalability. The last criterion is its integration to other systems. A catalog works with information a large volume of which already exists somewhere. Manual copying is not a relevant approach to consider. A catalog needs to automatically receive product offer information from the other systems that already have this information – ERP, production, warehouse, subcontractors. Very often one catalog is integrated into several such systems. It merges the information from these systems (for instance through EAN), supplements it with additional information inserted by users (pictures, calculations) and serves it swiftly to an e-shop or other sales channels.

In relation to ERP a question suggests itself:

Why do we need a smart product catalog when most of the information is already available in the ERP system?

There are two primary reasons:

• ERP handles a lot of various agendas which do not have to be dealt with in a catalog – finances, HR, purchase, dispatching or customer support. In addition, it is optimized for work by users via a user interface. On the other hand, it is not optimized for speed of provision of information to sales channels. In such a scenario an ERP system significantly lags behind in terms of performance. A product catalog is designed for fast provision of information to sales channels and in most cases has a runtime component which is specially designed for this purpose and which has information ready in advance for fast outward distribution. The difference is negligible for a few customers, but it is considerable in case of thousands. With higher numbers, an ordinary ERP system is in death throes.
• An ERP is usually a complex and therefore expensive system. As such, it is hard to scale if the e-shop operations start to grow. Increasing the ERP system performance usually means the purchase of an additional server and new expensive licenses. On the other hand, inflating a product catalog means the launch of another runtime component (usually in a cloud or on a virtual server) which connects to an existing designer component, extracts information from it and within a few minutes it provides data to customers.

Additional reasons which should not be forgotten include:

• An ERP system is not meant for sale and usually does not give support to various sales scenarios. Such scenarios include for instance a dynamic pricing policy based on various time-limited sales events or current volumes of inventory. Or the creation of product packages of existing products which can subsequently be sold as new products (and may be composed of the products of several suppliers).
• An ERP system usually contains large amount of product information but not all of it. For sales processes we need to integrate other systems as well anyway (either the company’s own systems or supplier systems) and unify product information from various sources together. In such a situation, it is simpler to do so in a dedicated system rather than through an expensive CRM modification.

This is probably hidden somewhere inside our e-shop

Exactly.  An overwhelming majority of “box” e-shop solutions contain some form of a product catalog. For a small e-shop, this is the best solution. We’ll get all of it in one piece and don’t have to bother with integration. For larger e-shops or ones which plan to grow and expand in future, this is a fundamental limitation. If a certain part of an e-shop box is no longer adequate (such as for the expansion on a new market), the entire system has to be replaced. Therefore, medium and large e-shops increasingly prefer a modular system composed of independent components to a monolithic system. We will discuss the advantages and disadvantages of a monolithic and a modular solution later.

To conclude and summarize: An e-shop needs to receive data from somewhere. This is what a product catalog is for. An ERP system contains product data but their provision to an e-shop is ineffective (slow and expensive). A smart product catalog has to be fast, flexible, scalable and integrated into other systems. This enables growth and expansion.

Next time we will, finally, get to some examples of smart product catalog applications in areas other than e-shops. We’ll have a look at banks, comparison engines, mobile operators, energy and other sectors.