Supplementing your Online Shop with Apache SOLR

Nowadays, if you want to have an online shop, you have a choice between a number of software solutions, from small and slim to large and comprehensive. Oxid eShop, Shopware, Magento, XT-Commerce are a couple of the more well-known solutions. Every vendor tries to provide the best experience for their target audience, so a couple of compromises is made to cover all use cases. All have in common a web-frontend with an underlying database. The latter is one such compromise. A database in itself is a compromise between storage and retrieval. Since all shopping systems come with a built-in search facility based on the database, searching is subject to a compromise.

To cater all but the most basic needs with a search facility, it needs to be tuned to find the right data, be fast at it and present the results in a meaningful manner. That is where Apache SOLR comes into play. Its main focus is on being a search engine, it does not suffer from the drawbacks that databases have.

So why use a specialized search for your shop?

• The customer wants to find, not search. When a user sees a description or designation on the detail page of a product, he expects to reach the product, or similar, by entering what he read into the search box. Standard shop search facilities have a limited number of searchable fields and no logic to determine how the content is to be indexed.
• Data in relational databases usually is normalized to some extent, so some complex relations can be very expensive in regard to memory usage and retrieval speed. Denormalization and redundancy have their own drawbacks. Search engines can be fed data prepared specifically for the retrieval of those relations.
• Searches on the database strain the resources that are needed for data retrieval and storage for the rest of the shopping experience.
• A specialized search solution can cater wishes that are simply not possible with database-dependent solutions.

What exactly can be done with such a search?

• Replacing the traditional search. As stated above, this relieves the database of some of its duties and can, done correctly, improve the performance of the online shop considerably.

• Autosuggest. Usually starting at 2 or 3 characters, a number of possible search strings is suggested. Alternatively previews of searches with the currently entered string can be displayed to give the user a hint whether he needs to further refine his search term.

• Autocomplete. Similar to Autosuggest, this gives the user a guideline for his searches. While typing, the most common completion for the search term is displayed. This can be in regard to search frequency, result set size or even preferences of the shop owner.
• Filtering. Technically not a “search”, it helps the user navigate a large set of products, especially in areas where categorization is difficult or ambiguous. The search engine replaces the database query that determines which products are in a category. That means the user can enter his preferences and only finds products that meet these criteria while he browses the shop
• Instant tagclouds. The queries that generate tagclouds are especially burdensome to a database that is designed to store products and categories. Solr can generate data for a tag-cloud on-the-fly without much overhead.

Why use SOLR?

So when the shop owner has accepted that the database-driven way of solving the described problems has issues, why should he decide to use SOLR? For a start, it’s open-source. There are no license fees, the software can be altered freely, knowledgeable programmers can look up how data is processed. If needed, extra functions can be added, the software can be tuned exactly for the purpose intended.

When not to use SOLR

Although a very powerful tool, there are still some counterarguments. SOLR is written in java, so if your shop runs on PHP, you need a dedicated webserver like jetty or tomcat. This allocates some disk space and a relatively large amount of memory. Next, you need quite a bit of know-how to implement a solution for your particular shop. A customized environment and good integration grade does not come ready-made. To keep control of your data, you need to be able to tune the software environment. For most cases, you also need to take care of filling all data aspects into SOLR that you need for your searches.

Conclusion

Searching facilities in online shops are in most cases based upon partial or unsharp matching or combining attributes, relational databases are not tuned to perform particularly well there. So if your products are not strictly hierarchical, or you have a lot of them, adding SOLR to your software set can greatly enhance the user experience. If you have the knowledge and skills or an apt partner to implement a solution tailored to your needs, this is a profitable supplement to your shop.