Why and how hybrid search boosts e-commerce sites
AI is affecting almost everything, including e-commerce search functions. E-commerce vendors are increasingly considering replacing traditional search capabilities with newer, AI-powered alternatives that are better at delivering relevant results in many cases.
However, there is a problem: while AI search might be extremely useful in certain situations, it also has significant downsides. AI search typically uses large language models, which can sometimes produce fantastic results but are also susceptible to hallucination flaws, resulting in perplexing product listings and making it difficult for shoppers to find what they’re looking for.
That is why implementing hybrid search capabilities is a superior option, as it provides e-commerce companies with the best of both worlds—traditional searches paired with AI-powered search results.
Classical search gives you more control and predictability over the search results than AI does. Thus, by combining classical and AI-powered searches in a hybrid search approach, it is feasible to profit from AI search where it makes sense while still exploiting classical search capability where it is a better match.
Here’s why most businesses should use a hybrid search strategy rather than relying solely on AI search, as well as practical tips for making hybrid search work.
What is a hybrid search?
Hybrid search is a website search feature that uses multiple techniques to interpret site visitors’ search queries and deliver relevant results. Typically, those techniques involve a combination of the following:
Classical search, also known as sparse vector or lexical search, This search technique aims to match specific search terms or keywords with corresponding results. For example, if you search for “black dress,” a lexical search will return results that include black dresses based on keyword matching. However, because this type of search is based on simple pattern matching, this query may also display items that simply have “black” in them, such as black dress shoes.
AI search, also called dense vector search. While specific approaches to AI search implementation differ, most AI search engines use language models to associate search terms with semantically related words or phrases. For example, a search for “lawn mower” using an AI engine may yield results that include weed whackers because the terms are semantically similar. A traditional search would not be able to produce these results unless you created an index that specifically linked the keywords “lawn mower” and “weed whacker.”
Hybrid search allows you to direct queries to both types of search engines and see a mix of results from each.
The need for hybrid search.
In some respects, hybrid search may seem like an unnecessarily complicated approach to search functionality.
If today’s AI search engines are capable of interpreting shopper intent without requiring the tedious development of complex search indexes to link search terms to the most relevant results, why wouldn’t e-commerce sites simply replace classic search engines with modern AI search?
The challenges of AI search
The answer is that, while AI search is great in many contexts, it has several shortcomings.
One major weakness of AI search is that it tends to perform poorly when visitors are searching for a very specific product and enter search terms unique to that product—like a model number that consists of a string of letters and numbers. In that case, an AI search engine would be likely to display a seemingly random list of products because it would not be able to establish semantic relationships between the model number (a term it has likely not encountered before) and the product the shopper is actually looking for.
There is also a risk that AI search engines will draw conclusions that lead to search results that are totally irrelevant to the shopper. For example, an AI model might associate the word “black” with “gray” because the terms are semantically related. In turn, someone who searches for “black dress” might see search results that include gray dresses, which is likely not what the user asked for.
In more extreme cases, AI models may generate search results that make little sense at all. This can happen when flaws in model design or training data lead to hallucinations, or events that cause a model to believe two terms are related when in fact they are not. If a search for “black dress” yields results that include pink pencils, for example, it’s probably because of a model hallucination issue.
AI search also makes it challenging to factor search facets accurately into results. You could potentially do this by using a separate algorithm to filter search results generated by AI, but that requires an additional step that wouldn’t be necessary when using a classical search engine that embeds facets into query processing.
The bottom line here is that when AI search works as intended, the results can be spectacular. But when things go wrong, results can be spectacularly bad—and it’s challenging to anticipate issues because the intricacies of LLMs make it virtually impossible to predict with total accuracy how a model will behave in response to a given search query.
The drawbacks of lexical search
On balance, classical lexical search engines have clear drawbacks, too.
For one, they’re not good at dealing with misspellings in search terms. They also typically can’t generate results for closely related products. Lexical search is exceptionally good at showing results that directly match what a shopper searches for—which is great when the search query is a product model number but less great when a user searches for “book” and receives search results that include book shelves and book bags—products that include the search keyword but are not closely related to what the shopper wants.
So, rather than settling for a type of search engine that excels in some areas but falls short in others, businesses can take advantage of hybrid search in order to generate the best possible search results in every context.
Putting hybrid search into practice
To take advantage of the benefits of hybrid search, retailers should first configure both lexical and AI search engines for their sites. Most sites already have lexical search functionality in place, and AI search features are increasingly becoming part of e-commerce software, so the lift necessary to implement both types of search is not particularly heavy.
From there, enabling hybrid search is a matter of configuring tools that assess each search query and determine whether to process it using classical search, AI search, or a combination of both. Websites can automatically make this determination based on factors such as:
Whether the query is an exact match for any existing lexical keywords, in which case leaning more heavily on classical search typically makes sense.
Whether the query contains any potentially misspelled or unusual terms, in which case AI search is likely to deliver better results.
In cases where the query returns no results from lexical search, running an AI search may help identify closely related products.
Typically, it makes sense to lean primarily on lexical search to generate results for straightforward queries that match closely with products. Meanwhile, AI search can take the lead for “fuzzy” terms or for helping to identify products that might be relevant for a search concept or phrase but whose names and descriptions don’t include the specific words the user has searched for.
If online retailers opt for a mix of classical and AI search results, they also consider how to order the results based on their level of confidence in the effectiveness of each type of search for a given query. For instance, in a situation where AI search might be less reliable because the search query includes a brand name that an AI model is not primed to associate with certain products, the results page could display lexical search results first, with some AI-generated results further down the list in case the lexical search, too, turns out not to be accurate.
Conclusion: Making search and sales easier
Enhancing e-commerce websites with hybrid search functionality provides a balanced approach to overcoming the limitations of both classical and AI-powered search. By integrating the precision and reliability of classical search with the context-aware capabilities of AI search, online retailers can enhance the relevance and accuracy of search results. Hybrid search addresses key challenges like handling exact-match searches, mitigating AI hallucination issues, and improving results for vague or misspelled queries.
Ultimately, the effect of any search strategy should be to make it as easy as possible for shoppers to find the products they’re looking for. As AI search continues to evolve, hybrid search strategies help e-commerce businesses to be adaptable, combining new advancements with proven search techniques to meet the needs of shoppers. Hybrid search techniques give retailers access to more options when processing search queries, which translates to a higher likelihood of displaying what shoppers want to see and getting them to click “buy.”