The word “Google” became a verb in 2006, when it found its way into the Merriam-Webster dictionary. But like all shifts in language, it had caught on long before official recognition caught up to everyday usage.
That happened because search engines like Google had caused a seismic shift in the way humans access information. The algorithms read linking behavior between pages to enable information retrieval on an unprecedented scale. And it was all wrapped up in a streamlined, easy-to-use interface that gave users (even non-technical ones) what they wanted, fast: lists of websites semantically linked to their search query.
Bad news for public libraries, but amazing for almost everything else. It’s become so ubiquitous that most of us struggle to imagine the “before time”.
Why you might not be Googling for much longer (or at least not in the same way)
None of this is news to anyone, so why am I talking about it? Simple: we’re now at another inflection point, heralded by the arrival of powerful new technology. Generating lists of websites ranked according to relevance is useful, but it’s just the starting point. The user still needs to sift, evaluate, analyze and synthesize that information.
And as Google itself has acknowledged, there is now a pressing need to go further. Lists of websites are not enough: users want “deeper insights and understanding.” In this post, we’re taking a look at how generative AI is changing how we search, paving the way to richer insights and better decision making.
But first, a brief diversion on semantic analysis.
Semantics: building search queries, word by word
Every word has what is known as a semantic domain, a range of other words that connect to a shared substrate. For example, the word “vehicle” is part of a semantic domain that includes words like car, plane, ship, and many more. “Vehicle” can also signify a means of achieving something, particularly in medical and scientific applications.
Search engines operate by identifying pages that contain keywords in the user’s search query. But one of the conventional limitations of keyword-based search is that it lacks the human intuition to situate the right part of the semantic domain. So, “vehicle”, taken out of context, can recall the entire semantic domain of “things that travel”, and the entirely different domain of “biological component that delivers drugs”. That’s too broad to be useful.
But there are many ways to refine a search query to get narrower and more relevant results:
- Quotation marks around a phrase to fetch exact matches of that phrase: “the costs of drug discovery” will fetch only results that contain those words and in that order.
- Hyphens to exclude certain results: if you want to get results about unicorn companies, but don’t want to deal with pages that talk about magical creatures, you can use: unicorn -creature
- If you need information in a specific format, you can ask Google to only return results in that format, as in: tech talent shortage filetype: pdf
From keyword search to generative AI: bridging the human/machine gap
Behaviors like this simulate the context-driven decision-making that we take for granted when we communicate with other humans using a natural language. And they are behaviors that generative AI can now automate.
Generative AI reads search queries using NLP algorithms that interpret language in the way that we do as humans – but on a much larger scale. This allows it to understand the intent behind a user’s query, and create content that matches it. In other words, generative AI makes that crucial step that search engines couldn’t make – interpreting the information on web pages for the human user. And, much like search engines did in the early 21st century, generative AI like ChatGPT combines this new power with a user-friendly interface that people love to engage with.
Why it isn’t quite over yet for search engines
As others have noted, search engines still have the edge over chatbots when it comes to crawling the web to find up-to-date information. Chatbots are trained on large but static data sets. For researchers in particular, this is a major limitation.
And much like search engines, the output of generative AI is only as good as the input it receives. In other words, it takes knowledge and skill to create an effective prompt for the AI to use. We can all expect to hear a lot more about prompt engineering as an in-demand competency as the AI revolution continues to unfold.
What’s needed is a way to leverage the real-time information gathering of a search engine, with the intuition and nuance of generative AI and NLP.
Similari: AI-based insights for the future
By leveraging the latest advances in AI, ML and NLP, Similari equips researchers and business development professionals with continually up-to-date insights on their industry and their competitors – all without the need for complex human-led prompt engineering.
Because it learns the habits and preferences of human users over time, Similari can refine searches by identifying the most relevant terms from a semantic domain, and tailoring results to specific business needs.
Get in touch to learn how Similari combines the intelligence and power of an AI chatbot, with the flexibility and 360-degree line of sight of a search engine.