FWIW, the AI features are not used to provide search results; they are all on-demand and triggered by the user (via Quick Answer, or Universal Summarizer, or the "discuss this site" feature).

The founder is well aware of the problems with AI and that is taken into account when deciding how to use it in Kagi.

See this link: https://blog.kagi.com/kagi-ai-search#philosophy

Generative AI is a hot topic, but the technology still has flaws. Critics of AI warn that “[AI] will degrade our science and debase our ethics by incorporating into our technology a fundamentally flawed conception of language and knowledge”.

From an information retrieval point of view, relevant to our context of a search engine, we should acknowledge the two main limitations of the current generation of AI.

Large language models (LLMs) should not be blindly trusted to provide factual information accurately. They have a significant risk of generating incorrect information or fabricating details (confabulating). This can easily mislead people who are not approaching LLMs pragmatically. (This is a product of auto-regressive nature of these models where the output is predicted one token at a time, and once it strays away from the “correct” path, for which the probablity grows exponentially with the length of the output, it is “doomed” to the end of output, without the ability to plan ahead or correct itself).

LLMs are not intelligent in the human sense. They have no understanding of the actual physical world. They do not have their own genuine opinions, emotions, or sense of self. We must avoid attributing human-like qualities to these systems or thinking of them as having human-level abilities. They are limited AI technologies. (In a way, they are similar to how a wheel can get us from point A to point B, sometimes much more efficiently than human body can, but it lacks the ability to plan and the agility of human body to get us everywhere a human body can)

These limitations required us to pause and reflect on the impact on search experience, before incoporating this new technology for our customers. As a result, we came up with an AI integration philosophy that is guided by these principles:

AI should be used in closed, defined context relevant to search (don’t make a therapist inside the search engine, for example) AI should be used to enhance the search experience, not to create it or replace it (similar to how we use JavaScript in Kagi, where search still works perfectly fine when JS is disabled in the browser) AI should be used to the extent that it enhances our humanity, not diminish it (AI should be used to support users, not replace them)