Building a Powerful Retrieval-Augmented AI Chatbot for WordPress

I’ve been developing ContentOracle AI Chat, a plugin that adds a retrieval-augmented ai chatbot to WordPress websites for the past six months. Today, I’d like to take a look at the main features of my new plugin, and share some lessons learned and problems solved in the process of building a powerful retrieval-augmented chatbot for WordPress.

Give it a Try

You can try the plugin out on this site, by visiting this page.

Powerful Features

Let’s begin by taking a look at the main features of my plugin, ContentOracle AI Chat.

No-Code

Throughout the course of my work with my clients, one thing that I noticed was that many small and medium sized businesses want to use AI to improve their customer service and site UX, but they lack the time and knowledge to make it happen. That’s why one of the pillars of the software I wanted to build was a no-code integration. To that end, I implemented my software in to parts: an easy-to-install, minimally technical WordPress plugin, and an api that it automatically connects to without any configuration apart from an API token.

The owner of a site using my plugin never has to know the api exists, as the plugin abstracts that behind the scenes, and the plugin integrates directly with the block editor to place the UI elements needed to facilitate the chat experience.

Highly Customizable

Thanks to the WordPress administrative dashboard and site editor, every aspect of the plugin can be customized to suit a particular user’s needs. You can use the prompt settings dashboard to customize the voice, vocabulary, and information provided to the ai chat agent. The method used to retrieve content (more on this later) can be configured on the embeddings settings page. Meanwhile, all the UI elements that users of the website will see can have their colors, text, and other visual features customized on a case-by-case basis using the block editor.

Content-Aware

At its core, my plugin is an implementation of retrieval-augmented generation (using information from your website to generate responses to your visitors) on top of WordPress. And uniquely, we require no integration with any third-party services. That means no vector database integration, or any other such service.

In its responses, the retrieval-augmented ai chatbot doesn’t just use information from the most relevant posts… it cites and links to each source it used right where it used it, and can provide inline calls to action and recommendations to help connect your site visitors with the content they are looking for. The type of content it cites and recommends can be configured in the general and prompt settings of the plugin (again, no coding required!).

Main Challenges

Building my retrieval-augmented ai chatbot for WordPress has been a true adventure that has vastly stretched my software development abilities. Let’s take a look at some of the biggest challenges I faced, and what I did to solve those problems.

Response Streaming

If you use any of the major online ai chat services, such as Grok or ChatGPT, one thing you will notice is that the ai agents always stream their responses. This means that the ai responses to your query come in piece-by-piece, or, to be more precise, token-by-token. This reduces the time from when users send their message, to when they get a response back. I wanted to implement something similar in my plugin.

Click here to see a streamed chat on the installation on this site!

This proved to be quite the challenge. Most ai APIs give you the option to stream a response back. I used this. The problem I ran into, however, was that in each packet, I was not guaranteed to receive a full json object. Sometimes, I would get partial objects, like ‘{“key”:”va’, where the next packet would complete the object.

To solve this, I implemented a class-based solution on my api in php that contains a buffer. The body content of each packet is appended to that buffer whenever a new packet arrives. Then, I scan from the beginning of the buffer to the end, counting the number of { and } that I see. I kept a score, starting at 0. Seeing a { gave +1, and a } gave -1. That way, whenever the “score” reached 0 (after the beginning of course), I knew I had received a full json packet. Then, I spliced it out of the buffer, and forwarded it on to the client site’s server, which would then forward it to the client who made the chat request.

Semantic Search in MySQL

One of the key features that sets my app apart from other options out there is my implementation of semantic search (matching relevant content to user queries by meaning, not keywords) via text embeddings without using a specialized vector database. A few benefits follow from this:

• Users of the plugin don’t have to spend time integrating there WordPress site with a vector database.
• Site content, user queries, and other data is not shared with a third party (it stays between your site and the ContentOracle AI api.

Now, the obvious challenge here is how to efficiently scale a vector search to a large WordPress site with many posts and products. The brute force method would be to embed the query from the user, and then use a distance or similarity metric such as Euclidean distance or cosine similarity. But, this does not scale well at all to sites with lots of content. That would become very slow on such a site.

Luckily, I was able to come up with a solution. By using binary codes based on the embeddings to generate related candidates more efficiently, I was able to drastically limit the number of cosine similarity calculations I had to do to find closely related embeddings. This, in turn, meant that my semantic text matching between user queries and site content improved drastically.

Extracting Block Supports Styles

When building custom blocks for the block editor, one neat feature that WordPress has is block supports. Block supports are certain preset style options that a block can support, such as text color, font size, or border thickness. When you use a certain block support, WordPress automatically generates inputs that editors can use to configure that style.

In simple blocks, block supports can be applied to a block using the useBlockProps hook on the outermost html tag. But, in more complex blocks that need different styles applied to different elements, this approach does not work.

It is an absolute nightmare to try to manually extract singular styles from block attributes. The attributes object that stores all block styles is a nested array where the values for each style are stored. But, there is little rhyme or reason to the locations in this array where style values are stored. And, depending on certain factors, it can be tricky to tell whether the provided value should be applied as an inline style, or as a class tag.

In order to solve this problem, I actually built my own php package to extract individual styles from the overall attributes array, one at a time. That way, I could pull out button color to apply to just the send button, while leaving the background color of the main block element to be controlled by the background color attribute. Feel free to click here for more details about how I did it.

Conclusion

ContentOracle AI Chat is a powerful implementation of a retrieval-augmented ai chatbot for WordPress. It’s no-code, highly customizable, and content-aware nature make it extremely easy for businesses to get started in using powerful modern ai feature to enhance their site UX. And it was not easy to get it to that state. A host of problems, including solving the issues of response streaming, semantic search, and block style extraction pushed me to grow as a developer more than any project I have worked on. And at the end of the day, that is what working on exciting projects like this is all about.

Want to Give it a Try?

Do you want a retrieval-augmented ai chatbot on your WordPress site? Then ContentOracle AI Chat is for you! Begin by trying it out on this site, and explore how it uses information from my site to answer your questions. After that, click here for an informative setup guide. And don’t worry, it is 100% free to get started… no credit card info required!