Boosting Your Blog Recommendations with Flowise & Weaviate: A Step-by-Step Guide

Table of Contents

Introduction

🎉 Ever wondered how to amp up your website’s blog recommendations? Let’s dive into a fun and effective integration of Flowise and Weaviate, bringing your website’s blog recommendations to a whole new level. Why, you ask? Well, here’s how this duo will work wonders for you:

  • Enhancing User Experience: More relevant reads for your audience.
  • Increasing Page Views and Time Spent: Keep readers engaged longer.
  • Driving Conversions: Guide readers towards desired actions.
  • Building Loyalty: Serve consistent quality content.
  • Leveraging Data: Use the power of analytics to refine recommendations.

Alright, let’s roll up our sleeves and dive into the good stuff! And don’t fret, we’ll keep things hands-on — less theory, more action. If you’ve landed here, I bet you’ve heard about Flowise and Weaviate, right? 😉

Step 1: Gathering Blog Data

First things first, we need the blog posts from your website. Here’s an example using blogs from irocket.services.

The following function, get_page_details, will be our magic wand to retrieve details of each blog post:

import requests
from bs4 import BeautifulSoup
import uuid


def get_page_details(url):
    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'}

    response = requests.get(url, headers=headers)
    if response.status_code != 200:
        return None

    soup = BeautifulSoup(response.content, 'html.parser')

    # Get the title of the page
    title = soup.title.string if soup.title else None

    # Get the description
    description_tag = soup.find('meta', attrs={'name': 'description'})
    if not description_tag:
        description_tag = soup.find('meta', attrs={'property': 'og:description'})
    description = description_tag['content'] if description_tag and 'content' in description_tag.attrs else None

    # Generate a UUID
    id = str(uuid.uuid4())

    return {
        "id": id,
        "title": title,
        "description": description,
        "url": url
    }

Step 2: Storing Blog Data

Before moving ahead, let’s save our collected blog details in a neat JSON file. This will come in handy later when we integrate with Weaviate:

import random
import time
import json

data = []

# Opens the file. The 'r' stands for 'read'.
with open('yourwebsite_blog_urls.txt', 'r') as file:

    # Iterates over each line in the file
    for line in file:
        # The line contains a trailing newline character that you probably don't want
        url = line.strip()
        print(url)

        # Get the details of the page
        data.append(get_page_details(url))

        # Sleep for a random amount of time between 1 and 3 seconds
        time.sleep(random.randint(1, 3))
        print('-------------------')

# Open the output file ('w' stands for 'write')
with open('yourwebsite_blog_data.json', 'w') as f:

    # Use json.dump to write data to the file
    json.dump(data, f)

Step 3: Weaviate Integration

Now that we have our data, it’s Weaviate’s time to shine. Start by setting up a free sandbox on Weaviate. Make sure to keep the Cluster URL and Cluster API Key handy; we’ll be using them soon.

Importing Weaviate Client:

Let’s get the Weaviate client up and running:

!pip install weaviate-client

import weaviate

client = weaviate.Client(
url="https://xxxxxx.weaviate.network", # Replace with your Cluster URL
auth_client_secret=weaviate.AuthApiKey(api_key="CLUSTER_API_KEY"), # Replace w/ your Weaviate instance API key
additional_headers={
"X-OpenAI-Api-Key": "OPENAI_API_KEY" # Replace with your inference API key
}
)

Setting Up Class Object:

Alrighty, now we’re going to set up a class object for our data:

client.schema.delete_class("irocket_openai") # Replace it with your own class name

# Open AI Vectorizer
class_obj = {
"class": "irocket_openai", # Replace it with your own class name
"description": "A class called document",
"vectorizer": "text2vec-openai",
# If set to "none" you must always provide vectors yourself. Could be any other "text2vec-*" also.
"moduleConfig": {
"text2vec-openai": {
"model": "ada", # Can also be any public or private Hugging Face model.
"modelVersion": "002",
"type": "text",
"options": {
"waitForModel": True
}
}
}
}

client.schema.create_class(class_obj)

Time for Import!

With everything set up, let’s import our blog data into Weaviate:

# Read the JSON file

import json

# Create the objects

with open('yourwebsite_blog_data.json', 'r', encoding='utf-8') as file:
data = json.load(file)

client.batch.configure(
batch_size=100, # Specify the batch size for auto batching
# num_workers=2, # Maximum number of parallel threads used during import
# dynamic=True, # Weaviate will dynamically adjust the batch size
)

with client.batch as batch:
# Batch import all Questions
for i, d in enumerate(data):
print(f"importing blogpost: {i + 1}")
    properties = {
        "title": d["title"],
        "url": d["url"],
        "description": d["description"]
    }

    batch.add_data_object(
        properties,
        "irocket_openai", # Replace with your own class
    )

Step 4: Testing Our Setup

Moment of truth! Let’s test our integration and see if our rocket is ready to launch 🚀:

response = (
client.query
.get("irocket_openai", ["url", "title", "description"])
.with_near_text({
"concepts": ["content syndication"]
})
.with_limit(4)
.with_additional(["distance", "id"])
.do()
)

response

If you got a response, congrats! You’re ready to integrate this with Flowise.

Step 5: Bring on the Flow with Flowise

So, you’ve gathered your data and integrated Weaviate. Next up: Setting up the flow in Flowise.

Now go to Flowise, and design a flow resembling the diagram below.

Memory: you can use either Buffer Memory or Motorhead Memory

ChatOpenAI: you need to use any model with 0613 (gpt-3.5-turbo-0613), we need to use OpenAI function calling in this.

OpenAI Function Agent Node:

Blog Recommendation Flow

Custom Tool: Add the following info as per your use case

Tool Name: Must be in small capital letters with underscore (e.g. blog_recom)

Tool Description: Description of what the tool does. This is for ChatGPT to determine when to use the tool

Output Schema: Add the query parameter that needs to be used for calling the function. Like: show me blog posts for Community. In this, community will be considered as query to search the similar blog posts within your data.

Here’s a sneak peek into the function that makes it all happen:

/*

* You can use any libraries imported in Flowise

* You can use properties specified in Output Schema as variables. Ex: Property = userid, Variable = $userid

* Must return a string value at the end of function
*/

const fetch = require('node-fetch');
const myQuery = $query;
const url = 'https://xxxxx.weaviate.network/v1/graphql'; // Replace with your CUSTER_URL
const options = {
'method': 'POST',
'headers': {
'Authorization': 'Bearer CLUSTER_API_KEY', // Replace with your CLUSTER_API_KEY
'X-OpenAI-Api-Key': 'OPENAI_API_KEY',
// Replace with your openai api key
'Content-Type': 'application/json'
},
body: JSON.stringify({
query: { Get { Irocket_openai(limit: 3, nearText: { concepts: ["${myQuery}"] }) { title description url _additional { id distance } } } },
variables: {}
})
};
try {
const response = await fetch(url, options);
const text = await response.text();
return text;
} catch (error) {
console.error(error);
return '';
}

Here is the final config for custom tool

Custom Tool Config

Test Drive Time!

Configured your custom tool? High-five! 🙌 Now, take a leisurely stroll to the chat button. Strike up a casual chat and type a query like, “show me blogs about [your favorite topic].”

If the system showers you with blog recommendations, break into your happy dance! 🕺💃 You’ve just blended the prowess of Flowise and Weaviate to offer standout blog recommendations.

Now grab the iframe and paste that into your website code.

Wrapping It Up

There you have it! A seamless, efficient way to cater to your readers’ unique tastes and keep them coming back for more. The future of blog recommendations is bright, and with tools like Flowise and Weaviate, the sky’s the limit.

🎈 Enjoyed this guide? Share it with fellow bloggers and content creators! And don’t forget to bookmark us for more tech magic and fun tutorials. Stay curious and keep exploring! 🌍 🔍

If you have questions or just want to chat, don’t hesitate to connect with me on LinkedIn. Always here to help and share more insights! 🌐 🤝