Introduction
š Ever wondered how to amp up your websiteās blog recommendations? Letās dive into a fun and effective integration of Flowise and Weaviate, bringing your websiteās blog recommendations to a whole new level. Why, you ask? Well, hereās how this duo will work wonders for you:
- Enhancing User Experience: More relevant reads for your audience.
- Increasing Page Views and Time Spent: Keep readers engaged longer.
- Driving Conversions: Guide readers towards desired actions.
- Building Loyalty: Serve consistent quality content.
- Leveraging Data: Use the power of analytics to refine recommendations.
Alright, letās roll up our sleeves and dive into the good stuff! And donāt fret, weāll keep things hands-on ā less theory, more action. If youāve landed here, I bet youāve heard about Flowise and Weaviate, right? š
Step 1: Gathering Blog Data
First things first, we need the blog posts from your website. Hereās an example using blogs from irocket.services.
The following function, get_page_details, will be our magic wand to retrieve details of each blog post:
import requests
from bs4 import BeautifulSoup
import uuid
def get_page_details(url):
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'}
response = requests.get(url, headers=headers)
if response.status_code != 200:
return None
soup = BeautifulSoup(response.content, 'html.parser')
# Get the title of the page
title = soup.title.string if soup.title else None
# Get the description
description_tag = soup.find('meta', attrs={'name': 'description'})
if not description_tag:
description_tag = soup.find('meta', attrs={'property': 'og:description'})
description = description_tag['content'] if description_tag and 'content' in description_tag.attrs else None
# Generate a UUID
id = str(uuid.uuid4())
return {
"id": id,
"title": title,
"description": description,
"url": url
}Step 2: Storing Blog Data
Before moving ahead, letās save our collected blog details in a neat JSON file. This will come in handy later when we integrate with Weaviate:
import random
import time
import json
data = []
# Opens the file. The 'r' stands for 'read'.
with open('yourwebsite_blog_urls.txt', 'r') as file:
# Iterates over each line in the file
for line in file:
# The line contains a trailing newline character that you probably don't want
url = line.strip()
print(url)
# Get the details of the page
data.append(get_page_details(url))
# Sleep for a random amount of time between 1 and 3 seconds
time.sleep(random.randint(1, 3))
print('-------------------')
# Open the output file ('w' stands for 'write')
with open('yourwebsite_blog_data.json', 'w') as f:
# Use json.dump to write data to the file
json.dump(data, f)Step 3: Weaviate Integration
Now that we have our data, itās Weaviateās time to shine. Start by setting up a free sandbox on Weaviate. Make sure to keep the Cluster URL and Cluster API Key handy; weāll be using them soon.
Importing Weaviate Client:
Letās get the Weaviate client up and running:
!pip install weaviate-client
import weaviate
client = weaviate.Client(
url="https://xxxxxx.weaviate.network", # Replace with your Cluster URL
auth_client_secret=weaviate.AuthApiKey(api_key="CLUSTER_API_KEY"), # Replace w/ your Weaviate instance API key
additional_headers={
"X-OpenAI-Api-Key": "OPENAI_API_KEY" # Replace with your inference API key
}
)Setting Up Class Object:
Alrighty, now weāre going to set up a class object for our data:
client.schema.delete_class("irocket_openai") # Replace it with your own class name
# Open AI Vectorizer
class_obj = {
"class": "irocket_openai", # Replace it with your own class name
"description": "A class called document",
"vectorizer": "text2vec-openai",
# If set to "none" you must always provide vectors yourself. Could be any other "text2vec-*" also.
"moduleConfig": {
"text2vec-openai": {
"model": "ada", # Can also be any public or private Hugging Face model.
"modelVersion": "002",
"type": "text",
"options": {
"waitForModel": True
}
}
}
}
client.schema.create_class(class_obj)Time for Import!
With everything set up, letās import our blog data into Weaviate:
# Read the JSON file
import json
# Create the objects
with open('yourwebsite_blog_data.json', 'r', encoding='utf-8') as file:
data = json.load(file)
client.batch.configure(
batch_size=100, # Specify the batch size for auto batching
# num_workers=2, # Maximum number of parallel threads used during import
# dynamic=True, # Weaviate will dynamically adjust the batch size
)
with client.batch as batch:
# Batch import all Questions
for i, d in enumerate(data):
print(f"importing blogpost: {i + 1}")
properties = {
"title": d["title"],
"url": d["url"],
"description": d["description"]
}
batch.add_data_object(
properties,
"irocket_openai", # Replace with your own class
)Step 4: Testing Our Setup
Moment of truth! Letās test our integration and see if our rocket is ready to launch š:
response = (
client.query
.get("irocket_openai", ["url", "title", "description"])
.with_near_text({
"concepts": ["content syndication"]
})
.with_limit(4)
.with_additional(["distance", "id"])
.do()
)
responseIf you got a response, congrats! Youāre ready to integrate this with Flowise.
Step 5: Bring on the Flow with Flowise
So, youāve gathered your data and integrated Weaviate. Next up: Setting up the flow in Flowise.
Now go to Flowise, and design a flow resembling the diagram below.
Memory: you can use either Buffer Memory or Motorhead Memory
ChatOpenAI: you need to use any model with 0613 (gpt-3.5-turbo-0613), we need to use OpenAI function calling in this.
OpenAI Function Agent Node:
Custom Tool: Add the following info as per your use case
Tool Name: Must be in small capital letters with underscore (e.g. blog_recom)
Tool Description: Description of what the tool does. This is for ChatGPT to determine when to use the tool
Output Schema: Add the query parameter that needs to be used for calling the function. Like: show me blog posts for Community. In this, community will be considered as query to search the similar blog posts within your data.
Hereās a sneak peek into the function that makes it all happen:
/*
* You can use any libraries imported in Flowise
* You can use properties specified in Output Schema as variables. Ex: Property = userid, Variable = $userid
* Must return a string value at the end of function
*/
const fetch = require('node-fetch');
const myQuery = $query;
const url = 'https://xxxxx.weaviate.network/v1/graphql'; // Replace with your CUSTER_URL
const options = {
'method': 'POST',
'headers': {
'Authorization': 'Bearer CLUSTER_API_KEY', // Replace with your CLUSTER_API_KEY
'X-OpenAI-Api-Key': 'OPENAI_API_KEY',
// Replace with your openai api key
'Content-Type': 'application/json'
},
body: JSON.stringify({
query: { Get { Irocket_openai(limit: 3, nearText: { concepts: ["${myQuery}"] }) { title description url _additional { id distance } } } },
variables: {}
})
};
try {
const response = await fetch(url, options);
const text = await response.text();
return text;
} catch (error) {
console.error(error);
return '';
}Here is the final config for custom tool
Test Drive Time!
Configured your custom tool? High-five! š Now, take a leisurely stroll to the chat button. Strike up a casual chat and type a query like, āshow me blogs about [your favorite topic].ā
If the system showers you with blog recommendations, break into your happy dance! šŗš Youāve just blended the prowess of Flowise and Weaviate to offer standout blog recommendations.
Now grab the iframe and paste that into your website code.
Wrapping It Up
There you have it! A seamless, efficient way to cater to your readersā unique tastes and keep them coming back for more. The future of blog recommendations is bright, and with tools like Flowise and Weaviate, the skyās the limit.
š Enjoyed this guide? Share it with fellow bloggers and content creators! And donāt forget to bookmark us for more tech magic and fun tutorials. Stay curious and keep exploring! š š
If you have questions or just want to chat, donāt hesitate to connect with me on LinkedIn. Always here to help and share more insights! š š¤
