Building an LLM-driven chatbot for seamless order management

Overview

In today’s digital era, customer expectations for seamless and personalized interactions are at an all-time high. To meet these demands, businesses are turning to innovative solutions such as AI-driven chatbots.

In this tutorial, we will provide a step-by-step guide to building an LLM-based chatbot for order management. Here’s an outline of the topics that will be covered:

• Load and process a sample customer transaction dataset.
• Utilise the OpenAI Language Model (LLM) for interpreting user queries and intentions. Specifically, the LLM is capable of determining whether the user wants to retrieve all placed orders, obtain specific order details, or cancel a particular order.
• Develop an interactive chat interface, created using Panel, that streamlines order management for users.

Here’s a brief demo of the functionality:

The source code for this application can be found here.

Online Retail Dataset

We’ll utilize the online retail dataset available on the UCI Machine Learning Repository. You can access the dataset here. Here’s a brief overview of the dataset as described on the website:

This is a transnational data set which contains all the transactions occurring between 01/12/2010 and 09/12/2011 for a UK-based and registered non-store online retail.

Let’s see a few rows from the dataset:

import pandas as pd

df = pd.read_csv("orders.csv")
df.head()

As observed, the CustomerID field in the dataset is currently stored as a float type. To enhance usability, we’ll convert it to string format. Additionally, we’ll create a list containing all valid CustomerID values to validate input and ensure that the customer exists in our records.

df["CustomerID"] = df["CustomerID"].apply(
    lambda x: str(int(x)) if not pd.isna(x) else ""
)
all_customers = df["CustomerID"].tolist()

Process Flow

The entire process can be divided into the following steps:

1. Initialization: The user is prompted to enter their Customer ID and the Today's Date. The date field is required for order cancellations.
2. User interaction: Users interact with the chatbot by submitting queries specifying actions they want to perform on a particular order ID.
3. Intent detection: The chatbot utilizes the OpenAI Language Model (LLM) to identify the action requested by the user (e.g., cancelling an order, retrieving order details).
4. Action Execution: Based on the detected intent, the chatbot performs the corresponding action:
   • If the intent is to fetch all orders placed by the customer, the chatbot retrieves the relevant order IDs.
   • If the intent is to get specific order details, the chatbot retrieves the details of the specified order.
   • If the intent is to cancel an order, the chatbot checks the eligibility of the order for cancellation (should have been placed in the last 14 days).
5. Response Generation: The responses generated by the chatbot may include confirmation of order cancellation, a list of all orders placed by the customer, or detailed information about a specific order.

The chatbot will continue to interact with the user, responding to additional queries and performing actions as requested.

Initialization

Before initiating interaction with the chatbot, users are required to input both the Customer ID and Today's Date. To streamline the process, a default Customer ID is provided for convenience. Should the user opt for a different Customer ID, we’ll verify its validity by cross-referencing it with the dataset’s records.

if not customerid_input.value:
    customerid_input.value = "15574"
elif customerid_input.value not in all_customers:
    return "Please provide a valid CustomerID"

Intent Detection

Now, let’s dive into the code responsible for discerning the intent of the user query. Presently, our application supports three distinct intents or actions:

1. GET_ORDERS: Retrieve the IDs of all orders placed by the customer.
2. ORDER_ITEM_DETAILS: Obtain details regarding a specific Order ID.
3. CANCEL: Initiate the cancellation process for a particular order.

To guide the OpenAI Language Model (LLM) effectively, we’ll furnish it with a concise set of instructions or prompts, specifying the task at hand and the desired output format.

system_prompt = f"""
        You're a system that determines the invoice number (order number) and the intent in the user query. 

        You need to return only the order number. If no invoice number or order number is found 
        return the string None.
        
        You need to return only the intent and no additional sentences.
        If relevant intent is not found then return the string None.

        Valid intents = ["CANCEL", "GET_ORDERS", "ORDER_ITEM_DETAILS"]
        
        You should return the response as a JSON.
    """

Next, we would use OpenAI’s chat completions API to generate responses from the GPT-3.5-turbo language model. A valid JSON response looks like this:

{"OrderID":"536364", "Intent":"ORDER_ITEM_DETAILS"}

response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        response_format={"type": "json_object"},
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": "Which items were ordered in 536364"},
            {
                "role": "system",
                "content": '{"OrderID":"536364", "Intent":"ORDER_ITEM_DETAILS"}',
            },
            {"role": "user", "content": "Can i cancel the order 458891"},
            {"role": "system", "content": '{"OrderID":"458891", "Intent":"CANCEL"}'},
            {"role": "user", "content": "Which orders have I placed"},
            {"role": "system", "content": '{"OrderID":"None", "Intent":"GET_ORDERS"}'},
            {"role": "user", "content": "Please get my orders"},
            {"role": "system", "content": '{"OrderID":"None", "Intent":"GET_ORDERS"}'},
            {"role": "user", "content": "Order ID 17850"},
            {"role": "system", "content": '{"OrderID":"17850", "Intent":"None"}'},
            {"role": "user", "content": "all products"},
            {"role": "system", "content": '{"OrderID":"None", "Intent":"None"}'},
            {"role": "user", "content": user_query},
        ],
        seed=42,
        n=1,
    )
    output = response.choices[0].message.content
    return json.loads(output)

Get all orders

If the detected intent from the user query is GET_ORDERS, the application proceeds to retrieve the IDs of all orders placed by the customer and then returns the list to the user.

def get_orders():
    """Function to fetch all orders of given customer"""

    customerId = customerid_input.value.strip()
    customer_orders = df.loc[df["CustomerID"] == customerId]
    order_ids = customer_orders["InvoiceNo"].unique().tolist()
    return ", ".join(order_ids)

Get order details

If the intent detected from the user query is ORDER_ITEM_DETAILS, the application verifies that the provided Order ID exists and pertains to the current user. Only upon confirmation of these conditions, the order details are displayed to the user.

def get_order_details(invoice_number):
    """Function to fetch order details"""

    order_details = df.loc[df["InvoiceNo"] == invoice_number]
    order_details = order_details.reset_index(drop=True)
    if len(order_details) == 0:
        return {
            "Found": False,
            "Reason": "This Order ID doesn't exist. Please input a valid order ID",
        }
    customerId = order_details.iloc[0]["CustomerID"]
    if not customerId:
        return {
            "Found": False,
            "Reason": "There is no information on the Customer  "
            "ID for this order. Please try another Order ID",
        }

    if customerId.strip() != customerid_input.value.strip():
        return {
            "Found": False,
            "Reason": "Sorry! The order "
            f"{invoice_number} belongs to a different customer",
        }
    return {"Found": True, "Data": order_details}

Cancel an order

Users are allowed to cancel their own orders if they were placed within the last 14 days from the input Today's Date. Below is the code segment responsible for determining eligibility for cancellation:

try:
    invoice_date = invoice_details.iloc[0]["InvoiceDate"]
    date_object = datetime.datetime.strptime(
        invoice_date.split()[0], "%m/%d/%y"
    ).date()
    date_difference = date_picker.value - date_object
except Exception:
    return create_cancellation_details(
        f"Failed to process " f"Invoice date for order {invoice_number}"
    )

if 14 >= date_difference.days >= 0:
    cancel = {"eligible": True}
    CANCELLATION_CONTEXT_DATA["CancelOrderId"] = invoice_number
    return cancel
else:
    return create_cancellation_details(
        f"Order {invoice_number} not eligible for cancellation. "
        f"We can only cancel orders placed in the last 14 days"
    )

If the order is eligible for cancellation, the user is prompted for confirmation. We manage the context information for order cancellations within a global variable CANCELLATION_CONTEXT_DATA.

CANCELLATION_CONTEXT_DATA = {
    "CancelOrderId": None,
    "CancelConfirmationPending": False,
    "CancelledOrders": [],
}

Once the user confirms Yes, the order is cancelled and added to the CancelledOrders list in CANCELLATION_CONTEXT_DATA.

Chat interface using Panel

For the purpose of this tutorial, we use Panel for creating an interactive chat interface.

chat_interface = pn.chat.ChatInterface(callback=callback, callback_exception="verbose")
chat_interface.send(
    "I am a customer chat assistant and I'll help you perform actions on your order!\n"
    "Please enter your customer ID and current date before further processing.\n\n"
    "You can deploy your own by signing up at https://ploomber.io",
    user="System",
    respond=False,
)

The FastListTemplate is used for creating the layout:

customerid_input = pn.widgets.TextInput(name="Customer ID", placeholder="15574")
date_picker = pn.widgets.DatePicker(
    name="Today's Date", value=datetime.datetime(2011, 6, 20)
)


pn.template.FastListTemplate(
    title="Customer Chatbot",
    sidebar=[app_description, customerid_input, date_picker],
    main=[chat_interface],
    sidebar_width=400,
).servable()

Deployment

We can deploy the application on Ploomber Cloud using the command line interface. With Ploomber Cloud, we can easily deploy our application on the cloud and share it with others, we can manage secrets and automate deployments through GitHub actions.

You first need to create a Ploomber Cloud account here. You also need to generate an API Key.

Install the Ploomber Cloud CLI as follows:

pip install ploomber-cloud

Set your Ploomber Cloud API key:

ploomber-cloud key YOURKEY

Steps for initialising the app:

• Download the dataset and save as orders.csv

• Create a requirements.txt file with the following contents:

panel
openai

• Create a .env file and make sure to include the OpenAI API key:

OPENAI_API_KEY=your_openai_key

• Initialize the project as a panel application by running:

ploomber-cloud init

You should see a ploomber_cloud.json file created. Now deploy the application:

ploomber-cloud deploy --watch

Challenges faced

Few challenges were encountered during the development of this application:

• Despite providing prompts, the OpenAI API occasionally returned responses in an incorrect format, particularly when multiple invalid requests were sent to the server. To address this, thorough validation of the response was necessary.
• Determining the criteria for order cancellation posed a challenge, as all orders were placed within the 2010-2011 period. We resolved this issue by introducing a Today's Date input field.
• Ensuring accurate data conversions and handling of missing values required careful attention throughout the development process.

Conclusion

In this tutorial, we’ve offered an extensive guide to constructing an LLM-based chatbot, addressing components like data processing, intent detection, action execution, and user interface development. By following these steps, developers can create chatbots proficient in understanding user queries, performing actions, and providing tailored responses. Additionally, we’ve demonstrated the deployment process on Ploomber Cloud.