Build and deploy an AI-powered API to process order information

Laura Funderburk

Feb 01, 2024 - 19 Min read

Introduction

In previous blogs, we explored how to leverage large language models (LLMs) along with retrieval augmented generation (RAG) to explore their ability to retrieve and answer questions from a knowledge base. We explored this for unstructured data in the form of audio and video, and for semi-structured data in the form of questions and answers to a personality test. In this blog, we will assess an LLMs capacity to retrieve information from a structured data source, and its ability to reason to answer questions about it with the goal of integrating an LLM-based approach for a chatbot application with a focus on order support.

We will develop and deploy an API as our solution. The API will take a question in natural language as input, and return an answer in natural language. The answer should be grounded in the knowledge base and be relevant to the question. The LLM should display its reasoning when answering the question. For this application the knowledge base will consist of a table of data containing purchase order information. The API will be developed using FastAPI, and will be deployed and hosted on Ploomber Cloud. This API can then be used to integrate with a chatbot application.

Deploying the application

We will package our application into .py scripts, a Dockerfile and a requirements.txt file. We will then deploy it on Ploomber Cloud through their command line interface. With Ploomber Cloud, we can easily deploy our application on the cloud and share it with others, we can manage secrets and automate deployments through GitHub actions.

Sample code

All code for this application can be found here.

The complete application contains an app.py file with code for the API, a pipelinehelper.py with code for an LLM-powered pipeline, a Dockerfile and a requirements.txt file. We will go through each of these files in detail.

Chatbot Technology: Understanding the Core and Varieties

Chatbots are software applications designed with an amalgamation of an application layer, a connecting database, and APIs. At their core, chatbots operate on the principle of pattern matching. This method enables them to classify text and curate relevant responses to user inquiries. Essentially, a chatbot’s interaction is dictated by its pre-programmed responses and logic. Notably, chatbots manifest in various forms, each tailored to specific usage scenarios. Predominantly, they are categorized into three distinct types:

Rule-Based Chatbots: As the foundational version of chatbots, these function through predefined user choices. Interaction with such bots involves selecting from available options, following which the chatbot processes the request and responds with options or buttons. Rule-based chatbots excel in addressing common queries but may fall short in handling complex or nuanced inquiries.
Independent (Keyword) Chatbots: Leveraging machine learning, these chatbots surpass the limitations of their rule-based counterparts. By analyzing user inputs and intentions, they craft responses that are not bound to preset options. These bots utilize a blend of customizable keywords and machine learning algorithms to effectively and efficiently address user queries, providing a more dynamic interaction.
NLP (Contextual) Chatbots: These bots combine the strengths of both rule-based and keyword chatbots. Through Natural Language Processing (NLP), they excel in deciphering the context and intent behind user requests. This capability allows them to handle multiple, varied requests from a single user with ease, offering a highly sophisticated and responsive user experience.

In this blog we will explore the latter category through the use of LLMs.

About the data and the application

We will build a RAG-based application that has access to data on purchases. The data we will be working with can be found on the UCI Machine Learning Repository here. According to the data source, it “is a transnational data set which contains all the transactions occurring between 01/12/2010 and 09/12/2011 for a UK-based and registered non-store online retail.”

We can read and explore the data using pandas:

import pandas as pd

# Read data
df = pd.read_csv("data.csv", encoding='latin1')

df.info()

Console output (1/1):

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 541909 entries, 0 to 541908
Data columns (total 8 columns):
 #   Column       Non-Null Count   Dtype  
---  ------       --------------   -----  
 0   InvoiceNo    541909 non-null  object 
 1   StockCode    541909 non-null  object 
 2   Description  540455 non-null  object 
 3   Quantity     541909 non-null  int64  
 4   InvoiceDate  541909 non-null  object 
 5   UnitPrice    541909 non-null  float64
 6   CustomerID   406829 non-null  float64
 7   Country      541909 non-null  object 
dtypes: float64(2), int64(1), object(5)
memory usage: 33.1+ MB

Let’s do a bit of cleanup - in this case we do not want to give the LLM access to entries with no known customer ID, so we will remove them from the data:

# Drop rows with empty CustomerID
df.dropna(subset=['CustomerID'], inplace=True)

# rename columns to all lower case
df.columns = [x.lower() for x in df.columns]

The data contains 8 columns, and 406829 rows where customer ID is not empty, each containing information on purchases made by customers. The columns are:

invoiceno: The invoice number, a unique identifier for each transaction
stockcode: The stock code, a unique identifier for each product
description: A description of the product
quantity: The quantity of the product purchased
invoicedate: The date and time of the transaction
unitprice: The price of the product
customerid: The customer ID, a unique identifier for each customer
country: The country where the customer resides

Observation

For purchases that were cancelled, the invoice number starts with the letter ‘C’ and number of items is negative. We will retain these entries in the dataset.

df[df['invoiceno'].str.contains("C")].head()

Console output (1/1):

	invoiceno	stockcode	description	quantity	invoicedate	unitprice	customerid	country
141	C536379	D	Discount	-1	12/1/2010 9:41	27.50	14527.0	United Kingdom
154	C536383	35004C	SET OF 3 COLOURED FLYING DUCKS	-1	12/1/2010 9:49	4.65	15311.0	United Kingdom
235	C536391	22556	PLASTERS IN TIN CIRCUS PARADE	-12	12/1/2010 10:24	1.65	17548.0	United Kingdom
236	C536391	21984	PACK OF 12 PINK PAISLEY TISSUES	-24	12/1/2010 10:24	0.29	17548.0	United Kingdom
237	C536391	21983	PACK OF 12 BLUE PAISLEY TISSUES	-24	12/1/2010 10:24	0.29	17548.0	United Kingdom

Packaging our code to read and process data

We can refactor our code into the following function.

import pandas as pd 

def read_and_clean_csv(customer_data):
    """
    Read and clean the csv file. This function removes NaN values from the CustomerID column, 
    and renames the columns to all lower case. It returns a dictionary of the dataframe.

    Args:
        customer_data (str): Path to the csv file.

    Returns:
        df_dict (dict): Dictionary of the dataframe.
    """
        
    try:
        # Source https://www.kaggle.com/datasets/carrie1/ecommerce-data?resource=download
        df = pd.read_csv(customer_data, encoding='latin1')

        # Drop rows with empty CustomerID
        df.dropna(subset=['CustomerID'], inplace=True)

        # rename columns to all lower case
        df.columns = [x.lower() for x in df.columns]

        # Save df to dict
        df_dict = df.to_dict("records")

        return df_dict
    
    except Exception as e:
        print("Unable to read file due to:", e)
        return {}

Preparing our data for an LLM to consume

We will transform our data from a dataframe object into a dictionary. We will also implement an LLM-powered pipeline using the Haystack 2.0 beta release. This will enable us to set up a pipeline that will allow us to query our data using natural language.

Requirements

We will create a virtual environment and install the required packages through a requirements.txt file. We will use conda to create the virtual environment, and pip to install the packages.

Our requirements file will contain the following:

# requirements.txt
haystack-ai==2.0.0b5
fastapi==0.109.0
torch==2.1.1
torchvision==0.16.1
transformers 
accelerate 
bitsandbytes 
sentence-transformers
python-dotenv
uvicorn

We can then execute:

conda create -name aihelper python==3.10
conda activate aihelper
pip install -r requirements.txt

Transforming the data into an appropriate data structure

We will transform the data into a list of dictionaries, where each dictionary represents a row in the data. We will then store the dictionaries into Haystack Document objects. This object acts as an abstraction layer between the data and the LLM.

Core assumption

The Haystack Document object contains two key fields we will leverage: content and meta. We will create a string with the content of each row, and store it in the content field. We will store the dictionary representing the row in the meta field.

from haystack.dataclasses import Document

def generate_haystack_documents(df_dict):
    """
    Generate a list of Haystack documents from a dataframe.

    Args:
        df_dict (dict): Dictionary of the dataframe.

    Returns:
        haystack_documents (list): List of Haystack documents.
    """
    try:
        haystack_documents = []

        # Create a list of Haystack documents
        for i in range(len(df_dict)):
            content_str = f"Name of item purchased: {df_dict[i]['description']}; \
                Quantity purchased: {df_dict[i]['quantity']}; \
                Price of item: {df_dict[i]['unitprice']}; \
                Date of purchase: {df_dict[i]['invoicedate']}; \
                Country of purchase: {df_dict[i]['country']}; \
                Customer ID: {df_dict[i]['customerid']}; \
                Invoice Number: {df_dict[i]['invoiceno']}; \
                Stock Code: {df_dict[i]['stockcode']}" ,
            haystack_documents.append(Document(
                content=content_str[0],
                id = f"ZOOA{str(1000000 + i)}",
                meta={
                    "invoiceno": df_dict[i]["invoiceno"],
                    "stockcode": df_dict[i]["stockcode"],
                    "description": df_dict[i]["description"],
                    "quantity": df_dict[i]["quantity"],
                    "invoicedate": df_dict[i]["invoicedate"],
                    "unitprice": df_dict[i]["unitprice"],
                    "customerid": df_dict[i]["customerid"],
                    "country": df_dict[i]["country"],
                },
            ))

        return haystack_documents
    
    except Exception as e:
        print("Unable to generate Haystack documents due to:", e)
        return []

The function above will return a list of Haystack Document objects. We can then use the generate_haystack_documents function to generate our Haystack Documents:

haystack_documents = generate_haystack_documents(df_dict)

This is what our Documents look like:

print(haystack_documents[0].content)

'Name of item purchased: WHITE HANGING HEART T-LIGHT HOLDER; Quantity purchased: 6; Price of item: 2.55; Date of purchase: 12/1/2010 8:26; Country of purchase: United Kingdom; Customer ID: 17850.0; Invoice Number: 536365; Stock Code: 85123A;'

print(haystack_documents[0].meta)

{'invoiceno': '536365',
 'stockcode': '85123A',
 'description': 'WHITE HANGING HEART T-LIGHT HOLDER',
 'quantity': 6,
 'invoicedate': '12/1/2010 8:26',
 'unitprice': 2.55,
 'customerid': 17850.0,
 'country': 'United Kingdom'}

Next we will look into storing the documents onto a Document store.

Storing the documents on a document store

The document store is a Haystack class that the LLM will be connected to when retrieving information. As this application is quite simple, we will use an InMemoryDocumentStore. This document store is not persistent, and will be destroyed when the application is shut down. For more complex applications, Haystack offers a variety of document stores, including Elasticsearch, Qdrant, and AstraDB. For more information, see here.

from haystack.document_stores.in_memory import InMemoryDocumentStore

def populate_document_store(haystack_documents):
    """
    Populate the document store with the Haystack documents.

    Args:
        haystack_documents (list): List of Haystack documents.
    """
    try:
        document_store = InMemoryDocumentStore(bm25_algorithm="BM25Plus")
        document_store.write_documents(documents=haystack_documents)
        return document_store
    
    except Exception as e:
        print("Unable to populate document store due to:", e)
        document_store = InMemoryDocumentStore(bm25_algorithm="BM25Plus")
        return document_store

We can then populate the document store:

document_store = populate_document_store(haystack_documents)

The function above will ensure all of our documents are stored in the document store. We can then use the document store to query our data.

Building an LLM-based pipeline

We can now use Haystack’s components and connect them into an LLM-powered pipeline. Our pipeline will include a prompt template and a series of components that will allow us to query our data. The prompt template will provide the LLM with instructions on how to answer questions, and the components connected into a pipeline will provide the LLM with the data it needs to answer the questions.

Crafting our prompt template

We will use the following template to generate our prompt. In it, we provide instructions for the role the LLM plays, and the types of questions it can answer. We also provide the LLM with the context it will use to answer the questions. The context is the content of the documents retrieved by the retriever. We will use the Jinja2 language to generate the prompt and allow it to parse through the content of the data. We will use the following template:

prompt_template = """
        You are an expert data analyst who helps customers and employees with 
        their questions about purchases and products.
        You use the information provided in the documents to answer the questions.
        
        
        Questions regarding the order: please ask the user to give you the invoice number.
        Questions regarding the product: please ask the user to give you the stock code.
        Questions regarding purchases made on a given day: please ask the user to give you the date of purchase.
        
        If you are asked to calculate the total price of a purchase, please ask the user to give you the invoice 
        number and add the total price of the items in the purchase.
        
        If you are asked to calculate the total number of items for a purchase, please ask the user to give you the 
        invoice number and add the total number of items in the purchase.
        
        If you are asked if an order can be cancelled, please ask the user to give you the invoice number and check 
        if that the order was placed after November 1 2010 and before December 2 2010. If it isn't, 
        you should respond with 'No, the order cannot be cancelled.'

        If the documents do not contain the answer to the question, say that ‘Answer is unknown.’
        Context:
        {% for doc in documents %}
            Purchase information: {{ doc.content }} 
            Invoice Number: {{ doc.meta['invoiceno'] }} 
            Stock Code: {{doc.meta['stockcode']}}
            Quantity purchased: {{doc.meta['quantity']}}
            Date of purchase: {{doc.meta['invoicedate']}}
            Price per item: {{doc.meta['unitprice']}} \n
        {% endfor %};

        Question: {{query}}
        
        \n Answer:
        """

Our pipeline will consist of the following components:

PromptBuilder: This component will generate a prompt for the LLM to consume. The prompt will be generated based on the question asked by the user. The prompt will be generated using our template.

InMemoryBM25Retriever: This component will retrieve the most relevant documents from the document store based on the question asked by the user. The documents will be retrieved using the BM25 algorithm. BM25, or Best Match 25, is a ranking algorithm for information retrieval and search engines. It enhances the traditional TF-IDF (Term Frequency-Inverse Document Frequency) model.

GPTGenerator: This component will generate an answer to the question asked by the user. The answer will be generated using the OpenAI’s GPT models.

We can initialize our LLM-powered pipeline as follows:

from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.builders.prompt_builder import PromptBuilder
from haystack.components.generators import GPTGenerator
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever

def initialize_pipeline(document_store, openai_key, prompt_template):
    """
    Initialize the pipeline for the AI helper.

    Args:
        document_store (DocumentStore): Populated Haystack document store.
        openai_key (str): OpenAI API key.
        prompt_template (str): Template for the prompt.

    Returns:
        prediction_pipeline (Pipeline): Prediction pipeline.

    """

    try:
        
        prompt_builder = PromptBuilder(prompt_template)
        retriever = InMemoryBM25Retriever(document_store=document_store)
        llm = GPTGenerator(api_key=openai_key, 
                        generation_kwargs={"temperature": 0},
                        model='gpt-4')

        # Initialize pipeline
        prediction_pipeline = Pipeline()
        prediction_pipeline.add_component("retriever", retriever)
        prediction_pipeline.add_component("prompt_builder", prompt_builder)
        prediction_pipeline.add_component("generator", llm)

        prediction_pipeline.connect("retriever.documents", "prompt_builder.documents")
        prediction_pipeline.connect("prompt_builder", "generator")

        return prediction_pipeline
    
    except Exception as e:
        print("Unable to initialize pipeline due to:", e)
        return None

With the function above, we can initialize our pipeline.

In this section we connected the components of our pipeline. We can now use the pipeline to query our data. We will now turn our attention to the API. In the next section, we will learn about how we can package our code into an API using FastAPI.

Assessing the pipeline’s reasoning capabilities

To test the capacity of the pipeline to answer questions correctly, we extracted order information for a given order number. We then asked the LLM to produce the order’s description and quantity. We repeated this test 100 times to evaluate the LLM-based pipeline’s capability to answer the questions correctly in a consistent manner.

Below is code for the this specific test:

def get_items_from_answer(order_number):
    """
    This function sets up the LLM response part of the test

    Args:
        order_number (str): Order number.

    Returns:
        items (list): List of tuples containing the item description and quantity.
    """

    query = f"Items for order with invoice number {order_number}"
    result = prediction_pipeline.run(data={"retriever": {"query": query}, 
                                        "prompt_builder": {"query": query},
                                        })
    final_answer= result['generator']['replies'][0]

    items = []
    for line in final_answer.strip().split("\n"):
        if "Item" in line:
            item_name = line.split(":")[1].strip().split(", ")[0]
            quantity = line.split(":")[2].strip()
            items.append((item_name, quantity))
    return items

def test_answers_against_ground_truth(test_df, items):
    """
    This function tests the LLM response against the ground truth

    Args:
        test_df (DataFrame): DataFrame containing the ground truth.
        items (list): List of tuples containing the item description and quantity.

    Returns:
        fail_flags (list): List of tuples containing the item description and quantity 
        that were not found.
    """
    
    fail_flags = []
    total_quantity_flag = False
    for item, quantity in items:
        if not test_df[(test_df['description'] == item) & (test_df['quantity'] == int(quantity))].empty:
            pass
        else:
            fail_flags.append((item, quantity))

    if len(fail_flags) == 0  & len(items)==test_df.shape[0]:
        print("All items found!")
        total_quantity_flag = True
    else:
        print("The following items were not found:")
        print(fail_flags)
        print("Or there are items in the dataframe the llm didn't identify")
        total_quantity_flag = False
    
    return fail_flags, total_quantity_flag


# Perform test
results = []
for i in range(100):
    test_llm = get_items_from_answer("536365")
    test_df = df[df['invoiceno'] == "536365"]

    test_result = test_answers_against_ground_truth(test_df, test_llm)
    results.append(test_result)

Results

The LLM-based pipeline was able to answer the questions correctly in 100% of the cases for the order number 536365. This is a promising result, and indicates that the LLM-based pipeline is able to reason and answer questions about the data in a consistent manner. We can repeat this test for a higher number of orders to increase confidence in the pipeline’s ability to answer questions correctly, or alternatively to assess what can be improved. Similar tests can be developed to assess the ability of the LLM to correctly calculate the total cost of a given order, for it to identify whether an order was cancelled or not, and for it to identify the country where a given order was placed.

Packaging our application

We will store the functions we created in a file called pipelinehelper.py. We will then create a file called app.py that will contain the code for our API. We will also create base models using Pydantic for the question and answer. We will use these models to define the input and output of our API.

Creating the base models

We will create our models as follows

from pydantic import BaseModel


# Define Pydantic models
class QueryModel(BaseModel):
    query: str

class ResponseModel(BaseModel):
    answer: str

The base models above are quite simple and assume that the user will provide a question as input, and the API will return an answer as output. We can then use these models to define the input and output of our API.

Completing API functionality

We can complete the app through the following code. The app has two entry points: a GET endpoint that returns a message, and a POST endpoint that takes a question as input and returns an answer as output. The POST endpoint will use the pipeline we created to generate the answer.

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import os
from dotenv import load_dotenv
import uvicorn

from pipelinehelper import (read_and_clean_csv, 
                            generate_haystack_documents, 
                            populate_document_store, 
                            initialize_pipeline)

# Load environment variables
load_dotenv(".env")

# Initialize FastAPI app
app = FastAPI()

# Define Pydantic models
class QueryModel(BaseModel):
    query: str

class ResponseModel(BaseModel):
    answer: str

# Load and prepare the pipeline
openai_key = os.getenv("OPENAI_KEY")
df_dict = read_and_clean_csv("data.csv")
haystack_documents = generate_haystack_documents(df_dict=df_dict)
document_store = populate_document_store(haystack_documents=haystack_documents)
prediction_pipeline = initialize_pipeline(document_store=document_store, openai_key=openai_key)

@app.post("/query", response_model=ResponseModel)
async def query_sales_data(query_model: QueryModel):
    try:
        query = query_model.query
        result = prediction_pipeline.run(data={"retriever": {"query": query}, 
                                               "prompt_builder": {"query": query},
                                               })
        return {"answer": result['generator']['replies'][0]}
    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))

@app.get("/")
async def read_root():
    return {"message": "Haystack AI Sales Query API"}

if __name__ == "__main__":
    
    uvicorn.run(app, host="0.0.0.0", port=80)

The application above ensures a simple API that can be used to query the data. We can then deploy the API to Ploomber Cloud.

Deploying the solution

Deploying our application with Ploomber Cloud using the CLI

We will now package our application into .py scripts, a Dockerfile and a requirements.txt file. We will then deploy it on Ploomber Cloud through their command line interface. With Ploomber Cloud, we can easily deploy our application on the cloud and share it with others, we can manage secrets and automate deployments through GitHub actions.

Sample Dockerfile

FROM python:3.11

COPY app.py app.py
COPY pipelinehelper.py pipelinehelper.py
COPY .env .env 
COPY requirements.txt requirements.txt
RUN pip install torch==2.1.1 torchvision==0.16.1 --index-url https://download.pytorch.org/whl/cpu
RUN pip install -r requirements.txt

ENTRYPOINT ["uvicorn", "app:app", "--host=0.0.0.0", "--port=80"]

In this case app.py contains code with the retriever pipeline and the Solara functionality. We will also need to create a .env file with our OpenAI API key. We will also need to create a requirements.txt file with the following packages:

haystack-ai==2.0.0b5
fastapi==0.109.0
torch==2.1.1
torchvision==0.16.1
transformers 
accelerate 
bitsandbytes 
sentence-transformers
python-dotenv
uvicorn

Initialize deployment

You will need to create an account on Ploomber Cloud. You can do so here. You will also need to generate an API key on Ploomber Cloud under ‘Account’ in https://www.platform.ploomber.io/ You will also need to install the Ploomber Cloud CLI. You can do so as follows:

pip install ploomber-cloud

Connect your local computer to your Ploomber Cloud account by running the following command:

ploomber-cloud key

Paste your API key when prompted. You can then initialize your deployment as follows:

ploomber-cloud init

This will create a ploomber-cloud.json file in your current directory. It will have your app id and the type (docker).

Deploy your application

You can deploy your application as follows:

ploomber-cloud deploy

This will deploy your application on the cloud. You can then access your application via the URL provided in the output of the command above.

Calling the API and sample questions and answers

Here are some questions we can ask the API:

What is the total cost for order with invoice number 537463?
What were the items in order 536365?
How many items were in order 536858?
Can I still cancel order with invoice number 536365?

Once we have deployed our application, we can call the API from the command line or via FastAPI through the web browser. We can use the following code to call the API from the command line.

For example, for the question What were the items in order 536365?, we can execute:

curl -X 'POST'
    'https://calm-violet-6179.ploomberapp.io/query'
    -H 'accept: application/json'
    -H 'Content-Type: application/json'
    -d '{
    "query": "What were the items in order 536365?"
    }'

This yields a JSON object with an answer key:

{
    "answer": "The items in order 536365 were:
                1. WHITE METAL LANTERN
                2. SET 7 BABUSHKA NESTING BOXES
                3. WHITE HANGING HEART T-LIGHT HOLDER
                4. CREAM CUPID HEARTS COAT HANGER
                5. RED WOOLLY HOTTIE WHITE HEART.
                6. GLASS STAR FROSTED T-LIGHT HOLDER
                7. KNITTED UNION FLAG HOT WATER BOTTLE"
    }

Similarly, for What is the total cost for order with invoice number 537463? you can run

curl -X 'POST'
    'https://calm-violet-6179.ploomberapp.io/query'
    -H 'accept: application/json'
    -H 'Content-Type: application/json'
    -d '{
    "query": "What is the total cost for order with invoice number 537463?"
    }'

This yields a JSON object with an answer key:

{
  "answer": "The total cost for order with invoice number 537463 is calculated as follows:
  - STRAWBERRY LUNCH BOX WITH CUTLERY: 6 * 2.55 = 15.3
  - LUNCH BOX WITH CUTLERY RETROSPOT: 6 * 2.55 = 15.3
  - POSTAGE: 4 * 18.0 = 72.0
  - DOORMAT RESPECTABLE HOUSE: 2 * 7.95 = 15.9
  - TABLECLOTH RED APPLES DESIGN: 4 * 8.5 = 34.0
  - GUMBALL COAT RACK: 36 * 2.1 = 75.6
  - WOODLAND STICKERS: 12 * 0.85 = 10.2
  - IVORY DINER WALL CLOCK: 2 * 8.5 = 17.0
  - RED RETROSPOT CUP: 16 * 0.85 = 13.6
  - WOODLAND CHARLOTTE BAG: 10 * 0.85 = 8.5
  
  Adding all these up, the total cost for order with invoice number 537463 is 15.3 + 15.3 + 72.0 + 15.9 + 34.0 + 75.6 + 10.2 + 17.0 + 13.6 + 8.5 = 277.4."
}

Final remarks

In this blog, we explored how to leverage large language models (LLMs) along with retrieval augmented generation (RAG) to explore their ability to retrieve and answer questions from a knowledge base. We explored this for unstructured data in the form of audio and video, and for semi-structured data in the form of questions and answers to a personality test. In this blog, we assessed an LLMs capacity to retrieve information from a structured data source, and its ability to reason to answer questions about it with the goal of integrating an LLM-based approach for a chatbot application with a focus on order support.

We identified a dataset containing purchase order information, and used it to build an LLM-based pipeline that can answer questions about the data. We then packaged our code into an API using FastAPI, and deployed it on Ploomber Cloud. We then tested the pipeline’s ability to answer questions about the data. We found that the pipeline was able to answer questions about the data in a consistent manner. We can use this pipeline a a chatbot application that can answer questions about purchase orders by adding a user interface that leverages the API. This application can be used to support order support teams, and to provide customers with information about their orders.

Deploy AI apps with Ploomber

Start for free

Build and deploy an AI-powered API to process order information

Introduction

Deploying the application

Sample code

Chatbot Technology: Understanding the Core and Varieties

About the data and the application

Packaging our code to read and process data

Preparing our data for an LLM to consume

Requirements

Transforming the data into an appropriate data structure

Core assumption

Storing the documents on a document store

Building an LLM-based pipeline

Crafting our prompt template

Assessing the pipeline’s reasoning capabilities

Results

Packaging our application

Creating the base models

Completing API functionality

Deploying the solution

Deploying our application with Ploomber Cloud using the CLI

Sample Dockerfile

Initialize deployment

Deploy your application

Calling the API and sample questions and answers

Final remarks

Deploy AI apps with Ploomber

Recent Articles

A systematic overview of prompt engineering through programming

Stop using 0.5 as the threshold for your binary classifier

Try Ploomber Cloud Now