Build a retrieval augmented generation (RAG) AI app using SDKs

This tutorial includes the following SDK languages and versions:

TypeScript Java Python v1 Python v2 C# Go
❌ ❌ ❌ ❌ ✅ ❌

TypeScript	Java	Python v1	Python v2	C#	Go
❌	❌	❌	❌	✅	❌

Everyone has heard of ChatGPT, the popular large language model (LLM) that can generate human like text, answering any question you may have with a certain degree of correctness. However, the biggest problem with these large language models is that they only have a limited amount of 'knowledge' to draw on - they are trained using data from the internet up to a particular date, and that is it.

For example, if you ask an LLM what todays date is, or what the weather is like in a particular city, it will not be able to answer you - it just doesn't have that data.

This lack of data also limits an LLMs ability to answer questions, or reason, based off your own data. For example, if you have a database of customer reviews, and you want to ask the LLM a question about the reviews, it will not be able to answer you as it simply doesn't have access to that data. This is where retrieval augmented generation (RAG) comes in, allowing you to retrieve data from your own systems to augment what the LLM can reason over.

This tutorial will show you how to develop an AI powered app that uses SDKs generated by liblab to provide RAG capabilities to an LLM. Specifically using augmenting ChatGPT with cat facts! Your app will use Microsoft Semantic Kernel as a AI app framework, and ChatGPT from OpenAI as the LLM.

Prerequisites

This tutorial assumes you already have:

The liblab CLI installed and you are logged in
An OpenAI account and API key ¹
.NET 8.0 or later installed²

¹ There is a small cost associated with using the OpenAI API, so make sure you have enough credits to run the code in this tutorial. This should only use a few cents to get started. ² There is a template repository that you can use to get started with this tutorial, which has a pre-configured dev container. To use this, you will need to have Docker installed but won't need .NET installed.

This tutorial also assumes that you have a basic understanding of what an LLM is, and have created prompts with tools like ChatGPT before.

Overview of some relevant concepts

Before we get started, let's go over some concepts that will be useful to know. If you already know these concepts, feel free to skip ahead to creating the app.

What is RAG?

Retrieval augmented generation, or RAG, is the term for augmenting the data that goes into the LLM by retrieving data from other systems, and use that data to help the LLM generate answers to your questions. For example, if you have an app that will prompt an LLM to give you the average sentiment of a particular product, you can use RAG to augment the LLM with the customer reviews of that product by extracting the relevant data from your reviews database, then sending that data to the LLM to reason over.

How does your app implement RAG?

If you have a prompt-based app that allows the user to interact using pure text, then your app will start from a prompt that is user generated, such as "What is the average sentiment of the cuddly llama toy". Your app will then use some kind of LLM-powered app framework that has plugins - these are add-ons in the app that can retrieve data. This framework will use the LLM to determine which plugins it needs to call to get data, then send that data back to the LLM to reason over with an updated prompt.

Create your RAG app

This tutorial will alk you through creating your first RAG app using an SDK generated by liblab. This app will use:

Microsoft Semantic Kernel as the AI app framework
ChatGPT from OpenAI as the LLM
Cat Facts API to get random cat facts

The steps you will take are:

Use a provided template to create a new AI powered app
Use liblab to generate an SDK for the Cat Facts API
Create a plugin to get random cat facts and use this in your app

1. Create a new AI powered app

To get started, we have created a basic sample project that is configured to use the Microsoft Semantic Kernel as the AI app framework, and is already connected to OpenAI and has the user interaction already provided. You will start with this sample project, then add plugins using generated SDKs.

Create the app

This project is available from a template repository on GitHub.

Select the button below to create a new control repo using this template.

RAG with SDKs template repo

Name the repo whatever you like, such as cat-facts-app, then select the Create repo button.
Open this repo. You can either clone it to your local machine, or open in a GitHub Codespace. If you open locally, or in a Codespace, you can use the included dev container to get started quickly. This includes .NET 8.0 and the liblab CLI.
Before you can run the app, you will need to configure your OpenAI API key in the appsettings.json file.
1. In the src folder, rename the provided appsettings.json.example file to appsettings.json.
2. Open the appsettings.json file and replace OpenAI:ApiKey with your API key.
  appsettings.json
```
{
    "OpenAI": {
        "Key": "<Your key here>",
        "ChatModel": "gpt-4"
    }
}
```
3. You can also configure the chat model in this file if you want to use a different model.
Build and run the app by running dotnet run from the src folder. This will start the app and you can interact with through the terminal.
Terminal
```
$ dotnet run
I am an AI assistant who also knows a load of cat facts!
User >
```

Ask a question and press enter to see the response.

Terminal

User > What is the airspeed velocity of an unladen swallow?
Assistant > What do you mean? African or European swallow?

Overview of the app

The app is a small console app, built around Microsoft Semantic Kernel. It has the following files:

src
├── appsettings.json
├── AppSettingsReader.cs
├── OpenAISettings.cs
├── Program.cs
└── RagWithSDKs.csproj

appsettings.json - this file contains the settings for the app, including the OpenAI API key, and the model name to use for chat
AppSettingsReader.cs - this file contains the AppSettingsReader class that reads the appsettings.json file, allowing you to access the OpenAI settings, getting back an OpenAISettings object
OpenAISettings.cs - this file contains the OpenAISettings class that wraps the OpenAI settings section from the appsettings.json file
Program.cs - this file contains the entry point for the app. It does the following:
1. Reads the OpenAI settings from the appsettings.json file
2. Creates a new Semantic Kernel KernelBuilder object that configures how your app will interact with the AI models, in this case using OpenAI chat completion - essentially ChatGPT.
3. Creates a chat history object. The way you interact with the LLM in code is by sending a complete history of the conversation so far including the latest prompt from the user. This way the LLM can use previous questions and responses to help create the next response. For example you can ask "What is the capital of England", then ask "How many people live there", and the LLM will be aware that you are referring to London, the answer to the first question, when answering the second.
  
  This history contains:
  - A system prompt - this is used to give the AI some basic context and rules. In this case we are naming the chatbot Libby the liblab llama. Setting this system prompt is an important part of prompt engineering and can have a big impact on the quality of the responses you get back.
  - User prompts - these are the questions you ask the AI
  - AI responses - these are the responses the AI gives back
  This history will have a single system prompt, and then a user prompt and AI response for each interaction you have with the AI, ending with the latest user prompt. Although it is called the 'history', it also contains the current prompt.
4. Creates a chat completion service. The act of passing in a history and getting back a response is known as chat completion.
5. Starts a loop that will allow you to interact with the AI. This loop will:
  1. Get a user prompt
  2. Add this to the history
  3. Call the chat completion service with the history
  4. Add the AI response to the history
  5. Print the AI response
6. This loop ends when the user enters an empty prompt
RagWithSDKs.csproj - the project file for the app

2. Use liblab to generate SDKs for the Cat Facts API

If you run the app and ask for a cat fact, the LLM will create one based off of whatever it learned from the internet when it was trained. This might be fine, but for our purposes, we want to limit the cat facts to a defined set. To do this, we will use the Cat Facts API to get random cat facts, so that the LLM only uses this as its cat facts data source.

Before we dig into how our app implements RAG to limit cat facts to this data source, we first need a way for it to call the cat facts API, and we will do this by generating a C# SDK that wraps this API using liblab.

The cat facts liblab config file

To make it easier to generate the cat facts SDK, the repo you created already contains a configured liblab config file.

info

The liblab config file provides configuration to liblab for SDK generation, such as the location of your API spec, the SDK languages you want, and other configuration options. You can read more in our config file overview.

You can find this config file in sdks/cat-facts/liblab.config.json. This file contains the following configuration:

sdks/cat-facts/liblab.config.json
{
  "sdkName": "CatFacts",
  "specFilePath": "https://catfact.ninja/docs/api-docs.json",
  "languages": [
    "csharp"
  ],
  // The API spec doesn't define the servers, so we need to provide the base URL
  "baseUrl": "https://catfact.ninja",
  "languageOptions": {
    "csharp": {
      "packageId": "liblab.Examples.CatFacts",
      "sdkVersion": "0.0.1"
    }
  }
}

This configuration tells liblab:

Name the SDK CatFacts
Use the cat facts API spec from https://catfact.ninja/docs/api-docs.json
Generate a C# SDK only
The base URL for the API is https://catfact.ninja. The URL of the API server is normally defined in the API spec, but in this case it isn't, so we need to provide it.
Set the generated C# library's PackageId to liblab.Examples.CatFacts
Set the Version in the generated C# library to 0.0.1

Generate the cat facts SDK

To generate the cat facts SDK:

run the following command from the sdks/cat-facts folder:
Terminal
```
liblab build
```
This will generate a C# SDK in the sdks/cat-facts/output/csharp folder. In this folder there will be 2 projects - CatFacts, the SDK itself, and Example, a simple example project for the SDK.
```
sdks/cat-facts/output/csharp
├── CatFacts
└── Example
```

note

Both the SDK and example project are .NET 6 projects. This is fine for the SDK as you can use it in any .NET project from 6 onwards, but if you want to run the example you will either need .NET 6 installed, or you can update the project to .NET 8.

3. Create a plugin to get random cat facts

We will be creating a Semantic Kernel Plugin, a class that implements one or more methods that are labelled as a KernelFunction. These kernel functions have text decorators that tell the LLM what they can do, and return a response that the LLM can use to reason over. To implement RAG using cat facts, we will create a plugin with a kernel function that can get a random cat fact from the cat facts API.

Create the plugin

To use the new SDK, you first need to add a reference to it. Do this by running the following command from the src folder:
Terminal
```
dotnet add reference ../sdks/cat-facts/output/csharp/CatFacts/CatFacts.csproj
```
Create a new folder in the src folder called Plugins. This is where the plugin will live.

Inside the src/Plugins folder, create a new file called CatFactPlugin.cs:

src
├── Plugins
│   └── CatFactPlugin.cs
├── appsettings.json
├── AppSettingsReader.cs
├── OpenAISettings.cs
├── Program.cs
└── RagWithSDKs.csproj

Add the CatFactPlugin class to this file, declaring it in the RagWithSDKs.Plugins namespace:
src/Plugins/CatFactPlugin.cs
```
namespace RagWithSDKs.Plugins;

public class CatFactPlugin
{
}
```

Add a readonly field to this class for the cat facts SDK client, and create this on declaration:

src/Plugins/CatFactPlugin.cs
namespace RagWithSDKs.Plugins;
using CatFacts;

public class CatFactPlugin
{
    private readonly CatFactsClient _client = new();
}

Declare a method on the CatFactPlugin class to get a cat fact from the SDK and return it:

src/Plugins/CatFactPlugin.cs
public class CatFactPlugin
{
    private readonly CatFactsClient _client = new();

    public async Task<string> GetCatFact()
    {
        Console.WriteLine("CatFactPlugin > Getting a cat fact from the Cat Facts API...");
        var response = await _client.Facts.GetRandomFactAsync();
        Console.WriteLine("CatFactPlugin > Cat fact: " + response.Fact);
        return response.Fact;
    }
}

This method logs to the console before and after calling the SDK so that you can see that the app is using this plugin when you run it.

To use this method as a kernel function, you need to add 2 attributes - Microsoft.SemanticKernel.KernelFunctionAttribute to mark it as a kernel function, and System.ComponentModel.DescriptionAttribute to provide a natural language description to the LLM of what this function actually does, so that the AI app framework can decide to use it when appropriate.

Add these attributes to the GetCatFact method, along with the relevant using statements:
src/Plugins/CatFactPlugin.cs
```
using System.ComponentModel;
using Microsoft.SemanticKernel;

public class CatFactPlugin
{
    ...

    [KernelFunction]
    [Description("Gets a cat fact.")]
    public async Task<string> GetCatFact()
    {
        ...
    }
}
```

Add the plugin to the kernel

Once the plugin is created, you need to add it to the kernel so that the AI app framework can use it. This is done in the Program.cs file by adding it to the kernel builder before the kernel is built.

In the Program.cs file, add a using statement for the RagWithSDKs.Plugins namespace:
src/Program.cs
```
using RagWithSDKs;
using RagWithSDKs.Plugins;
```

After the builder is created, add the CatFactPlugin to it:

src/Program.cs
// Create the kernel builder with OpenAI chat completion
var builder = Kernel.CreateBuilder().AddOpenAIChatCompletion(openAISettings.ChatModel,
                                                             openAISettings.Key);

// Add plugins to the kernel builder
builder.Plugins.AddFromType<CatFactPlugin>();

This is added to the builder by type, and it knows how to instantiate a new instance of the CatFactPlugin class.

Try out the cat facts plugin

Now that the plugin is added to the kernel, you can try it out by running the app and asking for a cat fact.

Build and run the app by running dotnet run from the src folder.

Ask for a cat fact:

Terminal
$ dotnet run
I am an AI assistant who also knows a load of cat facts and can create images!
User > tell me a cat fact
CatFactPlugin > Getting a cat fact from the Cat Facts API...
CatFactPlugin > Cat fact: A cat's normal temperature varies around 101 degrees Fahrenheit.
Assistant > A cat's normal temperature varies around 101 degrees Fahrenheit.
User > 

In the console output, you can see that the plugin is being used to get the cat fact from the Cat Facts API.

The LLM can do more than just return the cat fact as is, you can use this fact to generate a response based off another part of the response. For example, asking for a cat fact in the style of a pirate:
Terminal
```
User > Give me a fact about cats in the style of a pirate
CatFactPlugin > Getting a cat fact from the Cat Facts API...
CatFactPlugin > Cat fact: A group of cats is called a clowder.
Assistant > Arr matey! Be ye knowin' that a gatherin' of meowin' seafarers,
them cats, be called a clowder? Aye, a fine group of whiskered buccaneers they be!
```
You will see in the response that the LLM has requested a cat fact, retrieved it from the cat facts plugin, then used this fact to generate a response in the style of a pirate.

You might also notice that the prompt examples here are asking for cat facts in different ways - "tell me a cat fact" and "Give me a fact about cats". The LLM is able to reason over these prompts and the plugin description "Gets a cat fact." to decide to use the plugin.

note
One hard part of using LLMs is prompt engineering, and this includes making sure that the description of these kernel functions is not only clear, but encompasses all the ways that a user might ask for this information. For example, if you ask "tell me about cats", then the LLM might not use this plugin as it is for cat facts, and you have not asked explicitly for a fact.

🎉🎉🎉 Congratulations, you have built a RAG AI app using a generated SDK! 🎉🎉🎉

Conclusion

In this tutorial, you have created an AI powered app that uses SDKs generated by liblab to provide RAG capabilities to an LLM. This is one of the powerful features of liblab - being able to quickly generate an SDK in a language of your choice, making it easier to integrate that API into your AI app.

Prerequisites​

Overview of some relevant concepts​

What is RAG?​

How does your app implement RAG?​

Create your RAG app​

1. Create a new AI powered app​

Create the app​

Overview of the app​

2. Use liblab to generate SDKs for the Cat Facts API​

The cat facts liblab config file​

Generate the cat facts SDK​

3. Create a plugin to get random cat facts​

Create the plugin​

Add the plugin to the kernel​

Try out the cat facts plugin​

Conclusion​