One Kernel, Many Frameworks

Connecting LangChain, LlamaIndex, and more via Semantic Kernel

How many AI orchestrators have you heard about lately — LangChain, LlamaIndex, AutoGen, CrewAI— and found yourself feeling overwhelmed? With an ever-growing number of frameworks promising to simplify the orchestration of AI workflows, choosing the right one — or figuring out how to integrate multiple solutions — often creates more confusion than clarity. But here’s the good news: you don’t have to limit your choice to only one framework.

This article will guide you through leveraging Semantic Kernel’s powerful modularity and abstraction to effortlessly integrate your favorite components from multiple orchestrators though the concept of Plugins.

Anatomy of a Plugin

In my previous article, we introduced two key concepts in the context of AI agents: modularity and abstraction.

Abstraction involves decomposing and simplifying complexity, making intricate systems easier to understand and scale.

Modularity divides complex problems into smaller, reusable components, each designed to manage a distinct aspect of the overall challenge.

Those two concepts are deeply entangled and, when it comes to concrete implementation in Semantic Kernel, the easiest way to get startd is that of plugin.

At its core, a SK plugin is a class with a set of functions. So what’s the key element that appoint plugins as agentic components? Their semantic description. In fact, plugins’ functions are provided with a natural language description of their capabilities, so that the brain of the agent — the LLM — can literally read those labels and understand which plugin to invoke — and which function of the plugin to run — given a specific user’s query.

Let’s see a sample template of how a SK plugin might look like:

from semantic_kernel.functions import kernel_function

class MathPlugin:

    @kernel_function(
        name="add_numbers",
        description="Add two numbers together and return the result."
    )
    def add_numbers(self, number1: float, number2: float) -> float:
        """Returns the sum of two numbers."""
        print('add_numbers function invoked')
        return number1 + number2

    @kernel_function(
        name="multiply_numbers",
        description="Multiply two numbers and return the product."
    )
    def multiply_numbers(self, number1: float, number2: float) -> float:
        """Returns the product of two numbers."""
        print('multiply_numbers function invoked')
        return number1 * number2

    @kernel_function(
        name="square_number",
        description="Calculate and return the square of a number."
    )
    def square_number(self, number: float) -> float:
        """Returns the square of the given number."""
        print('square_number function invoked')
        return number ** 2

As you can see, we initialize a class called MathPlugin and, leveraging the kernel_function method in SK, we initialize 4 functions, each one provided with:

• A name
• A semantic description of its capabilities
• A proper function core logic

We could easily provide this plugin to an AI agent with the following code:

from semantic_kernel.agents import ChatCompletionAgent

# Create the Chat Completion Agent instance by specifying a list of plugins

agent = ChatCompletionAgent(
    service=AzureChatCompletion(service_id="default"),
    instructions="You are a helpful math assistant.",
    plugins=[MathPlugin()]
)

Now, this was an easy example of capabilities that you can quickly write in custom functions. However, there might be skills you want to infuse in your agents that are more sophisticated — chatting with mutliple tables into a SQL database, retrieving unstructured data, processing videos etc.

Here you typically have two routes:

• Writing those plugins from scratch
• Leveraging pre-built components

Since we want to minimize the amount of code we write, we will go for option 2. But what if Semantic Kernel doesn’t provide us with these pre-built components?

Integrating multiple frameworks as SK Plugins

There is no doubt that, among the available AI frameworks, there are some that excel for specific tasks or for specific use cases. For example, LangChain is powered by a vibrant community which led, over the past years, to one of the biggest catalog of pre-built components that you can leverage in your AI App. We also have Llama-Index, which is featured by great tools when it comes to knowledge retrieval.

The fact that different frameworks might excel at different skills makes answering the question “which is the best AI framework?” really hard to answer. But the good news is that, thanks to modularity and abstraction — and the way Semantic Kernel infuse them in its structure — we can pick what we need from different orchestrators and convert them into SK plugins.

Let’s demonstrate how to do that with the following scenario. We are going to build a media ecommerce store AI agent with two plugins:

• A SQL plugin which is gonna query data in a SQL database with many related tables. For this purpose, we are going to use LangChain’s pre-built SQLDatabase Toolkit.
• A RAG plugin which is gonna retrieve unstructured text about various artists. For this purpose, we are going to use Llama-index rertieval engine.

Let’s start!

SQL Plugin

For this plugin, we will leverage the SQL Database Toolkit in LangChain available here (I recommend you to familiarize with the code before going ahead with the reading).

For test purpose, we are going to use the Chinook database (you can download it here in SQlite). The database represents a digital media store, with tables having the following relationships:

Source: lerocha/chinook-database: Sample database for SQL Server, Oracle, MySQL, PostgreSQL, SQLite, DB2

Below you can see how to “wrap” this toolkit into a SK plugin:

from langchain_community.agent_toolkits.sql.toolkit import SQLDatabaseToolkit
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain.tools import BaseTool, StructuredTool, tool
from langchain_community.utilities.sql_database import SQLDatabase
from dotenv import load_dotenv
from langchain_openai import AzureChatOpenAI
from langchain_community.chat_message_histories import ChatMessageHistory
from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain.agents import AgentExecutor, create_openai_tools_agent
import os
from semantic_kernel.functions import kernel_function

class SQLAgentPlugin:
    def __init__(self):
        # Load environment variables
        load_dotenv()
        openai_api_version = os.getenv("AZURE_OPENAI_API_VERSION")
        azure_chat_deployment = os.getenv("AZURE_OPENAI_CHAT_DEPLOYMENT_NAME")

        # Initialize the database and LLM
        self.db = SQLDatabase.from_uri("sqlite:///chinook.db")
        self.llm = AzureChatOpenAI(
            openai_api_version=openai_api_version,
            azure_deployment=azure_chat_deployment,
        )

        # Initialize the toolkit and tools
        self.toolkit = SQLDatabaseToolkit(db=self.db, llm=self.llm)
        self.tools = self.toolkit.get_tools()

        # Define the prompt
        self.prompt = ChatPromptTemplate.from_messages(
            [
                ("system", """You are an agent designed to interact with a SQL database.
Given an input question, create a syntactically correct SQLite query to run, then look at the results of the query and return the answer.
Unless the user specifies a specific number of examples they wish to obtain, always limit your query to at most 5 results.
You can order the results by a relevant column to return the most interesting examples in the database.
Never query for all the columns from a specific table, only ask for the relevant columns given the question.
You have access to tools for interacting with the database.
Only use the below tools. Only use the information returned by the below tools to construct your final answer.
You MUST double check your query before executing it. If you get an error while executing a query, rewrite the query and try again.

DO NOT make any DML statements (INSERT, UPDATE, DELETE, DROP etc.) to the database.

To start you should ALWAYS look at the tables in the database to see what you can query.
Do NOT skip this step.
Then you should query the schema of the most relevant tables.
                """),
                MessagesPlaceholder("chat_history", optional=True),
                ("human", "{input}"),
                MessagesPlaceholder("agent_scratchpad"),
            ]
        )

        # Initialize the agent and executor
        self.message_history = ChatMessageHistory()
        self.agent = create_openai_tools_agent(self.llm, self.tools, self.prompt)
        self.agent_executor = AgentExecutor(agent=self.agent, tools=self.tools, verbose=True)
        self.agent_with_chat_history = RunnableWithMessageHistory(
            self.agent_executor,
            lambda session_id: self.message_history,
            input_messages_key="input",
            history_messages_key="chat_history",
        )
    @kernel_function(
        name="run_sql_query", 
        description="Invoke the SQL agent to run a SQL query."
    )
    def run_sql_query(self, user_input: str) -> str:
        """
        Takes user input, invokes the agent, and returns the agent's output.
        """
        result = self.agent_with_chat_history.invoke(
            {"input": user_input},
            config={"configurable": {"session_id": "<foo>"}},
        )
        return result["output"]

As you can see, I re-used the same code provided in the LangChain documentation, wrapped into a SK plugin — meaning that we initialized a class with one function. This latter, run_sql_query, is initialized as a @kernel_function and comes with a name and description.

To make the code structure as modular as possible, I stored this plugin as a .py file into a folder named Plugins (you can see a full explanation on how to do that here). Let’s see how to import it and add it to a SK agent:

from Plugins.sqlplugin import SQLAgentPlugin
from semantic_kernel.agents import ChatCompletionAgent

# Create the Chat Completion Agent instance by specifying a list of plugins

agent = ChatCompletionAgent(
    service=AzureChatCompletion(service_id="default"),
    instructions="You are a helpful assistant",
    plugins=[SQLAgentPlugin()]
)

And now we can run the code (I enabled the “verbose” mode in LangChain so that we can see the SQL tool working in the backend):

from semantic_kernel.contents import ChatHistory


chat_history = ChatHistory()

while True:
    # Get user input
    user_input = input("You: ")
    if user_input.lower() in ["exit", "quit"]:
        print("Exiting the chat. Goodbye!")
        break

    # Add the user input to the chat history
    chat_history.add_user_message(user_input)
    print(f"# User: {user_input}")

    # Invoke the agent for a response
    response = await agent.get_response(chat_history)
    print(f"# {response.name}: {response.content}")

Output:

# User: hi
# agent_JEUYZYwYuKsNhQCD: Hello! How can I assist you today?
# User: Which country's customers spent the most?


> Entering new AgentExecutor chain...

Invoking: `sql_db_list_tables` with `{}`


albums, artists, customers, employees, genres, invoice_items, invoices, media_types, playlist_track, playlists, tracks
Invoking: `sql_db_schema` with `{'table_names': 'customers'}`



CREATE TABLE customers (
 "CustomerId" INTEGER NOT NULL, 
 "FirstName" NVARCHAR(40) NOT NULL, 
 "LastName" NVARCHAR(20) NOT NULL, 
 "Company" NVARCHAR(80), 
 "Address" NVARCHAR(70), 
 "City" NVARCHAR(40), 
 "State" NVARCHAR(40), 
 "Country" NVARCHAR(40), 
 "PostalCode" NVARCHAR(10), 
 "Phone" NVARCHAR(24), 
 "Fax" NVARCHAR(24), 
 "Email" NVARCHAR(60) NOT NULL, 
 "SupportRepId" INTEGER, 
 PRIMARY KEY ("CustomerId"), 
 FOREIGN KEY("SupportRepId") REFERENCES employees ("EmployeeId")
)

/*
3 rows from customers table:
CustomerId FirstName LastName Company Address City State Country PostalCode Phone Fax Email SupportRepId
1 Luís Gonçalves Embraer - Empresa Brasileira de Aeronáutica S.A. Av. Brigadeiro Faria Lima, 2170 São José dos Campos SP Brazil 12227-000 +55 (12) 3923-5555 +55 (12) 3923-5566 luisg@embraer.com.br 3
2 Leonie Köhler None Theodor-Heuss-Straße 34 Stuttgart None Germany 70174 +49 0711 2842222 None leonekohler@surfeu.de 5
3 François Tremblay None 1498 rue Bélanger Montréal QC Canada H2G 1A7 +1 (514) 721-4711 None ftremblay@gmail.com 3
*/
Invoking: `sql_db_schema` with `{'table_names': 'invoices'}`
responded: The `customers` table does not contain a column named `spent_amount`. To find the total spent amount by country, we need to look at a table that contains spending information.

Let's list the schemas of other relevant tables that might contain spending information. The `invoices` and `invoice_items` tables are likely candidates.

I will retrieve the schemas for `invoices` and `invoice_items`.


CREATE TABLE invoices (
 "InvoiceId" INTEGER NOT NULL, 
 "CustomerId" INTEGER NOT NULL, 
 "InvoiceDate" DATETIME NOT NULL, 
 "BillingAddress" NVARCHAR(70), 
 "BillingCity" NVARCHAR(40), 
 "BillingState" NVARCHAR(40), 
 "BillingCountry" NVARCHAR(40), 
 "BillingPostalCode" NVARCHAR(10), 
 "Total" NUMERIC(10, 2) NOT NULL, 
 PRIMARY KEY ("InvoiceId"), 
 FOREIGN KEY("CustomerId") REFERENCES customers ("CustomerId")
)

/*
3 rows from invoices table:
InvoiceId CustomerId InvoiceDate BillingAddress BillingCity BillingState BillingCountry BillingPostalCode Total
1 2 2009-01-01 00:00:00 Theodor-Heuss-Straße 34 Stuttgart None Germany 70174 1.98
2 4 2009-01-02 00:00:00 Ullevålsveien 14 Oslo None Norway 0171 3.96
3 8 2009-01-03 00:00:00 Grétrystraat 63 Brussels None Belgium 1000 5.94
*/
Invoking: `sql_db_schema` with `{'table_names': 'invoice_items'}`
responded: The `customers` table does not contain a column named `spent_amount`. To find the total spent amount by country, we need to look at a table that contains spending information.

Let's list the schemas of other relevant tables that might contain spending information. The `invoices` and `invoice_items` tables are likely candidates.

I will retrieve the schemas for `invoices` and `invoice_items`.


CREATE TABLE invoice_items (
 "InvoiceLineId" INTEGER NOT NULL, 
 "InvoiceId" INTEGER NOT NULL, 
 "TrackId" INTEGER NOT NULL, 
 "UnitPrice" NUMERIC(10, 2) NOT NULL, 
 "Quantity" INTEGER NOT NULL, 
 PRIMARY KEY ("InvoiceLineId"), 
 FOREIGN KEY("TrackId") REFERENCES tracks ("TrackId"), 
 FOREIGN KEY("InvoiceId") REFERENCES invoices ("InvoiceId")
)

/*
3 rows from invoice_items table:
InvoiceLineId InvoiceId TrackId UnitPrice Quantity
1 1 2 0.99 1
2 1 4 0.99 1
3 2 6 0.99 1
*/
Invoking: `sql_db_query_checker` with `{'query': 'SELECT BillingCountry AS country, SUM(Total) AS total_spent FROM invoices GROUP BY BillingCountry ORDER BY total_spent DESC LIMIT 1;'}`
responded: The `invoices` table contains a column named `Total` which represents the total amount spent on each invoice. We can use this information to calculate the total spent by country.

Here is the query to find the country with the highest total spend:

```sql
SELECT BillingCountry AS country, SUM(Total) AS total_spent
FROM invoices
GROUP BY BillingCountry
ORDER BY total_spent DESC
LIMIT 1;
```

Let's check and execute this query.

```sql
SELECT BillingCountry AS country, SUM(Total) AS total_spent FROM invoices GROUP BY BillingCountry ORDER BY total_spent DESC LIMIT 1;
```
Invoking: `sql_db_query` with `{'query': 'SELECT BillingCountry AS country, SUM(Total) AS total_spent FROM invoices GROUP BY BillingCountry ORDER BY total_spent DESC LIMIT 1;'}`


[('USA', 523.06)]The country with the highest total spend is the USA, with a total spend of 523.06.

> Finished chain.
# agent_JEUYZYwYuKsNhQCD: The country whose customers spent the most is the USA, with a total spending amount of 523.06.
Exiting the chat. Goodbye!

Let’s now go ahead with the RAG Plugin.

RAG Plugin

For this plugin, we are going to leverage LlamaIndex, an AI orchestration framework which excels at integrating external knowledge bases to LLM-powered applications, providing a set of specialized tools for data ingestion, indexing and querying. This makes extremely convenient to use LlamaIndex for RAG-based agents, or any scenario where we are required to interact with some external knowledge.

Henceforth, we are going to leverage LlamaIndex for our RAG plugin. As a vector DB, we are going to leverage Azure AI Search, an AI-powered information retrieval platform that enables developers to build rich search experiences and generative AI applications by indexing and querying content from various sources.

The first thing that we need to do is creating an index in Azure AI Search — this will be our external knowledge base. To be consistent with the content of the Chinook database, I indexed some text files about a few artists’ life and work.

This is the example for Iron Maiden:

Artist Spotlight: Iron Maiden

Formed in Leyton, East London, in 1975 by bassist and primary songwriter Steve Harris, Iron Maiden emerged as a pioneering force in the New Wave of British Heavy Metal (NWOBHM). The band's name, inspired by a medieval torture device, reflects their dark and intense musical themes.​

Iron Maiden's self-titled debut album, released in 1980, introduced their raw energy and complex compositions to the world. The follow-up, Killers (1981), further solidified their reputation, but it was the addition of vocalist Bruce Dickinson for their third album, The Number of the Beast (1982), that catapulted them to international stardom. This album featured iconic tracks like "Run to the Hills" and the titular "The Number of the Beast," showcasing their signature galloping rhythms and intricate guitar work.​

Throughout the 1980s, Iron Maiden released a series of successful albums, including Piece of Mind (1983), Powerslave (1984), and Seventh Son of a Seventh Son (1988). Their elaborate live performances, complete with elaborate stage designs and their mascot, Eddie, became legendary.​

Despite lineup changes and shifts in the musical landscape, Iron Maiden continued to produce critically acclaimed albums into the 21st century, such as Brave New World (2000) and The Book of Souls (2015). Their enduring influence on the heavy metal genre is undeniable, inspiring countless bands and maintaining a dedicated global fanbase.​

Then, I indexed those 3 files into AI search as follows:

import logging
import sys
from azure.core.credentials import AzureKeyCredential
from azure.search.documents import SearchClient
from azure.search.documents.indexes import SearchIndexClient
from IPython.display import Markdown, display
from llama_index.core import (
    SimpleDirectoryReader,
    StorageContext,
    VectorStoreIndex,
)
from llama_index.core.settings import Settings
from llama_index.llms.azure_openai import AzureOpenAI
from llama_index.embeddings.azure_openai import AzureOpenAIEmbedding
from llama_index.vector_stores.azureaisearch import AzureAISearchVectorStore
from llama_index.vector_stores.azureaisearch import (
    IndexManagement,
    MetadataIndexFieldType,
)
import os
from dotenv import load_dotenv

load_dotenv()

aoai_api_key = os.getenv("AZURE_OPENAI_API_KEY")
aoai_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT")
aoai_api_version = os.getenv("AZURE_OPENAI_API_VERSION")

llm = AzureOpenAI(
    model="gpt-4o",
    deployment_name="gpt-4o",
    api_key=aoai_api_key,
    azure_endpoint=aoai_endpoint,
    api_version=aoai_api_version,
)

# You need to deploy your own embedding model as well as your own chat completion model
embed_model = AzureOpenAIEmbedding(
    model="text-embedding-3-large",
    deployment_name="text-embedding-3-large",
    api_key=aoai_api_key,
    azure_endpoint=aoai_endpoint,
    api_version=aoai_api_version,
)

search_service_api_key = os.getenv("AZURE_AI_SEARCH_API_KEY")
search_service_endpoint = os.getenv("AZURE_AI_SEARCH_ENDPOINT")
credential = AzureKeyCredential(search_service_api_key)
search_service_api_version = "2024-07-01"


# Index name to use
index_name = "chinook"

# Use index client to demonstrate creating an index
index_client = SearchIndexClient(
    endpoint=search_service_endpoint,
    credential=credential,
)

# Use search client to demonstration using existing index
search_client = SearchClient(
    endpoint=search_service_endpoint,
    index_name=index_name,
    credential=credential,
)

metadata_fields = {
    "artist": "artist"
}

vector_store = AzureAISearchVectorStore(
    search_or_index_client=index_client,
    filterable_metadata_field_keys=metadata_fields,
    index_name=index_name,
    index_management=IndexManagement.CREATE_IF_NOT_EXISTS,
    id_field_key="id",
    chunk_field_key="chunk",
    embedding_field_key="embedding",
    embedding_dimensionality=3072,
    metadata_string_field_key="metadata",
    doc_id_field_key="doc_id",
    language_analyzer="en.lucene",
    vector_algorithm_type="exhaustiveKnn",
)

# Load documents
documents = SimpleDirectoryReader("data/").load_data()
storage_context = StorageContext.from_defaults(vector_store=vector_store)

Settings.llm = llm
Settings.embed_model = embed_model
index = VectorStoreIndex.from_documents(
    documents, storage_context=storage_context
)

Note: make sure to match the embedding dimensionality with your embedding model! For example, the model I picked (text-embedding-3-large) generates vectors of 3072 elements, but other models might have different dimentionalities.

Now that we have our index, we can initialize our plugin:

from semantic_kernel.functions import kernel_function
from llama_index.core import StorageContext, VectorStoreIndex
from llama_index.llms.azure_openai import AzureOpenAI
from llama_index.embeddings.azure_openai import AzureOpenAIEmbedding
from llama_index.vector_stores.azureaisearch import AzureAISearchVectorStore, IndexManagement
from llama_index.core.vector_stores.types import VectorStoreQueryMode
from azure.core.credentials import AzureKeyCredential
from azure.search.documents.indexes import SearchIndexClient
from azure.search.documents import SearchClient
from llama_index.core.settings import Settings
from llama_index.core.settings import Settings
from llama_index.llms.azure_openai import AzureOpenAI
from llama_index.embeddings.azure_openai import AzureOpenAIEmbedding
from llama_index.vector_stores.azureaisearch import AzureAISearchVectorStore
from llama_index.vector_stores.azureaisearch import (
    IndexManagement,
    MetadataIndexFieldType,
)
from dotenv import load_dotenv
import os

class RAGPlugin:
    def __init__(self):
        load_dotenv(dotenv_path="c:/Users/vaalt/OneDrive/Desktop/Projects/github/adnoc prototype/.env", override=True)

        self.aoai_api_key = os.getenv("AZURE_OPENAI_API_KEY")
        self.aoai_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT")
        self.aoai_api_version = os.getenv("AZURE_OPENAI_API_VERSION")

        self.llm = AzureOpenAI(
            model="gpt-4o",
            deployment_name="gpt-4o",
            api_key=self.aoai_api_key,
            azure_endpoint=self.aoai_endpoint,
            api_version=self.aoai_api_version,
        )

        self.embed_model = AzureOpenAIEmbedding(
            model="text-embedding-3-large",
            deployment_name="text-embedding-3-large",
            api_key=self.aoai_api_key,
            azure_endpoint=self.aoai_endpoint,
            api_version=self.aoai_api_version,
        )
        
        Settings.llm = self.llm
        Settings.embed_model = self.embed_model

        self.search_service_api_key = os.getenv("AZURE_AI_SEARCH_API_KEY")
        self.search_service_endpoint = os.getenv("AZURE_AI_SEARCH_ENDPOINT")
        self.credential = AzureKeyCredential(self.search_service_api_key)

        self.index_name = "chinook"

        self.vector_store = AzureAISearchVectorStore(
            search_or_index_client=SearchIndexClient(
                endpoint=self.search_service_endpoint,
                credential=self.credential,
            ),
            filterable_metadata_field_keys={"artist": "artist"},
            index_name=self.index_name,
            index_management=IndexManagement.CREATE_IF_NOT_EXISTS,
            id_field_key="id",
            chunk_field_key="chunk",
            embedding_field_key="embedding",
            embedding_dimensionality=3072,
            metadata_string_field_key="metadata",
            doc_id_field_key="doc_id",
            language_analyzer="en.lucene",
            vector_store_query_mode=VectorStoreQueryMode.HYBRID,
        )

        storage_context = StorageContext.from_defaults(vector_store=self.vector_store)
        self.index = VectorStoreIndex.from_documents([], storage_context=storage_context)
        self.hybrid_retriever = self.index.as_retriever(vector_store_query_mode=VectorStoreQueryMode.HYBRID)

    @kernel_function(
        name="extract_text_content",
        description="Retrieve relevant context about popular artists."
    )
    def extract_text_content(self, user_query: str) -> str:
        result = self.hybrid_retriever.retrieve(user_query)
        extracted_texts = [node_with_score.node.text for node_with_score in result]
        combined_text = "\n\n".join(extracted_texts)
        print('RAG plugin invoked')
        return combined_text

Let’s try it (also in this case, I saved the plugin as a .py file into my Plugins folder):

from Plugins.ragplugin import RAGPlugin
from semantic_kernel.agents import ChatCompletionAgent

# Create the Chat Completion Agent instance by specifying a list of plugins

agent = ChatCompletionAgent(
    service=AzureChatCompletion(service_id="default"),
    instructions="You are a helpful assistant. Given the context provided by the rag plugin, you summarize and provide a concise answer to the user query",
    plugins=[RAGPlugin()]
)

from semantic_kernel.contents import ChatHistory


chat_history = ChatHistory()

while True:
    # Get user input
    user_input = input("You: ")
    if user_input.lower() in ["exit", "quit"]:
        print("Exiting the chat. Goodbye!")
        break

    # Add the user input to the chat history
    chat_history.add_user_message(user_input)
    print(f"# User: {user_input}")

    # Invoke the agent for a response
    response = await agent.get_response(chat_history)
    print(f"# {response.name}: {response.content}")

Outout:

# User: what's maiden song in 1980
RAG plugin invoked
# agent_BjDDMrCkrOkWIjgK: Iron Maiden's self-titled debut album, **"Iron Maiden"**, was released in 1980. This album includes notable songs such as "Running Free," "Phantom of the Opera," and "Iron Maiden."
Exiting the chat. Goodbye!

Putting all together

Now let’s create our media ecommerce AI agent. We are going to provide it with both the plugins and a specific set of instructions as follows:

instructions = """
You are a helpful assistant for a media store. 
You provide insightful information about songs, artists and albums' statistics around the world.
When users ask you questions about any media, you will check your internal database and provide the answer.
You also provide context about artists' life from your knowledge base.
"""

agent = ChatCompletionAgent(
    service=AzureChatCompletion(service_id="default"),
    instructions = instructions,
    plugins=[RAGPlugin(), SQLAgentPlugin()]
)

Let’s try it:

from semantic_kernel.contents import ChatHistory


chat_history = ChatHistory()

while True:
    # Get user input
    user_input = input("You: ")
    if user_input.lower() in ["exit", "quit"]:
        print("Exiting the chat. Goodbye!")
        break

    # Add the user input to the chat history
    chat_history.add_user_message(user_input)
    print(f"# User: {user_input}")

    # Invoke the agent for a response
    response = await agent.get_response(chat_history)
    print(f"# {response.name}: {response.content}")

Output:

# User: Hello
# agent_IxDKrVtELgiNVprU: Hi there! How can I assist you today?
# User: What's the most sold album from Iron Maiden?


> Entering new AgentExecutor chain...

Invoking: `sql_db_list_tables` with `{}`


albums, artists, customers, employees, genres, invoice_items, invoices, media_types, playlist_track, playlists, tracks
Invoking: `sql_db_schema` with `{'table_names': 'albums'}`



CREATE TABLE albums (
 "AlbumId" INTEGER NOT NULL, 
 "Title" NVARCHAR(160) NOT NULL, 
 "ArtistId" INTEGER NOT NULL, 
 PRIMARY KEY ("AlbumId"), 
 FOREIGN KEY("ArtistId") REFERENCES artists ("ArtistId")
)

/*
3 rows from albums table:
AlbumId Title ArtistId
1 For Those About To Rock We Salute You 1
2 Balls to the Wall 2
3 Restless and Wild 2
*/
Invoking: `sql_db_schema` with `{'table_names': 'artists'}`



CREATE TABLE artists (
 "ArtistId" INTEGER NOT NULL, 
 "Name" NVARCHAR(120), 
 PRIMARY KEY ("ArtistId")
)

/*
3 rows from artists table:
ArtistId Name
1 AC/DC
2 Accept
3 Aerosmith
*/
Invoking: `sql_db_query_checker` with `{'query': "SELECT ArtistId FROM artists WHERE Name = 'Iron Maiden';"}`
responded: The `albums` table does not have a `copies_sold` column, and the `artists` table does not contain a column for the number of copies sold either.

The correct schema for the `albums` table is:
```sql
CREATE TABLE albums (
 "AlbumId" INTEGER NOT NULL, 
 "Title" NVARCHAR(160) NOT NULL, 
 "ArtistId" INTEGER NOT NULL, 
 PRIMARY KEY ("AlbumId"), 
 FOREIGN KEY("ArtistId") REFERENCES artists ("ArtistId")
)
```

To find the albums by Iron Maiden, I will need to first find the `ArtistId` for Iron Maiden from the `artists` table and then query the `albums` table for albums with this `ArtistId`. However, without the `copies_sold` column, I cannot sort by it. 

Let's identify the `ArtistId` for Iron Maiden first.

```sql
SELECT ArtistId FROM artists WHERE Name = 'Iron Maiden';
```
Invoking: `sql_db_query` with `{'query': "SELECT ArtistId FROM artists WHERE Name = 'Iron Maiden';"}`


[(90,)]
Invoking: `sql_db_query_checker` with `{'query': 'SELECT Title FROM albums WHERE ArtistId = 90 LIMIT 5;'}`


```sql
SELECT Title FROM albums WHERE ArtistId = 90 LIMIT 5;
```
Invoking: `sql_db_query` with `{'query': 'SELECT Title FROM albums WHERE ArtistId = 90 LIMIT 5;'}`


[('A Matter of Life and Death',), ('A Real Dead One',), ('A Real Live One',), ('Brave New World',), ('Dance Of Death',)]Here are some albums by Iron Maiden:

1. A Matter of Life and Death
2. A Real Dead One
3. A Real Live One
4. Brave New World
5. Dance Of Death

> Finished chain.
RAG plugin invoked
# agent_IxDKrVtELgiNVprU: The most sold album from Iron Maiden is "The Number of the Beast." This iconic album catapulted them to international stardom and features legendary tracks such as "Run to the Hills" and the titular "The Number of the Beast."

Note that the final answer:

# agent_IxDKrVtELgiNVprU: The most sold album from Iron Maiden is "The Number of the Beast." This iconic album catapulted them to international stardom and features legendary tracks such as "Run to the Hills" and the titular "The Number of the Beast."

Consolidates the output from both the SQL plugin and RAG plugin. This is an example of a deliberate and autonomous choice of the agent, which decided to provide a comprehensive answer merging two different sources. We can decide to be more “prescrictive” in the way the agent orchestrates the plugin at the system message level or leveraging the built-in function FunctionChoiceBehavior() (which is set to Auto by default) in the Kernel Arguments (learn more here).

Conclusion

In this article, we saw how Semantic Kernel can act as the glue that binds components from different AI orchestrators. Its plugin-based architecture enables developers to unify diverse tools under a single, cohesive workflow without compromising flexibility or control. I see it as a great feature which really empodies the concept of agentic modularity, and that could be further scaled into multi-agent architecture.

References

• SQLDatabase Toolkit | 🦜️🔗 LangChain
• Azure AI Search — LlamaIndex
• Introduction to Azure AI Search — Azure AI Search | Microsoft Learn
• lerocha/chinook-database: Sample database for SQL Server, Oracle, MySQL, PostgreSQL, SQLite, DB2
• https://learn.microsoft.com/en-us/semantic-kernel/concepts/ai-services/chat-completion/function-calling/?pivots=programming-language-python