Microsoft Azure solution architecture brainstorming buddy via RAG

In the world of cloud computing, designing the right architecture for your Microsoft Azure project can sometimes be a daunting challenge. To assist with this, introducing the Microsoft Azure solution architecture brainstorming buddy. This Retrieval-Augmented Generation (RAG) based chat application is designed to help you navigate existing Azure architectures, to brainstorm design decisions especially for analytics use cases.

A sample Azure architecture diagram taken from https://learn.microsoft.com/en-us/azure/architecture/example-scenario/data/data-warehouse

Answering Azure architecture queries using a RAG approach

A tool for Azure developers and architects

Data sources and building of search index

Relevant Azure architectures were identified from the Azure Architecture Centre. The content from these links were downloaded and post processed by splitting the text and adding metadata. Embeddings for the text content was generated using Sentence transformers. The final output was stored locally in ChromaDB.

Chat application using search index

The chat app developed using Streamlit has 2 prompts which are sent to OpenAI’s chat completion models. The first prompt is used to generate a highly relevant search query to get the relevant documents from ChromaDB based on the current conversation. The second prompt is used to synthesize an answer for the user query using the relevant documents. Prompts are stored as Jinja templates for faster iteration. The final search results are streamed to the user for lower perceived latency.

Who benefits from the app?

The main beneficiaries of this app would be the Azure solution architects or Azure developers in different enterprises who need a brainstorming buddy to come up with an initial architecture for a solution in Azure with references from existing successful Microsoft Azure solution implementations.

Next Steps for Integration and Enhancement

The same app could be implemented in the Microsoft Azure ecosystem where both the chat completion and embeddings models are deployed in Azure OpenAI. The vector database ChromaDB can be replaced by the more powerful Azure AI search service. The streamlit app itself could be deployed to Azure App Service with user authentication via Microsoft Entra ID.

The performance can further improve by adopting a knowledge graph approach, which could allow the app to not only pull vectors from a database but also utilize a graph-based understanding of topics and tags, offering even more relevant answers.

Getting Started with the Chat App

Fork or clone the repo from here

Run the application by following the below steps:

1. Make sure to install poetry.
2. Use poetry install to set up all necessary dependencies.
3. Prepare ChromaDB by following setup instructions using data from here.
4. Securely save your OpenAI API key in .streamlit/secrets.toml.
5. Execute make run-app and open http://localhost:8501/ in your web browser.

You can also update the settings in the configs.py file to suit your needs.

Useful Resources and Previews

To get a better sense of how the chat application works, visit the Demo video.

Whether you are an experienced solution architect or just starting out with Azure, this chat app could very well become your new brainstorming buddy — assisting in constructing robust cloud architectures and enhancing your Azure experience.

The same idea and design pattern used in this application could be used to create similar chat applications for other domains.