Managing Prompt Templates in Python with Jinja

When working on dynamic text generation, whether it’s for generating personalized emails, or for generating prompts for different user inputs, managing and populating text based templates efficiently is crucial. One popular and effective way to handle such tasks in the Python ecosystem is by utilizing Jinja, a powerful templating engine. This templating engine is agnostic of LLM frameworks like Langchain and LlamaIndex and could be an useful lightweight solution for dynamic prompt template management when other features available in those libraries are not required.

Jinja logo

What is Jinja Template Language?

Jinja is a modern and designer-friendly templating language for Python, modeled after Django’s template system. It allows you to create dynamic HTML pages, emails, and other text-based data by injecting variables, using loops, conditional statements, and more.

Jinja templates provide a means to separate your logic from your presentation layer, making it easier to manage and update your text generation codebases. Additionally your jinja templates can be stored in your SCM, so making management of prompts is separate from the management of code.

How to install Jinja?

While Jinja is not included in the Python standard library, it is readily available via package managers such as pip. Installing Jinja is simple, and it integrates seamlessly with Python:

pip install Jinja2

Using Jinja Templates to Manage Prompt Templates

Let’s walk through a practical example where we use Jinja templates to create blog posts dynamically. Imagine you want to generate blog posts by passing specific values such as an overall blog topic, subtopics, etc. The first step would be to dynamically create a prompt that incorporates these values. Here’s how you can build such a prompt:

Step 1: Create a Jinja Template

The first step is to define a template in Jinja. Here is an example template stored in blog_post_template.jinja2:

Write a blog for a technical site on the topic {{ blog_topic }}.
{% if sub_topics|length > 0 %}
Sub topics inside the blog post:
{% for sub_topic in sub_topics %}
- {{ sub_topic }}
{% endfor %}
{% endif %}

As could be seen in the above prompt template, special placeholders in the template allow to write code that is similar to Python within the template itself which can be extremely useful for different scenarios. In the above template, only if there is atleast 1 sub topic the text “Sub topics inside the blog post:” will be rendered and then the following list of those sub topics will be rendered. In case there are no sub topics, then the whole section will be skipped and will not be rendered in the output.

Step 2: Create a Python Script to Populate the Template

Now, let’s write a Python script called generate_blog_post_prompt.py that reads this template, replaces the placeholders with actual values, and prints the final prompt for the blog post.

from jinja2 import Environment, FileSystemLoader

# Define the data to replace placeholders in the template.
data = {
    "blog_topic": "Types of machine learning",
    "sub_topics": [
        "Supervised learning",
        "Unsupervided learning",
        "Reinforcement learning"
    ]
}
# Load the Jinja template.
env = Environment(loader=FileSystemLoader('.'))
template = env.get_template('blog_post_template.jinja2')
# Render the template with data.
blog_post_prompt = template.render(data)
# Print the final blog post prompt.
print(blog_post_prompt)

While in the above script we populate the placeholders directly within the script as an example, these values could have been retrieved from the scripts arguments, a database, file or from the user via an API which interfaces a GUI.

Step 3: Execute the Script

Run the Python script to see the dynamically generated prompt to generate the blog post:

python generate_blog_post_prompt.py

The output will be:

Write a blog for a technical site on the topic Types of machine learning.
Sub topics inside the blog post:
- Supervised learning
- Unsupervided learning
- Reinforcement learning

You can try regenerating the prompt by providing an empty list for sub_topics and then check the output.

This prompt could then be passed to any LLM for the actual text generation e.g. the creation of the blog post in our example via the LLM’s completions endpoint.

Conclusion

Jinja templates provide an elegant and efficient way to manage and generate dynamic text in the Python ecosystem. By separating the presentation from the logic, you can easily manage prompt templates, making your code cleaner and easier to maintain. This approach is particularly valuable for generating prompts for creating blog posts, emails, or any content where the prompt structure is fixed, but the data is dynamic.