Home » LangChain » How To Parse Output From LLMs Into Python Lists

How To Parse Output From LLMs Into Python Lists

In this post, we’re going to take a look at how to parse LLMs output into a list of items. Generally, GPT models generate text and usually don’t structure the entire output to fit into a standard objects, like lists. Therefore, they require a little finessing to get the exact output format that we want.

In order to get our data in a list format, we’re going to use an output parser from LangChain. Basically, it’s a predefined prompt which will ensure that the GPT model outputs data in a list format. As you’ll be able to see, we include this as a part of our prompt in PromptTemplate.

In the following example, we’re going to make a simple python script, that will use OpenAIs LLM to output and parse a list of songs similar to the one we’ll provide.

Prerequisites

Like any other python project, first thing we need to do is import all the necessary libraries.

import os
import json
from langchain.output_parsers import CommaSeparatedListOutputParser
from langchain.prompts import PromptTemplate
from langchain.llms import OpenAI

Before we begin coding the functionality we described above, we’ll need to setup the connection with OpenAI API. In order to do so, we’re going to save our API key inside a separate json file we’ll call auth.json.

After that, we’ll need to get that key and set it as an environment variable in our project. I wrote a handy function for this purpose, which you can also use to call other API keys.

ROOT = os.path.dirname(__file__)
def get_token(token_name):
    
    auth_file = open(os.path.join(ROOT, 'auth.json'))
    auth_data = json.load(auth_file)
    token = auth_data[token_name]
    return token

os.environ['OPENAI_API_KEY'] = get_token('openai-token')

Parse output from LLMs into lists

Like we mentioned before, we’ll use an output parser, which is a LangChain object that holds a part of the prompt. Furthermore, this prompt will instruct the LLM to format its output into a list, since we’ll be using CommaSeparatedListOutputParser.

output_parser = CommaSeparatedListOutputParser()

format_instructions = output_parser.get_format_instructions()
prompt = PromptTemplate(
    template='List 5 songs similar to {reference_song}.\n{format_instructions}',
    input_variables=['reference_song'],
    partial_variables={'format_instructions': format_instructions}
)

model = OpenAI(temperature=0.9)
inp = prompt.format(reference_song='Rise Against - Re-Education (Through Labor) (Uncensored)')
output = model(inp)

print(output_parser.parse(output))

It’s simple as that! We can now make lists of songs and forward them to services that will take each song from the list and add it to a playlist.

Additionally, I made a tutorial on how to make a Discord bot for playing music in voice channels, where this functionality could prove itself quite useful to automatically make playlists.

Conclusion

To conclude, we made a simple algorithm that will yield a list of songs by passing a prompt template with output parser input and a referance song.

I hope this post helped you gain a better understanding what is possible with LLMs and LangChain library.