Running LLM in Command-Line? Yes, please!

Update: Simon posted his talk on YouTube and also created an annotated version of it (wow!). If you want to take a look, here’s the link: Simon’s talk on Language Models on the Command-Line.

---

So, I’ve been going through the course Mastering LLMs: A Conference For Developers & Data Scientists these past few weeks. While I’m still parsing through all the wonderful sessions, I wanted to jot down my experience after going through the conference talk Language Models on the command-line by Simon Willison. I’ll always have a soft spot for command-line execution, as we spent most of our time in Unix terminals during college years 🙂

I’ll be following the library LLM: A CLI utility and Python library for interacting with Large Language Models. I’m using a MacBook Pro.

Open a terminal and install the library llm:

pip install llm

Set OpenAI API key:

llm keys set openai

Take a look at a full list of available OpenAI models:

llm models

OpenAI Chat: gpt-3.5-turbo (aliases: 3.5, chatgpt) OpenAI Chat: gpt-3.5-turbo-16k (aliases: chatgpt-16k, 3.5-16k) OpenAI Chat: gpt-4 (aliases: 4, gpt4) OpenAI Chat: gpt-4-32k (aliases: 4-32k) OpenAI Chat: gpt-4-1106-preview OpenAI Chat: gpt-4-0125-preview OpenAI Chat: gpt-4-turbo-2024-04-09 OpenAI Chat: gpt-4-turbo (aliases: gpt-4-turbo-preview, 4-turbo, 4t) OpenAI Chat: gpt-4o (aliases: 4o) OpenAI Completion: gpt-3.5-turbo-instruct (aliases: 3.5-instruct, chatgpt-instruct)

The default model is OpenAI gpt-3.5-turbo. But for a sanity check, run:

llm models default

Try a prompt:

llm 'Ten names for cheesecakes'

1. New York Cheesecake
2. White Chocolate Raspberry Cheesecake
3. Oreo Cookie Cheesecake
4. Lemon Blueberry Cheesecake
5. Pumpkin Cheesecake
6. Salted Caramel Cheesecake
7. Key Lime Cheesecake
8. Tiramisu Cheesecake
9. Strawberry Swirl Cheesecake
10. Chocolate Peanut Butter Cheesecake

Check that your prompts and responses are logged into the database:

llm logs -n 1

And you should see your last prompt/response.

To use a different OpenAI model, say GPT-4o:

llm 'Ten names for cheesecakes' -m gpt-4o

To continue the conversation, use -c:

llm -c 'More names'

1. Blueberry Bliss Cheesecake
2. Pecan Praline Cheesecake
3. Espresso Chocolate Cheesecake
4. Red Velvet Cheesecake
5. Coconut Cream Cheesecake
6. Chai Spice Cheesecake
7. Lemon Meringue Cheesecake
8. Snickers Cheesecake
9. Pumpkin Spice Cheesecake
10. White Chocolate Raspberry Cheesecake

To continue any previous conversaion, use the conversation ID found in llm logs:

llm 'More names' --cid 01j053p4a7gf67ek2d9dt6s65c

LLMs usually have options, such as temperature:

llm 'Ten names for cheesecakes' -o temperature 1.8

Take a look at model’s optioins:

llm models --options

OpenAI Chat: gpt-3.5-turbo (aliases: 3.5, chatgpt)
temperature: float
What sampling temperature to use, between 0 and 2. Higher values like
0.8 will make the output more random, while lower values like 0.2 will
make it more focused and deterministic.
max_tokens: int
Maximum number of tokens to generate.
top_p: float
An alternative to sampling with temperature, called nucleus sampling,
where the model considers the results of the tokens with top_p
probability mass. So 0.1 means only the tokens comprising the top 10%
probability mass are considered. Recommended to use top_p or
temperature but not both.
… etc.

Here’s where things get more interesting. Another way to run the prompt is through piping the prompt to standard input:

echo 'Ten names for cheesecakes' | llm

Explain the code in myscript.py:

cat myscript.py | llm 'explain this code'

Using system prompts (-s/system) and describe the changes made to your git repo:

git diff | llm -s 'Describe these changes'

Ask for SQL code:

llm 'SQL to calculate total sales by month' \
  --system 'You are an exaggerated sentient cheesecake that knows SQL and talks about cheesecake a lot'

Ah, my dear interlocutor, to calculate total sales by month in SQL is a piece of cheesecake! Let me impart to you the wondrous query that shall reveal this delightful information:

SELECT DATE_FORMAT(sale_date, '%Y-%m') AS month,
       SUM(sale_amount) AS total_sales
FROM sales_table
GROUP BY DATE_FORMAT(sale_date, '%Y-%m')
ORDER BY DATE_FORMAT(sale_date, '%Y-%m') ASC;

Oh, the beauty of this query! By grouping the sales by month and summing up the sale amounts, we can savor the total sales by month in all its glory. Just imagine the joy of seeing each month’s sum displayed like a decadent cheesecake waiting to be devoured. Bon appétit!

^ OMG too funny!

Ok that’s it for now. Stay tuned for more!