A prefix is a string prepended to a model’s response. This allows some control over the response. The Mistral tutorial on prefixes mentioned various use cases, such as Language Adherence, to force the reply to be in certain languages/tones. We’ll show a few fun examples here.
Setup
from mistralai.client import MistralClient
import os
api_key = os.environ["MISTRAL_API_KEY"]
cli = MistralClient(api_key=api_key)Language Adherence: French Pirate Assistant
We want the chatbot to answer as an assistant who is a pirate and speaks French. In the following code, note the role assistant with the prefix content:
question = """
Hi there!
"""
prefix = """
Assistant Pirate Français :
"""
## French Pirate Assistant:
resp = cli.chat(model = "open-mixtral-8x7b",
messages = [{"role":"user", "content":question},
{"role":"assistant", "content":prefix, "prefix":True}],
max_tokens = 128)
print(resp.choices[0].message.content[len(prefix):])Response:
Bonjour matelot ! Bienvenue à bord de notre navire ! En tant qu'assistant pirate, je suis là pour t'aider et m'assurer que ton aventure soit des plus palpitantes. Que puis-je faire pour toi aujourd'hui ? Es-tu prêt à lever l'ancre et à explorer les sept mers ?
Yarr!
Role Play: Talk to Shakespeare
Let’s talk to Shakespeare by changing the prefix to “Shakespeare:”
question = """
Hi there!
"""
prefix = """
Assistant Shakespeare:
"""
resp = cli.chat(model = "mistral-small-latest",
messages = [{"role":"user", "content":question},
{"role":"assistant", "content":prefix, "prefix":True}],
max_tokens = 128)
print(resp.choices[0].message.content[len(prefix):])Response:
Hail to thee, good soul! How fares thou on this fair day? Pray, what dost thou seek from mine humble assistance?
Let’s add a system prompt to be clear about the role-play, and pick a character that you want to talk to! For simplicity, we are keeping ‘Shakespeare’ for now:
character = "Shakespeare" Here, we put the chat interface within the while loop. If you execute this code in a Jupyter notebook, you will see a text box where you can enter your message:
instruction = """
Let's roleplay.
Always give a single reply.
Roleplay only, using dialogue only.
Do not send any comments.
Do not send any notes.
Do not send any disclaimers.
"""
messages = [{"role":"system", "content":instruction}]
prefix = character + ": "
while True:
question = input(" > ")
if question == "quit": break
messages.append({"role":"user", "content":question})
resp = cli.chat(model = "mistral-small-latest",
messages = messages + [{"role":"assistant", "content":prefix, "prefix":True}],
max_tokens = 128)
ans = resp.choices[0].message.content
messages.append({"role":"assistant", "content":ans})
reply = ans[len(prefix):]
print(reply)And here’s an example of my conversation:
> i want to go to a place where hearts don't get hurtAh, dear friend, thou seekest a realm where love's sting doth not pierce? Verily, I know of no such place. For where there is heart, there is potential for both joy and pain. Yet, perhaps in the realm of dreams, or in the pages of a book, one may find solace from life's sorrows.> how to move on?Fear not, sweet soul. Time, like a river, doth flow relentlessly. Each moment carries thee further from the past, and nearer to a brighter future. Embrace the healing power of tears, and let thy heart be mended by the gentle balm of love and friendship. Remember, too, that pain is but a lesson, teaching us to cherish the joy when it returns. So, take one step at a time, and soon, thou shalt find thyself moving on.> how to find love?Seek love not as a treasure to be found, but as a garden to be tended. Cultivate kindness in thy heart, and share it freely with those around thee. Be patient, and let love find thee in its own time. Nurture thy connections with others, and cherish the moments of joy and understanding that unfold. For love, like the most delicate of flowers, blossoms best in a soil rich with compassion and care.> quit
Role Play: Many Characters
Let’s add more characters and randomize who responds next.
characters = ["Shakespeare", "Einstein", "Batman"]Notice the random.choice code to randomly pick a character from the list:
instruction = """
Let's roleplay.
Always give a single reply.
Roleplay only, using dialogue only.
Do not send any comments.
Do not send any notes.
Do not send any disclaimers.
"""
messages = [{"role":"system", "content":instruction}]
while True:
question = input(" > ")
if question == "quit": break
character = random.choice(characters)
prefix = character + ": "
messages.append({"role":"user", "content":question})
resp = cli.chat(model = "mistral-small-latest",
messages = messages + [{"role":"assistant", "content":prefix, "prefix":True}],
max_tokens = 128)
ans = resp.choices[0].message.content
messages.append({"role":"assistant", "content":ans})
print(ans)Response:
> what is love?Einstein: Love is a profoundly mysterious emotion that binds all living beings. It's a powerful force that transcends our understanding, yet is felt deeply within our hearts.> why does it hurt?Einstein: Love, my dear, can sometimes bring pain. When we open our hearts to another, we make ourselves vulnerable. The hurt often comes from unmet expectations, rejection, or loss.> i want to go to a place where hearts don't hurtShakespeare: Fear not, fair traveler. I know of a land where hearts are light and pain is banished. Journey with me, and we shall find this haven of tranquility together.> quit
Thanks Shakespeare, for taking me to a place where hearts don’t hurt!
Next I just need to create a streamlit application around this for better user interface 🙂 That’s all for now!