Chapter 7: Building Chat Applications
Source: Oai Assignment · GitHub · microsoft/generative-ai-for-beginners Authors: Microsoft (microsoft) Licence: MIT — https://spdx.org/licenses/MIT.html
OpenAI API Quickstart
This notebook is adapted from the Azure OpenAI Samples Repository that includes notebooks that access Azure OpenAI services.
The Python OpenAI API works with Azure OpenAI Models as well, with a few modifications. Learn more about the differences here: How to switch between OpenAI and Azure OpenAI endpoints with Python
Overview
"Large language models are functions that map text to text. Given an input string of text, a large language model tries to predict the text that will come next"(1). This "quickstart" notebook will introduce users to high-level LLM concepts, core package requirements for getting started with AML, a soft introduction to prompt design, and several short examples of different use cases.
Table of Contents
Overview
How to use OpenAI Service
1. Creating your OpenAI Service
2. Installation
3. Credentials
Use Cases
1. Summarize Text
2. Classify Text
3. Generate New Product Names
4. Fine Tune a Classifier
Build your first prompt
This short exercise will provide a basic introduction for submitting prompts to an OpenAI model for a simple task "summarization".
Steps:
- Install OpenAI library in your python environment
- Load standard helper libraries and set your typical OpenAI security credentials for the OpenAI Service that you've created
- Choose a model for your task
- Create a simple prompt for the model
- Submit your request to the model API!
1. Install OpenAI
%pip install openai python-dotenv
2. Import helper libraries and instantiate credentials
import os
from openai import OpenAI
from dotenv import load_dotenv
load_dotenv()
API_KEY = os.getenv("OPENAI_API_KEY","")
assert API_KEY, "ERROR: OpenAI Key is missing"
client = OpenAI(
api_key=API_KEY
)
3. Finding the right model
Models like GPT-4o and GPT-4o mini can understand and generate natural language, and are available on the OpenAI platform with different levels of power and speed suitable for different tasks.
# Select a general purpose chat model
model = "gpt-5-mini"
4. Prompt Design
"The magic of large language models is that by being trained to minimize this prediction error over vast quantities of text, the models end up learning concepts useful for these predictions. For example, they learn concepts like"(1):
- how to spell
- how grammar works
- how to paraphrase
- how to answer questions
- how to hold a conversation
- how to write in many languages
- how to code
- etc.
How to control a large language model
"Of all the inputs to a large language model, by far the most influential is the text prompt(1).
Large language models can be prompted to produce output in a few ways:
Instruction: Tell the model what you want Completion: Induce the model to complete the beginning of what you want Demonstration: Show the model what you want, with either: A few examples in the prompt Many hundreds or thousands of examples in a fine-tuning training dataset"
There are three basic guidelines to creating prompts:
Show and tell. Make it clear what you want either through instructions, examples, or a combination of the two. If you want the model to rank a list of items in alphabetical order or to classify a paragraph by sentiment, show it that's what you want.
Provide quality data. If you're trying to build a classifier or get the model to follow a pattern, make sure that there are enough examples. Be sure to proofread your examples — the model is usually smart enough to see through basic spelling mistakes and give you a response, but it also might assume this is intentional and it can affect the response.
Check your settings. The temperature and top_p settings control how deterministic the model is in generating a response. If you're asking it for a response where there's only one right answer, then you'd want to set these lower. If you're looking for more diverse responses, then you might want to set them higher. The number one mistake people make with these settings is assuming that they're "cleverness" or "creativity" controls.
Source: https://learn.microsoft.com/azure/ai-foundry/openai/overview
5. Submit!
# Create your first prompt
text_prompt = "Should oxford commas always be used?"
response = client.responses.create(
model=model,
input = [{"role":"system", "content":"You are a helpful assistant."},
{"role":"user","content":text_prompt},],
store=False,)
response.output_text
Repeat the same call, how do the results compare?
response = client.responses.create(
model=model,
input = [{"role":"system", "content":"You are a helpful assistant."},
{"role":"user","content":text_prompt},],
store=False,)
response.output_text
Summarize Text
Challenge
Summarize text by adding a 'tl;dr:' to the end of a text passage. Notice how the model understands how to perform a number of tasks with no additional instructions. You can experiment with more descriptive prompts than tl;dr to modify the model’s behavior and customize the summarization you receive(3).
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typically task-agnostic in architecture, this method still requires task-specific fine-tuning datasets of thousands or tens of thousands of examples. By contrast, humans can generally perform a new language task from only a few examples or from simple instructions - something that current NLP systems still largely struggle to do. Here we show that scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with prior state-of-the-art fine-tuning approaches.
Tl;dr
Exercises for several use cases
- Summarize Text
- Classify Text
- Generate New Product Names
prompt = "Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typically task-agnostic in architecture, this method still requires task-specific fine-tuning datasets of thousands or tens of thousands of examples. By contrast, humans can generally perform a new language task from only a few examples or from simple instructions - something that current NLP systems still largely struggle to do. Here we show that scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with prior state-of-the-art fine-tuning approaches.\n\nTl;dr"
#Setting a few additional, typical parameters during API Call
response = client.responses.create(
model=model,
input = [{"role":"system", "content":"You are a helpful assistant."},
{"role":"user","content":prompt},],
store=False,)
response.output_text
Classify Text
Challenge
Classify items into categories provided at inference time. In the following example, we provide both the categories and the text to classify in the prompt(*playground_reference).
Customer Inquiry: Hello, one of the keys on my laptop keyboard broke recently and I'll need a replacement:
Classified category:
prompt = "Classify the following inquiry into one of the following: categories: [Pricing, Hardware Support, Software Support]\n\ninquiry: Hello, one of the keys on my laptop keyboard broke recently and I'll need a replacement:\n\nClassified category:"
print(prompt)
#Setting a few additional, typical parameters during API Call
response = client.responses.create(
model=model,
input = [{"role":"system", "content":"You are a helpful assistant."},
{"role":"user","content":prompt},],
store=False,)
response.output_text
Generate New Product Names
Challenge
Create product names from examples words. Here we include in the prompt information about the product we are going to generate names for. We also provide a similar example to show the pattern we wish to receive. We have also set the temperature value high to increase randomness and more innovative responses.
Product description: A home milkshake maker Seed words: fast, healthy, compact. Product names: HomeShaker, Fit Shaker, QuickShake, Shake Maker
Product description: A pair of shoes that can fit any foot size. Seed words: adaptable, fit, omni-fit.
prompt = "Product description: A home milkshake maker\nSeed words: fast, healthy, compact.\nProduct names: HomeShaker, Fit Shaker, QuickShake, Shake Maker\n\nProduct description: A pair of shoes that can fit any foot size.\nSeed words: adaptable, fit, omni-fit."
print(prompt)
#Setting a few additional, typical parameters during API Call
response = client.responses.create(
model=model,
input = [{"role":"system", "content":"You are a helpful assistant."},
{"role":"user","content":prompt}],
store=False,)
response.output_text
References
For More Help
Contributors
- Louis Li