OER·harvester

← Back to the library
GitHub NOTEBOOK resource

Phi 3 Vision Nividia Nim

Repository 21 Lessons, Get Started Building with Generative AI

Licence
OPEN MIT
Authors
Microsoft (microsoft)
Updated
2024-09-12 · GitHub
Language
en detected
Length
61 words
Open original ↗
{ }
Jupyter notebook Converted to a read-only view · code is not executed

Source: Phi 3 Vision Nividia Nim · GitHub · microsoft/generative-ai-for-beginners Authors: Microsoft (microsoft) Licence: MIT — https://spdx.org/licenses/MIT.html

In [1]
import requests, base64
In [2]
invoke_url = "https://ai.api.nvidia.com/v1/vlm/microsoft/phi-3-vision-128k-instruct"
In [3]
with open("./img/demo.png", "rb") as f:
  image_b64 = base64.b64encode(f.read()).decode()
In [4]
len(image_b64)
In [5]
assert len(image_b64) < 180_000, \
  "To upload larger images, use the assets API (see docs)"
In [6]
stream = False
In [7]
headers = {
  "Authorization": "Bearer Your Nvidia NIM API Key",
  "Accept": "text/event-stream" if stream else "application/json"
}
In [8]
# payload = {
#   "messages": [
#     {
#       "role": "user",
#       "content": f'Tell me what is in the picture <img src="data:image/png;base64,{image_b64}" />'
#     }
#   ],
#   "max_tokens": 1024,
#   "temperature": 0.6,
#   "top_p": 1.0,
#   "stream": stream
# }

payload = {
  "messages": [
    {
      "role": "user",
      "content": f'Please create Python code for image, and use plt to save the new picture under imgs/ and name it phi-3-vision.jpg. <img src="data:image/png;base64,{image_b64}" />'
    }
  ],
  "max_tokens": 1024,
  "temperature": 0.6,
  "top_p": 1.0,
  "stream": stream
}
In [9]
response = requests.post(invoke_url, headers=headers, json=payload)
In [10]
if stream:
    for line in response.iter_lines():
        if line:
            print(line.decode("utf-8"))
else:
    print(response.json())
In [11]
code = response.json()["choices"][0]["message"]["content"]
In [12]
code
In [13]
# Find the starting index of the Python code block within the generated code
# The index function locates the substring '```python' and adds 9 to skip past it
begin = code.index('```python') + 9

# Slice the code string from the calculated starting index to the end
# This removes the initial part of the string up to and including '```python'
code = code[begin:]
In [14]
end = code.index('```')
In [15]
code = code[:end]
In [16]
code