OpenAI API

We provide a usage method compatible with the OpenAI API. You only need to install the StackFlow package to get started.

Preparation

Refer to AI Pyramid Software Package Update to complete the installation of the following model packages and software packages.

apt install lib-llm llm-sys llm-llm llm-openai-api

apt install llm-model-qwen2.5-1.5b-int4-ax650

Note

After installing a new model each time, you need to manually execute systemctl restart llm-openai-api to update the model list.

Curl Call

curl http://127.0.0.1:8000/v1/models \
  -H "Content-Type: application/json"

curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-xxxxxxxx" \
  -d '{
    "model": "qwen2.5-1.5B-Int4-ax650",
    "messages": [
      {"role": "developer", "content": "You are a helpful home assistant."},
      {"role": "user", "content": "Write a one-sentence bedtime story about a unicorn."}
    ]
  }' 

Python Call

from openai import OpenAI
client = OpenAI(
    api_key="sk-",
    base_url="http://127.0.0.1:8000/v1"
)

client.models.list()
print(client.models.list()) 

from openai import OpenAI
client = OpenAI(
    api_key="sk-",
    base_url="http://127.0.0.1:8000/v1"
)

completion = client.chat.completions.create(
  model="qwen2.5-1.5B-Int4-ax650",
  messages=[
    {"role": "developer", "content": "You are a helpful home assistant."},
    {"role": "user", "content": "Turn on the light!"}
  ]
)

print(completion.choices[0].message) 