Product Guide
Linux PC
AI Accelerator Card
LLM-8850 Card
Large Language Models
LLM
Real-Time AI Voice Assistant
XiaoZhi Voice Assistant
AtomS3R-M12 Volcengine Kit
Offline Voice Recognition
Home Assistant
Home Assistant OS
Industrial Control
IoT Measuring Instruments
Air Quality
Module13.2 PPS
VAMeter
T-Lite
Ezdata
Ethernet Camera
PoECAM
Wi-Fi Camera
Unit CamS3/-5MP
AI Camera
LoRa & LoRaWAN
Motor Control
Restore Factory Firmware
DIP Switch Usage Guide
Module GPS v2.0
Module GNSS
Module ExtPort For Core2
Module LoRa868 V1.2
OpenAI API
We provide a set of usage methods compatible with the OpenAI API, simply install the StackFlow package.
Preparation
- Refer to RaspberryPi & LLM8850 Software Package Acquisition Tutorial to complete the installation of the following model packages and software packages.
sudo apt install lib-llm llm-sys llm-llm llm-openai-apisudo apt install llm-model-qwen2.5-ha-0.5b-ctx-axclNote
Each time a new model is installed, you need to manually execute sudo systemctrl restart llm-openai-api to update the model list.
Curl Invocation
curl http://192.168.20.27:8000/v1/models \
-H "Content-Type: application/json"curl http://192.168.20.27:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer sk-xxxxxxxx" \
-d '{
"model": "qwen2.5-HA-0.5B-ctx-axcl",
"messages": [
{"role": "developer", "content": "You are a helpful home assistant."},
{"role": "user", "content": "Turn on the light!"}
]
}'Python Invocation
from openai import OpenAI
client = OpenAI(
api_key="sk-",
base_url="http://192.168.20.27:8000/v1"
)
client.models.list()
print(client.models.list())from openai import OpenAI
client = OpenAI(
api_key="sk-",
base_url="http://192.168.20.27:8000/v1"
)
completion = client.chat.completions.create(
model="qwen2.5-HA-0.5B-ctx-axcl",
messages=[
{"role": "developer", "content": "You are a helpful home assistant."},
{"role": "user", "content": "Turn on the light!"}
]
)
print(completion.choices[0].message)ChatBox Invocation
Get ChatBox
Click settings, and add the model provider.
In API Host, fill in the RaspberryPi's IP and API path, retrieve and add the installed models.
Create a new chat, select the qwen2.5-HA-0.5-axcl model provided by LLM8850.
Modify the maximum context message length to 0.
Supports setting System Prompt.
