Module LLM - LLM Generative Dialogue

This example demonstrates how to use the M5ModuleLLM library on the Arduino platform to call an LLM and implement generative dialogue functionality. The example uses USB serial communication, where questions are sent to the module and responses are received via a serial terminal program.

Preparation

Refer to Module LLM Arduino Quick Start to complete the basic environment setup and installation of the M5ModuleLLM driver library.
Refer to Module LLM Software Package Update Guide to complete the installation of the following model packages.

apt install llm-llm

apt install llm-model-qwen2.5-0.5b-prefill-20e

The hardware devices used in the following example include:

LLM CoreS3

/*
 * SPDX-FileCopyrightText: 2024 M5Stack Technology CO LTD
 *
 * SPDX-License-Identifier: MIT
 */
#include <Arduino.h>
#include <M5Unified.h>
#include <M5ModuleLLM.h>

#define CommSerialPort Serial

M5ModuleLLM module_llm;
String llm_work_id;
String received_question;
bool question_ok;

void setup()
{
    M5.begin();
    M5.Display.setTextSize(2);
    M5.Display.setTextScroll(true);

    /* Init usb serial */
    CommSerialPort.begin(115200);

    /* Init module serial port */
    int rxd = M5.getPin(m5::pin_name_t::port_c_rxd);
    int txd = M5.getPin(m5::pin_name_t::port_c_txd);
    Serial2.begin(115200, SERIAL_8N1, rxd, txd);

    /* Init module */
    module_llm.begin(&Serial2);

    /* Make sure module is connected */
    M5.Display.printf(">> Check ModuleLLM connection..\n");
    while (1) {
        if (module_llm.checkConnection()) {
            break;
        }
    }

    /* Reset ModuleLLM */
    M5.Display.printf(">> Reset ModuleLLM..\n");
    module_llm.sys.reset();

    /* Setup LLM module and save returned work id */
    M5.Display.printf(">> Setup llm..\n");
    m5_module_llm::ApiLlmSetupConfig_t llm_config;
    llm_config.max_token_len = 1023;
    llm_config.setParam("enable_temperature", true);
    llm_config.setParam("temperature", 0.7f);

    llm_config.setParam("enable_top_p_sampling", true);
    llm_config.setParam("top_p", 0.9f);

    llm_config.setParam("enable_top_k_sampling", true);
    llm_config.setParam("top_k", 50);

    llm_config.setParam("enable_repetition_penalty", true);
    llm_config.setParam("repetition_penalty", 1.05f);
    llm_config.setParam("penalty_window", 20);

    llm_work_id = module_llm.llm.setup(llm_config);

    M5.Display.printf(">> Setup finish\n");
    M5.Display.printf(">> Try send your question via usb serial port\n");
    M5.Display.setTextColor(TFT_GREEN);
    M5.Display.printf("e.g. \nHi, What's your name?\n");
    M5.Display.printf("(end with CRLF \\r\\n)\n\n");
}

void loop()
{
    /* Check comm serial port and get received question */
    question_ok = false;
    if (CommSerialPort.available()) {
        while (CommSerialPort.available()) {
            char in_char = (char)CommSerialPort.read();
            received_question += in_char;

            /* Check if question finish */
            if (received_question.endsWith("\r\n")) {
                received_question.remove(received_question.length() - 2);
                question_ok = true;
                break;
            }
        }
    }

    /* If question is ready */
    if (question_ok) {
        M5.Display.setTextColor(TFT_GREEN);
        M5.Display.printf("<< %s\n", received_question.c_str());
        M5.Display.setTextColor(TFT_YELLOW);
        M5.Display.printf(">> ");
        CommSerialPort.printf("<< \"%s\"\n", received_question.c_str());
        CommSerialPort.print(">> ");

        /* Push question to LLM module and wait inference result */
        module_llm.llm.inferenceAndWaitResult(llm_work_id, received_question.c_str(), [](String& result) {
            /* Show result on screen and usb serial */
            M5.Display.printf("%s", result.c_str());
            CommSerialPort.print(result);
        });

        /* Clear for next question */
        received_question.clear();

        M5.Display.println();
        CommSerialPort.println();
    }

    delay(20);
} 

Note: Parameters such as 'temperature', 'top_p', 'top_k', 'repetition_penalty' are only effective when the software version llm_llm >= 1.9. Please refer to Module LLM Software Package Update Guide to upgrade the pre-installed base software on ModuleLLM.

Upload the code to CoreS3.
Open the Serial Monitor in Arduino IDE, set the baud rate to 115200, and configure "Line Ending \r\n". Enter text in the Serial Monitor and press Enter to send. Module LLM will process the input and return a response.

Next Overview

Overview

Devices & Quick Start

Module LLM

LLM630 Compute Kit

Models

Qwen2.5

Qwen3

DeepSeek-R1

SmolVLM

MeloTTS

Whisper

Llama

Applications

Audio

CV Vision Application

Vision Language Model (VLM)

Large Language Model (LLM)

Voice Assistant

OpenAI API

Module LLM - LLM Generative Dialogue

Preparation

LLM CoreS3

On This Page