UnitV2 Arduino Tutorial

1. Preparation

Environment configuration: Refer to Arduino IDE Getting Started Tutorial to complete the IDE installation, then install the corresponding board manager and necessary driver libraries based on your actual development board.
Required driver libraries:
Required hardware products:

2. Notes

Pin Compatibility

Due to different pin configurations of each host device, for user convenience, M5Stack provides a pin compatibility table. Please modify the example code based on the actual pin connection situation.

3. Compile and Upload

Copy and paste the example code below into your project code area as needed, then select the device port (for details refer to Program Compile and Flash), click the compile and upload button in the top left of Arduino IDE, and wait for the program to finish compiling and uploading to the device.

4. UnitV2 Connection Methods

4.1 Ethernet Mode Connection (USB Connection)

This connection method requires you to first download and install the corresponding SR9900 driver according to your operating system. Then connect UnitV2 to the computer via the USB Type-C port. UnitV2 has a built-in wired network card, and the computer will automatically recognize it as a network adapter device, then automatically establish a network connection with UnitV2.

4.2 AP Mode Connection (Wi-Fi Connection)

This connection method does not require any drivers. After powering on UnitV2, it will automatically enable a Wi-Fi hotspot with the SSID UnitV2-XXXX (XXXX varies by device) and the password 12345678. Connect to this hotspot using a computer or mobile phone, and you can establish a network connection with UnitV2.

5. Example Programs

All the following examples are only applicable to M5Stack official firmware. They are used to switch functions and parse JSON data returned by UnitV2. For official firmware, see UnitV2 Firmware Update Tutorial. For more information about UnitV2, see here.

Notice

1. UnitV2's settings and return data format are all in JSON. For the JSON data format of each function, see the prompt before each example.
2.UnitV2 needs to be powered via the USB Type-C port to work. After plugging/unplugging the USB Type-C port, the unit will restart.
3.The functional settings of this unit will be lost after power off. After restarting, you need to reconfigure the function. However, saved data and parameter configurations under each function will not be lost after power off.

5.1 Camera Stream

This example can configure the UnitV2 to switch to the video streaming function, where the camera feed can be viewed in real time on the function page. Access the function page in a browser by visiting the domain unitv2.py or IP: 10.254.239.1. The page content is as follows:

For the JSON data format of the video stream function, see here.

cpp 
#include <M5Unified.h>
#include <M5GFX.h>
#include <Arduino_JSON.h>

M5GFX display;

void setup() {

    display.begin();
    display.setRotation(1);
    display.clear(TFT_WHITE);
    display.setFont(&fonts::FreeMonoBold9pt7b);
    display.setTextColor(TFT_BLACK);
    delay(100);
    display.drawString("UnitV2 Json Example", 5, 5);
    display.drawLine(0, 25, 320, 25, TFT_BLACK);

    Serial.begin(115200);
    Serial2.begin(115200, SERIAL_8N1, 13, 14);//PORT.C

    //Setting JSON
    JSONVar obj;
    obj["function"] = "Camera Stream";                                        
    obj["args"] = "";           
    String jsonString = JSON.stringify(obj);
    Serial2.println(jsonString);
    Serial2.flush();
}

void loop() {

    if (Serial2.available() > 0) {
      String line = Serial2.readStringUntil('\r');
      while (line.length() && line[0] != '{') {// clear '\0'
          line.remove(0, 1);
      }
      Serial2.flush();

      JSONVar camera_stream_obj = JSON.parse(line);

      // JSON.typeof(jsonVar) can be used to get the type of the var
      if (!(JSON.typeof(camera_stream_obj) == "undefined")){
        display.fillRect(0, 35, 320, 205, TFT_WHITE);
        Serial.println(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
        Serial.println("UnitV2 Camera Stream example");
        if (camera_stream_obj.hasOwnProperty("msg")) {
          Serial.print("msg : ");
          Serial.println(String(code_obj["msg"]).c_str());
          display.setCursor(0, 35);
          display.printf(" msg: %s\n", String(code_obj["msg"]).c_str());
        }
        if (camera_stream_obj.hasOwnProperty("running")) {
          Serial.print("running : ");
          Serial.println(String(code_obj["running"]).c_str());
          display.printf(" running: %s\n", String(code_obj["running"]).c_str());
        }        
        Serial.println("<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<");
      } else {
        return;
      }
    }
} 

5.2 Code Detector

This example can configure UnitV2 to switch to the QR code recognition function. This function can recognize common codes such as QR codes, barcodes, DataMatrix codes, etc. In a browser, visit the domain name unitv2.py or IP 10.254.239.1 to access the function page. The page content is shown below:

For the JSON data format of the QR code recognition function, see here.

cpp 
#include <M5Unified.h>                
#include <M5GFX.h>                  
#include <Arduino_JSON.h>          

M5GFX display;                
int code_cnt = 0;             

void setup() {
    display.begin();                 
    display.setRotation(1);        
    display.clear(TFT_WHITE);     
    display.setFont(&fonts::FreeMonoBold9pt7b);
    display.setTextColor(TFT_BLACK); 
    delay(100);                    
    display.drawString("UnitV2 Json Example", 5, 5); 
    display.drawLine(0, 25, 320, 25, TFT_BLACK);  
    Serial.begin(115200);          
    Serial2.begin(115200, SERIAL_8N1, 13, 14); 

    // Create initial JSON for configuration
    JSONVar obj;
    obj["function"] = "Code Detector";                         
    obj["args"] = "";                 
    String jsonString = JSON.stringify(obj); // Convert JSON to string
    Serial2.println(jsonString);       // Send JSON string to Serial2
    Serial2.flush();                   // Flush Serial2 buffer
}

void loop() {
    if (Serial2.available() > 0) {         
        String line = Serial2.readStringUntil('\r'); 
        while (line.length() && line[0] != '{') { // Remove non-JSON starting characters
            line.remove(0, 1);
        }
        Serial2.flush();       

        JSONVar code_obj = JSON.parse(line);

        if (!(JSON.typeof(code_obj) == "undefined")) { 
            display.fillRect(0, 35, 320, 205, TFT_WHITE);
            Serial.println(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
            Serial.println("UnitV2 Code Detector example");   

            // Configuration response
            if (code_obj.hasOwnProperty("msg")) {
                Serial.print("msg : ");
                Serial.println(String(code_obj["msg"]).c_str());
                display.setCursor(0, 35);
                display.printf(" msg: %s\n", String(code_obj["msg"]).c_str());
                if (code_obj.hasOwnProperty("running")) {
                  Serial.print("running : ");
                  Serial.println(String(code_obj["running"]).c_str());
                  display.printf(" running: %s\n", String(code_obj["running"]).c_str());
                }
            } else {
              if (code_obj.hasOwnProperty("running")) {
                  Serial.print("running : ");
                  Serial.println(String(code_obj["running"]).c_str());
                  display.setCursor(0, 35);
                  display.printf(" running: %s\n", String(code_obj["running"]).c_str());
              }
            }
           
            if (code_obj.hasOwnProperty("num")) {
                code_cnt = (int)code_obj["num"];
                Serial.printf("num = %d\n", code_cnt);
                display.printf(" num: %d\n", code_cnt);
            }
            
            if (code_obj.hasOwnProperty("code")) {
                JSONVar code_arr = code_obj["code"];
                for (int i = 0; i < (int)code_cnt; i++) {
                    display.fillRect(0, 80, 320, 160, TFT_WHITE);
                    display.setCursor(0, 80);
                    if (JSON.typeof(code_arr[i]) != "undefined") {
                        Serial.printf("------[Code %d]------\n", i);
                        Serial.printf("\tprob: %f\n", (double)code_arr[i]["prob"]);
                        Serial.printf("\tx: %d\n", (int)code_arr[i]["x"]);
                        Serial.printf("\ty: %d\n", (int)code_arr[i]["y"]);
                        Serial.printf("\tw: %d\n", (int)code_arr[i]["w"]);
                        Serial.printf("\th: %d\n", (int)code_arr[i]["h"]);
                        Serial.printf("\ttype: %s\n", (const char*)code_arr[i]["type"]);
                        Serial.printf("\tcontent: %s\n", (const char*)code_arr[i]["content"]);
                        display.printf("Code %d:\n", i);
                        display.printf(" prob: %.3f\n", (double)code_arr[i]["prob"]);
                        display.printf(" x: %d, y: %d\n", (int)code_arr[i]["x"], (int)code_arr[i]["y"]);
                        display.printf(" w: %d, h: %d\n", (int)code_arr[i]["w"], (int)code_arr[i]["h"]);
                        display.printf(" type: %s\n", (const char*)code_arr[i]["type"]);
                        display.printf(" content: %s\n\n", (const char*)code_arr[i]["content"]);
                    }
                    delay(200);
                }
            }
            Serial.println("<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<");
        }
    }
}

Scan the QR code on the left below; the recognition result is shown in the right image.

5.3 Object Recognition

This example can configure UnitV2 to switch to the object recognition function. The factory firmware comes with nanodet_80class and yolo_20class models, ready to use. If you wish to train a custom model, check the tutorial UnitV2 V-Training. In a browser, visit unitv2.py or IP 10.254.239.1 to access the function page. The page content is shown below:

For the JSON data format of the object recognition function, see here.

cpp 
#include <M5Unified.h>
#include <M5GFX.h>
#include <Arduino_JSON.h>

M5GFX display;
int obj_cnt = 0;

void setup() {
    display.begin();
    display.setRotation(1);
    display.clear(TFT_WHITE);
    display.setFont(&fonts::FreeMonoBold9pt7b);
    display.setTextColor(TFT_BLACK);
    delay(100);
    display.drawString("UnitV2 Json Example", 5, 5);
    display.drawLine(0, 25, 320, 25, TFT_BLACK);
    Serial.begin(115200);
    Serial2.begin(115200, SERIAL_8N1, 13, 14);//PORT.C
    //Setting JSON
    JSONVar obj;
    obj["function"] = "Object Recognition";                                
    obj["args"][0] = "yolo_20class";   // yolo_20class or nanodet_80class  
    String jsonString = JSON.stringify(obj);
    Serial2.println(jsonString);
    Serial2.flush();
}

void loop() {
    if (Serial2.available() > 0) {
        String line = Serial2.readStringUntil('\r');
        while (line.length() && line[0] != '{') { // clear '\0'
            line.remove(0, 1);
        }
        Serial2.flush();
        JSONVar rec_obj = JSON.parse(line);

        if (!(JSON.typeof(rec_obj) == "undefined")) {
            display.fillRect(0, 35, 320, 205, TFT_WHITE);
            Serial.println(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
            Serial.println("UnitV2 Object Recognition example");            

            if (rec_obj.hasOwnProperty("msg")) {
                Serial.print("msg = ");
                Serial.println((const char*) rec_obj["msg"]);
                display.setCursor(0, 35);
                display.printf(" msg: %s\n", (const char*) rec_obj["msg"]);
                if (rec_obj.hasOwnProperty("running")) {
                  Serial.print("running = ");
                  Serial.println(String(rec_obj["running"]).c_str());
                  display.printf(" running: %s\n", String(rec_obj["running"]).c_str());
                }
            } else {
              if (rec_obj.hasOwnProperty("running")) {
                  Serial.print("running = ");
                  Serial.println(String(rec_obj["running"]).c_str());
                  display.setCursor(0, 35);
                  display.printf(" running: %s\n", String(rec_obj["running"]).c_str());
              }
            }
            if (rec_obj.hasOwnProperty("num")) {
                obj_cnt = (int)rec_obj["num"];
                Serial.printf("num = %d\n", obj_cnt);
                display.printf(" num: %d\n", obj_cnt);
            }

            if (rec_obj.hasOwnProperty("obj")) {
                JSONVar obj_arr = rec_obj["obj"];
                for (int i = 0; i < (int)obj_cnt; i++) {
                    display.fillRect(0, 80, 320, 160, TFT_WHITE);
                    display.setCursor(0, 80);
                    if (JSON.typeof(obj_arr[i]) != "undefined") {
                        Serial.printf("------[obj %d]------\n", i);
                        Serial.printf("\tprob: %f\n", (double)obj_arr[i]["prob"]);
                        Serial.printf("\tx: %d\n", (int)obj_arr[i]["x"]);
                        Serial.printf("\ty: %d\n", (int)obj_arr[i]["y"]);
                        Serial.printf("\tw: %d\n", (int)obj_arr[i]["w"]);
                        Serial.printf("\th: %d\n", (int)obj_arr[i]["h"]);
                        Serial.printf("\ttype: %s\n", (const char*)obj_arr[i]["type"]);
                        
                        display.printf("obj %d:\n", i);
                        display.printf(" prob: %.3f\n", (double)obj_arr[i]["prob"]);
                        display.printf(" x: %d, y: %d\n", (int)obj_arr[i]["x"], (int)obj_arr[i]["y"]);
                        display.printf(" w: %d, h: %d\n", (int)obj_arr[i]["w"], (int)obj_arr[i]["h"]);
                        display.printf(" type: %s\n\n", (const char*)obj_arr[i]["type"]);
                    }
                    delay(200);
                }
            }
            Serial.println("<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<");
        }
    }
}

Scan the image on the left below; the recognition result is shown in the right image.

5.4 Color Tracker

This example can configure UnitV2 to switch to the color tracking function. The color tracking function requires color parameter settings after powering on UnitV2 to work properly. There are two ways to set the color parameters:
1.Click and select the color area to track in the function page image (operation shown below) or enter L, A, B values on the left and click the update button to set.

2.Use JSON settings in the code below. You can set by defining an ROI and auto-analyzing the LAB color value, or by setting the LAB color value directly. The default code uses ROI; to switch to LAB, uncomment #define USING_LAB and enter LAB values as shown:

    LAB["l_min"] = 0; 
    LAB["l_max"] = 255; 
    LAB["a_min"] = 179; 
    LAB["a_max"] = 201; 
    LAB["b_min"] = 162; 
    LAB["b_max"] = 184;  

Access the function page at unitv2.py or IP 10.254.239.1. The page content is shown below:

For the JSON data format of the color tracking function, see here.

cpp 
#include <M5Unified.h>
#include <M5GFX.h>
#include <Arduino_JSON.h>

// Enabling this macro definition will use LAB; otherwise, the default is to use ROI.
// #define USING_LAB

M5GFX display;

void setup() {
    display.begin();
    display.setRotation(1);
    display.clear(TFT_WHITE);
    display.setFont(&fonts::FreeMonoBold9pt7b);
    display.setTextColor(TFT_BLACK);
    delay(100);
    display.drawString("UnitV2 Json Example", 5, 5);
    display.drawLine(0, 25, 320, 25, TFT_BLACK);
    Serial.begin(115200);
    Serial2.begin(115200, SERIAL_8N1, 13, 14);//PORT.C
    //Setting JSON
    JSONVar obj;
    obj["function"] = "Color Tracker";                                
    obj["args"] = "";   
    String jsonString = JSON.stringify(obj);
    Serial2.println(jsonString);
    Serial2.flush();
    delay(200);
#if defined(USING_LAB) 
    JSONVar LAB;
    LAB["config"] = "Color Tracker";                                
    LAB["l_min"] = 0; 
    LAB["l_max"] = 255; 
    LAB["a_min"] = 179; 
    LAB["a_max"] = 201; 
    LAB["b_min"] = 162; 
    LAB["b_max"] = 184;   
    jsonString = JSON.stringify(LAB);
#else
    JSONVar ROI;
    ROI["config"] = "Color Tracker";                                
    ROI["x"] = 160; 
    ROI["y"] = 120; 
    ROI["w"] = 320; 
    ROI["h"] = 240;  
    jsonString = JSON.stringify(ROI);
#endif
    Serial2.println(jsonString);
    Serial2.flush();
    delay(200);
}

void loop() {
    if (Serial2.available() > 0) {
        String line = Serial2.readStringUntil('\r');
        while (line.length() && line[0] != '{') { // clear '\0'
            line.remove(0, 1);
        }
        Serial2.flush();
        JSONVar color_obj = JSON.parse(line);

        if (!(JSON.typeof(color_obj) == "undefined")) {
            display.fillRect(0, 35, 320, 205, TFT_WHITE);
            Serial.println(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
            Serial.println("UnitV2 Color Tracker example");            

            if (color_obj.hasOwnProperty("msg")) {
                Serial.print("msg = ");
                Serial.println(String(color_obj["msg"]).c_str());
                display.setCursor(0, 35);
                display.printf(" msg: %s\n", String(color_obj["msg"]).c_str());
                if (color_obj.hasOwnProperty("running")) {
                  Serial.print("running = ");
                  Serial.println(String(color_obj["running"]).c_str());
                  display.printf(" running: %s\n", String(color_obj["running"]).c_str());
                }
            } else {
              if (color_obj.hasOwnProperty("running")) {
                  Serial.print("running = ");
                  Serial.println(String(color_obj["running"]).c_str());
                  display.setCursor(0, 35);
                  display.printf(" running: %s\n", String(color_obj["running"]).c_str());
              }
              // ROI Setting Response
              if (color_obj.hasOwnProperty("a_cal")) {
                  double a_cal = (double)color_obj["a_cal"];
                  double b_cal = (double)color_obj["b_cal"];
                  double va    = (double)color_obj["va"];
                  double vb    = (double)color_obj["vb"];

                  int l_min   = (int)color_obj["l_min"];
                  int l_max   = (int)color_obj["l_max"];
                  int a_min   = (int)color_obj["a_min"];
                  int a_max   = (int)color_obj["a_max"];
                  int b_min   = (int)color_obj["b_min"];
                  int b_max   = (int)color_obj["b_max"];

                  Serial.printf("a_cal = %f\n", a_cal);
                  Serial.printf("b_cal = %f\n", b_cal);
                  Serial.printf("va = %f\n", va);
                  Serial.printf("vb = %f\n", vb);
                  Serial.printf("l_min = %d\nl_max = %d\n", l_min, l_max);
                  Serial.printf("a_min = %d\na_max = %d\nb_min = %d\nb_max = %d\n",
                                a_min, a_max, b_min, b_max);

                  display.printf(" a_cal: %f\n", a_cal);
                  display.printf(" b_cal: %f\n", b_cal);
                  display.printf(" va: %f\n", va);
                  display.printf(" vb: %f\n", vb);
                  display.printf(" l_min: %d\n", l_min);
                  display.printf(" l_max: %d\n", l_max);
                  display.printf(" a_min: %d\n", a_min);
                  display.printf(" a_max: %d\n", a_max);
                  display.printf(" b_min: %d\n", b_min);
                  display.printf(" b_max: %d\n", b_max);
                  delay(2000);
              }
            }

            if (color_obj.hasOwnProperty("cx")) {
                int cx   = (int)color_obj["cx"];
                int cy   = (int)color_obj["cy"];
                int r   = (int)color_obj["r"];
                int mx   = (int)color_obj["mx"];
                int my   = (int)color_obj["my"];

                Serial.printf("cx = %d\n", cx);
                Serial.printf("cy = %d\n", cy);
                Serial.printf("r  = %d\n", r);
                Serial.printf("mx = %d\n", mx);
                Serial.printf("my = %d\n", my);

                display.setCursor(0, 60);
                display.printf(" cx = %d\n", cx);
                display.printf(" cy = %d\n", cy);
                display.printf(" r  = %d\n", r);
                display.printf(" mx = %d\n", mx);
                display.printf(" my = %d\n", my);
            }

            Serial.println("<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<");
        }
    }
} 

When running successfully, the host will display the position, size, etc. of the tracked color block's center point in real time. Result:

5.5 Lane Line Tracker

Similar to color tracking, lane line tracking requires setting color parameters after powering on UnitV2. See Color Tracking above for parameter setting methods. Visit unitv2.py or IP 10.254.239.1 to access the function page:

The code below uses the ROI method by default. If you want to use the LAB method instead, uncomment #define USING_LAB in the code, and enter the LAB color values in the section shown below.

    LAB["l_min"] = 0; 
    LAB["l_max"] = 255; 
    LAB["a_min"] = 176; 
    LAB["a_max"] = 198; 
    LAB["b_min"] = 155; 
    LAB["b_max"] = 175;  

For the JSON data format of the lane line tracking function, see here.

cpp 
#include <M5Unified.h>
#include <M5GFX.h>
#include <Arduino_JSON.h>

// Enabling this macro definition will use LAB; otherwise, the default is to use ROI.
// #define USING_LAB

M5GFX display;

void setup() {
    display.begin();
    display.setRotation(1);
    display.clear(TFT_WHITE);
    display.setFont(&fonts::FreeMonoBold9pt7b);
    display.setTextColor(TFT_BLACK);
    delay(100);
    display.drawString("UnitV2 Json Example", 5, 5);
    display.drawLine(0, 25, 320, 25, TFT_BLACK);
    Serial.begin(115200);
    Serial2.begin(115200, SERIAL_8N1, 13, 14);//PORT.C
    //Setting JSON
    JSONVar obj;
    obj["function"] = "Lane Line Tracker";                                
    obj["args"] = "";   
    String jsonString = JSON.stringify(obj);
    Serial2.println(jsonString);
    Serial2.flush();
    delay(200);
#if defined(USING_LAB) 
    JSONVar LAB;
    LAB["config"] = "Lane Line Tracker";                                
    LAB["l_min"] = 0; 
    LAB["l_max"] = 255; 
    LAB["a_min"] = 176; 
    LAB["a_max"] = 198; 
    LAB["b_min"] = 155; 
    LAB["b_max"] = 175;   
    jsonString = JSON.stringify(LAB);
#else
    JSONVar ROI;
    ROI["config"] = "Lane Line Tracker";                                
    ROI["x"] = 160; 
    ROI["y"] = 120; 
    ROI["w"] = 320; 
    ROI["h"] = 240;  
    jsonString = JSON.stringify(ROI);
#endif
    Serial2.println(jsonString);
    Serial2.flush();
    delay(200);
}

void loop() {
    if (Serial2.available() > 0) {
        String line = Serial2.readStringUntil('\r');
        while (line.length() && line[0] != '{') { // clear '\0'
            line.remove(0, 1);
        }
        Serial2.flush();
        JSONVar line_obj = JSON.parse(line);

        if (!(JSON.typeof(line_obj) == "undefined")) {
            display.fillRect(0, 35, 320, 205, TFT_WHITE);
            Serial.println(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
            Serial.println("UnitV2 Lane Line Tracker example");            

            if (line_obj.hasOwnProperty("msg")) {
                Serial.print("msg = ");
                Serial.println(String(line_obj["msg"]).c_str());
                display.setCursor(0, 35);
                display.printf(" msg: %s\n", String(line_obj["msg"]).c_str());
                if (line_obj.hasOwnProperty("running")) {
                  Serial.print("running = ");
                  Serial.println(String(line_obj["running"]).c_str());
                  display.printf(" running: %s\n", String(line_obj["running"]).c_str());
                }
            } else {
              if (line_obj.hasOwnProperty("running")) {
                  Serial.print("running = ");
                  Serial.println(String(line_obj["running"]).c_str());
                  display.setCursor(0, 35);
                  display.printf(" running: %s\n", String(line_obj["running"]).c_str());
              }
              // ROI Setting Response
              if (line_obj.hasOwnProperty("a_cal")) {
                  double a_cal = (double)line_obj["a_cal"];
                  double b_cal = (double)line_obj["b_cal"];
                  double va    = (double)line_obj["va"];
                  double vb    = (double)line_obj["vb"];

                  int l_min   = (int)line_obj["l_min"];
                  int l_max   = (int)line_obj["l_max"];
                  int a_min   = (int)line_obj["a_min"];
                  int a_max   = (int)line_obj["a_max"];
                  int b_min   = (int)line_obj["b_min"];
                  int b_max   = (int)line_obj["b_max"];

                  Serial.printf("a_cal = %f\n", a_cal);
                  Serial.printf("b_cal = %f\n", b_cal);
                  Serial.printf("va = %f\n", va);
                  Serial.printf("vb = %f\n", vb);
                  Serial.printf("l_min = %d\nl_max = %d\n", l_min, l_max);
                  Serial.printf("a_min = %d\na_max = %d\nb_min = %d\nb_max = %d\n",
                                a_min, a_max, b_min, b_max);

                  display.printf(" a_cal: %f\n", a_cal);
                  display.printf(" b_cal: %f\n", b_cal);
                  display.printf(" va: %f\n", va);
                  display.printf(" vb: %f\n", vb);
                  display.printf(" l_min: %d\n", l_min);
                  display.printf(" l_max: %d\n", l_max);
                  display.printf(" a_min: %d\n", a_min);
                  display.printf(" a_max: %d\n", a_max);
                  display.printf(" b_min: %d\n", b_min);
                  display.printf(" b_max: %d\n", b_max);
                  delay(2000);
              }
            }

            if (line_obj.hasOwnProperty("x")) {
                int x  = (int)line_obj["x"];
                int y  = (int)line_obj["y"];
                double k  = (double)line_obj["k"];

                Serial.printf("x = %d\n", x);
                Serial.printf("y = %d\n", y);
                Serial.printf("k = %f\n", k);
                
                display.setCursor(0, 60);
                display.printf(" x = %d\n", x);
                display.printf(" y = %d\n", y);
                display.printf(" k = %f\n", k);                
            }

            Serial.println("<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<");
        }
    }
} 

Scan the left image; result shown on the right:

5.6 Target Tracker

This example can configure UnitV2 to switch to the target tracking function. You need to grab target data from the current camera frame after powering on UnitV2, either by selecting in the function page (unitv2.py or IP 10.254.239.1) or by code parameters below:

    obj["x"] = 160;  //start point x-coordinate           
    obj["y"] = 120;  //start point y-coordinate                             
    obj["w"] = 320;  //width                            
    obj["h"] = 240;  //heigth 

For JSON format, see here.

cpp 
#include <M5Unified.h>
#include <M5GFX.h>
#include <Arduino_JSON.h>

M5GFX display;

void setup() {
    display.begin();
    display.setRotation(1);
    display.clear(TFT_WHITE);
    display.setFont(&fonts::FreeMonoBold9pt7b);
    display.setTextColor(TFT_BLACK);
    delay(100);
    display.drawString("UnitV2 Json Example", 5, 5);
    display.drawLine(0, 25, 320, 25, TFT_BLACK);
    Serial.begin(115200);
    Serial2.begin(115200, SERIAL_8N1, 13, 14);//PORT.C
    //Setting JSON
    JSONVar obj;
    obj["function"] = "Target Tracker";                                
    obj["args"] = "";   
    String jsonString = JSON.stringify(obj);
    Serial2.println(jsonString);
    Serial2.flush();
    delay(200);
    JSONVar ROI;
    ROI["config"] = "Target Tracker";                                
    ROI["x"] = 160; 
    ROI["y"] = 120; 
    ROI["w"] = 320; 
    ROI["h"] = 240;  
    jsonString = JSON.stringify(ROI);
    Serial2.println(jsonString);
    Serial2.flush();
    delay(200);
}

void loop() {
    if (Serial2.available() > 0) {
        String line = Serial2.readStringUntil('\r');
        while (line.length() && line[0] != '{') { // clear '\0'
            line.remove(0, 1);
        }
        Serial2.flush();
        JSONVar target_obj = JSON.parse(line);

        if (!(JSON.typeof(target_obj) == "undefined")) {
            display.fillRect(0, 35, 320, 205, TFT_WHITE);
            Serial.println(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
            Serial.println("UnitV2 Target Tracker example");            

            if (target_obj.hasOwnProperty("msg")) {
                Serial.print("msg = ");
                Serial.println(String(target_obj["msg"]).c_str());
                display.setCursor(0, 35);
                display.printf(" msg: %s\n", String(target_obj["msg"]).c_str());
                if (target_obj.hasOwnProperty("running")) {
                  Serial.print("running = ");
                  Serial.println(String(target_obj["running"]).c_str());
                  display.printf(" running: %s\n", String(target_obj["running"]).c_str());
                }
            } else {
              if (target_obj.hasOwnProperty("running")) {
                  Serial.print("running = ");
                  Serial.println(String(target_obj["running"]).c_str());
                  display.setCursor(0, 35);
                  display.printf(" running: %s\n", String(target_obj["running"]).c_str());
              }
            }

            if (target_obj.hasOwnProperty("x")) {
                int x  = (int)target_obj["x"];
                int y  = (int)target_obj["y"];
                int w  = (int)target_obj["w"];
                int h  = (int)target_obj["h"];

                Serial.printf("x = %d\n", x);
                Serial.printf("y = %d\n", y);
                Serial.printf("w = %d\n", w);
                Serial.printf("h = %d\n", h);
                
                display.setCursor(0, 60);
                display.printf(" x = %d\n", x);
                display.printf(" y = %d\n", y);
                display.printf(" w = %d\n", w);
                display.printf(" h = %d\n", h);                
            }

            Serial.println("<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<");
        }
    }
} 

After grabbing target data successfully, the host displays position data in real time:

5.7 Motion Tracker

This example configures UnitV2 for motion tracking. Before using, set a background once for effective tracking. Access via browser: unitv2.py or IP 10.254.239.1.

For JSON format, see here.

cpp 
#include <M5Unified.h>
#include <M5GFX.h>
#include <Arduino_JSON.h>

M5GFX display;
int motion_cnt = 0;

void setup() {
    display.begin();
    display.setRotation(1);
    display.clear(TFT_WHITE);
    display.setFont(&fonts::FreeMonoBold9pt7b);
    display.setTextColor(TFT_BLACK);
    delay(100);
    display.drawString("UnitV2 Json Example", 5, 5);
    display.drawLine(0, 25, 320, 25, TFT_BLACK);
    Serial.begin(115200);
    Serial2.begin(115200, SERIAL_8N1, 13, 14);//PORT.C
    //Setting JSON
    JSONVar obj;
    obj["function"] = "Motion Tracker";                                
    obj["args"] = "";   
    String jsonString = JSON.stringify(obj);
    Serial2.println(jsonString);
    Serial2.flush();
    delay(200);
    JSONVar BCKG;
    BCKG["config"] = "Motion Tracker";                                
    BCKG["operation"] = "update";  
    jsonString = JSON.stringify(BCKG);
    Serial2.println(jsonString);
    Serial2.flush();
    delay(200);
}

void loop() {
    if (Serial2.available() > 0) {
        String line = Serial2.readStringUntil('\r');
        while (line.length() && line[0] != '{') { // clear '\0'
            line.remove(0, 1);
        }
        Serial2.flush();
        JSONVar motion_obj = JSON.parse(line);

        if (!(JSON.typeof(motion_obj) == "undefined")) {
            display.fillRect(0, 35, 320, 205, TFT_WHITE);
            Serial.println(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
            Serial.println("UnitV2 Target Tracker example");            

            if (motion_obj.hasOwnProperty("msg")) {//此处是为了接收一次配置返回消息
                Serial.print("msg = ");
                Serial.println(String(motion_obj["msg"]).c_str());
                display.setCursor(0, 35);
                display.printf(" msg: %s\n", String(motion_obj["msg"]).c_str());
                if (motion_obj.hasOwnProperty("running")) {
                  Serial.print("running = ");
                  Serial.println(String(motion_obj["running"]).c_str());
                  display.printf(" running: %s\n", String(motion_obj["running"]).c_str());
                }
            } else {
              if (motion_obj.hasOwnProperty("running")) {
                  Serial.print("running = ");
                  Serial.println(String(motion_obj["running"]).c_str());
                  display.setCursor(0, 35);
                  display.printf(" running: %s\n", String(motion_obj["running"]).c_str());
              }
            }

            if (motion_obj.hasOwnProperty("num")) {
                motion_cnt = (int)motion_obj["num"];
                Serial.printf("num = %d\n", motion_cnt);
                display.printf(" num: %d\n", motion_cnt);
            }

            // 遍历 obj 数组
            if (motion_obj.hasOwnProperty("roi")) {
                JSONVar obj_arr = motion_obj["roi"];
                for (int i = 0; i < (int)motion_cnt; i++) {
                    display.fillRect(0, 80, 320, 160, TFT_WHITE);
                    display.setCursor(0, 80);
                    if (JSON.typeof(obj_arr[i]) != "undefined") {
                        Serial.printf("------[roi %d]------\n", i);
                        Serial.printf("\tx: %d\n", (int)obj_arr[i]["x"]);
                        Serial.printf("\ty: %d\n", (int)obj_arr[i]["y"]);
                        Serial.printf("\tw: %d\n", (int)obj_arr[i]["w"]);
                        Serial.printf("\th: %d\n", (int)obj_arr[i]["h"]);
                        Serial.printf("\tangle: %f\n", (double)obj_arr[i]["angle"]);
                        Serial.printf("\tarea: %d\n", (int)obj_arr[i]["area"]);
                        
                        display.printf("roi %d:\n", i);
                        display.printf(" x: %d, y: %d\n", (int)obj_arr[i]["x"], (int)obj_arr[i]["y"]);
                        display.printf(" w: %d, h: %d\n", (int)obj_arr[i]["w"], (int)obj_arr[i]["h"]);
                        display.printf(" angle: %f\n", (double)obj_arr[i]["angle"]);
                        display.printf(" area: %d\n", (int)obj_arr[i]["area"]);
                    }
                    delay(200);
                }
            }

            Serial.println("<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<");
        }
    }
} 

When UnitV2 detects motion, it returns position data in real time:

5.8 Online Classifier

This example configures UnitV2 for the online classification function. Factory firmware includes Core, UnitV, StickC, Atom categories. To customize: click add to create, train to train, save&run to save and exit. Access via unitv2.py or IP: 10.254.239.1.

You can also use code functions enter_training_mode, save_and_start. This example creates and trains an earphone category 10 times.

Note

The reset operation (click reset or call enter_training_mode_with_all_classes_clear) permanently deletes all trained categories, including defaults, and they will not be restored after power cycle.

For JSON format, see here.

cpp 
#include <M5Unified.h>
#include <M5GFX.h>
#include <Arduino_JSON.h>

M5GFX display;
int cla_cnt = 0;

String enter_training_mode(int id, String name) {
    JSONVar json;
    json["config"] = "Online Classifier";                                
    json["operation"] = "train";  
    json["class_id"] = id; //0~N
    json["class"] = name;  //format: "class_name"
    return JSON.stringify(json);
}

String enter_training_mode_with_all_classes_clear(void) {
    JSONVar json;
    json["config"] = "Online Classifier";                                
    json["operation"] = "reset";  
    return JSON.stringify(json);
}

String save_and_start(void) {
    JSONVar json;
    json["config"] = "Online Classifier";                                
    json["operation"] = "saverun";  
    return JSON.stringify(json);
}

void setup() {
    display.begin();
    display.setRotation(1);
    display.clear(TFT_WHITE);
    display.setFont(&fonts::FreeMonoBold9pt7b);
    display.setTextColor(TFT_BLACK);
    delay(100);
    display.drawString("UnitV2 Json Example", 5, 5);
    display.drawLine(0, 25, 320, 25, TFT_BLACK);
    Serial.begin(115200);
    Serial2.begin(115200, SERIAL_8N1, 13, 14);//PORT.C
    //Setting JSON
    JSONVar obj;
    obj["function"] = "Online Classifier";                                
    obj["args"] = "";   
    String jsonString = JSON.stringify(obj);
    Serial2.println(jsonString);delay(10);

    // This command will erase the default classes preset in the factory firmware. It is not recommended to use.
    // jsonString = enter_training_mode_with_all_classes_clear();
    // Serial2.println(jsonString);delay(10);

    for (int i=0; i < 10; i++){
        jsonString = enter_training_mode(0, "earphone");
        Serial2.println(jsonString);delay(10);            
    }    
    delay(2000);

    jsonString = save_and_start();
    Serial2.println(jsonString);delay(10);
    Serial2.flush();
}

void loop() {
    if (Serial2.available() > 0) {
        String line = Serial2.readStringUntil('\r');
        while (line.length() && line[0] != '{') { // clear '\0'
            line.remove(0, 1);
        }
        Serial2.flush();
        JSONVar cla_obj = JSON.parse(line);

        if (!(JSON.typeof(cla_obj) == "undefined")) {
            display.fillRect(0, 35, 320, 205, TFT_WHITE);
            Serial.println(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
            Serial.println("UnitV2 Target Tracker example");            

            if (cla_obj.hasOwnProperty("msg")) {
                Serial.print("msg = ");
                Serial.println(String(cla_obj["msg"]).c_str());
                display.setCursor(0, 35);
                display.printf(" msg: %s\n", String(cla_obj["msg"]).c_str());
                if (cla_obj.hasOwnProperty("running")) {
                  Serial.print("running = ");
                  Serial.println(String(cla_obj["running"]).c_str());
                  display.printf(" running: %s\n", String(cla_obj["running"]).c_str());
                }
            } else {
              if (cla_obj.hasOwnProperty("running")) {
                  Serial.print("running = ");
                  Serial.println(String(cla_obj["running"]).c_str());
                  display.setCursor(0, 35);
                  display.printf(" running: %s\n", String(cla_obj["running"]).c_str());
              }
            }

            if (cla_obj.hasOwnProperty("class_num")) {
                cla_cnt = (int)cla_obj["class_num"];
                String best_match = String(cla_obj["best_match"]).c_str();
                double best_score = (double)cla_obj["best_score"];

                Serial.printf("class num = %d\n", cla_cnt);
                Serial.printf("best match = %s\n", best_match);
                Serial.printf("best score = %f\n", best_score);

                display.printf(" class num: %d\n", cla_cnt);
                display.printf(" best match = %s\n", best_match);
                display.printf(" best score = %f\n", best_score);
            }

            if (cla_obj.hasOwnProperty("class")) {
                JSONVar obj_arr = cla_obj["class"];
                for (int i = 0; i < (int)cla_cnt; i++) {
                    display.fillRect(0, 120, 320, 120, TFT_WHITE);
                    display.setCursor(0, 120);
                    if (JSON.typeof(obj_arr[i]) != "undefined") {
                        Serial.printf("------[class %d]------\n", i);
                        Serial.printf("\tname: %s\n", String(obj_arr[i]["name"]).c_str());
                        Serial.printf("\tscore: %f\n", (double)obj_arr[i]["score"]);
                        
                        display.printf("class %d:\n", i);
                        display.printf(" name: %s\n", String(obj_arr[i]["name"]).c_str());
                        display.printf(" score: %f\n", (double)obj_arr[i]["score"]);
                    }
                    delay(200);
                }
            }

            Serial.println("<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<");
        }
    }
}

When UnitV2 detects an object, it returns the category data:

5.9 Face Recognition

This example configures UnitV2 for face recognition. The factory firmware has no face data. To add: click add, train to train, save to save and exit. Access via unitv2.py or IP: 10.254.239.1.

You can also use code functions create_new_face, save_and_start. This example creates and trains Lena 10 times.

Note

The reset operation (click or call clear_all_faces) permanently deletes all trained faces and they will not be restored after power cycle.

For JSON format, see here.

cpp 
#include <M5Unified.h>
#include <M5GFX.h>
#include <Arduino_JSON.h>

M5GFX display;
int face_cnt = 0;

String create_new_face(int id, String name) {
    JSONVar json;
    json["config"] = "Face Recognition";                                
    json["operation"] = "train";  
    json["face_id"] = id; //0~N
    json["name"] = name;  //format: "face_name"
    return JSON.stringify(json);
}

String clear_all_faces(void) {
    JSONVar json;
    json["config"] = "Face Recognition";                                
    json["operation"] = "reset";  
    return JSON.stringify(json);
}

String save_and_start(void) {
    JSONVar json;
    json["config"] = "Face Recognition";                                
    json["operation"] = "saverun";  
    return JSON.stringify(json);
}

void setup() {
    display.begin();
    display.setRotation(1);
    display.clear(TFT_WHITE);
    display.setFont(&fonts::FreeMonoBold9pt7b);
    display.setTextColor(TFT_BLACK);
    delay(100);
    display.drawString("UnitV2 Json Example", 3, 3);
    display.drawLine(0, 23, 320, 23, TFT_BLACK);
    Serial.begin(115200);
    Serial2.begin(115200, SERIAL_8N1, 13, 14);//PORT.C
    //Setting JSON
    JSONVar obj;
    obj["function"] = "Face Recognition";                                
    obj["args"] = "";   
    String jsonString = JSON.stringify(obj);
    Serial2.println(jsonString);delay(10);

    // jsonString = clear_all_faces();
    // Serial2.println(jsonString);delay(10);

    for (int i=0; i < 10; i++){
        jsonString = create_new_face(0, "Lena");
        Serial2.println(jsonString);delay(10);            
    }    
    delay(2000);

    jsonString = save_and_start();
    Serial2.println(jsonString);delay(10);
    Serial2.flush();
}

void loop() {
    if (Serial2.available() > 0) {
        String line = Serial2.readStringUntil('\r');
        while (line.length() && line[0] != '{') { // clear '\0'
            line.remove(0, 1);
        }
        Serial2.flush();
        JSONVar face_obj = JSON.parse(line);

        if (!(JSON.typeof(face_obj) == "undefined")) {
            display.fillRect(0, 25, 320, 215, TFT_WHITE);
            Serial.println(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
            Serial.println("UnitV2 Face Recognition example");            

            if (face_obj.hasOwnProperty("msg")) {
                Serial.print("msg = ");
                Serial.println(String(face_obj["msg"]).c_str());
                display.setCursor(0, 25);
                display.printf(" msg: %s\n", String(face_obj["msg"]).c_str());
                if (face_obj.hasOwnProperty("running")) {
                  Serial.print("running : ");
                  Serial.println(String(face_obj["running"]).c_str());
                  display.printf(" running: %s\n", String(face_obj["running"]).c_str());
                }
            } else {
              if (face_obj.hasOwnProperty("running")) {
                  Serial.print("running : ");
                  Serial.println(String(face_obj["running"]).c_str());
                  display.setCursor(0, 25);
                  display.printf(" running: %s\n", String(face_obj["running"]).c_str());
              }
              if (face_obj.hasOwnProperty("status")) {
                  Serial.printf("status : %s\n", String(face_obj["status"]).c_str());
                  Serial.printf("\tx: %d\n", (int)face_obj["x"]);
                  Serial.printf("\ty: %d\n", (int)face_obj["y"]);
                  Serial.printf("\tw: %d\n", (int)face_obj["w"]);
                  Serial.printf("\th: %d\n", (int)face_obj["h"]);
                  Serial.printf("\tprob: %f\n", (double)face_obj["prob"]);
                  Serial.printf("\tname: %s\n", String(face_obj["name"]).c_str());

                  display.printf(" status: %s\n", String(face_obj["status"]).c_str());
                  display.printf(" x: %d\n", (int)face_obj["x"]);
                  display.printf(" y: %d\n", (int)face_obj["y"]);
                  display.printf(" w: %d\n", (int)face_obj["w"]);
                  display.printf(" h: %d\n", (int)face_obj["h"]);
                  display.printf(" prob: %f\n", (double)face_obj["prob"]);
                  display.printf(" name: %s\n", String(face_obj["name"]).c_str());
              }
            }

            if (face_obj.hasOwnProperty("num")) {
                face_cnt = (int)face_obj["num"];
                Serial.printf("num = %d\n", face_cnt);
                display.printf(" num: %d\n", face_cnt);
            }

            if (face_obj.hasOwnProperty("face")) {
                JSONVar obj_arr = face_obj["face"];
                display.fillRect(0, 60, 320, 180, TFT_WHITE);
                display.setCursor(0, 60);
                for (int i = 0; i < (int)face_cnt; i++) {
                    if (JSON.typeof(obj_arr[i]) != "undefined") {
                        int fx = (int)obj_arr[i]["x"];
                        int fy = (int)obj_arr[i]["y"];
                        int fw = (int)obj_arr[i]["w"];
                        int fh = (int)obj_arr[i]["h"];
                        double fprob = (double)obj_arr[i]["prob"];
                        double fmprob = (double)obj_arr[i]["match_prob"];
                        String fname = String(obj_arr[i]["name"]);
                        Serial.printf("------[face %d]------\n", i);
                        Serial.printf("\tx: %d\n", fx);
                        Serial.printf("\ty: %d\n", fy);
                        Serial.printf("\tw: %d\n", fw);
                        Serial.printf("\th: %d\n", fh);
                        Serial.printf("\tprob: %f\n", fprob);
                        Serial.printf("\tmatch prob: %f\n", fmprob);
                        Serial.printf("\tname: %s\n", fname.c_str());
                        display.printf("face %d:\n", i);
                        display.printf(" x:%d, y:%d,", fx, fy);
                        display.printf(" w:%d, h:%d\n", fw, fh);
                        display.printf(" prob: %f\n", fprob);
                        display.printf(" match prob: %f\n", fmprob);
                        display.printf(" name: %s\n", fname.c_str());

                        if (obj_arr[i].hasOwnProperty("mark")) {
                            JSONVar marks = obj_arr[i]["mark"];
                            for (int j = 0; j < 5; j++) {
                                if (JSON.typeof(marks[j]) != "undefined") {
                                    int mx = (int)marks[j]["x"];
                                    int my = (int)marks[j]["y"];
                                    display.printf("\t mark%d: x=%d, y=%d\n", j, mx, my);
                                    Serial.printf("\t\tmark%d: x=%d, y=%d\n", j, mx, my);
                                }
                            }
                        }
                    }
                    delay(200);
                }
            }
            Serial.println("<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<");
        }
    }
} 

Scan the face on the left; recognition result on the right:

5.10 Face Detector

Configures UnitV2 to the face detection function. Access via unitv2.py or IP: 10.254.239.1.

For JSON format, see here.

cpp 
#include <M5Unified.h>
#include <M5GFX.h>
#include <Arduino_JSON.h>

M5GFX display;
int face_cnt = 0;

void setup() {
    display.begin();
    display.setRotation(1);
    display.clear(TFT_WHITE);
    display.setFont(&fonts::FreeMonoBold9pt7b);
    display.setTextColor(TFT_BLACK);
    delay(100);
    display.drawString("UnitV2 Json Example", 5, 5);
    display.drawLine(0, 25, 320, 25, TFT_BLACK);
    Serial.begin(115200);
    Serial2.begin(115200, SERIAL_8N1, 13, 14);//PORT.C
    //Setting JSON
    JSONVar obj;
    obj["function"] = "Face Detector";                                
    obj["args"] = "";   
    String jsonString = JSON.stringify(obj);
    Serial2.println(jsonString);
    Serial2.flush();
}

void loop() {
    if (Serial2.available() > 0) {
        String line = Serial2.readStringUntil('\r');
        while (line.length() && line[0] != '{') { // clear '\0'
            line.remove(0, 1);
        }
        Serial2.flush();
        JSONVar face_obj = JSON.parse(line);

        if (!(JSON.typeof(face_obj) == "undefined")) {
            display.fillRect(0, 35, 320, 205, TFT_WHITE);
            Serial.println(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
            Serial.println("UnitV2 Face Detector example");            

            if (face_obj.hasOwnProperty("msg")) {
                Serial.print("msg = ");
                Serial.println(String(face_obj["msg"]).c_str());
                display.setCursor(0, 35);
                display.printf(" msg: %s\n", String(face_obj["msg"]).c_str());
                if (face_obj.hasOwnProperty("running")) {
                  Serial.print("running : ");
                  Serial.println(String(face_obj["running"]).c_str());
                  display.printf(" running: %s\n", String(face_obj["running"]).c_str());
                }
            } else {
              if (face_obj.hasOwnProperty("running")) {
                  Serial.print("running : ");
                  Serial.println(String(face_obj["running"]).c_str());
                  display.setCursor(0, 35);
                  display.printf(" running: %s\n", String(face_obj["running"]).c_str());
              }
            }

            if (face_obj.hasOwnProperty("num")) {
                face_cnt = (int)face_obj["num"];
                Serial.printf("num = %d\n", face_cnt);
                display.printf(" num: %d\n", face_cnt);
            }

            if (face_obj.hasOwnProperty("face")) {
                JSONVar obj_arr = face_obj["face"];
                display.fillRect(0, 70, 320, 170, TFT_WHITE);
                display.setCursor(0, 70);
                for (int i = 0; i < (int)face_cnt; i++) {
                    if (JSON.typeof(obj_arr[i]) != "undefined") {
                        int fx = (int)obj_arr[i]["x"];
                        int fy = (int)obj_arr[i]["y"];
                        int fw = (int)obj_arr[i]["w"];
                        int fh = (int)obj_arr[i]["h"];
                        double fprob = (double)obj_arr[i]["prob"];
                        Serial.printf("------[face %d]------\n", i);
                        Serial.printf("\tx: %d\n", fx);
                        Serial.printf("\ty: %d\n", fy);
                        Serial.printf("\tw: %d\n", fw);
                        Serial.printf("\th: %d\n", fh);
                        Serial.printf("\tprob: %f\n", fprob);
                        display.printf("face %d:\n", i);
                        display.printf(" x:%d, y:%d\n", fx, fy);
                        display.printf(" w:%d, h:%d\n", fw, fh);
                        display.printf(" prob: %f\n", fprob);

                        if (obj_arr[i].hasOwnProperty("mark")) {
                            JSONVar marks = obj_arr[i]["mark"];
                            for (int j = 0; j < 5; j++) {
                                if (JSON.typeof(marks[j]) != "undefined") {
                                    int mx = (int)marks[j]["x"];
                                    int my = (int)marks[j]["y"];
                                    display.printf("\t mark%d: x=%d, y=%d\n", j, mx, my);
                                    Serial.printf("\t\tmark%d: x=%d, y=%d\n", j, mx, my);
                                }
                            }
                        }
                    }
                    delay(200);
                }
            }
            Serial.println("<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<");
        }
    }
}

Scan the left face; detection result on the right:

5.11 Shape Detector

Configures UnitV2 to the shape detection function. Before use, set the background once. Access via browser: unitv2.py or IP: 10.254.239.1.

For JSON format, see here.

cpp 
#include <M5Unified.h>
#include <M5GFX.h>
#include <Arduino_JSON.h>

M5GFX display;
int shape_cnt = 0;

void setup() {
    display.begin();
    display.setRotation(1);
    display.clear(TFT_WHITE);
    display.setFont(&fonts::FreeMonoBold9pt7b);
    display.setTextColor(TFT_BLACK);
    delay(100);
    display.drawString("UnitV2 Json Example", 5, 5);
    display.drawLine(0, 25, 320, 25, TFT_BLACK);
    Serial.begin(115200);
    Serial2.begin(115200, SERIAL_8N1, 13, 14);//PORT.C
    //Setting JSON
    JSONVar obj;
    obj["function"] = "Shape Detector";                                
    obj["args"] = "";   
    String jsonString = JSON.stringify(obj);
    Serial2.println(jsonString);delay(10);
    JSONVar BCKG;
    BCKG["config"] = "Shape Detector";                                
    BCKG["operation"] = "update";  
    jsonString = JSON.stringify(BCKG);
    Serial2.println(jsonString);
    Serial2.flush();
    delay(200);
}

void loop() {
    if (Serial2.available() > 0) {
        String line = Serial2.readStringUntil('\r');
        while (line.length() && line[0] != '{') { // clear '\0'
            line.remove(0, 1);
        }
        Serial2.flush();
        JSONVar shape_obj = JSON.parse(line);

        if (!(JSON.typeof(shape_obj) == "undefined")) {
            display.fillRect(0, 35, 320, 205, TFT_WHITE);
            Serial.println(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
            Serial.println("UnitV2 Shape Detector example");            

            if (shape_obj.hasOwnProperty("msg")) {
                Serial.print("msg = ");
                Serial.println(String(shape_obj["msg"]).c_str());
                display.setCursor(0, 35);
                display.printf(" msg: %s\n", String(shape_obj["msg"]).c_str());
                if (shape_obj.hasOwnProperty("running")) {
                  Serial.print("running : ");
                  Serial.println(String(shape_obj["running"]).c_str());
                  display.printf(" running: %s\n", String(shape_obj["running"]).c_str());
                }
            } else {
              if (shape_obj.hasOwnProperty("running")) {
                  Serial.print("running : ");
                  Serial.println(String(shape_obj["running"]).c_str());
                  display.setCursor(0, 35);
                  display.printf(" running: %s\n", String(shape_obj["running"]).c_str());
              }
            }

            if (shape_obj.hasOwnProperty("num")) {
                shape_cnt = (int)shape_obj["num"];
                Serial.printf("num = %d\n", shape_cnt);
                display.printf(" num: %d\n", shape_cnt);
            }

            if (shape_obj.hasOwnProperty("shape")) {
                JSONVar obj_arr = shape_obj["shape"];
                for (int i = 0; i < (int)shape_cnt; i++) {
                    display.fillRect(0, 80, 320, 160, TFT_WHITE);
                    display.setCursor(0, 80);
                    if (JSON.typeof(obj_arr[i]) != "undefined") {
                        String name = String(obj_arr[i]["name"]);
                        int x = (int)obj_arr[i]["x"];
                        int y = (int)obj_arr[i]["y"];
                        int w = (int)obj_arr[i]["w"];
                        int h = (int)obj_arr[i]["h"];
                        double angle = (double)obj_arr[i]["angle"];
                        int area = (int)obj_arr[i]["area"];

                        Serial.printf("------[shape %d]------\n", i);
                        Serial.printf("\tname: %s\n", name.c_str());
                        Serial.printf("\tx: %d\n", x);
                        Serial.printf("\ty: %d\n", y);
                        Serial.printf("\tw: %d\n", w);
                        Serial.printf("\th: %d\n", h);
                        Serial.printf("\tangle: %f\n", angle);
                        Serial.printf("\tarea: %d\n", area);

                        display.printf("shape %d:\n", i);
                        display.printf(" name: %s\n", name.c_str());
                        display.printf(" x:%d, y:%d\n", x, y);
                        display.printf(" w:%d, h:%d\n", w, h);
                        display.printf(" angle: %f\n", angle);
                        display.printf(" area: %d\n", area);
                    }
                    delay(200);
                }
            }
            Serial.println("<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<");
        }
    }
}

Scan the left image; detection result on the right:

5.12 Shape Matching

Configures UnitV2 to the custom shape matching function. Before use, set the background once. Factory firmware contains no shape data. To add shape data: click add, then upload a white background black shape PNG file. The shape name is taken from the file name. In this example: arrow.png → shape name arrow. After uploading successfully, you can use the shape matching function. Access via unitv2.py or IP: 10.254.239.1.

For JSON format, see here.

Note

The example code below does not print all JSON fields. To view all, refer to JSON format and add them as needed.

cpp 
#include <M5Unified.h>
#include <M5GFX.h>
#include <Arduino_JSON.h>

M5GFX display;
int shape_cnt = 0;

void setup() {
    display.begin();
    display.setRotation(1);
    display.clear(TFT_WHITE);
    display.setFont(&fonts::FreeMonoBold9pt7b);
    display.setTextColor(TFT_BLACK);
    delay(100);
    display.drawString("UnitV2 Json Example", 5, 5);
    display.drawLine(0, 25, 320, 25, TFT_BLACK);
    Serial.begin(115200);
    Serial2.begin(115200, SERIAL_8N1, 13, 14);//PORT.C
    //Setting JSON
    JSONVar obj;
    obj["function"] = "Shape Matching";                                
    obj["args"] = "";   
    String jsonString = JSON.stringify(obj);
    Serial2.println(jsonString);delay(10);
    JSONVar BCKG;
    BCKG["config"] = "Shape Matching";                                
    BCKG["operation"] = "update";  
    jsonString = JSON.stringify(BCKG);
    Serial2.println(jsonString);
    Serial2.flush();
    delay(200);
}

void loop() {
    if (Serial2.available() > 0) {
        String line = Serial2.readStringUntil('\r');
        while (line.length() && line[0] != '{') { // clear '\0'
            line.remove(0, 1);
        }
        Serial2.flush();
        JSONVar shape_obj = JSON.parse(line);

        if (!(JSON.typeof(shape_obj) == "undefined")) {
            display.fillRect(0, 35, 320, 205, TFT_WHITE);
            Serial.println(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
            Serial.println("UnitV2 Shape Matching example");            

            if (shape_obj.hasOwnProperty("msg")) {
                Serial.print("msg = ");
                Serial.println(String(shape_obj["msg"]).c_str());
                display.setCursor(0, 35);
                display.printf(" msg: %s\n", String(shape_obj["msg"]).c_str());
                if (shape_obj.hasOwnProperty("running")) {
                  Serial.print("running : ");
                  Serial.println(String(shape_obj["running"]).c_str());
                  display.printf(" running: %s\n", String(shape_obj["running"]).c_str());
                }
            } else {
              if (shape_obj.hasOwnProperty("running")) {
                  Serial.print("running : ");
                  Serial.println(String(shape_obj["running"]).c_str());
                  display.setCursor(0, 35);
                  display.printf(" running: %s\n", String(shape_obj["running"]).c_str());
              }
            }

            if (shape_obj.hasOwnProperty("num")) {
                shape_cnt = (int)shape_obj["num"];
                Serial.printf("num = %d\n", shape_cnt);
                display.printf(" num: %d\n", shape_cnt);
            }

            if (shape_obj.hasOwnProperty("shape")) {
                JSONVar obj_arr = shape_obj["shape"];
                for (int i = 0; i < (int)shape_cnt; i++) {
                    display.fillRect(0, 80, 320, 160, TFT_WHITE);
                    display.setCursor(0, 80);
                    if (JSON.typeof(obj_arr[i]) != "undefined") {
                        String name = String(obj_arr[i]["name"]);
                        int x = (int)obj_arr[i]["x"];
                        int y = (int)obj_arr[i]["y"];
                        int w = (int)obj_arr[i]["w"];
                        int h = (int)obj_arr[i]["h"];
                        int area = (int)obj_arr[i]["area"];

                        Serial.printf("------[shape %d]------\n", i);
                        Serial.printf("\tname: %s\n", name.c_str());
                        Serial.printf("\tx: %d\n", x);
                        Serial.printf("\ty: %d\n", y);
                        Serial.printf("\tw: %d\n", w);
                        Serial.printf("\th: %d\n", h);
                        Serial.printf("\tarea: %d\n", area);

                        display.printf("shape %d:\n", i);
                        display.printf(" name: %s\n", name.c_str());
                        display.printf(" x:%d, y:%d\n", x, y);
                        display.printf(" w:%d, h:%d\n", w, h);
                        display.printf(" area: %d\n", area);
                    }
                    delay(200);
                }
            }
            Serial.println("<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<");
        }
    }
} 

Scan the left image; detection result on the right:

Next 1. Arduino Setup - 1. Arduino IDE Install

1. Arduino Setup

2. Devices & Examples

Tab5

Basic/Gray

M5GO

Fire

Core2/Core2 For AWS

CoreS3

Tough

Atom-Lite / Atom-Matrix

AtomS3 / AtomS3-Lite

Atom EchoS3R

Dial

Air Quality

DinMeter

Capsule

Cardputer / -Adv