2.2.0
How to run locally a c-model


ST Edge AI Core

How to run locally a c-model


for STM32 target, based on ST Edge AI Core Technology 2.2.0



r2.0

Overview

This article explains how to run locally the generated c-model in a C-based environment, to:

Tip

Python-based environment is covered by the “How to use the AiRunner package” article.

%STEDGEAI_CORE_DIR% indicates the location where the ST Edge AI pack is installed.

C-based environment

This environment is based on the same X86 AI runtime libraries which are used to perform the built-in validation flow. So, the main prerequisite is to have a compatible C/C++ tool chain to be able to link the final application with them. Note that in Windows® development environment, a MingGW-W64 tool chain is available in X-CUBE-AI pack, including the GNU Make utility.

Following source tree is used to illustrate this how-to. cubeai_lib_XX directory contains the ST.AI host/x86 C library files to build the generated c-model files in model directory.

<root_project>
        |- makefile
        |_ main.c                  /* entry point,  */
        |_ *.c/*.h                 /* extra/user test framework files */  
        |
        |_ model
        |   |_ network.c/.h        /* generated c-model files */
        |   |_ network_data.c/.h
        |   |_ network_config.h
        |
        \_ cubeai_lib_XX
            |_ include             /* X-CUBE-AI header files */ 
            |_ lib                 /* Libraries */

Retrieve the X86/host C library files

By OS, the expected X86 library files are located in:

%STEDGEAI_CORE_DIR%/Utilities/%OS%/targets/common/EmbedNets/tools/inspector/workspace/

All content of the include folder can be directly copied in <root_project>/cube_ai_lib_XX/inlude. Idem for the content of lib in <root_project>/cube_ai_lib_XX/lib. Note the the share libraries (*.dll or .so) are not necessary.

Build the test application

AI_LIB_DIR = $(root_project)/cubeai_lib_XX
...
CFLAGS += $(root_project)/model 

CFLAGS += -std=c99
CFLAGS += -I$(AI_LIB_DIR)/include

LIBS = -L$(AI_LIB_DIR)/lib/static -lruntime -lst_cmsis_nn -lcmsis_nn -lx86_cmsis -lm

Update the test application source tree with a new c-model

The generate command can be used with the --output/-o option to indicate the location of the generated files.

$ stedgeai generate -m <my_model> --target stm32 <my_options> -o %root_project%/model/

Test code

There is no difference between the [embedded inference API][X_CUBE_AI_API] used for a STM32 application and for the X86 test application. Setup/init sequence and the calls of the ai_<network>_XXX() functions are the same.

#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include <string.h>

#include "network.h"
#include "network_data.h"

/**
 * @brief Statically allocated buffers.
 * Buffers can be dynamically allocated using malloc and size information
 * given by the report in ai_network_get_report().
 */
ai_u8 activations[AI_NETWORK_DATA_ACTIVATIONS_SIZE];
ai_u8 in_data[AI_NETWORK_IN_1_SIZE_BYTES];
ai_u8 out_data[AI_NETWORK_OUT_1_SIZE_BYTES];

/* AI buffer IO handlers */
ai_buffer *ai_input;
ai_buffer *ai_output;

int main(int argc, char const *argv[])
{
    ai_handle network = AI_HANDLE_NULL;
    ai_error err;
    ai_network_report report;

    /** @brief Initialize network */
    const ai_handle acts[] = { activations };
    err = ai_network_create_and_init(&network, acts, NULL);
    if (err.type != AI_ERROR_NONE) {
        printf("ai init_and_create error\n");
        return -1;
    }

    /** @brief {optional} for debug/log purpose */
    if (ai_network_get_report(network, &report) != true) {
        printf("ai get report error\n");
        return -1;
    }

    printf("Model name      : %s\n", report.model_name);
    printf("Model signature : %s\n", report.model_signature);

    ai_input = &report.inputs[0];
    ai_output = &report.outputs[0];
    printf("input[0] : (%d, %d, %d)\n", AI_BUFFER_SHAPE_ELEM(ai_input, AI_SHAPE_HEIGHT),
                                        AI_BUFFER_SHAPE_ELEM(ai_input, AI_SHAPE_WIDTH),
                                        AI_BUFFER_SHAPE_ELEM(ai_input, AI_SHAPE_CHANNEL));
    printf("output[0] : (%d, %d, %d)\n", AI_BUFFER_SHAPE_ELEM(ai_output, AI_SHAPE_HEIGHT),
                                         AI_BUFFER_SHAPE_ELEM(ai_output, AI_SHAPE_WIDTH),
                                         AI_BUFFER_SHAPE_ELEM(ai_output, AI_SHAPE_CHANNEL));

    /** @brief Fill input buffer with random values */
    srand(1);
    for (int i = 0; i < AI_NETWORK_IN_1_SIZE; i++) {
        in_data[i] = rand() % 0xFFFF;
    }

    /** @brief Normalize, convert and/or quantize inputs if necessary... */

    /** @brief Perform inference */
    ai_i32 n_batch;

    /** @brief Create the AI buffer IO handlers
     *  @note  ai_inuput/ai_output are already initilaized after the
     *         ai_network_get_report() call. This is just here to illustrate
     *         the case where get_report() is not called.
     */
    ai_input = ai_network_inputs_get(network, NULL);
    ai_output = ai_network_outputs_get(network, NULL);

    /** @brief Set input/output buffer addresses */
    ai_input[0].data = AI_HANDLE_PTR(in_data);
    ai_output[0].data = AI_HANDLE_PTR(out_data);

    /** @brief Perform the inference */
    n_batch = ai_network_run(network, &ai_input[0], &ai_output[0]);
    if (n_batch != 1) {
        err = ai_network_get_error(network);
        printf("ai run error %d, %d\n", err.type, err.code);
      return -1;
    }

    /** @brief Post-process the output results/predictions */
    printf("Inference output..\n");
    for (int i = 0; i < AI_NETWORK_OUT_1_SIZE; i++) {
        printf("%d,", out_data[i]);
    }
    printf("\n");
    return 0;
}

C++ environment

The generated 'network.c' and 'network_data.c' files should be compiled with the C compiler. network_runtime static libraries are fully compiled with a C-compiler. For the C++ application code, the AI header files should be included as follow:

#include <iostream>
using namespace std;

extern "C" {
#include "network.h"
#include "network_data.h"
}