ST Edge AI Core - Command-line Interface
ST Edge AI Core Technology 4.1.0
r8.7
Overview
The stedgeai application is a console utility. It provides a complete and unified command-line interface (CLI) for compiling a pre-trained deep learning (DL) or machine learning (ML) model into an optimized library. This library can run on an ST device or target, enabling edge AI on microcontrollers (MCUs) with or without Neural-ART, microprocessors (MPUs), and smart sensors. It consists of three main commands: analyze, validate, and generate. Each command can be used independently with the same set of common options (model files, compression factor, output directory, etc.) and specific options. The supported-ops command lists the supported operators and associated constraints for a given deep learning framework.
Supported ST device/target
| target | description |
|---|---|
stm32[xx] |
The STM32
family of 32-bit microcontrollers based on the Arm Cortex®-M
processor. 'stm32xx' is requested to define a specific
stm32 series (see the “Supported
STM32 series” section) else 'stm32h7' is
used. |
sr5[xx] |
Series of MCUs based on the Arm Cortex®-M7 processor tailored for electrification solutions and guarantee efficient actuation of power conversion and e-drivetrain applications. Stellar E MCUs are the ideal SiC/GaN companion chips for digital control enhancement. |
sr6p[xx] |
Series
of MCUs (P families), based on Arm Cortex®-R52+ and Arm
Cortex®-M4 processors, designed to meet the demands of integrating
next-generation drivetrains and electrification solutions, including
X-in-1 vehicle motion control and domain-oriented systems. It
delivers a new level of real-time performance, safety and
determinism, along with a rich set of I/Os and peripherals.
'sr6p[xx]' is requested to define a specific Stellar P
series (see the “Supported
STELLAR P/G series” section) else
'stellar-p-r52' is used. |
sr6g[xx] |
Series
of MCUs (G families), based on Arm Cortex®-R52+ and Arm
Cortex®-M4 processors, to address the key challenges of
next-generation body and gateway integration, as well as zonal
architectures. It ensures performance, safety, and power efficiency,
combined with wide connectivity and high security.
'sr6g[xx]' is requested to define a specific Stellar G
series (see the “Supported
STELLAR P/G series” section) else
'stellar-g-r52' is used. |
sr6p3e |
Series
of MCUs (P families), based on Arm Cortex®-R52+ and Arm
Cortex®-M4 processors, with integrated Neural-ART Processing Unit
(NPU) enabling advanced Artificial Intelligence (AI) use cases such
as predictive maintenance and precise motor-control algorithms for
smoother operation. Interfaces have been upgraded to deliver more
efficient outputs and faster response times, a key requirement for
electrification applications. 'sr6p3e' is requested to
define a specific Stellar P3E series (see the “Supported
STELLAR P/G series”. |
ispu |
A new generation of MEMS sensors featuring an embedded intelligent sensor processing unit (ISPU) |
mlc |
MEMS sensors embedding the machine learning core (MLC) |
Synopsis
usage: stedgeai --model FILE --target TARGET [--type tflite|onnx|keras] [--name STR]
[--compression none|lossless|low|medium|high] [--use-f16 | --no-use-f16]
[--inputs-allocation | --no-inputs-allocation] [--outputs-allocation | --no-outputs-allocation]
[--allocate-activations | --no-allocate-activations]
[--allocate-states | --no-allocate-states] [--input-memory-alignment INT] [--output-memory-alignment INT]
[--workspace DIR] [--output DIR] [--split-weights | --no-split-weights] [--optimization OBJ]
[--memory-pool FILE] [--use-onnx-simplifier] [--fix-parametric-shapes FIX_PARAMETRIC_SHAPES]
[--input-data-type float32|int8|uint8] [--output-data-type float32|int8|uint8]
[--inputs-ch-position chfirst|chlast] [--outputs-ch-position chfirst|chlast]
[--prefetch-compressed-weights | --no-prefetch-compressed-weights] [--custom FILE]
[--c-api st-ai|legacy] [--cut-input-tensors CUT_INPUT_TENSORS] [--cut-output-tensors CUT_OUTPUT_TENSORS]
[--cut-input-layers CUT_INPUT_LAYERS] [--cut-output-layers CUT_OUTPUT_LAYERS]
[--const-in-const | --no-const-in-const]
[--hsp [BRAM_SIZE]] [--hsp-config [HSP_CONF_ID]] [--st-neural-art [ST_NEURAL_ART]]
[--compute-runtime-code-size | --no-compute-runtime-code-size | --crcs | --no-crcs]
[--pack [STR]] [--binary | --no-binary] [--dll | --no-dll] [--address ADDR]
[--copy-weights-at ADDR] [--relocatable [STR]] [--enable-epoch-controller] [--batch-size INT]
[--mode host|target|host-io-only|target-io-only] [--desc DESC] [--val-json FILE]
[--valinput FILE [FILE ...]] [--valoutput FILE [FILE ...]] [--range MIN MAX [MIN MAX ...]]
[--range-int MIN_INT MAX_INT [MIN_INT MAX_INT ...]] [--seed SEED] [--io-only] [--save-csv]
[--classifier] [--no-check] [--no-exec-model] [--with-report] [--no-report]
[--no-workspace] [-h] [--version] [--tools-version]
[--verbosity [0|1|2|3]] [--quiet]
analyze|generate|validate|supported-opsA short description of the options can be displayed with the following command:
$ stedgeai --help
...To know the versions of the main Python modules which are used:
$ stedgeai --tools-version
stedgeai - ST Edge AI Core v4.1.0
- Python version : 3.13.12
- Numpy version : 2.4.3
- TF version : 2.21.0
- TF Keras version : 3.15.1
- ONNX version : 1.22.0
- ONNX RT version : 1.25.1Options for a given target
The 'target' option
can be specified to know the options which are available for a given
target.
$ stedgeai --target mlc --help
usage: stedgeai [--target STR] [--output DIR] --device DEVICE [--script FILE]
[--json FILE] [--type {arff,ucf}] [--port COM] [--ucf FILE]
[--logs FILE | DIR] [--ignore-zero] [--tree FILE] [--arff FILE]
[--meta FILE] [--no-report] [--help] [--version] [--tools-version]
[--verbosity [{0,1}]]
generate|validate|analyze
ST Edge AI Core v1.0.0 (MLC 1.0.0)
...Command workflow
For each command, the same preliminary steps are applied. A report (txt file) is systematically created and fully or partially displayed. Additional JSON files (dictionary based) are generated in the workspace directory to be parsed by the external tools/script to retrieve the results. Note that they can also be used by a nonregression environment. The format of these files is out of the scope of this document.
<workspace-directory-path>\<name>_c_info.json
<output-directory-path>\<name>_<cmd_name>_report.txt'analyze'workflow- import the model
- map, render, and optimize internally the model
- log and display a report
- import the model
'validate'workflow- import the model
- map, render, and optimize internally the model
- execute the generated C-model (on the desktop or on the
board)
- execute the original model using the original deep learning runtime framework for x86
- evaluate the metrics
- log and display a report
- import the model
'generate'workflow- import the model
- map, render, and optimize internally the model
- export the specialized C-files
- log and display a report
- import the model
Enable AutoML pipeline for resource-constrained environment
The CLI can be integrated into an automatic or manual pipeline. It allows to design a deployable, and effective neural network architecture for a resource-constrained environment (that is, with low memory/computational resources and/or critical power consumption budgets). The main loop can be extended with a post-analyzing/validating step of the pretrained models. The candidates are checked according to the end-user target constraints thanks to the respective analyze and validate commands.
- Checking of the budgeted memory (ROM/RAM) can be done in the inner loop (topology selection/definition) before the time-consuming training process (or retraining process) to preconstraint the choices of the neural network architecture according to the memory budgets.
- Note that the “analyze” and “host validate” step can be merged, “analyze” information are also available in the “validate” reports.
Error handling
During the execution of a given command, after the parsing of the
arguments if an error is raised, the stedgeai
application returns -1 (else 0 is
returned).
A category and a short description prefix an error message.
| category | description |
|---|---|
| CLI ERROR | specific CLI error |
| LOAD ERROR | error during the load/import of the model or the connection with the board - OSError, IOError |
| NOT IMPLEMENTED | expected feature is not implemented - NotImplementedError |
| INTERRUPT | indicates that the execution of the
command has been interrupted (CTRL-C or kill system
signal) - KeyboardInterrupt, SystemExit |
| TOOLS ERROR, INTERNAL ERROR | internal error - ImportError, RuntimeError, ValueError |
Note
There is a specific attention to have explicit and relevant short description of the error messages. Unfortunately, this is not always the case, do not hesitate to contact the local support or to use the product ST Community channel/forum, “Edge AI”.
Example of specific error
$ stedgeai validate model.tflite --target ispu -t keras
...
E102(CliArgumentError): Wrong model files for 'keras'Analyze command
Description
The 'analyze' command is the primary command to
import, parse, and check an uploaded pretrained model. A detailed report provides
the main metrics to know if the generated code can be deployed on
the targeted device. It also includes the rendering information by
layer or/and operator (see “C-graph
description” section). It provides the size of the requested RT memory size to store the
kernel and specific network binary objects. After completion, the
user can be fully confident in the imported model in term
of supported layers/operators.
Examples
Analyze a model
$ stedgeai analyze -m <model_file_path> --target <target>Analyze Keras model saved in two separated files: model.json + model.hdf5
$ stedgeai analyze -m <model_file_path>.json -m <model_file_path>.hdf5 --target <target>Analyze a 32b float model with compression request
$ stedgeai analyze -m <model_file_path> -c low --target <target>Analyze a model with the input tensors placed in the activations buffer
$ stedgeai analyze -m <model_file_path> --allocate-inputs --target <target>
Common options
This section describes the common options for the analyze, validate, and generate commands. The specific options are described in the respective command section.
-m/--model FILE
Specify the model file paths (see the “Deep Learning (DL) framework
detection” section). Use the same -m option to
specify the weights file, if needed. Required.
Details
Deep Learning (DL) framework detection
Extensions of the model files are used to identify the DL
framework which should be used to import the model. If the
autodetection is ambiguous, the '--type/-t' option Use
the '--type/-t' option to select the correct
framework.
| DL framework | type (–type/-t) | file extension |
|---|---|---|
| Keras | keras |
.h5 or .hdf5
and .json |
| TensorFlow lite | tflite |
.tflite |
| ONNX | onnx |
.onnx |
--target STR
Specify the ST target device or series. Required.
-t/--type STR
Specify the original deep learning (DL) framework when the model file extensions do not identify it (see the “DL framework detection” section).
-w/--workspace DIR
Specify the working directory for intermediate files (default:
"./st_ai_ws/").
-o/--output DIR
Specify the output directory for generated C and report files
(default: "./st_ai_output/").
-n/--name STR
Specify the C name (C-string type) for the imported
model. The name prefixes the specialized neural network (NN) C-files
and API functions. It is also used for temporary files, allowing the
use of the same workspace and output directories for different
models (default: "network").
-c/--compression STR
- supported target:
stm32xx,sr5[xx],sr6p[xx],sr6g[xx],sr6p3e - unsupported target:
stm32n6/stm32h7w/ NPU,sr6p3ew/ NPU,mlc,ispu
Specify the compression level to apply to operators. Choose
none|lossless|low|medium|high (default:
lossless) to apply one level to all operators, or
provide a JSON file to set levels per operator.
Details
During the optimization passes of the imported model, different compression level can be applied. Underlying applied compression algorithms depend on the selected level and the configuration of the operator/layer itself.
| level | description |
|---|---|
none |
no compression |
lossless |
applied algorithms ensuring the accuracy (structural compression) |
low |
applied algorithms trying to reduce the size of the parameters with a minimum of accuracy loss |
medium |
more aggressive algorithms, the final accuracy loss can be more important |
high |
extreme aggressive algorithms (not used) |
Supported compressions by operator
Floating-point dense or fully connected layers:
'low/medium/high'enables the compression of weights or/and bias. With'low', a targeted compression factor of'4'is applied while with'medium\high' the targeted compression factor is‘8’`.ONNX-ML TreeEnsembleClassifier operator: With
'none'level, no compression or optimization is applied.'lossless'enables a first level of compression w/o loss of accuracy.'low','medium','high'enable compression weights.
Warning
Only float32 (or float) are supported by the code generator, this implies that during the import of the operator, the float64 (or double) values are converted in float32.
Specify a compression level by operator
By default, the compression process tries to apply globally the same compression level for all eligible operators. If the global accuracy is too much impacted or to force the compression, the user has the possibility to refine the expected compression level layer-by-layer.
A JSON file must be defined to indicate what is the compression level which must be applied for a given layer. The original name specifies the name of the layer.
{
"layers": {
"dense_2": {"factor": "high"},
}
}The option -c/--compression can be used to pass the
configuration file.
$ stedgeai analyze -m <model_file> --target stm32 -c <conf_file>.json
Note
Be aware that a specific layer could not be compressed if the gain of weight size is not enough.
--use-f16
- supported target:
stm32xx,stm32n6andstm32h7w/o NPU,sr5[xx],sr6p[xx],sr6g[xx],ispu,sr6p3ew/o NPU - unsupported target:
mlc,stm32n6/stm32h7w/ NPU,sr6p3ew/ NPU
Enable deployment of float16 kernel implementations for long short-term memory (LSTM) and gated recurrent unit (GRU) operations. Otherwise, the float32 kernel implementations are used. Applicable to all targets and floating-point operators, including ONNX LSTM, ONNX GRU, Keras LSTM, Keras GRU and TFlite.
For the float32 operator, ST Edge AI Core performs an offline conversion or cast of the provided weight tensors during model conversion. For the int8 operator, the conversion is deferred to runtime and performed during inference, which may increase execution time and may introduce some degradation in accuracy or precision, depending on the model, quantization parameters, and target hardware, in particular whether f16 vectorization is supported. The actual impact on accuracy should be evaluated using representative data, as it depends on the sensitivity of the model.
Example 1 - ONNX floating-point stacked LSTM model on STM32V8 target
The following table illustrates the use of --use-f16
for a typical floating-point stacked LSTM model (2 layers) with
sequence length 128. Head is a simple ONNX Gemm operator. The model
is deployed on an STM32V8xx target based on an Arm Cortex-M85 core
running at 800 MHz with MVE capability. In both
configurations, internal flash and RAM are used to store the
application, weights, and activations. The cosine similarity (cos)
and SNR are evaluated using random data.
| used option | macc | weights | activations | nodes | inf/s | cos | snr |
|---|---|---|---|---|---|---|---|
| default | 264,970 | 1,060,904 | 7,680 | 6 | 486 (6.21 cycles//macc) | 1.0 | 130 |
--use-f16 |
267,530 | 533,032 | 4,352 | 16 | 1,075 (2.78 cycles//macc) | 0.999 | 40 |
- With
--use-f16, the weights are now stored in float16 (~gain of 2), the activations and internal LSTM tensors use the float16 instead float32. No change for the ONNX Gemm operator which is always deployed in float32. - Runtime code size increases due to the additional conversion layers. The impact on the overall memory layout depends on the model.
Example 2 - ONNX GRU model on STM32V8 target
The following table illustrates the use of --use-f16
for the typical ONNX GRU models (2 layers, f32 and ONNX QDQ w8a8
version) with a simple ONNX Gemm. The model is deployed on an
STM32V8xx target based on an Arm Cortex-M85 core running at 800 MHz
with MVE capability. In
both configurations, internal flash and RAM are used to store the
application, weights, and activations. The cosine similarity (cos)
and SNR are evaluated using random data.
| model | used option | macc | weights | activations | nodes | inf/s | cos | snr |
|---|---|---|---|---|---|---|---|---|
| f32 | default | 8,707,450 | 367,216 | 79,600 | 5 | 16.64 (5.52 cycles//macc) | 1.0 | 128 |
| f32 | --use-f16 |
8,726,842 | 184,016 | 39,800 | 7 | 35.15 (2.61 cycles//macc) | 0.999 | 49 |
| w8a8 | default | 8,948,442 | 91,812 | 407,984 | 16 | 26.87 (3.33 cycles//macc) | 1.0 | inf |
| w8a8 | --use-f16 |
9,189,434 | 91,816 | 242,592 | 27 | 47.15 (1.85 cycles//macc) | 0.999 | 42 |
- As the ONNX GRU operators are fully quantized in QDQ format, the
weights are stored as int8 in both cases and
dequantized at runtime through conversion layers to use the
floating-point implementation: float32 without
--use-f16, and float16 with--use-f16. This is closer to the behavior of ONNX Runtime, helping to avoid accuracy loss. - There is no change for the quantized ONNX Gemm operator, which remains deployed in int8.
- For
w8a8, the additional activation size (vs f32 model) is due to the conversion operations between integer and floating-point formats, which are inserted before and after the kernel execution. - Runtime code size increases due to the additional conversion layers. The impact on the overall memory layout depends on the model.
Example 3 - ONNX GRU model on STM32H7 target without float16 or float32 vectorization support
The following table illustrates the use of --use-f16
for the typical ONNX GRU models (2 layers, f32 and ONNX QDQ w8a8
version) with a simple ONNX Gemm. The model is deployed on an
STM32H7xx target based on an Arm Cortex-M7 core running at 400 MHz
with standard DSP and floating-point unit. In both configurations,
internal flash and RAM are used to store the application, weights,
and activations. The cosine similarity (cos) and SNR are evaluated
using random data.
| model | used option | macc | weights | activations | nodes | inf/s | cos | snr |
|---|---|---|---|---|---|---|---|---|
| f32 | default | 8,707,450 | 367,216 | 79,600 | 5 | 7.37 (7.48 cycles//macc) | 1.0 | 128 |
| f32 | --use-f16 |
8,726,842 | 184,016 | 39,800 | 7 | 3.51 (15.68 cycles//macc) | 1.0 | 68 |
| w8a8 | default | 8,948,442 | 91,816 | 407,984 | 16 | 8.35 (6.43 cycles//macc) | 1.0 | inf |
| w8a8 | --use-f16 |
9,189,434 | 91,816 | 242,592 | 27 | 3.47 (15.06 cycles//macc) | 0.999 | 44 |
- Without dedicated float16 acceleration support, additional CPU
cycles are required to convert float16 data to float32 before
computation. This conversion is performed in the critical execution
path. As a result,
--use-f16does not provide an inference-time benefit in this configuration; it mainly reduces weight storage and peak memory usage.
--no-inputs-allocation
- supported target:
stm32xx,stm32n6/stm32h7w/ NPU,sr5[xx],sr6p[xx],sr6g[xx],ispu,sr6p3ew/ and w/o NPU - unsupported target:
mlc
Do not reserve space in the activations buffer for input buffers. Allocate them separately in user memory and provide them to the execution engine before inference. Refer to the “IO buffers into activations buffer” section.
--no-outputs-allocation
- supported target:
stm32xx,stm32n6/stm32h7w/ NPU,sr5[xx],sr6p[xx],sr6g[xx],ispu,sr6p3ew/ and w/o NPU - unsupported target:
mlc
Do not reserve space in the activations buffer for output buffers. Allocate them separately in user memory and provide them to the execution engine before inference. Refer to the “IO buffers into activations buffer” section.
--input-memory-alignment INT
- supported target:
stm32xx,sr5[xx],sr6p[xx],sr6g[xx],ispu,sr6p3e - unsupported target:
stm32n6/stm32h7w/ NPU,mlc,sr6p3ew/ NPU
Set the input buffer alignment in bytes (a multiple of 2) within the activations buffer. The default is 4 or 8 bytes, depending on the system bus width. Refer to the “I/O buffers into activations buffer” section.
--output-memory-alignment INT
- supported target:
stm32xx,sr5[xx],sr6p[xx],sr6g[xx],ispu,sr6p3e - unsupported target:
stm32n6/stm32h7w/ NPU,mlc,sr6p3ew/ NPU
Set the output buffer alignment in bytes (a multiple of 2) within the activations buffer. The default is 4 or 8 bytes, depending on the system bus width. Refer to the “I/O buffers into activations buffer” section.
--memory-pool FILE
- supported target:
stm32xx,sr5[xx],sr6p[xx],sr6g[xx],sr6p3e - unsupported target:
stm32n6/stm32h7w/ NPU,mlc,ispu,sr6p3ew/ NPU
Specify the path to the memory pool descriptor file. The file describes the memory regions required to support the multiheap use case. For STELLAR, you can find some memory pool descriptor files under the Utilities/<osname>/targets/stellar/resources/memory_pool_files folder.
Details
Description of the Memory pools
For advanced use-cases, the user has the possibility to pass
(through the '--memory-pool' option), a text file (JSON
file format) allowing to specify some properties of the targeted
device. The “1.0” JSON version allows only to
provide the descriptions of the memory pools.
{
"version": "1.0",
"memory": {
"mempools": [
{
"name": "sram",
"size": "128KB",
"usable_size": "96KB"
}
]
}
}| key | description |
|---|---|
"version" |
version/format of the JSON file. Only
"1.0" value is supported - mandatory |
"memory" |
key to describe the memory properties (dict) - optional |
"mempools" |
key to describe the memory pool (list of dict) - optional |
Memory pool description ("mempools" item)
| key | description |
|---|---|
"name" |
user name, if not defined a generic
name is generated: pool_{pos} - optional |
"size" |
indicate the total size - mandatory |
"usable_size" |
indicate the maximum size which can be
used - if not defined, size is used -
optional |
"address" |
indicate the base @ of memory pool
(ex. 0x2000000) - optional |
- value is defined as a string,
"B,KB,MB"can be used to indicate the size."0x"prefix indicates value in hexadecimal.
- no target device database is embedded in the CLI to check that the provided memory pool descriptors are valid.
- Note that if
"address"attribute is defined and if the associated memory pool is used, the value is used as-is to define the default address of the activations buffer by the generated code (see generated<network>_data.cfile).
A typical example of JSON file indicates that two budgeted memory
pools can be used to place the activations buffer.
"dtcm" memory pool is privileged to place the critical
buffers.
{
"version": "1.0",
"memory": {
"mempools": [
{
"name": "dtcm",
"size": "128KB",
"usable_size": "64KB"
},
{
"name": "ram_d1",
"size": "512KB",
"usable_size": "256KB"
},
{
"name": "default"
}
]
}
}
--fix-parametric-shapes
- supported target:
stm32xx,stm32n6/stm32h7w/ NPU,sr5[xx],sr6p[xx],sr6g[xx],ispu,sr6p3ew/ and w/o NPU - unsupported target:
mlc
Set the parametric dimensions in input tensor shapes (default:
1).
Details
Accepted formats are
| format | description |
|---|---|
| List of tuples | A list of tuples specifying for each
input tensor its input shape. The tensors order is the same shown by
Netron. Example: [(1,2,3),(1,3,4)] |
| Dictionary (tensors) | A dictionary which associates an input
tensor name with its value. Example:
{'input1':(1,5,6),'input0':(1,4,5)} |
| Dictionary (dimensions) | A dictionary which associates a
dimension name with its value. Example:
{'batch_size':1,'sequence_length':10,'sample_size':12} |
--split-weights
- supported target:
stm32xx,sr5[xx],sr6p[xx],sr6g[xx],ispu,sr6p3e - unsupported target:
stm32n6/stm32h7w/ NPU,mlc,sr6p3ew/ NPU
Generate one C array per weight or bias tensor instead of a single C array (the “weights” buffer) for the whole model (default: disabled). Refer to the “Split weights buffer” section.
-O/--optimization STR
- supported target:
stm32xx,sr5[xx],sr6p[xx],sr6g[xx],sr6p3e - unsupported target:
stm32n6/stm32h7w/ NPU,mlc,ispu,sr6p3ew/ NPU
Specify the objective
for the optimization passes.
Details
Optimization objectives
The '-O/--optimization' option selects the objective
for the optimization passes used to deploy the C model. The accuracy
and precision of the generated model is not impacted. By default
(w/o option), a trade-off (that is, balanced) is
considered.
| objective | description |
|---|---|
time |
apply the optimization passes to reduce the inference time (or latency). In this case, the size of the used RAM (activations buffer) can be impacted. |
ram |
apply the optimization passes to reduce the RAM used for the activations. In this case, the inference time can be impacted. |
balanced |
trade-off between the
'time' and the 'ram' objectives. Reduces
RAM usage yet minimizing impact on inference time |
The following figure illustrates the usage of the optimization
option. It is based on the 'Nucleo-H743ZI@480MHz' board
with the small MLPerf Tiny quantized models from https://github.com/mlcommons/tiny/tree/master/benchmark/training.
--c-api STR
- supported target:
stm32xx,sr5[xx],sr6p[xx],sr6g[xx],sr6p3e(forispuonlyst-aiis supported)
- unsupported target:
stm32n6/stm32h7w/ NPU,mlc,sr6p3ew/ NPU
Select the generated embedded inference C API:
legacy or st-ai (default:
st-ai for all supported targets). For more details,
refer to the “Embedded
Inference Client API” and “Embedded Inference Client
ST Edge AI API” articles.
--allocate-activations
- supported target:
stm32xx,sr5[xx],sr6p[xx],sr6g[xx],ispu,sr6p3e - unsupported target:
stm32n6/stm32h7w/ NPU,mlc,sr6p3ew/ NPU
(Experimental) Use this option only with the st-ai C
API to allocate the activation buffers in the runtime. Otherwise,
the application must provide them (default behavior).
--allocate-states
- supported target:
stm32xx,sr5[xx],sr6p[xx],sr6g[xx],ispu,sr6p3e - unsupported target:
stm32n6/stm32h7w/ NPU,mlc,sr6p3ew/ NPU
Use this option only with the st-ai C API to
allocate the state buffers in the runtime. Otherwise, the
application must provide them (default behavior).
--input-data-type
- supported target:
stm32xx,stm32n6/stm32h7w/ NPU,sr5[xx],sr6p[xx],sr6g[xx],ispu,sr6p3ew/ and w/o NPU - unsupported target:
mlc
For quantized models, specify the input data type for the
generated implementation. Specify one type per input
(in_data_type_1,in_data_type_2,...), or provide one
type to apply it to all inputs (supported values:
float32|int8|uint8).
Details
| model type | supported options |
|---|---|
| Keras (float) | int8 and
uint8 data type are not supported. float32
can be used but the original data type is unchanged. |
| ONNX (float) | int8 and
uint8 data type are not supported. float32
can be used but the original data type is unchanged. |
| TFlite (float) | int8 and
uint8 data type are not supported. float32
can be used but the original data type is unchanged. |
| TFlite (quantized) | int8, uint8
and float32 are supported. According the original data
types a converter is inserted. |
| ONNX (quantized)* | int8, uint8
and float32 are supported. According the original data
types a converter is inserted. |
(*) by default for this type of model, the original I/O data type (float32) is converted to int8 data type to feed directly the int8 kernels allowing to support efficiently the deployed ONNX QDQ models (see “QDQ format deployment” section).
--output-data-type
- supported target:
stm32xx,stm32n6w/ NPU,sr5[xx],sr6p[xx],sr6g[xx],ispu,sr6p3ew/ and w/o NPU - unsupported target:
mlc
For quantized models, specify the output data type for the
generated implementation. Specify one type per output
(out_data_type_1,out_data_type_2,...), or provide one
type to apply it to all outputs (supported values:
float32|int8|uint8).
--inputs-ch-position
- supported target:
stm32xx,stm32n6w/ NPU,sr5[xx],sr6p[xx],sr6g[xx],ispu,sr6p3ew/ and w/o NPU - unsupported target:
mlc
Specify the expected NCHW (channel-first) or NHWC (channel-last)
data layout for the inputs (refer to “How to change the
I/O data type or layout (NHWC vs NCHW)” article for more
details). Apply this setting to all inputs when the model has
multiple inputs (supported values: chfirst|chlast).
--outputs-ch-position
- supported target:
stm32xx,stm32n6/stm32h7w/ NPU,sr5[xx],sr6p[xx],sr6g[xx],ispu,sr6p3ew/ and w/o NPU - unsupported target:
mlc
Specify the expected NCHW (channel-first) or NHWC (channel-last)
data layout for the outputs (refer to “How to change the
I/O data type or layout (NHWC vs NCHW)” article for more
details). Apply this setting to all outputs when the model has
multiple outputs (supported values:
chfirst|chlast).
--compute-runtime-code-size
- supported target:
stm32xx,stm32n6/stm32h7w/ and w/o NPU,sr5[xx],sr6p[xx],sr6g[xx],ispu,sr6p3ew/ and w/o NPU - unsupported target:
mlc
Use --compute-runtime-code-size or
--no-compute-runtime-code-size with
analyze or generate to enable or disable
computation of the runtime code
size.
- Default for
analyze: enabled. - Default for
generate: disabled. - Explicit CLI value overrides defaults for each command.
--const-in-const
- supported target:
stm32xx,sr5[xx],sr6p[xx],sr6g[xx],sr6p3ew/o NPU - unsupported target:
mlc,stm32n6/stm32h7w/ NPU,sr6p3ew/ NPU
Generate the network descriptor structures as const C structure
objects in the .rodata section, reducing the amount of
RAM required. The size reduction depends on the model and on the
generated C API, in particular the
number of operators and associated weight and parameter tensors.
Note that tensor descriptors for weights and/or activations are not considered, in order to avoid API break, since these values may be updated at runtime.
Example
The following table illustrates the use of
--const-in-const with a typical representative model
(~10 nodes). Only the object related to the 'network.o'
file are reported.
| option | .text |
.rodata |
.data |
.bss |
gain |
|---|---|---|---|---|---|
| default | 2,734 | 2,390 | 280 | 24 | |
--const-in-const |
2,714 | 2,518 | 152 | 24 | -45% of ram |
--c-api legacy |
1,160 | 3,410 | 6,132 | 300 | |
--c-api legacy --const-in-const |
1,160 | 5,882 | 3,660 | 300 | -40% of ram |
--cut-input-tensors
For TFLite models, specify one tensor location or a comma-separated list of tensor locations at which to cut the input model. For ONNX models, specify one tensor name or a comma-separated list of tensor names at which to cut the input model.
--cut-output-tensors
For TFLite models, specify one tensor location or a comma-separated list of tensor locations at which to cut the input model. For ONNX models, specify one tensor name or a comma-separated list of tensor names at which to cut the input model.
--cut-input-layers
For TFLite models, specify one layer location or a comma-separated list of layer locations at which to cut the input model. For Keras models, specify one layer index or a comma-separated list of layer indices at which to cut the input model.
--cut-output-layers
For TFLite models, specify one layer location or a comma-separated list of layer locations at which to cut the input model. For Keras models, specify one layer index or a comma-separated list of layer indices at which to cut the input model.
--no-onnx-optimizer
- supported target:
stm32xx,stm32n6/stm32h7w/ NPU,sr5[xx],sr6p[xx],sr6g[xx],ispu,sr6p3ew/ and w/o NPU - unsupported target:
mlc
Disable the ONNX optimizer pass before importing the ONNX model.
--use-onnx-simplifier
- supported target:
stm32xx,stm32n6/stm32h7w/ NPU ,sr5[xx],sr6p[xx],sr6g[xx],ispu,sr6p3ew/ and w/o NPU - unsupported target:
mlc
Enable the ONNX simplifier pass before importing the ONNX model (default: disabled).
--st-neural-art STR
- supported target: stm32n6, stm32h7, sr6p3e with Neural-ART
- unsupported target:
stm32xx,sr5[xx],sr6p[xx],sr6g[xx],ispu,mlc
Set the selected profile (including the Neural-ART compiler options, refer to “Neural-ART compiler primer” article) from a well-defined configuration file.
--custom FILE
- supported target:
stm32xx,stm32n6/stm32h7w/ NPU,sr5[xx],sr6p[xx],sr6g[xx],ispu,sr6p3ew/ and w/o NPU - unsupported target:
mlc
Specify the path to the JSON configuration file for custom layers. Refer to the “Keras Lambda/custom layer support” article.
-v/--verbosity {0,1,2,3}
Set the verbosity level (amount of displayed information).
Supported values: 0, 1, 2
(default: 1).
--quiet
Disable the progress bar during command execution.
Out-of-the-box information
The first part of the log shows the used arguments and the main metrics of the C implementation.
$ stedgeai analyze -m ds_cnn.h5 --target stm32
..
Exec/report summary (analyze)
-------------------------------------------------------------------------------------------
model file : <model-path>\ds_cnn.h5
type : keras
c_name : network
compression : lossless
optimization : balanced
target/series : stm32h7
workspace dir : <workspace-directory-path>
output dir : <output-directory-path>
model_fmt : float
model_name : ds_cnn
model_hash : 0xb773f449281f9d970d5b982fb57db61f
params # : 40,140 items (156.80 KiB)
-------------------------------------------------------------------------------------------
input 1/1 : 'input_0', f32(1x49x10x1), 1.91 KBytes, user
output 1/1 : 'dense_1', f32(1x12), 48 Bytes, user
macc : 4,833,792
weights (ro) : 158,768 B (155.05 KiB) (1 segment) / -1,792(-1.1%) vs float model
activations (rw) : 55,552 B (54.25 KiB) (1 segment)
ram (total) : 57,560 B (56.21 KiB) = 55,552 + 1,960 + 48
-------------------------------------------------------------------------------------------
...
The initial subsection recalls the CLI arguments. Note that the
full raw command line is saved at the beginning of the generated
report file:
<output-directory-path>\network_<cmd>_report.txt
| field | description |
|---|---|
| model file | reports the full-path of the original
model files (-m/--model). If multiple files, there is
one line by file. |
| type | reports the -t/--type
value or inferred DL framework type |
| c_name | reports the expected C-name for the
generated C-model (-n/--name) |
| compression | reports the applied compression level
(-c/--compression) |
| optimization | reports the selected objective:
balanced (default), ram or
time, (-O/--optimization) |
| target/series | reports the selected target/series
(--target) |
| workspace dir | full-path of the workspace directory
(-w/--workspace) |
| output dir | full-path of the output directory
(-o/--output) |
The second part shows the results of the importing and rendering stages.
| field | description |
|---|---|
| model_fmt | designates the main format of the
generated model: float, ss/sa,
dqnn,.. |
| model_name | designates the name of the provided model. This is generally the name of the model file. |
| model_hash | provides the computed MD5 signature of the imported model files. |
| input | indicates the name, the format, the
shape, and the size in bytes of an input tensor. There is one line
by input. 'inputs (total)' field indicates the total
size (in bytes) of the inputs. |
| output | indicates the name, the format, the
shape, and the size of the output tensor. There is one line by
output. outputs (total) field indicates the total size
(in bytes) of the outputs. |
| params # | indicates the total number of parameters of the original model and its associated size in bytes. |
| macc | indicates the whole computational
complexity of the original model. Value is defined in
MACC operations: Multiply ACCumulated operations, refer
to “Computational
complexity: MACC and cycles/MACC” |
| weights (ro) | indicates the requested size (in bytes) for the generated constant RO parameters (weights and bias tensors). The size is 4 bytes aligned. If the value is different from the original model files, the ratio is also reported. (refer to “Memory-related metrics” section) |
| activations (rw) | indicates the requested size (in bytes) for the working RW memory buffer (also called activations buffer). It is mainly used as an internal heap for the activations and temporary results. (refer to “Memory-related metrics” section) |
| ram (total) | indicates the requested total size (in bytes) for the RAM including the input and output buffers. |
Note that when the --memory-pool is
passed, the next part 'Memory-pools summary' summarizes
the usage of the memory pools.
Memory-pools summary (activations/ domain)
--------------------------- ---- -------------------------- ---------
name id used buffer#
--------------------------- ---- -------------------------- ---------
sram 0 54.25 KiB (10.8%) 34
weights_array 1 155.05 KiB (15876800.0%) 35
input_0_output_array_pool 2 1.91 KiB (196000.0%) 1
dense_1_output_array_pool 3 48 B (4800.0%) 1
--------------------------- ---- -------------------------- ---------
Example of ‘input/output’ description
'input_0', f32(1x49x10x1), 1.91 KBytes, userIndicates that
input_0tensor has a size of 490 floating-point items (size in bytes =490 x 4B = 1.91KiB) with a(1x49x10x1)shape, the associated memory chunk will be provided by the application ('user'domain) application (refer to “I/O tensor description” section). On the contrary in'input_0', f32(1x49x10x1), 1.91 KBytes, activationsThe description is similar, however thanks the
--allocate-inputsoption, a specific region is reserved in the activations buffer for the input ('activations'domain).Compressed floating-point model example
For a “compressed” floating-point model, the compression gain for the
'weights'size, here -72.90% is the global difference between a 32b float model and the generated “compressed” C-model. Note that only the full-connected or dense layers can be compressed.$ stedgeai analyze -m dnn.h5 -c low --target stm32 ... compression : low ... input 1/1 : 'input_0', f32(1x490), 1.91 KBytes, user output 1/1 : 'dense_4', f32(1x12), 48 Bytes, user macc : 114,816 weights (ro) : 123,792 B (120.89 KiB) (1 segment) / -333,024(-72.9%) vs float model activations (rw) : 1,152 B (1.12 KiB) (1 segment) ram (total) : 3,160 B (3.09 KiB) = 1,152 + 1,960 + 48 ...Quantized TFLite model example - integer format
The following report shows the case where a TensorFlow lite quantized model is imported and the inputs are placed in the activations buffer. Note that for each input (or output), type/scale, and zero-point value are reported. Additional infos are displayed in the “IR Graph description” section.
$ stedgeai analyze -m <quantized_model_file>.tflite --allocate-inputs --target stm32 ... input 1/1 : 'Reshape_1', int8(1x1960), 1.91 KBytes, QLinear(0.101715684,-128,int8), activations output 1/1 : 'nl_3', int8(1x4), 4 Bytes, QLinear(0.003906250,-128,int8), user macc : 336,072 weights (ro) : 16,688 B (16.30 KiB) (1 segment) / -49,920(-74.9%) vs float model activations (rw) : 12,004 B (11.72 KiB) (1 segment) * ram (total) : 12,008 B (11.73 KiB) = 12,004 + 0 + 4 (*) input buffers can be used from the activations buffer ...
IR graph description
The outlined “graph” section (table form) provides a summary of
the topology of the network which is considered before the
optimization, render, and generation stages. The 'id'
column indicates the index of the operator from the original graph.
It is generated by the importer. The described graph is an internal
platform-independent representation (or IR) created during the
import of the model. Only training operators are ignored. Note that
if no input operator is defined, an “input” layer is added and the
nonlinearity functions are unfused.
| field | description |
|---|---|
| id | indicates the layer/operator index in the original model. |
| layer (type) | designates the name and type of the
operator. The name is inferred from the original name. In the case
where a nonlinearity function is unfused, the new IR-node is created
with the original name suffixed with '_nl' (see next
figure with the first layer) |
| shape | indicates the output shape of the layer. Follow the “HWC” layout or channel last representation (refer to “I/O tensor” section) |
| param/size | indicates the number of parameters and their sizes in bytes (4 bytes aligned) |
| macc | designates the complexity in multiply-accumulated operations, refer to “Computational complexity: MACC and cycles/MACC” |
| connected to | designates the name of the incoming operators/layers |
The right side of the table ('c_*' columns) reports
generated C-object after optimization and rendering stages.
| field | description |
|---|---|
| c_size | indicates the difference in bytes of
the size for the implemented weights/params tensors. If nothing is
indicated, the size is unchanged compared to the original size
('-/size' field) |
| c_macc | indicates the difference in MACC. If
nothing is displayed, the final complexity of the C-operator is
comparable to the complexity of the original layer/operator
('macc' field). |
| c_type | indicates the type of the c-operator.
The value between square is the index in the c-graph. The value
between parenthesis is the data type: '()' indicates a
float32 type, '(i)' for integer type,
'(c4, c8)' for the compressed floating-point layer
(size includes also the associated dictionary). Multiple c-operators
can be generated for an original operator. |
Footer summarizes the differences for the whole model including the requested RAM size for the activations buffer and for the I/O tensors.
model/c-model: macc=369,672/369,688 +16(+0.0%) weights=18,288/18,288
activations=--/6,032 io=--/2,111In the case where the optimizer engine has folded or/and fused
the IR nodes, the 'c_type' is empty.
The following figure is an example of an IR graph with a residual
neural network. As for the multiple branches, no specific
information is added, 'connected to' column allows to
know the connections.
Warning
For a compressed or quantized model, the MACC values (by layer or globally) are unchanged since the number of operations is always the same. Only the associated number of CPU cycles by MACC is changed. In particular, for the quantized models.
Number of operations per c-layer
Number of operations by generated C-layer ('c_id')
according to the type of data is provided. With the synthesis by
types of operations for the entire model, this information makes it
possible to know the partitioning of the operations in relation with
the types of data.
Number of operations per c-layer
----------------------------------------------------------------------------------------------
c_id m_id name (type) #op (type)
----------------------------------------------------------------------------------------------
0 1 quant_conv2d_conv2d (conv2d_dqnn) 230,416 (smul_s8_s8)
1 3 quant_conv2d_1_conv2d (conv2d_dqnn) 1,843,200 (sxor_s1_s1)
...
14 25 quant_depthwise_conv2d_3_conv2d (conv2d_dqnn) 28,800 (sxor_s1_s1)
...
16 28 quant_conv2d_7_conv2d (conv2d_dqnn) 1,638,400 (sxor_s1_s1)
17 30 activation (nl) 6,400 (op_f32_f32)
18 32 conv2d_conv2d (conv2d) 76,812 (smul_f32_f32)
----------------------------------------------------------------------------------------------
total 10,067,228
Number of operation types
---------------------------------------------
smul_s8_s8 230,416 2.3%
sxor_s1_s1 9,740,800 96.8%
op_s1_s1 12,800 0.1%
op_f32_f32 6,400 0.1%
smul_f32_f32 76,812 0.8%
| operation | description |
|---|---|
smul_f32_f32 |
floating-point macc-type operation |
smul_s8_s8 |
8-bit signed integer macc-type operation |
op_f32_f32 |
Floating point operation (nonlinearity, elemtwise op…) |
conv_s8_f32 |
converter operation; s8 -> f32 |
xor_s1_s1 |
binary operation (~macc) |
Complexity report per layer
The last part of the report summarizes the relative network
complexity in term of MACC and associated ROM size by layer. Note
that only the operators which contribute to the global
'c_macc' and 'c_rom' metrics are reported.
'c_id' indicates the index of the associated
c-node.
Complexity report per layer - macc=18,752,688 weights=7,552 act=3,097,600 ram_io=602,184
---------------------------------------------------------------------------------------------------
id name c_macc c_rom c_id
---------------------------------------------------------------------------------------------------
1 separable_conv1 || 1.8% || 1.6% [0]
1 separable_conv1_conv2d ||| 3.2% |||| 3.4% [1]
2 depthwise_conv2d_1 ||||||||| 10.3% |||||||| 8.5% [2]
3 conv2d_1 |||||||||||||||| 17.6% |||||||||||||| 14.4% [3]
5 dw_conv_branch1 |||||||| 9.3% |||||||| 8.5% [7]
6 pw_branch1 |||||||||||||||| 17.6% |||||||||||||| 14.4% [8]
7 dw_conv_branch0 |||||||| 9.3% |||||||| 8.5% [6]
8 batch_normalization_1 || 2.1% || 1.7% [9]
9 separable_conv1_branch2 |||||||| 9.3% |||||||| 8.5% [4]
9 separable_conv1_branch2_conv2d ||||||||||||||| 16.5% |||||||||||||| 14.4% [5]
10 add_1 || 2.1% | 0.0% [10, 11]
11 global_average_pooling2d_1 | 1.0% | 0.0% [12]
12 dense_1 | 0.0% |||||||||||||||| 16.2% [13]
12 dense_1_nl | 0.0% | 0.0% [14]
C-graph description
Additional “Generated C-graph summary” section is included in the
report (also displayed with '-v 2' argument). It
summarizes the main computational and associated elements
(c-objects) used by the C-inference engine (runtime library). It is
based on the c-structures generated inside the
'<name>.c' file. A complete graphic
representation is available through the UI (refer to [UM]).
The first part recalls the main structural elements: c-name, number of c-nodes, number of C-array for the data storage of the associated tensors. Input and output name of the I/O tensors.
Generated C-graph summary
---------------------------------------------------------------------------------------------------
model name : microspeech_01
c-name : network
c-node # : 5
c-array # : 11
activations size : 4352
weights size : 16688
macc : 336084
inputs : ['Reshape_1_output_array']
outputs : ['nl_2_fmt_output_array']
As illustrated in the following figure, the implemented c-graph (legacy API) can be considered as a sequential graph, managed as a simple linked list. Fixed-executing order is defined by the C-code optimizer according to two main criteria: data-path dependencies (or tensor dependencies) and the minimization of the RAM memory peak usage.
Each computational c-node is entirely defined by:
- operation type, parameters
- input tensors list: [I]
- optional weights/bias tensors list: [W]
- optional scratches tensors list: [S]
- outputs tensors list: [O]
C-Arrays table
'C-Arrays' table lists the objects allowing to
handle the base address, size, and metadata of the data memory
segments for the different tensors. For each item, number of items,
size in byte ('item/size'), memory segment location
('mem-pool'), type ('c-type') and short
format description ('fmt') are reported.
C-Arrays (11)
---------------------------------------------------------------------------------------------------
c_id name (*_array) item/size mem-pool c-type fmt comment
---------------------------------------------------------------------------------------------------
0 conv2d_0_scratch0 352/352 activations uint8_t ua8
1 dense_1_bias 4/16 weights const int32_t ss32
2 dense_1_weights 16000/16000 weights const uint8_t ua8
3 conv2d_0_bias 8/32 weights const int32_t ss32
4 conv2d_0_weights 640/640 weights const uint8_t ua8
5 Reshape_1_output 1960/1960 user uint8_t ua8 /input
6 conv2d_0_output 4000/4000 activations uint8_t ua8
7 dense_1_output 4/4 activations uint8_t ua8
8 dense_1_fmt_output 4/16 activations float float
9 nl_2_output 4/16 activations float float
10 nl_2_fmt_output 4/4 user uint8_t ua8 /output
---------------------------------------------------------------------------------------------------
| mem_pool | description |
|---|---|
| activations | part of the activations buffer |
| weights | part of a ROM segment |
| user | part of a memory segment owned by the user (client application level) |
| fmt | format description |
|---|---|
| float | 32b float numbers |
| s1/packed | binary format |
| bool | boolean format |
| c4/c8 | compressed 32b float numbers. The size includes the dictionary. |
| s, u, ua, ss, sa | integer or/and quantized format (refer
to [“Quantized models support”]]STAI_CORE_QUANT article).
'/ch(n)' indicates that per-channel scheme is used
(else per-tensor). |
C-Layers table
'C-Layers' table lists the c-nodes. For each node,
the c-name (name), type, macc, rom and associated
tensors (with the shape for the I/O tensors) are reported.
Associated c-array can be found with its name (or array id).
C-Layers (5)
---------------------------------------------------------------------------------------------------
c_id name (*_layer) id type macc rom tensors shape (array id)
---------------------------------------------------------------------------------------------------
0 conv2d_0 0 conv2d 320008 672 I: Reshape_1_output [1, 49, 40, 1] (5)
S: conv2d_0_scratch0
W: conv2d_0_weights
W: conv2d_0_bias
O: conv2d_0_output [1, 25, 20, 8] (6)
---------------------------------------------------------------------------------------------------
1 dense_1 1 dense 16000 16016 I: conv2d_0_output0 [1, 1, 1, 4000] (6)
W: dense_1_weights
W: dense_1_bias
O: dense_1_output [1, 1, 1, 4] (7)
---------------------------------------------------------------------------------------------------
2 dense_1_fmt 1 nl 8 0 I: dense_1_output [1, 1, 1, 4] (7)
O: dense_1_fmt_output [1, 1, 1, 4] (8)
---------------------------------------------------------------------------------------------------
3 nl_2 2 nl 60 0 I: dense_1_fmt_output [1, 1, 1, 4] (8)
O: nl_2_output [1, 1, 1, 4] (9)
---------------------------------------------------------------------------------------------------
4 nl_2_fmt 2 nl 8 0 I: nl_2_output [1, 1, 1, 4] (9)
O: nl_2_fmt_output [1, 1, 1, 4] (10)
---------------------------------------------------------------------------------------------------'id'designates the layer/operator index from the original model allowing to retrieve the link with the implemented node ('c_id').
The following figure illustrates a quantized model where the softmax operator is implemented in float requesting to insert two converters. Note that this is just an example, the softmax operator is fully supported in int8.
Runtime code size
“Runtime” identifies all involved kernel objects
(software components) which are requested to execute the deployed
c-model on a given device (also called runtime AI-stack). To compute
these information, the '--target' option is used
to know the targeted device and an embedded gcc-based compiler
application should be available in the PATH.
The first part indicates the final contribution by module
(generated c-file or library) and by type of memory segment.
'RT total' line sum-ups the different contributors.
'lib (toolchain)' indicates the contribution of the
used toolchain objects (including typically the low-level floating
point operations from the libm/libgcc libraries). The extra lines
weights/activations/io recalls the requested size for
respectively the weights, the activations buffer and the payload for
the input/output tensors (refer to “memory-related
metrics” section from “Evaluation report and
metrics” article).
| segment | description |
|---|---|
| text | size in bytes for the code |
| rodata | size in bytes for the const data (usually stored in nonvolatile memory device, FLASH type, except for ISPU) |
| data | size in bytes for the initialized data (stored in volatile memory device like embedded RAM, initial values is stored in FLASH, except for ISPU) |
| bss | size in bytes for the zero-initialized data (stored in RAM) |
$ stedgeai analyze -m <model_path> --target stm32h7 --c-api legacy
...
Requested memory size by section - "stm32h7" target
----------------------------- -------- -------- ------- --------
module text rodata data bss
----------------------------- -------- -------- ------- --------
NetworkRuntime910_CM7_GCC.a 19,100 0 0 0
network.o 482 213 1,520 116
network_data.o 48 16 88 0
lib (toolchain)* 104 0 0 0
----------------------------- -------- -------- ------- --------
RT total** 19,734 229 1,608 116
----------------------------- -------- -------- ------- --------
weights 0 16,688 0 0
activations 0 0 0 12,004
io 0 0 0 1,964
----------------------------- -------- -------- ------- --------
TOTAL 19,734 16,917 1,608 14,084
----------------------------- -------- -------- ------- --------
* toolchain objects (libm/libgcc*)
** RT AI runtime objects (kernels+infrastructure)
| module | description |
|---|---|
| NetworkRuntime910_CM7_GCC.a | kernel objects implementing the requested operators |
| network.o | specialized code/data to manage the c-model |
| network_data.o | specialized code/data to manage the weight/activation buffers |
Note that the
'<network>_params_data.o'file does not appear in the table, because it contains only the values of the weights (c-array form) which is represented by the'*weights*’ extra line.
The last part summarizes the whole requested memory size per type of memory. It also illustrates the breakdown between the RT objects and the main dimensioning memory-related metrics of the deployed c-model (that is, ROM/RAM metrics).
Summary - "stm32h7" target
---------------------------------------------------
FLASH (ro) %* RAM (rw) %
---------------------------------------------------
RT total 21,571 56.4% 1,724 11.0%
---------------------------------------------------
TOTAL 38,259 15,692
---------------------------------------------------
* rt/total
ISPU example
The following log illustrates an example for the
'ispu' target. In the final summary, as the firmware is
loaded in the internal RAM through a serial interface by a host
processor, the requested size to store the initialized value of the
.data section is not considered.
$ stedgeai analyze -m <model_path> --target ispu --c-api stai
...
Requested memory size by section - "ispu" target
------------------- -------- -------- ------ --------
module text rodata data bss
------------------- -------- -------- ------ --------
network_runtime.a 10,970 0 4 0
network.o 1,968 80 0 0
lib (toolchain)* 1,844 428 0 0
------------------- -------- -------- ------ --------
RT total** 14,782 508 4 0
------------------- -------- -------- ------ --------
weights 0 16,688 0 0
activations 0 0 0 12,004
states 0 0 0 0
io 0 0 0 1,964
------------------- -------- -------- ------ --------
TOTAL 14,782 17,196 4 13,968
------------------- -------- -------- ------ --------
* toolchain objects (libm/libgcc*)
** RT AI runtime objects (kernels+infrastructure)
Summary - "ispu" target
----------------------------------------------------------
Code RAM (ro) %* Data RAM (rw) %
----------------------------------------------------------
RT total 15,290 47.8% 4 0.0%
----------------------------------------------------------
TOTAL 31,978 13,972
----------------------------------------------------------
* rt/total
Validate command
Description
The 'validate' command allows validating the
generated/deployed model. Two modes (--mode option) are
considered: host and target. The detailed
descriptions of the used metrics are described in the “Evaluation report and
metrics” article.
Native host runtime environment for model evaluation
The native runtime environment executes the original model on the
host machine. This runtime relies on Python packages associated with
the framework used to evaluate the model, such as the TFLite
interpreter for the TFLite file, ONNX runtime for the ONNX-based
model, and the Keras package with TensorFlow backend for Keras
files. These runtimes are embedded into the ST Edge AI CLI
executable without modification. Use the
'stedgeai --tools-version' command to check the
versions of the associated packages. The purpose of these
environments is to generate references for comparison with
the predictions produced by the deployed model in
terms of accuracy. Execution time is not considered. Therefore, all
optimizations aimed at improving inference time for the quantized
models are disabled by default to utilize the reference kernels,
avoiding side effects related to the CPU-specific optimizations. For
the ONNX model, the ONNX runtime session is configured with the
onnxruntime.GraphOptimizationLevel.ORT_DISABLE_ALL
option and for the TFlite models, the experimental TFLite
interpreter option
(tf.lite.experimental.OpResolverType.BUILTIN_REF) to
use the reference kernels should be enabled with the additional
options: --validate.experimentale kref.
$ stedgeai validate -m <tflite_model>.tflite --target stm32h7 --validate.experimental krefValidation on host
Option: '--mode host' (Default)
The specialized NN generated c-files are compiled on the host and linked with a specific network-runtime library implementing the reference C-kernels closed to the target implementation.
Validation on target
Option: '--mode target -d <desc>'
This mode allows validation of the deployed model on the
associated board. Before executing the 'validate'
command, flash the board with a specific validation firmware that
includes a dedicated COM stack and the deployed C-model. For each
target, the method to deploy the model on the associated development
board can differ.
- How-to create an STM32 validation firmware using STM32Cube AI Studio (https://wiki.st.com/stm32mcu/wiki/AI)
- How to evaluate a
model deployed on the Neural-ART accelerator™ on STM32N6
board
- How-to create an ISPU
validation firmware
- How-to create a STELLAR validation firmware
When the board is flashed and started, the same validation process applies. Only the execution of the deployed C-model is delegated to the target.
Examples
Minimal command to validate a model with self-generated random input data, “Validation on desktop”, default mode: --mode host.
stedgeai validate -m <model_file_path> --target stm32Minimal command to validate a model on STM32H7-based board. Note that a complete profiling report including execution time by layer is generated by default.
stedgeai validate -m <model_file_path> --mode target --target stm32h7 --desc serialValidation of a 32b float model with compression factor.
stedgeai validate -m <model_f32p_file_path> -c medium --target stm32Validate a model with a custom dataset (input samples)
stedgeai validate -m <model_file_path> -vi test_data.csv --target stm32
Specific options
--mode
Specify the validation mode.
| mode | description |
|---|---|
'host' |
Default value - Performs validation on the host. |
'target' |
Perform a validation on target. |
'host-io-only' |
Alias equivalent to ‘–mode host –io-only’ - deprecated - default behavior |
'target-io-only' |
Alias equivalent to ‘–mode target –io-only’ |
--val-json
Specify the generated <network>_c_info.json
file to use during model validation. Use this option only with the
validate command in target mode. It improves execution time by
skipping preliminary passes, such as importing or compiling the
model file. All required information is extracted directly from the
provided JSON file. After validation, the file is updated with computed metrics, including
measured inference time information.
Note
Since the release 3.0, if this option is not specified, the
computed metrics and measured inference time information are stored
in an extra file:
"<network>_c_info_valid.json".
-vi/--valinput
Specify the custom test dataset. If omitted, use an internally generated random dataset. Refer to the “Input validation files” section.
-vo/--valoutput
Specify the expected custom output values. If the data are
already provided in an *.npz file through
-vi, this argument is ignored.
-b/--batches
Specify the number of random samples to generate (default:
10) or the number of custom test samples to use
(default: all).
-d/--desc
Specify the protocol and parameters for communicating with the
deployed C model using the syntax
<driver>[:parameters]. Required when target mode is selected. For example,
specify the COM port used to communicate with a board (see the “Serial COM port configuration”
section).
--full
- supported target:
stm32xx,sr5[xx],sr6p[xx],sr6g[xx],sr6p3e - unsupported target:
ispu,stm32n6/stm32h7w/ NPU,mlc,sr6p3ew/ NPU
Run extended validation to report the L2r error layer-by-layer for the floating-point Keras model. Otherwise, the L2r is evaluated only on the last or output layers.
--io-only
Run the deployed model without instrumentation to retrieve intermediate data.
--classifier
Treat the model as a classifier to compute the CM and ACC metrics. Otherwise, use autodetection to determine whether the model is a classifier.
--no-check
In target mode, skip most preliminary checks when debugging. The option checks only the C name and network I/O shape and format against the flashed target C model.
--no-exec-model
Skip execution of the original model with a host deep learning runtime; execute only the generated C model. See the “Evaluation report and metrics” article.
--range
Specify the minimum and maximum floating-point values for
generated random data (default: [0.0, 1.0[). To
generate uniform random data between -1.0 and
1.0, use --range -1 1. Refer to the “Random data
generation” section.
--range-int
Specify the minimum and maximum integer values for generated
random data. No default is set. This option overrides
--range when used. Refer to the “Random data
generation” section.
--seed
Specify the seed for the pseudorandom number generator. If omitted, a fixed seed is used.
--save-csv
Save all data in the respective *.csv
files. By default, only a limited part is saved for performance
reasons.
For the ISPU target, an additional option is defined to specify the file needed to load the ISPU program (see the “Validate command extension” section of the ISPU-specific documentation).
Serial COM port configuration
Use the '-d/--desc' option to configure the serial
COM driver for board access. By default, the autodetection mechanism
discovers a connected board at 115200 baud rate (default value:
default:115200) or 921600 baud rate for ISPU
target.
To set the baud rate to 921600
stedgeai validate -m <model_file_path> --target stm32 --mode target -d serial:921600To set the COM port to
COM16(Windows case) or/dev/ttyACM0(Linux case)stedgeai validate -m <model_file_path> --target stm32 --mode target -d serial:COM16 stedgeai validate -m <model_file_path> --target stm32 --mode target -d serial:/dev/ttyACM0To set the COM port to
COM16and the baud rate to 921600stedgeai validate -m <model_file_path> --target stm32 --mode target -d serial:COM16:921600
Extended complexity report per layer
If '-v 2' option is used, the “Complexity report per
layer” table is extended with a specific column to report the
metric according the data type: 'l2r' for the
floating-point models and 'rmse' for the integer or
quantized models.
stedgeai validate -m <model_file_path> --target stm32 -v 2
...
Complexity report (model)
------ ----------- ------------------------- ------------------------- ---------- -------- ------------------
m_id name c_macc c_rom c_id c_dur rmse (X-CROSS)
------ ----------- ------------------------- ------------------------- ---------- -------- ------------------
0 input_1 |||||||||| 12.0% |||||||||| 11.6% [0] 9.6%
1 conv2d_1 ||| 2.7% ||| 3.4% [1, 2] 5.6%
2 conv2d_2 |||||||||||||||| 19.2% |||||||||||||||| 17.9% [3] 14.6%
3 conv2d_3 ||| 2.7% ||| 3.4% [4, 5] 5.7%
4 conv2d_4 |||||||||||||||| 19.2% |||||||||||||||| 17.9% [6] 19.4%
5 conv2d_5 ||| 2.7% ||| 3.4% [7, 8] 5.6%
6 conv2d_6 |||||||||||||||| 19.2% |||||||||||||||| 17.9% [9] 13.9%
7 conv2d_7 ||| 2.7% ||| 3.4% [10, 11] 5.6%
8 conv2d_8 |||||||||||||||| 19.2% |||||||||||||||| 17.9% [12] 19.4%
9 pool_9 | 0.3% | 0.0% [13] 0.4%
11 MatMul | 0.0% ||| 3.3% [14] 0.1%
12 nl_12 | 0.0% | 0.0% [15] 0.1% 0.00000000e+00 *
------ ----------- ------------------------- ------------------------- ---------- -------- ------------------
macc=2,665,536 weights=24,368 act=18,252 ram_io=0
...(*) indicates the max value
Execution time per layer
Validation on target
The validation on target allows to have a full and accurate profiling report including:
- inference-time
- number of CPU cycles by MACC
- execution time per layer
- Device HW settings/configurations (clock frequency, memory configuration)
...
Running the ST.AI c-model (AI RUNNER)...(name=network, mode=TARGET)
Proto-buffer driver v2.0 (msg v3.1) (Serial driver v1.0 - COM4:115200) ['network']
Summary 'network' - ['network']
-----------------------------------------------------------------------------------
I[1/1] 'input_1' : int8[1,1,28,28], 784 Bytes, QLinear(0.012722839,-95,int8),
activations
O[1/1] 'output_1' : f32[1,10], 40 Bytes,
activations
n_nodes : 9
activations : 32640
weights : 1200584
macc : 12052856
hash : 0x00f1e2478590bea3e6ed23bba954f39f
compile_datetime : Nov 5 2024 11:58:56
-----------------------------------------------------------------------------------
protocol : Proto-buffer driver v2.0 (msg v3.1)
(Serial driver v1.0 - COM4:115200)
tools : ST.AI (st-ai api) v2.0.0
runtime lib : v10.0.0-9a75ee0c compiled with GCC 12.3.1 (GCC)
capabilities : IO_ONLY, PER_LAYER, PER_LAYER_WITH_DATA, SELF_TEST
device.desc : stm32 family - 0x450 - STM32H743/53/50xx and
STM32H745/55/47/57xx @480/240MHz
device.attrs : fpu,art_lat=4,core_icache,core_dcache
-----------------------------------------------------------------------------------
ST.AI Profiling results v2.0 - "network"
---------------------------------------------------------------
nb sample(s) : 10
duration : 28.016 ms by sample (28.010/28.023/0.004)
macc : 12052856
cycles/MACC : 1.12
CPU cycles : [13,447,454]
---------------------------------------------------------------
Inference time per node
----------------------------------------------------------------------------------------------
c_id m_id type dur (ms) % cumul CPU cycles name
----------------------------------------------------------------------------------------------
0 11 Conv2D (0x103) 1.255 4.5% 4.5% [ 602,299 ] ai_node_0
1 17 Conv2dPool (0x109) 20.223 72.2% 76.7% [ 9,707,063 ] ai_node_1
2 20 Transpose (0x10a) 0.795 2.8% 79.5% [ 381,426 ] ai_node_2
3 20 NL (0x107) 0.580 2.1% 81.6% [ 278,565 ] ai_node_3
4 23 Dense (0x104) 5.147 18.4% 99.9% [ 2,470,516 ] ai_node_4
5 26 Dense (0x104) 0.009 0.0% 100.0% [ 4,214 ] ai_node_5
6 26 NL (0x107) 0.001 0.0% 100.0% [ 292 ] ai_node_6
7 29 Softmax (0x10c) 0.003 0.0% 100.0% [ 1,652 ] ai_node_7
8 30 NL (0x107) 0.003 0.0% 100.0% [ 1,427 ] ai_node_8
----------------------------------------------------------------------------------------------
n/a n/a Inter-nodal 0.000 0.0% 100.0% n/a
----------------------------------------------------------------------------------------------
total 28.016 [ 13,447,454 ]
----------------------------------------------------------------------------------------------
Statistic per tensor
----------------------------------------------------------------------------------
tensor # type[shape]:size min max mean std name
----------------------------------------------------------------------------------
I.0 10 i8[1,1,28,28]:784 -128 127 -1.681 73.679 input_1
O.0 10 f32[1,10]:40 -7.937 -0.033 -4.356 1.900 output_1
----------------------------------------------------------------------------------
...
This report can be used to identify the main contributors in
terms of inference time and to re-fine the model accordingly.
'c_id' column references the index of the c-node (see
“C-graph description” section)
and the 'm_id' identifies the index from the original
model.
out-of-the-box execution
When 'target-io-only' mode or --io-only
options are used, the deployed model is only executed
out-of-the-box. Executing time or l2r per layer are no more
computed. This can be used to limit the traffic between the host and
the target reducing the validation time.
...
ST.AI Profiling results v2.0 - "network"
------------------------------------------------------------------
nb sample(s) : 10
duration : 28.016 ms by sample (28.007/28.044/0.010)
macc : 12052856
cycles/MACC : 1.12
CPU cycles : [13,447,610]
used stack/heap : 1300/0 bytes
------------------------------------------------------------------
Statistic per tensor
----------------------------------------------------------------------------------
tensor # type[shape]:size min max mean std name
----------------------------------------------------------------------------------
I.0 10 i8[1,1,28,28]:784 -128 127 -1.681 73.679 input_1
O.0 10 f32[1,10]:40 -7.937 -0.033 -4.356 1.900 output_1
----------------------------------------------------------------------------------
...
Validation on host
For validation on
host, the relative execution time per layer is not reported by
default; the '-v 2' option should be used to display
them. Nevertheless, it is important to note that these values are
only indicators. They depend on the implementation of the
kernels, which are not optimized, and the workload of the
desktop/host machine (see device.desc field). This
contrasts with the reported inference times for validation on the
target.
$ stedgeai validate -m <model_file_path> --target stm32 -v 2 [--mode host]
...
Running the ST.AI c-model (AI RUNNER)...(name=network, mode=HOST)
DLL Driver v2.0 - Direct Python binding
(<workspace-directory-path>\inspector_network\workspace\lib\libai_network.dll) ['network']
Summary 'network' - ['network']
-----------------------------------------------------------------------------------
I[1/1] 'input_1' : int8[1,28,28,1], 784 Bytes, QLinear(0.012722839,-95,int8),
in activations buffer
O[1/1] 'output_1' : f32[1,1,1,10], 40 Bytes, in activations buffer
n_nodes : 9
activations : 32640
weights : 1200584
macc : 12052856
hash : 0x00f1e2478590bea3e6ed23bba954f39f
compile_datetime : Nov 15 2024 12:49:14
-----------------------------------------------------------------------------------
protocol : DLL Driver v2.0 - Direct Python binding
tools : ST.AI (legacy api) v2.0.0
runtime lib : v10.0.0
capabilities : IO_ONLY, PER_LAYER, PER_LAYER_WITH_DATA
device.desc : AMD64, Intel64 Family 6 Model 165 Stepping 2, GenuineIntel,
Windows
-----------------------------------------------------------------------------------
NOTE: The duration and execution time per layer are just indications. They depend
on the host machine's workload.
ST.AI Profiling results v2.0 - "network"
------------------------------------------------------------------
nb sample(s) : 10
duration : 6.068 ms by sample (5.698/6.571/0.223)
macc : 12052856
------------------------------------------------------------------
DEVICE duration : 7.066 ms by sample (including callbacks)
HOST duration : 0.074 s (total)
used mode : Mode.PER_LAYER
number of c-node : 9
------------------------------------------------------------------
Inference time per node
--------------------------------------------------------------------------------
c_id m_id type dur (ms) % cumul name
--------------------------------------------------------------------------------
0 11 Conv2D (0x103) 0.144 2.4% 2.4% ai_node_0
1 17 Conv2dPool (0x109) 5.142 84.7% 87.1% ai_node_1
2 20 Transpose (0x10a) 0.035 0.6% 87.7% ai_node_2
3 20 NL (0x107) 0.009 0.2% 87.8% ai_node_3
4 23 Dense (0x104) 0.731 12.0% 99.9% ai_node_4
5 26 Dense (0x104) 0.002 0.0% 99.9% ai_node_5
6 26 NL (0x107) 0.001 0.0% 99.9% ai_node_6
7 29 Softmax (0x10c) 0.002 0.0% 100.0% ai_node_7
8 30 NL (0x107) 0.001 0.0% 100.0% ai_node_8
--------------------------------------------------------------------------------
n/a n/a Inter-nodal 0.001 0.0% 100.0% n/a
--------------------------------------------------------------------------------
total 6.068
--------------------------------------------------------------------------------
Statistic per tensor
----------------------------------------------------------------------------------
tensor # type[shape]:size min max mean std name
----------------------------------------------------------------------------------
I.0 10 i8[1,28,28,1]:784 -128 127 -1.681 73.679 input_1
O.0 10 f32[1,1,1,10]:40 -7.937 -0.033 -4.356 1.900 output_1
----------------------------------------------------------------------------------
...
'c_id'designates the c-layer index in the “C-graph description”.
Generate command
Description
The 'generate' command is used to generate the
specialized network and data C-files. According to the c-api option, the used
target, and other additional options, the generated files can be
different.
Generated files with “legacy” C-API option
With the 'legacy' C-API, the following files are
generated:
$ stedgeai generate -m <model_file_path> --target stm32 -o <output-directory-path> [--c-api legacy]
...
Generated files (7)
-----------------------------------------------------------
<output-directory-path>\<name>_config.h
<output-directory-path>\<name>.c
<output-directory-path>\<name>_data.c
<output-directory-path>\<name>_data_params.c
<output-directory-path>\<name>.h
<output-directory-path>\<name>_data.h
<output-directory-path>\<name>_data_params.h
Creating report file <output-directory-path>\network_generate_report.txt
...
'<name>.c/.h'files contain the topology of the C-model (C-struct definition of the tensors and the operators), including the embedded inference client API (refer to “Embedded Inference Client API” article) to use the generated c-model on the top of the optimized inference runtime library.'<name>_data_params.c/.h'files contain by default a simple C-array with the data of the weight/bias tensors. However, the'--split-weights'option allows having a C-array by tensor (refer to “Split weights buffer” section) and the'--binary'option creates a binary file with the data of the weight/bias tensors. The'--relocatable/-r'option (available only for stm32) allows generating a relocatable binary model including the topology definition, the requested kernels, and the weights in a single binary file (refer to “Relocatable binary model support” article).'<name>_data.c/.h'files contain the intermediate functions requested by the specialized init function to manage the C-array with the weights.
Generated files with “st-ai” C-API option
With the 'st-ai' C-API, the following files are
generated:
$ stedgeai generate -m <model_file_path> --target sr5[xx] -o <output-directory-path> --c-api st-ai
or
$ stedgeai generate -m <model_file_path> --target sr6p[xx] -o <output-directory-path> --c-api st-ai
or
$ stedgeai generate -m <model_file_path> --target sr6g[xx] -o <output-directory-path> --c-api st-ai
...
Generated files (5)
-----------------------------------------------------------
<output-directory-path>\<name>.c
<output-directory-path>\<name>_data.c
<output-directory-path>\<name>.h
<output-directory-path>\<name>_data.h
<output-directory-path>\<name>_details.h
Creating report file <output-directory-path>\network_generate_report.txt
...
'<name>.c/.h'files contain the topology of the C-model (C-struct definition of the tensors and the operators), including the embedded inference client API (refer to “Embedded Inference Client ST Edge AI API” article) to use the generated c-model on the top of the optimized inference runtime library.
'<name>_data.c/.h'files contain by default a simple C-array with the data of the weight/bias tensors. However, the'--split-weights'option allows having a C-array by tensor (refer to “Split weights buffer” section)
'<name>_details.h'files contain the debug information about the intermediate tensors (debug/advanced purpose)
For ISPU target, the generated output also contains the runtime library and its header files and is structured in a manner to correctly populate the provided templates. For more details refer to the “Generate command extension” section of the ISPU specific documentation.
Examples
Generate the specialized NN C-files (default options).
$ stedgeai generate -m <model_file_path> --target sr5[xx] or $ stedgeai generate -m <model_file_path> --target sr6p[xx] or $ stedgeai generate -m <model_file_path> --target sr6g[xx]Generate the specialized NN C-files for a 32b float model with compression factor.
$ stedgeai generate -m <model_file_path> --target stm32 -c mediumGenerate a ZIP archive with the specialized NN C-files and associated runtime files for a c-project based on the HighTec or GNU Compiler Collection toolchain.
$ stedgeai generate -m <model_file_path> --target sr5[xx] --pack htc,gcc or $ stedgeai generate -m <model_file_path> --target stm32n6 --st-neural-art --pack gcc
Specific options
--pack
- supported target:
stm32xx,stm32h7/stm32n6w/ NPU,sr5[xx],sr6p[xx],sr6g[xx],sr6p3ew/ and w/o NPU - unsupported target:
mlc,ispu
The --pack option creates a ZIP archive containing
the files produced by the generate command, along with the
associated runtime files for the selected target. It also includes
an additional metadata file, app_build_info.json, which
provides the key information needed to integrate the generated files
into a C project. This option is useful for simplifying model
export, packaging, and deployment workflows.
$ stedgeai generate -m model.tflite --target <target-series> <options> --pack gcc
The archive name follows this pattern:
stai_<tool-version>_<target-series>_<model-name>.zip
Following extra options (comma-separated) are available.
| Option | Effect |
|---|---|
gen-only |
Do not add runtime resources or toolchain libraries. Generated
package files are still collected, and a ZIP is still created unless
no-zip is also specified. |
no-zip |
Do not create the ZIP archive. The package directory is still generated in the workspace folder. |
<tool-chain>> |
Indicates the expected toolchain’s runtime libraries. By
default, all supported libraries libraries are package. Names depend
on the selected target and commonly include gcc,
iar, mdk, ht,
armclang or ghs. Multiple toolchain can be
specified. |
Unrecognized options are ignored and reported as warnings.
Package ZIP
The ZIP preserves the package’s directory structure, with package contents at the archive root rather than nested under an extra enclosing folder. Typically it contains:
generated/: generated C sources, headers, and model data needed by the application.runtime/: target runtime headers, libraries, and (for applicable devices) NPU runtime files. This directory is omitted withgen-only, and may also be absent if runtime resources are unavailable.- Top-level package metadata such as
app_build_info.json,README.md, andLICENSE.txtwhen present. The generation report and C-info JSON are also placed at the package root unlesskeep-allis used, in which case they stay undergenerated/.
The exact files vary by target, available runtime resources, and
selected options. no-zip skips creating the archive
only; the package directory and its contents are still
generated.
Project configuration file: ‘app_build_info.json’
This JSON file describes the generated package for use by build integrations. Paths in its file lists are relative to the package root. It records:
- Package and model details, including the tool version, target, model name, model file, and creation date.
- The model MACC count, summed across the generated graph or graphs.
inputs-allocationandoutputs-allocationindicate whether input and output buffers are allocated by the generated application. Values aretruewhen allocation is enabled,falsewhen the correspondingno-...-allocationoption is used, andnullwhen the C-info file has no tool arguments.inputsandoutputsare arrays describing the resolved graph buffers. Each tensor entry contains:name: buffer name.shape: dimensions of the tensor.nb-elem: number of tensor elements, calculated as the product ofshape.size-bytes: buffer size in bytes as reported by C-info.data-format: present when the buffer format is available. It containstype,bit-size, andc-typewhen the C type is recognized. Quantization valuesscaleandzeroare included when present.
- Generated and NPU C source files, include directories, and runtime libraries.
- Common, debug, and release C compiler defines.
mempool-fileslists non-empty memory pools. Each entry contains:name: memory-pool name.initializer: initializer file path relative to the package root, ornullwhen no corresponding generated file exists. Neural Art pools use the compiler output; other read-only pools use the generated data file.addr: absolute memory address as a hexadecimal string, ornullfor a relative address.rights: normalized toread-onlyorread/write.size: used memory size in bytes; pools with zero used bytes are omitted.npu-cacheable: included for Neural Art packages and indicates whether the pool is cacheable.
- Optional configuration details such as selected toolchains, NPU compiler version, Neural Art, MVE, and HSP enablement.
target options
- For
'stm32'target a set of specific options is defined (see “Generate command extension” section) to address the additional UCs:- generation of a shared library to run the model locally (on the
host machine) through a specific Python module (see “How to use the AiRunner
package” article)
- generation of relocatable binary object to be installed and executed anywhere in an STM32 device (see “Relocatable binary model support” article)
- generation of a shared library to run the model locally (on the
host machine) through a specific Python module (see “How to use the AiRunner
package” article)
Supported-ops command
Description
The 'suppoorted-ops' command is used to display the
list of the supported operators for a given deep learning framework
with the '-t/--type' option.
Else by default, all operators are listed.
Specific arguments
--with-report
Generate a Markdown report listing the operators and their associated constraints.
This option has been used to generate the following articles: “Keras toolbox support”, “TFLite toolbox support and “ONNX toolbox support”
Examples
Generate the list of the supported operators (default)
$ stedgeai supported-ops ST Edge AI Core v1.0.0 281 operators found Abs (ONNX), ABS (TFLITE), Acos (ONNX), Acosh (ONNX), Activation (KERAS), ActivityRegularization (KERAS), Add (KERAS), Add (ONNX), ADD (TFLITE), AlphaDropout (KERAS), And (ONNX), ARG_MAX (TFLITE), ARG_MIN (TFLITE), ArgMax (ONNX), ArgMin (ONNX), ArrayFeatureExtractor (ONNX), Asin (ONNX), Asinh (ONNX),...Generate the list of the supported Keras operators
$ stedgeai supported-ops -t keras ST Edge AI Core v1.0.0 Parsing operators for KERAS toolbox 62 operators found Activation, ActivityRegularization, Add, AlphaDropout, Average, AveragePooling1D, AveragePooling2D, BatchNormalization, Bidirectional, Concatenate, Conv1D, Conv2D, Conv2DTranspose, Cropping1D, Cropping2D, Dense, DepthwiseConv2D, Dropout, ELU, Flatten, GaussianDropout, GaussianNoise, GlobalAveragePooling1D, GlobalAveragePooling2D, GlobalMaxPooling1D, GlobalMaxPooling2D, GRU InputLayer, .. 30 custom operators found Abs, Acos, Acosh, Asin, Asinh, Atan, Atanh, Ceil, Clip, Cos, Exp, Fill, FloorDiv, FloorMod, Gather, CustomLambda, Log, Pow, Reshape, Round, Shape, Sign, Sin, Split, Sqrt, Square, Tanh, Unpack, Where, TFOpLambdaGenerate the list of the supported ONNX operators
$ stedgeai supported-ops -t onnxGenerate the list of the supported tflite operators
$ stedgeai supported-ops -t tfliteGenerate the list of the supported Keras operators with a full report
$ stedgeai supported-ops -t keras --with-report ... Building report.. creating file : <output-directory-path>/supported_ops_keras.md