ST Edge AI Core for MLC

Feature extraction

To ensure a correct model training, the features comprising the training set must be aligned 1:1 to the MLC hardware. For this reason, ST Edge AI Core enables an extra generation step for automatically extracting the features from data logs acquired using ST MEMS software tools (e.g., MEMS-Studio, Unico-GUI, Unicleo-GUI)

This is done using an MLC generation script (in .json format) containing all settings needed for extracting the features. An example of MLC generation script can be found below:

{
    "datalogs": [
        { "filename": "data/stationary.csv", "label": "stationary" },
        { "filename": "data/walking.csv",    "label": "walking"    },
        { "filename": "data/cycling.csv",    "label": "cycling"    },
        { "filename": "data/running.csv",    "label": "running"    },
        "..."
    ],
    "name": "LSM6DSV16X",
    "mlc_odr": "30 Hz",
    "input_type": "accelerometer_only",
    "accelerometer_fs": "8 g",
    "accelerometer_odr": "30 Hz",
    "decision_tree_count": 1,
    "window_length": 60,
    "filters": [
        { "filter_id": "filter_1", "filter_type": "BP", "input": "Acc_V2", "a2": "-1.66", "a3": "0.81", "gain": "0.09" },
        "..."
    ],
    "features": [
        { "feature_name": "MEAN", "input": "Acc_V2",         },
        { "feature_name": "MEAN", "input": "Acc_V2_filter_1" },
        "..."
    ],
    "arff": "features.arff"
}

For more information on the MLC generation script, please follow the link to the dedicated article: How to write an MLC generation script.

By providing the script to the generate command, ST Edge AI Core will extract all the features in .arff format:

$ stedgeai generate --target mlc --device lsm6dsv16x --json conf.json --type arff

Model training

After generating the features, the data can now be loaded inside the ML framework of choice (e.g., Python, MATLAB, Weka, etc.) for model training and or data science tasks. The following snippet of code shows how this can be achieved using Python:

# Import dependencies
import numpy as np
import pandas as pd
from scipy.io import arff
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier, export_text

# Load labels and features from ARFF
arff_data = arff.loadarff("features.arff")
df = pd.DataFrame(arff_data[0])
X = df.drop("class", axis=1).values.astype(np.float32)
y = np.array([LABELS.index(label.decode()) for label in df["class"]])
feature_names = list(df.drop("class", axis=1).columns)

# Split dataset into train set and test set
X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y, test_size=0.3)

# Train decision tree classifer
clf = DecisionTreeClassifier(
    max_depth=128,
    criterion="entropy",
    min_impurity_decrease=1e-3,
    ccp_alpha=1e-2,
    class_weight="balanced",
)
clf.fit(X_train, y_train)

# Define utility function for weka format conversion
def convert_sklearn_to_weka(clf, feature_names):
    dt = re.sub(r"\|--- ", r"", export_text(clf, feature_names=feature_names)).splitlines()
    new_dt = []
    line_idx = 1
    while line_idx < len(dt):
        match = re.search(r"class: (\d+)$", dt[line_idx])
        if match is not None:
            label = LABELS[int(match.group(1))]
            new_dt.append(f"{dt[line_idx - 1]}: {label}")
            line_idx += 2
        else:
            new_dt.append(dt[line_idx - 1])
            line_idx += 1
    return "\n".join(new_dt)

# Export model to weka textual format
dectree = convert_sklearn_to_weka(clf, feature_names)
with open("dectree.txt", "w") as f:
    f.write(dectree)

After the training, to proceed with the generation of the complete configuration, the user should have one or more decision tree files in Weka format. An example can be found below:

F6_ENERGY_ACC_V2_FILTER_1 <= 7.86
|   F5_ENERGY_ACC_V2 <= 62.14: stationary
|   F5_ENERGY_ACC_V2 >  62.14
|   |   F11_PEAK_DETECTOR_ACC_V2 <= 31.00
|   |   |   F5_ENERGY_ACC_V2 <= 83.12: stationary
|   |   |   F5_ENERGY_ACC_V2 >  83.12: walking
|   |   F11_PEAK_DETECTOR_ACC_V2 >  31.00
|   |   |   F11_PEAK_DETECTOR_ACC_V2 <= 37.50
|   |   |   |   F12_PEAK_DETECTOR_ACC_V2_FILTER_1 <= 17.50: stationary
|   |   |   |   F12_PEAK_DETECTOR_ACC_V2_FILTER_1 >  17.50: cycling
|   |   |   F11_PEAK_DETECTOR_ACC_V2 >  37.50: cycling
F6_ENERGY_ACC_V2_FILTER_1 >  7.86
|   F1_MEAN_ACC_V2 <= 2.92
|   |   F9_ZERO_CROSSING_ACC_V2 <= 43.00: walking
|   |   F9_ZERO_CROSSING_ACC_V2 >  43.00: cycling
|   F1_MEAN_ACC_V2 >  2.92: running

Sensor configuration

A second, and final, generation step will now use the same MLC generation script as before with some additional fields to integrate one or more decision trees. An example of MLC generation script can be found below:

{
    "dectree_features": [
        "F1_MEAN_ACC_V2",
        "F2_MEAN_ACC_V2_FILTER_1",
        "..."
    ],
    "dectrees": [
        {
            "filename": "dectree.txt",
            "results": [
                { "label": "stationary", "value": 1  },
                { "label": "walking",    "value": 4  },
                { "label": "cycling",    "value": 8  },
                { "label": "running",    "value": 12 },
            ],
            "metaclassifier": [ 2, 2, 2, 2 ]
        },
        "..."
    ],
    "ucf": "mlc_conf.ucf"
}

For more information on the MLC generation script, please follow the link to the dedicated article: How to write an MLC generation script.

By running the generate command again with the updated script, we can now generate the .ucf file containing the register map of the sensor:

$ stedgeai generate --target mlc --device lsm6dsv16x --json conf.json --type ucf

A C header file (.h) containing the same register map is also generated in the output folder to ease the integration inside the firmware of the MCU/MPU used for configuring the sensor at runtime.

Performance analysis

By using the analyze command, ST Edge AI Core can execute the MLC logic offline by inputting features (extracted previously using the generate command) inside a given decision tree with optional metaclassifier stage to obtain the final predictions:

$ stedgeai analyze --target mlc --device lsm6dsv16x --tree dectree.txt --arff features.arff --meta meta.txt

where meta.txt configures the metaclassifier stage by specifying rows with: class name, metaclassifier end counter, and metaclassifier subgroup. For example:

stationary 2 0
walking 2 1
running 2 2
cycling 2 3

For more information on metaclassifier please refer to the MLC application note of the specific device, e.g., MLC application note for LSM6DSV16X sensor.

In addition to generating raw MLC prediction logs, a report summarizing decision tree information and predictive performance is also generated:

ST Edge AI Core v2.0.0 (MLC 1.1.0)
Created date   : YYYY-MM-DD hh:mm:ss
Parameters     : analyze --target mlc --device lsm6dsv16x --tree dectree.txt --arff features.arff --meta meta.txt

Exec/report summary (analyze)
--------------------------------------------------------------------------------
target/series  : mlc
device name    : LSM6DSV16X
decision tree  : /path/to/workspace/dectree.txt
arff features  : /path/to/workspace/features.arff
metaclassifier : /path/to/workspace/meta.txt
output dir     : /path/to/workspace/mlc_ai_output
--------------------------------------------------------------------------------

Running the simulation...

Features used:
    F1_MEAN_ACC_V2
    F5_ENERGY_ACC_V2
    F6_ENERGY_ACC_V2_FILTER_1
    F9_ZERO_CROSSING_ACC_V2
    F11_PEAK_DETECTOR_ACC_V2
    F12_PEAK_DETECTOR_ACC_V2_FILTER_1

Decision tree info:
    Nodes  : 17
    Leaves : 9
    Depth  : 6

Results before meta-classifier:
    Class "stationary" accuracy  : 98.848053 %
                       recall    : 98.504272 %
                       precision : 97.052635 %
    Class "walking"    accuracy  : 99.012619 %
                       recall    : 98.933899 %
                       precision : 97.274628 %
    Class "cycling"    accuracy  : 99.122330 %
                       recall    : 96.868011 %
                       precision : 99.540230 %
    Class "running"    accuracy  : 99.725731 %
                       recall    : 99.088837 %
                       precision : 99.770645 %

    Global accuracy : 98.354362 %

              stationary (T)    walking (T)    cycling (T)    running (T)
stationary    461               4              10             0          
walking       5                 464            4              4          
cycling       2                 0              433            0          
running       0                 1              0              435        

Results after meta-classifier:
    Class "stationary" accuracy  : 99.780342 %
                       recall    : 100.000000 %
                       precision : 99.148933 %
    Class "walking"    accuracy  : 99.615601 %
                       recall    : 99.360344 %
                       precision : 99.148933 %
    Class "cycling"    accuracy  : 99.615601 %
                       recall    : 98.434006 %
                       precision : 100.000000 %
    Class "running"    accuracy  : 99.450851 %
                       recall    : 99.088837 %
                       precision : 98.639458 %

    Global accuracy : 99.231186 %

              stationary (T)    walking (T)    cycling (T)    running (T)
stationary    466               3              1              0          
walking       0                 466            0              4          
cycling       0                 0              440            0          
running       0                 0              6              435

Validation on target

Alternatively, ST Edge AI Core also supports validation on target using ProfiMEMS Evaluation Board and DIL-24 adapters with the MLC-based sensor of choice, e.g., DIL-24 adapter for LSM6DSV16X sensor.

Before using the ProfiMEMS board for MLC validation, the user must flash the correct firmware and insert the sensor adapter into the DIL-24 slot. For these steps, please refer to the ProfiMEMS board user manual By connecting the board via USB to a PC, the validate command can be used for loading the .ucf configuration and inject any data log inside the sensor to obtain MLC predictions:

$ stedgeai validate --target mlc --device lsm6dsv16x --ucf mlc_conf.ucf --logs testset.csv --ignore-zero

Again, in addition to generating raw MLC prediction logs, a report summarizing the overall performance is also generated:

ST Edge AI Core v1.0.0-A1 (MLC 1.1.0)
Created date  : YYYY-MM-DD hh:mm:ss
Parameters    : validate --target mlc --device lsm6dsv16x --ucf mlc_conf.ucf --logs testset.csv --ignore-zero

Exec/report summary (validate)
--------------------------------------------------------------------------------
target/series : mlc
device name   : LSM6DSV16X
ucf file      : /path/to/workspace/mlc_conf.ucf
output dir    : /path/to/workspace/mlc_ai_output
logs dir/file : /path/to/workspace/testset.csv
--------------------------------------------------------------------------------

Computing the metrics...
Accuracy report for decision tree 0
--------------------------------------------------------------------------------
notes: - computed against the provided ground truth values
       - 5880 samples (4 items per sample)
       - C0 may also indicates the initial state of DT0

acc: 100.00%, rmse: 0.0, mae: 0.0, l2r: 0.0, mean: 0.0, std: 0.0, nse: 1.0, cos: 1.0

4 classes (5880 samples)
----------------------------
C1       1620   .    .    .  
C4         .  1440   .    .  
C8         .    .  1560   .  
C12        .    .    .  1260

acc  : Classification accuracy (all classes)
rmse : Root Mean Squared Error
mae  : Mean Absolute Error
l2r  : L2 relative error
nse  : Nash-Sutcliffe efficiency criteria
cos  : Cosine similarity

Command

Type of command to execute, must be specified first. Available commands for MLC target:

Common options

Specific generate options

Specific validate options

Specific analyze options

Additional options

Examples

$ stedgeai analyze  --target mlc --device lsm6dsv16x --tree mytree.txt --arff myfeatures.arff

$ stedgeai validate --target mlc --device lsm6dsv16x --ucf myconf.ucf --logs test_data.csv

$ stedgeai generate --target mlc --device lsm6dsv16x --json myscript.json --type arff

$ stedgeai generate --target mlc --device lsm6dsv16x --json myscript.json --type ucf -o output_dir