ST Edge AI Core for MLC

Feature extraction

To ensure a correct model training, the features comprising the training set must be aligned 1:1 to the MLC hardware. For this reason, ST Edge AI Core enables an extra generation step for automatically extracting the features from data logs acquired using ST MEMS software tools (for example, MEMS Studio, Unico GUI, Unicleo GUI).

This is done by providing a MLC device settings file containing the desired device settings needed for extracting the features (ODR, full-scale, etc.). An example can be found below:

{
    "datalogs": [
        { "filename": "data/stationary.csv", "label": "stationary" },
        { "filename": "data/walking.csv",    "label": "walking"    },
        { "filename": "data/cycling.csv",    "label": "cycling"    },
        { "filename": "data/running.csv",    "label": "running"    },
        "..."
    ],
    "name": "LSM6DSV16X",
    "mlc_odr": "30 Hz",
    "input_type": "accelerometer_only",
    "accelerometer_fs": "8 g",
    "accelerometer_odr": "30 Hz",
    "decision_tree_count": 1,
    "window_length": 60,
    "filters": [
        { "filter_id": "filter_1", "filter_type": "BP", "input": "Acc_V2", "a2": "-1.66", "a3": "0.81", "gain": "0.09" },
        "..."
    ],
    "features": [
        { "feature_name": "MEAN", "input": "Acc_V2",         },
        { "feature_name": "MEAN", "input": "Acc_V2_filter_1" },
        "..."
    ],
    "features_output_filename": "features.arff"
}

For more information please follow the link to the dedicated article: MLC device settings.

By providing the settings to the generate command, ST Edge AI Core will extract all the features in the .arff file format:

$ stedgeai generate --target mlc --device lsm6dsv16x --settings mlc_settings.json --gen-type features

Model training

After extracting the features, the data can now be loaded inside the ML framework of choice (Python, MATLAB, Weka, etc.) for model training. The following snippet of code shows how this can be achieved using Python:

# Import dependencies
import numpy as np
import pandas as pd
from scipy.io import arff
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier, export_text

# Load labels and features from ARFF
arff_data = arff.loadarff("features.arff")
df = pd.DataFrame(arff_data[0])
X = df.drop("class", axis=1).values.astype(np.float32)
y = np.array([LABELS.index(label.decode()) for label in df["class"]])
feature_names = list(df.drop("class", axis=1).columns)

# Split dataset into train set and test set
X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y, test_size=0.3)

# Train decision tree classifer
clf = DecisionTreeClassifier(
    max_depth=128,
    criterion="entropy",
    min_impurity_decrease=1e-3,
    ccp_alpha=1e-2,
    class_weight="balanced",
)
clf.fit(X_train, y_train)

# Define utility function for weka format conversion
def convert_sklearn_to_weka(clf, feature_names):
    dt = re.sub(r"\|--- ", r"", export_text(clf, feature_names=feature_names)).splitlines()
    new_dt = []
    line_idx = 1
    while line_idx < len(dt):
        match = re.search(r"class: (\d+)$", dt[line_idx])
        if match is not None:
            label = LABELS[int(match.group(1))]
            new_dt.append(f"{dt[line_idx - 1]}: {label}")
            line_idx += 2
        else:
            new_dt.append(dt[line_idx - 1])
            line_idx += 1
    return "\n".join(new_dt)

# Export model to weka textual format
dectree = convert_sklearn_to_weka(clf, feature_names)
with open("dectree.txt", "w") as f:
    f.write(dectree)

After training and saving the model(s), the user should have one or more decision tree files in the Weka format. An example can be found below:

F6_ENERGY_ACC_V2_FILTER_1 <= 7.86
|   F5_ENERGY_ACC_V2 <= 62.14: stationary
|   F5_ENERGY_ACC_V2 >  62.14
|   |   F11_PEAK_DETECTOR_ACC_V2 <= 31.00
|   |   |   F5_ENERGY_ACC_V2 <= 83.12: stationary
|   |   |   F5_ENERGY_ACC_V2 >  83.12: walking
|   |   F11_PEAK_DETECTOR_ACC_V2 >  31.00
|   |   |   F11_PEAK_DETECTOR_ACC_V2 <= 37.50
|   |   |   |   F12_PEAK_DETECTOR_ACC_V2_FILTER_1 <= 17.50: stationary
|   |   |   |   F12_PEAK_DETECTOR_ACC_V2_FILTER_1 >  17.50: cycling
|   |   |   F11_PEAK_DETECTOR_ACC_V2 >  37.50: cycling
F6_ENERGY_ACC_V2_FILTER_1 >  7.86
|   F1_MEAN_ACC_V2 <= 2.92
|   |   F9_ZERO_CROSSING_ACC_V2 <= 43.00: walking
|   |   F9_ZERO_CROSSING_ACC_V2 >  43.00: cycling
|   F1_MEAN_ACC_V2 >  2.92: running

Sensor configuration

A second, and final, generation step will now use the same MLC device settings file as before with additional fields to integrate one or more decision trees in the final sensor configuration. An example can be found below:

{
    "dectree_features": [
        "F1_MEAN_ACC_V2",
        "F2_MEAN_ACC_V2_FILTER_1",
        "..."
    ],
    "dectrees": [
        {
            "filename": "dectree.txt",
            "results": [
                { "label": "stationary", "value": 1  },
                { "label": "walking",    "value": 4  },
                { "label": "cycling",    "value": 8  },
                { "label": "running",    "value": 12 },
            ],
            "metaclassifier": [ 2, 2, 2, 2 ]
        },
        "..."
    ],
    "config_output_filename": "mlc_conf.json"
}

For more information please follow the link to the dedicated article: MLC device settings.

By running the generate command again, now with the updated settings, we can generate the sensor configuration in the .json file format:

$ stedgeai generate --target mlc --device lsm6dsv16x --settings mlc_settings.json --gen-type config

A C header file and a minimal header file (.h) containing the same configuration are also generated in the output folder to allow for the integration inside the firmware of the MCU/MPU used for setting up the sensor at runtime.

Performance analysis

By using the analyze command, ST Edge AI Core can execute the MLC logic offline by inputting features (extracted previously using the generate command) inside a given decision tree with an optional metaclassifier stage to obtain the final predictions:

$ stedgeai analyze --target mlc --device lsm6dsv16x --tree dectree.txt --arff features.arff --meta meta.txt

where meta.txt configures the metaclassifier stage by specifying rows with: class name, metaclassifier end counter, and metaclassifier subgroup. For example:

stationary 2 0
walking 2 1
running 2 2
cycling 2 3

For more information on the metaclassifier please refer to the MLC application note of the specific device, for example, the MLC application note for LSM6DSV16X sensor.

In addition to computing raw MLC prediction logs, a report summarizing decision tree information and the overall model performance is also generated:

ST Edge AI Core v2.1.0 (MLC 1.2.0)
Created date   : YYYY-MM-DD hh:mm:ss
Parameters     : analyze --target mlc --device lsm6dsv16x --tree dectree.txt --arff features.arff --meta meta.txt

Exec/report summary (analyze)
--------------------------------------------------------------------------------
target/series  : mlc
device name    : LSM6DSV16X
decision tree  : /path/to/workspace/dectree.txt
arff features  : /path/to/workspace/features.arff
metaclassifier : /path/to/workspace/meta.txt
output dir     : /path/to/workspace/mlc_ai_output
--------------------------------------------------------------------------------

Running the simulation...

Features used:
    F1_MEAN_ACC_V2
    F5_ENERGY_ACC_V2
    F6_ENERGY_ACC_V2_FILTER_1
    F9_ZERO_CROSSING_ACC_V2
    F11_PEAK_DETECTOR_ACC_V2
    F12_PEAK_DETECTOR_ACC_V2_FILTER_1

Decision tree info:
    Nodes  : 17
    Leaves : 9
    Depth  : 6

Results before meta-classifier:
    Class "stationary" accuracy  : 98.848053 %
                       recall    : 98.504272 %
                       precision : 97.052635 %
    Class "walking"    accuracy  : 99.012619 %
                       recall    : 98.933899 %
                       precision : 97.274628 %
    Class "cycling"    accuracy  : 99.122330 %
                       recall    : 96.868011 %
                       precision : 99.540230 %
    Class "running"    accuracy  : 99.725731 %
                       recall    : 99.088837 %
                       precision : 99.770645 %

    Global accuracy : 98.354362 %

              stationary (T)    walking (T)    cycling (T)    running (T)
stationary    461               4              10             0          
walking       5                 464            4              4          
cycling       2                 0              433            0          
running       0                 1              0              435        

Results after meta-classifier:
    Class "stationary" accuracy  : 99.780342 %
                       recall    : 100.000000 %
                       precision : 99.148933 %
    Class "walking"    accuracy  : 99.615601 %
                       recall    : 99.360344 %
                       precision : 99.148933 %
    Class "cycling"    accuracy  : 99.615601 %
                       recall    : 98.434006 %
                       precision : 100.000000 %
    Class "running"    accuracy  : 99.450851 %
                       recall    : 99.088837 %
                       precision : 98.639458 %

    Global accuracy : 99.231186 %

              stationary (T)    walking (T)    cycling (T)    running (T)
stationary    466               3              1              0          
walking       0                 466            0              4          
cycling       0                 0              440            0          
running       0                 0              6              435

Validation on target

ST Edge AI Core also supports validation on target using the ProfiMEMS Evaluation Board (STEVAL-MKI109D or STEVAL-MKI109V3) and DIL24 adapters with the MLC-compatible sensor of choice, for example, the DIL24 adapter for LSM6DSV16X sensor.

Before using the ProfiMEMS board for MLC validation, the user must flash the firmware provided with MEMS Studio and insert the sensor adapter into the DIL24 slot. For flashing the board please follow the steps descibed in the user manual of MEMS Studio and/or of the ProfiMEMS board.

By connecting the board via USB to a PC, the validate command can be used for loading the sensor configuration and inject any data log inside the sensor to obtain MLC predictions:

$ stedgeai validate --target mlc --device lsm6dsv16x --mlc-conf mlc_conf.json --logs testset.csv --ignore-zero

Again, in addition to generating raw MLC prediction logs, a report summarizing the overall performance is also generated:

ST Edge AI Core v2.1.0 (MLC 1.2.0)
Created date  : YYYY-MM-DD hh:mm:ss
Parameters    : validate --target mlc --device lsm6dsv16x --mlc-conf mlc_conf.json --logs testset.csv --ignore-zero

Exec/report summary (validate)
--------------------------------------------------------------------------------
target/series : mlc
device name   : LSM6DSV16X
config file   : /path/to/workspace/mlc_conf.json
output dir    : /path/to/workspace/mlc_ai_output
logs dir/file : /path/to/workspace/testset.csv
--------------------------------------------------------------------------------

Computing the metrics...
Accuracy report for decision tree 0
--------------------------------------------------------------------------------
notes: - computed against the provided ground truth values
       - 5880 samples (4 items per sample)
       - C0 may also indicates the initial state of DT0

acc: 100.00%, rmse: 0.0, mae: 0.0, l2r: 0.0, mean: 0.0, std: 0.0, nse: 1.0, cos: 1.0

4 classes (5880 samples)
----------------------------
C1       1620   .    .    .  
C4         .  1440   .    .  
C8         .    .  1560   .  
C12        .    .    .  1260

acc  : Classification accuracy (all classes)
rmse : Root Mean Squared Error
mae  : Mean Absolute Error
l2r  : L2 relative error
nse  : Nash-Sutcliffe efficiency criteria
cos  : Cosine similarity

Command

Type of command to execute, must be specified first. Available commands for MLC target:

Common options

Specific generate options

Specific validate options

Specific analyze options

Additional options

Examples

$ st_ai.py analyze  --target mlc --device lsm6dsv16x --tree dectree.txt --arff features.arff

$ st_ai.py validate --target mlc --device lsm6dsv16x --mlc-conf mlc_config.json --logs test_data.csv

$ st_ai.py generate --target mlc --device lsm6dsv16x --settings mlc_settings.json --gen-type features

$ st_ai.py generate --target mlc --device lsm6dsv16x --settings mlc_settings.json --gen-type config -o output_dir