Summary table

Following table contains the list of the operators that can be imported, if the constraints or limitations are met.

• supported optional fused activation (or non-linearity): gelu, linear, relu, quantized_relu, relu_n1_to_1, relu_0_to_1, leaky_relu, relu6, elu, selu, sigmoid, hard_sigmoid, hard_swish, exponential, tanh, softmax, softplus, softsign, abs, acos, acosh, asin, asinh, atan, atanh, ceil, clip, cos, cosh, erf, flexerf, exp, floor, identity, log, logistic, neg, logical_not, prelu, probit, reciprocal, relu_generic, relu_thresholded, round, sign, sin, sinh, softmax_zero, sqrt, swish, tan

Note: The "Can be HW-assisted" column indicates whether the operator can leverage hardware acceleration. Activations and weights should be quantized (integer scale/offset static scheme).

Common constraints

• input and output tensors must be not dynamic.
  • variable-length batch dimension (i.e. (None,)) is considered as equal to 1
    
  • must not be greater than 6D
    
  • dimension must be in the range [0, 65536[
    
  • batch dimension is partially supported in the axis/axes parameters
    
• data type for the weights/activations tensors must be:
  • float32, int8, uint8
    
  • only int32 for the bias tensor is considered
    
  • for some operators, bool and binary types are also supported
    
• 1D operator is mapped on the respective 2D operator by adding a singleton dimension on the input: (12,3) -> (12, 1, 3)

Abs

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Acos

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Acosh

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Add

Performs element-wise operation

• category: eltwise operator
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

And

Performs boolean element-wise operation

• category: eltwise operator
  
• can be hardware assisted : Neural-ART
  
• input data types: bool
  
• output data types: bool

ArgMax

Computes the indices of the max elements of the input tensor’s element along the provided axis.

• category: generic layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: int32

ArgMin

Computes the indices of the min elements of the input tensor’s element along the provided axis.

• category: generic layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: int32

ArrayFeatureExtractor

Select elements of the input tensor besed of the indices passed

• category: onnx.ml
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• used to support for example the skit-learn algo:
  • Nu-Support Vector Classification (sklearn.svm.NuSVC)
    
  • C-Support Vector Classification. (sklearn.svm.SVC)
    
  • A random forest classifier (sklearn.ensemble.RandomForestClassifier)
    
  • …

Asin

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Asinh

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Atan

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Atanh

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

AveragePool

Downsamples the input

• category: pooling layer
  
• can be hardware assisted : Neural-ART, HSP (w8a8)
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• arbitrary strides, provided that they are smaller than the input size
  
• arbitrary pool sizes, provided that they are smaller than the input size
  
• The attribute dilations different from the default value is not supported
  
• The attribute ceil_mode is not supported when it makes the last sliding window start beyond the input and the padding at the beginning of a spatial dimension

AveragePoolGrad

Computes the AveragePool input gradient by redistributing downstream gradient according to AveragePool kernel, stride, and padding semantics. Supports ONNX AveragePoolGrad.

• category: average pooling gradient layer
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• Input: gradient w.r.t. AveragePool output.
  
• Output: gradient w.r.t. AveragePool input.
  
• Supports 1D/2D AveragePool attributes with ONNX defaults.
  
• count_include_pad controls the averaging denominator semantics.

BatchNormalization

Performs the normalization of the input

• category: normalization layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• Only one output (Y) is supported

BitShift

Performs element-wise operation

• category: eltwise operator
  
• input data types: float32
  
• output data types: float32

Cast

Cast elements of the input tensor to the specified output tensor data

• category: conversion layer
  
• can be hardware assisted : Neural-ART
  
• input data types: bool, int8, uint8, float32
  
• output data types: bool, int8, uint8, float32

Specific constraints/recommendations:

• The attribute saturate different from the default value is not supported
  
• The attribute round_mode different from the default value is not supported

Ceil

Applies an activation function to the input tensor

• category: activation layer
  
• input data types: float32
  
• output data types: float32

Clip

Applies an activation function to the input tensor

• category: activation layer
  
• input data types: float32
  
• output data types: float32

Concat

Performs concatenation of a list of inputs

• category: merge operator
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• concatenating on the batch dimension is not supported

ConcatTraining

Performs concatenation of a list of inputs

• category: merge operator
  
• input data types: float32
  
• output data types: float32

Constant

• input data types: float32
  
• output data types: float32

ConstantOfShape

Generates a tensor with given value and shape

• category: constant layer
  
• input data types: float32
  
• output data types: float32

Conv

Performs convolution operation

• category: convolutional layer
  
• can be hardware assisted : Neural-ART, HSP (w8a8)
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• arbitrary strides, provided that they are smaller than the input size
  
• arbitrary filter kernel sizes, provided that they are smaller than the input size
  
• input spatial dimensions must be smaller than 65536 for quantized layers

ConvGrad

Downsamples the input

• category: Conv2DGrad layer
  
• input data types: float32
  
• output data types: float32

ConvTranspose

Transposed convolutional layer

• category: convolutional layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• arbitrary strides, provided that they are smaller than the input size
  
• arbitrary filter kernel sizes, provided that they are smaller than the input size

Cos

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Cosh

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

DepthToSpace

Permutes the dimensions of the input according to a given pattern

• category: reshaping layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32, int8, uint8
  
• output data types: float32, int8, uint8

Specific constraints/recommendations:

• transposing the batch dimension is not supported

DequantizeLinear

Computes element-wise data conversion low precision to full precision, based on the scale/zeropoint parameters

• category: conversion layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32, int8, uint8
  
• output data types: float32, int8, uint8

Specific constraints/recommendations:

• The attribute block_size different from the default value is not supported
  
• The attribute output_dtype different from the default value is not supported
  
• Blocked quantization, requested through a scale whose rank matches the rank of the input, is not supported

Div

Performs element-wise operation

• category: eltwise operator
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Dropout

Applies Dropout to the input

• category: regularization layers
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• operator is dropped during the conversion

Einsum

Einsum (Einstein Summation) operator

• category: einsum operator
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• diagonal is not supported

Elu

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Equal

Performs logical element-wise operation

• category: eltwise operator
  
• input data types: float32
  
• output data types: bool

Erf

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Exp

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Expand

Constructs a tensor by tiling the input tensor

• category: reshaping layer
  
• input data types: float32, int8, uint8
  
• output data types: float32, int8, uint8

Specific constraints/recommendations:

• tiling on batch-dimension is not supported

Flatten

Flattens the non-batch input dimensions to a vector

• category: Reshaping operation
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• Flatten on the batch dimension is not supported
  
• operator is dropped during the conversion

Floor

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Gather

Gathers values along a specified axis

• category: activation function
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• Gather is not supported with indices dimensions > 2 (Batch is not considered), axis > 3and axis = 0, batch_dims attribute is not handled

GatherElements

Gathers Elements along a specified axis

• category: activation function
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• GatherElements is not supported with indices dimensions > 2 (Batch is not considered), axis > 3and axis = 0, batch_dims attribute is not handled

GatherND

Gathers slices from input tensor into an output tensor with shape specified by indices

• category: activation function
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• Batch_dims attribute is not handled. 4D or more indices (BATCH is not included) are not implementedLast dimension of indices > 4 with 2D inputs tensor case is not handled

Gelu

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Gemm

General Matrix multiplication: Compute Y = alphaAB + beta*C

• category: core layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

GlobalAveragePool

Downsamples the input

• category: pooling layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

GlobalMaxPool

Downsamples the input

• category: pooling layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Greater

Performs logical element-wise operation

• category: eltwise operator
  
• input data types: float32
  
• output data types: bool

GreaterOrEqual

Performs logical element-wise operation

• category: eltwise operator
  
• input data types: float32
  
• output data types: bool

GRU

Gated Recurrent Unit Not supported by the Neural-ART NPU, and the Neural-ART compiler. For float GRU, a float16 GRU kernel is available and can be used by passing the flag –use-f16 to the stedgeai generate command.

• category: recurrent layer
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• Supports float32, w8a8 (full int8) QDQ For w8a8 GRU, dequantization at runtime is performed, and a float kernel is used. What this means, is that the weights of the GRU are stored in int8 in ROM and they are dequantized to float32 at runtime before being fed to a floating point GRU kernel. The activations are also dequantized. This emulates the behaviour of onnxruntime QDQ. You can use a float16 kernel instead of float32 by passing the flag –use-f16 to the stedgeai generate command. Non-constant initial state is supported. Only a batch size of 1 is supported.

• layout=1 attribute is not supported Non-empty sequence_lens is not supported.
  

• Fused activation: gelu, linear, relu, quantized_relu, relu_n1_to_1, relu_0_to_1, leaky_relu, relu6, elu, selu, sigmoid, hard_sigmoid, hard_swish, exponential, tanh, softmax, softplus, softsign
  

• Fused recurrent activation: gelu, linear, relu, quantized_relu, relu_n1_to_1, relu_0_to_1, leaky_relu, relu6, elu, selu, sigmoid, hard_sigmoid, hard_swish, exponential, tanh, softmax, softplus, softsign

Hardmax

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• The value 1 is supported as the default value of the axis attribute, not the value -1

HardSigmoid

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• It is supported only for 1D tensor and only on the channel dimension

HardSwish

Applies an activation function to the input tensor

• category: activation layer
  
• input data types: float32
  
• output data types: float32

Identity

This layer should be used as a placeholder when no operation is to be performed.The layer just returns its inputs argument as output.

• category: Identity layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

InPlaceAccumulatorV2

Performs an in-place accumulation (or overwrite) of a tensor into an accumulator buffer. This operator is typically used to gather/accumulate gradients or partial results in-place and exposes a control flag to select overwrite vs accumulate behavior.

• category: utility / inplace operator
  
• input data types: float32, float32, bool
  
• output data types: bool, float32

Specific constraints/recommendations:

• Input 0: accumulator buffer (same shape as Input 1).
  
• Input 1: tensor to accumulate into the buffer (same dtype/shape as Input 0).
  
• Input 2: scalar boolean overwrite flag; when true the buffer is overwritten with Input 1, otherwise Input 1 is added element-wise to the buffer.
  
• Output 0: boolean success/status (single-element bool).
  
• Output 1: updated accumulator buffer (same shape as Input 0).
  
• All numeric tensors are expected to be float32 for the C implementation provided; integer/quantized formats are not supported.
  
• Shapes of Input 0 and Input 1 must match. Input 2 is expected to be a scalar boolean.
  
• This is an ONNX operator and not present in standard Keras/TensorFlow.

InstanceNormalization

Apply instance normalization

• category: normalization function
  
• input data types: float32
  
• output data types: float32

LabelEncoder

Maps each element in the input tensor to another value.

• category: LabelEncoder
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• Warning: This operator is not supported for C code generation, it is only used during iForest generation. Support is not complete.
  
• Attributes keys_floats, keys_strings, values_int64s, values_strings are not supported

LayerNormalization

Apply LayerNormalization along the provided axis

• category: normalization function
  
• input data types: float32
  
• output data types: float32

LeakyRelu

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

LeakyReluGrad

Computes the gradient of the LeakyReLU activation function. Supports ONNX LeakyReluGrad operator.

• category: activation gradient layer
  
• input data types: float32, float32
  
• output data types: float32

Specific constraints/recommendations:

• Inputs: [gradient w.r.t. output, activation input].
  
• Output: gradient w.r.t. activation input.
  
• Input and output shapes must match.
  
• Default alpha is 0.01 when the attribute is not set.
  
• Used in backward passes for LeakyReLU.

Less

Performs logical element-wise operation

• category: eltwise operator
  
• input data types: float32
  
• output data types: bool

LessOrEqual

Performs logical element-wise operation

• category: eltwise operator
  
• input data types: float32
  
• output data types: bool

LinearClassifier

LinearClassifier layer

• category: onnx.ml
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• used to support for example the skit-learn algo:
  • Linear Support Vector Classification. (sklearn.svm.LinearSVC)
    
  • Logistic Regression (aka logit, MaxEnt) classifier. (sklearn.linear_model.LogisticRegression)
    
  • Classifier using Ridge regression. (sklearn.linear_model.RidgeClassifier)
    
  • Linear Discriminant Analysis. (sklearn.discriminant_analysis.LinearDiscriminantAnalysis)

LinearRegressor

General Matrix multiplication: Compute Y = alphaAB + beta*C

• category: core layer
  
• input data types: float32
  
• output data types: float32

Log

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

LogSoftmax

Applies an activation function to the input tensor

• category: activation layer
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• It is supported only for 1D tensor and only on the channel dimension
  
• The value 1 is supported as the default value of the axis attribute, not the value -1

LpNormalization

Apply Lp-normalization along the provided axis

• category: normalization function
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• order of the normalization (1 and 2 supported)

LRN

Apply Local Response Normalization over local input regions

• category: normalization function
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

LSTM

Applies a long short-term memory (LSTM) layer to an input sequence (batch=1, timesteps, features). Not supported by the Neural-ART NPU, and the Neural-ART compiler. For float LSTM, a float16 LSTM kernel is available and can be used by passing the flag –use-f16 to the stedgeai generate command.

• category: recurrent layer
  
• input data types: float32, int8, int16
  
• output data types: float32, int8, int16

Specific constraints/recommendations:

• Supports float32, w8a8 (full int8) and w8a16 (int8 weights, int16 activations) QDQ. For w8a8 and w8a16 LSTM, dequantization at runtime is performed, and a float kernel is used. What this means, is that the weights of the LSTM are stored in int8 in ROM and they are dequantized to float32 at runtime before being fed to a floating point LSTM kernel. The activations are also dequantized. This emulates the behaviour of onnxruntime QDQ. You can use a float16 kernel instead of float32 by passing the flag –use-f16 to the stedgeai generate command. Non-constant initial states are supported. Only a batch size of 1 is supported.
  
• layout=1 attribute is not supported. Non-empty sequence_lens is not supported.
  
• Fused activation: gelu, linear, relu, quantized_relu, relu_n1_to_1, relu_0_to_1, leaky_relu, relu6, elu, selu, sigmoid, hard_sigmoid, hard_swish, exponential, tanh, softmax, softplus, softsign
  
• Fused recurrent activation: gelu, linear, relu, quantized_relu, relu_n1_to_1, relu_0_to_1, leaky_relu, relu6, elu, selu, sigmoid, hard_sigmoid, hard_swish, exponential, tanh, softmax, softplus, softsign

MatMul

General Matrix multiplication: Compute Y = alphaAB + beta*C

• category: core layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• Only up to 3D matrix multiplication is supported

Max

Computes the maximum (element-wise) a list of inputs

• category: eltwise operator
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

MaxPool

Downsamples the input

• category: pooling layer
  
• can be hardware assisted : Neural-ART, HSP (w8a8)
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• arbitrary strides, provided that they are smaller than the input size
  
• arbitrary pool sizes, provided that they are smaller than the input size
  
• The attribute dilations different from the default value is not supported
  
• The attribute storage_order different from the default value is not supported
  
• The attribute ceil_mode is not supported when it makes the last sliding window start beyond the input and the padding at the beginning of a spatial dimension

MaxPoolGrad

Computes the MaxPool input gradient using downstream gradient and pooling indices. Supports ONNX MaxPoolGrad.

• category: max pooling gradient layer
  
• input data types: float32, int32
  
• output data types: float32

Specific constraints/recommendations:

• Inputs: [gradient w.r.t. MaxPool output, MaxPool indices].
  
• Output: gradient w.r.t. MaxPool input.
  
• When indices are provided, overlapping windows can accumulate into the same input position.
  
• Supports 1D/2D MaxPool attributes with ONNX defaults.

Mean

Performs element-wise operation

• category: eltwise operator
  
• input data types: float32
  
• output data types: float32

Min

Computes the minimum (element-wise) a list of inputs

• category: eltwise operator
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Mod

Performs element-wise operation

• category: eltwise operator
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Mul

Performs element-wise operation

• category: eltwise operator
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Neg

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Normalizer

Normalize the input.

• category: onnx.ml
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• used to support for example the skit-learn algo:
  • Logistic Regression (aka logit, MaxEnt) classifier. (sklearn.linear_model.LogisticRegression)
    
  • Normalize samples individually to unit norm. (sklearn.preprocessing.Normalizer)
    
• Note that the implementation of the max norm is aligned with sklearn implementation, not the onnx implementation. The difference is that sklearn is using absolute values to get a positive divisor when computing the norm. Onnx implementation does not use absolute values.

Not

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Or

Performs boolean element-wise operation

• category: eltwise operator
  
• can be hardware assisted : Neural-ART
  
• input data types: bool
  
• output data types: bool

Pad

Pads an input tensor

• category: Reshaping layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• Axes are not supported for Pad operator
  
• wrap value for mode attribute is not supported

Pow

Performs element-wise operation

• category: eltwise operator
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

PRelu

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• Only 1 input is allowed for PRelu; the slope should be constant and placed into the initializers

QLinearAdd

Performs element-wise binary addition on 8 bit data types(with Numpy-style broadcasting support).

• category: eltwise operator
  
• can be hardware assisted : Neural-ART
  
• input data types: uint8, int8
  
• output data types: uint8, int8

QLinearAveragePool

Downsamples an input tensor X and applies average pooling across the tensor according to kernel sizes, stride sizes, and pad lengths

• category: pooling layer
  
• can be hardware assisted : Neural-ART
  
• input data types: uint8, int8
  
• output data types: uint8, int8

Specific constraints/recommendations:

• arbitrary strides, provided that they are smaller than the input size
  
• arbitrary pool sizes, provided that they are smaller than the input size
  
• The attribute ceil_mode is not supported when it makes the last sliding window start beyond the input and the padding at the beginning of a spatial dimension

QLinearConcat

Concatenate a list of tensors into a single tensor

• category: merge operator
  
• can be hardware assisted : Neural-ART
  
• input data types: uint8, int8
  
• output data types: uint8, int8

Specific constraints/recommendations:

• concatenating on the batch dimension is not supported

QLinearConv

Consumes a quantized input tensor

• category: convolutional layer
  
• can be hardware assisted : Neural-ART
  
• input data types: int8, uint8
  
• output data types: int8, uint8

Specific constraints/recommendations:

• arbitrary strides, provided that they are smaller than the input size
  
• arbitrary filter kernel sizes, provided that they are smaller than the input size
  
• input spatial dimensions must be smaller than 65536 for quantized layers

QLinearGlobalAveragePool

Downsamples an input tensor X and applies Average pooling acrossthe values in the same channel

• category: pooling layer
  
• can be hardware assisted : Neural-ART
  
• input data types: uint8, int8
  
• output data types: uint8, int8

QLinearMatMul

Matrix product consuming two quantized input tensors

• category: core layer
  
• can be hardware assisted : Neural-ART
  
• input data types: int8, uint8
  
• output data types: int8, uint8

QLinearMul

Performs element-wise binary multiplication on 8 bit data types(with Numpy-style broadcasting support).

• category: eltwise operator
  
• input data types: uint8, int8
  
• output data types: uint8, int8

QuantizeLinear

Computes element-wise data conversion full precision to low precision, based on the scale/zeropoint parameters

• category: conversion layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32, int8, uint8
  
• output data types: float32, int8, uint8

Specific constraints/recommendations:

• The attribute block_size different from the default value is not supported
  
• The attribute output_dtype different from the default value is not supported
  
• Blocked quantization, requested through a scale whose rank matches the rank of the input, is not supported
  
• The attribute saturate different from the default value is not supported
  
• The attribute precision different from the default value is not supported

Range

Generate a tensor containing a sequence of numbers that begin at start and extends by increments of delta up to limit (exclusive).

• category: generic layer
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• The attribute stash_type different from the default value is not supported

Reciprocal

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

ReduceL1

• category: reduction operation
  
• input data types: float32
  
• output data types: float32

ReduceL2

• category: reduction operation
  
• input data types: float32
  
• output data types: float32

ReduceLogSumExp

• category: reduction operation
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

ReduceMax

Computes the Max of the input tensor’s element along the provided axes

• category: reduction operation
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

ReduceMean

Computes the Mean of the input tensor’s element along the provided axes

• category: reduction operation
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

ReduceMin

Computes the Min of the input tensor’s element along the provided axes

• category: reduction operation
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

ReduceProd

Computes the Product of the input tensor’s element along the provided axes

• category: reduction operation
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

ReduceSum

Computes the Sum of the input tensor’s element along the provided axes

• category: reduction operation
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

ReduceSumSquare

Computes the Sum Square of the input tensor’s element along the provided axes

• category: reduction operation
  
• input data types: float32
  
• output data types: float32

Relu

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

ReluGrad

Computes the gradient of the ReLU activation function. Supports ONNX ReluGrad operator.

• category: activation gradient layer
  
• input data types: float32, float32
  
• output data types: float32

Specific constraints/recommendations:

• Inputs: [gradient w.r.t. output, activation input].
  
• Output: gradient w.r.t. activation input.
  
• Input and output shapes must match.
  
• Used in backward passes for ReLU.

Reshape

Reshapes a tensor

• category: Reshaping operation
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• The attribute allowzero different from the default value is not supported

Resize

Resize the input tensor: it calculates every value in the output tensor as a weighted average of neighborhood

• category: resizing operation
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• Cubic resize is not supported
  
• half_pixel_symmetric value for coordinate_tansformation_mode is not supported
  
• The attribute antialias different from the default value is not supported
  
• axes attribute is not supported
  
• The attribute keep_aspect_ratio_policy different from the default value is not supported

RNN

Fully-connected RNN where the output is to be fed back to input, Only batch=1 is supported.

• category: recurrent layer
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• stateless support only
  
• intial_h not supported
  
• return_state not supported

Round

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Scale

Performs element-wise operation

• category: eltwise operator
  
• input data types: float32
  
• output data types: float32

Scaler

Compute the Scale and Offset of input tensor

• category: onnx.ml
  
• input data types: float32
  
• output data types: float

Specific constraints/recommendations:

• used to support for example the skit-learn algo:
  • Gradient Boosting for classification. (sklearn.ensemble.GradientBoostingClassifier)
    
  • Transform features by scaling each feature to a given range. (sklearn.preprocessing.MinMaxScaler)
    
  • Standardize features by removing the mean and scaling to unit variance. (sklearn.preprocessing.StandardScaler)
    
  • Scale features using statistics that are robust to outliers. (sklearn.preprocessing.RobustScaler)
    
  • Scale each feature by its maximum absolute value. (sklearn.preprocessing.MaxAbsScaler)

ScatterND

Scatters updates into a tensor of shape equal to shape attribute according indices (TFLite),The output of the ScatterND layer is produced by creating a copy of the input data, and then updating its values to values specified by updates at specific index positions specified by indices (ONNX)

• category: activation function
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• 5D or more indices (BATCH is not included) are not implemented, 5D or more data (BATCH is not included) are not implemented

Selu

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Shape

Returns a tensor containing the shape of the input tensor

• category: Reshaping operation
  
• can be hardware assisted : Neural-ART
  
• input data types: float32, int8, uint8
  
• output data types: int32

Specific constraints/recommendations:

• Shape layer is not supported as the only layer in a network;Shape layer only supported with know shapes in input; when batch size is undetermined, it is set to 1
  
• end attribute is not supported
  
• The attribute start different from the default value is not supported

Sigmoid

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

SigmoidGrad

Computes the gradient of the Sigmoid activation function. Supports ONNX SigmoidGrad operator.

• category: activation gradient layer
  
• input data types: float32, float32
  
• output data types: float32

Specific constraints/recommendations:

• Inputs: [gradient w.r.t. output, activation output].
  
• Output: gradient w.r.t. activation input.
  
• Input and output shapes must match.
  
• Gradient formula: dX = dY * y * (1 - y).

Sign

Applies an activation function to the input tensor

• category: activation layer
  
• input data types: float32
  
• output data types: float32

Sin

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Sinh

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Size

Returns a tensor containing the number of elements of the input tensor

• category: Reshaping operation
  
• input data types: float32, int8, uint8
  
• output data types: int32

Specific constraints/recommendations:

• Size layer is not supported as the only layer in a network;Suze layer only supported with know shapes in input; when batch size is undetermined, it is set to 1
  
• end attribute is not supported
  
• The attribute start different from the default value is not supported

Slice

Crops the input

• category: reshaping layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32, int8, uint8
  
• output data types: float32, int8, uint8

Softmax

Applies an activation function to the input tensor

• category: activation layer
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• It is supported only for 1D tensor and only on the channel dimension
  
• The value 1 is supported as the default value of the axis attribute, not the value -1

SoftmaxCrossEntropyLoss

Computes the softmax cross-entropy loss between scores and integer labels. Supports ONNX SoftmaxCrossEntropyLoss operator with 2D scores input (batch, classes) and integer labels.

• category: loss layer
  
• input data types: float32, int32
  
• output data types: float32

Specific constraints/recommendations:

• Only 2D scores input (batch, classes) is supported.
  
• Labels must be integer indices (int32).
  
• Only “mean” reduction is supported; other reduction types are not.
  
• The “ignore_index” and “weight” attributes are not supported.
  
• Outputs: [loss, logit probabilities].

SoftmaxCrossEntropyLossGrad

Computes the gradient of the softmax cross-entropy loss with respect to the input scores. Supports ONNX SoftmaxCrossEntropyLossGrad operator.

• category: loss gradient layer
  
• input data types: float32, float32, int32
  
• output data types: float32

Specific constraints/recommendations:

• Inputs: [gradient w.r.t. loss, labels, scores].
  
• Only 2D scores input (batch, classes) is supported.
  
• Labels must be integer indices (int32).
  
• Only “mean” reduction is supported.
  
• The “ignore_index” and “weight” attributes are not supported.
  
• Output: gradient w.r.t. scores.

Softplus

Applies an activation function to the input tensor

• category: activation layer
  
• input data types: float32
  
• output data types: float32

Softsign

Applies an activation function to the input tensor

• category: activation layer
  
• input data types: float32
  
• output data types: float32

SpaceToDepth

Rearranges blocks of spatial data into depth

• category: reshaping layer
  
• input data types: float32, int8, uint8
  
• output data types: float32, int8, uint8

Split

Splits a tensor into a list of sub tensors

• category: split operator
  
• input data types: float32, int8, uint8
  
• output data types: float32, int8, uint8

Specific constraints/recommendations:

• Only supported if the number of splits is equal to the size of the splitting dimension

SplitTraining

Splits a tensor into a list of sub tensors

• category: split operator
  
• input data types: float32, int8, uint8
  
• output data types: float32, int8, uint8

Specific constraints/recommendations:

• Only supported if the number of splits is equal to the size of the splitting dimension

Sqrt

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Squeeze

Reshapes a tensor

• category: Reshaping operation
  
• input data types: float32
  
• output data types: float32

Sub

Performs element-wise operation

• category: eltwise operator
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Sum

Performs element-wise operation

• category: eltwise operator
  
• input data types: float32
  
• output data types: float32

SVMClassifier

SVMClassifier layer

• category: onnx.ml
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• used to support for example the skit-learn algo:
  • C-Support Vector Classification. (sklearn.svm.SVC)
    
  • Nu-Support Vector Classification (sklearn.svm.NuSVC)

SVMRegressor

Support Vector Machine regression prediction and one-class SVM anomaly detection

• category: onnx.ml
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• kernel_type argument supports one of the following ‘LINEAR’,‘POLY’,‘RBF’,‘SIGMOID’.
  
• used to support for example the skit-learn algo:
  • Unsupervised Outlier Detection. (sklearn.svm.OneClassSVM)

Tan

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Tanh

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

TanhGrad

Computes the gradient of the Tanh activation function. Supports ONNX TanhGrad operator.

• category: activation gradient layer
  
• input data types: float32, float32
  
• output data types: float32

Specific constraints/recommendations:

• Inputs: [gradient w.r.t. output, activation output].
  
• Output: gradient w.r.t. activation input.
  
• Input and output shapes must match.
  
• Gradient formula: dX = dY * (1 - tanh(x)^2).

ThresholdedRelu

Applies an activation function to the input tensor

• category: activation layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Tile

Constructs a tensor by tiling the input tensor

• category: reshaping layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32, int8, uint8
  
• output data types: float32, int8, uint8

Specific constraints/recommendations:

• tiling on batch-dimension is not supported

TopK

Retrieve the top-K largest or smallest elements along a specified axis

• category: topK operator
  
• can be hardware assisted : Neural-ART
  
• input data types: float32, int8, uint8
  
• output data types: float32, int8, uint8

Transpose

Permutes the dimensions of the input according to a given pattern

• category: reshaping layer
  
• can be hardware assisted : Neural-ART
  
• input data types: float32, int8, uint8
  
• output data types: float32, int8, uint8

Specific constraints/recommendations:

• transposing the batch dimension is not supported
  
• perm attribute must have a length equal to the rank of the input

TreeEnsembleClassifier

Tree ensemble classifier. Returns the top class for each of N inputs.

• category: onnx.ml
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• This layer supports compression levels.
  • none: disable compresion and optimization
  • lossless (default): enable optimization only
  • low, medium, high: enable optimization and weight compression
• used to support for example the skit-learn algo:
  • A random forest classifier (sklearn.ensemble.RandomForestClassifier)
    
  • An extra-trees classifier (sklearn.ensemble.ExtraTreesClassifier)
    
  • Gradient Boosting for classification. (sklearn.ensemble.GradientBoostingClassifier)
    
  • Histogram-based Gradient Boosting Classification Tree. (sklearn.ensemble.HistGradientBoostingClassifier)
    
  • A decision tree classifier. (sklearn.tree.DecisionTreeClassifier)
    
• For detailed information, see Machine Learning support (ONNX-ML operators) [ONNX-ML] article

TreeEnsembleRegressor

Tree Ensemble regressor. Returns the regressed values for each input in N.

• category: onnx.ml
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• Warning: This operator is not supported for C code generation, it is only used during iForest generation. Not all arguments are supported.
  
• used to support for example the skit-learn algo:
  • A Bagging classifier (sklearn.ensemble.BaggingClassifier)
    
  • An AdaBoost classifier (sklearn.ensemble.AdaBoostClassifier)
    
  • A Bagging regressor (sklearn.ensemble.BaggingRegressor)
    
  • A random forest regressor (sklearn.ensemble.RandomForestRegressor)
    
  • An extra-trees regressor (sklearn.ensemble.ExtraTreesRegressor)
    
  • Gradient Boosting for regression (sklearn.ensemble.GradientBoostingRegressor)
    
  • Histogram-based Gradient Boosting Regression Tree (sklearn.ensemble.HistGradientBoostingRegressor)
    
  • Isolation Forest Algorithm (sklearn.ensemble.IsolationForest)
    
• For detailed information, see Machine Learning support (ONNX-ML operators) [ONNX-ML] article

Unsqueeze

Reshapes a tensor

• category: Reshaping operation
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Upsample

Resize the input tensor: it calculates every value in the output tensor as a weighted average of neighborhood

• category: resizing operation
  
• can be hardware assisted : Neural-ART
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• Cubic resize is not supported
  
• half_pixel_symmetric value for coordinate_tansformation_mode is not supported
  
• The attribute antialias different from the default value is not supported
  
• axes attribute is not supported
  
• The attribute keep_aspect_ratio_policy different from the default value is not supported

Where

Where layer

• category: generic layer
  
• input data types: float32, int8, uint8, int16, uint16, int32, uint32, bool
  
• output data types: float32, int8, uint8, int16, uint16, int32, uint32, bool

Xor

Performs boolean element-wise operation

• category: eltwise operator
  
• input data types: bool
  
• output data types: bool

ZipMap

Creates a map from the input and the attributes.

• category: onnx.ml
  
• input data types: float32
  
• output data types: float32

Specific constraints/recommendations:

• used to support for example the skit-learn algo:
  • A decision tree classifier. (sklearn.tree.DecisionTreeClassifier)
    
  • …