Return op name tosa.abs as a bitstring.

tosa.abs - Elementwise abs operator.

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Elementwise absolute value operation.

This operation is not pure. Undefined behaviour may occur if the calculated result underflows.

Example:

%output = tosa.abs(%input1) : (tensor<21x3xf32>) -> tensor<21x3xf32>

Return op name tosa.add as a bitstring.

tosa.add - Elementwise addition operator.

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values
• input2 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Elementwise addition of input1 and input2. Axis of size 1 will be broadcast, as necessary. Rank of input tensors must match.

This operation is not pure. Undefined behaviour may occur if the calculated result overflows.

Example:

// Elementwise addition.
%out = tosa.add %input1, %input2 : tensor<12x6xf32>, tensor<12x6xf32> -> tensor<12x6xf32>

// Elementwise addition with broadcasting.
%out = tosa.add %input1, %input2 : tensor<12x6xsi32>, tensor<1x1xsi32> -> tensor<12x6xsi32>

Return op name tosa.add_shape as a bitstring.

tosa.add_shape - Elementwise addition of shapes.

Operands

• input1 - Single, Tosa_Shape, Shape with static rank and Index element type
• input2 - Single, Tosa_Shape, Shape with static rank and Index element type

Results

• output - Single, Tosa_Shape, Shape with static rank and Index element type

Description

Elementwise addition of input1 and input2. Size of shapes must match.

Return op name tosa.apply_scale as a bitstring.

tosa.apply_scale - Rescale scalar operator for Tosa tensor operators

Attributes

• rounding_mode - Single, Tosa_RoundingModeAttr, Supported rounding modes

Operands

• value - Single, Tosa_IntLike, signless-integer-like
• multiplier - Single, Tosa_IntLike, signless-integer-like
• shift - Single, Tosa_Int8Like, signless-integer-8-bit-like

Results

• output - Single, Tosa_IntLike, signless-integer-like

Description

Applies rescaling for fixed point values. This behavior is replicated in multiple quantized operations (mul, convolution, rescale, matmul, pooling).

The commonplace implementation is to use i64 operations to avoid integer overflow with target specific implementations can use native operations to avoid wider than necessary types.

Return op name tosa.argmax as a bitstring.

tosa.argmax - Perform argmax on the input.

Attributes

• axis - Single, I32Attr, 32-bit signless integer attribute
• nan_mode - Single, Tosa_NanPropagationModeAttr, Supported NaN propagation strategies

Operands

• input - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

This returns the index with the largest value across the given axis of the input tensor. If multiple locations have equal values, returns the first match along the search axis.

This operation is not pure. Undefined behaviour may occur if the max index is out of bounds for the output data type.

Return op name tosa.arithmetic_right_shift as a bitstring.

tosa.arithmetic_right_shift - Elementwise Arithmetic Right Shift.

Attributes

• round - Single, BoolAttr, bool attribute

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values
• input2 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Elementwise arithmetic right shift of input1 by the amount specified in input2. Axis of size 1 will be broadcast, as necessary. Rank of input tensors must match.

This operation is not pure. Undefined behaviour may occur if the specified shift is out of range.

Return op name tosa.assert_equal_shape as a bitstring.

tosa.assert_equal_shape - Verify two shapes are equal.

Attributes

• allow_broadcast - Single, BoolAttr, bool attribute

Operands

• input1 - Single, Tosa_Shape, Shape with static rank and Index element type
• input2 - Single, Tosa_Shape, Shape with static rank and Index element type

Description

Verify input1 and input2 are equal. If allow_broadcast is set, shapes which are broadcast compatible are allowed.

Return op name tosa.avg_pool2d as a bitstring.

tosa.avg_pool2d - Performs average pooling on the input.

Attributes

• kernel - Single, Tosa_IntArrayAttr2, i64 dense array attribute with exactly 2 elements
• stride - Single, Tosa_IntArrayAttr2, i64 dense array attribute with exactly 2 elements
• pad - Single, Tosa_IntArrayAttr4, i64 dense array attribute with exactly 4 elements
• acc_type - Single, anonymous/composite constraint, type attribute of 32-bit signless integer or 48-bit signless integer or 16-bit float or 32-bit float or bfloat16 type

Operands

• input - Single, Tosa_Tensor4D, 4D tosa-conformant tensor of number values
• input_zp - Single, Tosa_ScalarIntOrFloatTensor, tosa-conformant scalar tensor of unsigned integer or signless integer or floating-point values
• output_zp - Single, Tosa_ScalarIntOrFloatTensor, tosa-conformant scalar tensor of unsigned integer or signless integer or floating-point values

Results

• output - Single, Tosa_Tensor4D, 4D tosa-conformant tensor of number values

Description

This performs an average pooling over the given input tensor. A sliding window of size given by <kernel size> is passed over the input tensor, with the mean value being placed in the output tensor. When calculating the average, only the number of valid input tensor values, but not padding, are used to calculate the divisor.

This operation is not pure. Undefined behaviour may occur if the accumulated result overflows.

Return op name tosa.avg_pool2d_adaptive as a bitstring.

tosa.avg_pool2d_adaptive - Performs average pooling on the input with shape operands.

Attributes

• acc_type - Single, anonymous/composite constraint, type attribute of 32-bit signless integer or 48-bit signless integer or 16-bit float or 32-bit float or bfloat16 type

Operands

• input - Single, Tosa_Tensor4D, 4D tosa-conformant tensor of number values
• input_zp - Single, Tosa_ScalarIntOrFloatTensor, tosa-conformant scalar tensor of unsigned integer or signless integer or floating-point values
• output_zp - Single, Tosa_ScalarIntOrFloatTensor, tosa-conformant scalar tensor of unsigned integer or signless integer or floating-point values
• kernel - Single, Rank2TosaShape, Tosa shape type of rank 2
• stride - Single, Rank2TosaShape, Tosa shape type of rank 2
• pad - Single, Rank4TosaShape, Tosa shape type of rank 4

Results

• output - Single, Tosa_Tensor4D, 4D tosa-conformant tensor of number values

Description

This performs an average pooling over the given input tensor. A sliding window of size given by <kernel size> is passed over the input tensor, with the mean value being placed in the output tensor. When calculating the average, only the number of valid input tensor values, but not padding, are used to calculate the divisor. Compared to avg_pool2d, the kernel/stride/ pad values are provided as inputs.

Return op name tosa.bitwise_and as a bitstring.

tosa.bitwise_and - Bitwise AND operator.

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values
• input2 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Elementwise bitwise AND of input1 and input2. Axis of size 1 will be broadcast as necessary. Rank of input tensors must match.

Return op name tosa.bitwise_not as a bitstring.

tosa.bitwise_not - Bitwise NOT operator.

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Elementwise bitwise NOT of input tensor.

Return op name tosa.bitwise_or as a bitstring.

tosa.bitwise_or - Bitwise OR operator.

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values
• input2 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Elementwise bitwise OR of input1 and input2. Axis of size 1 will be broadcast as necessary. Rank of input tensors must match.

Return op name tosa.bitwise_xor as a bitstring.

tosa.bitwise_xor - Bitwise XOR operator.

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values
• input2 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Elementwise bitwise XOR of input1 and input2. Axis of size 1 will be broadcast as necessary. Rank of input tensors must match.

Return op name tosa.cast as a bitstring.

tosa.cast - Cast operation.

Operands

• input - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Casts a tensor from one data type to another.

Return op name tosa.cast_from_block_scaled as a bitstring.

tosa.cast_from_block_scaled - Apply scales from a scale tensor to the values in a value tensor

Attributes

• block_size - Single, Tosa_BlockSizeAttr, Block size for the block_scaled formats

Operands

• input_data - Single, Tosa_MXFPDataTensorAtLeast1D, tosa-conformant tensor of at least rank 1 of micro-scaling format number values
• input_scale - Single, Tosa_MXFPScaleTensorAtLeast1D, tosa-conformant tensor of at least rank 1 of micro-scaling format scale number values

Results

• output_data - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values

Description

Apply the scales from a scale tensor to the values in a value tensor, casting the result to the output type. The block dimension must be the last dimension of the tensor.

Note: This operation is deprecated. It will be removed in the future.

Return op name tosa.cast_to_block_scaled as a bitstring.

tosa.cast_to_block_scaled - Calculate scale tensor values per block, output to separate scale and data tensors.

Attributes

• block_size - Single, Tosa_BlockSizeAttr, Block size for the block_scaled formats

Operands

• input_data - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values

Results

• output_data - Single, Tosa_MXFPDataTensorAtLeast1D, tosa-conformant tensor of at least rank 1 of micro-scaling format number values
• output_scale - Single, Tosa_MXFPScaleTensorAtLeast1D, tosa-conformant tensor of at least rank 1 of micro-scaling format scale number values

Description

Calculate a scale value per block of input values and use that to calculate scaled data values from an input tensor. The output tensors are cast to the specified scale and value types. The block dimension will be the last dimension of the tensor.

Note: This operation is deprecated. It will be removed in the future.

Return op name tosa.ceil as a bitstring.

tosa.ceil - Elementwise ceil operator.

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Elementwise ceiling operation.

Return op name tosa.clamp as a bitstring.

tosa.clamp - Computes clamp(features, min, max).

Attributes

• min_val - Single, Tosa_IntOrFloatAttr, arbitrary integer attribute or arbitrary float attribute
• max_val - Single, Tosa_IntOrFloatAttr, arbitrary integer attribute or arbitrary float attribute
• nan_mode - Single, Tosa_NanPropagationModeAttr, Supported NaN propagation strategies

Operands

• input - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Clamp to an arbitrary minimum and maximum value. Maximum and minimum values are specified as values in the range of the input type. No zero point subtraction is done to the values, thus to clamp to the zero point value, the zero point itself should be supplied as the minimum value.

Return op name tosa.clz as a bitstring.

tosa.clz - Elementwise count leading zero operator.

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Elementwise count leading zeros operation.

Return op name tosa.concat as a bitstring.

tosa.concat - Concatenates tensors along one dimension.

This op has support for result type inference.

Attributes

• axis - Single, I32Attr, 32-bit signless integer attribute

Operands

• input1 - Variadic, Tosa_TensorAtLeast1D, variadic of tosa-conformant tensor of at least rank 1 of number values

Results

• output - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values

Description

Concatenate a list of tensors along a given axis. No data conversion happens during a concat operation.

Return op name tosa.concat_shape as a bitstring.

tosa.concat_shape - Concatenates a list of shapes.

Operands

• input - Variadic, Tosa_Shape, variadic of Shape with static rank and Index element type

Results

• output - Single, Tosa_Shape, Shape with static rank and Index element type

Description

Concatenates a list of shapes into a new shape with length equal to the sum of the lengths of the inputs.

Return op name tosa.cond_if as a bitstring.

tosa.cond_if - Conditional if operator.

Operands

• condition - Single, Tosa_I1Tensor, tosa-conformant tensor of 1-bit signless integer values
• input_list - Variadic, Tosa_Tensor, variadic of tosa-conformant tensor of number values

Results

• output_list - Variadic, Tosa_Tensor, variadic of tosa-conformant tensor of number values

Description

Evaluates a Boolean condition and then takes one of two distinct execution paths. This implements the semantic If-then-else structure.

Return op name tosa.const as a bitstring.

tosa.const - Constant operator.

Attributes

• values - Single, ElementsAttr, constant vector/tensor attribute

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

A node containing constant data for use as the input to an operation. May hold data in any of the supported data formats.

Example:

// Generic form
%out = "tosa.const"() {values = dense<0> : tensor<2x3xi32>} : () -> tensor<2x3xi32>

Return op name tosa.const_shape as a bitstring.

tosa.const_shape - Constant Shape operator.

Attributes

• values - Single, IndexElementsAttr, index elements attribute

Results

• output - Single, Tosa_Shape, Shape with static rank and Index element type

Description

A node containing a constant shape.

Example:

// Generic form
%out = "tosa.const_shape"() {values = dense<0> : tensor<4xindex>} : () -> !tosa.shape<4>

Return op name tosa.conv2d as a bitstring.

tosa.conv2d - 2D Convolution operator.

Attributes

• pad - Single, Tosa_IntArrayAttr4, i64 dense array attribute with exactly 4 elements
• stride - Single, Tosa_IntArrayAttr2, i64 dense array attribute with exactly 2 elements
• dilation - Single, Tosa_IntArrayAttr2, i64 dense array attribute with exactly 2 elements
• acc_type - Single, anonymous/composite constraint, type attribute of 32-bit signless integer or 48-bit signless integer or 16-bit float or 32-bit float or bfloat16 type
• local_bound - Optional, BoolAttr, bool attribute

Operands

• input - Single, Tosa_Tensor4D, 4D tosa-conformant tensor of number values
• weight - Single, Tosa_Tensor4D, 4D tosa-conformant tensor of number values
• bias - Single, Tosa_Tensor1D, 1D tosa-conformant tensor of number values
• input_zp - Single, Tosa_ScalarIntOrFloatTensor, tosa-conformant scalar tensor of unsigned integer or signless integer or floating-point values
• weight_zp - Single, Tosa_ScalarIntOrFloatTensor, tosa-conformant scalar tensor of unsigned integer or signless integer or floating-point values

Results

• output - Single, Tosa_Tensor4D, 4D tosa-conformant tensor of number values

Description

Performs a 2D convolution over the given tensor input, using the weight tensor. Implementations may choose to skip calculation of multiplies in the padding area.

This operation is not pure. Undefined behaviour may occur if the accumulated result overflows.

Return op name tosa.conv2d_block_scaled as a bitstring.

tosa.conv2d_block_scaled - Performs two dimensional convolution using block scaled tensors.

Attributes

• block_size - Single, Tosa_BlockSizeAttr, Block size for the block_scaled formats

Operands

• input_data - Single, Tosa_MXFPDataTensor4D, 4D tosa-conformant tensor of micro-scaling format number values
• input_scale - Single, Tosa_MXFPScaleTensor4D, 4D tosa-conformant tensor of micro-scaling format scale number values
• weight_data - Single, Tosa_MXFPDataTensor4D, 4D tosa-conformant tensor of micro-scaling format number values
• weight_scale - Single, Tosa_MXFPScaleTensor4D, 4D tosa-conformant tensor of micro-scaling format scale number values
• bias - Single, Tosa_Tensor1D, 1D tosa-conformant tensor of number values
• pad - Single, Rank4TosaShape, Tosa shape type of rank 4
• stride - Single, Rank2TosaShape, Tosa shape type of rank 2
• dilation - Single, Rank2TosaShape, Tosa shape type of rank 2

Results

• output - Single, Tosa_Tensor4D, 4D tosa-conformant tensor of number values

Description

Performs a 2D convolution over the given input data and scales, using the weight data and scales. Implementations may choose to skip calculation of multiplies in the padding area.

This operation is not pure. Undefined behaviour may occur if the accumulated result overflows.

// Note: This operation is deprecated. It will be removed in the future.

Return op name tosa.conv3d as a bitstring.

tosa.conv3d - 3D Convolution operator.

Attributes

• pad - Single, Tosa_IntArrayAttr6, i64 dense array attribute with exactly 6 elements
• stride - Single, Tosa_IntArrayAttr3, i64 dense array attribute with exactly 3 elements
• dilation - Single, Tosa_IntArrayAttr3, i64 dense array attribute with exactly 3 elements
• acc_type - Single, anonymous/composite constraint, type attribute of 32-bit signless integer or 48-bit signless integer or 16-bit float or 32-bit float or bfloat16 type
• local_bound - Optional, BoolAttr, bool attribute

Operands

• input - Single, Tosa_Tensor5D, 5D tosa-conformant tensor of number values
• weight - Single, Tosa_Tensor5D, 5D tosa-conformant tensor of number values
• bias - Single, Tosa_Tensor1D, 1D tosa-conformant tensor of number values
• input_zp - Single, Tosa_ScalarIntOrFloatTensor, tosa-conformant scalar tensor of unsigned integer or signless integer or floating-point values
• weight_zp - Single, Tosa_ScalarIntOrFloatTensor, tosa-conformant scalar tensor of unsigned integer or signless integer or floating-point values

Results

• output - Single, Tosa_Tensor5D, 5D tosa-conformant tensor of number values

Description

Performs a 3D convolution over the given input tensor. Implementations may choose to skip calculation of multiplies in the padding area.

This operation is not pure. Undefined behaviour may occur if the accumulated result overflows.

Return op name tosa.cos as a bitstring.

tosa.cos - Elementwise cos operator.

Operands

• input1 - Single, Tosa_FloatTensor, tosa-conformant tensor of floating-point values

Results

• output - Single, Tosa_FloatTensor, tosa-conformant tensor of floating-point values

Description

Elementwise cosine operation for values given in radians.

Return op name tosa.custom as a bitstring.

tosa.custom - Custom operator wrapper for Tosa

Attributes

• operator_name - Single, StrAttr, string attribute
• domain_name - Single, StrAttr, string attribute
• implementation_attrs - Single, StrAttr, string attribute

Operands

• input_list - Variadic, Tosa_Tensor, variadic of tosa-conformant tensor of number values

Results

• output_list - Variadic, Tosa_Tensor, variadic of tosa-conformant tensor of number values

Description

Hardware implementing TOSA may choose to add additional custom operators that are not expressed in the existing TOSA operations. These operators are not expected to be portable across TOSA implementations. The input and output signatures must be expressed in the corresponding TOSA node.

operator_name is a string that tells the backend which custom operator is being called.

domain_name is a string identifier which can help avoid name collisions on the identifier field.

implementation_attrs is a string which is a backend and identifier specific set of attributes to the custom operator.

input_list is the set of tensor inputs to the custom operator.

output_list is the list of tensors returned by the operator. The number of operators is backend specific.

Example:

%out = tosa.custom %in {domain_name = "tosa_mlir_test", operator_name =
       "custom_test", implementation_attrs = ""}: (tensor<10xi32>) ->
       (tensor<10xi32>)

Return op name tosa.depthwise_conv2d as a bitstring.

tosa.depthwise_conv2d - Depthwise 2D Convolution operator.

Attributes

• pad - Single, Tosa_IntArrayAttr4, i64 dense array attribute with exactly 4 elements
• stride - Single, Tosa_IntArrayAttr2, i64 dense array attribute with exactly 2 elements
• dilation - Single, Tosa_IntArrayAttr2, i64 dense array attribute with exactly 2 elements
• acc_type - Single, anonymous/composite constraint, type attribute of 32-bit signless integer or 48-bit signless integer or 16-bit float or 32-bit float or bfloat16 type
• local_bound - Optional, BoolAttr, bool attribute

Operands

• input - Single, Tosa_Tensor4D, 4D tosa-conformant tensor of number values
• weight - Single, Tosa_Tensor4D, 4D tosa-conformant tensor of number values
• bias - Single, Tosa_Tensor1D, 1D tosa-conformant tensor of number values
• input_zp - Single, Tosa_ScalarIntOrFloatTensor, tosa-conformant scalar tensor of unsigned integer or signless integer or floating-point values
• weight_zp - Single, Tosa_ScalarIntOrFloatTensor, tosa-conformant scalar tensor of unsigned integer or signless integer or floating-point values

Results

• output - Single, Tosa_Tensor4D, 4D tosa-conformant tensor of number values

Description

Performs 2D convolutions separately over each channel of the given tensor input, using the weight tensor. Implementations may choose to skip calculation of multiplies in the padding area.

This operation is not pure. Undefined behaviour may occur if the accumulated result overflows.

Return op name tosa.dim as a bitstring.

tosa.dim - Extract size of dimension from input tensor.

Attributes

• axis - Single, I32Attr, 32-bit signless integer attribute

Operands

• input1 - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values

Results

• output - Single, Tosa_Shape, Shape with static rank and Index element type

Description

Returns a length 1 shape_t of the size of the input tensor for the given axis.

Return op name tosa.div_ceil_shape as a bitstring.

tosa.div_ceil_shape - Elementwise ceiling divide of shapes.

Operands

• input1 - Single, Tosa_Shape, Shape with static rank and Index element type
• input2 - Single, Tosa_Shape, Shape with static rank and Index element type

Results

• output - Single, Tosa_Shape, Shape with static rank and Index element type

Description

Elementwise divide of input1 by input2. The result of the divide is rounded up.

Return op name tosa.div_floor_shape as a bitstring.

tosa.div_floor_shape - Elementwise floor divide of shapes.

Operands

• input1 - Single, Tosa_Shape, Shape with static rank and Index element type
• input2 - Single, Tosa_Shape, Shape with static rank and Index element type

Results

• output - Single, Tosa_Shape, Shape with static rank and Index element type

Description

Elementwise integer divide of input1 by input2. The result of the divide is rounded down.

Return op name tosa.equal as a bitstring.

tosa.equal - Returns the truth value of (input1 == input2) element-wise.

This op has support for result type inference.

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values
• input2 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_I1Tensor, tosa-conformant tensor of 1-bit signless integer values

Description

Elementwise comparison operation.

Return op name tosa.erf as a bitstring.

tosa.erf - Computes gauss error function of input.

Operands

• input - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Gauss error function: $ erf(x) = \frac{2}{\sqrt{\pi}} \int_{0}^{x} e^{-t^2} dt $ For quantized integer data types, the TABLE operator should be used instead with the following definition. The ERF table has 513 entries each of 16-bit precision and covering the input range -4.0 to +4.0 in steps of 1/64.

Return op name tosa.exp2_shape as a bitstring.

tosa.exp2_shape - Elementwise base-2 exponential of shapes.

Operands

• input - Single, Tosa_Shape, Shape with static rank and Index element type

Results

• output - Single, Tosa_Shape, Shape with static rank and Index element type

Description

Computation of raising two to the power of each element in input.

Return op name tosa.exp as a bitstring.

tosa.exp - Elementwise exp operator.

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Elementwise e to the x operation

Return op name tosa.fft2d as a bitstring.

tosa.fft2d - Performs FFT2D operation on the input.

Attributes

• inverse - Single, BoolAttr, bool attribute
• local_bound - Optional, BoolAttr, bool attribute

Operands

• input_real - Single, Tosa_Tensor3D, 3D tosa-conformant tensor of number values
• input_imag - Single, Tosa_Tensor3D, 3D tosa-conformant tensor of number values

Results

• output_real - Single, Tosa_Tensor3D, 3D tosa-conformant tensor of number values
• output_imag - Single, Tosa_Tensor3D, 3D tosa-conformant tensor of number values

Description

Performs a batched complex 2D Fast Fourier Transform over the input. The complex input values are constructed from the corresponding values in the input_real and input_imag tensors. The resulting values in the output are split into the output_real and output_imag tensors. No normalization is applied on either the forward or inverse versions of the operation.

Example:

 %output_real, %output_imag = tosa.fft2d %input_real, %input_imag : (tensor<8x9xf32>, tensor<8x9xf32>) -> (tensor<8x9xf32>, tensor<8x9xf32>)

Return op name tosa.floor as a bitstring.

tosa.floor - Elementwise floor operator.

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Elementwise floor operation.

Return op name tosa.gather as a bitstring.

tosa.gather - Gather operation.

Operands

• values - Single, Tosa_Tensor3D, 3D tosa-conformant tensor of number values
• indices - Single, Tosa_IndexTensor2D, 2D tosa-conformant tensor of 32-bit signless integer or 64-bit signless integer values

Results

• output - Single, Tosa_Tensor3D, 3D tosa-conformant tensor of number values

Description

Generate a tensor for which each element in the output is a subtensor of the values tensor based on the indices. N is the number of batches, W the number of indices in each batch, K the range of each index and C the number data channels for each index.

This operation is not pure. Undefined behaviour may occur if the specified indices are out of range.

Return op name tosa.greater as a bitstring.

tosa.greater - Returns the truth value of (input1 > input2) element-wise.

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values
• input2 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_I1Tensor, tosa-conformant tensor of 1-bit signless integer values

Description

Elementwise greater than comparison operation.

Return op name tosa.greater_equal as a bitstring.

tosa.greater_equal - Returns the truth value of (input1 >= input2) element-wise.

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values
• input2 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_I1Tensor, tosa-conformant tensor of 1-bit signless integer values

Description

Elementwise comparison operation.

Return op name tosa.identity as a bitstring.

tosa.identity - Identity operator.

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Returns a tensor with the same shape, type, and contents as the input.

Return op name tosa.intdiv as a bitstring.

tosa.intdiv - Integer divide operator.

Operands

• input1 - Single, Tosa_Int32Or64Tensor, tosa-conformant tensor of 32-bit signless integer or 64-bit signless integer values
• input2 - Single, Tosa_Int32Or64Tensor, tosa-conformant tensor of 32-bit signless integer or 64-bit signless integer values

Results

• output - Single, Tosa_Int32Or64Tensor, tosa-conformant tensor of 32-bit signless integer or 64-bit signless integer values

Description

Elementwise integer divide of input1 by input2. Axis of size 1 will be broadcast as necessary. Rank of input tensors must match. The result of the divide is truncated towards zero. Expected use is for operations on non-scaled integers. Floating point divide should use RECIPROCAL and MUL. Quantized integer divide should use TABLE (for 1/x) and MUL.

This operation is not pure. Undefined behaviour may occur on division by zero.

Return op name tosa.log2_ceil_shape as a bitstring.

tosa.log2_ceil_shape - Elementwise ceil base-2 logarithm of shapes.

Operands

• input - Single, Tosa_Shape, Shape with static rank and Index element type

Results

• output - Single, Tosa_Shape, Shape with static rank and Index element type

Description

Computation of the base two logarithm of each element in input. Result is rounded up.

Return op name tosa.log2_floor_shape as a bitstring.

tosa.log2_floor_shape - Elementwise floor base-2 logarithm of shapes.

Operands

• input - Single, Tosa_Shape, Shape with static rank and Index element type

Results

• output - Single, Tosa_Shape, Shape with static rank and Index element type

Description

Computation of the base two logarithm of each element in input. Result is rounded down.

Return op name tosa.log as a bitstring.

tosa.log - Elementwise log operator.

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Elementwise natural logarithm operation

Return op name tosa.logical_and as a bitstring.

tosa.logical_and - Returns the truth value of input1 AND input2 element-wise.

Operands

• input1 - Single, Tosa_I1Tensor, tosa-conformant tensor of 1-bit signless integer values
• input2 - Single, Tosa_I1Tensor, tosa-conformant tensor of 1-bit signless integer values

Results

• output - Single, Tosa_I1Tensor, tosa-conformant tensor of 1-bit signless integer values

Description

Elementwise logical AND of input1 and input2. Axis of size 1 will be broadcast, as necessary. Rank of input tensors must match.

Return op name tosa.logical_left_shift as a bitstring.

tosa.logical_left_shift - Elementwise Logical Left Shift.

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values
• input2 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Elementwise logical left-shift of input1 by the amount specified in input2. Axis of size 1 will be broadcast, as necessary. Rank of input tensors must match.

This operation is not pure. Undefined behaviour may occur if the specified shift is out of range.

Return op name tosa.logical_not as a bitstring.

tosa.logical_not - Returns the truth value of NOT input1 element-wise.

Operands

• input1 - Single, Tosa_I1Tensor, tosa-conformant tensor of 1-bit signless integer values

Results

• output - Single, Tosa_I1Tensor, tosa-conformant tensor of 1-bit signless integer values

Description

Elementwise logical NOT of input.

Return op name tosa.logical_or as a bitstring.

tosa.logical_or - Returns the truth value of x OR y element-wise.

Operands

• input1 - Single, Tosa_I1Tensor, tosa-conformant tensor of 1-bit signless integer values
• input2 - Single, Tosa_I1Tensor, tosa-conformant tensor of 1-bit signless integer values

Results

• output - Single, Tosa_I1Tensor, tosa-conformant tensor of 1-bit signless integer values

Description

Elementwise logical OR of input1 and input2. Axis of size 1 will be broadcast as necessary. Rank of input tensors must match.

Return op name tosa.logical_right_shift as a bitstring.

tosa.logical_right_shift - Elementwise Logical Right Shift.

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values
• input2 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Elementwise logical right shift of input1 by the amount specified in input2. Axis of size 1 will be broadcast, as necessary. Rank of input tensors must match.

This operation is not pure. Undefined behaviour may occur if the specified shift is out of range.

Return op name tosa.logical_xor as a bitstring.

tosa.logical_xor - Returns the truth value of input1 XOR input2 element-wise.

Operands

• input1 - Single, Tosa_I1Tensor, tosa-conformant tensor of 1-bit signless integer values
• input2 - Single, Tosa_I1Tensor, tosa-conformant tensor of 1-bit signless integer values

Results

• output - Single, Tosa_I1Tensor, tosa-conformant tensor of 1-bit signless integer values

Description

Elementwise logical XOR of input1 and input2. Axis of size 1 will be broadcast as necessary. Rank of input tensors must match.

Return op name tosa.matmul as a bitstring.

tosa.matmul - Matrix multiplication operator.

Operands

• a - Single, Tosa_Tensor3D, 3D tosa-conformant tensor of number values
• b - Single, Tosa_Tensor3D, 3D tosa-conformant tensor of number values
• a_zp - Single, Tosa_ScalarIntOrFloatTensor, tosa-conformant scalar tensor of unsigned integer or signless integer or floating-point values
• b_zp - Single, Tosa_ScalarIntOrFloatTensor, tosa-conformant scalar tensor of unsigned integer or signless integer or floating-point values

Results

• output - Single, Tosa_Tensor3D, 3D tosa-conformant tensor of number values

Description

Performs two dimensional matrix multiplications.

This operation is not pure. Undefined behaviour may occur if the accumulated result overflows.

Return op name tosa.matmul_t as a bitstring.

tosa.matmul_t - Matrix multiplication with transposed B operator.

Operands

• a - Single, Tosa_Tensor3D, 3D tosa-conformant tensor of number values
• b - Single, Tosa_Tensor3D, 3D tosa-conformant tensor of number values
• a_zp - Single, Tosa_ScalarIntOrFloatTensor, tosa-conformant scalar tensor of unsigned integer or signless integer or floating-point values
• b_zp - Single, Tosa_ScalarIntOrFloatTensor, tosa-conformant scalar tensor of unsigned integer or signless integer or floating-point values

Results

• output - Single, Tosa_Tensor3D, 3D tosa-conformant tensor of number values

Description

Performs two dimensional matrix multiplications. A matrix is of shape N x H x C. B matrix is of shape D x W x C. This is effectively a matrix multiply of A by the transposed B matrix. If the batched dimension of input B is of size 1, the B matrix is broadcast.

Return op name tosa.matmul_t_block_scaled as a bitstring.

tosa.matmul_t_block_scaled - Performs two dimensional matrix multiplications using block scaled tensors.

Attributes

• block_size - Single, Tosa_BlockSizeAttr, Block size for the block_scaled formats

Operands

• a_data - Single, Tosa_MXFPDataTensor3D, 3D tosa-conformant tensor of micro-scaling format number values
• a_scale - Single, Tosa_MXFPScaleTensor3D, 3D tosa-conformant tensor of micro-scaling format scale number values
• b_data - Single, Tosa_MXFPDataTensor3D, 3D tosa-conformant tensor of micro-scaling format number values
• b_scale - Single, Tosa_MXFPScaleTensor3D, 3D tosa-conformant tensor of micro-scaling format scale number values

Results

• output_data - Single, Tosa_Tensor3D, 3D tosa-conformant tensor of number values

Description

Performs two dimensional matrix multiplications using block scaled tensors. The block dimension is always the the last dimension of the tensor, so the result is effectively a matrix multiply of A by the transposed B matrix. If the N dimension of input B is of size 1, the B matrix will be broadcast.

This operation is not pure. Undefined behaviour may occur if the accumulated result overflows.

Note: This operation is deprecated. It will be removed in the future.

Return op name tosa.max_pool2d as a bitstring.

tosa.max_pool2d - Performs max pooling on the input.

Attributes

• kernel - Single, Tosa_IntArrayAttr2, i64 dense array attribute with exactly 2 elements
• stride - Single, Tosa_IntArrayAttr2, i64 dense array attribute with exactly 2 elements
• pad - Single, Tosa_IntArrayAttr4, i64 dense array attribute with exactly 4 elements
• nan_mode - Single, Tosa_NanPropagationModeAttr, Supported NaN propagation strategies

Operands

• input - Single, Tosa_Tensor4D, 4D tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor4D, 4D tosa-conformant tensor of number values

Description

This performs a max pooling over the given input tensor. A sliding window of size given by <kernel size> is passed over the input tensor, with the maximum value being placed in the output tensor.

Return op name tosa.max_pool2d_adaptive as a bitstring.

tosa.max_pool2d_adaptive - Performs max pooling on the input.

Attributes

• nan_mode - Single, Tosa_NanPropagationModeAttr, Supported NaN propagation strategies

Operands

• input - Single, Tosa_Tensor4D, 4D tosa-conformant tensor of number values
• kernel - Single, Rank2TosaShape, Tosa shape type of rank 2
• stride - Single, Rank2TosaShape, Tosa shape type of rank 2
• pad - Single, Rank4TosaShape, Tosa shape type of rank 4

Results

• output - Single, Tosa_Tensor4D, 4D tosa-conformant tensor of number values

Description

This performs a max pooling over the given input tensor. A sliding window of size given by <kernel size> is passed over the input tensor, with the maximum value being placed in the output tensor. Compared to MAX_POOL2D, MAX_POOL2D_ADAPTIVE has the kernel, stride, pad arguments as inputs rather than attributes.

Return op name tosa.max_shape as a bitstring.

tosa.max_shape - Elementwise maximum of shapes.

Operands

• input1 - Single, Tosa_Shape, Shape with static rank and Index element type
• input2 - Single, Tosa_Shape, Shape with static rank and Index element type

Results

• output - Single, Tosa_Shape, Shape with static rank and Index element type

Description

Elementwise maximum of input1 and input2.

Return op name tosa.maximum as a bitstring.

tosa.maximum - Elementwise Maximum.

Attributes

• nan_mode - Single, Tosa_NanPropagationModeAttr, Supported NaN propagation strategies

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values
• input2 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Elementwise max of input1 and input2. Axis of size 1 will be broadcast, as necessary. Rank of input tensors must match.

Return op name tosa.min_shape as a bitstring.

tosa.min_shape - Elementwise minimum of shapes.

Operands

• input1 - Single, Tosa_Shape, Shape with static rank and Index element type
• input2 - Single, Tosa_Shape, Shape with static rank and Index element type

Results

• output - Single, Tosa_Shape, Shape with static rank and Index element type

Description

Elementwise minimum of input1 and input2.

Return op name tosa.minimum as a bitstring.

tosa.minimum - Elementwise Minimum.

Attributes

• nan_mode - Single, Tosa_NanPropagationModeAttr, Supported NaN propagation strategies

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values
• input2 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Elementwise minimum of input1 and input2. Axis of size 1 will be broadcast, as necessary. Rank of input tensors must match.

Return op name tosa.mod_shape as a bitstring.

tosa.mod_shape - Elementwise modulo of shapes.

Operands

• input1 - Single, Tosa_Shape, Shape with static rank and Index element type
• input2 - Single, Tosa_Shape, Shape with static rank and Index element type

Results

• output - Single, Tosa_Shape, Shape with static rank and Index element type

Description

Elementwise modulo of input1 divided by input2.

Return op name tosa.mul as a bitstring.

tosa.mul - Multiplication operator.

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values
• input2 - Single, Tosa_Tensor, tosa-conformant tensor of number values
• shift - Single, Tosa_ScalarInt8Tensor, tosa-conformant scalar tensor of 8-bit signless integer values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Elementwise multiplication (Hadamard product) of input1 and input2. Axis of size 1 will be broadcast, as necessary. Rank of input tensors must match.

This operation is not pure. Undefined behaviour may occur if the specifed shift is out of range or the result overflows.

Return op name tosa.mul_shape as a bitstring.

tosa.mul_shape - Elementwise multiplication of shapes.

Operands

• input1 - Single, Tosa_Shape, Shape with static rank and Index element type
• input2 - Single, Tosa_Shape, Shape with static rank and Index element type

Results

• output - Single, Tosa_Shape, Shape with static rank and Index element type

Description

Elementwise multiplication of input1 and input2.

Return op name tosa.negate as a bitstring.

tosa.negate - Elementwise negate operator.

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values
• input1_zp - Single, Tosa_ScalarIntOrFloatTensor, tosa-conformant scalar tensor of unsigned integer or signless integer or floating-point values
• output_zp - Single, Tosa_ScalarIntOrFloatTensor, tosa-conformant scalar tensor of unsigned integer or signless integer or floating-point values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Elementwise negation operation.

This operation is not pure. Undefined behaviour may occur if the calculated result underflows or overflows.

Return op name tosa.pad as a bitstring.

tosa.pad - Pads a tensor with value specified.

Operands

• input1 - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values
• padding - Single, Tosa_Shape, Shape with static rank and Index element type
• pad_const - Single, Tosa_ScalarTensor, tosa-conformant scalar tensor of number values

Results

• output - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values

Description

Pads a tensor along the borders of each dimension with a supplied value. Returns a new tensor with the padding included. The pad_const value includes the zero point if the tensor uses a zero point.

Example:

%pad_const = "tosa.const"() {values = dense<3.14> : tensor<1xf32>} : () -> tensor<1xf32>
%padding = tosa.const_shape {values = dense<[1, 2, 3, 4]> : tensor<4xindex>} : () -> !tosa.shape<4>
tosa.pad %arg0, %padding, %pad_const: (tensor<1x2xf32>, !tosa.shape<4>, tensor<1xf32>)  -> (tensor<4x9xf32>)

Example 2:

%pad_const = "tosa.const"() {values = dense<3.14> : tensor<1xf32>} : () -> tensor<1xf32>
%padding = tosa.const_shape {values = dense<[-1, 2, 3, 4]> : tensor<4xindex>} : () -> !tosa.shape<4>
tosa.pad %arg0, %padding, %pad_const : (tensor<1x2xf32>, !tosa.shape<4>, tensor<1xf32>)  -> (tensor<?x9xf32>)

Return op name tosa.pow as a bitstring.

tosa.pow - Computes the power of one value to another.

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values
• input2 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Elementwise input1 value raised to the power of input2. Axis of size 1 will be broadcast, as necessary. Rank of input tensors must match.

This operation is not pure. Undefined behaviour may occur if specified exponent is negative.

Return op name tosa.reciprocal as a bitstring.

tosa.reciprocal - Elementwise reciprocal operator.

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Elementwise reciprocal operation. For integer operation, a TABLE should be used with the appropriate ranges.

Return op name tosa.reduce_all as a bitstring.

tosa.reduce_all - Reduce All operator.

This op has support for result type inference.

Attributes

• axis - Single, I32Attr, 32-bit signless integer attribute

Operands

• input - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values

Results

• output - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values

Description

Reduce a tensor along the given axis with a logical AND operation.

Return op name tosa.reduce_any as a bitstring.

tosa.reduce_any - Reduce Any operator.

This op has support for result type inference.

Attributes

• axis - Single, I32Attr, 32-bit signless integer attribute

Operands

• input - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values

Results

• output - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values

Description

Reduce a tensor along the given axis with a logical OR operation.

Return op name tosa.reduce_max as a bitstring.

tosa.reduce_max - Reduce Max operator.

This op has support for result type inference.

Attributes

• axis - Single, I32Attr, 32-bit signless integer attribute
• nan_mode - Single, Tosa_NanPropagationModeAttr, Supported NaN propagation strategies

Operands

• input - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values

Results

• output - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values

Description

Reduce a tensor along the given axis with a maximum operation.

Return op name tosa.reduce_min as a bitstring.

tosa.reduce_min - Reduce Min operator.

This op has support for result type inference.

Attributes

• axis - Single, I32Attr, 32-bit signless integer attribute
• nan_mode - Single, Tosa_NanPropagationModeAttr, Supported NaN propagation strategies

Operands

• input - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values

Results

• output - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values

Description

Reduce a tensor along the given axis with a minimum operation.

Return op name tosa.reduce_product as a bitstring.

tosa.reduce_product - Reduce Product operator.

This op has support for result type inference.

Attributes

• axis - Single, I32Attr, 32-bit signless integer attribute

Operands

• input - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values

Results

• output - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values

Description

Reduce a tensor along the given axis by computing the product of the axis.

This operation is not pure. Undefined behaviour may occur if the accumulated result overflows.

Return op name tosa.reduce_sum as a bitstring.

tosa.reduce_sum - Reduce Sum operator.

This op has support for result type inference.

Attributes

• axis - Single, I32Attr, 32-bit signless integer attribute

Operands

• input - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values

Results

• output - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values

Description

Reduce a tensor along the given axis by computing the sum of the axis.

This operation is not pure. Undefined behaviour may occur if the accumulated result overflows.

Return op name tosa.rescale as a bitstring.

tosa.rescale - Tosa rescale operator.

Attributes

• scale32 - Single, BoolAttr, bool attribute
• rounding_mode - Single, Tosa_RoundingModeAttr, Supported rounding modes
• per_channel - Single, BoolAttr, bool attribute
• input_unsigned - Single, BoolAttr, bool attribute
• output_unsigned - Single, BoolAttr, bool attribute

Operands

• input - Single, Tosa_Tensor, tosa-conformant tensor of number values
• multiplier - Single, Tosa_1DInt16Or32Tensor, 1D tosa-conformant tensor of 16-bit signless integer or 32-bit signless integer values
• shift - Single, Tosa_1DInt8Tensor, 1D tosa-conformant tensor of 8-bit signless integer values
• input_zp - Single, Tosa_ScalarIntOrFloatTensor, tosa-conformant scalar tensor of unsigned integer or signless integer or floating-point values
• output_zp - Single, Tosa_ScalarIntOrFloatTensor, tosa-conformant scalar tensor of unsigned integer or signless integer or floating-point values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

RESCALE is defined using an integer multiply, add, and shift.

Rescale supports two precisions of multiplier: 16-bit and 32-bit. The 32-bit multiplier version supports two rounding modes to enable simpler lowering of existing frameworks that use two stage rounding. All arithmetic is designed so that it does not overflow a 64-bit accumulator and that the result fits in 32 bits. In particular, a 48-bit value cannot be scaled with the 32-bit multiplier because the accumulator would need to have 80 bits.

The shift and value range are limited to allow a variety of implementations. The limit of 62 on shift allows the shift to be decomposed as two right shifts of 31.

This operation is not pure. Undefined behaviour may occur if the calculated result underflows or overflows.

Supported rescalings:

• This table is showing the supported conversions from the TOSA Specification.
• The MLIR dialect here can be used to represent other conversions.

Return op name tosa.reshape as a bitstring.

tosa.reshape - Reshape operator.

This op has support for result type inference.

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values
• shape - Single, Tosa_Shape, Shape with static rank and Index element type

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Returns a tensor with the same type/values as the input, with a new shape specified by the shape argument. Reshape may operate on tensors of any rank. No data conversion happens during a reshape operation.

Return op name tosa.reshape_block_scaled as a bitstring.

tosa.reshape_block_scaled - Reshape with support for block scaled tensors.

This op has support for result type inference.

Attributes

• block_size - Single, Tosa_BlockSizeAttr, Block size for the block_scaled formats

Operands

• input - Variadic, Tosa_Tensor, variadic of tosa-conformant tensor of number values
• new_value_shape - Single, Tosa_Shape, Shape with static rank and Index element type

Results

• output - Variadic, Tosa_Tensor, variadic of tosa-conformant tensor of number values

Description

Returns a tensor-list with the same type/values as the input, with a new shape specified by the shape argument. Reshape may operate on block-scaled tensors (values tensor followed by scale tensor) of rank 1 or higher; or a single non-block-scaled tensor of any rank. No data conversion happens during a reshape operation. Reshape must retain the relationship between values and their scale in a block for block-scaled content.

Note: This operation is deprecated. It will be removed in the future.

Return op name tosa.resize as a bitstring.

tosa.resize - Resize operation, supports various resize/upsample modes.

Attributes

• mode - Single, Tosa_ResizeModeAttr, Supported resize/upsampling strategies

Operands

• input - Single, Tosa_Tensor4D, 4D tosa-conformant tensor of number values
• scale - Single, Rank4TosaShape, Tosa shape type of rank 4
• offset - Single, Rank2TosaShape, Tosa shape type of rank 2
• border - Single, Rank2TosaShape, Tosa shape type of rank 2

Results

• output - Single, Tosa_Tensor4D, 4D tosa-conformant tensor of number values

Description

Resizes a tensor. Resize is only allowed in the H and W dimensions.

The height dimension is scaled by factor (scale_y_n/scale_y_d). The width dimension is scaled by factor (scale_x_n/scale_x_d).

The NEAREST_NEIGHBOR mode returns the value of the input tensor closest to the calculated sample position for both floating-point and integer data formats.

Floating-point BILINEAR mode returns a bilinearly interpolated output value based on the four closest input sample positions.

For integer BILINEAR interpolation mode, the output value must be scaled by 1/(scale_y_n * scale_x_n) in a following operation to complete the interpolation (for example with a RESCALE operator).

The output dimensions can be derived from the input dimensions by inverting the scale as described in the pseudocode. The [border_y, border_x] values adjust the output size to allow fractional sampling beyond integer input position (IH - 1,IW - 1).

The limit MAX_SCALE is applied to each scale ratio after reduction of the ratio. Individual scale numerator and denominator values are allowed to be larger than MAX_SCALE.

Return op name tosa.reverse as a bitstring.

tosa.reverse - Reverse operator.

Attributes

• axis - Single, I32Attr, 32-bit signless integer attribute

Operands

• input1 - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values

Results

• output - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values

Description

Returns a tensor with the same type/values as the input, with the data reversed along the given axis. No data conversion happens during a reverse operation.

Return op name tosa.rfft2d as a bitstring.

tosa.rfft2d - Performs RFFT2D operation on the input.

Attributes

• local_bound - Optional, BoolAttr, bool attribute

Operands

• input_real - Single, Tosa_Tensor3D, 3D tosa-conformant tensor of number values

Results

• output_real - Single, Tosa_Tensor3D, 3D tosa-conformant tensor of number values
• output_imag - Single, Tosa_Tensor3D, 3D tosa-conformant tensor of number values

Description

Performs a batched 2D real-valued Fast Fourier Transform over the input where the input tensor consists of real values producing complex valued output. The complex output values will be split into the output_real and output_imag tensor arguments. RFFT2D takes advantage of Hermitian symmetry to only calculate the first half of the final output axis. Implementations may choose to skip calculation of the imaginary values at (0,0), (0,W/2), (H/2,0), and (H/2, W/2). If the calculation is skipped, the result at that location must be zero.

Example:

 %ouput_real, %output_imag = tosa.rfft2d %input_real : (tensor<8x16xf32>) -> (tensor<8x9xf32>, tensor<8x9xf32>)

Return op name tosa.row_gather as a bitstring.

tosa.row_gather - Row gather operation.

Operands

• values - Single, Tosa_Tensor3D, 3D tosa-conformant tensor of number values
• indices - Single, Tosa_IndexTensor2D, 2D tosa-conformant tensor of 32-bit signless integer or 64-bit signless integer values
• row_count - Single, Tosa_ScalarInt32Tensor, tosa-conformant scalar tensor of 32-bit signless integer values

Results

• output - Single, Tosa_Tensor3D, 3D tosa-conformant tensor of number values

Description

Generate a tensor based on the input indices and row_count. The number of consecutive rows gathered for each index is specified in row_count. N is the number of batches, W is the number of indices in each batch, K is the range of each index, and C is the number of data channels for each index. The values tensor has shape [N, K, C] and the output tensor has shape [N, W * row_count, C].

This operation is not pure. Undefined behaviour may occur if the specified indices are out of range.

Return op name tosa.row_gather_block_scaled as a bitstring.

tosa.row_gather_block_scaled - Row gather operation for block-scaled and non-block-scaled data.

Attributes

• block_size - Single, Tosa_BlockSizeAttr, Block size for the block_scaled formats

Operands

• values - Variadic, Tosa_Tensor3D, variadic of 3D tosa-conformant tensor of number values
• indices - Single, Tosa_IndexTensor2D, 2D tosa-conformant tensor of 32-bit signless integer or 64-bit signless integer values
• row_count - Single, Tosa_ScalarInt32Tensor, tosa-conformant scalar tensor of 32-bit signless integer values

Results

• output - Variadic, Tosa_Tensor3D, variadic of 3D tosa-conformant tensor of number values

Description

Generate a tensor-list which contains a data tensor and an optional scale tensor based on the input indices and row_count. The number of consecutive rows gathered for each index is specified in row_count.

The variadic values operands and results model the flattened tensor_list_t form from the TOSA spec:

• non-block-scaled: 1 input value tensor and 1 result tensor
• block-scaled: data + scale input tensors and data + scale result tensors

This operation is not pure. Undefined behaviour may occur if the specified indices are out of range.

Note: This operation is deprecated. It will be removed in the future.

Return op name tosa.rsqrt as a bitstring.

tosa.rsqrt - Elementwise 1/sqrt operator.

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Elementwise reciprocal square root operation. For integer operation, a TABLE should be used with the appropriate ranges.

Return op name tosa.scatter as a bitstring.

tosa.scatter - Scatter operation.

Operands

• values_in - Single, Tosa_Tensor3D, 3D tosa-conformant tensor of number values
• indices - Single, Tosa_IndexTensor2D, 2D tosa-conformant tensor of 32-bit signless integer or 64-bit signless integer values
• input - Single, Tosa_Tensor3D, 3D tosa-conformant tensor of number values

Results

• values_out - Single, Tosa_Tensor3D, 3D tosa-conformant tensor of number values

Description

The values_out tensor is set to the values_in tensor with data modified as follows: data from the input tensor is inserted at the positions specified by the indices tensor. N is the number of batches, W the number of indices in each batch, K the range of each index and C the number data channels for each index. It is not permitted to repeat the same output index within a single SCATTER operation and so each output index occurs at most once. It follows that K >= W. In use cases that require multiple updates to the same output position, these must be decomposed into multiple SCATTER operations.

This operation is not pure. Undefined behaviour may occur if the specified indices are out of range or duplicate indices are provided.

Return op name tosa.select as a bitstring.

tosa.select - Elementwise select operator.

Operands

• input1 - Single, Tosa_I1Tensor, tosa-conformant tensor of 1-bit signless integer values
• input2 - Single, Tosa_Tensor, tosa-conformant tensor of number values
• input3 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Elementwise select of the output based on a condition.

Return op name tosa.sigmoid as a bitstring.

tosa.sigmoid - Computes elementwise sigmoid of input.

Operands

• input - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Applies the sigmoid logistic function to each element of the input tensor: $ sigmoid(x) = \frac{1}{1 + e^{-x}} $.

For quantized integer data types, the TABLE operator should be used instead. Each implementation may choose an appropriate TABLE given the scale and zero point of the input data. Eight or sixteen bit precision tables may be used based on the input tensor to the sigmoid function.

Return op name tosa.sin as a bitstring.

tosa.sin - Elementwise sin operator.

Operands

• input1 - Single, Tosa_FloatTensor, tosa-conformant tensor of floating-point values

Results

• output - Single, Tosa_FloatTensor, tosa-conformant tensor of floating-point values

Description

Elementwise sine operation for values given in radians.

Return op name tosa.slice as a bitstring.

tosa.slice - Slice operator.

Operands

• input1 - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values
• start - Single, Tosa_Shape, Shape with static rank and Index element type
• size - Single, Tosa_Shape, Shape with static rank and Index element type

Results

• output - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values

Description

Extracts a slice of input1, beginning at the start coordinates, and extending for size elements in each direction. No data conversion happens during a slice operation.

Return op name tosa.slice_shape as a bitstring.

tosa.slice_shape - Extract slice of a shape.

Operands

• input - Single, Tosa_Shape, Shape with static rank and Index element type
• start - Single, Tosa_ScalarInt32Tensor, tosa-conformant scalar tensor of 32-bit signless integer values
• size - Single, Tosa_ScalarInt32Tensor, tosa-conformant scalar tensor of 32-bit signless integer values

Results

• output - Single, Tosa_Shape, Shape with static rank and Index element type

Description

Extract a shape of size from input.

Return op name tosa.sub as a bitstring.

tosa.sub - Elementwise subtraction operator.

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values
• input2 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Elementwise subtraction of input1 and input2. Axis of size 1 will be broadcast as necessary. Rank of input tensors must match.

This operation is not pure. Undefined behaviour may occur if the calculated result underflows.

Return op name tosa.sub_shape as a bitstring.

tosa.sub_shape - Elementwise subtraction of shapes.

Operands

• input1 - Single, Tosa_Shape, Shape with static rank and Index element type
• input2 - Single, Tosa_Shape, Shape with static rank and Index element type

Results

• output - Single, Tosa_Shape, Shape with static rank and Index element type

Description

Elementwise subtraction of input1 and input2. Size of shapes must match.

Return op name tosa.table as a bitstring.

tosa.table - Table lookup operator.

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values
• table - Single, Tosa_Tensor1D, 1D tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Table lookup operation. For int8_t TABLE operation, perform a 256 entry table lookup returning an int8_t value. For int16_t tables, the int16_t input is treated as a fixed-point 9.7 value. The most significant 9 bits are used to index into the table. The fractional 7 bits are used to interpolate based on table[index] and table[index+1]. For int16_t inputs, the TABLE operator returns a 16.7 interpolated value in an int32_t. This value can then be input to the RESCALE operator to scale to the required output data type. Note that int16_t table has 513 values to handle table[index+1] when index=511.

An int16_t to int16_t table lookup can be constructed in TOSA as follows:

• Use the TABLE operator to produce a fixed point 16.7 interpolated result
• Use RESCALE (in_t=int32_t, out_t=int16_t, scale=1<<14, shift=21) to scale the output to int16_t range (or alternate scale as required)

This operation is not pure. Undefined behaviour may occur if the calculated slope is out of range.

Return op name tosa.tanh as a bitstring.

tosa.tanh - Computes elementwise hyperbolic tangent of input.

Operands

• input - Single, Tosa_Tensor, tosa-conformant tensor of number values

Results

• output - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Parameterized hyperbolic tangent: $ tanh(x) = \frac{1 - e^{-2x}}{1 + e^{-2x}} $.

For quantized integer data types, the TABLE operator should be used instead. Each implementation may choose an appropriate TABLE given the scale and zero point of the input data. Eight or sixteen bit precision tables may be used based on the input tensor to the tanh function.

Return op name tosa.tile as a bitstring.

tosa.tile - Tile operator.

Operands

• input1 - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values
• multiples - Single, Tosa_Shape, Shape with static rank and Index element type

Results

• output - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values

Description

Replicates input1 multiples times along each dimension.

Return op name tosa.transpose as a bitstring.

tosa.transpose - Transpose operator.

Attributes

• perms - Single, DenseI32ArrayAttr, i32 dense array attribute

Operands

• input1 - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values

Results

• output - Single, Tosa_TensorAtLeast1D, tosa-conformant tensor of at least rank 1 of number values

Description

Permutes the dimensions of the input tensor input1 based on the perms argument. Each value in the perms list must be a valid dimension of the input tensor and may not be repeated.

Return op name tosa.transpose_conv2d as a bitstring.

tosa.transpose_conv2d - Transpose 2D Convolution operator.

Attributes

• out_pad - Single, Tosa_IntArrayAttr4, i64 dense array attribute with exactly 4 elements
• stride - Single, Tosa_IntArrayAttr2, i64 dense array attribute with exactly 2 elements
• acc_type - Single, anonymous/composite constraint, type attribute of 32-bit signless integer or 48-bit signless integer or 16-bit float or 32-bit float or bfloat16 type
• local_bound - Optional, BoolAttr, bool attribute

Operands

• input - Single, Tosa_Tensor4D, 4D tosa-conformant tensor of number values
• weight - Single, Tosa_Tensor4D, 4D tosa-conformant tensor of number values
• bias - Single, Tosa_Tensor1D, 1D tosa-conformant tensor of number values
• input_zp - Single, Tosa_ScalarIntOrFloatTensor, tosa-conformant scalar tensor of unsigned integer or signless integer or floating-point values
• weight_zp - Single, Tosa_ScalarIntOrFloatTensor, tosa-conformant scalar tensor of unsigned integer or signless integer or floating-point values

Results

• output - Single, Tosa_Tensor4D, 4D tosa-conformant tensor of number values

Description

Performs a 2D transposed convolution over the given tensor input, using the weights tensor. Implementations may choose to skip calculation of multiplies by zero at fractional input positions.

This operation is not pure. Undefined behaviour may occur if the accumulated result overflows.

Return op name tosa.variable as a bitstring.

tosa.variable - Defines a variable

Attributes

• sym_name - Single, SymbolNameAttr, string attribute
• var_shape - Single, IndexElementsAttr, index elements attribute
• type - Single, TypeAttr, any type attribute
• initial_value - Optional, AnyAttr, any attribute

Description

Defines a new TOSA variable. This is a persistent mutable value across multiple TOSA graph invocations. Modifications are expressed using read/write semantics.

Return op name tosa.variable_read as a bitstring.

tosa.variable_read - read_buffer operator

Attributes

• name - Single, SymbolNameAttr, string attribute

Results

• output1 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Reads the value from a pseudo-buffer resource holding a persistent mutable tensor.

Return op name tosa.variable_write as a bitstring.

tosa.variable_write - write_buffer operator

Attributes

• name - Single, SymbolNameAttr, string attribute

Operands

• input1 - Single, Tosa_Tensor, tosa-conformant tensor of number values

Description

Assigns a value to the pseudo-buffer resource holding a persistent mutable tensor.

Return op name tosa.while_loop as a bitstring.

tosa.while_loop - output = input; While (Cond(output)) {output = Body(output)}

Operands

• input_list - Variadic, Tosa_Tensor, variadic of tosa-conformant tensor of number values

Results

• output_list - Variadic, Tosa_Tensor, variadic of tosa-conformant tensor of number values

Description

Generates and evaluates a Boolean condition and either executes a loop body or exits the loop. This action is performed repeatedly after updating and re-evaluating the Boolean condition every iteration. This implements the semantic foreach or while iterative loop structure.

Return op name tosa.yield as a bitstring.

tosa.yield - yield operator

Operands

• inputs - Variadic, Tosa_Tensor, variadic of tosa-conformant tensor of number values

Description

return operation within the conditional and body of structured control flow. Operation takes variadic operands but produces no results of its own.