Return op name xevm.blockload2d as a bitstring.

xevm.blockload2d - 2D block load

Attributes

• elem_size_in_bits - Single, I32Attr, 32-bit signless integer attribute
• tile_width - Single, I32Attr, 32-bit signless integer attribute
• tile_height - Single, I32Attr, 32-bit signless integer attribute
• v_blocks - Single, I32Attr, 32-bit signless integer attribute
• transpose - Single, I1Attr, 1-bit signless integer attribute
• pack_register - Single, I1Attr, 1-bit signless integer attribute
• cache_control - Optional, XeVM_LoadCacheControlAttr, Describe the cache settings for load operators

Operands

• ptr - Single, LLVM_AnyPointer, LLVM pointer type
• base_width - Single, I32, 32-bit signless integer
• base_height - Single, I32, 32-bit signless integer
• base_pitch - Single, I32, 32-bit signless integer
• x - Single, I32, 32-bit signless integer
• y - Single, I32, 32-bit signless integer

Results

• res - Single, anonymous/composite constraint, fixed-length vector of 8-bit integer or 16-bit integer or 32-bit integer or 32-bit float or tf32 type or 16-bit float or bfloat16 type values of ranks 1

Description

The xevm.blockload2d operation loads a two dimensional matrix tile from a base matrix residing in global memory. The parameters are:

• ptr - the base address of the base matrix containing the tile to load
• base_width - the width of the base matrix in number of bytes.
• base_height - the number of rows in the base matrix
• base_pitch - the physical stride between the first columns of the current row and the subsequent row in number of bytes.
• x, y, tile_width, tile_height - the starting offsets and shape of the tile to load in number of elements.
• elem_size_in_bits - the size in bits of the matrix element type
  • 32 for f32, tf32
  • 16 for f16, int16, bf16
  • 8 for int8
• v_blocks - number of consecutive tiles in innermost dimension direction to load
• transpose - transpose the tile in registers (useful for 32 bit element type)
• pack_register - pack element types narrower than register bit width. [M, N] => [M/factor, N, factor] where factor is register_size_in_bits / elem_size_in_bits
• cache_control - an enumerator that sets the cache behaviour

Notes:

• the transpose and pack_register parameters are mutual exclusive
• transposing the tile loaded is used for A matrix in backward path or used for the B matrix operand (D = C + A * B), where A has row-major layout and B should have column-major layout in memory.
• if the tile loaded contains out of bound elements of the matrix, they are filled with 0.

Example:

  %base_width_a = arith.constant 32 : i32
  %base_height_a = arith.constant 8 : i32
  %base_pitch_a = arith.constant 32 : i32
  %x = arith.constant 0 : i32
  %y = arith.constant 0 : i32
  %loaded_a = xevm.blockload2d %src, %base_width_a, %base_height_a, %base_pitch_a, %x, %y
                <{elem_size_in_bits=16 : i32, tile_width=16 : i32, tile_height=8 : i32,
                  v_blocks=1 : i32, transpose=false : i32, pack_register=false,
                  cache_control=#xevm.load_cache_control<Default>}>
                : (!llvm.ptr<1>, i32, i32, i32, i32, i32) -> vector<8xi16>

Return op name xevm.blockload as a bitstring.

xevm.blockload - subgroup block load

Attributes

• cache_control - Optional, XeVM_LoadCacheControlAttr, Describe the cache settings for load operators

Operands

• ptr - Single, LLVM_AnyPointer, LLVM pointer type

Results

• res - Single, anonymous/composite constraint, 8-bit signless integer or 16-bit signless integer or 32-bit signless integer or 64-bit signless integer or fixed-length vector of 8-bit signless integer or 16-bit signless integer or 32-bit signless integer or 64-bit signless integer values of ranks 1

Description

Reads one or more components of Result data for each invocation in the subgroup from the specified ptr as a block operation. The data is read strided, so the first value read is:

  ptr[ SubgroupLocalInvocationId ]

and the second value read is:

  ptr[ SubgroupLocalInvocationId + SubgroupMaxSize ]

Result type may be a scalar or vector type of scalar element type.

The parameters are:

• ptr - the base address to load from. Must be uniform across subgroup.
• cache_control - an enumerator that sets the cache behaviour

Example:

  %loaded_a = xevm.blockload %src,
                  <{cache_control=#xevm.load_cache_control<L1uc_L2uc_L3uc>}>
                : (!llvm.ptr<1>) -> vector<4xi16>

Return op name xevm.blockprefetch2d as a bitstring.

xevm.blockprefetch2d - 2D block prefetch

Attributes

• elem_size_in_bits - Single, I32Attr, 32-bit signless integer attribute
• tile_width - Single, I32Attr, 32-bit signless integer attribute
• tile_height - Single, I32Attr, 32-bit signless integer attribute
• v_blocks - Single, I32Attr, 32-bit signless integer attribute
• cache_control - Optional, XeVM_LoadCacheControlAttr, Describe the cache settings for load operators

Operands

• ptr - Single, LLVM_AnyPointer, LLVM pointer type
• base_width - Single, I32, 32-bit signless integer
• base_height - Single, I32, 32-bit signless integer
• base_pitch - Single, I32, 32-bit signless integer
• x - Single, I32, 32-bit signless integer
• y - Single, I32, 32-bit signless integer

Description

The xevm.blockprefetch2d operation prefetches a two dimensional tile from a larger base matrix residing in global memory. The parameters are:

• ptr - the base address of the base matrix containing the tile to prefetch
• base_width - the width of the base matrix in number of bytes.
• base_height - the number of rows in the base matrix
• base_pitch - the physical stride between the first columns of the current row and the subsequent row in number of bytes.
• x, y, tile_width, tile_height - the starting offsets and shape of tile to prefetch in number of elements.
• elem_size_in_bits - the size in bits of the matrix element
  • 32 for f32, bf32
  • 16 for f16, int16, bf16
  • 8 for int8, int4, int2
• v_blocks - number of tiles in innermost dimension direction to prefetch
• cache_control - an enumerator that sets the cache behaviour

Example:

  xevm.blockprefetch2d %ptr, %base_width, %base_height, %base_pitch, %x, %y
    <{elem_size_in_bits=8 : i32, tile_width=32 : i32, tile_height=8 : i32,
      v_blocks=1 : i32, cache_control=#xevm.load_cache_control<L1uc_L2uc_L3uc>}>
    : (!llvm.ptr<1>, i32, i32, i32, i32, i32)

Return op name xevm.blockstore2d as a bitstring.

xevm.blockstore2d - 2D block store

Attributes

• elem_size_in_bits - Single, I32Attr, 32-bit signless integer attribute
• tile_width - Single, I32Attr, 32-bit signless integer attribute
• tile_height - Single, I32Attr, 32-bit signless integer attribute
• cache_control - Optional, XeVM_StoreCacheControlAttr, Describe the cache settings for store operators

Operands

• ptr - Single, LLVM_AnyPointer, LLVM pointer type
• base_width - Single, I32, 32-bit signless integer
• base_height - Single, I32, 32-bit signless integer
• base_pitch - Single, I32, 32-bit signless integer
• x - Single, I32, 32-bit signless integer
• y - Single, I32, 32-bit signless integer
• stored_val - Single, anonymous/composite constraint, fixed-length vector of 8-bit integer or 16-bit integer or 32-bit integer or 32-bit float or tf32 type or 16-bit float or bfloat16 type values of ranks 1

Description

The xevm.blockstore2d operation stores a two dimensional tile into a larger matrix residing in global memory. The parameters are:

• ptr - the base address of the target matrix where to store the tile
• base_width - the width of the base matrix in number of bytes.
• base_height - the number of rows in the base matrix
• base_pitch - the physical stride between the first columns of the current row and the subsequent row in number of bytes.
• x, y, tile_width, tile_height - the starting offsets and shape of the tile to store in number of elements.
• elem_size_in_bits - the size in bits of the matrix element
  • 32 for f32, tf32
  • 16 for f16, int16, bf16
  • 8 for int8
• cache_control - an enumerator that sets the cache behaviour
• stored_val - the tile to store

Example:

  %base_width_c = arith.constant 64 : i32
  %base_height_c = arith.constant 8 : i32
  %base_pitch_c = arith.constant 64 : i32
  %x = arith.constant 0 : i32
  %y = arith.constant 0 : i32
  xevm.blockstore2d %dst, %base_width_c, %base_height_c, %base_pitch_c, %x, %y, %src
    <{elem_size_in_bits=32 : i32, tile_width=16 : i32, tile_height=8 : i32,
      cache_control=#xevm.load_cache_control<Default>}>
    : (!llvm.ptr<1>, i32, i32, i32, i32, i32, vector<8xi32>)

Return op name xevm.blockstore as a bitstring.

xevm.blockstore - subgroup block store

Attributes

• cache_control - Optional, XeVM_StoreCacheControlAttr, Describe the cache settings for store operators

Operands

• ptr - Single, LLVM_AnyPointer, LLVM pointer type
• val - Single, anonymous/composite constraint, 8-bit signless integer or 16-bit signless integer or 32-bit signless integer or 64-bit signless integer or fixed-length vector of 8-bit signless integer or 16-bit signless integer or 32-bit signless integer or 64-bit signless integer values of ranks 1

Description

Writes one or more components of val for each invocation in the subgroup to the specified ptr as a block operation. The data is written strided, so the first value is written to:

  ptr[ SubgroupLocalInvocationId ]

and the second value is written to:

  ptr[ SubgroupLocalInvocationId + SubgroupMaxSize ]

val type may be a scalar or vector type of scalar element type.

The parameters are:

• ptr - the base address to store to. Must be uniform across subgroup.
• val - the value to store
• cache_control - an enumerator that sets the cache behaviour

Example:

  xevm.blockstore %ptr, %val
    <{cache_control=#xevm.store_cache_control<L1uc_L2uc_L3uc>}>
    : (!llvm.ptr<1>, vector<4xi16>)

Return op name xevm.extf as a bitstring.

xevm.extf - Floating point extension from f8/bf8/f4 to f16/bf16

Attributes

• src_etype - Single, XeVM_ExtfSrcElemTypeAttr,
• dst_etype - Single, XeVM_ExtfDstElemTypeAttr,

Operands

• src - Single, anonymous/composite constraint, fixed-length vector of 8-bit signless integer or 4-bit signless integer values of ranks 1 or 8-bit signless integer or 4-bit signless integer

Results

• dst - Single, anonymous/composite constraint, fixed-length vector of 16-bit float or bfloat16 type values of ranks 1 or 16-bit float or bfloat16 type

Description

The xevm.extf operation extends a floating point value from f8/bf8/f4 format to f16/bf16 format. It is the inverse of xevm.truncf.

Return op name xevm.group_count.x as a bitstring.

xevm.group_count.x

Attributes

• range - Optional, LLVM_ConstantRangeAttr, A range of two integers, corresponding to LLVM's ConstantRange

Results

• res - Single, anonymous/composite constraint, 32-bit signless integer or 64-bit signless integer

Return op name xevm.group_count.y as a bitstring.

xevm.group_count.y

Attributes

• range - Optional, LLVM_ConstantRangeAttr, A range of two integers, corresponding to LLVM's ConstantRange

Results

• res - Single, anonymous/composite constraint, 32-bit signless integer or 64-bit signless integer

Return op name xevm.group_count.z as a bitstring.

xevm.group_count.z

Attributes

• range - Optional, LLVM_ConstantRangeAttr, A range of two integers, corresponding to LLVM's ConstantRange

Results

• res - Single, anonymous/composite constraint, 32-bit signless integer or 64-bit signless integer

Return op name xevm.group_id.x as a bitstring.

xevm.group_id.x

Attributes

• range - Optional, LLVM_ConstantRangeAttr, A range of two integers, corresponding to LLVM's ConstantRange

Results

• res - Single, anonymous/composite constraint, 32-bit signless integer or 64-bit signless integer

Return op name xevm.group_id.y as a bitstring.

xevm.group_id.y

Attributes

• range - Optional, LLVM_ConstantRangeAttr, A range of two integers, corresponding to LLVM's ConstantRange

Results

• res - Single, anonymous/composite constraint, 32-bit signless integer or 64-bit signless integer

Return op name xevm.group_id.z as a bitstring.

xevm.group_id.z

Attributes

• range - Optional, LLVM_ConstantRangeAttr, A range of two integers, corresponding to LLVM's ConstantRange

Results

• res - Single, anonymous/composite constraint, 32-bit signless integer or 64-bit signless integer

Return op name xevm.lane_id as a bitstring.

xevm.lane_id

Attributes

• range - Optional, LLVM_ConstantRangeAttr, A range of two integers, corresponding to LLVM's ConstantRange

Results

• res - Single, anonymous/composite constraint, 32-bit signless integer or 64-bit signless integer

Return op name xevm.local_id.x as a bitstring.

xevm.local_id.x

Attributes

• range - Optional, LLVM_ConstantRangeAttr, A range of two integers, corresponding to LLVM's ConstantRange

Results

• res - Single, anonymous/composite constraint, 32-bit signless integer or 64-bit signless integer

Return op name xevm.local_id.y as a bitstring.

xevm.local_id.y

Attributes

• range - Optional, LLVM_ConstantRangeAttr, A range of two integers, corresponding to LLVM's ConstantRange

Results

• res - Single, anonymous/composite constraint, 32-bit signless integer or 64-bit signless integer

Return op name xevm.local_id.z as a bitstring.

xevm.local_id.z

Attributes

• range - Optional, LLVM_ConstantRangeAttr, A range of two integers, corresponding to LLVM's ConstantRange

Results

• res - Single, anonymous/composite constraint, 32-bit signless integer or 64-bit signless integer

Return op name xevm.local_size.x as a bitstring.

xevm.local_size.x

Attributes

• range - Optional, LLVM_ConstantRangeAttr, A range of two integers, corresponding to LLVM's ConstantRange

Results

• res - Single, anonymous/composite constraint, 32-bit signless integer or 64-bit signless integer

Return op name xevm.local_size.y as a bitstring.

xevm.local_size.y

Attributes

• range - Optional, LLVM_ConstantRangeAttr, A range of two integers, corresponding to LLVM's ConstantRange

Results

• res - Single, anonymous/composite constraint, 32-bit signless integer or 64-bit signless integer

Return op name xevm.local_size.z as a bitstring.

xevm.local_size.z

Attributes

• range - Optional, LLVM_ConstantRangeAttr, A range of two integers, corresponding to LLVM's ConstantRange

Results

• res - Single, anonymous/composite constraint, 32-bit signless integer or 64-bit signless integer

Return op name xevm.memfence as a bitstring.

xevm.memfence - Work-item's memory fence.

Attributes

• scope - Single, XeVM_MemScopeAttr, Describe memory scopes
• addrspace - Single, XeVM_AddrSpaceAttr, Describe address spaces

Description

This operation ensures that all prior memory accesses of this work-item to addrspace are visible to all other work-items in scope. Parameters description:

• scope - specify the memory scope at which all other work-items should observe memory operations prior to the fence.
• addrspace - specify the address space of work-item's memory accesses to be affected by the fence.

Return op name xevm.mma as a bitstring.

xevm.mma - Subgroup matrix multiply-add

Attributes

• shape - Single, XeVM_MMAShapeAttr,
• types - Single, XeVM_MMATypesAttr,

Operands

• a - Single, anonymous/composite constraint, fixed-length vector of 8-bit integer or 16-bit integer or 32-bit integer or 32-bit float or tf32 type or 16-bit float or bfloat16 type values of ranks 1
• b - Single, anonymous/composite constraint, fixed-length vector of 8-bit integer or 16-bit integer or 32-bit integer or 32-bit float or tf32 type or 16-bit float or bfloat16 type values of ranks 1
• c - Optional, anonymous/composite constraint, fixed-length vector of 8-bit integer or 16-bit integer or 32-bit integer or 32-bit float or tf32 type or 16-bit float or bfloat16 type values of ranks 1

Results

• d - Single, anonymous/composite constraint, fixed-length vector of 8-bit integer or 16-bit integer or 32-bit integer or 32-bit float or tf32 type or 16-bit float or bfloat16 type values of ranks 1

Description

The xevm.mma is a cooperative operation where all threads/lanes in a subgroup participates and carries out matrix multiplication plus accumulation:

D = C + A x B

where the A, B, C input matrices and the result D have shapes:

- D : MxN
- C : MxN
- A : MxK
- B : KxN

Parameters:

• a - vector of matrix A elements.
• b - vector of matrix B elements.
• c - (optional) vector of matrix C elements.
• shape - the shape of the matrices, specified as M, N, and K values.
• types - the data types of the matrices, specified as D, A, B, and optionally C.

Example:

  %d = xevm.mma %a, %b, %c { shape=<m=8, n=16, k=16>, types=<d=f32, a=f16, b=f16, c=f32> }
         : (vector<8xi16>, vector<8xi32>, vector<8xf32>) -> vector<8xf32>

Return op name xevm.mma_mx as a bitstring.

xevm.mma_mx - Subgroup matrix multiply-add with MxN shape and MX scaling

Attributes

• shape - Single, XeVM_MMAShapeAttr,
• types - Single, XeVM_MMATypesAttr,

Operands

• a - Single, anonymous/composite constraint, fixed-length vector of 8-bit integer or 16-bit integer or 32-bit integer or 32-bit float or tf32 type or 16-bit float or bfloat16 type values of ranks 1
• b - Single, anonymous/composite constraint, fixed-length vector of 8-bit integer or 16-bit integer or 32-bit integer or 32-bit float or tf32 type or 16-bit float or bfloat16 type values of ranks 1
• scale_a - Single, anonymous/composite constraint, fixed-length vector of 8-bit signless integer values of ranks 1 or 8-bit signless integer
• scale_b - Single, anonymous/composite constraint, fixed-length vector of 8-bit signless integer values of ranks 1 or 8-bit signless integer
• c - Optional, anonymous/composite constraint, fixed-length vector of 8-bit integer or 16-bit integer or 32-bit integer or 32-bit float or tf32 type or 16-bit float or bfloat16 type values of ranks 1

Results

• d - Single, anonymous/composite constraint, fixed-length vector of 8-bit integer or 16-bit integer or 32-bit integer or 32-bit float or tf32 type or 16-bit float or bfloat16 type values of ranks 1

Description

The xevm.mma_mx is similar to xevm.mma has scale operands for A and B matrices. It is a cooperative operation where all threads/lanes in a subgroup participates and carries out matrix multiplication plus accumulation:

D = C + A x B

where the A, B, C input matrices and the result D have shapes:

- D : MxN
- C : MxN
- A : MxK
- B : KxN

Parameters:

• a - vector of matrix A elements.
• b - vector of matrix B elements.
• scale_a - vector or scalar of scaling factors for matrix A.
• scale_b - vector or scalar of scaling factors for matrix B.
• c - (optional) vector of matrix C elements.
• shape - the shape of the matrices, specified as M, N, and K values.
• types - the data types of the matrices, specified as D, A, B, and optionally C.

Return op name xevm.prefetch as a bitstring.

xevm.prefetch - Prefetch data into a cache subsystem.

Attributes

• cache_control - Optional, XeVM_LoadCacheControlAttr, Describe the cache settings for load operators

Operands

• ptr - Single, anonymous/composite constraint, LLVM pointer in address space 1 or LLVM pointer in address space 4

Description

Work-item issues a prefetch from global memory to cache:

• ptr - LLVM pointer with address space. Address space must be 1 (global) or 4 (generic)
• cache_control - specify caching options

Return op name xevm.subgroup_id as a bitstring.

xevm.subgroup_id

Attributes

• range - Optional, LLVM_ConstantRangeAttr, A range of two integers, corresponding to LLVM's ConstantRange

Results

• res - Single, anonymous/composite constraint, 32-bit signless integer or 64-bit signless integer

Return op name xevm.subgroup_size as a bitstring.

xevm.subgroup_size

Attributes

• range - Optional, LLVM_ConstantRangeAttr, A range of two integers, corresponding to LLVM's ConstantRange

Results

• res - Single, anonymous/composite constraint, 32-bit signless integer or 64-bit signless integer

Return op name xevm.truncf as a bitstring.

xevm.truncf - Floating point truncation from f16/bf16 to f8/bf8/f4

Attributes

• src_etype - Single, XeVM_TruncfSrcElemTypeAttr,
• dst_etype - Single, XeVM_TruncfDstElemTypeAttr,

Operands

• src - Single, anonymous/composite constraint, fixed-length vector of 16-bit float or bfloat16 type values of ranks 1 or 16-bit float or bfloat16 type

Results

• dst - Single, anonymous/composite constraint, fixed-length vector of 8-bit signless integer or 4-bit signless integer values of ranks 1 or 8-bit signless integer or 4-bit signless integer

Description

The xevm.truncf operation truncates a floating point value from f16/bf16 to f8/bf8/f4 format.