com.microsoft.GatedRelativePositionBias

com.microsoft · ONNX Runtime contrib operator · contrib since_version 1

Description

Scales a precomputed relative-position bias by a per-query gate. The biased query row for each head is projected through weight and bias to D values; the first D/2 sum to one gate and the last D/2 to another, both through a logistic, and output = (gate_u * (gate_r * eco_a - 1) + 2) * rel_pos. D must be even and at most 32. All gate arithmetic is float32 whatever the tensor type, and only the store narrows.

See the ONNX Runtime GatedRelativePositionBias contrib-operator spec for the reference semantics.

Inputs

Outputs

Attributes

Attributes and default values (overridable per request):

Type constraints

Implementation variants

One implementation is selected per call from the device capabilities, the request shapes and the dtypes; these notes say what each one covers.

• row — One workgroup per output row, streaming the bias row as vec4 words when the sequence length is a multiple of four and as scalars otherwise; the route every device can select. The gate is computed once into workgroup memory, so its cost is amortized over the whole row rather than repeated per element.
• row_head_tree — Projects head elements in parallel with a tree reduction of both float32 gate sums. Used when the head dimension fills at least four head lanes per projection lane or, on adapters with a variable or unreported subgroup range, for long vec4-aligned rows with a head-to-projection ratio of at least eight. Subgroup leaders combine sums in workgroup memory.
• row_head_subgroups — Projects head elements in parallel with a subgroups reduction of both float32 gate sums. Used when the head dimension fills at least four head lanes per projection lane or, on adapters with a variable or unreported subgroup range, for long vec4-aligned rows with a head-to-projection ratio of at least eight. Subgroup leaders combine sums in workgroup memory.

Device requirements

Some implementation variants require subgroups. These are route-specific capabilities, not package-wide requirements; availability also depends on the request shape and dtype.

Files

• metadata.json — kernel metadata (id, digests, per-variant templates, provenance)
• manifest.json — the op contract (source of truth)
• test.json — correctness cases
• bench.json — benchmark cases
• gated-relative-position-bias.wgsl.jinja

Use with @huggingface/kernels

npm install --save-exact @huggingface/[email protected]

Required output shapes and logical data types are inferred from the supplied inputs and attributes; result tensors are allocated automatically.

The version: 1 option selects the published kernel contract; it is independent of any operator opset, contrib since_version, or model version. It follows the v1 branch as fixes land. To pin exact artifact bytes, pass a 40-character commit revision instead of version.

Replace each *Data placeholder with a typed array containing the corresponding input data.

import { getKernel } from "@huggingface/kernels";

const kernel = await getKernel("webgpu-kernels/com.microsoft.GatedRelativePositionBias", { version: 1 });
const { outputT } = await kernel({
  queryLayerT: { data: queryLayerTData, shape: [2, 1, 3] },
  queryBiasT: { data: queryBiasTData, shape: [3] },
  relPosT: { data: relPosTData, shape: [1, 3, 1, 1] },
  weightT: { data: weightTData, shape: [1, 2] },
  biasT: { data: biasTData, shape: [2] },
  ecoAT: { data: ecoATData, shape: [1, 3, 1, 1] },
}, {
  attrs: { num_heads: 3 },
});