GPU computing

Kalandra uses one source language for CPU functions and eligible GPU numerical functions. The Apple Metal backend requires macOS 26 or newer with Apple silicon. Use the mixed CPU/Metal compiler distribution and its Apple toolchain prerequisites. The experimental Vulkan and CUDA backends on Linux and Windows use the same placement and transfer APIs. Windows GPU execution is experimental and has not been verified on physical Windows GPUs.

Before you begin

Install a package that includes the backend you want, plus its system prerequisites. This example uses float32 so it needs no software-arithmetic fallback. Save it as main.k. The same source works with the backend-specific commands below; no user-written shader is required.

• GPU package and system prerequisites
• Tensor shape and element-type rules

Transfer, dispatch and observe

fn project(input: Tensor[float32, ?, ?], weights: Tensor[float32, ?, ?]) -> Tensor[float32, ?, ?]:
  matmul(input, weights)

fn main() -> float32 ! TaskError:
  try parallel:
    gpu = devices.gpu()
    input: Tensor[float32, 1, 2] = tensor([[2, 3]])
    weights: Tensor[float32, 2, 1] = tensor([[4], [5]])
    a = try await transfer(input, gpu)
    b = try await transfer(weights, gpu)
    work = spawn project(a, b) on gpu
    resident = try await work
    host = try await transfer(resident, devices.cpu())
    reduceSum(host)

The steps are explicit:

1. Construct the input and weights as tensors on the CPU.
2. Use transfer and await to upload each input to the selected GPU.
3. Start project on that GPU with spawn … on gpu.
4. Await the result. It is still a resident GPU tensor.
5. Transfer it to devices.cpu(), then use the host result.

The ? dimensions on project accept runtime extents while keeping rank two. matmul checks that the inner dimensions match. transfer returns a task, so it uses the same try await error handling as other task work.

Compile a mixed application

kalandra main.k --target metal -O2 -o app
./app/program

For this example the numerical result is 23. The Metal executable displays this float32 result as hexadecimal IEEE-754 bits: 41b80000. This represents 23, not an integer result. The executable's result display is separate from a general-purpose printing API.

The output is an application directory containing program and its metallib resources. Keep them together when moving the application. The generated application can run without the compiler distribution; compatible hardware and system frameworks remain required.

Experimental Vulkan backend

A compiler built with Vulkan support accepts the following command for the same example:

kalandra main.k --target vulkan -O2 -o app
./app/program

GPU compilation requires system C++23 development headers in addition to the compiler's private tools. Execution requires the system Vulkan loader and a compatible physical GPU driver. Keep the generated executable and its SPIR-V resources together. This backend does not provide Raspberry Pi VideoCore acceleration or imply support for every Vulkan device. Linux RTX 5090 and Intel ARL are the current physical qualification devices; Windows GPU execution remains unqualified.

Vulkan currently defaults to fixed-tree matmul. Select --matmul-policy optimized explicitly to use the full-float32 tiled matrix kernel. Its private right-operand transpose consumes an extra GPU buffer and dispatch within the normal resource limits. Matrix optimization does not enable fast math for ordinary elementwise expressions.

Native float64 depends on the device. NVIDIA RTX 5090 passes the native four-operation edge corpus. Intel float64 division currently reports an unsupported capability; select --arithmetic hardware-with-software-fallback to permit the integer GPU companion. Intel kernels without float64 division remain native. Fallback applies to float64 arithmetic for the selected kernel as a whole while retaining native float32; it is not chosen separately for each instruction. Missing or malformed shader resources are errors, not reasons to select software arithmetic. Hardware-only mode never silently selects software.

Experimental Linux CUDA backend

A Linux compiler built with CUDA support accepts the same program with a different target:

kalandra main.k --target cuda -O2 -o app
./app/program

The initial hardware qualification target is Linux x86-64 with an NVIDIA RTX 5090. The compiler emits an application directory containing program, PTX modules, required companions and third-party notices. Deploy the complete directory. Execution requires a compatible NVIDIA driver with PTX 9.2 support. An installed standalone CUDA compiler requires system C++23 development headers but no CUDA toolkit; building the compiler itself needs CUDA development headers and NVRTC. Older drivers and other NVIDIA GPUs are not yet qualified. macOS continues to use Metal.

CUDA supports the same ten numeric types and ranks zero through eight. Float32 and float64 arithmetic use hardware instructions by default. The explicit software and hardware-with-software-fallback policies retain their normal meaning; a malformed module or driver fault never triggers a numerical fallback.

Matrix products default to fixed-tree. Explicit --matmul-policy optimized permits full-float32 fused multiply-add and a different accumulation order. Eligible aligned shapes use audited BSD-3-Clause CUTLASS SIMT kernels with bounded temporary workspace and deterministic contraction partitions. Other shapes, including empty dimensions and contraction tails, use a fixed-tree GPU companion. There is no CPU, TF32, half-precision or cuBLAS substitution. Unlike the Vulkan tiled implementation, the CUDA tiled kernel consumes both source operands in row-major order.

Submitted device work retains its inputs, outputs and temporary storage through actual completion. Cancellation takes effect at completed stage boundaries; it does not reset the GPU or interrupt another process. The sanitizer CLI variants are not available for CUDA.

Experimental Windows GPU support

Windows CUDA and Vulkan support is experimental. Compiler packages for x86-64 and ARM64 have passed build and installed-package checks, including native CPU execution and GPU program compilation. Physical Windows GPU execution, numerical correctness, cancellation behavior and performance have not yet been qualified. An ARM64 compiler package does not imply that NVIDIA provides a compatible ARM64 CUDA driver.

Use the matching Visual Studio developer command prompt with C++ Build Tools and the Windows SDK. A CUDA-enabled compiler accepts --target cuda; a Vulkan-enabled compiler accepts --target vulkan. CPU-only release packages do not gain GPU compilation by passing these flags. Generated applications use program.exe and require their complete PTX or SPIR-V directory, the appropriate system loader and a compatible vendor driver. Download the experimental Windows GPU packages from GitHub Releases.

• Track Windows GPU hardware qualification

Keep intermediates resident

A numerical helper can compose matrix products, elementwise operations and reductions. A later GPU task can consume an earlier resident tensor without a host round trip. Repeatedly uploading and downloading small tensors may dominate execution cost; residence is explicit so your program can choose a useful boundary.

Host coordination and CPU workers remain separate from shader execution even when compiled from one source program. Unsupported device work is diagnosed rather than silently executed on the CPU.

Numerical and control scope

GPU numerical programs support all ten native numeric types, dense ranks zero through eight, rank-two matmul, balanced reduceSum, explicit conversions and resident numerical functions and supported control flow. Float32 uses hardware arithmetic by default. Metal float64 GPU arithmetic requires --arithmetic hardware-with-software-fallback; CPU float64 remains on hardware under that policy. --arithmetic software selects the strict software implementation. GPU helpers also support local arrays, eligible records and enums, recursive calls and closures with supported captures, including eligible callbacks created on the CPU and passed to GPU tasks. GPU tasks can return eligible arrays and aggregates to the CPU, or return tensors that stay resident on the GPU. GPU functions can raise and handle declared application errors. A caller can recover from an escaping application error after awaiting the task. Host resources stay on the CPU. See the numerical guide for the complete CPU/device capability table.

• Read the numerical contract and graph example
• Task and device function reference