CMake Keywords#
Important
With version 3.0 all Kokkos CMake keywords are prefixed with Kokkos_ which is case-sensitive.
Recall that to set a keyword in CMake you used the syntax -Dkeyword_name=value
.
This page is organized in four sections:
Backend selection#
Default State: All backends are disabled by default. This ensures you explicitly choose the backends you need for your specific hardware setup. If no backend is enabled explicitly, the Serial backend will be enabled.
Enabling Backends:
You can enable backends by configuring with -DKokkos_ENABLE_<BACKEND>=ON
flag, where <BACKEND>
is replaced with the specific backend you want to
enable (e.g. -DKokkos_ENABLE_CUDA=ON
for CUDA).
- Restrictions:
Mutual Exclusion: You can only have one device backend (e.g., CUDA, HIP, SYCL) and one host parallel backend (e.g., OpenMP, C++ threads) enabled at the same time. This is because these backends manage parallelism in potentially conflicting ways.
Host Backend Requirement: At least one host backend must always be enabled. This is because Kokkos code execution typically starts on the host (CPU) before potentially being offloaded to devices (GPUs, accelerators). If you don’t explicitly enable a host backend, Kokkos will automatically enable the Serial backend, which provides a sequential execution model.
Serial backend#
Description/info |
|
---|---|
|
To build the Serial backend targeting CPUs |
Host parallel backends#
Description/info |
|
---|---|
|
To build the OpenMP backend targeting CPUs |
|
To build the C++ Threads backend |
|
[Experimental] To build the HPX backend |
Device backends#
Description/info |
|
---|---|
|
To build the CUDA backend targeting NVIDIA GPUs |
|
To build the HIP backend targeting AMD GPUs |
|
[Experimental] To build the SYCL backend targeting Intel GPUs |
|
[Experimental] To build the OpenMP target backend |
Options#
General options#
Description/info |
Default |
|
---|---|---|
|
Build benchmarks |
|
|
Build examples |
|
|
Build tests |
|
|
Create bindings for tuning tools |
|
|
Activate extra debug features - may increase compile times |
|
|
Use bounds checking - will increase runtime |
|
|
Debug check on dual views |
|
|
Enable deprecated code in the Kokkos 3.x series |
|
|
Enable deprecated code in the Kokkos 4.x series |
|
|
Aggressively vectorize loops |
|
|
Print all compiler warnings |
|
|
Check that headers are self-contained |
|
|
Perform extra large memory tests |
|
Backend-specific options#
Description/info |
Default |
|
---|---|---|
|
Activate experimental relaxed constexpr functions |
|
|
Activate experimental lambda features |
(see below) |
|
Use CUDA LDG intrinsics |
|
|
Enable relocatable device code (RDC) for CUDA |
|
|
Use unified memory (UM) by default for CUDA |
|
|
Instantiate multiple kernels at compile time - improve performance but increase compile time |
|
|
Enable relocatable device code (RDC) for HIP |
|
Kokkos_ENABLE_CUDA_LAMBDA
default value is OFF
until 3.7 and ON
since 4.0
Third-party Libraries (TPLs)#
The following options control enabling TPLs:
Description/info |
Default |
Notes |
|
---|---|---|---|
|
Whether to enable the HWLOC library |
|
|
|
Whether to enable the LIBDL library |
|
|
|
Whether to enable the oneDPL library when using the SYCL backend |
|
|
|
Whether to enable the rocThrust library when using the HIP backend |
|
(since Kokkos 4.3) |
The following options control finding and configuring non-CMake TPLs:
Description/info |
Default |
|
---|---|---|
|
Location of CUDA install prefix for libraries |
PATH Default: |
|
Location of HWLOC install prefix |
PATH Default: |
|
Location of LIBDL install prefix |
PATH Default: |
The following options control find_package
paths for CMake-based TPLs:
Description/info |
Default |
|
---|---|---|
|
Location of HPX prefix (ROOT) or CMake config file (DIR) |
PATH Default: |
Architectures#
CPU architectures#
Description/info |
Default |
|
---|---|---|
|
Optimize for the local CPU architecture |
|
|
Optimize for ARMv8.2 with SVE Support |
|
|
Optimize for AMDAVX architecture |
|
|
Optimize for ARMV80 architecture |
|
|
Optimize for ARMV81 architecture |
|
|
Optimize for ARMV8_THUNDERX architecture |
|
|
Optimize for the ARMV8_THUNDERX2 architecture |
|
|
Optimize for BDW architecture |
|
|
Optimize for HSW architecture |
|
|
Optimize for KNC architecture |
|
|
Optimize for KNL architecture |
|
|
Optimize for POWER8 architecture |
|
|
Optimize for POWER9 architecture |
|
|
Optimize for SKX architecture |
|
|
Optimize for SNB architecture |
|
|
Optimize for Sapphire Rapids architecture |
|
|
Optimize for Zen architecture |
|
|
Optimize for Zen2 architecture |
|
|
Optimize for Zen3 architecture |
|
GPU Architectures#
NVIDIA GPUs#
The Kokkos naming convention is to aggregate the eponym of the Nvidia GPU microarchitecture and the associated CUDA Compute Capability.
Kokkos_ARCH_<MICROARCHITECTURE><COMPUTE_CAPABILITY>
If the CUDA backend is enabled and no NVIDIA GPU architecture is specified, Kokkos will attempt to autodetect the architecture flag at configuration time.
NVIDIA GPUs |
Architecture |
Compute Capability |
Models |
Notes |
---|---|---|---|---|
|
Hopper |
9.0 |
H100 |
(since Kokkos 4.0) |
|
Ada Lovelace |
8.9 |
L4, L40 |
(since Kokkos 4.1) |
|
Ampere |
8.6 |
A40, A10, A16, A2 |
|
|
Ampere |
8.0 |
A100, A30 |
|
|
Turing |
7.5 |
T4 |
|
|
Volta |
7.2 |
||
|
Volta |
7.0 |
V100 |
|
|
Pascal |
6.1 |
P40, P4 |
|
|
Pascal |
6.0 |
P100 |
|
|
Maxwell |
5.3 |
||
|
Maxwell |
5.2 |
M60, M40 |
|
|
Maxwell |
5.0 |
||
|
Kepler |
3.7 |
K80 |
|
|
Kepler |
3.5 |
K40, K20 |
|
|
Kepler |
3.2 |
||
|
Kepler |
3.0 |
K10 |
AMD GPUs#
The Kokkos naming convention is to aggregate AMD_ and the architecture flag.
Kokkos_ARCH_AMD_<ARCHITECTURE_FLAG>
If the HIP backend is enabled and no AMD GPU architecture is specified, Kokkos will attempt to autodetect the architecture flag at configuration time.
AMD GPUs |
Architecture flags |
Models |
Notes |
---|---|---|---|
|
GFX942 |
MI300A, MI300X |
(since Kokkos 4.2) |
|
GFX940 |
MI300A (pre-production) |
(since Kokkos 4.2.1) |
|
GFX90A |
MI200 series |
(since Kokkos 4.2) |
|
GFX90A |
MI100 |
(since Kokkos 4.2) |
|
GFX906 |
MI50, MI60 |
(since Kokkos 4.2) |
|
GFX1100 |
7900xt |
(since Kokkos 4.2) |
|
GFX1030 |
V620, W6800 |
(since Kokkos 4.2) |
|
GFX90A |
MI200 series |
Prefer |
|
GFX908 |
MI100 |
Prefer |
|
GFX906 |
MI50, MI60 |
Prefer |
|
GFX900 |
MI25 |
removed in 4.0 |
Intel GPUs#
CMake option |
Architecture |
Models |
---|---|---|
|
Xe-HPC (Ponte Vecchio) |
Intel Data Center GPU Max 1550 |
|
Xe-HP |
|
|
Iris Xe MAX (DG1) |
|
|
Gen12LP |
Intel UHD Graphics 770 |
|
Gen11 |
Intel UHD Graphics |
|
Gen9 |
Intel HD Graphics 510, Intel Iris Pro Graphics 580 |
|
Just-In-Time compilation* |
* Kokkos_ARCH_INTEL_GEN
enables Just-In-Time compilation for Intel GPUs
whereas all the other flags for Intel compilers request Ahead-Of-Time
compilation.
Just-In-Time (JIT) compilation means that the compiler is invoked again when the binaries created are actually executed and only at that point the architecture to compile for is determined.
On the other hand, Ahead-Of-Time (AOT) compilation describes the standard model where the compiler is only invoked once to create the binary and the architecture to compile for is determined before the program is run.