Eigen  5.0.1-dev+master
 
Loading...
Searching...
No Matches
Offloading linear algebra to a GPU with the GPU module

The contrib module contrib/Eigen/GPU runs dense and sparse linear algebra on NVIDIA GPUs from ordinary host code. It dispatches to the CUDA math libraries: cuBLAS, cuSOLVER, cuSPARSE, cuFFT, NPP, and optionally cuDSS. You write no kernels, and your source files stay regular .cpp files.

This is a different feature from using Eigen in CUDA kernels, where Eigen's fixed-size objects run inside your own kernels, one small problem per thread. Use this module to run large operations on the GPU, such as a matrix product or a factorization.

Warning
Like the other contrib modules, this module has looser API-stability guarantees than the main library; see Experimental parts of Eigen. It supports NVIDIA CUDA only, not HIP or SYCL.

Setup

The module needs the CUDA Toolkit, version 11.8 or later. Sparse direct solvers also need cuDSS, which is installed separately and requires CUDA 12.0 or later. Define EIGEN_USE_GPU before including the header, because without it the header declares none of the module:

#define EIGEN_USE_GPU
#include <Eigen/Dense>
#include <contrib/Eigen/GPU>

The module is header-only, but your program must link NVIDIA libraries. Always link -lcublas -lcublasLt -lcudart, and add the libraries of the features you use:

FeatureAdditional link flags
DeviceMatrix, products, triangular solvesnone
Dense solvers-lcusolver
norm(), DeviceScalar arithmetic, /=, cwiseProduct -lnpps -lnppc
gpu::SparseContext, gpu::DeviceSparseView -lcusparse
ConjugateGradient on a gpu::DeviceSparseView -lcusparse -lnpps -lnppc
gpu::FFT -lcufft
Sparse direct solvers (define EIGEN_CUDSS before the include)-lcudss

With CMake, the CUDAToolkit package provides these libraries as targets. This program uses the dense solvers:

find_package(CUDAToolkit REQUIRED)
target_link_libraries(app PRIVATE CUDA::cublas CUDA::cublasLt CUDA::cusolver CUDA::cudart)

For other features, add CUDA::cusparse, CUDA::cufft, CUDA::npps and CUDA::nppc as the table requires. cuDSS is not part of that package: find it with find_package(cudss) and link its cudss target.

Dense matrices

GPU and CPU code coexist in one program, because there is no global switch: you choose the GPU by using the types in the Eigen::gpu namespace. A gpu::DeviceMatrix holds a dense, column-major matrix in device memory. Move data with fromHost() and toHost(). Between those calls, expressions look like their CPU counterparts:

MatrixXd A = ..., B = ...; // A is symmetric positive definite
auto d_A = gpu::DeviceMatrix<double>::fromHost(A);
auto d_B = gpu::DeviceMatrix<double>::fromHost(B);
gpu::DeviceMatrix<double> d_C = d_A * d_B; // cuBLAS
gpu::DeviceMatrix<double> d_X = d_A.llt().solve(d_B); // cuSOLVER
MatrixXd X = d_X.toHost();
Matrix< double, Dynamic, Dynamic > MatrixXd
DynamicĂ—Dynamic matrix of type double.
Definition Matrix.h:489

The expression d_A.llt().solve(d_B) factorizes on every evaluation. To reuse a factorization, or to check for numerical failure, use a solver class instead. The classes are gpu::LLT, gpu::LU, gpu::QR, gpu::SVD and gpu::SelfAdjointEigenSolver:

gpu::LLT<double> llt;
llt.compute(d_A);
if (llt.info() != Success) { ... }
gpu::DeviceMatrix<double> d_X1 = llt.solve(d_B1); // reuses the factor on the device
gpu::DeviceMatrix<double> d_X2 = llt.solve(d_B2);
@ Success
Definition Constants.h:457

The solvers also accept host matrices and handle the transfers themselves, which makes them the smallest change from CPU code:

gpu::LLT<double> llt(A); // instead of Eigen::LLT<MatrixXd> llt(A);
MatrixXd X = llt.solve(B);

Sparse matrices

Define EIGEN_CUDSS to enable gpu::SparseLLT, gpu::SparseLDLT and gpu::SparseLU. They take an ordinary SparseMatrix and host vectors:

#define EIGEN_USE_GPU
#define EIGEN_CUDSS
#include <Eigen/Sparse>
#include <contrib/Eigen/GPU>
SparseMatrix<double> A = ...;
VectorXd b = ...;
gpu::SparseLLT<double> llt(A); // instead of SimplicialLLT<SparseMatrix<double>> llt(A);
if (llt.info() != Success) { ... }
VectorXd x = llt.solve(b);
Matrix< double, Dynamic, 1 > VectorXd
DynamicĂ—1 vector of type double.
Definition Matrix.h:489

As with the CPU solvers, a sequence of matrices with one sparsity pattern can share the symbolic analysis, by calling analyzePattern() once and factorize() for each matrix.

For sparse matrix-vector products, gpu::SparseContext uses cuSPARSE. Its deviceView() also lets Eigen's own ConjugateGradient run on the GPU, with the identity preconditioner and real scalars:

#include <Eigen/IterativeLinearSolvers>
gpu::SparseContext<double> spmv;
auto mat = spmv.deviceView(A);
auto d_b = gpu::DeviceMatrix<double>::fromHost(b);
gpu::DeviceMatrix<double> d_x(A.rows(), 1);
d_x.setZero();
ConjugateGradient<gpu::DeviceSparseView<double>, Lower | Upper, IdentityPreconditioner> cg;
cg.compute(mat); // stores a pointer: mat and spmv must outlive cg
cg.solveWithGuessInPlace(d_b, d_x); // d_x holds the initial guess
@ Lower
Definition Constants.h:212
@ Upper
Definition Constants.h:214

A gpu::SparseContext caches one matrix at a time, so another deviceView() or a multiply() with a host matrix invalidates its earlier views. Use one context per matrix.

Asynchronous execution and threads

Operations are asynchronous by default. The host waits only where it must: in fromHost(), toHost(), info(), when a gpu::DeviceScalar, the result type of dot() and norm(), converts to a host scalar, and in the forms that return host data, such as solve() on a host matrix, and gpu::FFT and gpu::SparseContext::multiply() on host vectors. The dense solvers also wait when compute() takes a host matrix, and an internal workspace that has to grow waits for the work using it. Debug builds also wait to check the factorization, on the first solve() after compute() and on each llt().solve(). Transfers are usually the main cost, so keep intermediate results on the device and download only what the host needs.

Each thread gets a default gpu::Context, which holds a CUDA stream and the library handles. One Context is not thread-safe. To run work on separate streams, or to share a stream with your own CUDA code, create contexts explicitly. DeviceMatrix synchronizes across streams automatically:

gpu::Context ctx1, ctx2;
d_C1.device(ctx1) = d_A1 * d_B1; // runs on stream 1
d_C2.device(ctx2) = d_A2 * d_B2; // runs on stream 2, concurrently
gpu::Context ctx3(my_existing_stream); // borrows the stream without owning it

Errors

There are two kinds of failure:

  • A failed CUDA or library call stops the program, in release builds as well as debug builds. To handle such failures yourself, for instance by throwing an exception, define the macro EIGEN_GPU_CHECK_FAILED(error, expression, file, line) before including the module.
  • A numerical failure, such as a matrix that is not positive definite, is reported by info(). Release builds do not check it for you, and the one-shot llt().solve() form has no info(), so use a solver class and call info() where failure is possible. cuSOLVER 11.4.1 and 11.4.2, from CUDA 11.8 and 12.0, do not detect a non-positive-definite input to gpu::LLT: info() reports success and the factor contains NaNs. CUDA 12.1 and later report it.

Limitations

  • DeviceMatrix is not an Eigen expression type and does not derive from MatrixBase, so it cannot be passed to code templated on MatrixBase. It also has no coefficient access.
  • Each expression dispatches to NVIDIA library calls, usually one: norm() is a dot product and an NPP square root, and llt().solve() is a copy, a factorization and a solve. Unlike CPU expressions, GPU expressions do not fuse into a single loop, and an unsupported expression is a compile error.
  • Scalars are float, double, std::complex<float> and std::complex<double>, unless a class states otherwise.
  • Numerical behavior and performance are those of the NVIDIA libraries. In particular, the default compute types let cuBLAS choose faster algorithms that might not be bit-reproducible; define EIGEN_NO_CUDA_TENSOR_OPS to disable them.

Further reference

The module README is the complete reference. It covers the BLAS operations, FFT, solver configuration, stream control, and the mapping from each expression to its library calls.