Python GPU Libraries: CuPy, Numba, and RAPIDS Ecosystem for Data Science on GPU

CuPy provides a NumPy-compatible API executed on GPU, making it the easiest entry point for GPU-accelerated array computing. The `import cupy as cp` drop-in replacement for `import numpy as np` enables GPU execution for array operations. CuPy 13.x uses CUDA 12.4+ and cuTENSOR (NVIDIA's tensor contraction library) for efficient linear algebra, achieving 10-100x speedups over NumPy on H100 for common operations. A 10,000 x 10,000 matrix multiply on H100 takes 28ms with CuPy versus 3,400ms with NumPy on a 32-core CPU (121x speedup). Element-wise operations like `cupy.sin(x)` benefit from CUDA kernel fusion: CuPy's JIT compiler fuses multiple element-wise operations into a single kernel, reducing kernel launch overhead. `cupyx.jit.rawkernel` and `cupyx.jit.kerngen` provide user-defined CUDA kernel compilation at runtime, generating PTX code optimized for the target GPU architecture.

The memory management in CuPy uses a memory pool (`cupy.cuda.MemoryPool`) that caches GPU allocations to avoid the expensive `cudaMalloc`/`cudaFree` cycle. The pool allocates memory in blocks, with `cupy.cuda.set_allocator(cupy.cuda.MemoryPool().malloc)` set as the default allocator. This reduces allocation overhead from 5-50 microseconds per call to 0.1-0.5 microseconds. The pool's `free_block` callback on memory pressure triggers `cudaFree` only when the pool exceeds `cupy.cuda.memory.mempool.set_limit(size=80*1024**3)` (80 GB for H100). CuPy also supports unified memory (`cupy.cuda.alloc.UMalloc`) for data exceeding GPU VRAM, automatically migrating pages between GPU and CPU memory. For data science workflows processing 200 GB datasets on a single H100, unified memory with `cupy.cuda.runtime.setDevice(0)` enables processing beyond VRAM capacity with 40-60% performance of fully-resident data.

Numba is a JIT compiler that translates a subset of Python and NumPy to CUDA kernels using LLVM. A kernel is defined with `@cuda.jit` decorator and executed with `kernel[blocks, threads](args)`. The kernel launch configuration for H100 uses `threads=256` and `blocks=math.ceil(n / 256)` for data-parallel operations. Numba's CUDA backend generates PTX code optimized for the target GPU architecture (sm_90 for H100) through LLVM's NVPTX backend. Compilation occurs on the first invocation and is cached in `~/.numba/cache/` for subsequent runs. Numba 0.60+ supports FP16 arithmetic (`numba.float16`), improving memory bandwidth utilization for half-precision data science workloads.

For data science operations not covered by CuPy or RAPIDS, Numba enables custom GPU kernels. A custom k-means kernel in Numba achieves 2,100ms for 10M points x 128 dimensions x 100 clusters on H100, versus 15,200ms for scikit-learn on CPU (7.2x speedup). The kernel implements the assignment step in a single CUDA block with shared memory for centroid storage, reducing global memory reads from 128 to 2 per point per iteration. Numba's `cuda.to_device()` transfers data to GPU and `cuda.from_device()` retrieves results. The `cuda.devicearray` object supports zero-copy access to CuPy and RAPIDS arrays, enabling mixed workflows where CuPy handles array ops and Numba handles custom compute. The integration is seamless: `distances = cupy.zeros((n_points, n_clusters))` and `numba_kernel[blocks, threads](distances, points, centroids)` operates on the same GPU memory.

The RAPIDS ecosystem is the most comprehensive GPU data science platform, covering the full workflow: cuDF (DataFrames, pandas-compatible), cuML (ML algorithms, scikit-learn-compatible), cuGraph (graph analytics, NetworkX-compatible), cuSpatial (geospatial), and cuCIM (image processing). cuDF 24.x achieves 40-80x speedup over pandas for common DataFrame operations. A `df.groupby("category").agg({"value": ["sum", "mean", "std"]})` on 100M rows completes in 120ms with cuDF versus 8,400ms with pandas on 32-core CPU (70x speedup). The key optimization is the GPU-based hash table implementation for groupby operations, using a lock-free hash table in shared memory with Cuckoo hashing for collision resolution.

The integration between CuPy and RAPIDS is a core design principle. `cudf.DataFrame.to_cupy()` returns a CuPy array for numerical operations, and `cupy.array.to_cudf()` converts back. This enables workflows like: load data with cuDF, convert to CuPy for matrix operations, train with cuML, and store results back in cuDF. A complete GPU-accelerated XGBoost training pipeline: `import xgboost as xgb; dtrain = xgb.QuantileDMatrix(cudf_data.values, label=cudf_labels.values); model = xgb.train({"device": "cuda", "tree_method": "gpu_hist"}, dtrain)`. XGBoost 2.1+ uses the RAPIDS memory allocator (`RAPIDS_MEMORY_MANAGER="managed"`) which pools GPU memory across the entire RAPIDS stack, reducing peak memory usage by 25-35% versus each library using its own allocator.

Data science datasets (hundreds of GB to TB) often exceed GPU VRAM, requiring memory hierarchy management. The RAPIDS `ucx` and `dask-cuda` libraries extend GPU data processing across multiple GPUs and nodes. Dask cuDF partitions the DataFrame across GPUs using Dask's task graph, with each partition processed on its local GPU. For a 500 GB CSV dataset on 8x H100 (640 GB total GPU memory), Dask cuDF distributes the data at 62.5 GB per GPU and executes operations in parallel with near-linear scaling. The `dask_cuda.LocalCUDACluster(n_workers=8, device_memory_limit=0.8)` sets per-worker memory limit to 80% of GPU memory (64 GB), spilling to host memory when exceeded. The spill-to-host penalty is 20-40% for workloads exceeding the limit.

For single-GPU workflows with datasets larger than VRAM, cuDF 24.10+ introduces `cudf.pandas` mode, which provides automatic GPU fallback. Enabled by `import cudf.pandas`, this mode attempts to execute DataFrame operations on GPU and falls back to pandas on CPU for operations exceeding VRAM or lacking GPU implementation. In benchmarks with the 200 GB NYC taxi dataset on an H100 80 GB, `cudf.pandas` mode achieves 80% of full cuDF performance for operations fitting in VRAM and seamlessly falls back to CPU pandas for out-of-core operations. The fallback overhead is 5-15% for hybrid GPU/CPU workflows, representing a massive usability improvement versus manually partitioning the dataset. For the most cost-effective configuration on ClusterBid, an 8x H100 Dask cuDF cluster at $20/hr processes 2 TB datasets in minutes, replacing 40-80x CPU instances at comparable or lower cost.

The RAPIDS ecosystem is containerized for production GPU deployment. The `rapidsai/docker` repository provides optimized Docker images: `rapidsai/rapidsai:24.12-cuda12.4-runtime-ubuntu22.04-py3.11` (8.5 GB compressed). This image includes all RAPIDS libraries, CuPy, Numba, XGBoost, LightGBM, and cuML, pre-linked against CUDA 12.4. For production, the slim variant `rapidsai/rapidsai-core:24.12-cuda12.4-runtime` (3.2 GB) includes only cuDF, CuPy, and Numba, allowing custom ML library installation. RAPIDS images should always use `NVIDIA_VISIBLE_DEVICES=all` and `NVIDIA_DRIVER_CAPABILITIES=compute,utility` to ensure CUDA and cuDNN access. Kubernetes deployments use `resources.limits["nvidia.com/gpu"]: 1` with the RAPIDS image.

The cost efficiency of GPU data science versus CPU clusters is compelling. A single H100 at $2.50/hr running cuDF groupby achieves 70x speedup versus a 32-core CPU instance at $1.20/hr. The GPU costs 2.1x more but completes the workload 70x faster, delivering a 33x cost-performance advantage. For ETL workloads processing 10 TB daily, a GPU cluster reduces processing time from 8 hours (32-core CPU cluster, $9.60) to 7 minutes (1x H100, $0.29), a 33x cost reduction. The RAPIDS ecosystem is production-ready for GPU-accelerated data science and is deployed on H100 and A100 clusters across finance, healthcare, and e-commerce. On ClusterBid, H100 instances for RAPIDS workflows cost $2.35-2.80/hr with 80 GB HBM sufficient for most data science workloads.