ORACLE'S GPU PHILOSOPHY: BARE METAL BY DEFAULT
Oracle Cloud Infrastructure (OCI) took a contrarian approach to GPU compute: instead of virtualized GPU instances sharing a hypervisor, OCI offers bare metal GPU instances where the tenant owns the entire physical host. The BM.GPU.H100.8 instance provides 8x H100 80 GB SXM GPUs with NVLink 3.0, 4-core AMD EPYC processors dedicated to the tenant, and 2,048 GB system RAM. There is no virtualization layer overhead (typically 3-5 percent for GPU workloads on hypervisor-based instances), and the GPUs have direct PCIe passthrough without intermediate translation.
The bare metal approach affects the pricing model significantly. OCI's BM.GPU.H100.8 at $18.28/hr for the full 8-GPU box (effective $2.29/GPU/hr) undercuts AWS P5 on-demand ($4.10/GPU/hr) by 44 percent, GCP A3 High ($3.91/GPU/hr) by 41 percent, and Azure ND H100 v5 ($4.27/GPU/hr) by 46 percent. Even when compared to competing providers' reserved pricing, OCI's on-demand bare metal pricing beats or matches hyperscaler 3-year reservation rates.
| OCI Instance | Hardware | GPU Count | On-Demand $/hr | $/GPU/hr | vs AWS % | vs GCP % |
|---|---|---|---|---|---|---|
| BM.GPU.H100.8 | 8x H100 80GB | 8 | $18.28 | $2.29 | -44% | -41% |
| BM.GPU.A100.8 | 8x A100 80GB | 8 | $12.80 | $1.60 | -35% | -32% |
| BM.GPU4.8 | 8x A100 40GB | 8 | $9.96 | $1.25 | -49% | -46% |
| BM.GPU.L40S.4 | 4x L40S 48GB | 4 | $5.18 | $1.30 | N/A | N/A |
| VM.GPU.A100.8 | 8x A100 80GB VM | 8 | $14.08 | $1.76 | -28% | -25% |
| VM.GPU.H100.8 | 8x H100 80GB VM | 8 | $20.11 | $2.51 | -38% | -36% |
REAL-WORLD PERFORMANCE: BARE METAL VS HYPERVISOR GPU
OCI's bare metal advantage is measurable in production benchmarks. Training Llama-2 7B on 8 GPUs (FSDP, BF16) across 1000 steps: OCI BM.GPU.H100.8 completes in 28.4 minutes versus 30.2 minutes on AWS P5 (6.3 percent faster). The difference is the hypervisor tax: AWS Nitro hypervisor intercepts PCIe transactions for GPU memory access, adding 3-8 percent latency on GPU memory operations. OCI's bare metal provides direct GPU memory access, eliminating this overhead.
For multi-node training, OCI uses RoCE v2 (RDMA over Converged Ethernet) with 1,600 Gbps per node. This is competitive with AWS EFA but lacks the GCP Jupiter advantage. At 64 GPUs (8 nodes), OCI achieves 86 percent scaling efficiency versus AWS P5 at 88 percent and GCP A3 at 91 percent. OCI's inter-node networking is adequate for most training workloads but trails Google's custom network for large-scale distributed training.
| Benchmark (8 GPU) | OCI BM.H100.8 | AWS P5.48x | GCP A3 High | Azure NDv5 | |
|---|---|---|---|---|---|
| Llama-2 7B (1K steps) | 28.4 min | 30.2 min | 29.5 min | 30.8 min | 6% faster |
| GPT-2 1.5B (FP16) | 2,850 tok/s | 2,710 tok/s | 2,780 tok/s | 2,690 tok/s | 5% faster |
| Stable Diffusion XL | 4.8 im/s | 4.5 im/s | 4.6 im/s | 4.4 im/s | 7% faster |
| Memory BW (H2D) | 205 GB/s | 185 GB/s | 195 GB/s | 182 GB/s | 11% better |
NETWORKING AND STORAGE COST DIFFERENTIALS
OCI's networking cost structure is a hidden advantage. Data egress from OCI GPU instances to the internet is free for the first 10 TB/month per OCI tenancy (AWS charges $0.05-0.09/GB for the first 10 TB). For training workloads generating large checkpoint files (500 GB - 2 TB per run), this saves $50-180 per run versus AWS. Inter-region transfer is also 25-40 percent cheaper on OCI.
Block storage for GPU instances (boot volumes for model weights, datasets) costs $0.042/GB-month for block volumes with 200 MB/s baseline throughput, versus AWS gp3 at $0.08/GB-month. For a training dataset of 10 TB, storage costs $420/month on OCI versus $800/month on AWS. These infrastructure cost differentials compound the GPU compute savings, making OCI's total cost of AI infrastructure 35-50 percent below the hyperscaler average.
REGIONAL AVAILABILITY AND CAPACITY LIMITATIONS
OCI's GPU availability is the primary constraint. BM.GPU.H100.8 instances are available in 8 OCI regions: US East (Ashburn), US West (Phoenix), London, Frankfurt, Amsterdam, Mumbai, Seoul, and Sao Paulo. This is more limited than AWS (4 P5e regions) and far below Azure's 16-region ND-series deployment. Capacity is notably tight in Ashburn and Phoenix (wait times 3-6 weeks for 32+ GPU clusters).
OCI does not offer spot GPU pricing, which is a significant disadvantage for cost-sensitive training workloads. The company argues that bare metal instances cannot be interrupted cleanly for spot preemption, but this stance eliminates a major cost-saving mechanism that AWS (75-85 percent off) and GCP (60-70 percent off) provide. OCI's 1-month and 1-year commitments offer 20 percent and 33 percent discounts respectively, less aggressive than hyperscaler reserved instances.
| Feature | OCI BM.GPU | AWS P5 | GCP A3 | Azure ND |
|---|---|---|---|---|
| H100 Regions | 8 | 4 | 3 | 16 |
| Spot Pricing | No | Yes (85% off) | Yes (70% off) | Yes (72% off) |
| Max Cluster Size | 512 GPUs (req) | 20K+ | 16K+ | Unlimited |
| Typical Wait 64 GPU | 3-6 weeks | 1-3 weeks | 1-3 weeks | 1-2 weeks |
| Commitment Discount | 33% (1yr) | 53% (3yr) | 52% (3yr) | 58% (3yr) |
WHEN ORACLE CLOUD GPU MAKES SENSE
OCI's GPU offering is optimal for three specific profiles. First, teams with predictable steady-state GPU usage where reserved or monthly commitments cover 80+ percent of demand, maximizing OCI's 30-50 percent price advantage over hyperscalers. Second, GPU workloads sensitive to hypervisor overhead, including memory-bandwidth-bound inference serving and latency-sensitive GPU computing. Third, organizations already running Oracle databases or enterprise applications who can use OCI's unified billing and cross-service discounts.
OCI is suboptimal for bursty training workloads that benefit from spot instance pricing (no spot available), teams needing the broadest regional footprint for global inference deployment (OCI has 8 GPU regions versus 16+ on Azure), or workloads requiring multi-10K GPU clusters (OCI's capacity allocation process caps most tenants at 512 GPUs). The bare metal advantage also matters less for inference workloads where hypervisor overhead is a smaller fraction of total latency.