Text-to-Speech: Workload Profile
Text-to-Speech workloads have distinct GPU requirements compared to standard LLM inference. Models range from 0.5B-5B parameters. The primary performance bottleneck is Compute-bound, sequential generation. Key metrics: tokens/second, latency P50/P95/P99, batch throughput, and memory utilization. Production deployments typically use 1-2 GPUs per model GPUs.
Recommended GPU Configurations
Recommended GPUs for Text-to-Speech: L4, L40S, A10, T4. For models in the 0.5B-5B range, GPU selection depends on: model architecture (encoder-only, decoder-only, encoder-decoder, or diffusion), precision requirements (FP16/FP8/INT4), latency SLAs (real-time vs batch), and throughput targets. Batch sizes of Batch size 1-8 for real-time TTS are typical for production throughput optimization.
VRAM Requirements
VRAM needs for Text-to-Speech vary by model. A 0.5B-5B model at FP16 requires approximately 1 GB for model weights. KV cache for encoder-decoder architectures may require additional 1-4 GB per sequence. Diffusion models additionally need latent space working memory. INT4 quantization reduces weight memory by 75% but increases compute requirements.
Throughput and Latency Expectations
Typical throughput for Text-to-Speech on recommended hardware: varies by batch size. For real-time inference, latency targets should be 200-500ms P99. Batch inference can process at 2-10x real-time throughput depending on model complexity and GPU configuration. Production systems should maintain GPU utilization above 70% for cost efficiency.
Production Architecture Patterns
Production Text-to-Speech deployment patterns include: dedicated GPU instances per model version; pooled GPU clusters with dynamic model loading; autoscaling based on queue depth or GPU utilization; GPU-backed serverless inference for variable workloads; and multi-model serving on shared GPU instances using model parallelism or MIG partitioning.
Cost Optimization Strategies
Optimize costs for Text-to-Speech workloads: choose spot/preemptible GPUs for batch inference with checkpointing; use reserved instances for baseline traffic with on-demand overflow; implement GPU autoscaling to minimize idle capacity; use model quantization to reduce GPU requirements by 2-4x; and batch process non-real-time workloads during off-peak pricing periods.