Why blended request cost hides modality drift
Multimodal requests combine inputs with very different processing costs. A short text prompt may need a few hundred tokens of prefilling, while two attached images may expand into thousands of vision tokens plus encoder work, plus audio minutes may add a separate encoder pass. A single blended price per request hides which modality drives GPU time.
That hiding matters when traffic shifts. A product that starts text heavy can drift image heavy after a feature launch, and the old price tier quietly subsidizes the new cost center. Support teams then see margin erosion without a clear cause, because aggregate tokens per request still look flat while vision tokens per request climb.
Modality aware tracing assigns GPU seconds to text, image, and audio inputs using counted preprocessed units. The goal is not perfect causal attribution, since encoders, attention, and decoding interact. The goal is a repeatable allocation that follows measured execution time closely enough to guide pricing and capacity choices.
Define countable input units per modality
Count preprocessed units per modality before the main model runs. For text, count input tokens after tokenization. For images, count images, pixels, tiles, or vision tokens, depending on what the serving stack exposes. For audio, count seconds of audio and output frames or tokens from the audio preprocessor. The vLLM documentation on multimodal inputs shows how image, video, and audio payloads enter requests as distinct fields, which makes per modality counting practical at the gateway.
Emit those counts as span attributes on the request trace. The OpenTelemetry trace semantic conventions describe how spans carry operation specific attributes across polyglot services, which is the right pattern here: one inference span with text tokens, image units, and audio seconds attached. Keep the raw counts, not only ratios, so later pricing math can be recomputed.
Standardize units per model version. Vision token counts change when tile size or resolution limits change, and audio frame counts change when sample rate handling changes. Version the counting method with the model and preprocessor release, or historical comparisons will confuse a tokenizer change with a traffic change.
Attribute GPU seconds with a calibrated key
Measure GPU seconds per request from the serving engine, then split that total across modalities with a transparent allocation key. A practical key uses timed components where available: audio encoder seconds to audio, vision encoder seconds to image, and remaining prompt plus decode seconds split by token share. Where component timers are unavailable, fall back to calibrated weights from a controlled timing run.
Calibrate weights by sending single modality probes at representative sizes and recording GPU seconds per unit. For example, an illustrative timing run might find 0.0008 GPU seconds per text token, 0.012 GPU seconds per image tile, and 0.05 GPU seconds per audio second. Those figures are illustrative assumptions for showing the arithmetic, not measured performance claims.
Apply the weights to each mixed request to get attributed seconds per modality, then reconcile to measured total GPU seconds with a single scaling factor. If the weighted sum predicts 1.40 GPU seconds and the engine measured 1.54, multiply each modality share by 1.10. This keeps the trace math consistent with billing meters while preserving the modality proportions from the calibration.
| Modality | Count unit | Timer to prefer | Fallback weight source |
|---|---|---|---|
| Text | Tokenized input tokens | Prefill seconds by token share | Single modality text probes |
| Image | Tiles or vision tokens plus pixel count | Vision encoder seconds | Image only probes at standard resolutions |
| Audio | Audio seconds and encoder frames | Audio encoder seconds | Audio only probes at standard sample rates |
Worked example with a mixed request
Consider an illustrative mixed request with 600 text tokens, 8 image tiles, and 20 seconds of audio. Using the illustrative weights above, text contributes 0.48 GPU seconds, image contributes 0.096 GPU seconds, and audio contributes 1.00 GPU second, for a weighted sum of 1.576 GPU seconds. If the engine measured 1.70 GPU seconds, the reconciliation factor is 1.079, giving attributed totals of about 0.52 for text, 0.10 for image, and 1.08 for audio.
Convert to money only with explicitly hypothetical rates. Suppose a planning assumption of 2.00 dollars per GPU hour, stated as hypothetical with actual prices varying by provider and contract. Then 1.70 GPU seconds costs about 0.00094 dollars, because 1.70 divided by 3600 times 2.00 equals 0.00094. Audio carries about 64 percent of that cost in this example, even though the request looked text heavy by character count.
Repeat that arithmetic across a week of traces to get modality share of total GPU seconds. Per request examples explain the method, but pricing tiers should follow aggregate shares. A handful of dramatic audio requests should not set the price for a mostly text workload.
Turn attribution into targeted optimization
Attribute before optimizing. When image share dominates, test resolution caps, tile limits, thumbnail first flows, and prefix caching for repeated images. The vLLM multimodal inputs documentation describes practical controls such as multimodal limits per prompt and caching behavior that change how much vision work each request triggers. Measure attributed seconds again after each change.
When audio share dominates, test sample rate normalization, silence trimming, chunk length tuning, and transcript caching for repeated clips. When text share dominates, test prompt compression and output length controls. Each optimization should move its own modality line without relying on blended averages.
Limits of modality attribution
Allocation is an estimate, not a physical separation. Shared attention, shared KV cache, and batched execution mean the true marginal cost of adding one image to a mixed batch differs from its standalone probe cost. Treat attributed seconds as a planning signal with error bars, especially under continuous batching.
Probes also decay. Tokenizer updates, vision encoder swaps, quantization changes, and driver updates all shift the weights. Recompute the calibration monthly or after any serving stack change, and keep prior weight versions so cost history can be restated.
Finally, cost attribution does not settle fairness on its own. A modality with high GPU share may still deserve subsidy when it drives adoption or retention. Use the trace data to quantify the subsidy explicitly, then decide whether product strategy justifies it.