H100, A100 and RTX 4090 are the three most compared GPUs in AI projects. We explain the differences in architecture, memory, FP8 and NVLink, why a card with a higher hourly price can be cheaper per job, and which one to choose in which scenario.
One of the most common questions from people looking to rent a GPU is this: H100, A100 or RTX 4090? All three are widely used for AI workloads, but they come from different generations and different markets, and their hourly prices differ significantly.
In this article we explain the technical differences between the three cards, why cost should not be measured by hourly price alone, and which card makes more sense in which scenario. In the final section we also look at alternatives such as L40S and H200.
Three Cards, Three Different Worlds
NVIDIA A100 is a data centre card based on the Ampere architecture and was the standard for AI training for years. Its software ecosystem is very mature; almost every library and training script has been tested on A100.
NVIDIA H100 is the A100's successor, built on the Hopper architecture. Thanks to the Transformer Engine designed for transformer models and FP8 support, it offers a clear leap in large language model training and inference.
NVIDIA RTX 4090 is a consumer card based on the Ada Lovelace architecture. Although it was designed for gaming and content creation, its high compute and relatively low price make it a popular choice for experiments, small and medium model inference and image generation.
Technical Comparison
| Feature | H100 80 GB | A100 80 GB | RTX 4090 24 GB |
|---|---|---|---|
| Architecture | Hopper | Ampere | Ada Lovelace |
| Memory | 80 GB HBM3 (SXM version) | 80 GB HBM2e | 24 GB GDDR6X |
| Memory bandwidth | Very high (HBM3) | High (HBM2e) | Moderate (GDDR6X) |
| FP8 support | Yes (Transformer Engine) | No | Yes (Ada Tensor Cores) |
| BF16 / FP16 | Yes | Yes | Yes |
| NVLink | Yes | Yes | No |
| MIG (GPU partitioning) | Yes | Yes | No |
| Class | Data centre | Data centre | Consumer |
| CloudRUX starting price (hourly, excl. VAT) | $4.99 | $2.24 | $1.11 |
Some of these rows make a big difference in practice. Let us go through them one by one.
Memory: The Gap Between 80 GB and 24 GB
In AI workloads, VRAM is the first filter. FP16 inference needs about 2 bytes per parameter. That means an 8B model's weights take about 16 GB and fit in the RTX 4090's 24 GB with a small margin for KV cache. For 13B models and larger, the 4090 only works with quantisation.
The 80 GB on A100 and H100 opens up much more room: running mid-sized models at full precision, using long context windows, training LoRA with larger batch sizes and serving 70B models on a single card with 4-bit quantisation all become possible.
Bandwidth matters too. In language model inference especially, token generation speed depends heavily on memory bandwidth. H100's HBM3 memory is the strongest option here, and A100's HBM2e also offers significantly higher bandwidth than the 4090's GDDR6X.
FP8 and the Transformer Engine
One of the most important features separating H100 from A100 is FP8 support. FP8 represents numbers with 8 bits, reducing both memory use and compute time. H100's Transformer Engine manages which layers can safely run in FP8 during training, trying to limit accuracy loss.
A100 does not support FP8; its lowest common training precision is BF16 and FP16. The RTX 4090's Ada Tensor Cores support FP8 in hardware, but with 24 GB of memory and a consumer positioning it is not in the same league as H100 for large-scale training.
In short, H100 is substantially faster than A100 for transformer-based model training, and the gap widens further with software that can use FP8. The exact speed difference depends on the model, library and settings, so we recommend a short measurement with your own workload.
NVLink and Multi-GPU Work
Models that do not fit on a single card run across multiple GPUs. Data traffic between GPUs then directly affects performance. A100 and H100 support NVLink, which provides high-speed links between GPUs. The RTX 4090 has no NVLink; cards communicate over PCIe.
So for multi-GPU full training or serving large models split across several cards, A100 and H100 are clearly better suited. The RTX 4090 shines in jobs that can be done on a single card.
Cost Per Job, Not Hourly Price
The most common mistake in GPU selection is looking only at hourly price. What really matters is the total cost of finishing a job: cost per job = time x hourly price.
H100's hourly starting price ($4.99) is about 2.2 times that of A100 ($2.24). So if H100 runs your job more than 2.2 times faster than A100, H100 is cheaper overall; if not, A100 has the edge. The table below shows this logic with hypothetical speed ratios. These ratios are not measured benchmark results, just examples chosen to show how the calculation works.
| Scenario | A100 time and cost | H100 time and cost | Result |
|---|---|---|---|
| H100 1.5x faster (assumption) | 30 hours x 2.24 = $67.20 | 20 hours x 4.99 = $99.80 | A100 more economical |
| H100 2x faster (assumption) | 30 hours x 2.24 = $67.20 | 15 hours x 4.99 = $74.85 | A100 slightly cheaper, H100 finishes sooner |
| H100 3x faster (assumption) | 30 hours x 2.24 = $67.20 | 10 hours x 4.99 = $49.90 | H100 both faster and cheaper |
Prices exclude VAT. You should also factor in the value of time: finishing a training run in one day instead of two lets your team run more experiments and ship faster. That is why H100 is often a strong candidate for long transformer training runs that can use FP8.
The same logic applies to the RTX 4090. If the model fits in 24 GB and the job can be done on one card, its $1.11 starting price can deliver the lowest cost per job. If it does not fit, compromises such as quantisation or smaller batches lengthen the run and the advantage may disappear.
Which One for Which Scenario?
Choose the RTX 4090 if
- You are validating an idea, developing code or running short experiments,
- You serve small models such as 7B-8B or run QLoRA training on models up to 13B,
- You run visual workloads that fit on one card, such as Stable Diffusion, ComfyUI or 3D rendering.
Choose the A100 if
- You need 80 GB of memory but do not use FP8,
- You want a mature, widely tested software environment,
- You run budget-sensitive LoRA training without time pressure, or 70B 4-bit inference.
Choose the H100 if
- You run long transformer training jobs,
- Your software can use FP8,
- Fast results and short experiment cycles matter to you.
You can compare all cards and starting prices on our GPU cloud page.
Alternatives: L40S and H200
L40S 48 GB ($1.36 starting): a data centre card based on the Ada architecture with FP8 support. It is strong for inference and image generation. When the 4090's 24 GB is not enough but you do not need A100 or H100, for example for mid-sized LLM services and high-resolution image generation, it is a good middle ground.
H200 141 GB ($6.61 starting): a version of the Hopper architecture with larger and faster memory. Its 141 GB of HBM3e makes it easier to run large models split across fewer GPUs. It stands out when H100's memory is not enough, for example for serving 70B-class models at higher precision or LoRA training on large models.
For heavier jobs you can also consider the newest-generation B200 180 GB ($9.99 starting), the RTX PRO 6000 with 96 GB ($3.11 starting) and the economical 48 GB A40 ($0.74 starting).
Practical Notes
Whichever card you choose, you can access CloudRUX GPU servers over SSH and Jupyter, start immediately with ready PyTorch/CUDA templates or use your own container image. Running the same image on different cards makes comparison measurements fair and quick.
The temporary disk is deleted when the server stops; keep model weights and results on a persistent volume or back them up to S3 object storage. Because the servers run in our partner's data centers abroad, workloads with personal data can count as a cross-border transfer under KVKK; we recommend working with non-personal or anonymised data and help with standard contract processes when needed.
Frequently Asked Questions
Can I train a large language model on an RTX 4090?
Fine-tuning small models with QLoRA is possible. But with 24 GB of memory and no NVLink, it is not suitable for full training of large models or multi-GPU training.
Is the A100 outdated, or does it still make sense?
A100 is an older generation, but with 80 GB of memory and mature software support it still makes sense for many jobs. Its price advantage stands out especially in jobs that do not use FP8 and are not time-critical.
Is H100 always more expensive?
Per hour, yes, but not always per job. If your job runs fast enough on H100 compared with A100, total cost can be lower.
Are the prices final?
The figures are hourly starting prices per GPU, excluding VAT. Exact price and capacity are confirmed in a quote, and monthly reserved usage is discounted.
Rent H100, A100 and RTX 4090 with CloudRUX
The CloudRUX GPU service is in early access. You can rent H100, A100, RTX 4090 and other cards by the hour, with e-invoicing in Turkey and Turkish-speaking engineer support. Explore the cards on our GPU cloud page, or get in touch to choose the right card for your workload together and request a quote.