Renting a GPU server is now the fastest way to start an AI project. We explain which GPU makes sense for which job, from small LLM inference and 70B models to LoRA training, Stable Diffusion and 3D rendering, and how to estimate hourly cost with worked examples.
Almost every AI project starts with the same question: which GPU should I use? Buying a GPU server means a large investment and a long procurement process. And in the first weeks of a project, it is often unclear what you actually need.
Renting a GPU server removes that uncertainty. You use the card you need by the hour and shut the server down when the job is done. But if you pick the wrong card, you either pay for an unnecessarily expensive GPU or lose hours on a card that runs out of memory.
In this guide we walk through common workloads, suggest sensible GPU options for each, and show how to estimate the cost.
What to Check First When Choosing a GPU
Most people look at raw compute first. In AI workloads, however, the first filter is almost always VRAM, the memory on the card. If the model and data do not fit in memory, compute power does not matter.
We suggest asking these four questions in order:
- Does the model fit in memory? Model size, precision (FP16, 8-bit, 4-bit) and whether you train or run inference determine VRAM needs.
- How long will the job run? A few hours of experimentation and weeks of training call for different cards.
- Speed or hourly price? A faster card may cost more per hour but finish sooner and cost less in total.
- Which software stack? PyTorch, CUDA version, ComfyUI or your own container image all affect setup time.
Rough VRAM Estimates
Exact numbers vary with the model, context length and software, but these rules of thumb are good enough for a first estimate:
- FP16 inference: about 2 bytes per parameter, plus the KV cache, which grows with context length and concurrent users. For an 8B model, the weights alone take about 16 GB.
- 4-bit quantised inference: roughly 0.5 to 0.6 bytes per parameter. A 70B model drops to around 40 GB this way, again plus KV cache.
- Full fine-tuning: in 16-bit training with Adam optimizer states, roughly 16 bytes per parameter or more. That is why even full training of a 7B model does not comfortably fit on a single 80 GB card.
- LoRA and QLoRA: they train only small adapter layers, which cuts memory needs dramatically. QLoRA goes further by keeping the base model in 4-bit.
Treat these numbers as a starting point. Long context windows, large batch sizes and high concurrency raise memory needs noticeably, so always leave some headroom.
GPU Recommendations by Workload
The table below summarises workloads we see often and sensible GPU options for each. Prices are CloudRUX GPU hourly starting prices per GPU, excluding VAT.
| Workload | Recommended GPU | Starting price (hourly, excl. VAT) | Note |
|---|---|---|---|
| Small LLM inference (7B-8B) | L4 24 GB, RTX 4090 24 GB | $0.74 / $1.11 | FP16 weights fit in 24 GB; leave room for KV cache |
| 70B model inference | A100 80 GB, H100 80 GB (4-bit); H200 141 GB, B200 180 GB (higher precision) | $2.24 / $4.99 / $6.61 / $9.99 | FP16 weights take about 140 GB; needs quantisation or multiple GPUs |
| LoRA / QLoRA fine-tuning | RTX 4090 24 GB, A40 48 GB, L40S 48 GB, A100 80 GB | $1.11 / $0.74 / $1.36 / $2.24 | Move to 48 GB and 80 GB cards as model size grows |
| Full fine-tuning | A100 80 GB, H100 80 GB, H200 141 GB, B200 180 GB | $2.24 / $4.99 / $6.61 / $9.99 | Usually needs multiple GPUs except for small models |
| Stable Diffusion / ComfyUI | RTX 4090 24 GB, L40S 48 GB | $1.11 / $1.36 | 48 GB gives comfort for large models and high resolutions |
| Video generation | L40S 48 GB, RTX PRO 6000 96 GB, H100 80 GB | $1.36 / $3.11 / $4.99 | Video models are very memory hungry |
| 3D rendering | RTX 4090 24 GB, L40S 48 GB, RTX PRO 6000 96 GB | $1.11 / $1.36 / $3.11 | Scene size drives VRAM needs |
| Research and experiments | L4 24 GB, A40 48 GB, RTX 4090 24 GB | $0.74 / $0.74 / $1.11 | Fast iteration at low hourly cost |
These recommendations are a general roadmap. Exact price and capacity are confirmed in a quote, and monthly reserved usage is discounted. You can see every GPU option on our GPU cloud page.
A Quick Look at the Cards
L4 24 GB and A40 48 GB: the most economical options, starting at $0.74 per hour. L4 is efficient for small model inference and light workloads. A40 offers 48 GB at the same starting price, which makes it very valuable for mid-sized models and QLoRA experiments.
RTX 4090 24 GB: a consumer card, but strong for experiments, running small and medium models and image generation. Its 24 GB of memory quickly becomes a limit with large models.
L40S 48 GB: a data centre card that is strong for inference and image generation. It offers a good price-performance balance for Stable Diffusion, ComfyUI and mid-sized LLM services.
RTX PRO 6000 96 GB: 96 GB lets you run large models, video generation and heavy render scenes on a single card.
A100 80 GB: one of the standard AI training cards for years. Mature software support and 80 GB make it a safe choice for LoRA, full training and large model inference.
H100 80 GB: substantially faster than A100 for transformer training, and FP8 support widens the gap. It can lower the cost per job on long training runs.
H200 141 GB and B200 180 GB: H200's 141 GB of HBM3e makes it easier to run large models with less splitting. B200 is the newest generation, built for the heaviest training and inference jobs.
How to Estimate Hours and Cost
Cost in GPU rental is simple multiplication: hours used x number of GPUs x hourly price. The hard part is estimating hours. We suggest this method:
- Run a short trial on a small slice of your dataset.
- Note the number of samples or tokens processed per second.
- Roughly calculate total time from total data size and number of epochs.
- Add a buffer of around 20 to 30 percent for setup, debugging and evaluation.
Worked Example
Say you want to fine-tune an 8B model with LoRA on a dataset built from your company documents. After a short trial, you estimate that the whole job, including evaluation and reruns, needs about 40 hours on an A100 80 GB.
| Item | Value |
|---|---|
| GPU | A100 80 GB |
| Hourly starting price | $2.24 (excl. VAT) |
| Estimated usage | 40 hours |
| Calculation | 40 x 2.24 = $89.60 |
| Total | $89.60 + VAT |
If you do the same job with QLoRA on a 7B or 8B model, a cheaper card such as RTX 4090 or A40 may be enough. Even if the job takes a bit longer, total cost may drop. A trial of a few hours comparing two cards is often the best investment.
Billing is straightforward: prices are in USD, invoices are issued in Turkey as e-invoices, and the USD amount is converted to TRY at the invoice-date rate.
Setup and Working Environment
You can access CloudRUX GPU servers over SSH and Jupyter. Ready PyTorch and CUDA templates let you start right away without losing hours on drivers and libraries. You can also bring your own container image and run the same software stack as your local environment.
One important detail: the temporary disk is deleted when the server stops. To keep model weights, checkpoints and results, use a persistent volume or back up your data to S3 object storage. This habit prevents data loss and lets you shut the server down with peace of mind.
Data Location and KVKK
CloudRUX GPU servers run in our partner's data centers abroad. For workloads that include personal data, this can count as a cross-border transfer under KVKK, the Turkish data protection law.
We recommend running non-personal or anonymised workloads on the GPUs. Model training on non-personal data, image generation, rendering and research on open datasets usually fall outside this scope. Where you need to process personal data, the CloudRUX team helps with the standard contract processes.
Tips for Keeping Costs Under Control
- Start small: validate code and the data pipeline on a cheap card, then move long training to a larger one.
- Shut down idle servers: in an hourly model, every idle hour shows up on the invoice.
- Save checkpoints: regular checkpoints remove the cost of restarting from scratch after an interruption or bug.
- Consider reservation for steady use: if you will serve a model for months, monthly reserved usage is discounted.
- Pick the right precision: 8-bit or 4-bit quantisation for inference can let you move to a smaller, cheaper card if the quality loss is acceptable.
Frequently Asked Questions
Is it better to rent or buy a GPU?
If usage is irregular, the project is still experimental or you want to test different cards, renting usually makes more sense. If you are sure you will use the GPU almost continuously all year, compare buying with monthly reserved usage.
Which GPU should I start with?
For most projects, starting with a 24 GB or 48 GB card such as RTX 4090, L4 or A40 is sensible. When memory runs short or jobs get long, move to cards like A100 or H100.
Are the prices fixed?
The figures are hourly starting prices per GPU, excluding VAT. Exact price and capacity are confirmed in your quote, and monthly reserved usage is discounted.
What happens to my data when the server stops?
The temporary disk is deleted when the server stops. Keep the data you need on a persistent volume or back it up to S3.
CloudRUX GPU Server Rental
The CloudRUX GPU service is now in early access. We offer a wide choice of cards from L4 to B200, ready PyTorch/CUDA templates, SSH and Jupyter access, e-invoicing in Turkey and Turkish-speaking engineer support. See the cards and starting prices on our GPU cloud page, or get in touch to choose the right GPU for your workload and request a quote.