Cloud performance variability is the way the speed of the same server changes from hour to hour and day to day, even when your own workload stays the same. You measure it by running identical, repeatable tests on a schedule over several weeks and looking at the spread of results, especially the slow tail, rather than a single average. This guide explains what causes variability and gives a practical method you can run on any VPS or cloud instance.
A note on this article: an earlier version presented results from a “Tremhost Labs” study. We have removed those figures because they could not be independently verified. What follows is a method, not a set of findings.
Why the same server performs differently over time
- Shared physical hosts. Virtual machines share hardware. If CPU is not dedicated, other tenants’ workloads can take processor time from yours. On Linux this shows up as “steal” time.
- Storage contention. Disk throughput and latency can vary when many machines use the same storage system.
- Network conditions. Routes, congestion and peering between networks change through the day.
- Burst credits. Some cloud instance types allow short bursts of CPU, then slow down once credits run out.
- Your own stack. Cron jobs, backups, log rotation and cache expiry can create regular slow periods that look like provider problems.
What to measure
- CPU: a fixed compute benchmark (for example, sysbench cpu) and CPU steal time from tools such as top, vmstat or mpstat.
- Memory: a memory bandwidth test, which is usually stable but worth confirming.
- Disk: random read and write IOPS and latency with fio, using a test file larger than the server’s RAM so results are not just cache.
- Network: latency and packet loss to the places your users are (ping, mtr) and throughput with iperf3 to a server you control.
- Application: response times for a real page or API endpoint using a load tool such as k6 or wrk, from a separate machine.
How to design the test
Keep everything constant
Use the same operating system image, package versions, test parameters and test duration every time. Change one thing at a time. Record the server plan, location and kernel version alongside every result.
Test on a schedule
Run the suite several times a day, including business hours and night time, for at least two to four weeks. Variability often follows daily and weekly patterns that a one-off test will miss.
Test more than one instance
If you can, run the same tests on two or more servers of the same plan. That separates a single unlucky host from a pattern.
Store raw results
Save every run with a timestamp in a simple CSV or a time-series database so you can graph and re-analyse later.
A minimal test routine
You do not need a large lab. A simple routine on one server can include a short CPU benchmark, a one-minute fio random-read test, a ping and mtr run to a target near your users, and a k6 or wrk test against one important page from a second machine. Wrap these in a script that writes results with a timestamp, schedule it with cron every few hours, and leave it running. After a few weeks you will have enough data to see whether performance is steady or swings at certain times.
How to analyse the results
- Use percentiles, not just averages. Report the median (p50) and the slow tail (p95, p99). Users notice the slow requests.
- Calculate the spread. The coefficient of variation (standard deviation divided by mean) lets you compare stability across tests with different units.
- Plot over time. A time-series chart shows patterns, such as slowdowns at the same hour each day.
- Correlate with steal time and your own jobs. If slow periods line up with backups or cron jobs, the cause is yours.
- Be honest about sample size. A few days of data from one server supports only modest conclusions.
Reducing variability in practice
- Choose dedicated CPU where predictable performance matters. Tremhost’s KVM VPS plans have dedicated cores and reserved RAM.
- Use fast local storage. NVMe reduces disk latency for databases.
- Pick a location near your users to cut network delay. Tremhost VPS plans are available in Johannesburg, New York, Miami, Limburg and Roubaix; see the data centres page.
- Cache aggressively. Application and page caches smooth out slow backend moments.
- Schedule heavy jobs such as backups away from peak hours.
- Size with headroom so normal peaks do not push the server to its limits.
Turning measurements into decisions
Set a target you care about, for example “p95 response time for the checkout page under a set threshold during business hours”. Compare each server or plan against that target, not against a vendor’s headline figures. If a server regularly misses the target and steal time is high, move to dedicated cores or a larger plan. If it misses only during your own scheduled jobs, fix the schedule.
Uptime is a separate question from speed. Check each provider’s written service level agreement; Tremhost’s sets a 99.9% monthly availability target for hosting, email and servers.
Frequently asked questions
How long should a performance test run?
Long enough to capture daily and weekly cycles: at least two weeks, and ideally a month.
Is one benchmark run useful?
It is useful as a quick check, but it cannot tell you about variability.
What is CPU steal time?
The share of time your virtual CPU was ready to run but the hypervisor was serving another virtual machine. Consistently high steal time suggests contention.
Next step
To run these tests on dedicated cores, start a server on the Tremhost VPS hosting page and measure it yourself.
