To benchmark cloud databases such as Amazon RDS, Azure SQL Database and Google Cloud SQL fairly, you need the same engine, comparable instance sizes and storage, the same region distance from your application, a workload that resembles your own, and repeated runs over time. Without those controls, a benchmark tells you about the test set-up rather than the service. This guide sets out a method you can run yourself; it does not report results, because any result depends on choices you have to make for your workload.
What you are comparing
- Amazon RDS runs managed MySQL, PostgreSQL, MariaDB, Oracle and SQL Server on AWS.
- Azure SQL Database is Microsoft’s managed database service based on the SQL Server engine.
- Google Cloud SQL runs managed MySQL, PostgreSQL and SQL Server on Google Cloud.
Because the engines differ, a fair comparison usually means picking one engine that all the services you are testing support, such as PostgreSQL or SQL Server, rather than comparing different engines against each other.
What drives managed database performance
- Compute: vCPU count, CPU generation and memory. Memory decides how much of your working data stays in cache.
- Storage: the storage type, provisioned throughput and IOPS limits. Many services tie IOPS to volume size or tier.
- Network latency: the distance between your application and the database often matters more than raw database speed for chatty applications.
- High availability: synchronous standby replicas can add write latency in exchange for resilience.
- Engine configuration: default parameters differ between providers, for example memory settings and connection limits.
- Burst behaviour: some instance and storage types allow short bursts above their baseline, which can make short tests look better than sustained use.
Step-by-step benchmarking method
- Define the question. For example: which service gives the lowest 95th-percentile query latency for our order-processing workload at our expected load, and at what monthly cost?
- Choose one engine and version available on every service you test.
- Match the resources. Pick instance sizes with the same vCPU and memory where possible, and storage with comparable provisioned throughput. Record every setting.
- Match the topology. Run the load generator in the same provider and region as each database, at a similar distance, or deliberately from your real application location if that is what you want to measure.
- Use a realistic workload. Standard tools include sysbench for MySQL and PostgreSQL, pgbench for PostgreSQL and HammerDB for transactional workloads on several engines. Better still, replay a sample of your own queries.
- Load enough data. Use a dataset larger than memory if your production data is, so the test exercises storage as well as cache.
- Warm up, then measure. Run a warm-up period before recording, then measure for long enough to get past any burst credits.
- Record the right numbers: throughput (transactions or queries per second), latency at the median, 95th and 99th percentiles, error rates, and CPU, memory and IOPS on the database during the test.
- Repeat at different times and on different days, and report medians and spread, not a single best run.
- Test failure behaviour. Trigger a failover where the service allows it and measure how long writes are unavailable.
- Price the exact configuration you tested, including storage, IOPS, backups, standby replicas and data transfer.
Common mistakes
- Comparing a high-availability configuration on one service with a single instance on another.
- Running the load generator in a different region from the database.
- Using a dataset small enough to fit entirely in memory when production data does not.
- Stopping the test before burst credits run out.
- Leaving default parameters on one service and tuning another.
How to report a benchmark honestly
- Publish the engine version, instance types, storage settings, regions, parameter changes, tool versions, dataset size and test dates.
- Show the spread of results, not just averages.
- State the limits: one workload, one region, one point in time.
- Check each provider’s terms before publishing benchmark results, as some cloud agreements include conditions on this.
Beyond raw speed
For most businesses, the decision also rests on backups and point-in-time restore, failover, security controls, upgrade policy, monitoring and support. A slightly slower service with tested restores and automatic failover is often the better business choice. Engine support windows matter too: running an old major version can mean forced upgrades or extra charges, so check each provider’s version policy.
Where Tremhost fits
Tremhost runs managed databases, including MySQL, PostgreSQL and MongoDB, on Amazon RDS or on its own servers. The service includes daily backups with restore to any point in the last seven days, encryption in transit and at rest, a private network with an IP allow-list, monitoring and alerts, help tuning slow queries, and an optional standby that takes over if the primary fails. Tremhost also migrates existing databases and, through its cloud migration service, moves servers and applications with a tested switch-over and a rollback plan.
Frequently asked questions
Which cloud database is fastest?
There is no single answer. Results depend on engine, instance size, storage, region and workload, so run the method above with your own workload.
How long should a database benchmark run?
Long enough to pass warm-up and any burst allowance and to show stable results, and repeated on more than one day.
Should I benchmark with my own data?
Yes, if you can do it safely. Use an anonymised copy so personal data is not exposed in the test environment.
Next step
See what is included on the Tremhost managed databases page, or book a scoping call to talk through your workload.
