Input/output operations per second or IOPS (IO/s), throughput (MB/s or MiB/s), and response time (ms) are the common performance measurements for storage devices, such disk drives or RAID sets.
IOPS measure is generally used to describe storage device capability for handling random small block (4 – 8 KB in size) input-output (IO) requests that are typical for on-line transaction processing (OLTP) applications, databases, etc.
Throughput is usually used to measure device performance for large block (64 KB and more) random or sequential operations such file transfer and video/audio streaming.
Response time is the time it takes for a storage device to process an input-output request.
Conversion between IOPS and throughput can be performed as follows:
IOPS = throughput / block_size
throughput = IOPS * block_size
Where:
block_size – amount of data transferred during a single IO operation.
Let’s review the factors affecting performance of hard disk drives (HDD), also know as spinning disk drives.
The time it takes for a drive to process a single IO request can be described in the following simplified form:
Tio = Ta + Tl + Tt
Where:
Ta – access time (also known as seek time) is the time required for the head assembly to be positioned on the track that contain the requested sector(s). Drive specification sheets usually list three different access time parameters:
full stroke – time to position to the last (innermost) track/cylinder;
single track – time to move between adjacent tracks/cylinders;
average access time.
Tl – rotational latency is the time between positioning the head on the required track and the moment the requested sector travels under the head. Rotational latency value falls in the range of 0 (zero) and the the time equal one revolution period of the platter. Average rotational latency is equal to one-half of the one revolution period.
Tt – transfer time is the time it takes to transfer the data between the disk media and the host interface. It can be estimates as:
Tt = block_size / Rs
Where:
block_size– amount of data transferred during a single IO operation.
Rs – sustained transfer rate. Sustained transfer rate values are also listed on the drive specification sheets.
We can calculate drive’s average IO processing time Tio using average access time, average latency, and sustained transfer rate parameters from the specification sheet. Than we can estimate disk’s IOPS capabilities as:
IOPS = 1 / Tio
Let’s take Seagate Savvio 2.5” SAS 15K rpm drive as an example. The drive has average access time of 2.9 ms for reads and 3.3 ms for writes, average rotational latency of 2.0 ms, and sustained transfer rate in the range between 151 MB/s and 202 MB/s.
Using lower minimum and maximum sustained transfer rate numbers of 151 MB/s and 202 MB/s, as well as block size of 4 KB, we can calculate Savvio’s average IO processing time for small block IO operations:
for reads with minimum transfer rate:
Ta= 2.9 ms; Tl= 2.0 ms; Tt= 4 / 151 = 0.027 ms; Tio= 2.9 + 2.0 + 0.027 = 4.927 ms
for reads with maximum transfer rate:
Ta= 2.9 ms; Tl= 2.0 ms; Tt= 4 / 202 = 0.02 ms; Tio= 2.9 + 2.0 + 0.02 = 4.92 ms
for writes with minimum transfer rate:
Ta= 3.3 ms; Tl= 2.0 ms; Tt= 4 / 151 = 0.027 ms; Tio= 3.3 + 2.0 + 0.027 = 5.327ms
for writes with maximum transfer rate:
Ta= 3.3 ms; Tl= 2.0 ms; Tt= 4 / 202 = 0.02 ms; Tio= 3.3 + 2.0 + 0.02 = 5.32ms
This gives us Savvio’s IOPS numbers for random small block IO operations:
for reads with minimum transfer rate: IOPS= 1 / 4.927 = 203
for reads with maximum transfer rate: IOPS= 1 / 4.927 = 203
for writes with minimum transfer rate: IOPS= 1 / 5.327 = 188
for writes with maximum transfer rate: IOPS= 1 / 5.32 = 188
Using the same approach we can calculated IOPS for different types of enterprise-class disk drives from Seagate and Toshiba. The results are presented in the table below together with the IOPS numbers obtained form other sources.
*Information Sources:
[1] – http://www.yellow-bricks.com/2009/12/23/iops/
[2] – http://www.aarondelp.com/2009/10/its-now-all-about-iops.html
[3] – http://www.tomshardware.com/forum/251893-32-raid-raid
In a theoretical situation when IO requests come as a single synchronous thread (the next request comes only after the previous one has been processed) disk response time is equal to the IO processing time.
Tr = Tio
In the real life IO requests come asynchronously, and in multiple threads. For example, request can come from different processes within the OS or from multiple external clients or systems. The new requests that come while disk is still processing the previous request will have to wait in the queue. The terms of queue length or queue depth are used to describe the number of waiting or outstanding IO requests.
On one hand, queue length grows will increase average response time:
Tr = (Q+1) * Tio
Where:
Q – average queue length;
Tio – average processing time for a single IO request.
On the other hand, as many practical tests have shown [4, 5], maintaining non-zero length queue can significantly improve disk performance. The improvement is attributed to the drive’s queue manager functionality. Queue manager re-orders incoming requests to optimize head movements and as a result reduce the processing time (Tio) for individual requests.
Obviously, optimal combination of IO performance, response time, and queue length will depend on the specific system requirements and load. For a “general purpose” storage system reasonable response time is considered to be below 20 ms [6].
Transfer time Tt for small blocks (4-8 KB) is usually insignificant in comparison with the combined access and latency time. Often Tt is not taken into account for the purpose of IOPS calculations.
With the increasing block size (>= 64KB) Tt becomes more apparent and leads to gradual decrease in disk IOPS.
The following table shows random IOPS and throughput numbers calculated for 64 KB and 512 KB block sizes.
Note: The range of the values is defined by the variation of sustained transfer rate and the difference in access time for read and write operations. The lower number of the range corresponds to write operations with the minimum sustained transfer rate, the higher number – to read operations with the maximum sustained transfer rate.
Sequential IO operations require significantly shorter head movements in comparison to random IO operations. During sequential OI operations head assembly moves only between adjacent tracks and stays in the position until all blocks on the track are accessed (read or written).
In a simplified form, drive throughput for large block sequential IO can be evaluated using the formula below. This formula does not account for any drive configuration details, such as number of platters, heads, platter capacity, etc. It is also assumed that the head switch time is equal to the single track access time.
throughput = Rs * Tt / ( Ta + Tl + Tt )
Where:
Rs – sustained disk transfer rate;
Ta – access time. In this case – single track access time, i.e. time required to move between adjacent tracks;
Tl – rotational latency.
Tt – transfer time. In this case, since all sectors on the track need to be accessed, transfer time is equal to the period of one revolution.
The following table presents disk throughput calculations for large block sequential IO operations.
Note: When multiple disk drives share a single SAS or FC channel, the performance for sequential operations will start to be limited by the bandwidth of the channel. SAS-2 channel is capable of supporting a maximum bandwidth of up to 6 Gb/s or 600 MB/s.