vioperf
The vioperf
utility quickly tests the performance of your host's input and output subsystem. The utility performs the following tests:
- sequential write
- sequential rewrite
- sequential read
- skip read (read non-contiguous data blocks)
The utility verifies that the host reads the same bytes that it wrote and prints its output to STDOUT. The utility also logs the output to a JSON formatted file.
For data in HDFS, the utility tests reads but not writes.
Syntax
vioperf [--help] [--duration=<INTERVAL>] [--log-interval=<INTERVAL>] [--log-file=<FILE>] [--condense-log] [--thread-count=<N>] [--max-buffer-size=<SIZE>] [--preserve-files] [--disable-crc] [--disable-direct-io] [--debug] [<DIR>*]
Minimum and Recommended I/O Performance
- The minimum required I/O is 20 MB/s read/write per physical processor core on each node, in full duplex (reading and writing) simultaneously, concurrently on all nodes of the cluster.
- The recommended I/O is 40 MB/s per physical core on each node.
-
The minimum required I/O rate for a node with 2 hyper-threaded six-core CPUs (12 physical cores) is 240 MB/s. Vertica recommends 480 MB/s.
For example, the I/O rate for a node with 2 hyper-threaded six-core CPUs (12 physical cores) is 240 MB/s required minimum, 480 MB/s recommended.
Disk Space vioperf Needs
vioperf
requires about 4.5 GB to run.
Options
Option | Description |
---|---|
--help |
Prints a help message and exits. |
--duration |
The length of time
|
--log-interval |
The interval at which the log file reports summary information. The default interval is 10 seconds. This option uses the same interval notation as |
--log-file |
The path and name where log file contents are written, in JSON. If not specified, then |
--condense-log |
Directs |
--thread-count=<N> |
The number of execution threads to use. By default, |
--max-buffer-size=<SIZE> |
The maximum size of the in-memory buffer to use for reads or writes. Specify the units with any of these suffixes:
|
--preserve-files |
Directs |
--disable-crc |
Directs |
--disable-direct-io |
When reading from or writing to a local file system, Disabling this behavior can produce more realistic performance results but slows down the operation of |
--debug |
Directs |
<DIR> |
Zero or more directories to test. If you do not specify a directory, To test reads from a directory on HDFS:
|
Returns
The utility returns the following information:
Heading | Description |
---|---|
test |
The test being run (Write, ReWrite, Read, or Skip Read) |
directory |
The directory in which the test is being run. |
counter name |
The counter type of the test being run. Can be either MB/s or Seeks per second. |
counter value |
The value of the counter in MB/s or Seeks per second across all threads. This measurement represents the bandwidth at the exact time of measurement. Contrast with counter value (avg). |
counter value (10 sec avg) |
The average amount of data in MB/s, or the average number of Seeks per second, for the test being run in the duration specified with |
counter value/core |
The counter value divided by the number of cores. |
counter value/core (10 sec avg) |
The counter value (10 sec avg) divided by the number of cores. |
thread count |
The number of threads used to run the test. |
%CPU |
The available CPU percentage used during this test. |
%IO Wait |
The CPU percentage in I/O Wait state during this test. I/O wait state is the time working processes are blocked while waiting for I/O operations to complete. |
elapsed time |
The amount of time taken for a particular test. If you run the test multiple times, elapsed time increases the next time the test is run. |
remaining time |
The time remaining until the next test. Based on the |
Example
Invoking vioperf
from a terminal outputs the following message and sample results:
[dbadmin@v_vmart_node0001 ~]$ /opt/vertica/bin/vioperf --duration=60s The minimum required I/O is 20 MB/s read and write per physical processor core on each node, in full duplex i.e. reading and writing at this rate simultaneously, concurrently on all nodes of the cluster. The recommended I/O is 40 MB/s per physical core on each node. For example, the I/O rate for a server node with 2 hyper-threaded six-core CPUs is 240 MB/s required minimum, 480 MB/s recommended. Using direct io (buffer size=1048576, alignment=512) for directory "/home/dbadmin" test | directory | counter name | counter value | counter value (10 sec avg) | counter value/core | counter value/core (10 sec avg) | thread count | %CPU | %IO Wait | elapsed time (s)| remaining time (s) -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Write | /home/dbadmin | MB/s | 420 | 420 | 210 | 210 | 2 | 89 | 10 | 10 | 5 Write | /home/dbadmin | MB/s | 412 | 396 | 206 | 198 | 2 | 89 | 9 | 15 | 0 ReWrite | /home/dbadmin | (MB-read+MB-write)/s | 150+150 | 150+150 | 75+75 | 75+75 | 2 | 58 | 40 | 10 | 5 ReWrite | /home/dbadmin | (MB-read+MB-write)/s | 158+158 | 172+172 | 79+79 | 86+86 | 2 | 64 | 33 | 15 | 0 Read | /home/dbadmin | MB/s | 194 | 194 | 97 | 97 | 2 | 69 | 26 | 10 | 5 Read | /home/dbadmin | MB/s | 192 | 190 | 96 | 95 | 2 | 71 | 27 | 15 | 0 SkipRead | /home/dbadmin | seeks/s | 659 | 659 | 329.5 | 329.5 | 2 | 2 | 85 | 10 | 5 SkipRead | /home/dbadmin | seeks/s | 677 | 714 | 338.5 | 357 | 2 | 2 | 59 | 15 | 0
When evaluating performance for minimum and recommended I/O, include the Write and Read values in your evaluation. ReWrite and SkipRead values are not relevant to determining minimum and recommended I/O.