How to verify the prerequisites using vertica validation utilities before install or upgrade of Data Repository ?

Document ID : KB000043260
Last Modified Date : 18/09/2018
Show Technical Document Details
Introduction:

Question:  

How to verify the prerequisites using vertica validation utilities before install or upgrade of Data Repository ?

Answer: 

 

Before upgrading the data repository following steps need to be completed to check if the server meets the minimum 

requirements.

 

Verify that the Data Repository host has at least 2 GB of swap space.

 

Verify that the Data Repository hosts use the ext4 or ext3 file system for data and catalog directories.

 

Note: The database performs best with the ext4 file system.

 

Verify that you are not using Logical Volume Manager (LVM) for data and catalog directories.

 

Verify Hardware and Network Performance

 

Vertica provides a set of utilities that test the performance of your hardware for the Data Repository. To verify that the environment is ideal for the database, run these tests before you upgrade the Data Repository.

 

 

Note: If you have already installed the Data Repository, you can perform these tests at any time to verify performance. The utilities are available on each node in /opt/vertica/bin.

 

If the tests do not meet the recommendations, fix the issues before you continue the installation or upgrade.

 

vcpuperf

 

This utility measures the CPU processing speed of the host and compares the speed against benchmarks for common server CPUs. The utility measures how long the server requires to complete the test, and determines whether CPU throttling is enabled.

 

Follow these steps:

 

Execute the following command on each Data Repository node:

 

./vcpuperf > /tmp/vcpuperf.out

 

Verify that the  performance meets the following requirements

 

The CPU time is consistent with the benchmark values in the output.

 

The low load time and high load time are within 10 microseconds. If the difference is grater than 50 microseconds, CPU throttling might be enabled on your system. Disable CPU throttling.

 

Example:

 

The following example shows the return from this utility:

 

$ /opt/vertica/bin/vcpuperf

Compiled with: 4.1.2 20080704 (Red Hat 4.1.2-52)

Expected time on Core 2, 2.53GHz: ~9.5s

Expected time on Nehalem, 2.67GHz: ~9.0s

Expected time on Xeon 5670, 2.93GHz: ~8.0s

This machine's time:

CPU Time: 7.740000s

Real Time:7.740000s

Some machines automatically throttle the CPU to save power.

This test can be done in <100 microseconds (60-70 on Xeon 5670, 2.93GHz).

Low load times much larger than 100-200us or much larger than the corresponding high load time

indicate low-load throttling, which can adversely affect small query / concurrent performance.

This machine's high load time: 67 microseconds.

This machine's low load time: 64 microseconds.

 

This test was performed on a system with 2.67-GHz processors, so the real time is acceptable. The difference between the high load time and low load time is within the expected tolerance.

 

For more information about this utility, see the Vertica documentation.

 

vioperf

 

This utility tests the performance of the disk input and output (I/O). The utility performs a series of reads and writes.

 

Follow these steps:

 

Execute the following commands on each Data Repository node:

 

./vioperf /data > /tmp/vioperf.out --duration=60sec

 

/data is the full path of the data directory.

 

./vioperf /catalog > /tmp/vioperf.out --duration=60sec

 

/catalog is the full path of the catalog directory.

 

Verify the Write and Read counter values at least 40 MB/s per core.

 

The recommended I/O is 40 MB/s per physical core on each node. For example, the recommended I/O rate for a node with 2 hyper-threaded six-core CPUs (12 physical cores) is 480 MB/s.

 

Important! If the thread count column shows a value of 1, the utility cannot determine the number of cores. Add the following arguement to the command to run the utility:

 

--thread-count=CORES

 

Cores defines the number of cores in the system as a fixed integer.

 

Example:

 

The following example shows the return from this utility for the data directory:

 

The minimum required I/O is 20 MB/s read and write per physical processor core on each node, in full duplex i.e. 

reading and writing at this rate simultaneously, concurrently on all nodes of the cluster. The recommended I/O 

is 40 MB/s per physical core on each node. For example, the I/O rate for a server node with 2 hyper-threaded 

six-core CPUs is 240 MB/s required minimum, 480 MB/s recommended.Using direct io (buffer size=1048576, alignment=512) for directory "/drdata"

 

testdirectorycounter namecounter value counter (value 10 sec avg)Counter value /core counter value/core (10 sec avg) thread count%CPU%IO Waitelapsed time (s)remaining time (s)
Write/drdataMB/s87387354.562554.5625 162940105
Write/drdataMB/s86886854.2554.0625162830150
ReWrite/drdata(MB-read+MB-write)/s275+275275+27517.1875+17.1875  17.1875+17.1875 161321105
ReWrite/drdata(MB-read+MB-write)/s242+242178+17815.125+15.125   11.125+11.125  16717150
Read/drdataMB/s73573545.937545.9375161123105
Read/drdataMB/s78678649.12549.125162625150
SkipRead/drdata seeks/s 45114511281.938281.938161419105
SkipRead/drdata seeks/s 44774407279.812275.43816315150

 

 

This server has 16 cores. The Read and Write counter values indicate the I/O is greater than 40 MB/s per core.

For more information about this utility, see the Vertica documentation.

 

vnetperf

 

This utility tests the network performance of the Data Repository hosts. The utility measures network latency and the throughput for the TCP and UDP protocols.

 

Important! This utility causes a high network load and affects database performance. Do not run this utility while the database is running.

 

Follow these steps:

 

Log in as a user that has passwordless ssh between the nodes.

 

Execute the following command on one of the Data Repository nodes:

 

./vnetperf --hosts DAhost,DRhost1,DRhost2,DRhost3 > /tmp/vnetperf.out

 

Specify the hostname or IP address of the Data Aggregator host and each Data Repository host.

Verify that the network performance meets the following requirements:

  • Round-trip time (RTT) latency of 200 microseconds or less
  • Clock skew under 1 second
  • Throughput of 800 MB/s or more
    The utility runs a series of throttled tests. Verify the throughput for the highest speed test.

For more information about this utility, see the Vertica documentation.

 

Additional Information:

 

Please refer to the following wiki link for more information 

 

https://docops.ca.com/ca-performance-management/3-6/en

Instructions:
Please Update This Required Field