Close

StorNext - cvadmin tricks

Cvadmin, a powerful command line tool to query the running file systems on the MDC.

Common cvadmin usage: cvadmin  (no arguments)

If run on a client, and it shows the file systems available, it means that the metadata connection is working correctly.  On an HA system, you can determine which MDC is the primary by which entries have an asterisk (*) next to it.

StorNext Administrator

Enter command(s) For command help, enter "help" or "?".

List FSS

File System Services (* indicates service is in control of FS): 
 1>*data_vol1[0]              located on mdc1:32892 (pid 17650) 
 2>*data_vol2[0]              located on mdc1:32900 (pid 17649) 
 3> data_vol1[1]              located on mdc2:32825 (pid 9623) 
 4> data_vol2[1]              located on mdc2:32826 (pid 9624) 
Select FSM "none"

Inside cvadmin use select to view info about a file system

cvadmin> select data_vol1

This shows an overview of the file system. Type who to see which clients are connected to that file system, how long they have been connected, and information on licensing

snadmin (data_vol1) > who
Who (File System "data_vol1")

#    SNFS  I.D.  Type  Location         Up Time       License Expires
--   ----------  ----  ---------------  ---------     ---------------
0>               FSM   mdc1             12d 2h 10m     N/A
1>               CLI   client_01-san    12d 2h 10m     N/A
3>   151762DD74  CLI   mdc-1             1d 1h 56m     Permanent
4>               ADM   mdc1              0d 0h 0m      N/A
10>              CLI   192.168.10.110   12d 2h 13m     Permanent
12>  151762DD74  CLI   192.168.10.121    6d 0h 42m     Permanent

Type who long to get a more detailed output on the file system status and it’s stripegroup statistics.

Index     : 10
 Type      : Client
 Location  : 192.168.10.110
 Up Time   : 12d 2h 13m
 License   : I.D. 15177A3B98 permanent
 Latency   : 37037us measured on 02-17-2014 21:39:15
 Revision  : 4.3.2 build 30118 for Linux 2.6.32-71.el6.x86_64 x86_64
 Build Date: Fri Jan  4 02:04:48 MST 2013
 Build Dir : /home/mlund/nightly/VM-0-RedHat60AS-26x86-64-SP0/sn/buildinfo

 Index     : 12
 Type      : Client
 Location  : 192.168.10.121
 Up Time   : 6d 0h 45m
 License   : I.D. 15177A3B98 permanent
 Latency   : 77us measured on 02-17-2014 21:37:08
 Revision  : 4.3.2 build 30118 for Windows v7.0 AMD64
 Build Date: Fri Jan  4 02:08:57 MST 2013
 Build Dir : C:/sbx/sn/buildinfo
 Reconnects: 6 times in the last 8 days

This is a quite helpful option that can help to quick check the right IP assignments and versions (not provided for Xsan).

cvadmin -e 'select data_vol1;who long' | grep -e Location -e Revision
  Location  : mdc-1 (192.168.10.2)
  Location  : 192.168.20.110 (192.168.10.110 -> 192.168.1.2)
  Revision  : 4.3.2 build 30118 for Linux 2.6.32-71.el6.x86_64 x86_64
  Location  : 192.168.20.121 (192.168.10.121 -> 192.168.1.2)
  Location  : mdc-1 (192.168.10.2) (192.168.10.2 -> 192.168.10.2)
  Revision  : 4.3.2 build 30118 for Linux 2.6.32-71.el6.x86_64 x86_64

Another useful option that can help troubleshooting performance issues is the latency-test.  While there are many Linux tools as netcap, iperf, ttcp and so on, cvadmin has a build in tool to measure the latency to a client.

Inside the cvadmin tool run a latency test against a single client or all attached clients. It is based on the client index number that you can see via who, not the IP address of that client.

 snadmin (data_vol1) > latency-test 10 
 Test started on client 1o (192.168.20.110)...latency 1582us
 snadmin (data_vol1) > latency-test all
 Test started on client 10 (192.168.20.110)...latency 142857us
 Test started on client 12 (192.168.20.121)...latency 216us

As you can see, the repeated test result for client 10 is quite different. This may happen as the client is busy with some other activities. I have seen clients spiking with those latency numbers and I would recommend to test client by client rather than do a broadcast like latency test.

If you still need more information why this latency number is high, you can gather more details about the client running stats IPADDR (not stat) inside cvadmin. Note, this time the IP is required and it won’t accept the client ID like in the example before.

snadmin (data_vol1) > stats 192.168.10.110 
File paths shown are relative to the mount point for data_vol1
Top files in bytes read/written:
  read bytes  write bytes   read count  write count  Relative File Path
  1529444352            0        14834            0  scanity/05.03.2014/picture/project/reel04/2048x1488/0370453.dpx
   410200064            0        1684             0  scanity/05.03.2014/picture/project/reel04/2048x1488/0370454.dpx

Cvadmin is the tool used to stop, start, and activate file systems. Inside cvadmin use the command “start data_vol1” to start that file system on the local machine. If that file system is already running and active on another MDC (fail over /HA mode), then the local MDC will start it in secondary (standby) mode.

If you use “stop data_vol1” on an HA system, it will stop both the primary and secondary instances of it. You can be more specific about which to stop by specifying a file system with a number, such as “stop 1“, where 1 is a file system in a list.

The internal commands of cvadmin can be used on the regular linux command line by using the -e and -F argument. This becomes useful for scripting or other non-interactive activities. For example, to see who is connected to file system data_vol1 without entering the cvadmin prompt, use the following.

    cvadmin -e "select; who -F data_vol1"

This will output the list of connections the same as if you typed who inside cvadmin.
To see a list of the current file systems like you would upon first entering the cvadmin prompt, use the select command without specifying a file system.

    cvadmin -e select

Leave a Reply

Your email address will not be published. Required fields are marked *