Hello,
A few days ago, I asked:
----------------------------------------------------------------------
When review sar reports, I often note four "disks" that are very busy:
device %busy avque r+w/s blks/s avwait avserv
sd20 51 0.5 37 4632 0.0 13.9
sd175 45 0.5 37 4631 0.0 12.4
sd314 63 0.8 37 4632 0.0 21.0
sd316 52 0.6 37 4633 0.0 15.0
I'm at a loss on how to figure out how to convert these device names
into something useful. How can I tell which filesystems these are on?
I've tried following the links in /dev and /devices, but to no avail.
The disks are in a set of RSM2000 trays. We're running Raid Manager
6.0, Veritas Filesystem 2.3.1.1, and Veritas Volume Manager 2.4.
----------------------------------------------------------------------
Many thanks to those people who offered their insight (in order of
appearance, e-mail addresses withheld to minimize harvesting by
spammers):
Kevin Sheehan
Michael Wang
Richard Smith
Stefan Voss
Niall O Broin
Nick Hindley
Oliver Giraud
Jackie Rosinsky
Rob Hill
Jeff Kennedy
Ray Trzaska
Several people suggested iostat, some sent scripts, some suggested I
check the links in the /dev/tree, and and a couple of people described
how to do convert the sd instance number into a cNtNdN number. However,
No one even attempted to answer the question on how to find which file-
system on the affected device was responsible for the busy disks.
In the end, iostat doesn't do the job, and none of the scripts could
handle LUNs created under rm6.
Piecing together all the information sent to me, I was able to modify
and combine all of the scripts to create an all-in-one script that
could convert from sdN to cNtNdNsN and back, as well as listing all
such devices on the system. The script is attached to this message
for anyone who is interested. Feedback is welcome.
Here are the responses I received:
1. Several people mentioned using the "-n" option to iostat. Unfor-
tunately, this is only available under 2.6, and we are running
2.5.1.
Also, iostat does not provide names for all disks. I'm not sure
under what conditions it doesn't work, though. I haven't had the
time to investigate this fully, though anyone is welcome to explain
this to me if they know. Here is an example from a 2.6 machine:
# uname -a
SunOS mensa 5.6 Generic_105181-10 sun4u sparc SUNW,Ultra-Enterprise-10000
# iostat -xn | head
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.1 0.3 1.0 3.5 0.0 0.1 3.5 156.0 0 0 c0t0d0
0.1 0.0 0.5 2.9 0.0 0.0 0.0 28.9 0 0 c0t1d0
0.1 0.0 0.5 2.9 0.0 0.0 0.0 31.3 0 0 c0t2d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.2 0 0 c0t5d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.2 0 0 c0t6d0
1.1 0.7 78.7 32.1 0.0 0.0 0.8 17.2 0 1 sd20
0.9 0.7 77.2 31.9 0.0 0.0 0.3 12.2 0 1 sd35
0.1 0.3 1.7 2.9 0.0 0.1 3.4 153.0 0 0 c3t0d0
Unfortunately, it's the "sdxx" devices that I'm interested in.
What are they? Which filesystems do they belong to?
2. Several people either mentioned or sent me a script called
"whatdev," or slight variations thereof. This was apparently
published by a Sun engineer a while ago, and can be found in the
Answerbook (search for "whatdev" in docs.sun.com). I was orig-
inally looking for this script when I sent out my query, but I
could not remember what it was called.
One problem with whatdev is that it only looks in /dev/dsk for
devices, where the rm6 devices are symlinks to /devices/pseudo
rather than /devices/sbus. whatdev doesn't know how to handle
this:
/dev/dsk/c0t0d0s2 ---- sd0
/dev/dsk/c0t1d0s2 ---- sd1
/dev/dsk/c0t2d0s2 ---- sd2
/dev/dsk/c0t3d0s2 ---- sd3
/dev/dsk/c0t4d0s2 ---- sd4
/dev/dsk/c0t5d0s2 ---- sd5
/dev/dsk/c0t6d0s2 ---- sd6
/dev/dsk/c1t5d0s2 ---- /dev/dsk/c1t5d2s2 ---- /dev/dsk/c1t5d4s2 ---- /dev/dsk/c1t5d6s2 ---- /dev/dsk/c3t0d0s2 ---- sd45
/dev/dsk/c3t1d0s2 ---- sd46
/dev/dsk/c3t2d0s2 ---- sd47
/dev/dsk/c3t3d0s2 ---- sd48
/dev/dsk/c3t4d0s2 ---- sd49
/dev/dsk/c3t5d0s2 ---- sd50
/dev/dsk/c3t6d0s2 ---- sd51
/dev/dsk/c4t4d1s2 ---- /dev/dsk/c4t4d3s2 ---- /dev/dsk/c4t4d5s2 ---- /dev/dsk/c4t4d7s2 ---- /dev/dsk/c5t4d0s2 ---- /dev/dsk/c5t4d1s2 ---- /dev/dsk/c5t4d2s2 ---- /dev/dsk/c5t4d3s2 ---- /dev/dsk/c5t4d4s2 ---- /dev/dsk/c5t4d5s2 ---- /dev/dsk/c5t4d6s2 ---- /dev/dsk/c5t4d7s2 ----
Also, the number of "sd" devices outnumbers the devices in
/dev/dsk, so whatdev doesn't even print a complete list.
3. Nick Hindley sent two scripts, one to convert a given sd instance
number to cNtNdN format, one to go the other way around. Adding
the path /dev/osa/dev/dsk to the scripts gives them the ability
to resolve all disks available on the system.
4. Stefan Voss sent a summary from a question that was asked last year,
titled "Recognizing iostat disk names." It goes into great detail
on how to follow the links in the /dev tree. Most of this I
already knew, but the important thing was that this post showed me
that I also need to check in /dev/osa/dev/dsk for systems using
the Open Storage Array software.
Using the information from this post, I was able to modify all
the scripts sent to me so they would work in an Open Storage Array
environment.
5. Jackie Rosinsky explained the relationship between sd instance
numbers (as found in /etc/path_to_inst) and entries in the /dev/dsk
tree: "minor number of slice 0 is equal to the instance number
times 8." So, in my original question, sd20 would have a minor
number of 160:
# cd /dev/osa/dev/dsk
# ls -lL | grep 160
brw-r----- 1 root sys 32,160 Jan 28 1998 c1t5d0s0
# cd /dev
# ls -lLR | grep 160
crw-r----- 1 root sys 32,160 Jan 28 1998 rsd20a
brw-r----- 1 root sys 32,160 Jan 28 1998 sd20a
brw-r----- 1 root sys 32,160 Jan 28 1998 c1t5d0s0
crw-r----- 1 root sys 32,160 Jan 28 1998 c1t5d0s0
brw-r----- 1 root sys 32,160 Jan 28 1998 sd@5,0:a
crw-r----- 1 root sys 32,160 Jan 28 1998 sd@5,0:a,raw
c1t5d0s0 is the same device that is reported by the attached
script.
6. Ray Trzaska mentioned that early versions of the rm6 software
caused the disk busy status to be badly out of sync with what was
the real load. We're running 6.01. Not sure which early versions
had this problem.
Now, all the disks mentioned in the original post are actually RAID 5
LUNS, and several filesystems have been striped on each. Is there
any easy way to find out which filesystem is the busy one responsible
for the busy LUN?
Here is one way: once I have the LUN hardware location (eg. c1t5d0s2),
I can use vxprint to determine the disk name under Veritas Volume
manager, eg. RSM0_0. From there, I can use vxprint again to display
all volumes created on this disk, and from there I can use df to display
the filesystems mounted on these volumes.
Just curious if there is an easier way, or if someone has already
written such a script. I'm going to go ahead and write my own if no
one else has one.
In my case, there are five filesystems striped across the four LUNs in
question. How can I find out which is the filesystem responsible for
the busy disks?
Thanks again,
Mike van der Velden
Insurance Corporation of British Columbia
==
The manual said, "NT4 or better," so I installed Solaris.
_________________________________________________________
DO YOU YAHOO!?
Get your free @yahoo.com address at http://mail.yahoo.com
This archive was generated by hypermail 2.1.2 : Fri Sep 28 2001 - 23:13:16 CDT