Summary: I/O Performance on SUN Drives

From: david@orange.com
Date: Fri Mar 03 1995 - 05:24:40 CST


This is the summary to my query:
(Note: I've included responses at the end of this summary)

> To all:
>
> We have an Oracle database running on a SPARCserver 2000. We are looking
> into using an 18GB SPARCstorage Array for the database and have been told
> that the outer half of the disk will be about twice as fast as the inner
> half of the disk, and, therefore that we should only use the outer half of
> the disk for performance reasons. Does anyone have experience with this?
> Is this accurate?

Almost everybody said the same thing:

(1) Because of ZBR (Zoned Bit Recording), the lowest numbered cylinders
     move more data past the heads per unit time than the higher cylinders.

(2) There is no need to only use half of the disk. Infrequently used
     files should be placed on the inner part of the disk in order to
     maintain maximum performance.

The question remains as to how significant this difference is. Butch Deal
indicated that it doesn't matter where you write to because of the 4MB NVRAM
cache in the SPARCstorage array. Donald McLachlan experimented with this
on his 4GB disk split logically in 2 and found that writing and reading
a 300MB files from the outer cylinders was faster than from the inner
cylinders. However, the difference in time real time was only 2.3 seconds
for writing and 2.1 seconds for reading.

It appears that the drives should be split in some fashion in order to
place the most heavily accessed files on the outer cylinders and the least
accessed files on the inner cylinders.

Thanks to:

deal@ait.nrl.navy.mil Butch Deal
sdr@rdga3.att.com S. D. Raffensberger
jim.leahy@valcom.com Jim Leahy
Kevin.Sheehan@uniq.com.au Kevin Sheehan
don@mars.dgrc.doc.ca Donald McLachlan
jeffw@jane.tiac.net Jeff Wasilko
mshon@sunrock.East.Sun.COM Michael J. Shon
Mark_Mellman@msmail.mpi.com Mark Mellman
strombrg@hydra.acs.uci.edu Dan Stromberg
ncanas@mickey.Solsuni.pt Nuno.Canas

David James
Orange Systems
david@orange.com

----- Begin Included Message -----

>From deal@ait.nrl.navy.mil Fri Feb 24 12:36 EST 1995
From: deal@ait.nrl.navy.mil (Butch Deal NRL)
Date: Fri, 24 Feb 1995 12:36:24 -0500
X-Mailer: Mail User's Shell (7.2.5 10/14/92)
To: david@orange.com
Subject: Re: I/O Performance on SUN Drives
Content-Type: text
Content-Length: 463

who on earth told you that. that kind of disk manipulations is from
pretty old hardware.
with raid and with the battery backed up write buffer on the sun raid, it
doesn't matter what part of the disk you write too. Also with most
modern disks it doesn't matter either.
Butch

-- 
#include <std/*>
The Butcher                  
Butch Deal                   deal@ait.nrl.navy.mil
--------------------------------------------------------------------------------

----- End Included Message -----

----- Begin Included Message -----

>From sdr@rdga3.att.com Fri Feb 24 18:15 EST 1995 From: sdr@rdga3.att.com (S. D. Raffensberger 52882 (RD)) To: david@orange.com Date: Fri, 24 Feb 95 14:30:18 EST Original-From: rdga3!sdr (S. D. Raffensberger 52882 (RD)) Original-To: orange.com!david Subject: Re: I/O Performance on SUN Drives Content-Type: text Content-Length: 1136

Dave

1. I don't believe that you can reliably use the outer half of a disk. When you format, you specify cylinders, which leads one to believe this is possible. However, I asked this question at the last Solaris class I attended and was told that the SCSI driver and disks will hide the true disk information and all you are predictably setting is file- system sizes.

2. Assuming that the disk has the same number of sectors on the outer tracks as the inner tracks, then the number of sectors that pass under the head per revolution is the same throughout the disk. The only exception to this might be disks from the CDC division of Seagate. About two years ago, CDC developed "zoned recording" which creates groups of cylinders based upon how many whole sectors will phy- sically fit into them. These disks pack more sectors into the outer cylinders. The kicker is that I've never heard that the SUN disk drivers can take advantage of that. Notice that the format command makes no provisions for it.

I doubt that you'd see a measurable difference in speed.

Steve Raffensberger AT&T

----- End Included Message -----

----- Begin Included Message -----

>From jim.leahy@valcom.com Fri Feb 24 21:16 EST 1995 Date: Fri, 24 Feb 95 15:48:08 EST From: jim.leahy@valcom.com (Jim Leahy) To: david@orange.com Subject: Re: I/O Performance on SUN Drives Content-Type: text Content-Length: 831

Although I have no practical working experience with this theory due to the laws of physics this makes alot of sense... Look at any disk or bicycle wheel. If you draw a straight line(spoke) from the center (axle) to the outside(tire). The two points, the axle and the wheel cross the same point at exactly the same time. The points on the outside of wheel (disk) cover a greater distance in the same amount of time. Therefore travelling at a faster speed. Now if I was a disk head reading data it would be more efficient to minimize the disk head movement from cylinder to cylinder by putting the data on the outside cylinders.

Reality: The storage array drives a 3.5" single platter. Will the difference between 3.0" from center and 1.0" from center make a difference ?

Who knows ?

Can't wait for the summary on this one ;-)

----- End Included Message -----

----- Begin Included Message -----

>From kevin@uniq.com.au Sat Feb 25 03:26 EST 1995 From: Kevin.Sheehan@uniq.com.au (Kevin Sheehan {Consulting Poster Child}) Date: Sat, 25 Feb 1995 19:27:13 EST X-Mailer: Mail User's Shell (7.1.2 7/11/90) To: david@orange.com Subject: Re: I/O Performance on SUN Drives Content-Type: text Content-Length: 1409

[ Regarding "I/O Performance on SUN Drives", david@orange.com writes on Feb 24: ]

> To all: > > We have an Oracle database running on a SPARCserver 2000. We are looking > into using an 18GB SPARCstorage Array for the database and have been told > that the outer half of the disk will be about twice as fast as the inner > half of the disk, and, therefore that we should only use the outer half of > the disk for performance reasons. Does anyone have experience with this? > Is this accurate?

Huh? For disks that do not use Variable Zone Recording (different number of sectors at different cylinder offsets) there are always the same number of sectors per track. The disk doesn't spin any slower in the middle. In other words, the media speed is no different in the middle.

If you arrange your data poorly, seeks to the center and then back out to the edge will cost you time. As an example, if you stipe two very busy file systems on the same disks, then you have guaranteed to move the head back and forth between them. But this is a different problem entirely.

In short, unless the number of sectors per track changes, the media speed is constant. Wasting half a disk is not warranted at all.

The caveat about Variable Zone Recording is that there will be fewer sectors per track towards the center, but a small percentage difference, not half...

l & h, kev

----- End Included Message -----

----- Begin Included Message -----

>From don@mars.dgrc.doc.ca Sun Feb 26 14:36 EST 1995 Date: Sun, 26 Feb 95 14:37:55 EST From: don@mars.dgrc.doc.ca (Donald McLachlan) To: david@orange.com Subject: Re: I/O Performance on SUN Drives Cc: Content-Type: text Content-Length: 2310

I think you will get a better rate of return using many small drives rather than fewer larger drives as access/seeks are spread across multiple spindles.

I have never heard this, but it sounds logical. Let me see ... I have a 4 G drive split (logically) in 2. Both filesystems are in approx the same state of fullness and are probably quiescent (Sunday aft).

mars don> df /drx /ars Filesystem kbytes used avail capacity Mounted on /dev/sd6g 2076848 1340552 528612 72% /drx /dev/sd6h 2076848 1397184 471980 75% /ars

mars don> dkinfo sd6 sd6: SCSI CCS controller at addr 10080000, unit # 16 3694 cylinders 21 heads 108 sectors/track a: No such device or address b: No such device or address c: 8377992 sectors (3694 cyls) starting cylinder 0 d: No such device or address e: No such device or address f: No such device or address g: 4188996 sectors (1847 cyls) starting cylinder 0 h: 4188996 sectors (1847 cyls) starting cylinder 1847

I checked the write speeds by creating a 300Mbyte file (large enough to break the cache) and the times were insignificantly different.

mars# /bin/time dd bs=1024k if=/dev/zero of=/drx/junk count=300 300+0 records in 300+0 records out 122.7 real 21.8 user 97.0 sys mars# ^drx^ars^ /bin/time dd bs=1024k if=/dev/zero of=/ars/junk count=300 300+0 records in 300+0 records out 120.4 real 22.4 user 94.1 sys

I checked the read speeds the same way. The number again are insignificantly faster different again, but they are consistent with the write test.

mars# /bin/time dd bs=1024k if=/drx/junk of=/dev/null 00+0 records in 300+0 records out 83.7 real 20.4 user 54.7 sys mars# ^drx^ars^ /bin/time dd bs=1024k if=/ars/junk of=/dev/null 300+0 records in 300+0 records out 81.6 real 20.2 user 53.5 sys

Based on my test I'd say the info you received may be correct, but the difference is so small it is not worth throwing away 1/2 the available disk space toachieve the gains.

Don

-- Donald McLachlan e-mail donald.mclachlan@crc.doc.ca Communications Research Centre / DRX office 613-998-2845 3701 Carling Ave., fax 613-998-9648 Ottawa, Ontario lab 613-998-2423 K2H 8S2

----- End Included Message -----

----- Begin Included Message -----

>From jeffw@jane.tiac.net Sun Feb 26 11:15 EST 1995 Date: Sun, 26 Feb 95 11:16:53 EST From: jeffw@jane.tiac.net (Jeff Wasilko) To: david@orange.com Subject: Re: I/O Performance on SUN Drives Reply-To: jeffw@jane.tiac.net Newsgroups: info.sun-managers Organization: Jeff's Personal System Cc: Content-Type: text Content-Length: 477

On ZBR (Zone Bit Recording) disks, the data density is higher at the outside of the disk, so the transfer rate is better. You might want to optimize by putting heavily used tables on the outside of the disk and lesser used tables on the inside of the disk.

For example, one disk (the Sun 1.3G SCSI disk) runs from 3.25-4.5 MB/sec.... -- Jeff @ Home....

"I'll be youah race-cah drivah..." -- Jewel "Pahrk youah race-cah in Havahad Yahd?" -- Anja [smoe]

----- End Included Message -----

----- Begin Included Message -----

>From mshon@sunrock.East.Sun.COM Mon Feb 27 09:02 EST 1995 Date: Mon, 27 Feb 1995 09:03:42 +0500 From: mshon@sunrock.East.Sun.COM (Michael J. Shon {*Prof Services} Sun Rochester) To: david@orange.com Subject: Re: I/O Performance on SUN Drives Cc: mshon@sunrock.East.Sun.COM X-Sun-Charset: US-ASCII Content-Type: text Content-Length: 3215

| the outer half of the disk will be about twice as fast as the inner |half of the disk,

This is true of virtually all SCSI (and probably other) disk drives made today, since it is a characteristic of the drive electronics, and not the interface.

The encoding used on the media (ZBR Zoned Bit Recording) records at a constant density, even though the surface moves under the heads at different speeds , depending on how far from the center the head is. The lowest numbered cylinders move more data past the heads per unit time than the higher cylinders.

| and, therefore that we should only use the outer half of |the disk for performance reasons.

Nonsense. You will achieve *maximum* performance from the outer part of the disk, but that does NOT mean that you should not use the rest of it.

When you lay out the disk from a functional standpoint, try to use the inner cylinders for data which is accessed less frequently, or for which the system response time does not depend. Be careful not to force the disks to make frequent long seeks from the outer to the inner cylinders. Losses due to seek time are *far* greater than those due to transfer time.

The DBA needs to know something about disk geometry, and needs to be closely involved in order to get the right things in the right places.

On the radical opposite end of the spectrum, I have heard of one person who used an entire storage array for a single striped filesystem to hold all of the files for his Oracle database. He got very high throughput and very low disk utilization (that's good) by having the load spread over so many spindles. For performance, it was essentially optimal. Backup and recovery were slightly more tricky.

There's nothing strange about Sun's disks. Just buy the largest number of the smallest disks you can reasonably use, and you will get the greatest potential I/O throughput in ops per second. Of course, it is hard to manage this kind of setup without a tool like DiskSuite or Volume Manager.

__________________________________________________________________________

/\ Michael Shon Sun Authorized Technical Consultant \\ \ Sun Microsystems, Inc. \ \\ / 345 Woodcliff Drive / \/ / / Fairport, NY 14450-4284 / / \//\ \//\ / / Phone: (716) 385-5065 / / /\ / Fax: (716) 385-8754 / \\ \ EMail: michael.shon@East.Sun.COM \ \\ \/

|From sun-managers-relay@ra.mcs.anl.gov Fri Feb 24 22:35 EST 1995 |Sender: sun-managers-relay@ra.mcs.anl.gov |Precedence: junk |Date: Fri, 24 Feb 1995 10:44:16 +0500 |From: david@orange.com |Reply-To: david@orange.com |Followup-To: junk |To: sun-managers@eecs.nwu.edu |Subject: I/O Performance on SUN Drives |Cc: david@orange.com | |To all: | |We have an Oracle database running on a SPARCserver 2000. We are looking |into using an 18GB SPARCstorage Array for the database and have been told |that the outer half of the disk will be about twice as fast as the inner |half of the disk, and, therefore that we should only use the outer half of |the disk for performance reasons. Does anyone have experience with this? |Is this accurate? | |David James |Orange Systems |david@orange.com |

----- End Included Message -----

----- Begin Included Message -----

>From Mark_Mellman@msmail.mpi.com Mon Feb 27 09:48 EST 1995 To: david@orange.com (david) From: Mark_Mellman@msmail.mpi.com Organization: Millennium Pharmaceuticals, Inc. Date: Mon, 27 Feb 1995 09:50:26 EDT Subject: RE: I/O Performance on SUN Drives Content-Type: text Content-Length: 476

Hi David,

I was once involved in tuning an Oracle DB on a SPARC machine. We did find that using the outer half of the disk does increase performance, but not twice. In my opinion, using RAID in the proper way, with your 18 1GB drives, should give you the performance you need. Please summarize, I would like to see what other people have to say on this matter. One person you may want to get to respond to this would be Hal Stern from Sun.

Mark Mellman mellman@mpi.com

----- End Included Message -----

----- Begin Included Message -----

>From strombrg@hydra.acs.uci.edu Mon Feb 27 20:02 EST 1995 To: david@orange.com Subject: Re: I/O Performance on SUN Drives Date: Mon, 27 Feb 1995 17:04:01 -0800 From: Dan Stromberg - OAC-CSG <strombrg@hydra.acs.uci.edu> Content-Type: text Content-Length: 971

If you're using a disk with a variable number of sectors per track, you end up needing fewer track-to-track seeks, to read the same amount of data, when doing that reading from the outer tracks - because they are longer, they can hold more sectors.

Note that not all disks have a variable number of sectors per track. I suspect (I'm not a disk expert!) that newer drives tend to have that, tho.

In message <9502241544.AA15620@orange.com>you write: >To all: > >We have an Oracle database running on a SPARCserver 2000. We are looking >into using an 18GB SPARCstorage Array for the database and have been told >that the outer half of the disk will be about twice as fast as the inner >half of the disk, and, therefore that we should only use the outer half of >the disk for performance reasons. Does anyone have experience with this? >Is this accurate? > >David James >Orange Systems >david@orange.com

Dan Stromberg - OAC/CSG strombrg@uci.edu

----- End Included Message -----

----- Begin Included Message -----

>From ncanas@mickey.Solsuni.pt Wed Mar 1 08:39 EST 1995 Date: Wed, 1 Mar 1995 14:33:01 +0100 From: ncanas@mickey.Solsuni.pt ((Nuno.Canas) Presales Manager) To: david@orange.com Subject: Re: I/O Performance on SUN Drives X-Sun-Charset: US-ASCII Content-Type: text Content-Length: 250

Yes it is !!!

I've some experience with that. You should also use disk stripping with all disks.

You shoul use (if you can) about 10% of the size of the disks. Let's say 100 or 150 MB per 1.05 GB disk in order to increase the performance.

//Nuno

----- End Included Message -----



This archive was generated by hypermail 2.1.2 : Fri Sep 28 2001 - 23:10:17 CDT