Well the final polls are in. Most of the responses were for more information on DISK_PAK.
I did receive some replies on disk fragmentation.
----- Begin Included Message -----
>From adam%bwnmr4@harvard.harvard.edu Tue Sep 7 11:50:33 1993
Return-Path: <adam%bwnmr4@harvard.harvard.edu>
From: adam%bwnmr4@harvard.harvard.edu (Adam Shostack)
Subject: Re: Disk Pak for Unix
To: twhitely@tr1072.to.ford.com
Date: Tue, 7 Sep 93 11:55:47 EDT
Content-Length: 1162
You wrote:
| Hello Sun Guru's
|
| I am currently looking at a third party software package called Disk
| Pak for Unix. It is a disk defragmenter. It will show you how you
| disk is set up and also the arrangement of the disk as
| far as files, inodes and free disk space with any fragmentation present. My biggest concern
| with this package is that this package will break down the disk and reorder the disk. This
| brings up a question of liability. Has anyone used Disk_Pak? If so, how do you favor this package?
| Please email me all criticisms and concerns to twhitely@tr1072.to.ford.com. If you are also interested
| in some info on this package, send me email.
I'd use dump/restore if you really feel your disks need
degfragmenting. But, if you look at the way inodes work,
fragmentation is not nearly so big a problem on unix as under dos.
Adam
-- Adam Shostack adam@bwnmr4.harvard.edu Systems Manager 617-732-7692 Surgical Planning Lab, Dept of Radiology Fax 732-7963 Brigham and Womens Hospital, Boston----- End Included Message -----
----- Begin Included Message -----
>From @srlns1.srl.ford.com,@vm.cnuce.cnr.it:ACHILLM@IMIHSRA.BITNET Wed Sep 8 03:36:58 1993 Return-Path: <@srlns1.srl.ford.com,@vm.cnuce.cnr.it:ACHILLM@IMIHSRA.BITNET> Date: Wed, 08 Sep 93 09:31:29 GMT From: Martin Achilli <ACHILLM%IMIHSRA.BITNET@vm.cnuce.cnr.it> Subject: Re: Disk Pak for Unix To: Ted Whitely <twhitely@tr1072.to.ford.com> Content-Length: 1152
Hello, up to now,what I do is to (yearly) back up and restore all filesystems complete ly. I do a double backup for safety. And before restoring, I rebuild filesystem s on the disks. This operation improves large file access performance (we work with Magnetic Re sonance and Nuclear Medicine data: ave file size 1 Mb). I suppose the program is like the DOS compress utility, so presumably you must have at least one back up of your disks data. Please send me more info, although I expect a program like this costs about 500 dollars (for one host ?), so perhaps, if you don't have too many disks, it is cheaper to do what I do once a year.. :-)
Martin
------------------------------------------------------------------------- Martin Achilli Consiglio Nazionale Delle Ricerche * Tel: +392/26433648 Ist. di Neuroscienze e Bioimmagini * Fax: +392/26415202 c/o Medicina Nucleare * Email: ACHILLM at HSR.IT (Internet) Ospedale San Raffaele * at IMIHSRA (Bitnet) Via Olgettina 60 * 20132 MILANO (Italy) * - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
----- End Included Message -----
----- Begin Included Message -----
>From dsnmkey@guinness.ericsson.se Wed Sep 8 04:32:33 1993 Return-Path: <dsnmkey@guinness.ericsson.se> Date: Wed, 8 Sep 93 10:31:49 +0200 From: dsnmkey@guinness.ericsson.se (Martin Kelly) To: twhitely@tr1072.to.ford.com Subject: Re: Disk Pak for Unix Content-Length: 1560
Hello,
> If you are also interested in some info on this package, send me email
I'm interested. Can you semd me an info pack ?
> This brings up a question of liability.
It is most likely that the Disk Pak utility has no 'liability' if you lose data or software. You are expected to provide your own backups. You can minimize the risk of data loss by dumping the disk to a free filesystem or disk (rather than tape), running the Disk Pak program the way you want to, and then run a diff on all the files and directories (this may take a while). Then you will know that the Pak'd disk contains the exact same data as the backup and the original disk.
If you meant to ask about reliability, then I do not know how reliable it is. I guess you will have to ask the vendors.
However, the question arises if sufficient gains will be made even after Pak'ing your disk. This is very much data set dependent and application dependent. Your mileage may vary .....
Best Regards,
Martin.
============================================================================ Martin Kelly Tel: +31 1612 29358 Unix Analyst/Specialist Fax: +31 1612 29071 Ericsson Data Services Nederland, BV. (DSN) PO Box 209 5120 AE Rijen MEMO: ERI.DSN.DSNMKEY The Netherlands E-Mail: dsnmkey@guinness.ericsson.se DSN - Providers of Unix and Networking Expertise to the Corporation ============================================================================
----- End Included Message -----
----- Begin Included Message -----
>From bobr@houston.nam.SLB.COM Wed Sep 8 09:18:05 1993 Return-Path: <bobr@houston.nam.SLB.COM> Date: Wed, 8 Sep 93 08:17:37 CDT From: bobr@houston.nam.SLB.COM ( Bob Reardon ) To: twhitely@tr1072.to.ford.com Subject: Re: Disk Pak for Unix Content-Length: 4654
Here is the reply from a previous question about fragmentation. Not exactly what you asked about but may be useful anyway.. ========================================================================
>From sun-managers-relay@ra.mcs.anl.gov Fri Jun 11 21:04:17 1993 Return-Path: <sun-managers-relay@ra.mcs.anl.gov> Received: from ra.mcs.anl.gov by houston.nam.SLB.COM (4.1/SMI-4.9) id AA11374; Fri, 11 Jun 93 21:04:14 CDT Received: by ra.mcs.anl.gov id AA19287 (5.65c/IDA-1.4.4 for sun-managers-outbound); Fri, 11 Jun 1993 11:40:58 -0500 Sender: sun-managers-relay@ra.mcs.anl.gov Received: from delta.eecs.nwu.edu by ra.mcs.anl.gov with SMTP id AA19281 (5.65c/IDA-1.4.4 for <sun-managers@ra.mcs.anl.gov>); Fri, 11 Jun 1993 11:40:52 -0500 Precedence: bulk Received: from cgi.com (cggate.cgi.com) by delta.eecs.nwu.edu with SMTP id AA04484 (5.65c/IDA-1.4.4 for <sun-managers@eecs.nwu.edu>); Fri, 11 Jun 1993 11:40:33 -0500 Date: Fri, 11 Jun 1993 12:43:19 -0400 (EDT) From: HILL@cgi.com (Bear) Reply-To: HILL@cgi.com (Bear) Followup-To: junk Message-Id: <930611124319.240020f5@CGIVA> Subject: Fragmentation Summary To: sun-managers@eecs.nwu.edu X-Vmsmail-To: SMTP%"sun-managers@eecs.nwu.edu" Content-Length: 3412 X-Lines: 94 Status: RO
First off..Thanks to all who responded. You're all such a knowledgable crowd. Also my apologies to those I may have missed. In summary:
1. Dump the disk to tape, newfs the filesystem, and restore the tapes. 2. Try to tune with tunefs or mkfs parameters 3. Unix Central catalog lists a product called Eagles Software Disk_Pak (800)532-1771 (913)823-7257 4. Spend your time instead by placing often-used common filesystems in the center of the disk, minimizing the seek time to go to the user filesystems at either end. 5. You could try a fsck while unmounted...this will show %fragmentation. 6. And the most interesting is the following:
>From: SMTP%"stern@sunne.east.sun.com" 11-JUN-1993 11:11:31.26 To: HILL@cgi.com CC: Subj: Re: Anyone know of good defragmentation software?
Date: Fri, 11 Jun 93 11:07:40 EDT >From: stern@sunne.east.sun.com (Hal Stern - NE Area Systems Engineer) Message-Id: <9306111507.AA14455@sunne.East.Sun.COM> To: HILL@cgi.com Subject: Re: Anyone know of good defragmentation software?
be careful about "fragmentation". there is no internal fragmentation in the UNIX filesystem -- files only contain fragments in the last data block, never in the middle of a file. so your file will comprise full data blocks for most/all of its extent.
where you run into problems is with block placement, and "working set distribution". the optimal block placement strategy in BSD unix (FFS) tries to drop the blocks down so that you minimize rotational delay. the sunos 4.1.x UFS+ puts the blocks down in "clusters" of 56k, making longer contiguous stretches that give you much improved performance with SCSI disks (or other disks that move a fairly large buffer in one shot). the block placement can run into problems with a full disk, and end up putting blocks where you have to seek/rotate quite a bit to read the whole file. the file isn't exactly fragmented, but it is suboptimally laid out.
finally, there's the issue of what files you're using and how they are arranged on the disk. if you can keep a whole working set together, again you minimize seek times.
what can you do? (a) dump and restore the filesystem. this palces all blocks optimally (b) use something like nfswatch to track what files are used most frequently. then run restore with a list of files, and restore them in "favored" order. this lets you keep frequently used files together to cut down on seek times.
--hal
This summary was made possible from Knowledge grants from the following Contributors.
SMTP%"jdavis@cs.arizona.edu" SMTP%"fetrow@biostat.washington.edu" SMTP%"Birger.Wathne@vest.sdata.no" SMTP%"smc@goshawk.lanl.gov" SMTP%"leclerc@clamart.est.slb.com" SMTP%"lsil!mhost!lonfs01!allen@fernwood.mpk.ca.us" SMTP%"barnes@sde.mdso.vf.ge.com" SMTP%"schmelin@cad.gmeds.com" SMTP%"jeff@access.digex.net" SMTP%"lem@cgi.com" SMTP%"poc@cgi.com" SMTP%"lemke@mitl.research.panasonic.com" SMTP%"rwolf@dretor.dciem.dnd.ca" SMTP%"bert@penril.com" SMTP%"long-morrow@cs.yale.edu" SMTP%"stern@sunne.east.sun.com" SMTP%"exudnw@exu.ericsson.se" SMTP%"morrow@cns.ucalgary.ca"
Thanks, -Bear -------------------------------------------------------------------------- Scott A. Hill (Bear) Senior Hardware/Network Engineer Carnegie Group Inc. 5 PPG Place Pittsburgh, PA 15222 (412)-642-6900 --------------------------------------------------------------------------
----- End Included Message -----
----- Begin Included Message -----
>From jeff@access.digex.net Wed Sep 8 16:26:20 1993 Return-Path: <jeff@access.digex.net> Date: Wed, 8 Sep 1993 16:25:48 -0400 From: Jeff Mallory <jeff@access.digex.net> To: twhitely@tr1072.to.ford.com Content-Length: 2907
I got a catalog from UnixCentral listing this product and read its claims. Piqued my interest `cause it seemed to try to draw parallels between DOS-style file cluster usage and Unix inode/cylinder group usage. Having just read a (most excellent) Hal Stern article about unix file systems, I said they cannot do what they are claiming because the wrong they say they right doesn't exist in UFS. So, I called UnixCentral and asked for more product info, then faxed them parts of Hals` article that basically rebutted the whole idea of unix file fragmentation. A salesperson called me and said they had passed on the info to Eagle (the makers of Disk Pak) and they were writing up a response (this was ~2 months ago.....).
So, I sent a note to Hal laying out this interchange and he replied that SunWorld (I think) had approached him about reviewing Disk Pak and he had told them then what I had sent to UnixCentral about unix "fragmentation" of disks.
(sounds of rummaging through 'net notes), A ha! It was actually in a sunmgr reply that Hal talked about defragging:
>From: SMTP%"stern@sunne.east.sun.com" 11-JUN-1993 11:11:31.26 To: HILL@cgi.com CC: Subj: Re: Anyone know of good defragmentation software?
Date: Fri, 11 Jun 93 11:07:40 EDT >From: stern@sunne.east.sun.com (Hal Stern - NE Area Systems Engineer) Message-Id: <9306111507.AA14455@sunne.East.Sun.COM> To: HILL@cgi.com Subject: Re: Anyone know of good defragmentation software?
be careful about "fragmentation". there is no internal fragmentation in the UNIX filesystem -- files only contain fragments in the last data block, never in the middle of a file. so your file will comprise full data blocks for most/all of its extent.
where you run into problems is with block placement, and "working set distribution". the optimal block placement strategy in BSD unix (FFS) tries to drop the blocks down so that you minimize rotational delay. the sunos 4.1.x UFS+ puts the blocks down in "clusters" of 56k, making longer contiguous stretches that give you much improved performance with SCSI disks (or other disks that move a fairly large buffer in one shot). the block placement can run into problems with a full disk, and end up putting blocks where you have to seek/rotate quite a bit to read the whole file. the file isn't exactly fragmented, but it is suboptimally laid out.
finally, there's the issue of what files you're using and how they are arranged on the disk. if you can keep a whole working set together, again you minimize seek times.
what can you do? (a) dump and restore the filesystem. this palces all blocks optimally (b) use something like nfswatch to track what files are used most frequently. then run restore with a list of files, and restore them in "favored" order. this lets you keep frequently used files together to cut down on seek times.
--hal
Jeff Mallory jeff@millie.loc.gov
----- End Included Message -----
----- Begin Included Message -----
>From ups!uniq.com.au!glenn@warrane.connect.com.au Wed Sep 8 21:23:46 1993 Return-Path: <ups!uniq.com.au!glenn@warrane.connect.com.au> Date: Thu, 9 Sep 93 08:11:26 EST From: glenn@uniq.com.au (Glenn Satchell - Uniq Professional Services) To: twhitely@tr1072.to.ford.com Subject: Re: Disk Pak for Unix Content-Length: 2831
With the SunOS' BSD Fast File System disk fragmentation is not such a big deal any more. Normally I just do a full backup/newfs/restore about once per year to defragment my disks.
regards, -- Glenn Satchell glenn@uniq.com.au | "This is a unix system. Uniq Professional Services Pty Ltd ACN 056 279 335 | I can do this easy." PO Box 70, Paddington, NSW 2021, (Sydney) Australia | Phone 02 360 7434 Pager 016 287 000 Fax 02 331 2572 | - Lex, Jurassic Park "Sun Accredited System Consultants" |
----- End Included Message -----
I don't believe that a Sun filesystem fragments constantly. Most replies quote that defragmenting was done once a year. This is good for users that have a very small network, but for my case, it can be a nightmare cause I have a very large network with over 550 nodes! We also have over 30 servers containing more than 4gig's of disk space. Disk_Pak for unix will let you defragment disks actively or interactively. By using disk pak interactively, defragementing can be done overnight or over the weekend. Whether or not the software is reliable, every good administrator knows to keep up to date backups of everything in the event of lost data. Disk_Pak also has an X interface that allows you to scan the disk to see how badly fragmented it is. A large network which might require a network license, the cost would be about $3000. For those who would like more info or prices on Disk_Pak for unix, here is the following information:
Eagle Software (Disk_pak for Unix) 123 Indiana Avenue P.O. Box 16 Salina, Kansas 67402-0016 Phone (913) 823-7257 Fax (913) 823-6185 Contact: Darin Boyer, Products Representative
PS. Thanks for everyone's responses.
############################################################################# # # # _____ ______/ ___________/ _____________ _/ _/ # # _/ _/ __ / _/ _/ # # _/ ___________/ __ _/ / _/ _/ _/ # # _/ _/ __ / _/ _/ _/ # # __/ ___________/ ______________/ _/ _/ __/ _/ # # # # Ted Whitely Unix Tech. Support # # System Administrator Ford Motor Company # # Truck Operations Danou Technical Center # # TSD Systems Group Suite 6000 # # Internet: twhitely@tr1072.to.ford.com 16630 Southfield Rd. # # Phone: (313) 390-5259 Allen Park, MI 48101 # # FAX #: (313) 845-7501 U.S.A. # #############################################################################
This archive was generated by hypermail 2.1.2 : Fri Sep 28 2001 - 23:08:10 CDT