Fragmentation Summary

From: Bear (HILL@cgi.com)
Date: Fri Jun 11 1993 - 19:51:00 CDT


First off..Thanks to all who responded. You're all such a knowledgable crowd.
Also my apologies to those I may have missed. In summary:

1. Dump the disk to tape, newfs the filesystem, and restore the tapes.
2. Try to tune with tunefs or mkfs parameters
3. Unix Central catalog lists a product called Eagles Software Disk_Pak
   (800)532-1771 (913)823-7257
4. Spend your time instead by placing often-used common filesystems in the center
   of the disk, minimizing the seek time to go to the user filesystems at either
   end.
5. You could try a fsck while unmounted...this will show %fragmentation.
6. And the most interesting is the following:

>From: SMTP%"stern@sunne.east.sun.com" 11-JUN-1993 11:11:31.26
To: HILL@cgi.com
CC:
Subj: Re: Anyone know of good defragmentation software?

Date: Fri, 11 Jun 93 11:07:40 EDT
>From: stern@sunne.east.sun.com (Hal Stern - NE Area Systems Engineer)
Message-Id: <9306111507.AA14455@sunne.East.Sun.COM>
To: HILL@cgi.com
Subject: Re: Anyone know of good defragmentation software?

be careful about "fragmentation". there is no internal
fragmentation in the UNIX filesystem -- files only
contain fragments in the last data block, never in
the middle of a file. so your file will comprise
full data blocks for most/all of its extent.

where you run into problems is with block placement,
and "working set distribution". the optimal block
placement strategy in BSD unix (FFS) tries to drop
the blocks down so that you minimize rotational delay.
the sunos 4.1.x UFS+ puts the blocks down in "clusters"
of 56k, making longer contiguous stretches that give
you much improved performance with SCSI disks (or other
disks that move a fairly large buffer in one shot).
the block placement can run into problems with a full
disk, and end up putting blocks where you have to
seek/rotate quite a bit to read the whole file. the
file isn't exactly fragmented, but it is suboptimally
laid out.

finally, there's the issue of what files you're using
and how they are arranged on the disk. if you can
keep a whole working set together, again you minimize
seek times.

what can you do?
(a) dump and restore the filesystem. this palces all blocks
        optimally
(b) use something like nfswatch to track what files are
        used most frequently. then run restore with
        a list of files, and restore them in "favored" order.
        this lets you keep frequently used files together
        to cut down on seek times.

--hal

This summary was made possible from Knowledge grants from the following Contributors.

SMTP%"jdavis@cs.arizona.edu"
SMTP%"fetrow@biostat.washington.edu"
SMTP%"Birger.Wathne@vest.sdata.no"
SMTP%"smc@goshawk.lanl.gov"
SMTP%"leclerc@clamart.est.slb.com"
SMTP%"lsil!mhost!lonfs01!allen@fernwood.mpk.ca.us"
SMTP%"barnes@sde.mdso.vf.ge.com"
SMTP%"schmelin@cad.gmeds.com"
SMTP%"jeff@access.digex.net"
SMTP%"lem@cgi.com"
SMTP%"poc@cgi.com"
SMTP%"lemke@mitl.research.panasonic.com"
SMTP%"rwolf@dretor.dciem.dnd.ca"
SMTP%"bert@penril.com"
SMTP%"long-morrow@cs.yale.edu"
SMTP%"stern@sunne.east.sun.com"
SMTP%"exudnw@exu.ericsson.se"
SMTP%"morrow@cns.ucalgary.ca"

Thanks,
-Bear
--------------------------------------------------------------------------
Scott A. Hill (Bear)
Senior Hardware/Network Engineer
Carnegie Group Inc.
5 PPG Place
Pittsburgh, PA 15222
(412)-642-6900
--------------------------------------------------------------------------



This archive was generated by hypermail 2.1.2 : Fri Sep 28 2001 - 23:07:55 CDT