SUMMARY: Re: Splitting a monster file

From: Mark Drummond (mark.drummond@rmc.ca)
Date: Fri Jul 14 2000 - 20:57:14 CDT


Lot's of great ideas, some of which I had thought of, others not. Many
of them will work, I am just to damn lazy to do them. To many people
answered so to heck with ya, I'm not naming all of you. plsssssst!

Here are the ideas, in a nutshell as it were.

1. moving the file to tape was perhaps the most popular. Put it on tape,
then read it back in running the input through split. I like this one,
had not thought of it.

2. move the file to someplace where there is sufficient space (~5GB)
(that came to mind), or better yet, just direct split to send it's
output files to some other filesystem that has sufficient space
(~2.5GB).

3. compress the file, then split it, then uncompress the splits. This
came to mind. Would this actually work? I may give this one a whirl just
for the sake of trying it. Would splitting the compressed file not, at
the least, hack up some lines in the log file?

4. "window" sections of the file using head and tail. Never thought of
that one.

5. use dd though this might hack up some lines, no?

6. write your own program (perl or C) to do the job for you. though of
it but again, I'm lazy. My general attitude is that there has to be some
way of handling this task from the command line.

Some people wondered why I wanted to chop it up. webalizer would not
accept it as input, presumably for the same reason vi can't open it ...
vi (and others) are not 'largefile' compatible.

-- 

- Hail, hail, the gang's all here -

S U BEFORE POSTING please READ the FAQ located at N ftp://ftp.cs.toronto.edu/pub/jdd/sun-managers/faq . and the list POLICY statement located at M ftp://ftp.cs.toronto.edu/pub/jdd/sun-managers/policy A To submit questions/summaries to this list send your email message to: N sun-managers@ececs.uc.edu A To unsubscribe from this list please send an email message to: G majordomo@sunmanagers.ececs.uc.edu E and in the BODY type: R unsubscribe sun-managers S Or . unsubscribe sun-managers original@subscription.address L To view an archive of this list please visit: I http://www.latech.edu/sunman.html S T



This archive was generated by hypermail 2.1.2 : Fri Sep 28 2001 - 23:14:12 CDT