Some weeks ago I posted a question about using a dedicated Sparc 5 for
a combined WWW, news, ftp, etc server. Here is a summary of the responses I
received:
A. The consensus was that a Sparc 5 would be adequate. The one real
concern was that it was upgradable only by replacing the box.
B. The real bottleneck would be disk and network access. Buffered ethernet
and multiple SCSI buses were recommended.
C. The most cpu intensive process would be mail. There was a split as
to whether mail should have its own machine or not. The second most cpu
intensive process would be news.
D. The local console should not be used for other than maintaining the
machine.
E. Ordinary users should not have logins on this machine.
F. Estimates for the size of the news partition ranged from 2 to 3 GBytes.
We will probably go the Sparc 5 route. At this time we are looking at 48
MBytes of memory, about 10 GBytes of disk, and 2 SCSI busses. We will try putting
mail on this machine initially and see what happens (we have about 650 users,
somewhat less than those people who recommended against mail on the same machine
-- we are going to be the WWW server for the whole college but mail, ftp, news,
etc will be for the department only).
There is a usenet calculator at:
http://www.netpart.com/janus/usenet.html
The calculator says that assuming 30 day expiration for comp and sci, and 7 days
for everything else but alt, we will need about 5.5 GBytes in two years. Alt
would add about 2.5 to that.
Thanks for everyone's help.
Here is the original post followed by (somewhat edited) answers:
> Our department has decided to purchase a machine to be used as a WWW
> server for the college which has about a dozen departments. We anticipate
> fairly heavy use of the server, mostly by current students. We have also
> been considering using the same machine for a department mail, news, and
> ftp server. I am looking for some advice on the advisability of this
> course of action and any suggestions about specifics.
>
> A. Is it reasonable to put all these functions on one separate
> machine? Can we expect to be able to use the machine for any
> other purpose (for example, the machine the administrator uses
> to monitor the network)?
>
> B. I recall hearing that there was about 1GByte of disk space
> necessary now to maintain a full complement of news groups (we
> are currently running a small subset). Is that correct? Would
> twice whatever is the minimum necessary space be advisable?
> Four times?
>
> C. How much processing power is necessary for this server? Can we
> get away with a Sparc 5? Do we have to have something with more
> power? Is there any special hardware we should consider for this
> machine (like hardware for faster disk access)?
>
> These are the principal questions I have, but any relavent advice would
> be appreciated (especially if there are important questions I forgot to ask).
=================================================================================
*********************************************************************************
Let me describe what we have done for our college, and you can
interpolate to your situation. Please also note that a lot is
dependant upon the skill and flexibility of the system adminstrator,
support staff, and the SA's supervisor, as well as what the general
user will want to accomplish.
The College of Liberal Arts & Sciences runs an environment very much
like you have described. A single Sparc 10 Model 30 handles Usenet,
WWW, gopher, ftp, SMTP, Majordomo (list-server), XNTP (network time
protocol), DNS, etc for our entire college. As you see, these are all
"services-at-large" that do not require local (ie, "shell") access.
The minute you actually have the ability for users to telnet/rlogin
into the machine and run their own applications, you will need much
more power.
The accounts on the system (with the exception of the administrators
and info-maintenence accounts) cannot actually run programs on our SS
10/30. They _can_ telnet in and check their email via a locally
modified copy of "pine", but they do not have shell or ftp access.
Currently, only a selected few have the ability to actually have "home
pages" for WWW, but we plan to change this by NFS-mounting our Novell
server sometime in the near future.
Currently, we serve email, DNS, and WWW for twenty-one departments in
our college. The BIND also runs as a secondary nameserver for campus.
Our machine is one of two campus-wide NNTP servers, and thus not only
serves the college, but serves every other college to some degree.
The load via 'uptime(1)' can often get up to three or four during the
day. However, due to how the machine has been configured for such a
"server" role, performance is only slightly impaired. However, read
my comments about using the console below.
The machine is configured as follows:
Model 30 CPU module
64 megs RAM (148 megs swap)
local X console for administrator
6.5 gigs local HD
buffered ethernet port & dual scsi buses
local CD-ROM (for software installation)
local 8mm (8500) (for system backups)
Major software:
SunOS 4.1.3, w/ custom kernel
NCSA httpd (1.1) [WWW]
INN (1.4sec) [NNTP]
Sendmail (8.6.9) [SMTP]
Majordomo (1.92) [Mail-lists]
XNTP (3.x) [NTP]
BIND (4.9.2b) [DNS]
Gopher (1.1)
Qualcomm POPd (2.1.1) [POP]
Currently, I do use the local console. However, we are going to
change that fact sometime in the near future. During the day, there
can be enough users causing a big enough "hit" that using the console
is just not feasable (I have been told that the OS swaps out the X
server and local X apps to free extra buffers for disk access). The
administrator should probably have a second, much smaller box to
actually use the console of. Even an old SS1 would work for this use.
We have a 2.1 gig disk dedicated to our Usenet partition on the second
SCSI bus. This allows us to have a six-day expiration, except for the
alt.* groups, which are three days. Interpolate this to your wished
expiration times. We do not restrict any newsgroups.
For services-at-large, a properly configured sparc 5 with no one using
the local console should provide acceptable performance. However, you
may want to look at a more expandable system for enhancements down the
road. We are also considering replacing the current single model 30
CPU with a dual model 41. If you get a Sparc 5, such future upgrades
without replacing the box itself will be impossible.
Of course, if you want to actually let people login to the machine and
run any processes that they want to, I would suggest getting two
machines -- one of the services, and one for the users.
No matter what people tell you, don't skimp on the RAM. More physical
memory means less swapping, and performance increases dramatically.
I've seen some CPU-poor machines perform quite admirably with
sufficient memory, whereas faster CPU boxes with less memory actually
"feel" slower than the slow machine with the added memory.
You have asked some good ones, except you haven't stated how the users
will be accessing the system. Pur users don't usually directly
"login" to the system. Instead, they use their client software to
connect to the server.
*********************************************************************************
=================================================================================
*********************************************************************************
Not with all that stuff.. We've got 1400+ users on one machine and
another one handles news, www, and gopher. We are also putting
anything else on it.. But that's about it.. It's still not heavily
loaded, a sparc 2. But as it gets used more and more expect heavy loads.
If it's used for mail, make it a mail host, and have users log in on
another machine.. But even then, I wouldn't... The file system on it
will get taxed heavily, between spooling mail, home directories and
the like.. Best thing is to put your mail system on one and the other
things like web site on another box..
I heard it was a minimum 2 gig for one week of news.
Like I said, we've got a sparc 2 (basicaly a slow sparc 5) and it's
handled it all well for now.. We've got all our users on another
machine, and it does get hammered when there's heavy work on it, but
we've installed a packaged called nov that sped up the indexing for
news so now recall time isn't so bad for our news. As for network
admin, we've got a dedicated RS/6000 that we use for experiments and
soon to be network admin. Right now we run etherman/interman and the
like off of our www box. Load never goes over 2 unless something is
going on abnormal.
*********************************************************************************
=================================================================================
*********************************************************************************
I would be very interested to hear whether a lot of people recommend
a Linux 486 or Pentium for this ...
*********************************************************************************
=================================================================================
*********************************************************************************
It depends on the expected load. How many simultaneous users (for each
service) do you expect to serve? Do you intend to support a full or just
a partial newsfeed?
Mail, news, ftp, and www are all orthogonal services, so it's perfectly
possible (and a good idea!) to set up separate servers.
Assume news will be twice the size it is now in 18 months, perhaps less.
Fast disks will help. A SS5 might do, depending on the expected load.
*********************************************************************************
=================================================================================
*********************************************************************************
I can't comment on all your questions, but one you ask is about the space
required to provide news service. At my last assignment, I setup and ran a
news server. I found that in order to keep a full compliment of newsgroups with
reasonable expiration I needed 1.5-2.0 gig for my news spool. Your mileage may
vary, particularly since that server carried a lot of internal newsgroups and
it was a very large company with a lot of internal traffic. On the otherhand,
I had about 1.2 gig and kept my head above water only by accelerating expiration
of newsgroups that tended toward large file sizes. Given you are running a
more general setup, I would believe this is not a real option for you. Oh,
I should also point out that I was not keeping an overview database for threaded
newsreaders due to my space shortage and the newsreaders the people I was
serving used (none used threading).
*********************************************************************************
=================================================================================
*********************************************************************************
When I was working for UF in the CIS department, we had mail, news, and ftp on
one (old) IBM/RT. It died under the strain and was completely useless. We
took a Sparc2, moved news and ftp to it, added gopher and WWW (gopher pretty
much is unused now). It handled the load pretty well, but when we tried to add
mail it became unusable. We moved on to a Sparc10 and moved mail off to a
seperate (IPC) machine. Both these systems are usable and the loads aren't
overly high. However, we found that mail was quickly outgrowing the IPC. (We
have about 1000 users at UF CIS). I would suggest putting mail and news on a 5
or a 10 and FTP, WWW, gopher, etc on another 5 or 10.
We had 1.4 gigs for news and kept a 1 week expire for the big 7+ gnu, alt, and
local/state groups. We do expire really big groups (like alt.binaries.*)
nightly though. We have 1.4 gigs for FTP and use most of it at any one time.
Of course, we also have a lot of distribution stuff on there, some of which
takes up a couple of hundred meg each. We have a 1 gig disk on the mail server
and it runs at about 30-40% usually. I'd suggest at least a 2 gig disk for
everything to start, a four would be better.
*********************************************************************************
=================================================================================
*********************************************************************************
If it is powerful enough, yeah. You are going to have a LOT of
disk usage, which leads me to believe that you would be better
off with two sparc 5's or 10's rather than one sparc 1000 or
whatever.
I would guess that, depending on your expiration settings, 1.5
gig is a minimum.
I really don't think you have to worry too much about processing
power. Your problem is going to be the network and disk access.
*********************************************************************************
=================================================================================
*********************************************************************************
This is typically what is done. The internet services server
is put in the DMZ ( De Militarized Zone ), or on a subnet that is
closest to the internet, and firewalls between it, and the other
subnets. To give you a head start, give this machine a name, like
benji, and then give it the aliases of www, ftp, news, ns, etc. That
way, if you were ever to pull those functions from that machine,
you would just move the aliases. ( I would add dns secondary to
this machine as well.... )
Since your post appeared on a Sun group, I would say anthing
it could handle. I would add it to the list of machines that could
be telnet'ed to from the outside with Skey, perhapps. I would take
off NIS, NFS, etc. You may want to read firewalls ( email firewalls-majodomo@
greatcircle.com with body of subscribe ).
I just talked to our news providers about that. I would give
it 3+ gb for the full distribution, just to make it to midyear. The
new PC's and MAC's that slip to the internet have not yet peaked, and
news is filling up 2 gb easy.
We have a machine much like you are talking about. It is hampered
first by mail ( it is also our mail relay ), then news, then ftp,
then www. It is a SUN3 ( it barely keeps up ). So a sparc5 should
be fine. If you get anything faster, you will be hampered by your network
hardware ( unless you have fddi, switched eithernet ).
You should work with your network security people on this. Once you
have a very public server like this, the internet will hammer it. Unless
you decide that it is not to have internet access at all. That is probably
a mistake, IMHO. This machine should be either put on a DMZ subnet, or
secured. Our ftp and www get about 2000 inquieries per month which is
mostly customers. With a student users group, it should be considerably
higher. Even with our low usage, we have had numerous crackers attack
our net ( not successfully, of course :-).
*********************************************************************************
=================================================================================
*********************************************************************************
This really depends on a few things. Do you think that your site will really be
using WWW so much that a dedicated CPU is necessary? Do you intend to run httpd
out of inetd? If so, then you probably will want a dedicated server. There
is also the security concern that if someone finds a security hole in httpd or
somewhere else in the WWW world, at least they've only comprimised a single host
and you can control how much damage they can do.
If you do this for security reasons, then you will probably not want to use
the machine for things like network monitoring. However, if that is not your
concern, then you may, depending on how much use it's getting and how big your
CPU is.
At my site, for example, WWW traffic isn't really high enough that a dedicated
server is necessary so I have it on the same machine that does news and
anonymous ftp. Again, no critical functions so if it gets hacked, we don't go
down, but enough work to justify its own machine.
News will eat as much space you allow it to. We have a 1GB disk for news and
we don't get all newsgroups, although we get many of the USENET groups (at last
count we are getting 1,077 newsgroups out of a possible 9,841, call it 10%.) Our
disk under normal conditions runs from 40-65% full. We like having the extra
space in case of the occasional floods. We expire almost all of our groups in
5-7 days. All of these factors need to be considered when choosing a size. We
find 1GB works well, you may need more. Again, this isn't an exact science so
I really can't give you an exact number but perhaps you can use my numbers as a
guideline.
We are currently using a Sun 4/300 but are finding that to be too slow (mostly
because of news which can get pretty CPU hungry). We have found that a SPARC
2 works well (why they changed it down to a 4/300 I don't know). I'm not really
familiar with the SPARC 5, but a SPARC 2, I think is a 40 Mhz box. I think
you're probably fine. Unless you have very high and very critical data on this
box, a SPARC-10 or bigger is probably overkill. We don't consider news and WWW
critical but some sites might.
*********************************************************************************
=================================================================================
*********************************************************************************
I'm getting away with a SPARC ELC w/ 12MB of memory for all that.
It sits on my desk.
WWW servers can be surprisingly lightweight.
*********************************************************************************
=================================================================================
*********************************************************************************
We have all of ours on one machine. universe.concorde.com is CNAMEd
as news.concorde.com, www.concorde.com, and ftp.concorde.com. It's
also our backup machine and about to be turned into our Internet
gateway.
Estimate a daily feed is about 500Mb these days, which isn't too far
off. You get that once a day. You might want to keep a week of news.
If I do 500Mb x 7, I end up with 3.5Gb ...
Our machine, as detailed above, is but a lowly SS-2. :-) (We're
updating it to a 10 when we get our T1 in two weeks.)
*********************************************************************************
=================================================================================
Thanks to:
Dallas N Antley <dna@clas.ufl.edu>
james mularadelis <jamesm@matrix.newpaltz.edu>
Stefan Mochnacki <stefan@centaur.astro.utoronto.ca>
John DiMarco <jdd@db.toronto.edu>
gibian@typhoon.sol.hanscom.af.mil (Marc Gibian)
spp@vx.com (Stephen Potter)
bf6y34v@is000913.bell-atl.com (Jordan)
pamela@jupiter.Legato.COM (Pamela Pledger)
adamfox@super.org (Adam Fox)
Dave Fetrow <fetrow@biostat.washington.edu>
Joe Turner <jdt@concorde.com>
This archive was generated by hypermail 2.1.2 : Fri Sep 28 2001 - 23:10:16 CDT