SUMMARY: Server Management

Apologies for the late Summary.  The vendor selection process went from
September of '03 to mid-November of '03.  I wanted to close the book on the
whole issue prior to sending the summary..

A hearty thanks goes out to all who responded (Adam Ronthal, Hans Jacobsen,
Larry McCann, Rich Teer, Jose Vicente Nunez Zuleta, Vince Merrell, Pam
Eavenson, Neil Quiogue, and Rich Kulawiec).

All answers were very useful, but pretty close to what I expected.  I
figured that most shops would roll their own tools, especially where there
was no budget, as I have done in the past.  There are a wide variety of
solutions that cost nothing but time to put together.  On the other hand,
the server management/provisioning software is at the shallow end of the
shrink-wrap market, so, few environments are using off-the-shelf solutions.


Here is a list of the information that was shared with me.
Commercial products:  CiRBA (piggy-backs on Sun Management Center)
Roll Your Own Tools:  Perl, MySQL, CVS/RCS, rsync, ssh, tftp, & expect
Open Source Tools for monitoring:  Big Brother (http://www.bb4.com), Ganglia
(http://ganglia.sourceforge.net), Orca (http://www.orcaware.com), Net-SNMP,
OpenNMS (http://www.opennms.org), AIDE
(http://sourceforge.net/projects/aide)
Open Source Tools for Server Consistency:  cfengine
(http://www.cfengine.org) and cfengine (http://www.cfengine.org as much
about philosophy as tools)

As far as my situation goes, I had the following requirements:
F-1.  For each server, record configuration in a searchable form to a
central database at some interval (including OS, applications, packages,
disks, and low-level things like OBP and RTOS versions).
F-2.  Push out/Roll back packages, patches, configuration changes, and
scripts remotely from the administrative interface.
F-3.  Administrative interface authentication needs to integrate into
existing LDAP and RSA authentication facilities
F-4.  Administrative interface needs to have robust logging for
security/auditing purposes.
F-5.  Solution needs to be flexible enough to monitor changes to TWC custom
configuration files.
F-6.  Solution must support these environments (non-exhaustive list:
Windows NT 4.0, Windows 2000, Windows 2003, Sun Solaris 2.6, 7, 8, & 9, as
well as Red Hat Linux AS 2.1)
F-7.  Product must allow flexibility in grouping of servers. (i.e. - group
by hardware type/platform, application, functional role, environment
(DEV/QA/PILOT/CERT/PROD/etc), location, OS, etc.)
F-8.  Product must allow servers to belong to multiple groups, so if we need
to patch/deploy/config/etc. a certain subset of servers, we can pinpoint
that group only or a much larger group.

I had a descent budget to work with and a project load for my team that made
building this ourselves out of the question.  The building and testing alone
would have taken months and months and would have only worked on the *nix
variants in my shop, not Windoze.

Once I had buy-in from the Microsoft side of the house, we narrowed the
field to Opsware and Bladelogic (only Tivoli and Sun Management Center were
considered beyond Opsware and Bladelogic).

Both fit our requirements, but Opsware was slow and clunky.  The interface
was difficult to navigate.  Bladelogic, on the other hand, was clean,
intuitive,  and simple to use.  For once in my career, the Windows team and
the Unix team agreed on something more than where to have lunch -- Blade
logic was the best choice.  We then had them in for an onsite live
demonstration a selection of our servers.  Bladelogic lived up to its press
and we bought it for all of our Solaris, Linux, and Windoze servers.

We were not able to get the licensing through our legal department prior to
end of year freeze, so we are kicking off the implementation next week
(instead of the late-November target).  Implementation is slated to take 4-6
weeks; although, since I have installed and configured the software twice
during onsite visits, I will be completely productive within 2-3 days on 150
Solaris hosts and 10 Linux hosts.  It pays to be hands on during vendor
POCs, eh?  Also, we are rolling out the application, database, and reporting
servers on Red Hat AS 2.1.

That's all folks.

Thanks again, everyone.

Ben Green
Charlotte, NC

>  -----Original Message-----
> From: 	Ben Green [mailto:awk@carolina.rr.com]
> Sent:	Monday, September 08, 2003 4:53 PM
> To:	'sunmanagers@sunmanagers.org'
> Subject:	Server Management
>
> Greetings managers,
>
> I work in a data center that has about 150 Sun Servers consisting of at
> least one of every server that Sun currently manufactures (except some of
> the new small servers and the blade technologies).  I inherited this data
> center in December and completed its move from Denver, CO to Charlotte, NC
> in late May.
>
> I am now working with my team to manage this collection of servers that
> will grow to more than 200 over the next 6 months.  These inherited
> servers are mostly Solaris 8, but are at many different patch levels with
> a multitude of configuration differences.
>
> What we want to do is this:
>  - manage all of the servers at a single console app or client and open
> this up ina  role-base management for Tier II support
>  - group servers by application, environment (DEV/QA/PROD/etc.), Operating
> System (Solaris 7/8/9), or by type (www/oracle/sybase/weblogic/etc.)
>  - initiate an application/patch installation, script run,  or
> configuration change on a whole group at once
>  - track changes to files (sort of a host-based IDS in a way)
>  - complete application and hardware configuration (down to RTOS and OBP)
> logged to a central database for reporting and DR purposes
>
> So far, we've found BladeLogic and Opsware.   Both do these things.  We've
> also been looking at our own home-grown solutions utilizing open source
> technologies.  I would like to entertain more vendors than just two and
> the home-grown solutions could cause more work in mainenance and fixes to
> scripts and processes.
>
> Now, for my question...  What tools/packages/applications have you used to
> solve the above problems in the data centers in which you have worked and
> about how many servers did you manage?
>
> Thanks for any information.  I will summarize.
>
> Ben Green
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers