For those not familiar with the E10000 and its ability to be subdivided into
logical machines, I have included a brief description at the bottom.
Here is my original question:
Brian Exelbierd wrote:
> We are considering the purchase of a UE10000 for use in an OLTP,
> Datawarehouse/DSS, etc. evironment. I am wondering if anyone has any
> advice, comments, or real life experience with the Dynamic System
> Domains.
>
> One advantage we are looking at is the ability to lump unused resources
> in with other "partitions" to speed processing during peak periods.
> Because the apps we are looking at are not truly parallel we cannot just
> load a parallel db, and run them side by side.
>
> Other vendors trying to sell us clustering solutions are claiming that
> you cannot repartition the box and keep it online. They state that
> repartitioning is a many hour downtime ordeal. Sun's whitepaper says
> otherwise, and I feel certain their sales force will honor that party
> line. Some real life commentary would be greatly appreciated.
Thanks for tons of responses and information from:
Bill Hathaway <wdh@poss.com>
Matthew Stier <Matthew.Stier@tddny.fujitsu.com>
Andrew Mitchell <afm@colltech.com>
Craig Gates <cgates@fnoc.navy.mil>
Thomas Lester <tlester@spain.iakom.com>
Alan Morris SR Nashville <alanm@sunnash.East.Sun.COM>
ray trzaska <rtrzaska@uk.mdis.com>
Special Thanks to Peter L. Wargo <plw@ncgr.org> who I wound up trading over
5 letters with about various items.
I was assured that you can really dynamically reallocate system boards
without any downtime or need to stop the domains being affected. The time
for the move was reported to average between 2 and 15 minutes depending on
the I/O situation. There are 2 issues to be aware of:
1 - When removing a system board from a domain there will
occasionally/usually be a slight pause for all processing in the domain.
This apparently occurs as non-swappable memory and processes are relocated
to the remaining system board(s). This is non-fatal, brief, and only occurs
in the domain losing the system board and not in the domain gaining
resources.
2 - When planning your system you should try and set it up so that you will
not move I/O whenever possible. There are system patches for most I/O
devices that you should make sure are in place. There are 2 basic kinds of
I/O devices, suspendable and non-suspendable (this vocabulary may be
incorrect).
If a non-suspendable I/O device (i.e. a tape drive) is in use you cannot
remove the system board that is "hosting" that device, and in fact may be
unable to remove any system boards at that time. You must either wait for
the device to finish, or force it idle. This is because these devices
cannot guarantee the ability to pick back up where they paused without
timing out, or attempting to access the backplane during the move.
Suspendable I/O must simply be remapped in order to be maintained. i.e. You
cannot remove the only ethernet card from a domain and expect it to stay on
the network, however if you have 2 ethernet cards in the domain you could
use Alternate Pathing (AP) to maintain the connection. AP is part of the
E10000 software. AP is not dynamic, apparently you must manually issue the
commands to reroute the I/O. AP will work for disks as well, but Peter
Wargo has run into trouble with it. He uses Dynamic Multi-Pathing (DPM)
which is part of the Veritas software. According to him, DPM and AP do not
work well together.
As I was not involved in the actual negotiations for this box (don't you
love being the last man into a project :) I do not know the actual sale
process. I am informed that there is an interview with a sun engineer and
that extensive planning and confirmation is done before a sale is agreed
to. SUN doesn't want you to get one of these and have it not work out.
Alan Morris of SUN assured me that SUN will provide real life examples and
user contacts upon request. I was glad to get these non-vendor aided
examples from this list in addition to what we are asking our rep for.
Since no good story is without rumor, ... Rumor has it that a University in
California is using perl to dynamically repartition their E10000 as load
levels change in the various domains.
Finally to say something that should go without saying, but I have it on
good authority has been asked. You cannot remove all the resources (system
boards) from a domain and expect that it will remain active.
Later,
bex
-- Brian ExelbierdThe E10000 is a unique product (as far as my market research has shown). The system consists of what is essentially a large rack with a "gigaplane" back plane. In this rack you can place up to 16 system boards. Each system board has its own cpu(s) (up to 4), memory, and sbus cards (scsi, ethernet, etc). The boards are all independent. Once the system is installed you group the boards into logical machines. I believe that you are allowed only about 6 logical machines (called domains) currently, but support for more will be added soon. There is a very good whitepaper on this cool feature at http://www.sun.com/servers/enterprise/10000/wp/domains.pdf . There is an overview whitepaper (along with several others) at http://www.sun.com/servers/enterprise/10000/wp/
This archive was generated by hypermail 2.1.2 : Fri Sep 28 2001 - 23:12:50 CDT