Sorry for the delay in the summary, as always you folks are great.
My problem was that NIS couldn't deal with entries in /etc/group and
/etc/netgroup larger than size "x". I didn't know what "x" was.
Everyone suggested that I simply break my netgroups up into smaller netgroups
and then concatinate them together to get the netgroup I wanted, like this:
little1 (,user1,) (,user2,)
little2 (,user3,) (,user4,)
big little1 little2
Alexander Dupuy (dupuy@hudson.cs.columbia.edu), further suggests that:
>> The best fix (you say this is the netgroup map) is to rearrange the
>> netgroups into a hierarchy, which defeats the line-length limit for the
>> netgroup map. However, the netgroup.byhost and netgroup.byuser maps may
>> still exceed the allowable length (especially for the *.* key). A
>> reasonable workaround is to filter that key out with grep -v, as we do:
>>
>> netgroup.time: $(DIR)/netgroup
>> @(sed -e '/^#/d' -e 's/#.*$$//' -e 's/[ ][ ]*$$//' \
>> -e '/^$$/d' $(DIR)/netgroup $(CHKPIPE))\
>> | $(MAKEDBM) - $(YPDBDIR)/$(DOM)/netgroup
>> @($(REVNETGROUP) < $(DIR)/netgroup -u $(CHKPIPE))| grep -v '^\*.\*' | $
>> (MAKEDBM) - $(YPDBDIR)/$(DOM)/netgroup.byuser
>> @($(REVNETGROUP) < $(DIR)/netgroup -h $(CHKPIPE))| $(MAKEDBM) - $(YPDBD
>> IR)/$(DOM)/netgroup.byhost
>> @touch netgroup.time;
>> @echo "updated netgroup";
>> @if [ ! $(NOPUSH) ]; then $(YPPUSH) -d $(DOM) netgroup; fi
>> @if [ ! $(NOPUSH) ]; then $(YPPUSH) -d $(DOM) netgroup.byuser; fi
>> @if [ ! $(NOPUSH) ]; then $(YPPUSH) -d $(DOM) netgroup.byhost; fi
>> @if [ ! $(NOPUSH) ]; then echo "pushed netgroup"; fi
What I was looking for was how BIG the little entries could be before having to
break them up (and what program contained the limit). The answer was provided
by Richard Elling (elling@eng.auburn.edu) among others who said:
>> the correct answer is c. ndbm is limited to 4096 byte fields under
>> SunOS 4.1.x. It is 1024 bytes under pre-4.1 SunOS. If your netgroup
>> has more than that many characters, then split them up into collections
>> of netgroups.
I also recieved a couple of scripts (one sh the other perl) from folks to help
in breaking up the netgroups. I've enclosed them below.
Thanks once again one and all.
John M. Vogtle Internet: jmvogtle@syr.EDU
Systems Administrator AT&T: (315) 443-5772
Syracuse University US Snail: 222 Machinery Hall
Syracuse University
"One planet is all you get." Syracuse, New York 13244-1260
>> From: "Paul (Cliffy) Palmer" <palmer@pluto.lab.oasis.contel.com>
%<--------------------------------CUT-HERE--------------------------------
#!/bin/sh
#
#
# SCCS HEADER: @(#) fix-netgroup 1.8 7/16/91
# MODULE: fix-netgroup
# REVISION: 1.8
# DELTA DATE: 7/16/91
#
#
# This shell script builds /etc/netgroup out of two pieces:
# a standard section contained in the file /etc/netgroup.proto
# and a section built from the hosts table.
#
# WARNING: This script should only be run on the netgroup NIS
# master. It is intended to only be run by the Makefile
# in /var/yp.
#
# HISTORY
# 12/22/89 BJT,en first_words - coerce j to be numeric.
# 06/17/91 BPP added WARNING comment.
#
SYSTEMS_PATH=/etc/systems
NETGROUP_PATH=/etc/netgroup
NETGROUP_PROTO_PATH=/etc/netgroup.proto
DOMAINNAME=`domainname`
NETGROUP_LINE_LIMIT=15
strip_comments () {
sed -e '/#/ s/#.*//' -e '/^$/ d'
}
convert_host_to_group () {
domain=${1}
shift
for host
do
echo "${host} (${host},-,${domain})"
done
return 0
}
host_list () {
awk '{print $1}' ${1}
}
word_count () {
echo ${#}
}
first_words () {
words=${1}; shift 1
awk_prog='{
for (i = 1; i <= j+0; i++)
printf ("%s ", $i);
}'
echo ${*} | awk "${awk_prog}" j=${words} -
}
last_words () {
shift ${1}; shift
echo ${*}
}
partition_groups () {
limit=${1}; shift
index=0
list="${*}"
while [ `word_count ${list}` -gt ${limit} ]; do
index=`expr ${index} + 1`
echo "domain${index} "`first_words ${limit} ${list}`
list=`last_words ${limit} ${list} domain${index}`
done
echo "domain ${list}"
}
build_netgroup_from_hosts () {
line_limit=${1}; shift
domain=${1}; shift
systems_path=${1}; shift
if [ -f ${NETGROUP_PROTO_PATH} ]; then
cat ${NETGROUP_PROTO_PATH}
fi
list=`cat ${systems_path} | strip_comments | host_list`
convert_host_to_group ${domain} ${list}
partition_groups ${line_limit} ${list}
cat <<END
general domain
END
}
build_netgroup_from_hosts \
${NETGROUP_LINE_LIMIT} \
${DOMAINNAME} \
${SYSTEMS_PATH} \
>${NETGROUP_PATH}
cutcutcutcutcutcutcutcutcutcutcutcutcutcutcutcutcutcutcutcutcutcutcutcutcut
>> From: nicky@davinci.concordia.ca
Here it is. It came with another program but I seem to
have removed it. You might want to post to comp.lang.perl
or have a look at the archives in tut.cis.ohio-state.edu
for more. It's probably there with the other files.
Great Stuff...
Nicky
-------------------- Cut Here ------------------
#!/usr/local/bin/perl
# $Header: mknetgrp,v 1.6 89/08/04 12:00:49 aks Exp $
# mknetgrp [-v] [-i input_file] [-o output_file] [-l record-length]
#
# This script will parse the netgroup input file, writing a source
# suitable for parsing by "makedbm" (part of YP).
#
# The whole reason for this script is due to a gross bug in
# dbm(3) (even ndbm(3) has the same limitation), in that no single
# record can be longer than 1024 (or 4096 for ndbm) chars.
# Netgroups of students can get QUITE large, and so must be broken
# up into smaller groups.
#
# To make it easy, the human-source doesn't have to worry about
# these restrictions, but the machine-source cannot have a
# netgroup record exceeding the limitations of dbm. This program
# will parse each netgroup record, and, if it finds a large
# record, breaks into smaller chucks with sequentially-named
# subgroups.
#
# The algorithm makes two passes over the file: the first just
# reads in all existing netgroup names; the second reads each
# record and if it's not too large, writes it out immediately. If
# it is too large, then as much of the group members as will fit
# onto the first record is written, including the new subgroup
# name which is created from the symbolically-incremented parent
# group name (an easy trick courtesy of Perl). This is the reason
# for pass 1: to ensure that any new subgroups names are unique.
# The next subgroup is written, and then the process repeated
# until the record is entirely written.
#
# The input format is more relaxed than the actual "makedbm" input
# format: if the subsequent line begins with a space or tab, it is
# assumed to be a continuation of the previous group, intervening
# comment lines not withstanding (in other words, you can pretty
# much place a comment line anywhere, without worrying about
# interrupting a continuation).
#
# The above "freedom" implies that a netgroup record must begin
# with the netgroup name in column one. The items which follow
# must be the subgroup names or triples.
#
# Some configuration parameters
#
$Debug = 0;
$MaxRecLen = 512; # max "ndbm" record size
$MAXLINELEN = 80; # for pretty output to terminal
$[ = 0; # origin 0 indexing
@NetGroupNames = ();
$NetGrpIn = "/etc/netgroup.master";
$NetGrpOut = "/etc/netgroup";
# Keep track of # of groups in /out.
$InCount = 0;
$OutCount = 0;
$Verbose = 0;
$MaxNameLen = 0;
# Process any arguments
while ($_ = shift) {
if (!/^-[viodl]$/) {
print "usage: mknetgrp [-v] [-d] [-i in] [-o out] [-l maxrec]\n";
exit 1;
}
if (/^-i$/) { $NetGrpIn = shift; next; }
if (/^-o$/) { $NetGrpOut = shift; next; }
if (/^-l$/) { $MaxRecLen = int(shift); next; }
if (/^-v$/) { $Verbose++; next; }
if (/^-d$/) { $Debug++; next; }
}
if ($Debug) {
$NetGrpIn = "netgroup.master";
$NetGrpOut = "netgroup";
}
open(NETGRPIN,$NetGrpIn) || die "Can't open $NetGrpIn\n";
if ($Verbose) {
print "Input netgroup file is $NetGrpIn\n";
print "Output netgroup file is $NetGrpOut\n";
printf "Maximum record length is %d bytes\n",$MaxRecLen;
}
# Pass 1: scan the file for just netgroup names
while (<NETGRPIN>) {
chop; s/\\$//;
next if /^\s/ || /^$/;
@line = split(' ');
$NetGroupNames{$line[0]} = 1;
if ($Verbose) {
local($len) = length($line[0]);
$MaxNameLen = $len if $len > $MaxNameLen;
}
}
# Open the output file
seek(NETGRPIN,0,0) || "Could not rewind $NetGrpIn!\n";
open(NETGRPOUT,">$NetGrpOut.tmp")
|| die "Can't open $NetGrpOut for output!\n";
print NETGRPOUT
"# This netgroup file was produced automatically by \"mknetgrp\"\n";
@lt = localtime(time);
printf NETGRPOUT "# on %02d/%02d/%02d from %s\n",$lt[4]+1,$lt[3],$lt[5],$NetGrpIn;
print NETGRPOUT "# DO NOT MODIFY THIS FILE!\n";
@Group = ();
while (<NETGRPIN>) {
chop;
next if /^$/ || /^#/ || /^\s*#/; # don't write blanks or comments
$cont = /^[\t ]/; # set flag if continuation
s/\s(#.*|\\)$//; # strip any comments & continuations
s/\s*,\s*/,/g; # remove blanks inside of tuples
s/\(\s+/(/g;
s/\s+\)/)/g;
s/\\$//;
@list = split(' ');
if (!$cont) { # if new list
$OutCount += do PrintGroup() if $#Group >= $[;
$InCount++;
@Group = @list;
if ($Verbose) {
local($name) = $Group[0];
local($len) = length($name);
printf "%s:%s",$name,(" " x ($MaxNameLen+2-$len));
}
} else { # a continuation
push(@Group,@list); # add to the group list
}
}
$OutCount += do PrintGroup() if $#Group >= $[;
close NETGRPIN;
close NETGRPOUT;
if (!$Debug && $> == 0) { # if root and not debug
unlink "$NetGrpOut.old" if -e "$NetGrpOut.old";
rename($NetGrpOut,"$NetGrpOut.old") if -e $NetGrpOut;
rename("$NetGrpOut.tmp",$NetGrpOut);
}
printf "%d groups in; %d groups out.\n", $InCount, $OutCount if $Verbose;
exit 0;
# This subroutine prints the current accumulated group list
# in "grp".
# Return the # of groups printed.
sub PrintGroup {
local($grplen);
local($grpname) = shift(@Group);
local($firstgrp) = $grpname;
local($line);
local($member);
local($grpcount) = 1;
local($totalgrplen) = 0;
local($incname) = '01';
$line = "$grpname\t";
$grplen = length($line);
foreach $member (@Group) {
if ((length($line) + length($member) + 1) >= ($MAXLINELEN - 8)) {
print NETGRPOUT $line . "\\\n";
$grplen += length($line) + 2;
$line = "\t";
}
if ($grplen >= ($MaxRecLen - (length($grpname) + 4))) {
local($newgrpname) = $grpname;
$newgrpname .= $incname if $grpname !~ /\d$/;
$newgrpname = $firstgrp . ++$incname
while ($NetGroupNames{$newgrpname});
$grpname = $newgrpname;
$NetGroupNames{$grpname} = 1;
print NETGRPOUT $line . $grpname . "\n";
$line = "$grpname\t";
$totalgrplen += $grplen + length($grpname) + 1;
$grplen = length($line);
$grpcount++;
}
$line .= $member . ' ';
}
print NETGRPOUT ($line . "\n");
if ($Verbose) {
$totalgrplen += $grplen + length($line) + 1;
printf "%3d members", ($#Group + 1 - $[);
printf ", %2d subgroups", ($grpcount-1) if $grpcount > 1;
printf ", %4d bytes.\n", $totalgrplen;
}
return $grpcount;
}
cutcutcutcutcutcutcutcutcutcutcutcutcutcutcutcutcutcutcutcutcutcutcutcut
This archive was generated by hypermail 2.1.2 : Fri Sep 28 2001 - 23:06:19 CDT