|Upgrading to Debian Lenny||The hydrus standard Debian install|
I finally had some time to upgrade my FreeBSD servers to 7.1. This followed the upgrade process I originally outlined after my first attempt. However, I've learnt a few things since then...
First, there's no need for the
cvsup port, as a similar
capability is part of the base system, called
operates identically (for all practical purposes). So the upgrade
steps now are:
csup -g -L 2 /usr/local/etc/cvsup/src-supfile
cd /usr/src ./usr.sbin/mergemaster/mergemaster.sh -p make buildworld make buildkernel KERNCONF=CRIMSON make installkernel KERNCONF=CRIMSON [reboot to single-user ] fsck -p mount -u / mount -a -t ufs swapon -a cd /usr/src make installworld mergemaster -U # replaces unaltered files automatically # get rid of old, unused, files make -DBATCH_DELETE_OLD_FILES delete-old make -DBATCH_DELETE_OLD_FILES delete-old-libs # Enable networking so that package downloads will work /etc/rc.d/hostname start /etc/rc.d/netif start /etc/rc.d/routing start # get ports tree up-to-date csup -g -L 2 /usr/local/etc/cvsup/ports-supfile # replace all packages sh /home/mark/bin/pkg_upgrade.sh [Reboot into multi-user (normality has now been resumed...)]
Some warnings about the above process:
make delete-oldwith the flag
-DBATCH_DELETE_OLD_FILESunless you are sure you are removing all the old releases packages.
/home/mark/bin/pkg_upgrade.shis described in the journal entry for upgrading to FreeBSD 6.3. While it does its best, there are somethings it cannot cope with. In the 7.1 upgrade, a number of ports that were available as packages in 6.3 were not available in 7.1, for example
py-libxml2. I had to add these manually seeing the download errors in the log file.
So far, FreeBSD 7.1 has been working just fine. However, I have
noticed that the memory usage, as shown by
top, is much
more than it used to be on FreeBSD 6.3. This is very noticable on
crimson, as it only has 64MB. Here's a typical top display:
last pid: 71173; load averages: 0.05, 0.03, 0.01 up 17+19:50:57 17:01:35 47 processes: 1 running, 46 sleeping CPU: 0.8% user, 0.0% nice, 0.8% system, 0.8% interrupt, 97.7% idle Mem: 21M Active, 11M Inact, 21M Wired, 2060K Cache, 14M Buf, 3012K Free Swap: 128M Total, 19M Used, 109M Free, 14% Inuse PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 71173 mark 1 96 0 3488K 1428K RUN 0:00 1.24% top 17349 root 1 96 0 4664K 1172K select 5:45 0.00% ntpd 17418 root 1 96 0 6836K 1780K select 4:19 0.00% sendmail 17395 root 1 96 0 6852K 2172K select 3:53 0.00% httpd 17166 root 1 96 0 3176K 740K select 0:43 0.00% syslogd 17428 root 1 8 0 3204K 740K nanslp 0:37 0.00% cron 17722 bind 4 96 0 26464K 8056K select 0:29 0.00% named 17241 root 1 96 0 3204K 832K select 0:08 0.00% rpcbind 17295 root 1 4 0 3120K 500K - 0:05 0.00% nfsd 17422 smmsp 1 20 0 5948K 1396K pause 0:05 0.00% sendmail 17382 dhcpd 1 96 0 3120K 1268K select 0:05 0.00% dhcpd 71024 mark 1 96 0 8352K 3012K select 0:02 0.00% sshd 71026 mark 1 20 0 4448K 2052K pause 0:01 0.00% tcsh 17294 root 1 4 0 3120K 592K accept 0:01 0.00% nfsd 17413 root 1 96 0 5700K 1268K select 0:01 0.00% sshd 71021 root 1 4 0 8352K 3028K sbwait 0:01 0.00% sshd 66257 www 1 4 0 6852K 2488K accept 0:01 0.00% httpd
You can see that 19M of swap is in use. With 6.3, the amount of swap was normally around 2-3M. However, this does not seem to cause any performance problems, and may be just down to how the new malloc works in FreeBSD 7.1, or possibly because the new version of bind needs 26M!
As part of the upgrade to FreeBSD 7.1, I re-worked the kernel
configuration files for crimson and chrome to use the
config "include file" facility, along with nooptions,
nodevice etc. This allows a configuration file to be based on
GENERIC, but removes elements which are not required. I was in two
minds about doing this, as the level of removal from the GENERIC
config file for both crimson and chrome is large, but it's done now.
One tool I would have liked, but could not find, was something that would output the final configuration file, with elements added and removed. I couldn't find anything that did this, so I wrote one in python. Using this tool, the CRIMSON effective configuration file looks like:
ident CRIMSON # machine i386 # # cpu I486_CPU cpu I586_CPU # option UFS_ACL option KBD_INSTALL_CDEV option KSE option _KPOSIX_PRIORITY_SCHEDULING option STOP_NMI option PREEMPTION option MD_ROOT option MSDOSFS option NFSLOCKD option SOFTUPDATES option ATA_STATIC_ID option PSEUDOFS option SYSVSEM option SCSI_DELAY=5000 option STACK option INET option SCHED_ULE option COMPAT_FREEBSD6 option NFS_ROOT option UFS_GJOURNAL option COMPAT_FREEBSD5 option COMPAT_FREEBSD4 option ISAPNP option GEOM_BSD option SYSVMSG option SYSVSHM option PROCFS option CD9660 option GEOM_PART_GPT option COMPAT_43TTY option FFS option NFSCLIENT option UFS_DIRHASH option SCTP option GEOM_MBR option GEOM_LABEL option ADAPTIVE_GIANT option NFSSERVER # makeoption DEBUG=-g # device ppc device eisa device random device pty device atkbd device pci device psm device bpf device md device atadisk device uart_ns8250 device rl device mem device ata device uart device ppbus device loop device isa device kbdmux device ppi device ether device atkbdc device splash device atapifd device fdc device miibus device io device atapicd device sis device plip device vga device sio device npx device lpt device sc
Here's the python code:
""" NAME config.py - Displays effective version of FreeBSD kernel config SYNOPSIS python config.py [-n] config_file -n - do not attempt to process the DEFAULT configuration file DESCRIPTION config.py will process a FreeBSD kernel configuration file and output the effective configuration generated, taking account of values provided in any include files, and the impact of the various 'no..' keywords which are used to remove previously defined values. NOTES MODIFICATION HISTORY Mnemonic Rel Date Who config 1.0 20090216 mpw Created. """ import sys import getopt import os import shlex class Lex(shlex.shlex): def sourcehook(self,filename): f = open(filename) return (filename,f) class Config: def __init__(self): self.lex = None self.elements = ((("ident",),dict(),1), (("machine",),dict(),1), (("maxusers",),dict(),1), (("cpu","nocpu"),dict(),0), (("option","options","nooption","nooptions"),dict(),0), (("makeoption","makeoptions","nomakeoption", "nomakeoptions"),dict(),0), (("device","nodevice","nodevices"),dict(),0)) return def process_element(self,tok,target): for e in self.elements: if tok in e: if e == 1: e.clear() # only one element value allowed if tok[0:2] == "no": if target in e: del e[target] else: e[target] = 0 return def build(self,filename,process_default): try: f = open(filename) except IOError: return 1 self.lex = Lex(f) # extend word token characters for makeoptions self.lex.wordchars = self.lex.wordchars+"-=" self.lex.source = "include" # keyword for include file self.lex.dir = os.path.dirname(filename) self.lex.commenters = "#" # comment character if process_default: try: fd = open("DEFAULTS") except IOError: return 2 self.lex.push_source(fd) tok = "start" while len(tok) != 0: tok = self.lex.get_token() target = self.lex.get_token() self.process_element(tok,target) return 0 def display(self): for e in self.elements: for k in e.keys(): print e,k print "#" return if __name__ == '__main__': process_default = True try: opts,args = getopt.getopt(sys.argv[1:],'-n') for o,v in opts: if o == '-n': process_default = False except getopt.GetoptError,e: print "config.py: illegal argument:",e.opt sys.exit(1) if len(args) == 0: print >>sys.stderr,"config.py: usage: python [-n] config.py "\ "<CONFIG_FILE>" sys.exit(1) config = Config() result = config.build(args,process_default) if result == 0: config.display() elif result == 1: print >>sys.stderr,"config.py: Unable to open",args elif result == 2: print >>sys.stderr,"config.py: Unable to open DEFAULT config file"\ " (consider -n)"
|Upgrading to Debian Lenny||The hydrus standard Debian install|