A FreeBSD security hole made me decide to upgrade crimson from 4.7 to 4.8. This meant I had to figure out how to use the FreeBSD source upgrading tools embodied in cvsup, make buildworld and the rest.
The main sources of information were the FreeBSD Handbook, Chapter 24, and a post by SuperMule (on the apparently defunct BSDForums.org, now replaced by www.daemonforums.org) who did a very good job of summarising the necessary steps to get and compile the FreeBSD source tree.
This entry is therefore a summary of a summary, plus a discussion of the specific difficulties I encountered during this process.
The main steps of the process are:
cvsup -g -L 2 /usr/local/etc/cvsup/src-supfilecd /usr/src ./usr.sbin/mergemaster/mergemaster.sh -p make buildworld make buildkernel KERNCONF=CRIMSON make installkernel KERNCONF=CRIMSON [reboot to single-user (see below)] cd /usr/src make installworld mergemaster [reboot to multi-user (normality has now been resumed...)]
Each of these steps are covered in more detail below.
Copy the file /usr/share/examples/cvsup/stable-supfile to /usr/local/etc/cvsup/src-supfile. This becomes your local master cvsup file for the FreeBSD source tree. Since I wanted to track 4.8 stable (i.e. just major patches and security fixes), the src-supfile looked like this for me (comments and unchanged text omitted):
*default host=cvsup.uk.FreeBSD.org *default base=/usr/local/etc/cvsup *default prefix=/usr # The following line is for 4-stable. If you want 3-stable or 2.2-stable, # change "RELENG_4" to "RELENG_3" or "RELENG_2_2" respectively. *default release=cvs tag=RELENG_4_8 *default delete use-rel-suffix
For a helpful write-up of the significance of the tag options,
please see this summary,
provided by Phoenix, one of the moderators on the now defunct
http://www.bsdforums.org site, now risen from the ashes as www.daemonforums.org.
To run cvsup against the supfile, which will create the appropriate source tree on your local machine, issue the command:
cvsup -g -L 2 /usr/local/etc/cvsup/src-supfile
Depending on how you are connected to the Internet, this could take a few hours
This step is a preparation for the mergemaster step which follows make installworld, and is intended to offer some automated support for the changes to files in /etc which follows an operating system upgrade. You will need think about which files in /etc you have modified and wish to keep, in preparation for the mergemaster step.
mergemaster will store its working files in a directory, with the default location of /var/tmp/temproot. If this directory has contents when you run this pre-buildworld step, it starts comparing the 'crucial' files. This won't be the case the very first time you run mergemaster -p, but might be for subsequent invocations. Therefore, I suggest taking the option to delete the contents of /var/tmp/temproot at this stage, but NOT at the post-buildworld step.
In order to use the latest and greatest version of mergemaster (i.e. the one that you've just cvsup'd into /usr/src), issue the following command:
./usr.sbin/mergemaster/mergemaster -p
This step recompiles everything in FreeBSD, save the kernel. On a 2GHz machine, this took around 1 hour. On a 120MHz machine, over 8 hours.
This step builds a new kernel against the new objects created in buildworld step. It is suggested that the GENERIC kernel configuration is used for the first kernel build against a newly made world, but I was reasonably comfortable that my custom kernel configs would be OK. Hence the make command issued was:
make buildkernel KERNCONF=AMBER (on amber) make buildkernel KERNCONF=CRIMSON (on crimson)
This step installs the newly built kernel onto the machine. Ensure you use the same KERNCONF option as used with buildkernel. For crimson, the command would be:
make installkernel KERNCONF=CRIMSON
Now comes the acid test - booting the new kernel. Shutdown the machine, and on the reboot, enter the boot loader by hitting a key during the 10 second grace period before the kernel is loaded. Enter the command "boot -s" to boot in single-user mode. Once the machine gives you a shell prompt, issue the following commands:
fsck -p # preen the filesystems mount -u / # remount / in read/write mode mount -a -t ufs # mount the rest of the file systems (all ufs) swapon -a # turn on swap space
This step copies the newly built binaries, etc into their correct
places in the file system. Make sure you cd to /usr/src before
issuing the make installworld command.
The final step is to run mergemaster to refresh the /etc scripts
and configuration files (no -p command argument is required). mergemaster will compare each new file
against the existing one. If differences are found, it will show you
the changes, and ask what it should do with it. This can be a large
number of files if your upgrade delta is significant. If you have
made a note of the changes you wish to keep, this should be a fairly
simple process. Note that keeping your modified file means 'deleting the
temporary file'. In my case, I accepted everything new except:
/etc/hosts/etc/motd/etc/dhclient.conf/etc/inetd.conf/etc/mail/aliases/etc/mail/local-host-namesOnce everything is complete, you might want to save disk space by removing the contents of /usr/obj, which is where all the new binaries were placed pro-tem:
cd /usr/obj chflags -R noschg * rm -rf *
To test out the whole process, I decided to upgrade the IBM Thinkpad (amber) to 4.8, before I attempted my web server. Everything went smoothly, and the only problem I experienced was the failure of dhclient to pick up an address from my DHCP server via the wireless card. Since this only occurred after I'd allowed mergemaster to update the configuration files in /etc, I figured this was only a configuration problem. For details on this see DHCP and a wireless card (reprise).
Following my flush of success with amber, I turned my attention to crimson. The whole process took a lot longer, since crimson only has 64Mb of memory and a 120MHz Pentium 1 processor.
My problems with crimson only started after I successfully upgraded. I knew I'd have to re-install sendmail and openssh-portable, since the ports versions were ahead of those with the 4.8 world. I originally thought I could just "make deinstall" and "make reinstall", but it's not that simple. The "make reinstall" failed on libmilter.a. I figured that I'd need to re-compile everything against 4.8, so I issued a "make clean" and then a "make install". That worked fine. The same technique also worked with openssh-portable and apache. Except apache died on restarting it (signal 11)...
A quick search via google showed someone had experienced the same problem, and had identified the cause as mod_php, which I also had installed (to support phpnuke). I tested this by commenting our the php related lines in httpd.conf. Yep, apache started now. Maybe I had to re-build everything related to apache? So, I rebuilt mod_php3 and mod_php4 from scratch. Tried to re-start apache with the php elements back in. Nope, same problem. Maybe the same had to be done with mysql-client and mysql-server? Rebuilt mysql version 3.23.58. Nope, same problem.
Was it both mod_php versions that were causing a problem? I tried just removing the php3 lines. Apache started. Aha - since mod_php3 didn't seem to be required by phpnuke, I just de-installed mod_php3 and viewed the problem as solved. I suspect that I didn't really have to rebuild all these ports, but I'll probably find out next time I need to upgrade the operating system.
New security patches meant I went through this process again this weekend. However, I encountered a new problem when I tried to overwrite the default sendmail with the latest version from ports: the Makefile gave me the message:
You can't override the base sendmail this way. your version FreeBSD use mailwrapper. Please install with normal PREFIX and activate the port version with cd /usr/ports/mail/sendmail && make mailer.conf
I've not seen this message last week, using the procedure I had successfully run before. The Makefile (new as of 28th September) was now checking for the existence of /etc/mail/mailer.conf if DESTDIR was null. As far as I knew, I wasn't using mailwrapper. OK, I thought, this must be the new way of installing a later version of sendmail, using the mailer.conf file to point to the correct version of sendmail. I followed the instructions, but sendmail would not run. First problem was that the new sendmail was located in /usr/local/sbin, and rc.sendmail was still looking for /usr/sbin/sendmail. I set the sendmail_program variable in /etc/rc.conf and tried again. Sendmail started, but then died, saying it could not find /usr/libexec/mail.local. It was right, there was no such file. This was getting too difficult; I issued a "make deinstall", then moved /etc/mail/mailer.conf to /etc/mail/mailer.conf_ORIG, and issued the make command with DESTDIR set to null., and the other variables set to replace the based installed sendmail. This time, everything worked as before. I'm not sure if I should rename the mailer.conf file back, but I'll see how sendmail runs.
Later, I tried the same upgrade on amber, where it didn't really matter if sendmail worked or not. This time, I did a little reading of the handbook, and realised that I should treat the new version of sendmail in the same way as a different MTA. This time, I just set sendmail_enable="NONE" in the /etc/rc.conf file, and copied the /usr/local/etc/rc.d/sendmail.sh.sample to /usr/local/etc/sendmail.sh. This then ensures the FreeBSD version of sendmail is not invoked, and the ports version is started at boot time, as all files ending in .sh located in /usr/local/etc/rc.d are invoked.
Using this method means I wouldn't have to re-install sendmail after a buildworld operation, and I'll follow this approach next time I need to upgrade the operating system.
