17 May 2009- I have made a few changes to my FreeBSD computers. Topaz is an Intel Server with dual 2.4 GHz Xeon's. Ruby is an Intel system with core duo with 3.16 GHz cpus. The proposed replacement for Crystal, an 866 MHz machine, is a P4 3.4 GHz system but it is running hot. In speed, I am going from 5.5 GHz to 14.4 GHz. A buildworld on the old Ruby required close to 1.5 hours. On the new ruby, a buildworld takes 15 minutes.
For the non-informed, you may be wondering what a FreeBSD is. If you are curious, you need to visit FreeBSD.org. In the almost 40 years of being around computers, I have seen a number of changes. I found that I couldn't train most people to diagnose computer problems. They lacked some sort of pattern recognition and couldn't make the connection that I could see. A friend cracked that trying to teach them how to do diagnostics was similar to having a Catholic Priest (who are supposed to be celibate) teach sex education. Only being raised around Catholics, I didn't know if it was true or not; however, the comparison always seemed appropriate. One of the most impressive diagnostic programmers I ever met worked for Cray. It was his job to fix nano-second timing glitches in Cray Y/MP's with 16-cpus. I thought my diagnostic skills paled in comparison to his. The problems I was trying to track down may take hours of computer time before they would occur. The skills required were different. His 3D view of his world was different than mine.
My FreeBSD environment consists of different cpus. They range from a P-II 400 to an AMD 2400+ XP. I don't have any Intel P-4's. A friend benchmarked the kind of stuff that we do and because of the extra data paths that the AMD cpu's have, they will produce 2x more results than the same clock speed P4. The majority of them have at least 256 MB of memory and 40 GB or more HD. The really typical system has 200-300 GB of HDs and 1 GB of memory. They all have the build environment spread across multiple HDs and controllers. My fast builds have the system on one HD/controller and /usr/src and /usr/obj on their own HD/controllers. Because I have a mix of cpus, I don't build with cpu options. I have NFS mounted /usr/src and /usr/obj from a different system to recover one that developed serious build problems.
The typical file systems are as follows: / 500 MB, /swap 500-2048MB, /tmp and /var are each 1.5 GB, /usr/src and /usr/obj are each 1.5 GB. The ../src and ../obj are on different HDs and controllers.
I had the opportunity to use the Xerox-82x that started the gui revolution. It didn't go anywhere from a marketing standpoint. For that to happen required a couple of enterprising individuals to focus that look and feel onto a computer they called an Apple. Later, Microsoft implemented a similar version and called it Microsoft Windows. I never had the opportunity to use an Apple. The fact that they claimed to invent the look and feel always bugged me. Windows, on the other hand, was really primitive until Windows 98 and NT-4.0 came along. What I find to be really important on computers is finding a user interface that you are really comfortable with. You can't pay attention to the marketeer's because they always think their version is the best. If you look around, you will find one that stands out above all of the others. That is the system you need to be using. I am down to two. They are Windows XP and FreeBSD. No matter what I do, there are some things that I want to do that can only be done on Windows. The rest can be done with the system that won my heart and soul and that is FreeBSD. It didn't start out that way.
In January 1999, I used parts removed from various PC's that I upgraded to create a system with a P-133 cpu and running one of the open source versions of Unix. I started out using Red Hat Linux 5.1. I had problems with Linux in general. I was used to this controlled environment, which Linux didn't come close to satisfying. Figuring this out required almost a week. I depend on software that is developed on Linux. Discovering this required much longer. At any rate, I had been using Linux for about a week when a friend brought over his 4-cd set of FreeBSD version 2.2.8. Just to be cautious and not burn any bridges behind me, I didn't replace Linux but removed the HD and replaced it with another one of my "too drives" (too old, too slow, and too small), which had been a 3.1GB spare. I used this clean drive to install FreeBSD on. My friend that brought over the 4-cd set talked me through my first install. Since, I was using "too-equipment", the install went rather slowly but it worked on the first boot. I even had access to the Internet through Windows 2000 Server beta1 using RRAS/natd. They would change something every time they did a new build and I would have to re-install and configure RRAS/natd all over again. That was when the real switch occurred because I found that after the third time I was really irritated (totally PO'ed is closer) and spent the time configuring user-ppp and dialed my ISP via the FreeBSD system. My Windows 2000 Server has never done a dial-out since then. Some time ago the modem on the W2K Server died and a live one has never been powered up.
This was an interesting situation because when I started using FreeBSD, it had been a year since I had really used a Unix system. When I was laid off at Siemens Power Corp. and retired, I ended up working for a company that mostly used PC's running DOS, which were connected to embedded systems, or Windows 95. They had an old Sun Sparc but other than trying the root password I never used it for anything. It was kept around because the Monk Monte Carlo program would only run on it.
The embedded system was interesting for a time because of the strange problems we were encountering. It turned out that a Pentium was fast enough to see a design flaw in the main PC interface board . The system would measure the percentage of steam in high flow rate, high temperature, high pressure water on its way from a heating device (power reactor or furnace) to a steam generator. The interface board toggled the data ready line when it started saving a byte into the on board memory. The P-90 could catch the data when the memory was in transition. This occurred because each bit of data was stored on a different memory chip. The memory was dual ported and could be read while it was being written to. The speed variation between the memory chips was enough that the PC would occasionally receive bogus data from the interface board. The bogus data would be caught by the checksum but we would lost the next measurement. All we had to do then is add a slight delay after we received the data good byte before we tried retrieving the data from the interface board. The small delay was enough to ensure that all of the data bits were set in memory. The money ran out on this project and I was retired again.
Historically, I am not a big fan of Windows 9x. I think NT or what Microsoft now calls Windows 2000 is a much better system. I also think that if you don't have access to the various operating systems, you don't appreciate the pluses and minuses on each system. In my mind, there is no OS that deserves a 10 in all categories. A Staff Scientist at Battelle Northwest told me one time, that if the developer's have access to various machines, in a years time, the application will be running on the OS that performs the best for that software. The software may end up on a Mac, a PC running Windows, an HP-UX workstation, a Sun workstation, a Cray, or a massively parallel system. If you only have a single choice, you are clueless when it comes to choice of OS and software performance. There are things that I want to play with that only work on Unix and FreeBSD is my choice.
As you can imagine, the original configuration didn't last too long. It was the slowest computer I had running. I decided that it was time to try a Celeron and see what they were worth in my environment. Some thing's are really fast but Seti_at_Home was running 1.5 times slower than my NT systems, which are mostly P-II 400's. The Celeron was a 433 MHz. I expected more but wasn't surprised because of the speed of the memory. Eventually I upgraded from PC66 memory to PC100 memory. I had been told that I would see a significant improvement. I started finishing a Seti work unit (wu) 8,300 seconds faster, which was an improvement of around 15%. After a week or two, I replaced the Celeron 433 with a Celeron 300a running at 450Mhz. This made a significant difference. A Seti work unit processing time was 13,000 seconds faster with the 300a@450 than with the 433a and PC-100 memory. The change was 54,000 seconds/wu to 33,000 seconds/wu. For an OS, I am currently running 4-Stable, which is currently at version 4.3. I cvsup every time I see a major fix to one of the processes I use. It is an easy way of maintaining a system in a "current" state. I have a relatively clean system and it can be used in case someone develops build problems on one of the support lists.
I have a Monte Carlo program (Monk) that I used to use that last ran on a Cray. It was mostly written in Fortran 66. It is a patchwork quilt program with years of upgrading built into it. The goal was to make it run on FreeBSD. When I first started converting it, it had layers of problems. The fatal one was an array out of bounds. The program was calculating a bad index and trying to find it was difficult. I am really frustrated right now because of the state of debugging tools. I am trying to use the GNU gdb debugger with ddd as the frontend. It is not nearly as good a debugger as I have in Microsoft's Visual Studio. Don't get me wrong, MSVD doesn't win all of the points in a comparison. When it encountered the array out of bounds, it just rolled over and died. Code-Forge and ddd left me on the line with the error but I couldn't see what the problem was. Once I found the spot where the error was occurring, I could place a breakpoint on the line of code and see the problem on NT. It was obvious that something was being calculated that was out of range. I just didn't know why. One of the F66 options took care of the indexing problem on NT. I find the error on one system and was able to fix it first on the other system. The heterogeneous environment once more produced the wining combination.
This crossover win occurred because I couldn't do a quickwatch to look at the array of indexes with ddd even though it pointed to where the computer program was dying. In 36 years of programming, finding the line of code causing the problem is 80% of the solution. I was able to make the NT command line version run first because I was using Compaq's DEC Visual Fortran 5.0, which depends on Microsoft's Visual Debugger. It was all a matter of just a few enhancements. They made the difference. With a system running, however, I was able to add some diagnostic writes while trying some options. I was surprised on the first try because I ended up with a 70MB output file. I eventually found a setting that made the FreeBSD version run. The sample problems only run for a maximum of a minute, so we aren't going to see differences in processor and OS operating speeds. I expect the NT version to run faster because it uses a lot of accesses to memory, which the P-II, with the larger L2 cache, has proved superior to the Celeron with Seti_at_home. There isn't too much need for this program but it was nice to make it work on all of my computers. The conversion involved much more tha n compiler options. There were subroutines with entry points that depended on argument persistence and that had to be programmed around. The argument problem ended up requiring about two of the three man months involved in the conversion. It also contributed to most of the year of elapsed time involved with the conversion.
The latest problem was the overclocked Celeron 300a at 450. The cooling was inadequate when running setiathome full time and building XFree86 3.3.6 from source. The system would hang and after this happened twice I pulled the heatsink off of the Celeron. It had cooked the thermal pad. For a view of the heatsink, see celeron_300a_heatsink. The damaged areas are clearly visable. I used heatsink goop and tried that but it didn't help. I found a much larger PPGA cooling fan and heatsink but it was too late. A week later, the system became really flaky and the next week it would hardly boot. It died while I was substituting parts. I tried memory, which didn't help, and then I reinstalled the Slot1 Celeron 433a and everything was back to normal but much slower. While this was going on, I decided up to upgrade to FreeBSD 4.0. I had developed some diffs that would make Code Crusader run on 4.0. They had the side effect that it would also build as a port on FreeBSD 3.4. It was a peer group solution. I saw a PR and understood what the problem was. If you add that change to my original 4.0 changes and then have someone take it all and make a working set, you have an upgrade to Code Crusader that worked.
The environment was a massive change. The file systems are all shared via NFS mounts once you get past the firewall and gateway computer. I'm getting to the point that I really like the combination of KDE and FreeBSD. There is so much going on and anyone can contribute to the environment. Does that mean I am going to convert all of my systems to FreeBSD? No, but they may all boot it as one of the choices. I have too much invested in time and money with full fledged software to give up features for something less even if it is free. This was written around 1999 and in November 2002, my systems consist of 1 W2K Server, 2 standalone FreeBSD machines, and 4 that multiboot. Ruby, on of the standalone FreeBSD machines, completed a "portupgrade -afp" in around 6 hours. The environment is XFree86-4, and kde-3. A long session was checking file ownership in a portupgrade of cvsup-mirror. Not adding a -x cvsup-mirror to the portupgrade was a major mistake.
My latest is combining songs from 30 minute CD's into monster 74 minute combinations with songs I really like. The software to burn the audio material on to CD-R disks runs on Windows. Later, after FreeBSD 4.0 was released, burncd would burn CD's. The visual environment of Roxio's EZ CD Creator or Ahead's Nero5 is a hands down winner. This is a much cleaner solution than the old audio tapes I use to create to listen to in the Fiero or the old Ford 4x4 pickup. The Bonneville only plays CD's so we start out much better :). Unfortunately, I find they are scratched playing them. So, I don't play an original. I CD copy them and then play the copy. You can do this really easy with all of the Window's CDROM burners. The CD copy work is always done on a Windows system. If the Bonneville would only play MP3-CD's, I would have the best of everything just shy of a portable DVD player. The MP3-CD's that I currently create typically play for 5 hours. You can let the CD player play them sequentially or random. I typically choose random. I usually create CD's with at least 2 artists and using a random play sequence is almost like listening to an FM-radio station without any advertising.
It is 12 October 2002 and I have had to replace "ruby". I could start a task and ruby would hang. I had spares of everything but the motherboard. I decided last night that I would take care of that problem. One of the local computer stores had a bare bones system. It was a case, Gigabyte motherboard, and an AMD 2000+ XP cpu. Everything was already installed in the case. I had everything pluged in around 3 am but waited until I woke up so that I was not tired when I double checked everything. It ran on the first boot. This is one of the reasons I like the basic Unix systems. You can basically change motherboards and it will boot on the first time you turn it on. I can move my system builds on to ruby like I have wanted to do but didn't because ruby was too slow for what I wanted to use it for.
Back to Top
This page exists because I have had several requests for my upworld script. There are updated comments for newer systems such as FreeBSD-7.1 further down the page. For example, my upworld was split into 5 separate shell scripts and I only use them to update my systems. For historical reasons, I haven't redone this front section.
I cut and pasted the gist of what I was sending in my email replies and produced this web page. If you are having problems doing a buildworld, it won't help you. What it will do is help you document where your build is failing. The first question anyone has when they try to help you diagnose what went wrong is "what did you change". This script logs everything that was changed. Being somewhat lazy when it comes to retyping something, I thought I would create this page and add all of the caveat's that I knew of and then I wouldn't have to provide much more than a link to this page.
My script is nothing more than the following
#! /bin/sh cd /root/cvsup cvsup -g -L 2 stable-supfile 2>&1 | tee /var/log/build/cvsup.log cd /var/log/build # Now convert the log to html cvsuplog < cvsup.log > cvsup-`date "+%Y%m%d-%H%M"`.html # We have updated the source and now it is time to build the world. cd /usr/src make buildworld 2>&1 | tee /var/log/build/bworld-`date "+%Y%m%d-%H%M"`.log make buildkernel KERNCONF=RUBY 2>&1 | tee /var/log/build/bkernel-`date "+%Y%m%d-%H%M"`.log make installkernel KERNCONF=RUBY 2>&1 | tee /var/log/build/ikernel-`date "+%Y%m%d-%H%M"`.log make installworld 2>&1 | tee /var/log/build/iworld-`date "+%Y%m%d-%H%M"`.logYou can open this script in its own browser window by clicking Upworld. Before it can be something to be proud of, it needs a lot more than what I have done. You have to edit my script to replace RUBY, which is the name of my server's kernel, with the name you have chosen for your kernel.. You also have to "chmod 744 upworld" before you can run it. It provides a complete, time stamped log of what was changed between the last update and the current one.
Upworld assumes that you have a copy of Ben Smithurst's cvsuplog on your system. If you click the link to cvsuplog, it will take your browser to Ben's site and you can download cvsuplog. There are a number of scripts out in FreeBSD land that will convert a cvsup.log to HTML. The first one I came across was cvsuplog and it did everything I wanted. If there is anything clever in this script, it is cvsuplog that contributes the cleverness. I depend on that somewhat magical transformation from a log file into html that points to the cvsweb.cgi process at freebsd.org.
In my root home directory I have two subdirectories and they are /root/bin and /root/cvsup. In my /root/cvsup directory, I have copied the sample "cvsup" scripts. I use "stable-supfile". I also changed the structure of /var/log to include /var/log/build and then added that path to each of the capture files. This keeps the logs out of /usr/src and places them in a place designed for logs. You have to have a large /var filesystem or your can overwhelm it. I like to have some control so I create a /var parition when I add FreeBSD on to a computer. It is currently set to around 10 GB.
My simple upworld script depends on everything working and should die if it doesn't. That didn't occur on one occasion and some time in the near future, I will try to add status checking to the makes. I need a build that doesn't work to do this.
I usually fire it off from a telnet session and do "time upworld" if I want to see how long it takes. The build I made before I wrote this showed the following time information and it is
4525.721u 919.036s 1:45:20.37 86.1% 1328+1543k 74449+15900io 16430pf+0w
By current standards, this is pretty slow. A typical time on my AMD 1600+ XPs is 21-22 minutes for the elapsed time. A copy of my times.log from coral, which is an 1600+ XP, can be viewed by clicking coral-log. The faster 1600 is called opal and with 512 MB of DDR memory, it will do the buildworld in slightly over 21 minutes. You can also see the effect of using different numbers of HDs during the build and the effect of supplying a "-j?" parameter on a single processor system. In my case, supplying a -j on all of my single processor systems slowed the buildworld down across the board. It jumped up at -j2 and decreased to -j4 and then started increasing.
I have been known to use "at" to start it during the night. The only shock is finding a 6 MB log file in the morning mail. I have only had one session die and it didn't error off the make. I probably should check the return from make if there is one. That is something I can do in the near future. There is always the possibility of an error and that is why you have to check the logs. Having a log around is really good because you don't have to rerun the make in order to capture what died.
One of the reasons for the scripts existence is people popping up on freebsd-stable and claiming FreeBSD Stable won't build. I always have one computer that is doing nothing more than running Setiathome. I telnet or use ssh as is the real case and fire off "upworld". I will know in an hour or so if something is broken on the current version of FreeBSD-Stable. I copy where my system did or didn't have problems building the current stable. That gives them an idea if it is a local problem to their system. If I have problems, the cvsup.log will usually let me try to track it down to a cvs-commit and then I can email a copy of the problem to the commiter. If you have a system with problems, you really don't want to reboot until you are sure you have a good system. What you should see for a good build is log file sizes similar to
jade# ll *1108*
-rw-r--r-- 1 root wheel - 428878 Nov 8 09:52 bkernel-20001108-0942.log
-rw-r--r-- 1 root wheel - 5967365 Nov 8 09:42 bworld-20001108-0814.log
-rw-r--r-- 1 root wheel - 10723 Nov 8 09:52 ikernel-20001108-0952.log
-rw-r--r-- 1 root wheel - 503133 Nov 8 09:58 iworld-20001108-0952.logVersion 7.1 Changes
For 7.1-stable, a current buildworld is 10MB if you allow profiling and 8.8MB if you don't. The buildkernels on 7.1-stable are running around 2.6MB depending on the options you use.
The setup that I use now depends on the operations being separated into individual scripts. I have an upstable, which does the cvsup and cvsuplog. Then, I have a mkworld, mkkernel, inkernel, and inworld. The scripts do each of the make processes on their own. The only reason these scripts exist is that I simply got tired of typing the long strings and decided that everything could be lumped into a simple scripts. This has the benefit of always invoking a complicated script correctly. A mistyped mwkorld won't be recognized but a mkworld will. The complete process from cvsup to reboot to a fully updated system is around an hour on a dual Intel 2.4GHz Xeon based system. A buildworld is finished in around 50 minutes.
The most important line in the tail of the bworld-*-*.log file is the one that contains "World build complete" line. You can run "mkworld && mkkernel" at the same time but you have to tail the bworld output to make sure that your buildworld completed before you commit to an install of the kernel. On my systems, the install of the world is always done from booting to single user mode. People do complete updates and then reboot but in the time since 1997, which is when I started using FreeBSD, I have had a number of kernels that would panic when you rebooted. If your new kernel panics, you simply boot the kernel from /boot/kernel.old. Since, at that point, you are still using your old system, you can carry on as if nothing was wrong until they fix the problem. However, you can have serious problems if you are running an old kernel with a new world.
A mkworld with -j6 set, which was determined by running tests to be the optimum, required 48 minutes to do a buildworld of FreeBSD-7.1-stable on a dual Xeon 2.4GHz system and Sata-300 HDs. It is 17 minutes slower if you run it from a KDE session and have profiling turned on. The compiler output consumes that much time updating the KDE konsole window. Running from an ssh session is equally time consuming.
Back to Top
Adding a USB Mouse was one of the easiest features I ever added to my FreeBSD kernel. I started out using "moused". The usbd_conf also starts moused so you have to disable moused. The important lines in "MYKERNEL", which are all set in GENERIC, were the following:
# USB support
device uhci # UHCI PCI->USB interface
device ohci # OHCI PCI->USB interface
device usb # USB Bus (required)
device ugen # Generic
device uhid # "Human Interface Devices"
device ukbd # Keyboard
device ulpt # Printer
#device umass # Disks/Mass storage - Requires scbus and da
device ums # Mouse
device uscanner # Scanners
# USB Ethernet, requires mii
#device aue # ADMtek USB ethernet
#device cue # CATC USB ethernet
#device kue # Kawasaki LSI USB ethernetNote: I have turned off the Storage and NIC options.
The important lines in my /etc/rc.conf were then
#moused_enable="YES"
usbd_enable="YES"All you have to do then is make your kernel and reboot. Like I said it is very easy.
| Cert Coordination Center | eEye Digital Security |
| F-Secure (was Data Fellows) | Symantec Norton Anti Virus |
| Command Software Systems Anti Virus |
|
|
