Dilemma: I’ve got an old desktop system with less than 20 GB of disk space. On that system, I wanted to store backups of my laptop and our photos of Lennie. I also wanted to use it as a machine that will allow me to build software from time to time (which can take up a few GB of disk space). Without even starting backing up the many GBs of photos yet, I’ve nearly filled up the 20GB, and that’s after deleting a bunch of stuff. I happen to have another desktop system that has been on its last legs for a while, but it’s got a 65 GB hard drive.

Objective: Put the 65 GB hard drive in the working desktop system and make it operational.

Dilemma 2: I don’t really know anything about hardware. I forged ahead anyway, though, and I got it working. Here’s how.

Disclaimer: I did this on a Ubuntu Linux box. I have no idea how it’d go on any non-Linux box (so don’t ask). If you try these steps and you or your computer blow up or become otherwise harmed or incapacitated, note well that by reading further, you’re asserting that you’ve proceeded at your own risk.

1. Remove hard drive from dying computer. (This is pretty straightforward and doesn’t merit more detailed instructions.)
2. Read the back edge of the hard drive (the part with the pins) to figure out how to make it a slave so that the computer doesn’t get confused when it tries to boot up with two drives, This usually involves pulling a little plastic tab that slips over two pins off and sliding it back down onto two other prongs. I learned this the hard way by installing the drive, having the computer fail to boot, and remembering that something like this had to be done, whereupon I consulted the back edge of the hard drive.
3. Install the hard drive in a free bay in the working computer. This is pretty straighforward and, like initial removal, will require a screwdriver.
4. Cross your fingers that you’ve got the necessary cables available in your working computer’s case. If you don’t, you should probably consult some other guide. I was lucky. One cable is a ribbon cable with a plastic component that will receive the long array of pins on the back edge of the disk drive. The other is a little bundle of colored cables capped by a white plastic piece. It’s got four or five holds in the back for receiving pins. These plug into the obvious spots on the back edge of the disk drive.
5. Now boot up. The computer should boot as it did previously.
6. Get a terminal window open. If you don’t know how to do this, you probably shouldn’t go any further.
7. Find out what the disk is named. The following command will give you some output about your disks and partitions. You should look for one that’s the same size as the disk you installed and one that’s not listed in the output of the command “df”. If you don’t see one, something went wrong with the hardware install and I probably can’t help you.sudo fdisk -l
8. Make a directory to mount the drive on. I used /bak: Sosudo mkdir /bak
9. Now give your user permission on that directory. You might have to do something more involved if you need to allow more than one user to access the disk. This works for me, though.sudo chown -R houston /bak
10. Now partition the disk using fdisk. This is weird if you don’t know what you’re doing. I’ll walk you through my steps, which may or may not be the best to have taken.
    1. First do the following command, which takes you into a little interactive prompt (where /dev/hdb is the disk; note that it differs from /dev/hdb1, which I guess represents the partition on that disk; also note that you should substitute your disk identifier as above):sudo fdisk /dev/hdb
    2. At the command, enter “d” and then “1″ to delete the first partition on the disk. If there are other partitions, delete them as well.
    3. Then press “w” to write the changes to the disk and exit. This’ll probably take a few seconds or minutes.
    4. Next do sudo fdisk /dev/hdb again (substituting your disk identifier)
    5. Press “n” to add a partition. Chances are that you want just one, in which case you can just accept the defaults in the ensuing prompts.
    6. Then press “w” to write the changes to disk and exit.
11. Now format the new disk as an ext3 disk. Note that the /dev/hdb1 below is the same as what I discovered in step 7 to be my new disk. Substitute your value there.sudo mkfs /dev/hdb1 -t ext3
12. Ok, now you’ve got an ext3-formated disk with one partition. All that’s left is to mount it so that it can be used. To do this, edit /etc/fstab and add the following line to the end (again, substituting the drive identifier and directory the disk should be mounted on in columns one and two):/dev/hdb1 /bak ext3 defaults 0 0
13. Finally, typing the following command should mount the drive so that you can begin writing to it (substitute your path if needed):sudo mount /bak
14. Since we added the entry to /etc/fstab, the drive should come back mounted on reboot as well.

The other day, I was looking for some QuarkXpress-like publishing software for Linux after working with much frustration in Open Office to come up with a template for a publication I’m thinking about doing some work on. Open Office is great software, but it’s just not ideal for this kind of layout as far as I can tell. I found something called Scribus that seems a little rough around the edges but that I think’ll do the trick. Unfortunately, I had a lot of trouble getting it installed. It needed Qt, and that had some prerequisites that I had problems with. When I had trouble doing a manual install of this stuff, I decided to use yum to try to install qt, and at some point in the process, I used yum to uninstall it. And yum decided to uninstall a bunch of other stuff at the same time. Which didn’t really cause a problem until I had to reboot yesterday and gnome wouldn’t start. (Incidentally, I wound up finding an rpm for Scribus that installed with no problem and rendered all this pain really superfluous and thus doubly painful.)

So I decided to just reinstall my system. Since getting the widescreen and wireless working on this laptop was a pain originally, I made an effort to save all the info I recalled having needed to do so, figuring it’d be a pretty quick matter as a result. But I forgot to save my xorg.conf, and the wireless stuff didn’t work as seamlessly as one might have hoped. And though I thought I had blogged the steps when I originally set up this system just in case I needed to do it again, I hadn’t. Luckily, it didn’t take as long this time, though it still wasn’t exactly a walk in the park. So here’s a quick walkthrough for future reference.

For the widescreen, I need a tool called 915resolution. Once you get that installed, for this particular laptop, I add the following lines to /etc/rc.local:

/usr/sbin/915resolution 66 1280 800
/usr/sbin/915resolution 34 1280 800
/usr/sbin/915resolution 45 1280 800
/usr/sbin/915resolution 54 1280 800

Then, I edit /etc/X11/xorg.conf so that the screen, monitor, and video sections look like this:

Section "Monitor"
  Identifier   "Monitor0"
  VendorName   "--> LCD"
  ModelName    "1280X800@75HZ"
  Option       "CalcAlgorithm" "CheckDesktopGeometry"
  Option            "dpms"
  HorizSync    30.0 - 82.0
  VertRefresh  58.0 - 75.0
  UseModes     "Modes0"
EndSection

Section "Screen"
        Identifier "Screen0"
        Device     "Videocard0"
        Monitor    "Monitor0"
        DefaultDepth     24
        SubSection "Display"
                Viewport   0 0
                Depth     24
                Modes    "1280x800" "1920x1200" "1680x1050" "1440x900"
        EndSubSection
EndSection

Section "Modes"
  Identifier   "Modes0"
  # 1280x800 @ 75.00 Hz (GTF) hsync: 62.62 kHz; pclk: 107.21 MHz
  Modeline "1280x800_75.00"  107.21  1280 1360 1496 1712  800 801 804 835  -HSync +Vsync
EndSection

Some tutorials suggest that I need to get the ATI driver, but I’ve found that it’s not necessary. So, that takes care of the widescreen setup for this model (Sony VAIO VGN-FS550).

Next up, networking. I initially tried to cobble this together more or less piecemeal as I had done previously (thanks to the process of trial and error). But what ultimately worked was the directions that start here, which links to this site, which is the real jumping-off point. Essentially, here are the steps:

1. Download and install the ieee80211 subsystem.
2. Download a recent ipw2200 driver (which requires the ieee80211 subsystem).
3. Download and install recent firmware.
4. Copy the load, unload, and other similar scripts from the ipw2200 directory to, say, /usr/local/wireless
5. Add the following to your /etc/rc.local:

   modprobe ipw2200
   cd /usr/local/wireless
   /usr/local/wireless/load #ipw2200 driver loader
   

6. Reboot or execute those commands by hand.
7. Then just bring your wireless interface up as you normally would.

The installs listed in the first two steps are pretty routine. Just do the standard make and make install. I don’t think I had to do any other steps to make those work. The firmware files have to be copied into the hotplug directory on your system, which you can find the location for by looking for the FIRMWARE_DIR line in /etc/hotplug/firmware.agent.

After getting the wireless working, I ran into a couple more problems. For both wired and wireless interfaces (though I initially thought it was just the wireless), I started getting random disconnects. When I looked at the output from dmesg, I kept getting the error “no IPv6 routers present.” After googling around, I checked /etc/modprobe.conf and made sure the line for eth1 specified ipw2200 as the module rather than eepro100, which was what was listed when I checked. So far, this has kept my link up for a few minutes. If this is the sort of thing that degrades over time, it might not appear just yet and I might not be out of the woods, but in the short term, that change (and a subsequent reboot) appears to have done the trick.

I’m working on a community dashboard for work that’ll help me monitor different ways in which people are interacting with the community. One metric I thought it’d be neat to have was how many people have filed more than X bug reports during the last period Y (month, say, or week) and what the average for that period over time has been. This involves a pretty complicated query wherein you have to group by both user and period, which is kind of tricky. It’s also very expensive: With only about 1200 bugs in our database, this query was taking six seconds when I was testing it out. That’s clearly unacceptable. So I poked around a bit and discovered the GROUP_CONCAT function, which lets you group by one column and output the relevant grouped data in a delimited column. So say your query is something like this:

select DATE_FORMAT(creation_ts,’%Y-%m’) date, group_concat(reporter) cnt from bugs group by DATE_FORMAT(creation_ts,’%Y-%m’)

Here I’m getting all months in the database and a listing of all reporters of bugs for that month. The “cnt” column is a comma-delimited lists of user ids for those who reported bugs in the given month, duplicates included (though you can specify DISTINCT to eliminate duplicates). The query returns almost instantly and gives me something I can parse pretty easily in the programming language of my choice.

In my case, I split the “cnt” column on commas for each result and tally the ids for users who appear in the list X or more times. I then pass this data back to a function that does math to get averages and to find a count for the current month, and voila, community metrics.

I blog this here because it’s the sort of thing I might want to remember later, and I always put that stuff here rather than at my work blog, where my regular readers probably figure this should actually go.

Or, how I learned to stop worrying and import the bugs. Hrrmmmm.

So my company’s looking at switching from using the beastly Bugzilla bug tracking software to using something called FogBugz. It seems a decent enough piece of software, though the install process went as follows:

1. Download and untar install files.
2. Learn from the readme that it requires a PHP extension, a daemon, and (for me) a PHP upgrade. This is all rather more intrusive than most bug software I’ve seen, though to be fair, Bugzilla also has a lot of dependencies.
3. Upgrade PHP.
4. Run a script at the command line to make sure dependencies are all worked out.
5. Start the daemon.
6. Run a wizard on the Web to set up the database, etc.
7. Buy and submit license key.
8. Worry about importing bugs from old bug tracker into new one.

This last item is the one that’s taken up some four hours of my morning today. FogBugz apparently ships as an asp application, but there’s a PHP version that we’re using. For the Windows version, there’s a script to import from Bugzilla, but it’s hosed on Linux. It seems to tap into stuff set in the PHP extension that’s Windows-specific or something. So I worked a little magic using some PEAR classes and was able to connect to Bugzilla’s xml interfact to retrieve bugs.

But I was getting authentication errors. Apparently, if you’re not logged in, you can’t see bugs using Bugzilla’s xml interface. I tried a couple of things and finally just used links at the command line with the -source flag to get all of our bugs as text files. I had to browse to our Bugzilla install using links and login first, though, so that the cookie would be set and subsequent connections would be able to see the xml data rather than the “NotPermitted” error.

So, now I’ve got 700ish bug reports in a directory. Time to hack the import script to make it work using my data rather than the obfuscated methods available through the Zend extension. This was just a little tedious and mainly consisted of finding objects and function calls and replacing them with arrays and keys that hold the corresponding data from my parsed xml. That is, the shipped code gets an xml file and creates objects to hold the data, while my code slurps in the xml and creates a nested array for each bug; I had to make the import script read my arrays rather than looking for objects. There were a couple of little hitches that were easily fixed, and before too long, I had my 700ish bugs imported into FogBugz.

But I noticed when reviewing my new bug list that I had a bunch of bugs that weren’t mine, and others had bugs that should have been mine. Moreover, most of the bugs were assigned to just two of us. Why this happened I have no idea. To fix it, I wrote a script that reconciles user ids between the two systems and changes bug assignment in FogBugz for each bug to what the assignment in Bugzilla originally was. For example, say user X in Bugzilla has numeric id 5, while he has numeric id 22 in FogBugz (just because of the way the users were imported). Luckily, both systems use the email address as the login, so I could query both databases, get the numeric id for each, and then do a query for each bug that says (in English) “Change the assigned-to field for bug 700 to the numeric id 22 because the numeric id for assigned-to in Bugzilla is 5, and user 5 in Bugzilla corresponds to user 22 in FogBugz because they map to the same email address.”

And voila, my bug list is as it should be. I’m sure we’ll find problems here and there, but by and large, I’d say it was as successful an import as could have been hoped for.

/var/mailman/bin/withlist -l -r fix_url <list_name> -u <new_hostname> -v

This trick comes in handy when your company’s gotten a new URL and you want the subscription confirmation messages and various other things to use the new URL. It also doesn’t hurt to poke around in /var/mailman/Mailman/mm_cfg.py and change a couple of things and restart the mailman process.

It’s midnight on a Saturday night, and I’m at a dork party I wasn’t sure I’d go to. It’s at a big house, and I’m one of thirteen in one of the den/living rooms. We’re all scattered about on couches, folding chairs, and the floor. There’s enough cable running around on the floor to, to I don’t know what. To put a small-to-medium sized business’s IT staff to shame. There’s much talk of browsers and xcode and extensions and so on and so forth. I donned my best dork shirt for the event (it reads /(bb|[^b]{2})/, which means something to dorks) and have already gotten a good cackle from somebody over it. I occasionally have to stop my diligent work to go to a terminal and type “iwconfig eth1 essid ‘bret’; ifup eth1″ to get back on the network, because I keep getting dropped. The other den full of people is apparently more social than my room, which is full of relatively quiet people. I’m probably the quietest. It’s not nearly as painful as it could be. If only I could get this makefile to work blah blee bloopety blooblah.

Just a quick reminder to myself that the easiest fix to the “Packets out of order” error that Ruby on Rails sometimes throws is to edit the mysql adapter to prevent it from trying to load code that handles mysql’s new password hashing algorithm. Basically, in mysql 4.1.x, a new password hashing algorithm was introduced, and Ruby on Rails tries to connect using the new algorithm if it determines that your server version is greater than or equal to 4.1.x. If your actual mysql passwords are in the old format (16 characters), the connection will fail.

Several fixes have been suggested, including installing C bindings for the mysql driver (gem install mysql), which didn’t work for me. Ultimately, working from a cryptic suggestion by a coworker, I went into my ruby libs and edited /usr/local/lib/ruby/site_ruby/1.8/active_record/connection_adapters/mysql_adapter.rb and commented out the line seen commented out below:

begin
            require 'active_record/vendor/mysql'
            #require 'active_record/vendor/mysql411'
rescue LoadError

This prevents the code that does the 4.1.x substitutions from executing and allows your install of Ruby on Rails to play nicely with mysql users that have the old password format.

My company is looking to hire a couple of full-time senior developers. The following criteria must be met:

• You must already live near or be willing to relocate to Silicon Valley.
• You must have a minimum of five years of application development, with a focus on the following skills, in descending order of importance:
  • Mozilla application development
  • XUL
  • XPCOM
  • C++
  • javascript
  • CSS
• Experience developing on two of the following three platforms: PC, Mac, Linux
• Experience working on Open Source Software projects is a plus.
• All the usual buzz words (self-starter, team-player, etc.) apply.

Send me a resume at abcdefg@hijklmnopqrstuvw.xyz (but replace abcdefg with my first name and hijklmnopqrstuvw.xyz with learnhouston.com) if you think you’re the person for the job. If you’re not the person for the job, but you might know who is, have that person send me a resume and include your contact information. There’s a small reward for whoever turns up a person we ultimately hire. Please don’t apply or send me resumes for those who don’t meet the criteria listed here.

A few weeks ago, I posted about a shell script I had written to help automate the building of an extension for Firefox. Building an extension’s not a hard process, but it can be a pain to do over and over because it involves zipping some files and putting them into a directory, then zipping that directory and another file into another archive. It’s just kind of a pain. This week, I’ve been hit with a need to generate different versions of my extension for different platforms in order to accommodate styling differences among the platforms. (Macs have nice rounded text inputs for some of their widgets, for example.) I could find no way to do a simple sniff to determine what CSS file to use for a given XUL document within an extension, so I figured I’d need to generate separate packages for each platform, complete with specialized style sheets. So I was looking at having to generate packages for each platform. Even with my nifty little build script, that was going to be kind of a pain and was going to necessitate keeping three copies of my code and making updates in three places. What a pain.

To get around this, I wrote a new script this morning that builds packages from a base directory and incorporates changes per platform. So I make core changes to one code base and make changes to platform-specific code only within the given platform’s directories. When I run the build script, for each platform specified in the script, it copies the base files into a temp directory, then copies platform-specific files in on top of them. Then it builds and exports the package to an export directory specified within the script. It should make this whole process substantially less painful. The script appears below.

#!/bin/sh
##############################################################
#                                                            #
# Multi-platform Firefox extension build script.             #
#                                                            #
#    Author:    Daryl L. L. Houston                          #
#               http://daryl.learnhouston.com                #
#   Version:    1.0                                          #
# Rev. Date:    Tue Apr 12 11:01:51 EDT 2005                 #
#                                                            #
#     Usage:    Current directory should contain directories #
#               "base" and platform-specific directories.    #
#               Base directory contains install.rdf and      #
#               files common to all platforms. For example:  #
#                                                            #
#                  + contents/                               #
#                    + r2/                                   #
#                      - contents.rdf                        #
#                      - overlay.xul                         #
#                      - overlay.js                          #
#                  + skin/                                   #
#                    + r2/                                   #
#                      - r2.css                              #
#                  + locale/                                 #
#                    + en_US/                                #
#                      + r2/                                 #
#                        - overlay.dtd                       #
#                  - install.rdf                             #
#                                                            #
#              Platform-specific directories contain files   #
#              in addition to or different from base files.  #
#              So if the Mac package needs a different       #
#              stylesheet, you create the skin hierarchy     #
#              under a directory named (eg) "mac" and put    #
#              the Mac stylesheet in there. It will be       #
#              dropped into the Mac package in place of the  #
#              base package's style sheet.                   #
#                                                            #
#              Packages are named (eg) "extname_mac.xpi"     #
#              where the "extension" variable below is       #
#              set to "extname" and are placed in the        #
#              directory specified in "export_path" below.   #
#                                                            #
##############################################################

#Set these variables.

#Final export directory.
export_path='/home/houston/code/extensions/xpi'

#Package name. This should match chrome:name from contents.rdf
extension='r2'

#Platform directories. Will be used to create final xpi names.
#Corresponding directories must also exist.
dirs=(mac windows linux)

#No need to edit below this line.

for idx in $(seq 0 $((${#dirs[@]} - 1)))
do
	#Need to make a temporary build work space
	mkdir temp

	echo ' '
	echo '===== '${dirs[$idx]} '===='

	#Remove old xpi files.
	oldxpi=$export_path'/'$extension'_'${dirs[$idx]}'.xpi'
	if [ -e $oldxpi ]
	then
		echo '+ Removing ' $oldxpi
		rm $oldxpi
	fi

	#Copy common files into build directory
	echo '+ Copying files into ' ${dirs[$idx]} ' temp build directory'
	cp -Rf base/* temp/

	#Now copy (and force) files from the current platform build directory
	#into temp directory. This adds new files and overwrites base files.
	cp -Rf ${dirs[$idx]}/* temp/

	#Make chrome directory
	if [ -d temp/chrome ]
	then
		rm -rf temp/chrome
	fi
	mkdir temp/chrome

	#Now go into temp directory and zip up the files copied there, name the zip
	#with a .jar extension, and move into the chrome directory.
	cd temp
	echo '+ Creating jar for '${dirs[$idx]}
	file=$extension.jar
	zip -r $file content skin locale
	mv $file chrome/

	#Create the xpi by zipping install.rdf and chrome
	#and giving the file a .xpi extension.
	echo '+ Creating package for '${dirs[$idx]}
	file=$extension'_'${dirs[$idx]}'.xpi'
	zip -r $file install.rdf chrome/

	#Move the newly generated extension to the export directory
	echo '+ Moving xpi file to output directory for '${dirs[$idx]}
	mv $file $export_path

	#Go up a directory and remove temp directory in anticipation
	#of iteration for next platform. Lather, rinse, repeat
	cd ..
	rm -rf temp

done

Until today, every time I’ve clicked a mailto link in Firefox, the clunky old Evolution email program has started churning, and I’ve smacked myself on the forehead and cursed as I waited for that behemoth of an application to load. You see, I use the lightweight Thunderbird, which loads in an instant and is as flexible and powerful as I need it to be. So I wait for Evolution to finish loading a compose window, close that down, copy the mailto link into a Thunderbird composition window, and write my email. It’s a pain every time.

One of the RSS feeds I keep an eye on is Linux Journal’s, and an article entitled “Ten Mysteries of about:config” caught my eye there today. It’s a decent little article by Nigel McFarlane, author of Firefox Hacks, which I just received in the mail this week and which looks promising. One of the mysteries Nigel explains is how to set a default mail client in Firefox on Linux (it’s easy enough in Windows because it’s governed at the system level). It takes only three steps:

• Open about:config in your browser.
• Set the preference named “network.protocol-handler.external.mailto” to true; if it doesn’t exist, create it and set it to true.
• Create a preference named “network.protocol-handler.app.mailto” and set it to the path to Thunderbird. In my case, this is /usr/local/thunderbird/thunderbird.

That’s all there is to it. When I did this and clicked a mailto link, a Thunderbird composition window shot to the foreground and I was in business.