Archive for June, 2006

Quick and dirty apache logfile analysis

June 28th, 2006 by daryl

Dilemma: I had a bunch of rotated apache log files that I wanted to check traffic patterns in to see if some link changes I had made to a site were affecting traffic. Specifically, for the domain in question, I had tried to route requests to certain urls over to another domain by changing links in the html, and I wanted to see if it was actually impacting traffic. So I wrote the following little bash script to iterate over a set of log files and print the date and the line count for specified search strings.

#!/bin/bash

if [ -z "$1" ]
then
echo “Usage: $0 search_string [log_file_prefix] [log_directory]”
echo “log_file_prefix defaults to ‘access_log.’”
echo “log_directory defaults to ‘.’”
exit
fi

if [ -z "$2" ]
then
2=”access_log.”
fi

if [ -z "$3" ]
then
3=”.”
fi

FILES=$(ls $3)

for FILE in $FILES
do
EPOCH=${FILE##*$2}
DATE=$(echo $EPOCH|awk ‘{print strftime(”%c”,$1)}’)
COUNT=$(cat $3/$FILE | grep “GET $1″ |wc -l)
echo “Looking for $1 in log for $DATE: $COUNT”
echo “”
done

So say I execute the command as follows: “./log_parse /about access_log_www. www”. It would scan for requests beginning with “/about” in all log files whose names begin with “access_log_www.” in the directory “www”. The script assumes that rotated log files are suffixed with a timestamp, and it writes the time based on that timestamp. Output looks something like this:

[root@www logs]# ./log_parse /about access_log_www. www
Looking for /about in log for Thu 08 Jun 2006 07:00:00 PM CDT: 726

Looking for /about in log for Fri 09 Jun 2006 07:00:00 PM CDT: 681

Looking for /about in log for Sat 10 Jun 2006 07:00:00 PM CDT: 28

It’s certainly not a full-service solution for log analysis, but it makes a quick check of one-off traffic patterns over time pretty easy to spot.

Flock

June 23rd, 2006 by daryl

So, people who know me know I work for Flock, which has something to do with computers. We actually produce a web browser built on the platform that Firefox is built on. When we first started talking big a little less than a year ago, we sang out about how we were going to revolutionize browsing. Talking big got us in a little trouble because we were over-hyped too early. We’ve since changed our message a bit. Rather than being the saviors of the internets, we’re just trying to make the web a little easier to use for some people. If you like to blog, share your bookmarks, and use photo services like flickr and photobucket, maybe Flock is something you’d be interested in. Otherwise, it’s probably not. Here’s some of what I personally like about our product; these things are what would make me think about using Flock if I didn’t feel bound to because I’m an employee.

Built-in news/feeds integration. Most blogs and many news sites you see nowadays display little orange radio-wave icons and advertise things like “RSS2.0″ and “Atom.” These are just techie ways of saying “here’s a list of recent articles that your software may be able to keep track of.” The idea is that you can plug these feeds into your feed reading software, which will notify you of updates so that you don’t have to go out each day and click through to the dozen sites you like to keep up with. There have long been Firefox extensions and standalone applications that would consume these feeds, but I’ve never liked any of them. In the case of the extensions, the interface has always bugged me. In the case of applications, well, I really don’t need another application open on my desktop. Flock has feed-reading built in. I subscribe to the feeds for the blogs I like to read. When one of them has new content, a button in my browser lights up to let me know. So no more going out and visiting the site to see what’s new. When I click the news button, a sidebar opens up that gives me a tree view of my feeds (sort of like an email view, with folders and items on the side), which I can scan easily for updated content. This feature really makes my life easier and is probably my favorite thing in the browser.

Next, there’s the star button. It’s a little button connected to the urlbar. If you’re at a site that you like and would like to save for later, just click the star button. It’ll turn orange, and when you later visit sites you’ve starred, it’ll turn orange to let you know you’ve already starred them. If you’re adventurous, you can make an advanced options box appear that’ll let you apply tags (just think of them as ad hoc labels that are easier to use than nested folders) to the things you star. We call the things you star “favorites.” And there’s a favorites manager that lets you see a list view of the things you’ve starred. You can search through them by tag, sort by recent, organize them into collections (which you can make appear in your toolbar), and go back to their advanced options to change settings. What’s more, if you’re so inclined, you can set your favorites to be synced up with an online bookmarks service like delicious or Shadows so that your favorites are available to you anywhere, any time. I like our favorites system because it lets me do some level of organization of my links without making me manipulate a bunch of nested folders; I can quickly apply multiple labels that let me get to my links later in several ways without having to file them redundantly in multiple folders.

Favorites interact with the search box, which widget can be very useful but is something I often overlook. If you’ve used Firefox before, you’re already familiar with the search box embedded in the browser interface. You can select from among several search engines to direct your searches to, and you can easily perform searches without first having to navigate to the search page itself. I have muscle memory that makes me hit my home button and search straight from Google, so the search box hasn’t historically been something I’ve used (hence my often overlooking it now). But there’s one thing that makes it very handy in Flock: As you type search strings, it searches through your browser history and your favorites and populates a flyout with relevant searches. In other words, it’s a quick interface to search for relevance among things you’ve already flagged as relevant by visiting or starring. And when I say it searches through your browser history, I don’t mean it looks for strings in the url. It does a full text search on pages you’ve visited. So even if you didn’t star that page about platypuses that you found so interesting, if you want to get back to it without combing through thousands of results from Google, you can just start typing “platypus” in the search box, and sites you’ve visited that contained the word will magically appear in the flyout.

Next, there’s photo integration. We have this thing called a topbar that’s a little slice of screen that pops open between the browser buttons and the content window. It’s a pretty good size for displaying thumbnails of photos, and it’s got a neat little slidey interaction that makes it easy to zoom through photos it contains. The photos currently can be set up to pull from accounts at flickr and photobucket, and you can use the topbar to view your own photos or those of other users. The topbar will also notify you by lighting up a browser button when your contacts post new photos, so there’s no more running out to click through your contacts’ pages to see what’s new — Flock just brings it all to you with no effort on your part. This functionality is of pretty limited use to me because I don’t follow photos very closely, and the ones I tend to care about usually get posted to kodakgallery, which Flock doesn’t yet support and which sends me email invitations to see photos anyway. What is useful to me to the extent that I use it is the fact that Flock comes with a photo uploader that makes it pretty easy for me to send photos to flickr. As a Linux user, I’ve always had to do it the painful way, using file upload widgets and navigating through files by name to upload. This is definitely not a good way to batch upload photos. Now I can just open the uploader, browse files or drag files to the window, rotate and crop, and upload. If I were inclined to use photo services very frequently, this would be a great tool for me, especially since most of the tools out there that simplify photo uploading don’t work in Linux. Rumor has it that you can also now drag photos into textareas in the browser and they’ll get uploaded and posted automatically, but I haven’t tested that functionality out yet and so can’t verify that it works.

The last two features that stand out to me are things that used to excite me but that now completely underwhelm me. The first is the blog editor. It’s built right into Flock and lets you edit and save posts locally before pushing to your blog service. One nice thing is that it’s got drag and drop goodness, and I’ll bet that by this time, if you drag photos into it, they get pushed to your photo hosting service. Generally, I’m about as happy editing blog posts in my blog’s user interface, and as far as blog editing usability goes, I prefer the extension produced by Performancing to our blog editor. This is really sort of a shame, as blog integration was one of the things we wanted to address best and first, and in my opinion, we’ve regressed a bit on that front. The other feature I used to like that I now find almost useless is what we’re calling web snippets. This used to be a little sidebar or floaty window that you could drag photos and text snippets to for later use in blog posts. We later added the ability to add notes to the web snippets, and I found this useful as sort of a blogging to-do list. But we crammed the tool down into a little area at the bottom of the screen and made it pretty much unusable because of the way it uses its real estate. In theory, it helps you assemble things to put into blog posts, but in reality, for me, at least, it sits down in the status bar unused. These two features were very appealing to me when we started working on them around a year ago, but they’re now the least useful things we’ve added to the browser.

Which is ok, because, as I think I’ve indicated, we’ve added other useful things. The news functionality alone makes me want to use Flock. I think we’ve also produced a beautiful browser. As much credit as Firefox is due for helping to bring choice back to the web browser industry, the browser’s default theme is heinous. Any time I run Firefox now to test something, I’m shocked at how clunky the buttons and toolbars look, and I find it a little oppressive. Using Flock just feels smoother to me because our interface looks smoother. This is of course purely a matter of taste, and it should be noted that both Flock and Firefox can be re-skinned to look entirely different and better (or worse). I think that for the time being, Flock provides a better experience out of the box, though.
So, that’s what I’ve been doing for the last 18 months. I do primarily web programming and system administration these days, though I’ve been involved at some point in pretty much every part of the business. Last week, we released our first public beta (we’ve been in developer alpha for months), and feedback so far is pretty good. We’re still recommending that people try Flock out but not be disappointed if we’re not quite ready for prime time or if the browser’s a little buggy. If any of the features I’ve described sound useful to you, I invite you to try the browser out. You can download it here. If you’re not game, no hard feelings; as I said at the beginning of this post, Flock isn’t for everybody.

Fish tale

June 1st, 2006 by daryl

Fish tattoo designsIt’s not so much a question any more of whether I’m getting a tattoo but rather of what and where. For a while now, I’ve been thinking about getting a tattoo. That there are a couple of shows on cable right now showcasing the art has probably influenced me. It’s a strange yearning for me. Many people get tattoos to symbolize or commemorate something in particular — the death of a loved one, for example, or religious faith. I don’t have anything in particular I want to say with body art. I just think it might be neat to have a tattoo.

Not having anything in particular I want to express causes a bit of a dilemma in that I have to come up with something meaningful to me or go with something arbitrary and probably boiler-plate (so to speak). In trying to come up with something not completely lame, I’ve felt a little like a high school student told to write a thesis and working hard to come up with a thesis and support it.

I thought about doing something with pi, but I don’t think pi stands alone as a tattoo for a guy who can barely tell you what pi even is (for the record, it’s the ratio of a circle’s circumference to its diameter). Carl Sagan said beautiful and interesting things about pi in his novel Contact (the movie version of which I recently re-watched). Pi in some form or another is something that I notice on a more or less daily basis. Beginning in college, I noticed that I frequently happened to look at the clock when it read 3:14, and Mleeka and I settled on pi day (March 14, or 3/14) for our wedding date when we had no other special day in mind. In response to my mini-obsession with pi, my parents recently got me a pi dish (a pie dish with the pi symbol in the bottom and a few dozen digits of pi around the rim). Pi by itself has no special significance to me, but it is a number and a symbol I have enjoyed in a layman’s way for several years, so it’s a solid candidate. I still hold that it doesn’t stand alone as a pictogram to be engraved on my body, though.

So what else? Mleeka proposed a whale. If you know me reasonably well, you probably know that I have another mini-obsession with Moby Dick. Tattooing is very much present in that venerable tome, and it’s not lost on me that scrimshaw (animal bones and teeth carved with elaborate designs by whalers, which art form I have yet another mini-obsession with) and tattooing share something of a kinship as carving arts. So a whale tattoo maybe makes sense for me. But it’s a little too close to blue-anchor-retro to really be fitting for me. I think maybe there should be a law against getting a whale or otherwise seriously nautical tattoo unless you actually have something to do with whales or sailing. Plus I think it’d be really easy to get a crappy looking whale tattoo.

Now what? Let’s turn to word-origin and -play.

According to Melville (who was wrong, but quaintly wrong), a whale is a fish with its tail turned sideways. So, whale = fish. The latin word for fish (plural, I think) is “pisces.” So whale = fish = pisces. And pisces, if you’ll grant me some license here, is a homophone for “pi seas” (which calls to mind for me the phrase “chicken of the sea,” which is actually a word-playful way of saying “fish”). So whale = fish = pisces = pi seas = fish = whale = one nice big circle of equivocation and linguistic chicanery that appeals to me while at the same time touching at least tangentially (recall from high school geometry precisely what a tangent is?) on a couple of the ideas that have appealed to me for tattoos.

But wait, there’s more.

In college, I latched onto the fish (and the hunt for the fish and catch and release and the fish as a religious symbol and losing the fish and all manner of other similar things) as a metaphor meaningful enough that it became the centerpiece (and a part of the title) of a manuscript I wrote while completing my writing minor. This was all before I read Moby Dick, mind you, and Melville did a much better job in his treatment of the metaphor than I managed, naturally. The point, though, is that the pi and the whale ideas crystalizing in my mind through word-play (something else that’s a largish part of who I am) to converge on the idea of the fish, which has also been meaningful to me in what I once considered my own little art and thought — well, it’s nice. It’s appealing, if not as significant as paying homage to a dead loved one by duplicating his or her tattoo (for example).

What remains is to figure out exactly what kind of fish I want to get, if I do in fact determine I have the stones to get a tattoo. I’m not interested in a colorful fish or even a realistic one (see note above about how easy it probably is to get a bad whale tattoo). Stylized Koi look great on some people, but they’re just not my bag. I’ve looked around at a lot of images online and finally settled (at least for the moment) on a derivative design of my own creation (or derivation), two drafts of which are pictured here. The basic shapes, especially toward the front of the fish, come from a tee-shirt design I found. The innards of the fish pictured on the shirt are rather more maze-like and less fish-like than the treatment in my current draft. I added gills (note that they look a little pi-like) and modified the spine and tail pretty substantially. I’m not terribly happy with the back half of the fish, which seems a little inconsistent stylistically with the rest of the image, so I’d probably take one of these images to a tattoo artist to provide a starting point.

So there you have it. The “what” I mentioned initially is more or less resolved, and I need now to figure out the where (on my body), which may require the advice of a tattoo artist.

If I chicken out (and if I do, I think it’ll be because I harbor a tiny worry that I’d regret a tattoo in 20 years), I hope you’ll think of me as the chicken of the sea = pi of the seas = pisces = fish = whale. And we all know how messing with the whale turned out for Ahab, now, don’t we?