Category Archives: Daily

Another case where SSDs don’t help

Posted on by

The project I’m working on involves compiling a huge number of source files which means we need a sizable amount of scratch space on which to run our experiments.  Our main compute server was running a little short on disk space, so it seemed like the appropriate time to add a new disk. We considered both solid state and traditional spinning platter drives.  The SSD seemed like the better performing options, but we eventually settling on an HDD, as the price of SSDs (due to restriced suppliers) was still prohibitive.

I was curious though how much performance improvement an SSD might have yielded, so I ran a small experiment on my desktop, which does have a (small) SSD.  I built the clang C/C++ compiler version 3.2, first off the HDD (a typical 7200RPM 750GB affair, then off my SSD (a Crucial Sandforce MLC device), and finally off of a ramdisk (Linux tmpfs).

The result?

Less than a 1% difference in compile time between the 3 options.

Similar to the case of Lightroom, it looks like compiling, at least for a mid-sized project (500MB of source) doesn’t benefit from an SSD vs. a hard disk.  Considering that ramdisk and HDD performance were virtuall identical, it seems quite likely that the whole thing never even left the memory of the operating system’s disk cache.

File compression on UNIX

Posted on by

I’ve been moving around a lot of data lately, particularly over the network, so it seemed like a good idea to settle on a compression regimen.  Networks are fast and all, especially at school, but moving multiple gigabytes of data still doesn’t happen instantly.  So I did a comparison of the current mainstream compression programs on Linux.  The system had a fast SSD drive, so operations were mainly CPU bound.

The contenders

  • bzip2 – a fairly popular replacement for gzip, though generally believed to be slower for archiving and unarchiving.
  • compress – interesting for historical purposes and accessing old archives, but no longer really used otherwise.
  • gzip – intended as a free compress replacement, it’s still the most commonly used UNIX compression tool.
  • lzip – an FSF-endorsed LZMA-based encoder claiming higher efficiency than more common tools.
  • lzop – uses a similar algorithm to gzip, but claims to be much faster and so particularly useful for large data files.
  • xz – an LZMA-based encoder claiming high efficiency and speed.
  • zip – still the de-facto standard on Windows, but not particularly popular on Linux.

The Test

To compare, I compressed and decompressed a 220MB tar archive, containing a distribution of the clang C/C++ compiler.  For all program other than compress which only has one setting, I tried the minimum compression setting (-1), the maximum compression setting (-9) and the default setting (no option).

Continue reading →

Revisiting Robert Jordan’s Wheel of Time

Posted on by

Wheel of Time

I began reading Robert Jordan’s Wheel of Time series of novels when I was in junior high school.  They fell under the ‘epic fantasy’ subheading in the local library, but I think ‘exhaustive fantasy’ might have been a more appropriate heading.  At the time, the series had just expanded to 5 books.  At 600+ pages a book, it was by far the longest continuous story I’d ever to read.  And of course it wasn’t yet finished.

Continue reading →

A respite

Posted on by

It is 66 degrees out, overcast, and drizzle has been falling intermittently for the past few hours.  In short, perfect.  Thank you, weather gods, for a long-awaited and much appreciated break from unpleasantness that is an east coast summer.

Long island sound

Looking out from Rye on Long Island Sound

Session B begins

Posted on by

P7023090

Even after 4 years experience with it, I’m still not all that comfortable with the semester system.  Fifteen week blocks just seem too long, with things inevitably starting to drag around week 6 or so.

It also makes the contrast with the summer term even more jarring.  At Columbia, there are two summer sessions, each six weeks long.  As a result, classes that normally met twice a week for 75 minutes now meet twice a week for 190 minutes.  It’s a lot.

Still, it’s nice to have already finished one session and have started the second one on the first week of July.  Hopefully this one goes as smoothly as the last.  So far (today was the first class), so good.

Too clever, by half

Posted on by

My late grandfather was a bright man.  He was the first of his family to go to finish high school, attended City University on scholarship where he received a Bachelor’s degree in Economics and was ABT for the Master’s degree before leaving to go into business for himself.  The IQ test that he took while in the army in WWII classified him as a genius.  Throughout his life, he was a voracious reader who seemed to have been interested in every topic at one point or another, and had done extensive research on all of them.

He often presumed, with good reason, to know more about a given topic than anybody else.  This made him a great person to get advice from, but a poor one to give advice to, for he very rarely listened.  This was particularly true on topics that regarded money.

The downside of all this was that it has taken my uncle and a lawyer the better part of several months to untangle the finances of his estate.  He did everything himself, the result being that he left behind a mess of papers that only he could understand.  And since he did everything himself, none of the stuff is accessible electronically making things like stocks a royal pain to deal with.  The irony of course is that he always believed that his papers were well-organized and set up, and probably for him it was.

I’m not sure there is a particular lesson to be drawn, but after spending much of the day with my uncle and grandmother dealing with banks and lawyers, it’s fairly clear that it is possible to be too smart for one’s own good, and that smart people can just as easily be blind to what they do not understand as everybody else.

Ward Pound Ridge

Posted on by

P6242864

Despite the fact that it’s directly adjacent to New York City, Westchester county actually has quite a lot of open areas, once you get beyond the main suburbs.  One of my favorite parks is the Ward Pound Ridge Reservation, barely a half hour drive from suburban Westchester.

I’ve been to the park more than a dozen times in the past few years, but always in fall or winter.  So in the morning, I got up early, and took the drive up the Saw Mill River Parkway toward Katonah.

I opted for a short route out to one of the park’s lookouts.  It was a pleasant walk, but somehow not nearly as picturesque as I remembered.  In fact minus the colored leaves or the snow, it was almost monotonous winding through the trees and up and down the little hills, each almost indistinguishable from the rest.  It was good to be outside after several 90+ degree days, but I certainly didn’t get many interesting photos.  In fact the drive ended up being the more picturesque part of the trip.  Nothing is ever quite as one remembers it, I guess…

Servlets

Posted on by

The last part of the project for my database class involves building a web front-end for the database we’ve designed and built (information on music, like iTunes).  Fair enough.

Unfortunately, the choice of implementation languages was Java (which I do know) and PHP (which I don’t).  So I was more or less obliged to build the thing using Java servlets.

Mind you, I have no doubt that servlets are quite indispensable in many places.  Creating a HTML front-end for SQL queries is not one of those places though.  Hopefully it works for the presentation tomorrow, but in its current state the code ranks as some of the ugliest I’ve ever written.

Guess that’s good motivation to learn PHP before the next time…

‘You are here because of them’

Posted on by

Cory Booker, mayor of Newark, New Jersey, gave the commencement speech at Stanford University yesterday.  His was the 4th commencement speech I’ve heard there, and I daresay it was the best by a fair margin, though I suspect more than a few in the audience were disappointed.

The traditional commencement speech is a tricky affair: balancing out the expectations of graduates, their families, and the university worthies organizing the show.  Comedy, advice and a certain amount of praise is the norm, although the formula varies widely.

What mayor Booker offered however was a series of life lessons, wrapped in a compelling personal narrative and leavened with more than a touch of humor.  This he did without sounding particularly preachy, speaking smoothly and steadily for more than half an hour with no discernible prompting or notes.  I found it a compelling performance.

I guess if I were to summarize his message in a sentence, it would be that all of us are where we are thanks to a whole community of people, and that it is by working together in such communities that worthwhile things are achieved.  But really, it is worth reading in full**.

I’m not terribly up-to-date on how Newark is doing these days, but if Cory Booker is half as effective at governing as he is at delivering speeches, the city is definitely on the right track.

Meanwhile I still can’t believe the best my graduating class (2006) could get was Tom Brokaw.  Inspiring he was not…

** Stanford has now posted a full transcript of Cory Booker’s speech.  Video of the event is available on Youtube.

Congratulations, class of ‘012

Posted on by

P6172087

 

My cousin received his Master’s degree today at Stanford (Department of Electrical Engineering)!

It was a typical June day in Palo Alto (read: warm – the black gowns did not look comfortable), and the event had the usual combination of seriousness and hijinks.  Plenty of crazy costumes on the field for the Wacky Walk, including a row of dominos, who obligingly collapsed on the field partway into the proceedings.

The department ceremony proved as hard to photograph as last year.  Harder actually, as I was using a manual focus lens on my Olympus E-PM1 (a 30-year old Nikon 135/2.8).  I did get a few photos though.  Probably will stick to autofocus lenses in the future, if I can.

I also made a visit to Sunnyvale, to the annual De Anza Tutor’s picnic.  Some familiar faces.  Looks like everybody is doing well (many have now transferred to Cal, SJSU, UCLA and UCSD).