For now though, winter is back on schedule.  There were even people out skiing at Rockefeller State Park.

Swan Lake – Frozen and Snowed Over.

It’s hard to know what to say at such an event.  Words seem painfully inadequate.  She was an extraordinary person in many ways, but above all she had a knack for bringing cheer and joy wherever she went.  In our hurried, impersonal world, that is indeed a rare gift.

One of the more poignant moments at the memorial came when her close friend explained that Carol had said previously that she didn’t want any memorial to be held.  ”I know.  But it’s not for you.  It’s for us,” the friend replied.  And it was.

Goodbye Carol.

Looking back at old posts and the memories they conjure up, I recognize the skinny, sharp, cynical, occasionally confused kid who posted his musings on these digital pages.  I’m not sure quite what he’d think of the current version of me, but I hope wouldn’t be too terribly disappointed.

And if I could send a message to that fellow (not that he would have listened!) what with hindsight being a benefit and all, it would have been: Don’t take yourself (or anybody else) so seriously.  Life is just too short.*

*Seriousness is a leading cause of chronic overcautiousness among other damaging neuroses.  Such a condition, if untreated by ample quantities of levity and silliness, frequently degenerates into a state known as ‘being a tiresome bore.’

Still, given that the 2012 ‘election-cycle’ is almost upon us, and given the absurd spectacle we are now greeted with, now seems as good a time as any to say what I think, and what I think (writ large) is:

The Democrats deserve to lose this election.

Let us return briefly to the jubilant scene just three years ago when Barack Obama, surrounded by millions of cheering Americans, was inaugurated as the next president of the United States.

It was a moment to remember.  Beyond the symbolism of an African-American, the son of a single mother winning the most powerful elected office in the land, there was the political facts.  For the first time since 1994, the Democrats would control both houses of Congress and the White House.  For the first time since 1976, a Democratic presidential candidate had won more than half the popular vote.

Meanwhile, the opposition Republican party was weakened and directionless.  A Republican president had just presided over beginning of the largest economic crisis since the Great Depression.  They suffered damaging losses in Congress, including their entire New England house contingent.  So-called independents broke decisively against Republicans, as did young people, women, Latinos and African Americans.  The party appeared in danger of becoming the party of aging white men.  The finger-pointing and recriminations among the leadership were non-stop.

Even the traditional anti-Democratic sources were uncharacteristically silent.  The economic collapse had weakened Wall Street, leaving it temporarily silent.  Most big businesses were worried about their own problems, unable or unwilling to rock the boat.

In short, we not only had an eloquent new president with a compelling life-story on our hands, we had a once-in-a-generation political opportunity as a hugely popular president backed by large Congressional majorities faced a frightened and fragmented opposition.  Not since the days of Lyndon Johnson had liberals had such strong reasons to hope for the legislative enactment of a progressive social agenda.

Alas, it was not to be.  Congressional leaders fell to squabbling.  The president refused to get his hands dirty in the legislating process, allowing important bills to languish in Congressional committees.  Meanwhile, the economic distress continued, and the president’s conservative economic policies had little visible positive impact.  The lack of progress and change left many voters confused and disheartened, as did the reemergence of a Republican opposition determined to block the president at every turn.  In the 2010 midterm elections, Democrats were swamped, and all hope for major progress legislation died with the end of the 111th Congress.

While there are many reasons for this failure, the primary one was a failure in leadership.  The president did not lead.  He did not seize the political moment.  He did not effectively use his majorities and popularity when he had them and now they are gone.

Fundamentally, the Obama presidency’s lack of leadership is underlaid by a profound political naiveté.  Unable or unwilling to see the opposition for what it was, he wasted precious time and political capital soliciting their input, meeting their leaders, and negotiating with them when he did not have to.  It takes a special lack of savvy to fall victim to the same trick again and again, but that is precisely what Obama has done.  Unwilling to call a spade a spade, he has negotiated repeatedly with people whose primary goal is to see him fail.  He has played directly into their hands, and now he is suffering the consequences.

Politics is war.  You gain no credit for being a gentleman.  There’s no ‘A’ for effort.  Results are the only metric by which one is judged.  The president should have taken his political capital and spent it ramming through legislation rather than dallying in meetings with people whose votes he would never get.  Rather than compromise, he should have fought for the most liberal legislation that had any decent chance of passage.

Moreover, he should have used whatever advantages he had to weaken the opposition.  Why not ‘go nuclear’ and end the filibuster once and for all?  Why not fill all vacant judgeships with the most liberal judges available through recess appointments?  Why not pass a truly massive stimulus, investing directly in transportation and energy, and funded by a revamped tax-code that raises marginal tax rates on the well off back the 1960 levels?  Why not push through immigration reform, including fast-tracking citizenship for immigrants in good standing (most will Democratic)?  Why not reduce disproportionate federal aid and pork to Republican-friendly districts?  Why not purge the SEC of corrupt former Wall-streeters and target the wizards behind the financial instruments that led to the meltdown?  Why not break up the institutions deemed ‘too big to fail’?  Why not allow regular Americans to opt into the same healthcare plan that Congress has?

I’m no savvy career politician, but it doesn’t take a genius to see that Obama, for all his inspiration, was the wrong man for the job.  The lack of leadership, the lack of combativeness and the unwillingness to pull out all the stops have all proven fatal.  All in all, the president and the Democrats simply don’t seem to be willing to do what’s necessary to make real change happen.  Perhaps that’s a feature, or perhaps that’s a bug, but the truth of the matter is that makes them failures on two fronts: they are neither competent enough to destroy the opposition, nor are they courageous enough to get done what needs to be done.

Here is the view from lower Manhattan on a gray, blustery morning.

The whole situation seems to have disintegrated long past the point of parody, with disabled toddlers being groped by TSA agents and holiday cupcakes being confiscated as contraband, but there we are.

What makes the ‘security’ edicts of the DHS and TSA so pernicious is that while it is fairly easy to add new restrictions, it is all but impossible to discontinue existing ones.  Any sane discussion of efficacy or utility is invariably shutdown at the first mention of ’9/11′.

Aside from the casual dismissal of basic rights and common sense, air travel ‘security’ is also a ridiculously expensive endeavor, that has yielded no tangible returns.  When asked, the TSA is unable to point to a single case where its screenings have actually resulted in the apprehension of a would-be terrorist.  In the meantime, over $1 trillion has been spent on this security theater, which has been proven time and time again to be unable to catch even major security breaches (guns and explosives in carry on baggage, fake boarding passes and IDs, etc.).

The wasting of vast sums of money, the reflexive dismissal of legitimate concerns and the overall arrogance and unpleasantness of the whole arrangement leaves a very bad taste.  I’ve done my best to curtail my own air travel, and to seek alternatives wherever possible.  It doesn’t look likely that the situation will improve in the foreseeable future, but hopefully as more people vote with their feet and wallets, the alternatives will improve.

I’m not generally a ‘government has to go’ type, but the DHS and TSA are clearly examples of self-sustaining bureaucracies with no genuine value save perhaps to the people who it employs.  Accordingly, anybody who runs on a platform of ending them has my vote.

strlang is a programming language I created with the goal of making string manipulation simple and straightforward.  It is an imperative language with a minimalist syntax.  The language and its compiler were written as part of the Programming Languages and Translators (COMS 4115) course at Columbia University in Fall 2011.

Features

1. Basic data types are strings, numbers and maps (sets of key-value pairs)
2. Full-set of operators for arithmetic, string manipulation (including basic regular expressions) and map construction
3. C-like structure including functions, loops, conditionals and expressions
4. No keywords

The strlang compiler performs syntax and static-semantic checking.  Valid programs are compiled to a linearized form of C++ (only one block per function, all operands are stored in variables, loops and conditionals are simplified to labels and jumps).  The compiler itself is written in OCaml and comes with a testsuite.  It relies on the C++ STL for its underlying data-types, and the PCRE library for regular expression support.

Documentation, including a tutorial and a language reference manual, is included in the source distribution.  Building strlang requires a current OCaml distribution.  To use it, you will also need a working C++ compiler and STL installation as well as an installation of the PCRE library.

Overview: strlang-slides.pdf – 1.4MB
Download: strlang-0.1.tar.gz – 1.5MB

This photo is from last year, descending White Mountain Peak in eastern California in the middle of October.  Meanwhile it’s 50 degrees and sunny outside my window here in New York in mid-December.

One of the benefits of being on the east coast is that we do have four full seasons.  Of course as my photo suggests, you can have seasons in California too.  You just may need to drive to a different part of the state to see them.

Like many folks who began using Java around ten years ago, I found it very useful with two exceptions: it forced you to use objects for everything, and it was slow.

Since then, nothing much has changed about the object-oriented nature of Java, although some of the more annoying limitations of the early days have been more or less dealt with.

The more serious concern has generally been performance.  Java was slow.  The reason why isn’t exactly a mystery.  Java programs compile to byte-code.  C and C++ programs compile to native code.  Whereas running a compiled C program is more or less a straightforward matter of executing one instruction after another, running a compiled Java program means running the Java byte code through an interpreter on the Java virtual machine.  The interpreter has been made more clever over the years (it’s now a so-called Just-In-Time Compiler – JITC), but the basic principle remained.  Java added another level of abstraction to running programs, and abstraction is not free.

A few years ago, I converted my favorite pi calculating benchmark to Java and ran it with the then current tools (JDK 1.3 if I recall correctly).  It was slower than the native C version, but not enormously so.

Developments

Since then several things have changed.  First, Microsoft’s .NET platform has become more than an odd curiosity.  Like Java, .NET is basically a large runtime library packaged with a virtual machine (CLR) and its own bytecode format (called CIL).  Interestingly, Microsoft encourages not only users of their C# Java clone, but also users of C/C++ to target this platform.  In other words, your C++ programs can now also be compiled to run in a virtual machine.

Meanwhile, Sun (now Oracle) and all the C/C++ compiler projects have continued to work on improving performance.  Intel in particular is on roll, seemingly adding a dozen new specialized instructions every year to their microprocessors, and then figuring out clever ways their compilers can take advantage of these.  SSE2, SSE3, PNI, AVX, and whatnot.

The Test

The pi_css5 benchmark uses floating point arithmetic and fast-fourier transforms to efficiently compute large numbers of digits of pi.  It makes no use of special object-oriented features, and was written originally in ANSI C.  Happily, the code is portable and with a little massaging can be used with pretty much any C++, Java or .NET compiler.  The actual test consists simply of running the program to compute 4 million digits of pi.

All measurements were performed on a 1.8GHZ Intel Core i5 processor running Windows XP.  Measurements are of time to execute (lower is better).

Setup

The Java code was compiled with the earliest Java compiler I could find (JDK 1.1.8) and run under the various JVM versions.  JDK 1.1 was released in early 2000.  The current version as of late 2011 is JDK 1.7.  Interestingly, using bytecode compiled with newer versions of the JDK made no difference in performance.

The C# code was compiled with Microsoft Visual C# 2010.  This code was run in the current 4.0 release of the .NET environment.

The C++ code was compiled with Visual C++ 2010 and Intel C++ 12.  As Visual C++ 2010 allows you to target both .NET (CLR) generating bytecode, or x86 directly by producing machine code, I tried both versions.

The Java code was also tested in the IKVM virtual machine.  IKVM is a peculiar Java Virtual Machine which executes inside the .NET virtual machine.  In other words, it translate Java bytecode to .NET bytecode aka CIL.

Observations

1) Java has made substantial performance improvements.  Specifically, the JIT in JDK 1.6 and later is within spitting distance (10%) of the optimized x86 code produced by Microsoft’s Visual C++.  Given that MSVC++ is probably the most widely used C++ compiler available, that’s quite an achievement.

2) There’s a penalty for using C# over C++ with .NET.  This is surprising considering that the bytecode ought to look quite similar (floating point addition, multiplication and whatnot, combined with occasional calls to the math library).

3) The best native compiler (Intel C++) is beating the best JITC (JVM 1.7) by around 50%.  That’s a sizable margin and more or less what I saw the last time I compared the two in late 2006.  My suspicion is that the Intel compiler is generation FP code that takes advantage of newer processor features (SSE2 vectors and whatnot) whereas the JIT is stuck using generic 80-bit x87 FP arithmetic.

4) IKVM is sort of an odd case.  It’s about 10% slower than C# on .NET.  But considering that it’s working with two levels of abstraction instead of 1, that’s actually surprisingly good.  In fact, it beats several of the older JVMs including 1.5.

Code

pi_css5_src.tgz Source code for Java, C# and C++ versions of the program.

Notes

1) JDK 1.1 and 1.2 by default don’t allocate sufficient memory to run pi_css5 for 4 million digits.  To increase stack space, I used the -mx option.  -mx131000000 (i.e. 128MB) did the trick.

2) MSVC++ 10 compiler was invoked with the /O2 /Ot /Oy /GL optimization flags for the native code (optimize for speed, omit frame pointer, perform whole program optimizations).  For .NET, I used just /O2 (the others aren’t supported when targeting CLR).

3) Intel C++ 12 compiler was invoked with the /fast optimization flag.  I wish every compiler had a -fast flag.

Of course even in the early days, processor speed correlated only loosely to actual performance.  Most computing tasks were limited by disk speed, memory speed or user input.  Moreover, processor architecture made and makes a major difference.  Intel’s ‘Prescott’ Pentium 4 design reached 3.8GHZ almost 5 years ago, while most of today’s machines have a nominal clock speed of 2.5-3.0GHZ.  Yet even for simple tasks, today’s machine are considerably faster, as Intel and AMD have made their designs more efficient.

Multi-core processors are the current vogue.  Due to thermal constraints, the traditional doubling of processor clock speeds every 18 months is a thing of the past.  So manufacturers have taken the approach of bundling 2, 4 and now 6 ‘cores’ on a single package.  The upside is that for tasks that can be parallelized, the performance gains are enormous (potentially 6x).  The downside is that many (most?) tasks don’t parallelize well, so a lot of that additional processing power is going to sit idle.

One other clever approach Intel and AMD have taken is dynamic clock-scaling.  For a long while, processors in laptop computers have had the ability to slow themselves down in order to maximize battery life, particularly when the processor isn’t very busy.  With various ‘boost’ modes, today’s processors are doing the opposite – increasing the clock speed during periods of heavy load in order to increase performance.  They can get away with this by essentially disabling other cores which aren doing anything.

My Macbook Air for example has an Intel Core i5 that nominally is 2 cores running at 1.7GHZ.  In fact, much of the time it essentially just 1 core in operation, and consequently runs at 2.7GHZ.  Since most of my tasks run on only 1 processor, it’s a beneficial tradeoff.  Moreover, Intel has actually designed it so that for short periods, the processor will exceed its official speed rating even when both cores are active (until the temperature reaches a certain threshold).

The chart of clock speed vs. time is probably going to be even less meaningful in the future, but it is more than a little amazing to remember that my first machine, less than 20 years ago, ran at only 20MHZ.

In any case, if you’re interested in finding out the actual clock speed that the processor is running at, there’s a program in Larry McVoy’s LMBench benchmark suite that can compute it (with the caveat that the program itself induces load, which can affect the clock).  The suite is fairly large and has a number of pieces, so I extracted the mhz program and disentangled the code from its dependencies..

mhz.tar.gz - C source, Windows/Mac/Linux x86 binary.