#/usr/bin env bash 

When I ran this script with "./foo.sh", I got this error message

./foo.sh: line 1: #!/usr/bin/env: No such file or directory

This drove me nuts for a while, and I finally figured out the problem.  Although the line looked just fine in vi, I ran

head -1 foo.sh | hd

And got 

00000000 ef bb bf 23 21 2f 75 73  72 2f 62 69 6e 2f 65 6e  |…#!/usr/bin/en|
00000010 76 20 62 61 73 68 0a                                                 |v bash.|
00000017

I decided that I was tired of installing all of this stuff, and dealing with the excessive memory and performance impact, and the fact that Apple's Time Machine sees every little change to the VM image as an opportunity for a full, many GB backup.  I decided to give Wine a shot.  Wine is not an emulator (as the acronym implies), and it doesn't require that you get Windows either, because it just maps (actually, it is a major miracle) the Windows API to the Mac API.  No virtual machine, no windows, no resource hog, no backup problem.  However, instead of installing Wine, I splurged and bought Codeweaver's CrossOver for Mac, which takes the Wine project and wraps it with installation usability.

I am running CrossOver with my genealogy program, RootsMagic 4, and I would call it "OK", installation was incredibly simple to initiate, given their application-specific scripts.  I had, however, to attempt the install twice because the first failed with an error.  RootsMagic runs fairly well, but the window ordering often gets confused, and I have to look behind windows or click on the program icon in the dock to get back to the active window.  In fairness, Codeweaver's compatability page gives RootsMagic a Silver rating, so they set expectations appropriately.

Overall, Crossover is good enough that I paid for it after the evaluation period.  A tip: don't but it right away and you will get an offer for 25% off the list price of $39.99.  RootsMagic gives the same discount too.

First, Cloudera offers a virtual machine based on Ubuntu with Hadoop pre-installed and configured.  It can run in either local/standalone mode or pseudo-distributed mode.  I have run it in both the VMWare player and Virtual Box on my Mac.

Second, since the rest of my class emphasizes Python, I was looking at the Python interface options as an alternative to the normally Java-based Hadoop.  I came across Dumbo, which not only provides a Python interface, but also significantly streamlines the code compared to the Java interface.  Also, it is easy to change between local/standalone mode, where debug is easier, to distributed mode, for deployment.  This makes Dumbo very attractive in educational settings.

Dumbo is super simple to install in the Cloudera VM:

wget -O ez_setup.py http://bit.ly/ezsetup

sudo python ez_setup.py dumbo

The Dumbo documentation is somewhat limited, but here is the main reference (be sure to checkout the blog link on that page for additional info).

For example, here is the canonical WordCount example written in Python/Dumbo:

def mapper(key,value):
   for word in value.split(): yield word,1
def reducer(key,values):
   yield key,sum(values)
if __name__ == "__main__":
   import dumbo
   dumbo.run(mapper,reducer)

And here is that example written in the standard Hadoop Java interface:

package org.apache.hadoop.examples;

import java.io.IOException;
import java.util.StringTokenizer;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Reducer;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
import org.apache.hadoop.util.GenericOptionsParser;

public class WordCount {

  public static class TokenizerMapper 
       extends Mapper<Object, Text, Text, IntWritable>{
    
    private final static IntWritable one = new IntWritable(1);
    private Text word = new Text();
      
    public void map(Object key, Text value, Context context
                    ) throws IOException, InterruptedException {
      StringTokenizer itr = new StringTokenizer(value.toString());
      while (itr.hasMoreTokens()) {
        word.set(itr.nextToken());
        context.write(word, one);
      }
    }
  }
  
  public static class IntSumReducer 
       extends Reducer<Text,IntWritable,Text,IntWritable> {
    private IntWritable result = new IntWritable();

    public void reduce(Text key, Iterable<IntWritable> values, 
                       Context context
                       ) throws IOException, InterruptedException {
      int sum = 0;
      for (IntWritable val : values) {
        sum += val.get();
      }
      result.set(sum);
      context.write(key, result);
    }
  }

  public static void main(String[] args) throws Exception {
    Configuration conf = new Configuration();
    String[] otherArgs = new GenericOptionsParser(conf, args).getRemainingArgs();
    if (otherArgs.length != 2) {
      System.err.println("Usage: wordcount <in> <out>");
      System.exit(2);
    }
    Job job = new Job(conf, "word count");
    job.setJarByClass(WordCount.class);
    job.setMapperClass(TokenizerMapper.class);
    job.setCombinerClass(IntSumReducer.class);
    job.setReducerClass(IntSumReducer.class);
    job.setOutputKeyClass(Text.class);
    job.setOutputValueClass(IntWritable.class);
    FileInputFormat.addInputPath(job, new Path(otherArgs[0]));
    FileOutputFormat.setOutputPath(job, new Path(otherArgs[1]));
    System.exit(job.waitForCompletion(true) ? 0 : 1);
  }
}

Upon launching VMWare Version 2, I received this message:

" 64-bit Kernel incompatibility. The version of VMware Fusion does not work with the 64 bit kernel of Mac OS X you are currently running. To use VMware Fusion, you need to restart your computer using the 32-bit kernel of Mac OS X.   Quit VMware Fusion.  "

However, the message doesn't say how to launch lion in 32-bit mode.  It turns out that if you restart Lion and hold down the "3" and "2" keys while it is restarting, you are then in 32-bit mode, and VMWare Fusion launches just fine.

In general, I like to do a scorched Earth clean install to clean out cruft and avoid the traditional bugs that occur with an OS install done in place, but I decided to try the in place route.  After backing up with SuperDuper!, and burning a DVD of the Lion installation image (this will probably be handy some day), I installed Lion and it was ready in about a 1/2 hour.  No problems, although if you do Utilities -> System Information -> Software -> Applications, you can see PowerPC apps that won't work anymore (no Rosetta in Lion).  I didn't care about any of them anymore.  I also abandoned Quicken last year, so I didn't have that problem to deal with.

I suffered from mouse dyslexia for a day (because Lion reverses the convention for scroll direction to become more consistent with the touch devices), and then decided to get a Magic Trackpad, which gives me much more function than my Magic Mouse.  I am still don't know if the Natural Scrolling is really natural, but I will give it some more time.

I found a great article (more like a book) about Lion, both the exterior features as well as the internals, including commentary, written by John Siracusa at Ars Technica.  It was such a good source of info about security issues, Objective-C memory management strategies, recovery partitions, whole disk encryption, and other meaty issues that I splurged and got the eBook version from Amazon for $4.99.

The results: Spotify has 86/100 albums, and 76/100 are available in their entirety.  Pretty good results, but room for improvement.

I have been very impressed with the ease-of-use of the Squarespace creation and management tools.  It only took me a few hours to learn the basics.  The only major hang-up was figuring out how to use a unique style on certain pages (you can do it, but it will cost extra, so I didn't).  Also, it took me a while to understand that most widgets can't be placed unless you pick a style with at least one sidebar (sometimes also confusingly called a navigation area).  My Quantum News RSS feed didn't work initially, but I sent a note to support and it magically works now.  I really like having the site stats integrated as part of the same hosting environment.

Now, we will see how long I maintain the posting momentum this time!