Top 5 java programmer reactions to scala

Posted by Christian on July 5, 2011 15 comments

1. What are all these f…ing square brackets?
Don’t be afraid.  They are a lot like <java generics>.  In scala-land we use the more generic term ‘type parameters’  (because it makes us sound smarter).  In Java we mostly use generics for collections, but we’re not forced to.  Scala differs from Java in that you are forced to use type parameters on collections.  It’s a good thing trust me.  Also, Scala has something called ‘type inference’ which sounds scary, but just means the compiler is really smart about figuring out types, which means you don’t have to be as explicit.  This makes using type parameters easier, and as a result they are just used more frequently in Scala.  Trust me, it’s the sh*t.

2. Why is the f…ing compiler so f…ing slow?
Yea…. suck it up and buy a few extra cores.  See above about the compiler being ‘really smart’.  You don’t get something for nothing.  Now might be a good time to upgrade to that 8-core machine you’ve been eying.  If you’re programming in Scala you can probably afford it.

3. Holy f…ing underscores
Yea_they_are_everywhere.  The first time you’ll probably encounter them is in import statements.  I.e. import org.example._  is just like import org.example.*;
They are generally used as a wild card, but they do have different meanings in different contexts.  It won’t take you long to sort it out.  After a couple months I was at peace with _.

4. WTF is => ?
Fat arrow my friend, fat arrow.
When you’re first starting to visually parse => throughout Scala code, think function arguments are on the left of =>, and the function body is on the right.  Note that a function may take no arguments too.

Quick example:

someList.sortWith{ (L,R) => L > R }

L and R are the two function arguments neatly tucked into parens, and the stuff on the right is the function body.  Go with it.

5. I thought map was a data structure but you scala f…ers are using ‘map’ as a verb.
Yup.  Forget ‘map’ as just a hash map.  In scala-land (as with other functional programming languages I’m told) map says, “I’m going to take something and change it to something else, i.e. I’m going to MAP it”.  Like, “I’m going to map that integer by multiplying it by 41″.  Or, “I’m going to MAP OVER all those integers and multiply each one by 41″.

The other day I mapped over a set of blueberries by putting them in my blender for 30 seconds.

Map that sh*t!  You’ll love it.

Experimenting with Scala 2.9 sys.process

Posted by Christian on June 17, 2011 3 comments

I’ve been wanting to experiment with scala 2.9′s new features in sys.process. I did some noodling around in the console to familiarize myself:

sys.process has implicits defined on string so you can do fun things like:

scala> "ls /tmp" !
051494e06911f
a
activemq.xml
aprxWqPUc
geplugin_introspect_install_mutex
ics27375
ics9432
...

So “ls -/tmp” is being executed in the external environment (OS X in this case) and the results are returned. There is a variation that gets you results as lines in a Stream :

scala> "ls  /tmp" lines_!
res20: Stream[String] = Stream(051494e06911f, ?)

Let’s find all .jpg files in my home dir that are bigger than, say, 150K :

scala> val jpegFiles = "find /Users/cparker -type f -name *.jpg".lines_!                                              
jpegFiles: Stream[String] = Stream(/Users/cparker/.Trash/Recovered files/TEMP_com.apple.iWork.Pages_69047_329765062_1/image1.jpg, ?)
 
scala> val largeJpegs = jpegFiles.filter{ fn => (new java.io.File(fn)).length > 150000 }
res4: scala.collection.immutable.Stream[String] = Stream(/Users/cparker/.Trash/Recovered files/TEMP_com.apple.iWork.Pages_69047_329765062_1/image1.jpg, ?)

Let’s peek at the EXIF data in these jpeg files too see the ‘Model’ in the EXIF data; presumably the device that was used to capture the image. To extract the EXIF, we’ll use : http://code.google.com/p/metadata-extractor/wiki/GettingStarted

Couple helper methods:

def extractMetadata(f:java.io.File):Option[Metadata] = {
  try {
    Some(JpegMetadataReader.readMetadata(f))
  } catch {
    case _ => None
  }
}
 
def extractFilteredTags(md:Metadata,f:String):Iterable[String] = {
  for { dir <- md.getDirectories
        tag <- dir.getTags if tag.toString.contains(f)
      } yield tag.toString
}

Now let’s filter out the tags that contain “Model”:

scala> val modelTagLists = largeJpegs.map{ new java.io.File(_) }.flatMap{ extractMetadata _ }.map{ extractFilteredTags(_,"Model") }
res8: scala.collection.immutable.Stream[Iterable[String]] = Stream(List(), ?)

Let’s use a hash map to tally the number of photos captured with each model:

val uniqueModels = new scala.collection.mutable.HashMap[String,Int]
 
for {
  tagList <- modelTagLists
  tag <- tagList
} { 
    if (uniqueModels.containsKey(tag)) {
      uniqueModels.put(tag, uniqueModels(tag) + 1)
    } else {
      uniqueModels.put(tag, 1)
    }
  }

Here are the top 10 models:

scala> uniqueModels.toList.sortWith{ (L,R) => L._2 > R._2 }.take(10).foreach(println(_))
([Exif IFD0] Model - C4100Z,C4000Z,1658)
([Exif IFD0] Model - Nikon SUPER COOLSCAN 5000 ED,759)
([Exif IFD0] Model - Canon PowerShot SD1100 IS,687)
([Canon Makernote] Canon Model ID - 37945344,571)
([Exif IFD0] Model - iPhone 3GS,217)
([Exif IFD0] Model - BlackBerry 8100 Series,217)
([ICC Profile] Device Model Description - IEC 61966-2.1 Default RGB colour space - sRGB,105)
([Exif IFD0] Model - iPhone,86)
([Exif IFD0] Model - ePhoto CL30clik!,54)
([Exif IFD0] Model - CYBERSHOT,20)

It’s like a history of digital cameras that I’ve owned. I haven’t thought about this old camera in a long time.

Good to see these features in scala. Makes me think about all sorts of little utilities I’d like to have.

Scala meets Fly Fishing

Posted by Christian on June 10, 2011 No comments

I’m planning a fly fishing trip with friends to Jackson Hole, WY. Fishing will be challenging this year because the record snowfall in the mountains nearby is causing unusually high and fast streams as the snow melts (not to mention some flooding in other places around the country).

The USGS monitors stream levels and kindly makes this available for free. I saw this as an opportunity to have some fun with Scala and ext-js. We’re starting a project using ext-js at work so I’m trying to come up to speed.

Being kind of an open-data junkie, I’ve used a lot of different graphing packages and visualization libraries. Ext-js’s is pretty sweet. I love the idea of only having to write JSON and a little javascript to be able to produce great looking graphs.

There wasn’t much scala to this… just translating from their XML to JSON, and doing a little caching. I ended up using grizzly for an embedded REST server along with jersey. I just like having everything in one place.

Code is below. Try the app here. My friends and I will be monitoring throughout this summer.

package streamflow
 
import com.sun.grizzly.http.embed.GrizzlyWebServer
import com.sun.grizzly.http.servlet.ServletAdapter
import com.sun.jersey.spi.container.servlet.ServletContainer
import scala.io.Source.fromInputStream
import com.mongodb._
import scala.collection.JavaConversions._
import java.text.SimpleDateFormat
import java.io.File
import scala.actors.Actor
import scala.actors.Futures._
import java.util.Date
import javax.servlet.ServletRequest
import javax.ws.rs.core.{UriInfo, Context, Request}
import javax.ws.rs._
import scala.io.Source._
import org.joda.time.format._
import xml.XML
import scala.collection.mutable.HashMap
import org.joda.time.{Minutes, DateTime}
 
case class CachedResult (date:DateTime, text:String)
 
object GrizzlyServer {
 
  def getURLText(url:String):String = {
    println("\n\nmaking request %s \n\n".format(url))
 
    val current = new DateTime
 
    resultCache.get(url) match {
      case Some(cr) =>
        if ( Minutes.minutesBetween(cr.date,current).getMinutes > cacheTimeoutMins && cr.text.size > 0 ) {
          try {
            val text = fromURL(url).mkString
            resultCache.put(url,CachedResult(new DateTime,text))
            text
          } catch {
            case _ =>
              println("caught exception trying to get new text, sticking with old")
              cr.text
          }
        } else {
          println("returning cached result")
          cr.text
        }
      case None =>
        try {
          val text = fromURL(url).mkString
          resultCache.put(url,CachedResult(new DateTime,text))
          text
        } catch {
          case _ =>
            println("ERROR, won't cache")
            ""
        }
    }
  }
 
  val resultCache:HashMap[String,CachedResult] = HashMap.empty
 
  val cacheTimeoutMins = 60
 
  def startGriz() {
    val port = 9191
    val baseUri = "http://localhost:%s/".format(port)
    val gws = new GrizzlyWebServer(port)
 
    val jerseyAdapter = new ServletAdapter
    jerseyAdapter.addInitParameter("com.sun.jersey.config.property.packages", "streamflow")
    jerseyAdapter.setContextPath("/")
    jerseyAdapter.setHandleStaticResources(true)
    jerseyAdapter.setServletInstance(new ServletContainer)
 
    gws.addGrizzlyAdapter(jerseyAdapter, Array("/"))
 
    println("starting grizzly")
    gws.start()
  }
}
 
 
@Path("streamflow")
case class StreamFlow {
 
 
  @Path("/stationDetails/{stationId}")
  @GET
  def getStationDetails(@PathParam("stationId") stationId:String):String = {
    val usgsAPITemplate = "http://interim.waterservices.usgs.gov/NWISQuery/GetDV1?SiteNum=%s&ParameterCode=00060&StatisticCode=00003&StartDate=2011-01-01&EndDate=2011-01-05"
    val xmlDoc = XML.loadString( GrizzlyServer.getURLText(usgsAPITemplate.format(stationId)) )
    val siteName = (xmlDoc \\ "siteName").text
    val lat = (xmlDoc \\ "geogLocation" \ "latitude").text
    val lng = (xmlDoc \\ "geogLocation" \ "longitude").text
 
    """ { "stationName" : "%s",
        "lat" : %s,
        "lng" : %s }
    """.format(siteName,lat,lng)
  }
 
 
  @Path("/compare")
  @GET
  def compare(
    @DefaultValue("30") @QueryParam("priorDays") priorDays:Int,
    @QueryParam("stationId") stationId:String
  ) = {
 
    // call USGS API
    val usgsAPITemplate = "http://interim.waterservices.usgs.gov/NWISQuery/GetDV1?SiteNum=%s&ParameterCode=00060&StatisticCode=00003&StartDate=%s&EndDate=%s"
 
    val thisYearToday = new DateTime
    val thisYearPrior = thisYearToday.minusDays(priorDays)
    val lastYearToday = thisYearToday.minusYears(1)
    val lastYearPrior = lastYearToday.minusDays(priorDays)
 
    val dtf = DateTimeFormat.forPattern("yyyy-MM-dd")
    val monthDayDtf = DateTimeFormat.forPattern("MM-dd")
    val sourceDtf = DateTimeFormat.forPattern("yyyy-MM-dd'T'HH:mm:ss")
 
    def simplifyDate(ds:String) =  monthDayDtf.print(sourceDtf.parseDateTime(ds))
 
    // THIS YEAR
    val xmlText = GrizzlyServer.getURLText(usgsAPITemplate.format(stationId,dtf.print(thisYearPrior), dtf.print(thisYearToday)))
    println("\n\nRECEIVED " + xmlText)
    val xmlDoc = XML.loadString(xmlText)
    val rawData1 = (xmlDoc \\ "value").map{ x => (x.attribute("dateTime") match { case Some(d) => simplifyDate(d.text); case _ => "" }, x.text) }
    val siteName = (xmlDoc \\ "siteName").text
 
    // LAST YEAR
    val xmlText2 = GrizzlyServer.getURLText(usgsAPITemplate.format(stationId,dtf.print(lastYearPrior), dtf.print(lastYearToday)))
    println("\n\nRECEIVED2 " + xmlText2)
    val xmlDoc2 = XML.loadString(xmlText2)
    val rawData2 = (xmlDoc2 \\ "value").map{ x => (x.attribute("dateTime") match { case Some(d) => simplifyDate(d.text); case _ => "" }, x.text) }
 
    val combinedData = rawData1.zip(rawData2)
    println("raw combined " + combinedData)
 
    val jsonTemplate = """ { "date" : "%s", "currentYearFlow" : "%s", "lastYearFlow" : "%s" } """
    val jsonResult = combinedData.map{ raw =>
      val ((currentYearDate,currentYearFlow),(lastYearDate,lastYearFlow)) = raw
      jsonTemplate.format(currentYearDate,currentYearFlow,lastYearFlow) + ","
    }
    """{ "data" : [ %s ], "siteName" : "%s" } """.format(jsonResult.mkString.init,siteName)
  }
}
 
 
 
object StreamFlowApp extends App {
  println("starting streamflow")
  GrizzlyServer.startGriz()
}

And the ext-js code:

Ext.require('Ext.chart.*');
Ext.require(['Ext.Window', 'Ext.fx.target.Sprite', 'Ext.layout.container.Fit']);
 
Ext.onReady(function () {
 
 
    var startingStationId = '13013650';
 
 
    Ext.define('StreamFlowCompare', {
        extend: 'Ext.data.Model',
        fields: [
            {name: 'date', type: 'string'},
            {name: 'currentYearFlow',  type: 'int'},
            {name: 'lastYearFlow',  type: 'int'}
        ]
    });
 
    var getStreamProxy = function(stationId) {
        return new Ext.data.proxy.Ajax({
            type: 'ajax',
            url : 'http://localhost:8080/streamflow/compare?priorDays=30&stationId='+stationId,
            reader: {
                type: 'json',
                root: 'data'
            }
        });
    }
 
 
    var remoteStore = new Ext.data.Store({
        model: 'StreamFlowCompare',
        proxy: getStreamProxy(startingStationId),
        autoLoad: true
    });
 
 
    Ext.define('StationDetails', {
        extend: 'Ext.data.Model',
        fields : [
            {name : 'stationName', type : 'string'},
            {name : 'lat', type : 'float'},
            {name : 'lng', type : 'float'}
        ]
    });
 
    var getStationDetailsProxy = function(stationId) {
        return new Ext.data.proxy.Ajax({
            type: 'ajax',
            url : 'http://localhost:8080/streamflow/stationDetails/' + stationId,
            reader: {
                type: 'json'
            }
        });
    }
 
    var stationDetailsStore = Ext.create('Ext.data.JsonStore', {
        model : 'StationDetails',
        proxy : getStationDetailsProxy(startingStationId)
    });
 
    var localStore = Ext.create('Ext.data.JsonStore', {
        fields: ['date', 'currentYearFlow', 'lastYearFlow'],
        data: [
            {'date':'metric a', 'currentYearFlow':5, 'lastYearFlow':12},
            {'date':'metric b', 'currentYearFlow':51, 'lastYearFlow':15},
            {'date':'metric c', 'currentYearFlow':24, 'lastYearFlow':52},
            {'date':'metric d', 'currentYearFlow':22, 'lastYearFlow':32},
            {'date':'metric e', 'currentYearFlow':10, 'lastYearFlow':72},
            {'date':'metric f', 'currentYearFlow':17, 'lastYearFlow':92},
            {'date':'metric g', 'currentYearFlow':22, 'lastYearFlow':32},
            {'date':'metric h', 'currentYearFlow':25, 'lastYearFlow':22},
            {'date':'metric i', 'currentYearFlow':42, 'lastYearFlow':52},
            {'date':'metric j', 'currentYearFlow':55, 'lastYearFlow':42},
            {'date':'metric k', 'currentYearFlow':89, 'lastYearFlow':82},
            {'date':'metric l', 'currentYearFlow':65, 'lastYearFlow':42},
            {'date':'metric m', 'currentYearFlow':44, 'lastYearFlow':12}
        ]
    });
 
 
    var rivers = Ext.create('Ext.data.Store', {
    fields: ['stationId', 'name'],
    data : [
        {"stationId":"13013650", "name":"Snake (near Moose)"},
        {"stationId":"13023000", "name":"Greys"},
        {"stationId":"13014500", "name":"Gros Ventre"},
        {"stationId":"13018350", "name":"Flat Creek"},
        {"stationId":"13016450", "name":"Fish Creek (near Wilson)"},
        {"stationId":"13027500", "name":"Salt River"}
    ]
    });
 
 
   var chart = Ext.create('Ext.chart.Chart',  {
            xtype: 'chart',
            style: 'background:#fff',
            animate: true,
            store: remoteStore,
            shadow: true,
            theme: 'Category1',
            legend: {
                position: 'right'
            },
            axes: [{
                type: 'Numeric',
                minimum: 0,
                position: 'left',
                fields: ['currentYearFlow', 'lastYearFlow'],
                title: 'Stream flow in CFM',
                minorTickSteps: 1,
                grid: {
                    odd: {
                        opacity: 1,
                        fill: '#ddd',
                        stroke: '#bbb',
                        'stroke-width': 0.5
                    }
                }
            }, {
                type: 'Category',
                position: 'bottom',
                fields: ['date'],
                title: 'Date (month-day)',
                calculateCategoryCount:true,
                grid:false,
                label: {
                  rotate: { degrees: 70 }
                }
            }],
            series: [{
                type: 'line',
                highlight: {
                    size: 1,
                    radius: 1
                },
                axis: 'left',
                smooth: false,
                fill: true,
                xField: '',
                yField: 'currentYearFlow',
                markerConfig: {
                    type: 'circle',
                    size: 0,
                    radius: 0,
                    'stroke-width': 0
                }
            },{
                type: 'line',
                highlight: {
                    size: 1,
                    radius: 1
                },
                axis: 'left',
                smooth: false,
                fill: true,
                xField: '',
                yField: 'lastYearFlow',
                markerConfig: {
                    type: 'circle',
                    size: 0,
                    radius: 0,
                    'stroke-width': 0
                }
            }]
   }
   );
 
    var remoteStoreLoadFunc = function(records,operation,success) {
        console.log('loaded records');
        chart.setLoading(false);
    }
 
    var winTitle = 'River flow rate 2010 vs. 2011  station: ';
 
    var stationDetailsLoadFunc = function(records,operation,success) {
      var r = records[0].data;
      win.setTitle(winTitle +  r.stationName + '  location:  ' + r.lat + ', ' + r.lng);
    }
 
    var win = Ext.create('Ext.Window', {
        width: 800,
        height: 600,
        hidden: false,
        maximizable: true,
        title: 'River flow rate comparison this year vs. last year : ',
        renderTo: Ext.getBody(),
        layout: 'fit',
        tbar: [{
            xtype: 'combo',
            fieldLabel: 'Select Stream',
            store: rivers,
            queryMode: 'local',
            displayField: 'name',
            valueField: 'stationId',
            value:startingStationId,
            allowBlank:false,
            autoSelect:true,
            listeners:{
                'select': function(field, value) {
                    chart.setLoading(true);
                    console.log('selected ' + value[0].data.stationId);
                    remoteStore.setProxy(getStreamProxy(value[0].data.stationId));
                    remoteStore.load(remoteStoreLoadFunc);
 
                    stationDetailsStore.setProxy(getStationDetailsProxy(value[0].data.stationId))
                    stationDetailsStore.load(stationDetailsLoadFunc)
                }
            }
        }
        ],
        items: [chart]
    });
 
    stationDetailsStore.setProxy(getStationDetailsProxy(startingStationId))
    stationDetailsStore.load(stationDetailsLoadFunc)
});

Murmurings from Scaladays

Posted by Christian on June 3, 2011 No comments

Great to see things like this coming from scaladays. http://engineering.foursquare.com/2011/01/21/rogue-a-type-safe-scala-dsl-for-querying-mongodb/

Will try harder to make it out next year.

Observer Pattern with Actors

Posted by Christian on May 31, 2011 No comments

I had a sleepless night recently (It’s been windy here in Colorado). While trying to get to sleep I was thinking about refactoring a recent code kata to use Actors for its Observer / Observable mechanism.

I came up with a couple simple traits, ActObserver and ActObservable. First, ActObserver:

trait ActObserver extends Actor {
  protected def handleNotify(o: ActObservable)
  def act() = loop {
    react {
      case Notify(x) =>
        println("we are being notified with " + x)
        handleNotify(x)
      case x =>
        println("received an unknown message " + x)
    }
  }
  start()
}

Simple stuff. Just an abstract method leaving implementers to handle the notify. Then the observable:

trait ActObservable extends Actor {
  private val observers: ListBuffer[ActObserver] = new ListBuffer[ActObserver]()
 
  protected def notifyObservers() {
    observers.foreach { o =>
      println("sending notify to " + o)
      o ! Notify(this)}
  }
 
  private def addObserver(o:ActObserver) = {
    observers += o
    println("added observer %s to %s".format(o,this))
  }
 
  private def removeObserver(o:ActObserver) = observers -= o
 
  def act() = loop {
    react {
      case AddObserver(o) =>
        addObserver(o)
        println("adding observer " + o)
      case RemoveObserver(o) =>
        removeObserver(o)
        println("removing observer " + o)
    }
  }
  start()
}

The Observable is even cleaner since there is nothing really specific to a particular observer.

Some client code to exercise the thing:

object ActorListener extends App {
 
  val t1 = new Table(1)
  val t2 = new Table(2)
 
  val w1 = new Waitress("Wilma")
  val w2 = new Waitress("Betty")
 
  val k = new Kitchen
 
  t1 ! AddObserver(w1)
  t2 ! AddObserver(w2)
 
  k ! AddObserver(w1)
  k ! AddObserver(w2)
 
  // wait for messages to catch up
  Thread.sleep(1000)
 
  t1.checkGuests
  t2.checkGuests
 
  k.checkOrders
}

Output:

added observer Waitress(Wilma) to Kitchen()
added observer Waitress(Betty) to Table(2)
adding observer Waitress(Wilma)
added observer Waitress(Wilma) to Table(1)
adding observer Waitress(Wilma)
adding observer Waitress(Betty)
added observer Waitress(Betty) to Kitchen()
adding observer Waitress(Betty)
sending notify to Waitress(Wilma)
sending notify to Waitress(Betty)
sending notify to Waitress(Wilma)
sending notify to Waitress(Betty)
we are being notified with Table(1)
we are being notified with Table(2)
waitress Wilma : table Table(1) needs attention
waitress Betty : table Table(2) needs attention
we are being notified with Kitchen()
we are being notified with Kitchen()
waitress Wilma : the kitchen needs attention Kitchen()
waitress Betty : the kitchen needs attention Kitchen()

Seems to work OK. I wrestled a bit with adding type parameters to the traits (i.e. ActObserver[Waitress], ActObservable[Table]. The problem I ran into was that this seems to make it difficult for an observer to observe things of different types. It seems easiest for the observer to pattern match on any observable message received.

Let me know if you think otherwise. Thanks!

Combining partial functions with fold

Posted by Christian on May 23, 2011 5 comments

This pattern occurred to me while working on some code for an actor’s react {} method.  This is probably obvious to scala veterans but I patted myself on the back after sorting this one out.

So an actor’s react method requires a list of functions from Any to Unit.  First, I setup a type alias for PartialFunction[Any,Unit] as that’s a long one to keep typing:

type AnyToUnit = PartialFunction[Any,Unit]

Then define a list of partial functions:

val f1: AnyToUnit = { case i:Int => println("you have an int") }
val f2: AnyToUnit = { case f:Float => println("you have an float") }
val f3: AnyToUnit = { case b:Boolean => println("you have an boolean") }
val f4: AnyToUnit = { case s:String => println("you have a string") }
val funcList = List(f1,f2,f3,f4)

Now fold the tail into the head using orElse :

val combined = funcList.tail.foldLeft(funcList.head) { (functions,f) => functions orElse f }

Trying it out:

scala> combined(true)you have an boolean
scala> combined(1)you have an int
scala> combined(45.5f)you have an float
scala> combined("hello")you have a string

No need to chain a bunch of orElses together.

Video Games With Processing

Posted by Christian on May 20, 2011 No comments

Last winter, as a type of ‘code kata’, I got into writing cheesy video games for my kids using scala.  I wanted to write more scala and learn processing so it seemed like a good idea.

I took me a few days to get a pattern down and once I did it was pretty easy to turn out simple games.  My kids loved the idea that they could describe a game and I could have it for them the next day.

Processing turns out to be pretty simple to use.  It feels a lot like writing an applet, because you extend a PApplet, which extends Applet.  There is a draw method, which gets called n times per second to update the display, and you have methods for drawing simple shapes and displaying images and such.

In my games, each shape has a current x,y position (just 2d here folks), along with a speed and direction.  So with each frame you just calculate a new position based on speed and direction.  Pretty basic stuff.

I don’t know if we’re feeding our kids too much cereal or what but they come up with some crazy sh*t for ideas.  My daughter’s idea was “… a beaver chases a tiger around and throws apples at him…”.  OK.  My son wanted to play space invaders where his teacher was one of the invaders.  Sure.

The images are mostly gathered from google searches (my complements to their original authors).  My son composed his own music for ‘face invaders’.

I’ll spare you all the code (ping me if you’re really curious) as it’s a bit long for a gist (around 500 lines).  Here is what they ended up looking like:

 

 

They are available to play as Web Start applications:

play Beaver vs Tiger

play Face Invaders

Let me know what you get for a high score ;)

Jersey vs Spring MVC with Scala

Posted by Christian on May 12, 2011 No comments

In need of a REST API on my current project, we recently reached for Spring MVC’s latest version with their newly added ‘full-scale REST support’.  I’ve had good luck with Spring MVC in the past (and Spring in general of cos).

We ended up with methods like this:

@RequestMapping(value=Array("/hello"), method=Array(RequestMethod.GET))
@ResponseBody
def getItems(
   @RequestParam(value = "start", required=false) start:java.lang.Integer,
   @RequestParam(value = "limit", required=false) limit:java.lang.Integer):RESTResponse = {
   ...
}

view raw gistfile1.scala This Gist brought to you by GitHub.

We want callers of the API to specify optional parameters for start and limit, and we’ll translate those to the appropriate database query.  If start and limit are unspecified we want to return the first N items where N is some reasonable limit (typical stuff).

Unfortunately Spring MVC insists on passing null when non-required request parameters are left empty.  Using java types is therefore required.  For example, if you try to use scala Int you get this nasty exception :

java.lang.IllegalStateException: Optional int parameter 'start' is not present 
but cannot be translated into a null value due to being declared as a primitive type. 
Consider declaring it as object wrapper for the corresponding primitive type.
view raw gistfile1.txt This Gist brought to you by GitHub.

It’s not too terrible to have code like:

Option[Int](limit) match {
  case Some(x) => ...
  case None => ...
}

view raw gistfile1.scala This Gist brought to you by GitHub.

… but I’d much rather specify defaults in the arguments to the method.  Unfortunately scala default arguments don’t work, presumably because the argument is being specified… as null.  Argh!

My attempts at implicit conversion of null to various Option types were also fruitless.  Granted I may have thrown in the towel a little soon on this front… but any solution along these lines feels dirty.

Enter Jersey.

The same method, with Jersey looks like this (with some additions):

@GET
@Produces(Array(MediaType.APPLICATION_JSON))
def getItems(
  @DefaultValue("0") @QueryParam("start") start: Int,
  @DefaultValue("25") @QueryParam("limit") limit:Int,
  @DefaultValue("[]") @QueryParam("sort") sort: JSONReceiver[Array[Sort]],
  @DefaultValue("[]") @QueryParam("filter") filter: JSONReceiver[Array[Filter]]): RESTResponse = { ...

view raw gistfile1.scala This Gist brought to you by GitHub.

That’s a bit more like it… simple default value annotations.

The JSONReceiver stuff has to do with converting to JSON with Jackson (future post).

I like this much better because:
• We’re able to use Scala types
• No manual option conversion code

Some plumbing tips with Jersey: if you want your controller classes to be spring components you’ll want to use the JerseySpringServlet, which is aware of your spring beans.  In your web.xml file this looks like:

<servlet>
  <servlet-name>Jersey Spring Web Application</servlet-name>
  <servlet-class>com.sun.jersey.spi.spring.container.servlet.SpringServlet</servlet-class>

  <init-param>
    <param-name>javax.ws.rs.Application</param-name>
    <param-value>services.Application</param-value>
  </init-param>

  <init-param>
    <param-name>com.sun.jersey.api.json.POJOMappingFeature</param-name>
    <param-value>true</param-value>
  </init-param>
</servlet>
view raw gistfile1.xml This Gist brought to you by GitHub.

This uses the ‘application style’ of bootstrapping your Jersey stuff… i.e. you point it to a class that extends something like PackagesResourceConfig, or similar… this is well documented.

So we don’t want Spring MVC, but we want the rest of Spring 3… this causes some dependency problems.  The maven dependency section probably describes it best:

<dependency>
  <groupId>com.sun.jersey.contribs</groupId>
  <artifactId>jersey-spring</artifactId>
  <version>1.7-ea04</version>
  <exclusions>
    <exclusion>
      <groupId>org.springframework</groupId>
      <artifactId>spring-aop</artifactId>
    </exclusion>

    <exclusion>
      <groupId>org.springframework</groupId>
      <artifactId>spring-beans</artifactId>
    </exclusion>
    
    <exclusion>
      <groupId>org.springframework</groupId>
      <artifactId>spring-context</artifactId>
    </exclusion>

    <exclusion>
      <groupId>org.springframework</groupId>
      <artifactId>spring-web</artifactId>
    </exclusion>
  </exclusions>
</dependency>

view raw gistfile1.xml This Gist brought to you by GitHub.

The jersey-spring module, which contains the jersey-spring servlet, depends on spring 2, which conflicts with spring 3.  Luckily the spring 3 modules are backwards compatible, so excluding them here and explicitly including spring 3 elsewhere in the pom works fine.

So we’re using Jersey for the time being.  Let me know anyone has a different experience with Scala and Spring MVC.

Life, part 2

Posted by Christian on May 10, 2011 No comments

In a previous post I was talking about building the simplest possible version of Conway’s game of Life.  It works but is pretty slow.

I added one optimization which made a pretty big difference.  I was naively considering each ‘cell’ of the ‘board’ in each turn.  In my most recent update, I just take all the live cells, compute the ‘dead neighbors’ and only consider that set of cells.  Pretty simple change, but I’m easily getting 30 ticks a second now.

I was able to add more cells to the board.  I also added some slight randomness to the live cell colors just for fun.

Cells also aren’t allowed to grow off the edges of the board.

This morning I think I’ll add the ability to read in starting cell positions from JSON so I can try different patterns without recompiling.

Addictive little thing.

New code for the core game engine is here.  The visualization hasn’t changed enough to make it worth reposting.

package life


trait Status

case object ALIVE extends Status {
  override def toString = "ALIVE"
}

case object DEAD extends Status {
  override def toString = "DEAD"
}

case class Point(x: Int, y: Int)

case class Cell(p: Point, live: Status, inPlay: Boolean = false) {

    override def equals(obj: Any) = obj match {
    case c: Cell => this.p == c.p
    case _ => false
  }

  override def hashCode() = p.hashCode
}

case class Board(cells: Set[Cell],xLimit:Int = 50, yLimit:Int = 50) {

  def neighborPoints(p: Point): List[Point] = {
    List(
      Point(p.x, p.y - 1),
      Point(p.x, p.y + 1),
      Point(p.x - 1, p.y - 1),
      Point(p.x - 1, p.y),
      Point(p.x - 1, p.y + 1),
      Point(p.x + 1, p.y - 1),
      Point(p.x + 1, p.y),
      Point(p.x + 1, p.y + 1)
    )
  }

  def neighborCells(point: Point): List[Cell] = {
    neighborPoints(point).flatMap {
      neighborPoint =>
        cells.filter(c => c.p.x == neighborPoint.x && c.p.y == neighborPoint.y).headOption match {
          case Some(c:Cell) => Some(c)
          case None => if (
                            neighborPoint.x > 0 &&
                            neighborPoint.y > 0 &&
                            neighborPoint.x <= xLimit &&
                            neighborPoint.y <= yLimit)
                       Some(Cell(neighborPoint,DEAD)) else
                       None
        }
    }
  }

  def deadNeighbors(point: Point) = neighborCells(point).filter(_.live == DEAD)

  def liveNeighborCounts(point: Point): Int = neighborCells(point).foldLeft(0) {
    (acc, p) => acc + (if (p.live == ALIVE) 1 else 0)
  }

  def turn: Board = {

    val deadNeighborCells = cells.filter(_.live == ALIVE).flatMap {
      x => deadNeighbors(x.p)
    }

    val newCells = (deadNeighborCells -- cells ++ cells).flatMap {
      cell =>
        (cell.live, liveNeighborCounts(cell.p)) match {
          case (ALIVE, n) if (n <= 1) => None
          case (ALIVE, n) if (n >= 2 && n <= 3) => Some(cell) // stay alive
          case (ALIVE, n) if (n > 3) => None
          case (DEAD, n) if (n == 3) => Some(cell.copy(live = ALIVE))
          case _ => None
        }
    }

    //println("new cells " + newCells.size)
    this.copy(cells = newCells)
  }

  override def toString = {
    cells.filter(_.live == ALIVE).foldLeft("") {
      (acc, cell) => acc + "%s,%s:%s\n".format(cell.p.x, cell.p.y, cell.live)
    }
  }
}


object Life {

  def main(args: Array[String]) {

    val fCells = Set(
      Cell(Point(15, 15), ALIVE, true),
      Cell(Point(14, 15), ALIVE, true),
      Cell(Point(14, 16), ALIVE, true),
      Cell(Point(14, 17), ALIVE, true),
      Cell(Point(13, 16), ALIVE, true)
    )

    var board: Board = Board(fCells)

    println("starting board : %s".format(board))

    println("cell 2,1 neighbor counts " + board.liveNeighborCounts(Point(2, 1)))
    println("cell 1,1 neighbor counts " + board.liveNeighborCounts(Point(1, 1)))
    println("cell 3,1 neighbor counts " + board.liveNeighborCounts(Point(3, 1)))

    while (true) {
      board = board.turn
    }
  }
}

view raw gistfile1.scala This Gist brought to you by GitHub.

Discovering Conway’s Game of Life

Posted by Christian on May 9, 2011 3 comments

I didn’t know what ‘Life’ was until a few months ago.

I first heard it mentioned by a co-worker.   Then just a few weeks ago at the ’2-day scala meetup’ it came up again as the subject of a code retreat. If you’ve never heard of it, I’ll spare you an explanation and just send you here, where you’ll find an excellent definition and examples.

It’s deceptively simple.  After reading the basic rules I was able to have a nieve implementation working with visualization in about an hour.  If you haven’t played with processing yet, I highly recommend it if you like hacking data visualizations.

It’s amazing to me how dynamic the behavior can be given the very simple rules of the ‘game’.  My kids love watching.

The real education, I suppose, comes in optimizing the code to crank out generations as fast as possible.  My Scala implementation, which is is zero optimized, only gets about 5 generations per second.

This week in Boulder there is a meetup dedicated to working on Life where I’ll hopefully get some tips.

Here’s what I have so far :

The game engine:

package life


trait Status

case object ALIVE extends Status {
  override def toString = "ALIVE"
}

case object DEAD extends Status {
  override def toString = "DEAD"
}

case class Point(x: Int, y: Int)

case class Cell(p: Point, live: Status, inPlay:Boolean=false) {

  override def equals(obj: Any) = obj match {
    case c: Cell => this.p == c.p
    case _ => false
  }

  override def hashCode() = p.hashCode
}

case class Board(cells: Set[Cell]) {

  def neighborPoints(p: Point): List[Point] = {
    List(
      Point(p.x, p.y - 1),
      Point(p.x, p.y + 1),
      Point(p.x - 1, p.y - 1),
      Point(p.x - 1, p.y),
      Point(p.x - 1, p.y + 1),
      Point(p.x + 1, p.y - 1),
      Point(p.x + 1, p.y),
      Point(p.x + 1, p.y + 1)
    )
  }

  def neighborCells(point: Point): List[Cell] = {
    neighborPoints(point).flatMap{ neighborPoint =>
        cells.filter(c => c.p == neighborPoint).headOption
    }
  }

  def liveNeighborCounts(point: Point): Int = neighborCells(point).foldLeft(0) {
    (acc, p) => acc + (if (p.live == ALIVE) 1 else 0)
  }

  def turn: Board = {


    val newCells = cells.map {
      cell =>
        (cell.live, liveNeighborCounts(cell.p)) match {
          case (ALIVE, n) if (n <= 1) => cell.copy(live = DEAD) // death from under population
          case (ALIVE, n) if (n >= 2 && n <= 3) => cell // stay alive
          case (ALIVE, n) if (n > 3) => cell.copy(live = DEAD) // death from over population
          case (DEAD, n) if (n == 3) => cell.copy(live = ALIVE) // reborn
          case _ => cell
        }
    }
    Board(newCells)
  }

  override def toString = {
    cells.filter(_.live == ALIVE).foldLeft("") {
      (acc, cell) => acc + "%s,%s:%s\n".format(cell.p.x, cell.p.y, cell.live)
    }
  }
}


object Life {

  def main(args: Array[String]) {

    val fCells = Set(
      Cell(Point(15, 15), ALIVE,true),
      Cell(Point(14, 15), ALIVE,true),
      Cell(Point(14, 16), ALIVE,true),
      Cell(Point(14, 17), ALIVE,true),
      Cell(Point(13, 16), ALIVE,true)
    )

    var board: Board = Board(fCells)

    val inPlayCells = finalCells.filter(_.live == ALIVE).flatMap{ cell =>
      board.neighborCells(cell.p)
    }.map{cell => cell.copy(inPlay = true)} ++ fCells

    println("starting board : %s".format(board))

    println("cell 2,1 neighbor counts " + board.liveNeighborCounts(Point(2, 1)))
    println("cell 1,1 neighbor counts " + board.liveNeighborCounts(Point(1, 1)))
    println("cell 3,1 neighbor counts " + board.liveNeighborCounts(Point(3, 1)))

    while(true) {
      board = board.turn
    }
  }

}

view raw gistfile1.scala This Gist brought to you by GitHub.

and the processing visualization:

package life

import java.awt._
import processing.core.PApplet._
import processing.core.PConstants._
import processing.core.PApplet
import VisualLifeMain.{windowHeight,windowWidth,board}

class VisualLifeFrame extends Frame("Life") {
  setLayout(new BorderLayout())
  val embed = new LifeApplet()
  add(embed, BorderLayout.CENTER)
  embed.init()
  pack
  setLocation(100, 100)
  setVisible(true)
}


object VisualLifeMain {
  val windowWidth = 850
  val windowHeight = 680
  var board:Board = Board(Set.empty)

  def main(args: Array[String]) {
    println("starting visual life")

    val fullCells =
      for {
        i <- (1 to 35)
        j <- (1 to 35)} yield Cell(Point(i, j), DEAD)


    val liveCells = Set(
      Cell(Point(1, 3), ALIVE,true),
      Cell(Point(2, 3), ALIVE,true),
      Cell(Point(3, 3), ALIVE,true),
      Cell(Point(3, 2), ALIVE,true),
      Cell(Point(2, 1), ALIVE,true)
    )

    val fCells = Set(
      Cell(Point(15, 15), ALIVE,true),
      Cell(Point(14, 15), ALIVE,true),
      Cell(Point(14, 16), ALIVE,true),
      Cell(Point(14, 17), ALIVE,true),
      Cell(Point(13, 16), ALIVE,true)
    )

    val rnd = new java.util.Random
    val randCells = (1 to 200).map { n =>
      Cell( Point(rnd.nextInt(35)+1,rnd.nextInt(35) + 1), ALIVE,true)
    }

    val finalCells = (fullCells.toSet -- randCells) ++ randCells
    println("total cell count " + finalCells.size)
    println("alive cells " + finalCells.filter(_.live == ALIVE).size)

    board = Board(finalCells)

    new VisualLifeFrame
  }
}


class LifeApplet extends PApplet {

  override def init() {
    setSize(VisualLifeMain.windowWidth, VisualLifeMain.windowHeight)
    setPreferredSize(new Dimension(VisualLifeMain.windowWidth, VisualLifeMain.windowHeight))

    super.init()
  }

  override def setup() {
    size(VisualLifeMain.windowWidth, VisualLifeMain.windowHeight)
    rectMode(CORNER)
    ellipseMode(CENTER)
    imageMode(CENTER)
    frameRate(20)
  }

  override def draw() {
    background(230.0f,0.9f)

    stroke(128f)
    strokeWeight(1.0f)
    val cellWidth =20
    for{
      x <- 1 to (windowWidth/cellWidth)
    }  { line(cellWidth*x,0,cellWidth*x,windowHeight) }

    for {
      y <- 1 to (windowHeight/cellWidth)
    } line (0,cellWidth*y, windowWidth, cellWidth*y)

    //println("drawing alive cells " + board.cells.filter(_.live == ALIVE).size)

    fill(0f,0f,128f)
    board.cells.filter(_.live == ALIVE).foreach { cell =>
      rect(cell.p.x * cellWidth, cell.p.y * cellWidth, cellWidth,cellWidth)
    }
    board = board.turn
  }

}

view raw gistfile1.scala This Gist brought to you by GitHub.