Sunday, December 30, 2007

Scala the statically typed dynamic language

There's been a lot of chatter lately about static vs. dynamic languages, code size, code density, etc. Many people lump Scala into the statically typed bucket because Scala has static typing. But what most folks don't realize is that Scala has most of the flexibility and syntactic economy of languages like Ruby and Python while supporting compiler checking of types to flag errors.

You may be confused. Over the next few postings, I'm going to demonstrate that most of the safe (taking input off the wire and doing method lookup on that input is not safe) constructs that dynamically typed languages have are available in Scala.

But first, you have to read the following:

  • If you think I'm a crappy programmer and want to make posts about my stupidity and your superiority, go someplace else. But before you do, take a look in the lift repository. Most of that code is mine. I'm not the best programmer I've ever met, but I'm not stupid or some variant.

  • I've been coding professionally for nearly 30 years. I've used virtually every popular language for commercial projects (except Lisp.) I've done nearly 2 years of Ruby. I've done 11 years of Java. I built the first real-time spreadsheet (Mesa) in a combination of Objective-C and C++. I understand, very, very intimately the difference between statically typed and dynamically typed languages. This isn't to say I can't learn more, but for those who have 2 years of Ruby under their belt after a couple of 200 level Java classes in college, don't bother flaming me.

So, one of the places that dynamic languages shine is the ability to pass an object (or a collection of objects) to a method and as long as the objects implement particular methods (e.g., name and age), the method will execute just fine. For example, a method that returns a string formatted with the name and age of each of the objects in the collection. This is particularly useful for functions that format instances of classes that have similar method naming, but not a similar base class.

Just like Ruby and Python, Scala can do this. The only thing that you have to do is declare the methods that the objects must implement as the "type" of the incoming parameters. Think of this extra "typing" as merely good documentation.

First, let's define a couple of classes:

case class Pet(name: String, age: Int, price: Int)
case class Antique(name: String, year: Int, cost: Int) {
def age = 2008 - year

In Scala, case class creates a class that can be economically instantiated (no need for 'new') and has accessors for its parameters as well as deep comparators, toString, and hashCode methods.

Next, let's create some collections of Pets and Antiques:

val pets = Pet("Elwood", 10, 25) :: Pet("Archer", 4, 600) :: Nil
val stuff = Antique("Couch", 1873, 8000) :: Antique("Vase", 1749, 2000) :: Nil

And let's define a method that takes a collection of anything with name and age methods and returns a nicely formatted string:

def nameAndAge(in: Seq[{def name: String; def age: Int}]) = =>" is "+in.age+" years old").mkString("\n")

And let's call the methods:

println("Pets: "+Ext.nameAndAge(pets))
println("Stuff: "+Ext.nameAndAge(stuff))

Cool. It works like Ruby and Python. Yeah, yeah, you have to write a little extra stuff describing the methods used. So what? You want to do this anyway because it's part of documenting your code. The 20 or so characters are characters that wind up in the HTML documentation of the method. And, you get some free tests for this method because if you try to call it with parameters that don't match, the compiler flags the problem for you. You save some typing because you can write fewer tests.

But it gets better with Scala. If you don't have a uniform naming convention (for example, Pet has a price property and Antique has a cost property.) So in Ruby and Python, you can't use the same method to format the name and price/cost of both Pet and Antique. In Scala, there are two ways to do this using some excellent mojo called "view bounds".

You can define a lightweight class that contains just name and price: NameAndPrice. Next, you define a method that takes a list of things that can be converted into a NameAndPrice (that's what the <% means.)

case class NameAndPrice(name: String, price: Int)

def nameAndPrice[T <% NameAndPrice](in: Seq[T]) = =>" price $"+in.price).mkString("\n")

The next thing to do is define a method that converts Pet or Antique into a NameAndPrice. You only need to define this method once and the method will be applied to all cases where you need to convert from Pet or Antique to NameAndPrice:

implicit def petToNaP(in: Pet) = NameAndPrice(, in.price)
implicit def stuffToNaP(in: Antique) = NameAndPrice(, in.cost)

Once again, our code compiles and runs correctly:

println("Pets (view): "+nameAndPrice(pets))
println("Stuff (view): "+nameAndPrice(stuff))

But it gets even better. We can combine these two mechanisms to allow arbitrary classes to be passed to a method and do implicit conversion if the class doesn't have the cost method but has a price method. First, let's look at a variant of the original example:

def cost[T <% {def cost: Int}](in: T) = in.cost

println("Stuff (ext & view): "+ =>" cost $"+cost(i)).mkString("\n"))

Next, let's convert something with a "price" method into something with a "cost" method:

case class CostHolder(cost: Int)

implicit def priceToCost(in: {def price: Int}) = CostHolder(in.price)

And the code compiles and does the right thing:

println("Pets (ext & view): " =>" cost $"+cost(i)).mkString("\n"))

So, Scala can do everything that Ruby and Python can do in terms of taking objects or collections of objects and "dynamically calling methods on them. But Scala does them one better with the ability to normalize method names across classes so you can use the same methods for classes with different naming conventions.

There are a couple of "points" that folks may make about this code:

  • Those "implicit" things look really dangerous. They can be, but they are also scoped so one "invites" the implicit conversions into a module with an import statement. They are far less dangerous than AOP or Ruby's meta-programming/open classes, yet they serve the same purpose.

  • There must be some performance penalty for not using classes, but just calling things by method name. Yes, there is, but the JVM HotSpot compiler mitigates this issue. The cost of one of these method calls is about 5x the cost of a regular method call... or about the same as the cost of a Ruby method call vs. a Java method call. The nice thing is you only have to use them when you need them, so the performance hit is localized rather than a global problem with all your code.

  • All this creation of objects much cost a lot garbage collection-wise. Actually, no. The JVM's generational garbage collector is optimized for situations like this and because the implicitly created objects are short-lived, they'll GC very nicely.

That's it for the first installment. In the next installment, I'll show you have to dynamically change an object dispatch table at run-time so you can create your own proto-type style classes and then change the class behavior at runtime.


Viktor said...

Good stuff Dave!

And I think it's great you left the term "Structural subtyping" out of the post! :)



Jorge Ortiz said...

If your view is on a structural subtype, you can replace the case class with an anonymous class:

implicit def priceToCost(in: {def price: Int}) = new { def cost = price }

AlBlue said...

Although I know that ... :: ... :: Nil is creating a list, I'm betting most of the people having done a few hours of a Ruby class won't.

Why not use the equivalent but slightly less punctual List(a,b) instead?

Also, whilst you can do this in Scala, arguably one could do the same in Java with interfaces, right?

David Pollak said...


I avoided List(a,b) just because the type of the collection is immaterial to the discussion and I didn't want to sidetrack on Array vs. List vs. ???.

One can do this with Interfaces, but it starts getting really messy and it's one of the, IMHO, solid points that dynamic languages have going for them. You don't have to add the INamePrice interface or have an interface for every field and have some wacky ICost with IName nonsense that COM devolved into.

Put another way, Scala is statically typed with optional duck typing... the opposite of Objective-C et. al. with their duck typing with optional static typing.

Reginald Braithwaite said...

As David pointed out, the problem with interfaces (or virtual base classes, or anything else along the same lines) is that you have to declare each class as implementing the interface.

This quickly creates a thicket of interfaces for each class. besides the verbosity, they do not add much value to the class declaration, as David points out it's the method you want to document.

IMO, interfaces associated with concrete classes should be semantically significant. I don't think "ICanHazNameAndAge" adds any value since you can already see that in the declaration :-)

Reginald Braithwaite said...


I like the post, but the title left me a little confused. You seem to be describing how Scala provides the benefits of latent or implicit typing while also guaranteeing type safety.

I like that.

When I think of "Dynamic" typing, I think of actual run-time changes to type, such as method_mising handling, singleton classes, and the oft-maligned open classes.

Ben said...


I agree with Reginald. I think the point your making is a good one, and I like Scala, but I was confused by the use of the word dynamic. I'm a Squeak hacker at the moment, and I was waiting for something like method_missing or open classes.

Still, I like the post.


David Pollak said...

Ben and Reginald - you'll get method_missing and "open classes" (changing the dispatch table) in my next post.

Paul Prescod said...

You said: "Over the next few postings, I'm going to demonstrate that most of the safe (taking input off the wire and doing method lookup on that input is not safe) constructs that dynamically typed languages have are available in Scala."

David: do you mean "safe" in a programming language sense or a security sense? Because the security of taking input off the wire and doing method lookup depends on a host of factors outside of the language. (firewalls, for a start) If we were to conflate type safety and security-safety then we'd have to discard relational databases as "unsafe" because SQL is more or less "dynamic method lookup."

Along the same lines: Java has some tools that allow you to circumvent type safety and do dynamic method lookup and calling when appropriate (e.g. when Java code is being called by a dynamic language). Does Scala have equivalents? And in particular, will you discuss dynamic loading of Scala and/or Java code?

Even strongly Java-centric systems like Eclipse and Tomcat rely on dynamic code modules which may throw type exceptions at (dynamic) link time.

Graven said...

Well, I thought I'd have to wait till es4 to play with static verification/structural typing in a live project — and it turns out I can already have it in Scala. Very cool.

Looking forward dynamic dispatch!

Paddy3118 said...

If you compare your definition of duck typing:

"So, one of the places that dynamic languages shine is the ability to pass an object (or a collection of objects) to a method and as long as the objects implement particular methods (e.g., name and age), the method will execute just fine."

With that from the
Wikipedia article
, specifically the "Comparison with other type systems", you would find that Ruby and Python, being dynamic, don't have to rely on a statically determined interface, or method signature as Scala does.

If you take a look at
this example
You will see some of the extra flexibility you get.


On the Scala "non-uniform naming scheme handling feature" then you are rigth in that Python does not do it that way. The Python developers haven't yet seen the need to explicitely support this kind of thing in the language, (maybe they haven't been introduced to Scala); but the normal way of handling this would be to try and call the first method - catch the exception due to the method not being present and in the exception handler, try the alternative.

- Paddy.
P.S. My tone is meant to be informative, supportive of Python, and certainly not denigrating.

Jeremy said...


The advantage of view bounds is that they work for any type A for which there is an implicit function from A -> B in scope (where B is the required type). The function that requires a B doesn't need to be changed to add a case for each different type A.

Thus there is no need for handling exceptions or if-then-else'ing.


binil said...

David, eagerly looking forward to the next post in this series.

David Pollak said...

And I'm eagerly awaiting writing it, but I've been sidetracked by paying clients, new business development and a couple of very charming 3 year olds. Maybe I'll get to something this weekend. :-)

binil said...

Nag nag! ;)

sneJ said...

I thought I had a handle on Scala, having read my way through most of the online tutorial, but this post made my eyes pop out — I did not realize that Scala's static typing was this flexible.

Currently my favorite/primary language is Objective-C 2, with some straying into Ruby and Python, but Scala increasingly looks like it'll give me the best of both worlds.

I just got the big fat Artima book in the mail yesterday, so it's time to practice my Scala this weekend! Thanks again for writing this.

Daya Sharma said...

Just wondering when that next post about method_missing and open classes (like in Ruby) will be ??

I am holding my breath to see that magic in scala ...

rich_morin said...

As of 10/2009, I don't see any follow-up postings that cover method_missing, open classes, or other features that I tend to expect in dynamic languages. Wassup?

Wade Arnold said...

I'm with Daya and Rich in hoping to see a post on method_missing. Coming from Rails and specifically a Active Record mindset I am having a hard time with some of the concepts in Lift. I would think an Active Record scala trait would make building apps pretty easy. Maybe there is something better that I don't know about? I can't find anything in google either other than this post!

inderjeet said...

Register asap in CodeFest'11 IT-BHU

The Codefest'11 team has set out to unleash a yet another coding extravaganza. We hope that your participation would raise the level of competition in Codefest'11
Feel free to contact:
* Mohit Bansal
* Saket Saurabh

Interesting facts said...

Thanks for sharing your info. I really appreciate your efforts and I will be waiting for your further write ups thanks once again.
html5 server| ffmpeg hosting

Kyle said...

Awesome article Dave - thanks!!