Friday, January 4, 2008

maven for scala

I'll try to provide a mini-guide about maven for scala's project. For more info, go to maven home or plugin's home.

Introduction to maven


Maven is a builder like make or ant, written in java. It's a commande line tool, IDE (Eclipse, Netbeans, IDEA) have plugins to handle and integrate project powered by maven. It could be used to create lib (jar), webapps (ear) and any other type of "artifact". It prefers convention over configuration, and configuration over instruction. What that mean exactly ?
  • every action have a default configuration (= the convention).
  • every action is a goal defined into a plugin (aka mojo), and for common case, you will try to use existing plugin instead of calling (more) low level instruction (like copy file,...)
Before creating your first scala project, You need to know some info:

a command line tool
"mvn" is the name of the command line tool to call maven 2.x. To display help, run mvn help
the project descriptor : the file [prj]/pom.xml
It's the file when every project information are stored (name, version, dependencies, license, mailing-list,...)
the build lifecycle :
The build lifecycle is defined by a sequence of phases, the main are :
  • compile - compile the source code of the project
  • test - test the compiled source code using a suitable unit testing framework. These tests should not require the code be packaged or deployed
  • package - take the compiled code and package it in its distributable format, such as a JAR.
  • integration-test - process and deploy the package if necessary into an environment where integration tests can be run
  • install - install the package into the local repository, for use as a dependency in other projects locally
  • deploy - done in an integration or release environment, copies the final package to the remote repository for sharing with other developers and projects.
A phase is depend of the previous one, so when you request the phase test, then the phase compile is done before,...
Directory layout
see below for a scala project
repository
Maven use repositories (local and remote) to store and to retrieve artifacts and their descriptor (pom). Artifacts are jar, war,... and they could be used as dependencies, maven plugin,...
By default, maven search artifact in the central repository. A "dedicated" repository for Scala stuff is available at http://scala-tools.org/repo-releases/.

Your first scala project with maven


In the following, we will run maven for the first time. Maven download what it need to work from remote repositories, and cache the downloaded artifact into its local repository (default is $HOME/.m2/repository). It only download what it need for the requested phases/goals (lazy downloading). So the first runs could be very long.

Step 0: installation


  • install jdk 1.5+ (eg : on my box $HOME/bin/soft-linux/jdk-1.5.0_03)
  • install maven 2.0.8+ (eg : on my box $HOME/bin/soft-java/apache-maven-2.0.8)
    • download it
    • unarchive it
    • add the apache-maven-2.0.8/bin directory to your PATH
    • check that maven is in the path:
      • go into any directory outside the maven installation
      • run mvn help, you should see
        usage: mvn [options] [<goal(s)>] [<phase(s)>]

        Options:
        -q,--quiet Quiet output - only show errors
        ...

Step 1: create a project


You could create a project skeleton with your favorite file system tools (following directory layout as below) or you could use archetypes. Maven Archetypes are project'skeleton that could be used to create new project.
mvn org.apache.maven.plugins:maven-archetype-plugin:1.0-alpha-7:create \
-DarchetypeGroupId=org.scala-tools.archetypes \
-DarchetypeArtifactId=scala-archetype-simple \
-DarchetypeVersion=1.1 \
-DremoteRepositories=http://scala-tools.org/repo-releases \
-DgroupId=your.proj.gid -DartifactId=your-proj-id

At the end of the process you should see something like
...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1 second
[INFO] Finished at: Sat Jan 05 17:39:47 CET 2008
[INFO] Final Memory: 6M/63M
[INFO] ------------------------------------------------------------------------

!! Success you now have a empty project under your-proj-id directory with the following directory layout :
your-proj-id/
|-- pom.xml
`-- src
|-- main
| `-- scala
| `-- your
| `-- proj
| `-- gid
| `-- App.scala
`-- test
`-- scala
`-- your
`-- proj
`-- gid
`-- AppTest.scala

In fact, the project is not empty it contains an helloworld application (App.scala) and a JUnit test (AppTest.scala).
In the next step, you will request phase (or goals). The results will be put under your-proj-id/target directory. The target directory is the working directory where every plugin put the result of computation. If you want to clean up, request the goal "clean"
mvn clean

Step 2: compile the project


# only compile
mvn compile

If it's the first time you use maven with scala, the build should failed with a message like
...
[ERROR] FATAL ERROR
[INFO]
------------------------------------------------------------------------
[INFO] The PluginDescriptor for the plugin Plugin [org.scala-tools:maven-scala-plugin] was not found.
[INFO]
...


Cause :

the pom.xml (autogenerated) doesn't specify wish version to use for maven-scala-plugin, so maven try to use the latest available localy, and none was previously downloaded.

Solutions :
  • edit the pom.xml and define a version for the plugin

  • request to download the latest available on remote repositories


I prefer the second solution (in this case):
# only compile
mvn -U compile

now you should see
...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
...

Step 3: compile and running test


The skeleton create a JUnit test AppTest.scala as sample, try to compile and run it
# compile + compile test + run test
mvn test

you should get :
...
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running your.proj.gid.AppTest
Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.054 sec <<< FAILURE!

Results :

Failed tests:
testKO(your.proj.gid.AppTest)

Tests run: 2, Failures: 1, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] There are test failures.

Please refer to /home/dwayne/tmp/your-proj-id/target/surefire-reports for the individual test results.

BUILD FAILURE, it's not good! So read the log on console :
  • there is 2 tests and one of them failed
  • the failed test is the method testKO from the class your.proj.gid.AppTest
  • see the content of the directory .../your-proj-id/target/surefire-reports for details

So you could read the problem in .../your-proj-id/target/surefire-reports/your.proj.gid.AppTest.txt
...
testKO(your.proj.gid.AppTest) Time elapsed: 0.01 sec <<< FAILURE!
junit.framework.AssertionFailedError
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.assertTrue(Assert.java:20)
at junit.framework.Assert.assertTrue(Assert.java:27)
at your.proj.gid.AppTest.testKO(AppTest.scala:26)
at your.proj.gid.AppTest.testKO(AppTest.scala:26)
...

So edit the test and fix it (it's easy), and rerun test until it pass.
Why the empty project is created with a failed test? to check that test are running and are used.

Step 4: generate the jar


# compile + run test + generate the jar
mvn package

If you fixed the test in Step 3, then a jar should be generated under the target directory. The jar doesn't contains the test classes, only the classes from src/main/scala/...

Step 5: start coding


  • add scala file under src/main/scala/... or src/test/scala/...
  • run the phases or goal you wish,...
  • if you need more lib (dependencies), edit the pom.xml and add <dependecy> node. By default you could declare dependency available on central repo (I suggest to use mvnrepository as a search engine in central repo), or in http://scala-tools.org/repo-releases/ (browse the directory, no search engine available :()

Conclusion


I expect this overlook and quick tutorial could help you to start playing with maven and scala. I plan to write other articles about "maven for scala" (about, the pom.xml and repositories). If you want to know more about maven and don't want to wai futur article, I suggest you browse the documentation of maven. I also suggest you to take a look at the maven-scala-plugin 2.x documentation, you'll see how to generate scaladoc, choose the scala version, or run a scala console with the project dependencies.

If you have question ask, If you want I detail some point here or in a futur article, ask.

14 comments:

frohoff said...

Looks great; can the jvm 1.5 be targeted with the maven plugin yet?

David Bernard said...

Yes, you could use any scala argument (edit your pom.xml) :

<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
<configuration>
<scalaVersion>${scala.version}</scalaVersion>
<args>
<arg>-target:jvm-1.5</arg>
</args>
<jvmArgs>
<jvmArg>-Xmx512m</jvmArg>
<jvmArg>-Xms64m</jvmArg>
</jvmArgs>
</configuration>
</plugin>

(sorry, I don't know how to format comment)

Patrick said...

Great writeup--I tried running the archetype:create command but got the following error
[WARNING] org.apache.velocity.runtime.exception.ReferenceException: reference : template = archetype-resources/pom.xml [line 32,column 16] : ${scala.version} is not a valid reference.

I tried passing scala.version as an argument/system property (-Dscala.version) but that didn't work either. System is Ubuntu Linux, Gutsy. Scala was installed with apt-get. Know how I can fix this?

Thanks
Patrick

David Bernard said...

short : Don't worry, this is a "normal" warning.
long : archetype use velocity to generate project template. Velocity try replace ${...} by a "predefined/evaluated" value/expression. Maven also use the ${...} syntax in pom.xml to evaluate properties when you build. So when the pom.xml to generate use "properties", the archetype warn about undef expression when it try to generate the pom.xml. Those value are evaluate after the project generation, when you'll use the pom.xml to build your project.

Is it clear ?

vainqueur said...

I've managed to compile OSGi bundles with maven and scala-tools.

See http://jeunef.wordpress.com/2008/01/04/osgi-maven-and-scala/

Leo Soto M. said...

Isn't there an overlap between maven and sbaz when it comes to manage dependencies between scala components? Or is maven capable of using sbaz repositories?

David Bernard said...

Hi,

I don't think maven can using sbaz repository. But it's possible to developpe an extension to maven to allow it to use sbaz.

I fact, I don't know a lot sbaz (IMHO, the doc a little obscur), so I can't really do a comparison and I could be wrong.
So yes, there is probably a overlap between maven's repository and sbaz (maven's repository are older, and allow access to java library see mvnrepository.com as example).

I don't agree, when you say sbaz manage dependencies, from my understanding, sbaz allow you to upload/dowload library but doesn't manage dependencies. If in the pom.xml of my project, I say I need lib X in version vX, maven take care of the (transitive) dependencies of X, And I don't need to search in the doc or the Manifest of the lib, which other lib I need to install. And if later I upgrade to vX+1 and some dependencies are not more required by lib X, they'll be not added (and I don't need to remove then like with a ant build (without ivy)).

Leo Soto M. said...

Yeah, I agree that maven is better resolving dependencies than sbaz, although sbaz has basic dependency management as far as I can see ("sbaz help pack" shows an option to specify them).

Anyway, your guide is very helpful. Thanks you very much!

Scot said...

It's not working for me, I get

[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Error creating from archetype

Embedded error: OldArchetype does not exist.
http://repo1.maven.org/maven2/org

David Bernard said...

A new version maven-archetype-plugin was release in Feb 2008, the new version (1.0 -> 2.0) break some features. until fixes (in the plugin), use the old version, try (copy/paste):

mvn org.apache.maven.plugins:maven-archetype-plugin:1.0-alpha-7:create \
-DarchetypeGroupId=org.scala-tools.archetypes \
-DarchetypeArtifactId=scala-archetype-simple \
-DarchetypeVersion=1.0 \
-DremoteRepositories=http://scala-tools.org/repo-releases \
-DgroupId=your.proj.gid -DartifactId=your-proj-id

Muddle-headed Wombat said...

Thank you so much; this tutorial was exactly what I needed.

I'm new to both Scala and to Maven, and there were two minor things that had me confused momentarily. I thought I'd just mention them in case they help someone else:

1. After step 1 (creating the project), I had to cd into the project directory in order for the command in step 2 to work. In retrospect, I guess it's obvious that you need to do that.

2. After completing the steps here, I wanted to actually see "my" scala application run. I used this command to do that:

scala -cp ./target/project_name-1.0-SNAPSHOT.jar group_ID.App

Kenny's Playground said...

Thanks for your article. I was very helpful to me.

If you're using Eclipse plugIn, you can create Scala Project easily, too.

Plz, check my article, too. thanks. again.

http://kennyground.blogspot.com/2009/04/create-scala-project-using-eclipse.html

Deepak said...

mvn command seem to be broken as the repo URL is missing now.

Use:

mvn org.apache.maven.plugins:maven-archetype-plugin:1.0-alpha-7:create \
-DarchetypeGroupId=org.scala-tools.archetypes \
-DarchetypeArtifactId=scala-archetype-simple \
-DarchetypeVersion=1.1 \
-DremoteRepositories=http://central.maven.org/maven2 \
-DgroupId=com.ebay.kt.reporting -DartifactId=hadoop_reporting

Change is:
DremoteRepositories=http://central.maven.org/maven2 \

Alex Edward said...

You have done a fantastic job, i really like the topic of your blog it is both informative and interesting
Hedford