Building Xtext projects with Maven Tycho

Last year I showed how Xtext 0.7.2 projects can be build with Maven. It requires the installation of all related plugins as Maven artifacts in a Maven2 repository, which is a rather hard and error prone task. I was asked many times since then whether I would install Xtext 1.0 artifacts or even milestone versions into the openArchitectureWare Maven2 repository. Although I can really understand this requirement my hope was that I could avoid this and show a way to enable the build with Maven Tycho. Now the time has come that Tycho is mature enough and finally I had the time to do the necessary extension for the Fornax Maven Workflow Plugin to support the recently added MWE2 workflow engine, which is used from Xtext now as default. Maven Tycho and the Fornax plugin will allow to build Xtext projects in the most natural way possible.

A basic requirement for software builds is that everything can be build just from the sources. Also it is a good practice that no generated sources are checked in into a source repository. Unfortunately this often does not hold for Xtext projects, since it is a quite hard task to execute the Xtext generator in a build process. Yes, it is possible, but until now users often have to solve it again and again. Search the net, most projects seen there will just check in all the sources. The Fornax plugin and Maven will standardize this behavior.

What do you need to enable Xtext DSL builds with Tycho? Not much. Basically a Maven 3 installation (the beta is enough) and access to the internet to access the public repositories. A better idea is to use a repository manager in the intranet (like Nexus) to act as proxy for the public repositories. You will have to add some POMs to your projects. That’s it.

I’ll show now how to enable the basic project that you get from the Xtext project wizard get build with Maven Tycho.

1. Install Maven 3

Download Maven 3 from Apache, unpack it and put the bin folder on your path. Check the installation by typing ‘mvn –version‘ in a shell. You should something like this:

mvn --version
Apache Maven 3.0-beta-1 (r935667; 2010-04-19 19:00:39+0200)
Java version: 1.6.0_20
Java home: /System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home
Default locale: de_DE, platform encoding: MacRoman
OS name: "mac os x" version: "10.5.8" arch: "x86_64" Family: "mac"

2. Create the Xtext project

Select in Eclipse File / New / Project / Xtext Project and leave all the defaults. The project name should be org.xtext.example.mydsl for this tutorial. The projects will be created and almost empty for now. The aim is to get everything build by just adding the POMs.

3. Adjust plugin manifests

With the default Manifests the build does not execute successfully. The build fails because it has problems to resolve the logging infrastructure. It is sufficient to add Import-Package entries to resolve SLF4J and Logback. Open the plugin Manifest of the org.xtext.example.mydsl project, go to the Dependencies page and add the following Imported Packages:

• ch.qos.logback.classic
• ch.qos.logback.core.joran.spi
• org.slf4j

For each of them open the Properties dialog and set the “optional” flag. Otherwise you could get problems when deploying these plugins. (Thanks to Christian Amman for this tip!)

Open the Manifest of org.xtext.example.mydsl.generator, add the same packages and additionally

• org.apache.commons.logging

Again, set the imported packages to optional.

4. Add POM files

Each project will get a POM file. A Parent POM will contain the common settings and will act as a reactor POM also. Normally the Parent POM is placed in the directory above the modules, or here the plugin projects. This would be the workspace root. In this example we will place the Parent POM in the DSL grammar project. To make builds easier this POM will have the default name pom.xml. On the other side we have to add also a POM for the grammar project itself. Since it will be in the same directory we have to find another name, and we will call it pom-grammar.xml. For the other projects (.generator and .ui) we will stay with pom.xml.

4.1 org.xtext.example.mydsl/pom.xml

The Parent POM configures the Tycho plugins and aggregates the grammar project, UI project and the generator. Also the required repositories are configured here. Note that the folder “src” is added as resource path, this will add this folder to the classpath. The workflow execution would fail if the workflow module cannot be found on the classpath.

<!--?xml version="1.0" encoding="UTF-8"?-->
	4.0.0
	p2.osgi.bundle
	org.xtext.example.mydsl.parent
	1.0.0
	pom
	org.xtext.example.mydsl.MyDsl - Parent

		0.9.0

		./pom-grammar.xml
		../org.xtext.example.mydsl.ui
		../org.xtext.example.mydsl.generator

		<!-- The src directory must be named as resource dir to put it on the build classpath. This is required to resolve the 			workflow module named in the .mwe2 file -->

			src

				org.sonatype.tycho
				tycho-maven-plugin
				${version.tycho}
				true

				org.sonatype.tycho
				target-platform-configuration
				${version.tycho}

					p2

					org.fornax.toolsupport
					fornax-oaw-m2-plugin
					3.1.0-SNAPSHOT

						mwe2

							generate-sources
								run-workflow

			p2.eclipse.helios

http://download.eclipse.org/releases/helios

			p2

		<!-- At the moment the Fornax plugin is only available as snapshot -->

			fornax-snapshots

http://www.fornax-platform.org/archiva/repository/snapshots/

				false

				true

org.xtext.example.mydsl/pom-grammar.xml

The build of the grammar project needs to execute the workflow src/org/xtext/example/mydsl/GenerateMyDsl.mwe2. All other settings are already managed by the parent.

<!--?xml version="1.0" encoding="UTF-8"?-->
	4.0.0

		p2.osgi.bundle
		org.xtext.example.mydsl.parent
		1.0.0
		../org.xtext.example.mydsl/pom.xml

	org.xtext.example.mydsl
	eclipse-plugin
	org.xtext.example.mydsl.MyDsl - Grammar

				org.fornax.toolsupport
				fornax-oaw-m2-plugin

					src/org/xtext/example/mydsl/GenerateMyDsl.mwe2

org.xtext.example.mydsl.ui/pom.xml

The build configuration of the UI plugin is even easier. Only the reference to the parent and the minimum project coordinates need to be configured.

<!--?xml version="1.0" encoding="UTF-8"?-->
	4.0.0

		p2.osgi.bundle
		org.xtext.example.mydsl.parent
		1.0.0
		../org.xtext.example.mydsl/pom.xml

	org.xtext.example.mydsl.ui
	eclipse-plugin
	org.xtext.example.mydsl.MyDsl - UI

org.xtext.example.mydsl.generator/pom.xml

The generator plugin must execute its workflow. Note that the src directory and the compile directory of the grammar project are set as resource directory.

<!--?xml version="1.0" encoding="UTF-8"?-->
	4.0.0

		p2.osgi.bundle
		org.xtext.example.mydsl.parent
		1.0.0
		../org.xtext.example.mydsl/pom.xml

	org.xtext.example.mydsl.generator
	eclipse-plugin
	org.xtext.example.mydsl.MyDsl - Generator

			src
			<!-- Add the grammar classes to the classpath -->
			../org.xtext.example.mydsl/target/classes

				org.fornax.toolsupport
				fornax-oaw-m2-plugin

					mwe2
					src/workflow/MyDslGenerator.mwe2

5. Execute the build

Now the build should be executable. Run ‘mvn clean install‘ in the root of the org.xtext.example.mydsl project. Here is an excerpt of the console log you should get. The complete log can be viewed here.

mvn install
[INFO] Scanning for projects...
[WARNING] No explicit target runtime environment configuration. Build is platform dependent.
[INFO] Resolving target platform for project MavenProject: p2.osgi.bundle:org.xtext.example.mydsl:1.0.0 @ /Users/thoms/temp/tycho/trunk/trunk/org.xtext.example.mydsl/pom-grammar.xml
log4j:WARN No appenders could be found for logger (org.apache.commons.httpclient.HttpClient).
log4j:WARN Please initialize the log4j system properly.
[INFO] Adding repository http://download.eclipse.org/releases/helios
[INFO] Adding repository http://download.eclipse.org/releases/helios
[WARNING] No explicit target runtime environment configuration. Build is platform dependent.
[INFO] Resolving target platform for project MavenProject: p2.osgi.bundle:org.xtext.example.mydsl.ui:1.0.0 @ /Users/thoms/temp/tycho/trunk/trunk/org.xtext.example.mydsl.ui/pom.xml
[INFO] Adding repository (cached) http://download.eclipse.org/releases/helios
[WARNING] No explicit target runtime environment configuration. Build is platform dependent.
[INFO] Resolving target platform for project MavenProject: p2.osgi.bundle:org.xtext.example.mydsl.generator:1.0.0 @ /Users/thoms/temp/tycho/trunk/trunk/org.xtext.example.mydsl.generator/pom.xml
[INFO] Adding repository (cached) http://download.eclipse.org/releases/helios
[WARNING] No explicit target runtime environment configuration. Build is platform dependent.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] org.xtext.example.mydsl.MyDsl - Parent
[INFO] org.xtext.example.mydsl.MyDsl - Grammar
[INFO] org.xtext.example.mydsl.MyDsl - UI
[INFO] org.xtext.example.mydsl.MyDsl - Generator
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building org.xtext.example.mydsl.MyDsl - Parent 1.0.0
[INFO] ------------------------------------------------------------------------
....
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] org.xtext.example.mydsl.MyDsl - Parent ............ SUCCESS [0.681s]
[INFO] org.xtext.example.mydsl.MyDsl - Grammar ........... SUCCESS [26.418s]
[INFO] org.xtext.example.mydsl.MyDsl - UI ................ SUCCESS [3.103s]
[INFO] org.xtext.example.mydsl.MyDsl - Generator ......... SUCCESS [10.844s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS

5. Miscalleneous

5.1 ANTLR 3 download

Xtext relies on ANTLR 3, which cannot be hosted at Eclipse.org due to incompatible licenses. Therefore Xtext has introduced an automatic download (FYI: This behavior is implemented in AntlrToolFacade) that you have to confirm once:

*ATTENTION*
It is recommended to use the ANTLR 3 parser generator (BSD licence - http://www.antlr.org/license.html).
Do you agree to download it (size 1MB) from 'http://download.itemis.com/antlr-generator-3.0.1.jar'? (type 'y' or 'n' and hit enter)


Now an automatic build is not so responsive to questions like these. The plugin will confirm this download automatically.

5.2 Workflow execution problems

If the Maven plugin does not execute the MWE2 workflow properly it is not so responsive at the moment to name the root cause. But when running the Maven command with “-X” it will output the full Java command including classpath to the console. Copy the full command and try to execute it from the same directory. This might help to detect the problem you have.

/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/bin/java -classpath /Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.xpand/1.0.0.v201006150611/org.eclipse.xpand-1.0.0.v201006150611.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.xtend/1.0.0.v201006150611/org.eclipse.xtend-1.0.0.v201006150611.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.emf.mwe.core/1.0.0.v201006150535/org.eclipse.emf.mwe.core-1.0.0.v201006150535.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.emf.ecore/2.6.0.v20100614-1136/org.eclipse.emf.ecore-2.6.0.v20100614-1136.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.emf.common/2.6.0.v20100614-1136/org.eclipse.emf.common-2.6.0.v20100614-1136.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.apache.commons.cli/1.0.0.v20080604-1500/org.apache.commons.cli-1.0.0.v20080604-1500.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.emf.mwe2.runtime/1.0.0.v201006150446/org.eclipse.emf.mwe2.runtime-1.0.0.v201006150446.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/com.ibm.icu/4.2.1.v20100412/com.ibm.icu-4.2.1.v20100412.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.xtext/1.0.0.v201006170321/org.eclipse.xtext-1.0.0.v201006170321.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.emf.ecore.xmi/2.5.0.v20100521-1846/org.eclipse.emf.ecore.xmi-2.5.0.v20100521-1846.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.xtext.util/1.0.0.v201006170321/org.eclipse.xtext.util-1.0.0.v201006170321.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/com.google.collect/0.8.0.v201006170321/com.google.collect-0.8.0.v201006170321.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/com.google.inject/2.0.0.v201003051000/com.google.inject-2.0.0.v201003051000.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.antlr.runtime/3.0.0.v200803061811/org.antlr.runtime-3.0.0.v200803061811.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.emf.mwe.utils/1.0.0.v201006150535/org.eclipse.emf.mwe.utils-1.0.0.v201006150535.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.emf.mwe2.launch/1.0.0.v201006150907/org.eclipse.emf.mwe2.launch-1.0.0.v201006150907.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.emf.mwe2.language/1.0.0.v201006150907/org.eclipse.emf.mwe2.language-1.0.0.v201006150907.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.xtext.common.types/1.0.0.v201006170321/org.eclipse.xtext.common.types-1.0.0.v201006170321.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.xtend.typesystem.emf/1.0.0.v201006150611/org.eclipse.xtend.typesystem.emf-1.0.0.v201006150611.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/ch.qos.logback.classic/0.9.19.v20100519-1505/ch.qos.logback.classic-0.9.19.v20100519-1505.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/ch.qos.logback.core/0.9.19.v20100419-1216/ch.qos.logback.core-0.9.19.v20100419-1216.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.apache.commons.logging/1.1.1.v201005080502/org.apache.commons.logging-1.1.1.v201005080502.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.slf4j.api/1.5.11.v20100519-1910/org.slf4j.api-1.5.11.v20100519-1910.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/ch.qos.logback.slf4j/0.9.19.v20100519-1910/ch.qos.logback.slf4j-0.9.19.v20100519-1910.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.slf4j.log4j/1.5.11.v20100419-1106/org.slf4j.log4j-1.5.11.v20100419-1106.jar org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher /Users/thoms/Development/workspaces/oaw-v5-test/org.xtext.example.mydsl.generator/src/workflow/MyDslGenerator.mwe2

5.3 Fornax Plugin Snapshot version

The Maven plugin org.fornax.toolsupport:fornax-oaw-m2-plugin is not released yet. Yesterday I have added the necessary support for MWE2 and deployed a snapshot version 3.1.0-SNAPSHOT. Some minor issues have to be solved before it will be released.

5.4 Missing log output

The plugin forks a JVM with help of the Java ant task type. At the moment the output from the forked JVM does not get redirected to the Plugin’s log. This is one issue to solve for the Plugin before release. It makes debugging problems a bit harder, but when copying the executed Java command that you get from the debug output and execute yourself you get a better impression of the error.

5.5 Download sources

For your convenience you can download the example project here.

About these ads