Karsten's Blog

August 18, 2010

Building Xtext projects with Maven Tycho

Filed under: M2T, Maven, Xtext — kthoms @ 9:19 PM

Last year I showed how Xtext 0.7.2 projects can be build with Maven. It requires the installation of all related plugins as Maven artifacts in a Maven2 repository, which is a rather hard and error prone task. I was asked many times since then whether I would install Xtext 1.0 artifacts or even milestone versions into the openArchitectureWare Maven2 repository. Although I can really understand this requirement my hope was that I could avoid this and show a way to enable the build with Maven Tycho. Now the time has come that Tycho is mature enough and finally I had the time to do the necessary extension for the Fornax Maven Workflow Plugin to support the recently added MWE2 workflow engine, which is used from Xtext now as default. Maven Tycho and the Fornax plugin will allow to build Xtext projects in the most natural way possible.

A basic requirement for software builds is that everything can be build just from the sources. Also it is a good practice that no generated sources are checked in into a source repository. Unfortunately this often does not hold for Xtext projects, since it is a quite hard task to execute the Xtext generator in a build process. Yes, it is possible, but until now users often have to solve it again and again. Search the net, most projects seen there will just check in all the sources. The Fornax plugin and Maven will standardize this behavior.

What do you need to enable Xtext DSL builds with Tycho? Not much. Basically a Maven 3 installation (the beta is enough) and access to the internet to access the public repositories. A better idea is to use a repository manager in the intranet (like Nexus) to act as proxy for the public repositories. You will have to add some POMs to your projects. That’s it.

I’ll show now how to enable the basic project that you get from the Xtext project wizard get build with Maven Tycho.

1. Install Maven 3

Download Maven 3 from Apache, unpack it and put the bin folder on your path. Check the installation by typing ‘mvn –version‘ in a shell. You should something like this:

mvn --version
Apache Maven 3.0-beta-1 (r935667; 2010-04-19 19:00:39+0200)
Java version: 1.6.0_20
Java home: /System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home
Default locale: de_DE, platform encoding: MacRoman
OS name: "mac os x" version: "10.5.8" arch: "x86_64" Family: "mac"

2. Create the Xtext project

Select in Eclipse File / New / Project / Xtext Project and leave all the defaults. The project name should be org.xtext.example.mydsl for this tutorial. The projects will be created and almost empty for now. The aim is to get everything build by just adding the POMs.

3. Adjust plugin manifests

With the default Manifests the build does not execute successfully. The build fails because it has problems to resolve the logging infrastructure. It is sufficient to add Import-Package entries to resolve SLF4J and Logback. Open the plugin Manifest of the org.xtext.example.mydsl project, go to the Dependencies page and add the following Imported Packages:

  • ch.qos.logback.classic
  • ch.qos.logback.core.joran.spi
  • org.slf4j

For each of them open the Properties dialog and set the “optional” flag. Otherwise you could get problems when deploying these plugins. (Thanks to Christian Amman for this tip!)

Open the Manifest of org.xtext.example.mydsl.generator, add the same packages and additionally

  • org.apache.commons.logging

Again, set the imported packages to optional.

4. Add POM files

Each project will get a POM file. A Parent POM will contain the common settings and will act as a reactor POM also. Normally the Parent POM is placed in the directory above the modules, or here the plugin projects. This would be the workspace root. In this example we will place the Parent POM in the DSL grammar project. To make builds easier this POM will have the default name pom.xml. On the other side we have to add also a POM for the grammar project itself. Since it will be in the same directory we have to find another name, and we will call it pom-grammar.xml. For the other projects (.generator and .ui) we will stay with pom.xml.

4.1 org.xtext.example.mydsl/pom.xml

The Parent POM configures the Tycho plugins and aggregates the grammar project, UI project and the generator. Also the required repositories are configured here. Note that the folder “src” is added as resource path, this will add this folder to the classpath. The workflow execution would fail if the workflow module cannot be found on the classpath.

<!--?xml version="1.0" encoding="UTF-8"?-->
	4.0.0
	p2.osgi.bundle
	org.xtext.example.mydsl.parent
	1.0.0
	pom
	org.xtext.example.mydsl.MyDsl - Parent

		0.9.0

		./pom-grammar.xml
		../org.xtext.example.mydsl.ui
		../org.xtext.example.mydsl.generator

		<!-- The src directory must be named as resource dir to put it on the build classpath. This is required to resolve the 			workflow module named in the .mwe2 file -->

			src

				org.sonatype.tycho
				tycho-maven-plugin
				${version.tycho}
				true

				org.sonatype.tycho
				target-platform-configuration
				${version.tycho}

					p2

					org.fornax.toolsupport
					fornax-oaw-m2-plugin
					3.1.0-SNAPSHOT

						mwe2

							generate-sources
								run-workflow

			p2.eclipse.helios

http://download.eclipse.org/releases/helios

			p2

		<!-- At the moment the Fornax plugin is only available as snapshot -->

			fornax-snapshots

http://www.fornax-platform.org/archiva/repository/snapshots/

				false

				true

org.xtext.example.mydsl/pom-grammar.xml

The build of the grammar project needs to execute the workflow src/org/xtext/example/mydsl/GenerateMyDsl.mwe2. All other settings are already managed by the parent.

<!--?xml version="1.0" encoding="UTF-8"?-->
	4.0.0

		p2.osgi.bundle
		org.xtext.example.mydsl.parent
		1.0.0
		../org.xtext.example.mydsl/pom.xml

	org.xtext.example.mydsl
	eclipse-plugin
	org.xtext.example.mydsl.MyDsl - Grammar

				org.fornax.toolsupport
				fornax-oaw-m2-plugin

					src/org/xtext/example/mydsl/GenerateMyDsl.mwe2

org.xtext.example.mydsl.ui/pom.xml

The build configuration of the UI plugin is even easier. Only the reference to the parent and the minimum project coordinates need to be configured.

<!--?xml version="1.0" encoding="UTF-8"?-->
	4.0.0

		p2.osgi.bundle
		org.xtext.example.mydsl.parent
		1.0.0
		../org.xtext.example.mydsl/pom.xml

	org.xtext.example.mydsl.ui
	eclipse-plugin
	org.xtext.example.mydsl.MyDsl - UI

org.xtext.example.mydsl.generator/pom.xml

The generator plugin must execute its workflow. Note that the src directory and the compile directory of the grammar project are set as resource directory.

<!--?xml version="1.0" encoding="UTF-8"?-->
	4.0.0

		p2.osgi.bundle
		org.xtext.example.mydsl.parent
		1.0.0
		../org.xtext.example.mydsl/pom.xml

	org.xtext.example.mydsl.generator
	eclipse-plugin
	org.xtext.example.mydsl.MyDsl - Generator

			src
			<!-- Add the grammar classes to the classpath -->
			../org.xtext.example.mydsl/target/classes

				org.fornax.toolsupport
				fornax-oaw-m2-plugin

					mwe2
					src/workflow/MyDslGenerator.mwe2

5. Execute the build

Now the build should be executable. Run ‘mvn clean install‘ in the root of the org.xtext.example.mydsl project. Here is an excerpt of the console log you should get. The complete log can be viewed here.

mvn install
[INFO] Scanning for projects...
[WARNING] No explicit target runtime environment configuration. Build is platform dependent.
[INFO] Resolving target platform for project MavenProject: p2.osgi.bundle:org.xtext.example.mydsl:1.0.0 @ /Users/thoms/temp/tycho/trunk/trunk/org.xtext.example.mydsl/pom-grammar.xml
log4j:WARN No appenders could be found for logger (org.apache.commons.httpclient.HttpClient).
log4j:WARN Please initialize the log4j system properly.
[INFO] Adding repository http://download.eclipse.org/releases/helios
[INFO] Adding repository http://download.eclipse.org/releases/helios
[WARNING] No explicit target runtime environment configuration. Build is platform dependent.
[INFO] Resolving target platform for project MavenProject: p2.osgi.bundle:org.xtext.example.mydsl.ui:1.0.0 @ /Users/thoms/temp/tycho/trunk/trunk/org.xtext.example.mydsl.ui/pom.xml
[INFO] Adding repository (cached) http://download.eclipse.org/releases/helios
[WARNING] No explicit target runtime environment configuration. Build is platform dependent.
[INFO] Resolving target platform for project MavenProject: p2.osgi.bundle:org.xtext.example.mydsl.generator:1.0.0 @ /Users/thoms/temp/tycho/trunk/trunk/org.xtext.example.mydsl.generator/pom.xml
[INFO] Adding repository (cached) http://download.eclipse.org/releases/helios
[WARNING] No explicit target runtime environment configuration. Build is platform dependent.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] org.xtext.example.mydsl.MyDsl - Parent
[INFO] org.xtext.example.mydsl.MyDsl - Grammar
[INFO] org.xtext.example.mydsl.MyDsl - UI
[INFO] org.xtext.example.mydsl.MyDsl - Generator
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building org.xtext.example.mydsl.MyDsl - Parent 1.0.0
[INFO] ------------------------------------------------------------------------
....
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] org.xtext.example.mydsl.MyDsl - Parent ............ SUCCESS [0.681s]
[INFO] org.xtext.example.mydsl.MyDsl - Grammar ........... SUCCESS [26.418s]
[INFO] org.xtext.example.mydsl.MyDsl - UI ................ SUCCESS [3.103s]
[INFO] org.xtext.example.mydsl.MyDsl - Generator ......... SUCCESS [10.844s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS

5. Miscalleneous

5.1 ANTLR 3 download

Xtext relies on ANTLR 3, which cannot be hosted at Eclipse.org due to incompatible licenses. Therefore Xtext has introduced an automatic download (FYI: This behavior is implemented in AntlrToolFacade) that you have to confirm once:

*ATTENTION*
It is recommended to use the ANTLR 3 parser generator (BSD licence - http://www.antlr.org/license.html).
Do you agree to download it (size 1MB) from 'http://download.itemis.com/antlr-generator-3.0.1.jar'? (type 'y' or 'n' and hit enter)

Now an automatic build is not so responsive to questions like these. The plugin will confirm this download automatically.

5.2 Workflow execution problems

If the Maven plugin does not execute the MWE2 workflow properly it is not so responsive at the moment to name the root cause. But when running the Maven command with “-X” it will output the full Java command including classpath to the console. Copy the full command and try to execute it from the same directory. This might help to detect the problem you have.

/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/bin/java -classpath /Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.xpand/1.0.0.v201006150611/org.eclipse.xpand-1.0.0.v201006150611.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.xtend/1.0.0.v201006150611/org.eclipse.xtend-1.0.0.v201006150611.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.emf.mwe.core/1.0.0.v201006150535/org.eclipse.emf.mwe.core-1.0.0.v201006150535.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.emf.ecore/2.6.0.v20100614-1136/org.eclipse.emf.ecore-2.6.0.v20100614-1136.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.emf.common/2.6.0.v20100614-1136/org.eclipse.emf.common-2.6.0.v20100614-1136.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.apache.commons.cli/1.0.0.v20080604-1500/org.apache.commons.cli-1.0.0.v20080604-1500.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.emf.mwe2.runtime/1.0.0.v201006150446/org.eclipse.emf.mwe2.runtime-1.0.0.v201006150446.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/com.ibm.icu/4.2.1.v20100412/com.ibm.icu-4.2.1.v20100412.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.xtext/1.0.0.v201006170321/org.eclipse.xtext-1.0.0.v201006170321.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.emf.ecore.xmi/2.5.0.v20100521-1846/org.eclipse.emf.ecore.xmi-2.5.0.v20100521-1846.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.xtext.util/1.0.0.v201006170321/org.eclipse.xtext.util-1.0.0.v201006170321.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/com.google.collect/0.8.0.v201006170321/com.google.collect-0.8.0.v201006170321.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/com.google.inject/2.0.0.v201003051000/com.google.inject-2.0.0.v201003051000.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.antlr.runtime/3.0.0.v200803061811/org.antlr.runtime-3.0.0.v200803061811.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.emf.mwe.utils/1.0.0.v201006150535/org.eclipse.emf.mwe.utils-1.0.0.v201006150535.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.emf.mwe2.launch/1.0.0.v201006150907/org.eclipse.emf.mwe2.launch-1.0.0.v201006150907.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.emf.mwe2.language/1.0.0.v201006150907/org.eclipse.emf.mwe2.language-1.0.0.v201006150907.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.xtext.common.types/1.0.0.v201006170321/org.eclipse.xtext.common.types-1.0.0.v201006170321.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.eclipse.xtend.typesystem.emf/1.0.0.v201006150611/org.eclipse.xtend.typesystem.emf-1.0.0.v201006150611.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/ch.qos.logback.classic/0.9.19.v20100519-1505/ch.qos.logback.classic-0.9.19.v20100519-1505.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/ch.qos.logback.core/0.9.19.v20100419-1216/ch.qos.logback.core-0.9.19.v20100419-1216.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.apache.commons.logging/1.1.1.v201005080502/org.apache.commons.logging-1.1.1.v201005080502.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.slf4j.api/1.5.11.v20100519-1910/org.slf4j.api-1.5.11.v20100519-1910.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/ch.qos.logback.slf4j/0.9.19.v20100519-1910/ch.qos.logback.slf4j-0.9.19.v20100519-1910.jar:/Users/thoms/.m2/repository/p2/osgi/bundle/org.slf4j.log4j/1.5.11.v20100419-1106/org.slf4j.log4j-1.5.11.v20100419-1106.jar org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher /Users/thoms/Development/workspaces/oaw-v5-test/org.xtext.example.mydsl.generator/src/workflow/MyDslGenerator.mwe2

5.3 Fornax Plugin Snapshot version

The Maven plugin org.fornax.toolsupport:fornax-oaw-m2-plugin is not released yet. Yesterday I have added the necessary support for MWE2 and deployed a snapshot version 3.1.0-SNAPSHOT. Some minor issues have to be solved before it will be released.

5.4 Missing log output

The plugin forks a JVM with help of the Java ant task type. At the moment the output from the forked JVM does not get redirected to the Plugin’s log. This is one issue to solve for the Plugin before release. It makes debugging problems a bit harder, but when copying the executed Java command that you get from the debug output and execute yourself you get a better impression of the error.

5.5 Download sources

For your convenience you can download the example project here.

April 15, 2010

Xpand builder deadlock situation

Filed under: Eclipse, M2T — Tags: , , , — kthoms @ 1:48 PM

Today a colleague in my project came with an issue she experienced with several workspaces, which contain amongst others Xpand projects. After starting Eclipse during the initial build phase Eclipse hang and did nothing. In the workspace log we found indication for a deadlock situation, since there one of the last entries stated that thread Main was waiting for thread Main, and this happened during class loading of the XtendXpandBuilder. We debugged the situation and from the stacktraces of the running threads we could see that one thread tried to refresh a resource that was out of sync.

The problem was solved by starting Eclipse with the -refresh platform option. The out-of-sync resource was synchronized before the builder runs.

November 17, 2009

Eclipse DemoCamp November 26th in Frankfurt

Filed under: Eclipse, M2T, Xtext — Tags: , , , , , , , , , , , — kthoms @ 9:05 PM

Next week, Thursday Nov. 26th, there will be an Eclipse DemoCamp in Frankfurt, starting 05:30 PM until 09:00 PM at Saalbau Gallus. I am happy that my current project schedule allows me being in Frankfurt at this time. There are already over 40 registered attendees, this is quite impressing. There are 6 interesting talks presented at this DemoCamp:

  • Using Eclipse TMF in a large scale modeling and implementation project from Dr. Nikolai Busse (Deutsche Börse Systems AG). The project Nikolai is talking about is the one I am working on most of this year. I already had the opportunity to talk about this project at the CodeGeneration 2009 and W-JAX 09 conference. Nikolai will no doubt give an really interesting talk about an extraordinary project.
  • redView – dynamische Views für Business Applications from Ekkehard Gentz. Ekke has created a fascinating project using technologies like EMF, CDO, openArchitectureWare, UML2. Ekke is also a member on the openArchitectureWare team and it is always a pleasure to talk with him. Last time we met was last week at the W-JAX, but unfortunately I had to leave on thursday before his talk. Now I have the chance to see it finally.
  • JPA (EclipseLink) in OSGI Anwendungen from Karsten Voigt (IBM Global Business Services). EclipseLink will be the reference implementation of the JPA 2.0 standard. This framework has already a long history and many experience was put into it. Although Hibernate is most widely spreaded I think that EclipseLink will play a bigger role in the future. EclipseLink has smooth integration with EMF with Teneo, which makes Domain Driven Design with JPA persistence really easy.
  • Eclipse Code Recommenders – How much can the IDE predict what you will write in the next seconds? from Marcel Bruch (TU Darmstadt). Marcel presents his research project where a more intelligent way for producing IDE proposals is developed. Sounds interesting.
  • Das ist hier Usus… from Leif Frenzel and Stefan Schürle (adrena objects). They will present projectusus, which aims to provide Eclipse plugins that derive settings for Eclipse and code quality plugins like Checkstyle and EclEmma. The clue is that the settings try to match automatically what’s a project’s usus.
  • RCP goes Web 2.0 – Eclipse Rich Ajax Platform from Benjamin Muskalla (EclipseSource). Let’s see what RAP has new…

6 talks are really a lot for a DemoCamp. Besides presentations also networking is important at a DemoCamp. Let’s hope there is enough space for talking. Ralph Müller from the Eclipse Foundation will be there (where on the world is he not?), some people from SAP, Heidelberger Services AG, some of my colleagues from itemis, and several others. And it will be the first time I meet Attila Bak in person, who I now from the Fornax Project since some years already, but until now just via email.

Maybe I can met YOU also there? Registering is easy, and the event is free of charge. To register, just enter yourself to the DemoCamp’s Wiki page. To edit it, you’ll just need an account there. If you don’t want to open account just for that, just send me a mail and I’ll add you.

 

November 16, 2009

Misleading error message for JavaBeautifier dependencies

Filed under: M2T — Tags: , , — kthoms @ 9:34 PM

The postprocessor org.eclipse.xpand2.output.JavaBeautifier comes with the plugin org.eclipse.xpand, but needs additional dependencies to work. Under the hood the beautifier leverages JDT, so the dependency to JDT is obvious. Normally the missing plugin dependencies can be easily derived from the error messages, since the package name of some missing class usually maps to the plugin identifier where the class resides. However, you might recognize the error message
java.lang.NoClassDefFoundError: org/eclipse/jface/text/BadLocationException
and think that the class must be in plugin org.eclipse.jface.text. But that’s not true, although there is of course a plugin org.eclipse.jface.text. Actually, class BadLocationException is in the plugin org.eclipse.text and is reexported from the JFace plugin.

The actual dependencies that have to be added when using the JavaBeautifiers are:

  • org.eclipse.jdt.core
  • org.eclipse.text
  • org.eclipse.core.resources
  • org.eclipse.core.runtime

May 19, 2009

Founded Google Code project for the Hybridlabs Beautifier

Filed under: Hybridlabs Beautifier, M2T, Open Source, openArchitectureWare — kthoms @ 6:02 AM

The Hybridlabs beautifier has become a famous component for oAW. It is a Java code formatter based on the Jalopy library and what it makes it especially interesting is its feature of auto-organizing Java imports. Since M2T is now upcoming users ask for porting the component to M2T Xpand.

I have asked Karsten Klein, the founder of the Hybridlabs beautifier, to host the projectat a public site and join development efforts. Karsten is at the moment too busy on his normal projects to work on sparetime projects like his beautifier. So I got agreement from Karsten to support him founding the project.

Yesterday I created the Hybridlabs Beautifier project at Google Code. The project is now public, so it is not necessary to send patches and feature requests to Karsten per Mail, better use the given Infrastructure of the project from now on. I took the sources from version 1.1.9, set up the project structure and provided a Maven build for the project. I decided to make to increment the version here to 2.0.0, since project organization changed and M2T support was added.

Internally the code change is really small. The main component, JavaImportBeautifier, now imports both PostProcessor interfaces, from oAW4 Xpand2 and from Eclipse M2T Xpand. This way the component could kept backward compatible to oAW4 while at the same time being compatible for M2T Xpand. A good example why it makes sense to program against interfaces. A drawback of this approach is that this component has now a class dependency to both, which makes it necessary to have oAW4 Xpand on the classpath when using the component within M2T. I excluded all transitive dependencies for oaw-core-xpand2, so dependencies are rather small.

As a result I have created a bundle of the current build including dependencies and uploaded it. This bundle can be downloaded here.

The Silver is the New Black Theme. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 375 other followers