Mavenizing the project structure of Xtext projects

The structure of typical Xtext projects does not match the standard layout of Maven projects. Xtext projects more adhere to standard Eclipse project layout, which means

  • manual implemented Java sources are in folder /src
  • the plugin Manifest is /META-INF/MANIFEST.MF
  • generated sources go to /src-gen

In my customer’s project Xtext sources are built with Maven, and also the sources produced by Xtext are produced within the Maven build, using the Fornax Workflow Maven plugin. Until now we have adjusted the Maven build to match the standard Xtext project structure, which requires some configuration in the build section of the POM like follows:

   <build>
        <sourceDirectory>src</sourceDirectory>
        <resources>
            <resource>
                <directory>src</directory>
            </resource>
            <resource>
                <directory>src-gen</directory>
            </resource>
        </resources>
        <plugins>
            <plugin>
                <groupId>org.fornax.toolsupport</groupId>
                <artifactId>fornax-oaw-m2-plugin</artifactId>
                <version>3.0.3</version>
                <configuration>
                    <checkFilesets>
                        <checkFileset>
                            <directory>${basedir}</directory>
                            <includes>
                                <include>src/**</include>
                            </includes>
                            <excludes>
                                <exclude>**/.svn/**</exclude>
                            </excludes>
                        </checkFileset>
                    </checkFilesets>
                    <outletSrcDir>src-gen</outletSrcDir>
                    <outletSrcOnceDir>src</outletSrcOnceDir>
                    <outletResOnceDir>resources</outletResOnceDir>
                    <properties></properties>
                    <workflowDescriptor>xtext/example/GenerateMyDsl.mwe</workflowDescriptor>
                    <workflowEngine>mwe</workflowEngine>
                </configuration>
                <executions>
                    <execution>
                        <phase>generate-sources</phase>
                        <goals>
                            <goal>run-workflow</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>build-helper-maven-plugin</artifactId>
                <executions>
                    <execution>
                        <id>add-source</id>
                        <phase>generate-sources</phase>
                        <goals>
                            <goal>add-source</goal>
                        </goals>
                        <configuration>
                            <sources>
                                <source>src</source>
                            </sources>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

Now requirements changed and the project structure should conform to the Maven structure, which means:

  • src/main/java: Contains hand-written Java code
  • src/main/resources: All non-Java hand-written code (including Workflow and Xtext grammar)
  • target/generated/java: Contains generated code

Another requirement  from the build team is that the actual output directory ‘target’ should be configurable. This means mainly that we have to use properties that Maven uses to refer to the source and target directories (project.build.sourceDirectory and project.build.directory), so that the build could override just these settings by passing a system property and all output gets produced to and compiled from an alternative directory structure.

Of course this is possible with small changes, but you have to know where.

Xtext Generator Worklow

In order to produce the output of the Xtext generator to different directories than default (src, src-gen => src/main/java, target/generated/java) the MWE workflow file has to be changed. For the generator component we have to override the properties srcPath and srcGenPath. Further, these output directories should be parametrized by the same properties that Maven uses, namely project.build.directory and project.build.directory. These properties need to be configured in the Generate<MyDSL>.properties with defaults.

Generate<MyDSL>.mwe

<workflow>
  ..
  <component class="org.eclipse.xtext.generator.Generator">
    <pathRtProject value="${runtimeProject}"/>
    <pathUiProject value="${runtimeProject}.ui"/>
    <projectNameRt value="${projectName}"/>
    <projectNameUi value="${projectName}.ui"/>
    <srcPath value="/${project.build.sourceDirectory}"/>
    <srcGenPath value="/${project.build.directory}/generated/java"/>
    ..
  </component>
  ..
</workflow>
<pre>

Generate<MyDSL>.properties

basedir=.
project.build.directory=target
project.build.sourceDirectory=src/main/java
grammarURI=classpath:/.../MyDSL.xtext
file.extensions=...
projectName=...

pom.xml: build section

The Java source directories also contain non-Java resources, like the workflow, grammar file etc. Normally this would go to the resources directory for Maven projects, but Xtext (0.7.2) cannot be configured to produce resource files to another directory than Java sources. On the other side those resources need to be found on the classpath during the build. This requires that we add the Java directories as resource directories in the build section of the POM. This settings already existed before, it just has been adopted.

Before:

    <build>
        <sourceDirectory>src</sourceDirectory>
        <resources>
            <resource>
                <directory>src</directory>
            </resource>
            <resource>
                <directory>src-gen</directory>
            </resource>
        </resources>
        ...
   </build>

After:

    <build>
        <resources>
            <resource>
                <directory>${project.build.sourceDirectory}</directory>
            </resource>
            <resource>
                <directory>${project.build.directory}/generated/java</directory>
            </resource>
        </resources>
         ...
     </build>

The <sourceDirectory> entry is not necessary anymore, since the main Java source directory is now src/main/java, which follows the Maven standard directory layout and gets automatically compiled.

maven-clean-plugin

Xtext produces output into two projects – the grammar project and the UI project. Now if the generated UI sources are produced below /target calling ‘mvn clean install’ would have an undesired side effect. Maven builds both modules after each other, so when the UI module is built with the goals ‘clean install’ the target directory is removed and the previous generated sources get lost.

The solution is that the maven-clean-plugin must be deactivated for the UI module and the grammar module must clean the target directory for the UI module.

pom.xml – Grammar project

<plugin>
    <artifactId>maven-clean-plugin</artifactId>
    <configuration>
        <filesets>
            <fileset>
                <directory>../my.dsl.ui</directory>
                <includes>
                    <include>${project.build.directory}</include>
                </includes>
            </fileset>
        </filesets>
    </configuration>
</plugin>

pom.xml – UI project

<plugin>
    <artifactId>maven-clean-plugin</artifactId>
    <configuration>
        <skip>true</skip>
    </configuration>
</plugin>

Eclipse settings

The new project structure affects, of course, the project settings for Eclipse. The settings apply for the grammar and UI plugin.

build.properties

Change the source folders in the build.properties file:

source.. = src/main/java/,target/generated/java/
bin.includes = META-INF/,\
 .,\
 plugin.xml

Java build path

Open the project properties and change the  Java Build Path settings:

  • Source Folder src -> src/main/java
  • Source Folder src-gen -> target/generated/java
  • Default outputr folder: bin -> target/classes
Advertisements

9 thoughts on “Mavenizing the project structure of Xtext projects

  1. Hi! Nice work!
    But instead of multiplying the conf files, maybe you could add a section in your pom to declare the properties given in “Generate.properties” and active the filtering in maven to dynamically replace the variables in “Generate.mwe”.

    Anyway, it’s a shame that eclipse products are so hard to integrate with CI engines such as Hudson…

    Regards
    Xavier

    • Thanks Xavier!
      I have good news rgd. integration of Eclipse builds in Hudson. I’m currently working with Maven Tycho, wrote an article and giving presentations about it on a conference and Eclipse DemoCamp. With Tycho setting up Eclipse builds will be easier and integrate seamlessly into Hudson, since it is Maven based and Hudson comes with out-of-the-box support. I will blog about this soon.
      The build environment in the project I wrote is CMake based, and Maven is integrated into CMake.
      ~Karsten

  2. Hi Karsten,

    thanks for posting this. How about adding tycho support to the Xtext project wizard. I’m thinking of an checkmark to enable Tycho support. If set the wizard could generate the project layout according to your post.

    Christoph

    • Hi Christoph!

      When using Tycho it is not necessary to adjust the project layout. For Tycho the Eclipse build.properties and plugin Manifest is leading. I’m working on Tycho support right now and will present Tycho at the JAX next week. At the moment I’m trying to also support MWE2 in Tycho with the Fornax Workflow plugin, but this is not so easy. At least I might contribute a fragment for that, when finding time.

      ~Karsten

    • Oh, did not know from your post. Seems that you already documented the steps I mentioned also. Anyway, double documentation is far better than no documentation 🙂

      I think Tycho will be the way to go for building Xtext projects in the future. At the moment I’m struggling with Adoption of MWE2 in the Fornax Maven plugin. You could compile Xtext sources with Tycho now already, but could not trigger the Xtext generator within Maven for now. This will change soon.

  3. Hi,

    When will Fornax maven plugin 3.1.0 be available? Is there some way to use maven for Xtext 1.0 with 3.0.3?

    Please let me know your comment.

    With Best Regards,
    Diwakar

    • Cannot tell when it can be released, at the moment I have no time for it. Problem is MWE2 at the moment. The plugin 3.0.3 should work with Xtext 1.0 when
      a) the workflow is rewritten for MWE 1, which the plugin supports
      b) Maven Tycho is used, since the Eclipse & Xtext jars are not deployed to a Maven2 repository and Tycho could use the P2 repositories
      I plan to blog on Xtext, Tycho and Fornax when it all works together.

  4. Hi ,

    thank your for this great article.
    After hours of searching, I’ve made my mwe running with normal eclipse-plugins structure.
    This would be a good thing to be added to the common fornax wiki

    Best regards
    Markus

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s