Tycho build works also for oAW 4.3 projects

Yesterday I showed how Xtext 1.0 projects can be build with Maven Tycho. So why shouldn’t the same setup don’t work for ‘old’ openArchitectureWare Xpand projects? This works like a charm. I took the simple project that the Xpand project wizard creates and added the POM, and it worked! You can download the sample project here. See the POM below, a setup for other oAW Xpand projects should be rather similar.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>p2.osgi.bundle</groupId>
	<artifactId>org.openarchitectureware.example.xpand.tycho</artifactId>
	<version>1.0.0</version>
	<packaging>eclipse-plugin</packaging>
	<name>org.openarchitectureware.example.xpand.tycho</name>

	<properties>
		<version.tycho>0.9.0</version.tycho>
	</properties>

	<build>
		<resources>
			<resource>
				<directory>src</directory>
			</resource>
		</resources>
		<plugins>
			<plugin>
				<groupId>org.sonatype.tycho</groupId>
				<artifactId>tycho-maven-plugin</artifactId>
				<version>${version.tycho}</version>
				<extensions>true</extensions>
			</plugin>
			<plugin>
				<groupId>org.sonatype.tycho</groupId>
				<artifactId>target-platform-configuration</artifactId>
				<version>${version.tycho}</version>
				<configuration>
					<resolver>p2</resolver>
				</configuration>
			</plugin>
			<plugin>
				<groupId>org.fornax.toolsupport</groupId>
				<artifactId>fornax-oaw-m2-plugin</artifactId>
				<version>3.1.0-SNAPSHOT</version>
				<configuration>
					<workflowEngine>oaw</workflowEngine>
					<workflowDescriptor>workflow/generator.oaw</workflowDescriptor>
				</configuration>
				<executions>
					<execution>
						<phase>generate-sources</phase>
						<goals>
							<goal>run-workflow</goal>
						</goals>
					</execution>
				</executions>
			</plugin>
			<plugin>
				<artifactId>maven-clean-plugin</artifactId>
				<version>2.4.1</version>
				<configuration>
					<filesets>
						<fileset>
							<directory>src-gen</directory>
							<includes>
								<include>**/**</include>
							</includes>
						</fileset>
					</filesets>
				</configuration>
			</plugin>
		</plugins>
	</build>
	<repositories>

		<repository>
			<id>p2.openarchitectureware</id>
			<url>http://www.openarchitectureware.org/updatesite/milestone/</url>
			<layout>p2</layout>
		</repository>
		<repository>
			<id>p2.eclipse.helios</id>
			<url>http://download.eclipse.org/releases/helios</url>
			<layout>p2</layout>
		</repository>
		<!-- At the moment the Fornax plugin is only available as snapshot -->
		<repository>
			<id>fornax-snapshots</id>
			<url>http://www.fornax-platform.org/archiva/repository/snapshots/</url>
			<releases>
				<enabled>false</enabled>
			</releases>
			<snapshots>
				<enabled>true</enabled>
			</snapshots>
		</repository>
	</repositories>
	<pluginRepositories>
		<pluginRepository>
			<id>fornax-snapshots</id>
			<url>http://www.fornax-platform.org/archiva/repository/snapshots/</url>
			<releases>
				<enabled>false</enabled>
			</releases>
			<snapshots>
				<enabled>true</enabled>
			</snapshots>
		</pluginRepository>
	</pluginRepositories>
</project>

Lazy evaluation in Xpand

Usually code generation is a purely sequential process. Since the model does not change during the generation of an artifact all content can be computed in the template where it is needed for the output. But sometimes there is the wish to defer the output to a later point of time during the generation of an artifact.

The typical use case for this is import statements. If for example you want to generate a Java class and want to import all used types then the following alternatives are given:

  • Compute the types that the about-to-be-generated class will use
  • Print out all type names full qualified whenever needed and organize the imports with a postprocessor. For Java code generation the Hybridlabs Beautifier is used widely.

However, both approaches do not seamlessly solve the problem. What really is needed is some kind of lazy evaluation in Xpand. Therefore Jos Warmer wrote a feature proposal once. The feature that he proposed for Xpand is called Insertion Point. The idea was to mark some point in the Xpand template where some code will be inserted at a later point of time. Code is evaluated into this insertion point when the content can be derived easier.

From this feature proposal Feature Request #261607 was created in Eclipse Bugzilla system. In this bug entry, and also offline, a lively discussion arose in the team. The challenges for this feature request were:

  • The Xpand language has a rather small set of keywords. Adding this feature, which is used in some cases only, should not introduce too much changes to the Xpand language
  • An implementation should not break existing code
  • The solution should enable decent evaluation, e.g. avoiding duplicates, sorting

The latest proposal just introduces just one new keyword in Xpand, but requires an implementation pattern with Xtend function. The proposal is to add a keyword ONFILECLOSE to the EXPAND statement. By calling EXPAND with ONFILECLOSE the evaluation of the EXPAND statement is deferred until the FILE statement is closed. Any state that is used by the called definition is computed during the FILE evaluation. The EXPAND statement has to be evaluated with the execution context which is active when reaching the EXPAND statement.

Let’s see this by example. As example we take the project that Xpand’s project wizard creates, with small changes. The entities and types have now an additional ‘packageName’ attribute, and the both entities have been assigned different packages ‘entities1′ and ‘entities2′. Additionally entity ‘Person’ has a feature ‘birthday’ of type Date, which is mapped to java.util.Date. Therefore class Person.java has to import entities2.Address and java.util.Date. The used types should be collected when rendering instance variables and accessor methods, but inserted earlier in the code.

First take a look at the template code:


As you can see the definition ImportBlock is called for each Type instance (e.g. Entity and DataType instances) in the collection returned by the Xtend function UsedType(). In an alternative approach this function would be responsible to compute all the types that will be used by this Entity instance. But for the new implementation it just creates an empty list and returns that one. The implementation of the UsedType() function (in file GeneratorExtensions.ext) is:

create List[Type] UsedType (Entity e) : (List[Type]) {};

So at the time the Xpand engine reaches EXPAND ImportBlock… the list would be empty. Now note the ONFILECLOSE keyword at the end of the EXPAND statement. This one tells the engine now that the evaluation of this Xpand statement should be deferred until the file is about to be closed. During evaluation of the template code, in the FOREACH loop, another extension function addUsedType(f.type) is called. This one adds the type of the current processed feature to the collection returned by UsedTypes(). Therfore it is important that the UsedType() function uses the create keyword, since we want to create the collection on first access for one Entity and return the same instance when UsedTypes() is called later for that Entity again.

The function addUsedType() is used in the template like this:

«addUsedType(f.type)»

Xpand would print out the result of the function as string, but we don’t want to produce any output by calling the function. Therefore we assure that the function adds the type to the collection and returns an empty string:

addUsedType (Entity e, Type t) : UsedType(e).add(t) -> "";

During evaluation it could be that types were used multiple times within the template, but we want just one import statement per type. Further we don’t need imports for types from the java.lang package (here the packageName information for the DataType instances String and Integer is null) or for types that are in the same package like the entity. Therefore we transform the UsedType() collection before finally invoking the ImportBlock definition.

«EXPAND ImportBlock FOREACH UsedType()
 .select(t|t.packageName!=null && t.packageName!=this.packageName).toSet()
 .sortBy(t|t.qualifiedName()) ONFILECLOSE»

Conclusion

Lazy evaluation allows code generated with Xpand in an non-sequential matter. The proposed solution using ONFILECLOSE solves the desired Insertion Point feature by adding just one additional keyword to the Xpand language. It does not break existing template code. When accepted this code will be contributed to Xpand 0.8.0-M6 soon. For those who want to test it I have created a feature patch for the org.eclipse.xpand feature. The example project with the sources listed in this article can be downloaded here.

Feedback is welcome and is best placed in the bugzilla feature request.

Speaking at W-JAX 09

wjax09_button_speaker_enMy colleague Heiko Behrens and I will have a session at the upcoming W-JAX conference.
The session is entitled “Mastering differentiated MDSD Requirements at Deutsche Boerse AG“, and we will share our experience about successfully applying MDSD approaches that we gained through our consulting tasks at the Deutsche Boerse. The project at the Deutsche Boerse is in many ways challenging and a great example how model driven software development really helps to master complexity. The Deutsche Boerse is developing together with the New York ISE a new Global Trading System (GTS), which is scheduled to start operations in 2011. High performance and reliability is crucial for this system, and it would not be possible to deliver both without means of modeling and code generation. Of course we leverageEclipse Modeling and openArchitectureWare for that. We held this session already at the Code Generation 2009 with overwhealming feedback, and in the meantime the story continued and we will include the new experiences into our talk.

Xtext 0.7.2 artefacts installed to Maven repository

Finally I came to deploy the Xtext 0.7.2 artefacts to the Maven repository at openarchitectureware.org. I almost forgot that I did not that so far, but Michael Clay dropped me a mail today.

Maintaining the repository is a hard an error prone task. With each updated plugin the POM files need to be adjusted manually. Therefore each plugin manifest must be opened and the dependencies compared. Fortunately this stayed stable through the Xtext 0.7.x releases and also the underlying Eclipse libraries did not need to be updated after the Galileo release. I hope in future Eclipse plugins will be automatically deployed to a central Maven repository. There is currently some discussion on this in Eclipse Bug#283745.

I have optimized the manual deployment process a bit for myself. In my workspace I set up a project reflecting the repository structure. The jars are placed like they are stored in the repository with pattern {artifactId}-{version}.jar. Slightly different for the POMs. In the repository they are stored with pattern {artifactId}-{version}.pom, but in my workspace slightly different with {artifactId}-{version}.jar.pom.

The reason for is that I configured an External Tool configuration in Eclipse for deployment, which invokes the Maven deployment using “mvn deploy:deploy-file”. The property “pomFile” is computed from the selected resource through the {selected_resource_loc} variable (which ends with .jar), appended with “.pom”.

This way the jar file just needs to be selected and the deployment is executed with the external tool launcher.

[INFO] Scanning for projects...
[INFO] Searching repository for plugin with prefix: 'deploy'.
[INFO] ------------------------------------------------------------------------
[INFO] Building Maven Default Project
[INFO] task-segment: [deploy:deploy-file] (aggregator-style)
[INFO] ------------------------------------------------------------------------
[INFO] [deploy:deploy-file]
Uploading: scp://openarchitectureware.org/home/oaw/m2/repository/org/eclipse/xtext/xtext-core/0.7.2/xtext-core-0.7.2.jar
1495K uploaded
[INFO] Retrieving previous metadata from releases.openarchitectureware.org
[INFO] Uploading repository metadata for: 'artifact org.eclipse.xtext:xtext-core'
[INFO] Uploading project information for xtext-core 0.7.2
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 37 seconds
[INFO] Finished at: Thu Sep 24 21:06:43 CEST 2009
[INFO] Final Memory: 3M/6M
[INFO] ------------------------------------------------------------------------

Of course the repository must be configured in the ~/.m2/settings.xml file. The repository is accessed with SCP, authentication is done with public/private key.

<servers>
    <server>
      <id>releases.openarchitectureware.org</id>
      <username>oaw</username>
	  <passphrase>XXXXXXXXXXXX</passphrase>
      <filePermissions>775</filePermissions>
      <directoryPermissions>775</directoryPermissions>
      <privateKey>/Users/thoms/.ssh/id_rsa</privateKey>
    </server>
</servers>

Posted via email from Karsten’s Blog

Building TMF Xtext projects with Maven

Normally Xtext projects are built within the Eclipse IDE. But what do you do if the requirement is to have server side builds for for Xtext based projects, including the Grammar project itself, and of course without checking in generated sources? Setting up a working build process for Eclipse plugin projects is often a hell. I often face this requirement, and this is where Maven is a suitable alternative. This article explains how you can build Xtext based projects with Maven.

At first this requires that you have Maven installed on your machine and that the mvn command is on your path. Type on your shell

mvn -version

and it will answer something like

Maven version: 2.0.10
Java version: 1.6.0_07
OS name: "mac os x" version: "10.5.7" arch: "x86_64" Family: "mac"

This is all you basically need.

Xtext projects consist of

  1. A grammar project
  2. An UI project
  3. A generator project

The aim is to run the generator, which of course requires that at least the grammar project is built. We will also build the UI project, but it won’t result into a deployable plugin yet. I plan to investigate on this later. For the moment the projects are just compiled, packaged, and sources generated.

For this article I created projects using the Xtext project wizard, which creates by default the projects

  1. org.xtext.example.mydsl
  2. org.xtext.example.mydsl.ui
  3. org.xtext.example.mydsl.generator

Additionaly a further project was added:

  • org.xtext.example.mydsl.parent.

All projects can be downloaded here as Zip. Your project structure will look like this:

Bild 17

Maven Repositories

The most important thing for Maven based builds is that every artifact and plugin required must be available in a Maven Repository.

This is where the story begins, because Eclipse itself has no Maven repository, and thus every artifact (i.e. plugin jar) required within the build must be made available in a Maven Repository. This is at the moment maintained manually. I just deployed all required artifacts to the Maven Repository at openArchitectureWare.org. It contains now the latest 0.7.1 build of TMF Xtext, M2T Xpand and EMFT MWE.

Next repository we need is the one where to get Maven plugins from. Maven itself has just a small kernel and all features are added by plugins. Most of them are quite common and central available (e.g. Compiler Plugin), others are available in project specific repositories. For running Xpand based code generators the Fornax oAW Maven plugin is used, which is available through the Fornax Maven Repository.

In this example we make us of a small plugin (Replacer Plugin) which is available at the JBoss Maven Repository.

Last but not least the central Maven repository will be used to fetch some common artifacts that are not available in both of them. So we need the following repositories:

POM Files

Maven builds are described by Project Object Models (POM). The POMs are stored in pom.xml files for each subproject. These files are basically the build scripts for the projects and this is where all the magic goes in. Mainly it is “just” adding the right POMs to the projects that enable your projects to be built by Maven. This is totally non invasive. For more information about POMs read the Maven POM Reference.

In this article I show just excerpts from the POMs, so make sure to get the sources. Gaps in the POM are marked with “…”.

Parent Project

The org.xtext.example.parent project is the so called “parent project”. The parent’s POM is parent to all POMs from the submodules. The parent pom.xml

  • aggregates all subprojects as modules (Multi-module project)
  • declares common used Maven plugins
  • declares Maven Repositories to use

Module aggregation

We want to build all submodules when running the build on the parent project. This is done by declaring modules by relative paths. Since the projects are organized within an Eclipse workspace all projects including the parent are on the same level.

<modules>
  <module>../org.xtext.example.mydsl</module>
  <module>../org.xtext.example.mydsl.generator</module>
  <module>../org.xtext.example.mydsl.ui</module>
</modules>

Repository configuration

The required repositories are declared within a profile named “RepositoryManagement”, which is activated by default.

<profiles>
 <profile>
  <activation>
   <activeByDefault>true</activeByDefault>
  </activation>
  <id>RepositoryManagement</id>
  <repositories>
   <repository>
    <id>releases.openarchitectureware.org</id>
    <name>openArchitectureWare Release Repository</name>
    <url>http://www.openarchitectureware.org/m2</url>
    <releases>
     <enabled>true</enabled>
    </releases>
    <snapshots>
     <enabled>false</enabled>
    </snapshots>
   </repository>
  </repositories>
  <pluginRepositories>
   <pluginRepository>
    <id>releases.archiva.fornax-platform.org</id>
    <name>Archiva Managed Release Repository</name>
    <url>http://www.fornax-platform.org/archiva/repository/releases</url>
    <releases>
     <enabled>true</enabled>
    </releases>
    <snapshots>
     <enabled>false</enabled>
    </snapshots>
   </pluginRepository>
   ...
  </pluginRepositories>
 </profile>
</profiles>

Plugin configuration

Maven plugins are configured in the build section of a POM. Xtext projects require at least Java 1.5, so the Maven compiler plugin is configured so.

<build>
 ...
 <plugins>
  <plugin>
   <groupId>org.apache.maven.plugins</groupId>
   <artifactId>maven-compiler-plugin</artifactId>
   <configuration>
    <source>1.5</source>
    <target>1.5</target>
   </configuration>
  </plugin>
...
 </plugins>
</build>

Grammar Project

The Grammar project org.xtext.example.mydsl contains a workflow that runs the Xtext generator to create the Xtext implementation artifacts. By default the file is GenerateMyDsl.mwe in package org.xtext.example.

Bild 18

When running a build this generator workflow should be executed, generating the Xtext sources. Afterwards the sources should be compiled.

Project parent and basic descriptor tags

The project’s parent is declared by groupId, artifactId and version in the <parent> section. Since groupId and version should always be the same as the parent they are just inherited. This way the must only declared once.

<parent>
  <groupId>org.xtext.example.mydsl</groupId>
  <artifactId>mydsl-parent</artifactId>
  <version>1.0.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>mydsl-grammar</artifactId>
<name>TMF Xtext Example - Grammar</name>

Dependencies

For generating the sources and compilation the Grammar project needs just 2 dependencies:

<dependencies>
  <dependency>
    <groupId>org.eclipse.xtext</groupId>
    <artifactId>xtext-generator</artifactId>
    <version>0.7.1</version>
  </dependency>
  <dependency>
    <groupId>de.itemis.xtext</groupId>
    <artifactId>antlr</artifactId>
    <version>0.7.1</version>
  </dependency>
</dependencies>

All other required dependencies are included by Maven’s transitive dependency management.

Alternative Source/Resource paths

Xtext projects don’t follow Maven’s default paths for Java sources and resources. By default Maven will compile Java sources only from the src/main/java folder and will add src/main/resources. For Xtext projects there are two source folders, /src and /src-gen. To compile both an additional plugin from Codehaus, the Build Helper Maven Plugin, is needed.

<build>
  ...
  <sourceDirectory>src</sourceDirectory>
  <resources>
    <resource>
      <directory>src</directory>
    </resource>
    <resource>
      <directory>src-gen</directory>
    </resource>
  </resources>
  <plugins>
    <plugin>
      <groupId>org.codehaus.mojo</groupId>
      <artifactId>build-helper-maven-plugin</artifactId>
      <version>1.3</version>
      <executions>
        <execution>
          <id>add-source</id>
          <phase>generate-sources</phase>
          <goals>
            <goal>add-source</goal>
          </goals>
          <configuration>
            <sources>
              <source>src-gen</source>
            </sources>
          </configuration>
        </execution>
      </executions>
    </plugin>
    ...
  </plugins>
</build>

Workflow execution

To execute the MWE workflow we need the Fornax oAW Maven Plugin. Maven builds have a lifecycle, where predefined lifecycles exists. The oAW Maven plugin must be executed in the generate-sources phase. Since version 3.0.0 the Fornax plugin supports both workflow engines, oAW 4 Workflow Engine and oAW 5 MWE. Current version is 3.0.1.

To enable execution of MWE workflows the configuration section must be configured with <workflowEngine>mwe</workflowEngine>. The <workflowDescriptor> parameter specifies the path to the workflow file.

<build>
  ...
  <plugins>
    <plugin>
       <groupId>org.fornax.toolsupport</groupId>
       <artifactId>fornax-oaw-m2-plugin</artifactId>
       <version>3.0.1</version>
       <configuration>
         <outletSrcDir>src-gen</outletSrcDir>
         <outletSrcOnceDir>src</outletSrcOnceDir>
       </configuration>
       <executions>
         <execution>
           <phase>generate-sources</phase>
           <goals>
             <goal>run-workflow</goal>
           </goals>
           <configuration>
             <workflowDescriptor>org/xtext/example/GenerateMyDsl.mwe</workflowDescriptor>
             <workflowEngine>mwe</workflowEngine>
           </configuration>
         </execution>
       </executions>
     </plugin>
   </plugins>
   ...
</build>

UI Project

Configuring the POM for the UI project org.xtext.example.mydsl.ui is pretty similar to the Grammar project. In this project even no workflow execution is required, just source paths must be adjusted to /src and /src-gen (see above, Grammar project).

Dependencies

The UI plugin has 2 dependencies:

  1. The Grammar project
  2. org.eclipse.xtext:xtext-ui-common
<dependencies>
 <dependency>
  <groupId>${project.groupId}</groupId>
  <artifactId>mydsl-grammar</artifactId>
  <version>${project.version}</version>
 </dependency>

 <dependency>
  <groupId>org.eclipse.xtext</groupId>
  <artifactId>xtext-ui-common</artifactId>
  <version>0.7.1</version>
 </dependency>
</dependencies>

Generator Project

The POM for the Generator project again is similar to the Grammar project. This plugin of course executes a workflow during the build, so the Fornax oAW Maven plugin needs to be configured again. Again the source paths must be configured in the build section, and the Build Helper plugin used. See the Grammar project for this configuration.

Dependencies

The Generator project needs also just 2 dependencies:

  1. The Grammar project
  2. org.eclipse.xtext:xtext-util

All other dependencies come in transitive.

<dependencies>
  <dependency>
   <groupId>${pom.parent.groupId}</groupId>
   <artifactId>mydsl-grammar</artifactId>
   <version>${pom.parent.version}</version>
  </dependency>
  <dependency>
   <groupId>org.eclipse.xtext</groupId>
   <artifactId>xtext-util</artifactId>
   <version>0.7.1</version>
  </dependency>
</dependencies>

Workflow execution

The configuration of the oAW Maven plugin is similar to the Grammar project. Of course the path of the workflow file differs. To avoid unneccessary generator execution the plugin can be configured to just run the workflow when specified resources change. This is done through the checkResources section. In this case the generator will only run when the model file src/model/MyModel.dsl changes.

<build>
 ...
 <plugins>
  <plugin>
   <groupId>org.fornax.toolsupport</groupId>
   <artifactId>fornax-oaw-m2-plugin</artifactId>
   <version>3.0.1</version>
   <configuration>
    <checkResources>
     <checkResource>
      src/model/MyModel.mydsl
    </checkResource>
   </checkResources>
   <outletSrcDir>src-gen</outletSrcDir>
  </configuration>
  <executions>
   <execution>
    <phase>generate-sources</phase>
    <goals>
     <goal>run-workflow</goal>
    </goals>
    <configuration>
     <workflowDescriptor>workflow/MyDslGenerator.mwe</workflowDescriptor>
     <workflowEngine>mwe</workflowEngine>
    </configuration>
   </execution>
  </executions>
 </plugin>
 ...
 </plugins>
</build>

Running the build

Now it is time to build the projects with Maven. Note that we only added the described pom.xml files to the projects that the Xtext Wizard created.

Open a shell, go to the org.xtext.example.parent folder and type

mvn clean install

The Maven build starts…

[INFO] Scanning for projects...
[INFO] Reactor build order:
[INFO]   TMF Xtext Example - Parent
[INFO]   TMF Xtext Example - Grammar
[INFO]   TMF Xtext Example - Generator
[INFO]   TMF Xtext Example - UI
[INFO] ------------------------------------------------------------------------
[INFO] Building TMF Xtext Example - Parent
[INFO]    task-segment: [clean, install]
[INFO] ------------------------------------------------------------------------

These are the first log statements that the build prints out. You can see from here in which order Maven will build the projects. First the parent project will be built, followed by Grammar project, Generator project and UI project.

Downloading artifacts

When you start with a clean Maven installation Maven will download all plugins and artifacts required for the build.

...
Downloading: http://www.openarchitectureware.org/m2/org/eclipse/emf/ecore/xmi/2.5.0/xmi-2.5.0.pom
583b downloaded
Downloading: http://www.openarchitectureware.org/m2/org/eclipse/m2t/xpand/m2t-xpand-xpand/0.7.1/m2t-xpand-xpand-0.7.1.pom
1K downloaded
Downloading: http://www.openarchitectureware.org/m2/org/eclipse/m2t/xpand/m2t-xpand/0.7.1/m2t-xpand-0.7.1.pom
3K downloaded
Downloading: http://www.openarchitectureware.org/m2/org/eclipse/m2t/xpand/m2t-xpand-xtend/0.7.0/m2t-xpand-xtend-0.7.0.pom
1K downloaded
Downloading: http://www.openarchitectureware.org/m2/org/eclipse/m2t/xpand/m2t-xpand/0.7.0/m2t-xpand-0.7.0.pom
3K downloaded
Downloading: http://www.openarchitectureware.org/m2/org/eclipse/emf/mwe/emf-mwe/0.7.0/emf-mwe-0.7.0.pom
24K downloaded
Downloading: http://www.openarchitectureware.org/m2/org/eclipse/emf/mwe/emf-mwe-core/0.7.0/emf-mwe-core-0.7.0.pom
2K downloaded
...

This is already a lot! All of the downloaded artifacts will be stored in your local repository, which is usually located at ~/.m2/repository. Once the artifacts are downloaded you don’t need to access the internet again. You can than go to offline mode by adding the -o option:

mvn -o clean install

You may need to configure a proxy to access the public Maven repositories. Read “Configuring a proxy” to get more information.

Generating the Xtext artifacts

When running the build on the Grammar project the artifacts from the Xtext grammar file are generated. You will see that the Fornax oAW Maven plugin starts execution and the Xtext generator is run.

[INFO] [fornax-oaw-m2:run-workflow {execution: default}]
[INFO] Fornax oAW/MWE Maven2 Plugin V3.0.1
0    [main] INFO  eclipse.emf.mwe.core.WorkflowRunner  - --------------------------------------------------------------------------------------
174  [main] INFO  eclipse.emf.mwe.core.WorkflowRunner  - EMF Modeling Workflow Engine 0.7.1, Build v200907170432
174  [main] INFO  eclipse.emf.mwe.core.WorkflowRunner  - (c) 2005-2009 openarchitectureware.org and contributors
174  [main] INFO  eclipse.emf.mwe.core.WorkflowRunner  - --------------------------------------------------------------------------------------
175  [main] INFO  eclipse.emf.mwe.core.WorkflowRunner  - running workflow: org/xtext/example/GenerateMyDsl.mwe
175  [main] INFO  eclipse.emf.mwe.core.WorkflowRunner  -
882  [main] INFO  lipse.emf.mwe.utils.StandaloneSetup  - Registering platform uri '/Users/thoms/Development/workspaces/oaw-v5-test'
2794 [main] INFO  e.core.container.CompositeComponent  - DirectoryCleaner: cleaning directory '../org.xtext.example.mydsl/src-gen'
2795 [main] INFO  ipse.emf.mwe.utils.DirectoryCleaner  - Cleaning /Users/thoms/Development/workspaces/oaw-v5-test/org.xtext.example.mydsl/../org.xtext.example.mydsl/src-gen
2820 [main] INFO  e.core.container.CompositeComponent  - DirectoryCleaner: cleaning directory '../org.xtext.example.mydsl.ui/src-gen'
2820 [main] INFO  ipse.emf.mwe.utils.DirectoryCleaner  - Cleaning /Users/thoms/Development/workspaces/oaw-v5-test/org.xtext.example.mydsl/../org.xtext.example.mydsl.ui/src-gen
2824 [main] INFO  e.core.container.CompositeComponent  - Generator
2929 [main] INFO  ipse.xtext.generator.LanguageConfig  - generating infrastructure for org.xtext.example.MyDsl with fragments : ImplicitRuntimeFragment, ImplicitUiFragment, GrammarAccessFragment, EcoreGeneratorFragment, ParseTreeConstructorFragment, ResourceFactoryFragment, AntlrDelegatingFragment, JavaValidatorFragment, JavaScopingFragment, FormatterFragment, LabelProviderFragment, TransformerFragment, OutlineNodeAdapterFactoryFragment, JavaBasedContentAssistFragment, DelegatingGeneratorFragment

Done!

When the build is through you will get the build summary. All generators are run, sources are compiled, jar files are created and installed into your local repository.

...
...
[INFO] Installing /Users/thoms/Development/workspaces/oaw-v5-test/org.xtext.example.mydsl.ui/target/org.xtext.example.mydsl.mydsl-ui_1.0.0-SNAPSHOT.jar to /tmp/maven-rep/org/xtext/example/mydsl/mydsl-ui/1.0.0-SNAPSHOT/mydsl-ui-1.0.0-SNAPSHOT.jar
[INFO]
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] ------------------------------------------------------------------------
[INFO] TMF Xtext Example - Parent ............................ SUCCESS [4.743s]
[INFO] TMF Xtext Example - Grammar ........................... SUCCESS [24.037s]
[INFO] TMF Xtext Example - Generator ......................... SUCCESS [2.600s]
[INFO] TMF Xtext Example - UI ................................ SUCCESS [2.766s]
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 34 seconds
[INFO] Finished at: Mon Jul 20 15:07:14 CEST 2009
[INFO] Final Memory: 67M/160M
[INFO] ------------------------------------------------------------------------

Of course on first execution this will take much longer, since every needed dependency is downloaded. On second execution no download is needed anymore.

oAW 4 migration with cartridge adaption

In one of my projects we make use of the Fornax oAW 4 cartridges Hibernate and JavaBasic. This week we migrated our generator projects to Eclipse Galileo (we use the EPP Modeling Package at the moment) with MWE, M2T Xpand and TMF Xtext. (Note: I’ll name this package here oAW 5 although it does not exist yet, but this is another topic).

Now these cartridges are not available for oAW 5 yet and will not in the near future. So we have these alternatives:

  • migrate the cartridges
  • replace the cartridges by a equivalent, project specific cartridge which takes the same input and produces the same output
  • adapt the cartridges and leave them as is

We decided to try adapting it. Now the problem is that the cartridges are oAW 4 specific and won’t work without changes. We faced some problems, and luckily we were able to solve them. Some are quite hacks ;-)

Dependencies

The required plugins have changed. With oAW 4.3.1 only the aggregator plugin org.openarchitectureware.dependencies and org.openarchitectureware.uml2.adapter were required dependencies. In our generator project we now have these dependencies (besides project specific ones):
org.eclipse.xtend.typesystem.uml2;bundle-version="0.7.0",
org.eclipse.xtend.util.stdlib;bundle-version="0.7.0",
org.eclipse.xpand;bundle-version="0.7.0",
org.eclipse.xtend;bundle-version="0.7.0",
org.eclipse.jface.text;bundle-version="3.5.0",
org.eclipse.xtend.typesystem.emf;bundle-version="0.7.0",
org.eclipse.core.runtime;bundle-version="3.5.0",
org.eclipse.jdt.core;bundle-version="3.5.0"

Workflow

The standard workflow is packaged into the cartridge jars and cannot be changed. The namespaces of the used components has changed. Looking up the component class names with CTRL+SHIFT+T (Open Type) quickly revealed the package names the workflow components have now.
Screenshot
Configuration of the components did not change.

Tipp: if you want to know which workflow component are available then open the interface org.eclipse.emf.mwe.WorkflowComponent and press F4. This opens the type hierarchy and allows you to browse and inspect all known WorkflowComponent implementations. The most interesting ones are derived from AbstractWorkflowComponent2.

Bild 15

Loading profiles

Formerly profiles were just located by classpath. We had the Persistence.profile.uml in folder model, which was on classpath root. This profile could not be loaded anymore. The path was changed to a platform resource URI (platform:/resource/PROJECT/model/Persistence.profile.uml).

With the UML profile file from Fornax we had the problem that when instantiating the ProfileMetaModel instance with that file our generator process hang. It turned out that it was tried to open a HTTP connection to load “http://www.eclipse.org/uml2/2.1.0/UML“, which is declared as xmlns:uml in the model file. Eclipse did not have a internet connection nor should it require. The profile was loaded in MagicDraw 16 and reexported, and the file changed to xmlns:uml="http://www.eclipse.org/uml2/2.0.0/UML". After doing so the profile could be loaded with ProfileMetaModel. Since we use Galileo, the installed UML2 plugins have version 3.0.0.

Stdlib Extensions

The cartridges make use of the oAW’s stdlib extensions. The extensions exist unchanged, but now can be found in the plugin org.eclipse.xtend.util.stdlib with the same package name. So the namespace of those extensions changed and workflow execution failed as follows:


ERROR - [ERROR]: Unable to load extension file : org::openarchitectureware::util::stdlib::io(Element: EvaluationException : Unable to load extension file : org::openarchitectureware::util::stdlib::io
org::fornax::cartridges::uml2::hibernate::checks::CartridgeProperties.chk[967,28] on line 27 'getHibernatePropertiesFile()'
; Reported by: -UNKNOWN-)
EvaluationException : Unable to load extension file : org::openarchitectureware::util::stdlib::io
org::fornax::cartridges::uml2::hibernate::checks::CartridgeProperties.chk[967,28] on line 27 'getHibernatePropertiesFile()'
at org.eclipse.internal.xtend.expression.ast.Expression.evaluate(Expression.java:56)

To solve this we added extension files with the same name as the old stdlib file and same package, and just reexport the new stdlib extensions. For example, for the IO extensions a file org/openarchitectureware/util/stdlib/io.ext was created with the following content:

extension org::eclipse::xtend::util::stdlib::io reexport;

Dieter Moroff had already the idea to provide a stdlib compatibility package that contain all oAW 4 stdlib extensions mapped this way. I think it would be great for all users to have that, it would make migrations more smooth. Maybe I find some time to realize it. Main question would be where to host this plugin. I think the sources should be kept at Sourceforge in the oAW 4 repository, and the plugin should be available through an update site on openArchitectureWare.org.

In our own templates that we have under development we just replaced the usage of org::openarchitectureware::util::stdlib... by org::eclipse::xtend::util::stdlib...

PLE feature

The cartridges make use of the org.openarchitectureware.util.featureconfig plugin, which has not been migrated yet. We had to replace the behavior. The cartridges use this plugin mainly to conditionally execute workflow components. Since we had to replace the workflow files anyway we changed all <feature isSelected="FEATURENAME">...</feature> occurances by <if cond="${FEATURENAME}">...</if>, which has the same behavior, but configuration of the features is done by workflow properties and <if> is a built-in feature of the workflow engine. The if-components were not documented in the MWE reference yet, so we filed a bug which already has been fixed now.
The cartridge makes also use of the features.ext extensions, which defines a function isFeatureSelected(). We replaced the behavior by using the stdlib properties extensions. Therefore we configured our features in a properties file and read them with the PropertiesReader component.
<component>
<propertiesFile value="featureconfig.properties"/>
</component>

We created a file org/openarchitectureware/util/featureconfig/features.ext:

extension org::eclipse::xtend::util::stdlib::properties;

Boolean isFeatureSelected (String feature) : getProperty(feature).trim()=="true";

Changed namespace of internal types

oAW has a built-in typesystem (with types Type, Operation etc.) whose types were in oAW 4 in the namespace oaw. So if you have the expression uml::Class.metaType.name the result would be oaw::Type. With Eclipse M2T the namespace of those types changed to xpand2, so oaw::Type is now xpand2::Type. Normally you don’t work with the internal types directly until you do some reflection stuff, but extensions in the Fornax JavaBasic cartridge make use of those types. This leads to errors like this already when an extension is loaded and analyzed:

ERROR - Error in Component hibernateCartridge_transformations of type org.eclipse.xtend.XtendComponent:
EvaluationException : Couldn't resolve type for 'oaw::Type'. Did you forget to configure the corresponding metamodel?
org::fornax::cartridges::uml2::javabasic::extensions::DataType.ext[5047,76] on line 135 'boolean typeOf(uml::Type type1,oaw::Type type2)'
nofile[0,26] on line 1 'createDefaultValues(model)'

ERROR – Workflow interrupted. Reason: Couldn’t resolve type for ‘oaw::Type’. Did you forget to configure the corresponding metamodel?

To work around this the extension files need to be replaced. Since they are packaged in a jar it is required to copy them and make them available through the classpath before the cartridge. We use the approach to have a compatibility plugin which contains this .ext file as well as the stdlib compatibility extensions.

Problems with UML2 typesystem

We experienced some severe problems with the org.eclipse.xtend.typesystem.uml2 plugin that lead to EvaluationExceptions through our generator process. Basically the plugin is just a migrated version of the org.openarchitectureware.uml2.adapter plugin, but the behavior of the underlying EMF typesystem changed slightly but significant. It was not possible to handle different typesystems at the same time it (e.g. for M2M transformation), which blocked us. We opened Bug#283469 and hunted it down. Some other issues with the typesystem were detected and fixed. A patched plugin you can download from here. We created a folder dropins/oaw5-patch/plugins and put it there until the official service release will be available.
As a positive side effect it is not necessary anymore to register an UML2MetaModel nor EmfRegistryMetaModel instance in the workflow when a ProfileMetaModel is used, since it will delegate to them internally.

Hybridlabs Beautifier

The cartridges make use of the Hybridlabs Beautifier 1.1.9, which is only compatible with oAW 4. The new version 2.0.0 is now able to work also with M2T Xpand.

Conclusion

Our projects including cartridge adaption and bugfixing were migrated in about 4 days. The scenario is a real-world case, and some issues (esp. the mentioned bugs) we faced required deep knowledge about the implementation. I think most projects can be migrated even easier now, but if you face problems it might be worth ordering expert help before spending too much time. Experienced users of oAW 4 will not have much problems migrating the Xpand based code generators.

Founded Google Code project for the Hybridlabs Beautifier

The Hybridlabs beautifier has become a famous component for oAW. It is a Java code formatter based on the Jalopy library and what it makes it especially interesting is its feature of auto-organizing Java imports. Since M2T is now upcoming users ask for porting the component to M2T Xpand.

I have asked Karsten Klein, the founder of the Hybridlabs beautifier, to host the projectat a public site and join development efforts. Karsten is at the moment too busy on his normal projects to work on sparetime projects like his beautifier. So I got agreement from Karsten to support him founding the project.

Yesterday I created the Hybridlabs Beautifier project at Google Code. The project is now public, so it is not necessary to send patches and feature requests to Karsten per Mail, better use the given Infrastructure of the project from now on. I took the sources from version 1.1.9, set up the project structure and provided a Maven build for the project. I decided to make to increment the version here to 2.0.0, since project organization changed and M2T support was added.

Internally the code change is really small. The main component, JavaImportBeautifier, now imports both PostProcessor interfaces, from oAW4 Xpand2 and from Eclipse M2T Xpand. This way the component could kept backward compatible to oAW4 while at the same time being compatible for M2T Xpand. A good example why it makes sense to program against interfaces. A drawback of this approach is that this component has now a class dependency to both, which makes it necessary to have oAW4 Xpand on the classpath when using the component within M2T. I excluded all transitive dependencies for oaw-core-xpand2, so dependencies are rather small.

As a result I have created a bundle of the current build including dependencies and uploaded it. This bundle can be downloaded here.