Karsten's Blog

April 5, 2013

Preparations for CodeGeneration 2013

Filed under: MDSD, Xtext — Tags: , , , , , — kthoms @ 10:44 AM

As I am addicted to code generation and DSLs, the CodeGeneration conference in Cambridge is always a must each year. Last year I could not make it, since I had the chance to speak at EclipseCon North America, which was in the same week. This year Mark took EclipseCon into his considerations (it was last week), so me and my colleagues from itemis will be there again. Actually, this year we will be more itemis guys then ever. Mark already assumed in his opening words at CG2011 that almost everyone from itemis would be there, this year we prove itemis is larger. I think our company is so close related to the conference theme that it is natural that we have lots to present, and much interest to hear others about what they are doing and have learned in the past.

screenshot 2013-04-05 um 10.55.19

Before the actual CodeGeneration conference starts on Wednesday, there are some pre-conference activities. My colleagues Holger Schill and Moritz Eysholdt will hold an intensive 2-day Xtext workshop on monday and tuesday.

I will arrive monday noon, since I take the early flight directly from my hometown Dortmund to London Luton. From there, I have to take a 2 hour bus trip to Cambridge. In the evening, I plan to meet Holger, Moritz, Meinte Boersma and hopefully some others in the Castle Inn pub. When you arrive on monday, drop into the Castle Inn roughly at 20 PM (I guess we go to a restaurant before). You can reach me there on my mobile phone: screenshot 2013-04-05 um 12.30.02

DSC00412

On tuesday this year’s Language Workbench Challenge summit takes place. We have 14 submissions (wow!) for the LWC13 assignment. I have been working on the Xtext submission together with 2 colleagues, Johannes Dicks and Thomas Kutz. The results are available as open-source project lwc13-xtext at Eclipselabs. We have prepared a detailed step-by-step tutorial as submission paper. The resulting document LWC13-XtextSubmission.pdf is available for download. On the project homepage I have placed today a quick start tutorial. Oh boy, this project did cost some time. The actual solution is not much code, but as often, it is harder to write less code than more. It could be even less, but we took care of that the code is readable and understandable. And writing the document is at least that much work as the implementation.

screenshot 2013-04-08 um 13.52.06

Every presenter has only 15 minutes to present their approach. 15 minutes presentation for that much work. I guess that the other participants did invest also quite some time. Both of my co-authors got the chance to visit the conference, and Thomas will support me with the presentation. He will demo the resulting JSF application and DSL source code while I do the main talking. We did a test run of the talk yesterday evening, and easily exceeded 20 minutes. I think Angelo will bring his egg timer again, which begs no pardon with the talk time of speakers. But only that way we will be able to run 14 talks on one day. We will have to restrict on the most important aspects only.

Besides my colleagues Thomas and Johannes also Sven Lange will join us then. I have the pleasure to work with him in my long-time project at Deutsche Börse (German Stock Exchange), now since over half a year. Sven is a highly motivated, skilled and smart person. It is still the same project I have reported about at CodeGeneration 2009 together with my former colleague Heiko Behrens. Sven is full-time working on this project, while I am for 20% scheduled. We have migrated here a huge code generator project from Xpand to Xtend. This alone would be worth an experience report session. Sven is working on Xtend support for IntelliJ, which he might present in a Lightning Talk on wednesday.

Wednesday the conference will start. I will have my main talk “Alive and Kicking with Code Generation” together with Dr. Boris Baginski from ATOSS Software AG after lunch at 13:45.

screenshot 2013-04-05 um 10.57.34

Currently we are finalizing our presentation slides. Boris has been ill for some days and busy with a new release of their ASES product, a workforce management suite. This is really an interesting customer and project. They are evolving this product now for 25 years, and they make use of code generation for ages. I think one can say that it helped them to survive in their business, some contestants did not manage to make larger platform shifts and died. Most of them tried a big-bang replacement, but the business is too fast evolving so that the target is moving steadily. Boris and I will speak about this product and how it has been evolved over the years. ATOSS was one of the first major projects using openArchitectureWare 4 (which mainly means Xpand), and now they are currently preparing a shift to Xtend.

I am glad that this talk is already on wednesday, I never come to rest until I finished some talk. After it, I can just relax and enjoy the conference. I am expecting some interesting insights on different approaches. Especially experience reports are interesting for me. I did not finally decide which sessions I will attend. At the moment I plan to see John Hutchinson with “The Use of Model-Driven Development in Industry” in the morning, and Darius Silingas with “Why MDA Fails: Analysis of Unsuccessful Cases” in the afternoon.

In the evening it is again time for the punting boat tour. I already attended three times, but it will be great fun again for sure. Let’s hope the weather is not too bad. I saw a prediction of ~10°C and possibility of light shower. In the past we had luck, and on a warm, sunny day the tour is double fun. However, I’ll better put an umbrella into the suitcase.

On thursday I have again an active part in the hands-on session “Have Your Language Built While You Wait”, which is hosted by Risto Pohjonen from MetaCase. The idea of this session is that attendees can get a DSL with the language workbench of their choice built with the help of experts for this workbench. Of course I will assist on Xtext. If you had no chance to visit the Xtext workshop this might be your chance to get some hands on Xtext. This session was already run last year successfully. Last year my colleague Benjamin Schwertfeger took over the Xtext part, since we were at EclipseCon.

There are also some other talks around Xtext and Xtend. Both have been released in version 2.4 on March 20th, which brings some interesting new features. Most notably in regard to code generation are the Active Annotations. I guess this is also part of what Sven Efftinge will adress as future of code generation in his keynote “The Past, Present and Future of Code Generation” wednesday morning. More details he will present together with Sebastian Zarnekow in the tutorial “Internal DSLs with Xtend” (thursday 10:45-12:00). The last Xtext related talk will be from Moritz Eysholdt, called “Executable Specifications for Xtext Languages” (friday 10:45-12:15). I am actually not sure which of these talks I will attend personally. They are most relevant for my work, and I don’t work close enough with them to catch everything new in Xtext on my own. Thus, I’ll definetely would learn important aspects. On the other side, there are also other interesting talks in parallel.

The coming week will be an intensive experience with lots to learn and interesting persons to meet. Although I will really enjoy this time, I will be glad when I finally come back home. At the moment, my family is ill and I hope that I get not infected these days. I have been looking forward and worked for this event, so I am crossing fingers when I can board monday morning healthy.

I am sure the organizing team around Mark and Jacqui will do again a great job.

DSC00363

See you there and let’s make this event special!

July 7, 2011

Game Development – A whole new area for Eclipse Modeling?

Filed under: Eclipse, MDSD, Uncategorized, Xtext — Tags: , , — kthoms @ 3:15 PM

The Eclipse Modeling Project provides the world’s leading set of tools and frameworks that are used for successfully applying model driven software development techniques in various areas. Successful adoption are known in Enterprise Computing, Embedded System Development, Mobile Development etc. But what about Game Development? I have not heard about Game productions that use Eclipse Modeling or Model Driven Software Development in general so far. I cannot know about all projects in the world, but at least it is an indicator that this development technique is at least not wide adopted in the branch of Game Development.

Game Development is highly complex, developed in multidisciplinary teams under high time pressure and quality requirements. And the complexity is even growing, whilst time pressure also. Time-to-market is everything there. If your game comes too late, you are out. If you don’t use the latest technologies, you are lame. How could such projects ever be successful just by coding and hacking? I could imagine that game developers are just too busy with developing their games in a traditional way to think how they could gain speed and quality by applying software engineering techniques like MDSD.

I would not wonder if they associate MDSD with drawing UML diagrams and wasting time clicking and drawing useless stuff. Model Driven Software Development is everything else than useless. It helps raising the level of abstraction, speeding up development and gaining quality. If applied correctly, of course. Of course they think their kind of software development is special and completely different than other disciplines. But let me say, it’s not the case. Every piece of software has generic parts, schematic parts and parts that don’t fit into one of the previous sections. And for the schematic parts, MDSD can always help. Don’t tell me that a multi-million, mission-critical enterprise project is less challenging than game development.

One of the most promising things for game development can be the usage of Domain Specific Languages (DSLs), especially textual ones. With Xtext 2.0 the development of textual DSLs with tight integration of expression languages and code generators has become easier than ever before. If you don’t ever tried Xtext, do it!

On October 4th there will be an interesting workshop at the 10th International Conference on Entertaining Computing (ICEC 2011) in Vancouver, Canada. The “1st Workshop on Game Development and MDSD” will bring experts from both worlds together. The Call for Position Papers is running now, deadline is July 30th. If you are in one of those businesses, submit a propopsal or attend the workshop. I think both “worlds” can really win a lot from working together. This workshop could be a good start.

August 19, 2010

Tycho build works also for oAW 4.3 projects

Filed under: Maven, openArchitectureWare, Xpand — kthoms @ 10:16 PM

Yesterday I showed how Xtext 1.0 projects can be build with Maven Tycho. So why shouldn’t the same setup don’t work for ‘old’ openArchitectureWare Xpand projects? This works like a charm. I took the simple project that the Xpand project wizard creates and added the POM, and it worked! You can download the sample project here. See the POM below, a setup for other oAW Xpand projects should be rather similar.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>p2.osgi.bundle</groupId>
	<artifactId>org.openarchitectureware.example.xpand.tycho</artifactId>
	<version>1.0.0</version>
	<packaging>eclipse-plugin</packaging>
	<name>org.openarchitectureware.example.xpand.tycho</name>

	<properties>
		<version.tycho>0.9.0</version.tycho>
	</properties>

	<build>
		<resources>
			<resource>
				<directory>src</directory>
			</resource>
		</resources>
		<plugins>
			<plugin>
				<groupId>org.sonatype.tycho</groupId>
				<artifactId>tycho-maven-plugin</artifactId>
				<version>${version.tycho}</version>
				<extensions>true</extensions>
			</plugin>
			<plugin>
				<groupId>org.sonatype.tycho</groupId>
				<artifactId>target-platform-configuration</artifactId>
				<version>${version.tycho}</version>
				<configuration>
					<resolver>p2</resolver>
				</configuration>
			</plugin>
			<plugin>
				<groupId>org.fornax.toolsupport</groupId>
				<artifactId>fornax-oaw-m2-plugin</artifactId>
				<version>3.1.0-SNAPSHOT</version>
				<configuration>
					<workflowEngine>oaw</workflowEngine>
					<workflowDescriptor>workflow/generator.oaw</workflowDescriptor>
				</configuration>
				<executions>
					<execution>
						<phase>generate-sources</phase>
						<goals>
							<goal>run-workflow</goal>
						</goals>
					</execution>
				</executions>
			</plugin>
			<plugin>
				<artifactId>maven-clean-plugin</artifactId>
				<version>2.4.1</version>
				<configuration>
					<filesets>
						<fileset>
							<directory>src-gen</directory>
							<includes>
								<include>**/**</include>
							</includes>
						</fileset>
					</filesets>
				</configuration>
			</plugin>
		</plugins>
	</build>
	<repositories>

		<repository>
			<id>p2.openarchitectureware</id>
			<url>http://www.openarchitectureware.org/updatesite/milestone/</url>
			<layout>p2</layout>
		</repository>
		<repository>
			<id>p2.eclipse.helios</id>
			<url>http://download.eclipse.org/releases/helios</url>
			<layout>p2</layout>
		</repository>
		<!-- At the moment the Fornax plugin is only available as snapshot -->
		<repository>
			<id>fornax-snapshots</id>
			<url>http://www.fornax-platform.org/archiva/repository/snapshots/</url>
			<releases>
				<enabled>false</enabled>
			</releases>
			<snapshots>
				<enabled>true</enabled>
			</snapshots>
		</repository>
	</repositories>
	<pluginRepositories>
		<pluginRepository>
			<id>fornax-snapshots</id>
			<url>http://www.fornax-platform.org/archiva/repository/snapshots/</url>
			<releases>
				<enabled>false</enabled>
			</releases>
			<snapshots>
				<enabled>true</enabled>
			</snapshots>
		</pluginRepository>
	</pluginRepositories>
</project>

April 29, 2010

Mavenizing the project structure of Xtext projects

Filed under: Eclipse, Maven, MDSD, Xtext — Tags: , , , , — kthoms @ 4:23 PM

The structure of typical Xtext projects does not match the standard layout of Maven projects. Xtext projects more adhere to standard Eclipse project layout, which means

  • manual implemented Java sources are in folder /src
  • the plugin Manifest is /META-INF/MANIFEST.MF
  • generated sources go to /src-gen

In my customer’s project Xtext sources are built with Maven, and also the sources produced by Xtext are produced within the Maven build, using the Fornax Workflow Maven plugin. Until now we have adjusted the Maven build to match the standard Xtext project structure, which requires some configuration in the build section of the POM like follows:

   <build>
        <sourceDirectory>src</sourceDirectory>
        <resources>
            <resource>
                <directory>src</directory>
            </resource>
            <resource>
                <directory>src-gen</directory>
            </resource>
        </resources>
        <plugins>
            <plugin>
                <groupId>org.fornax.toolsupport</groupId>
                <artifactId>fornax-oaw-m2-plugin</artifactId>
                <version>3.0.3</version>
                <configuration>
                    <checkFilesets>
                        <checkFileset>
                            <directory>${basedir}</directory>
                            <includes>
                                <include>src/**</include>
                            </includes>
                            <excludes>
                                <exclude>**/.svn/**</exclude>
                            </excludes>
                        </checkFileset>
                    </checkFilesets>
                    <outletSrcDir>src-gen</outletSrcDir>
                    <outletSrcOnceDir>src</outletSrcOnceDir>
                    <outletResOnceDir>resources</outletResOnceDir>
                    <properties></properties>
                    <workflowDescriptor>xtext/example/GenerateMyDsl.mwe</workflowDescriptor>
                    <workflowEngine>mwe</workflowEngine>
                </configuration>
                <executions>
                    <execution>
                        <phase>generate-sources</phase>
                        <goals>
                            <goal>run-workflow</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>build-helper-maven-plugin</artifactId>
                <executions>
                    <execution>
                        <id>add-source</id>
                        <phase>generate-sources</phase>
                        <goals>
                            <goal>add-source</goal>
                        </goals>
                        <configuration>
                            <sources>
                                <source>src</source>
                            </sources>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

Now requirements changed and the project structure should conform to the Maven structure, which means:

  • src/main/java: Contains hand-written Java code
  • src/main/resources: All non-Java hand-written code (including Workflow and Xtext grammar)
  • target/generated/java: Contains generated code

Another requirement  from the build team is that the actual output directory ‘target’ should be configurable. This means mainly that we have to use properties that Maven uses to refer to the source and target directories (project.build.sourceDirectory and project.build.directory), so that the build could override just these settings by passing a system property and all output gets produced to and compiled from an alternative directory structure.

Of course this is possible with small changes, but you have to know where.

Xtext Generator Worklow

In order to produce the output of the Xtext generator to different directories than default (src, src-gen => src/main/java, target/generated/java) the MWE workflow file has to be changed. For the generator component we have to override the properties srcPath and srcGenPath. Further, these output directories should be parametrized by the same properties that Maven uses, namely project.build.directory and project.build.directory. These properties need to be configured in the Generate<MyDSL>.properties with defaults.

Generate<MyDSL>.mwe

<workflow>
  ..
  <component class="org.eclipse.xtext.generator.Generator">
    <pathRtProject value="${runtimeProject}"/>
    <pathUiProject value="${runtimeProject}.ui"/>
    <projectNameRt value="${projectName}"/>
    <projectNameUi value="${projectName}.ui"/>
    <srcPath value="/${project.build.sourceDirectory}"/>
    <srcGenPath value="/${project.build.directory}/generated/java"/>
    ..
  </component>
  ..
</workflow>
<pre>

Generate<MyDSL>.properties

basedir=.
project.build.directory=target
project.build.sourceDirectory=src/main/java
grammarURI=classpath:/.../MyDSL.xtext
file.extensions=...
projectName=...

pom.xml: build section

The Java source directories also contain non-Java resources, like the workflow, grammar file etc. Normally this would go to the resources directory for Maven projects, but Xtext (0.7.2) cannot be configured to produce resource files to another directory than Java sources. On the other side those resources need to be found on the classpath during the build. This requires that we add the Java directories as resource directories in the build section of the POM. This settings already existed before, it just has been adopted.

Before:

    <build>
        <sourceDirectory>src</sourceDirectory>
        <resources>
            <resource>
                <directory>src</directory>
            </resource>
            <resource>
                <directory>src-gen</directory>
            </resource>
        </resources>
        ...
   </build>

After:

    <build>
        <resources>
            <resource>
                <directory>${project.build.sourceDirectory}</directory>
            </resource>
            <resource>
                <directory>${project.build.directory}/generated/java</directory>
            </resource>
        </resources>
         ...
     </build>

The <sourceDirectory> entry is not necessary anymore, since the main Java source directory is now src/main/java, which follows the Maven standard directory layout and gets automatically compiled.

maven-clean-plugin

Xtext produces output into two projects – the grammar project and the UI project. Now if the generated UI sources are produced below /target calling ‘mvn clean install’ would have an undesired side effect. Maven builds both modules after each other, so when the UI module is built with the goals ‘clean install’ the target directory is removed and the previous generated sources get lost.

The solution is that the maven-clean-plugin must be deactivated for the UI module and the grammar module must clean the target directory for the UI module.

pom.xml – Grammar project

<plugin>
    <artifactId>maven-clean-plugin</artifactId>
    <configuration>
        <filesets>
            <fileset>
                <directory>../my.dsl.ui</directory>
                <includes>
                    <include>${project.build.directory}</include>
                </includes>
            </fileset>
        </filesets>
    </configuration>
</plugin>

pom.xml – UI project

<plugin>
    <artifactId>maven-clean-plugin</artifactId>
    <configuration>
        <skip>true</skip>
    </configuration>
</plugin>

Eclipse settings

The new project structure affects, of course, the project settings for Eclipse. The settings apply for the grammar and UI plugin.

build.properties

Change the source folders in the build.properties file:

source.. = src/main/java/,target/generated/java/
bin.includes = META-INF/,\
 .,\
 plugin.xml

Java build path

Open the project properties and change the  Java Build Path settings:

  • Source Folder src -> src/main/java
  • Source Folder src-gen -> target/generated/java
  • Default outputr folder: bin -> target/classes

February 26, 2010

Prototyping a Configuration Toolkit for Eclipse SMILA with Xtext

Filed under: Eclipse, MDSD — Tags: , , , , , — kthoms @ 2:06 PM

I got the chance to get a bit more familiar with Eclipse SMILA and started development of a configuration toolkit with Xtext. Target is to develop a prototype which enables an easier setup of a valid SMILA configuration by use of a textual DSL with all the benefits which you get from using such a DSL, like semantic validation, content assist etc. SMILA is configured by a bunch of XML files conforming to defined XSDs. Sometimes information is spread around different configuration files, and misconfiguration leads to runtime errors or even to no error at all.

But lets start with SMILA first…

What is SMILA about?

SMILA is an extensible framework for building search solutions to access unstructured information in the enterprise. Besides providing essential infrastructure components and services, SMILA also delivers ready-to-use add-on components, like connectors to most relevant data sources. Using the framework as their basis will enable developers to concentrate on the creation of higher value solutions, like semantic driven applications etc.

To give a rough imagination: You can configure different kinds of agents which search media for information (e.g. files, web pages etc.), and relevant data is extracted from those resources and published to some queue (ActiveMQ is used by default). Listeners react on entries and execute BPEL processes to process the information. Final goal is to index the data in stores, which can be searched by clients. Lucene is used by default as indexing engine.

Getting SMILA running

The SMILA project provides distributions for Windows and Linux. Since I’m working on a Mac I could not use them. So I followed the development guideline to setup a dev environment. In my fresh workspace I checked out first the trunk, but switched back to tag 0.5-M3 to have the same state as the distributions.

After finishing the checkout I finally was able to follow again the good 5 Minutes to Success tutorial. But don’t expect you can finish the tutorial in 5 minutes ;-) One word to mention: SMILA requires Java 6, and my development IDE is started by default with Java 5. So I needed to configure Java 6 for my target platform and also had to add the RCP delta pack, since 1.6 requires 64 bit libraries on Mac.

Contained in the sources is a example configuration project SMILA.application, which can be started by a launch configuration in the SMILA.launch project. Here is a small screenshot of the SMILA.application project structure.

The application contains several XML configuration files and their XSDs in a structure which reflects the plugins that are used. The tutorial explains small changes to the configuration and which files have to be changed, but for setting up a brand new project it might become more complicated if one is not familiar with the structure.

Starting the prototype

First I have to make clear that the following is early development state. I plan to extend the functionality when getting some time again. Since I’m involved often at customers I cannot tell how fast I progress now. At least I get the possibility to spend some days in the near future on it, so I’m expecting to have something useful in the near future.

I created the Xtext projects for the SMILA DSL and added some first rules. After running the MWE workflow Xtext generated the project infrastructure.

SMILA project wizard

When looking at the example project I recognized that a normal project setup would require copy/paste of an existing one and changing some files. Therefore extending the generated project wizard seemed to be a good starting point. The extended wizard now lets you set up a SMILA application with all the required files.

After finishing the wizard a project in the workspace is created. All static resources (esp. project structure and XSDs) are copied from the UI plugin into the new project and as a start some files are generated using Xpand with the information filled in into the wizard.

The wizard generated from the SimpleProjectWizardFragment was not so extensible for my case as it should be, so I had to copy some code from the generated classes and provide a manual implementation with some copied code. I think the fragment could be changed easily to improve and I will set up a change request on that later and post it to bugzilla.

At the moment the project wizard generates the following artifacts from the information provided on the pages:

  • SMILA DSL model file
  • log4j.properties
  • Launch configuration
  • Tomcat server config

Here you can see the project the wizard created:

Crawler configuration

The first configuration I targeted to describe with the DSL is the configuration of the FileSystemCrawler and FeedAgent. This is pretty straight forward, nearly 1:1 mapping. Here’s an excerpt from the appropriate configuration file “feed.xml” shipped with the example:

<DataSourceConnectionConfig
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:noNamespaceSchemaLocation="../org.eclipse.smila.connectivity.framework.crawler.filesystem/schemas/FileSystemDataSourceConnectionConfigSchema.xsd"
>
  <DataSourceID>file</DataSourceID>
  <SchemaID>org.eclipse.smila.connectivity.framework.crawler.filesystem</SchemaID>
  <DataConnectionID>
    <Crawler>FileSystemCrawler</Crawler>
  </DataConnectionID>
  ...
  <Attributes>
    <Attribute Type="Date" Name="LastModifiedDate" HashAttribute="true">
      <FileAttributes>LastModifiedDate</FileAttributes>
    </Attribute>
    <Attribute Type="String" Name="Filename">
      <FileAttributes>Name</FileAttributes>
    </Attribute>
    <Attribute Type="String" Name="Path" KeyAttribute="true">
      <FileAttributes>Path</FileAttributes>
    </Attribute>
    <Attribute Type="String" Name="Content" Attachment="true">
      <FileAttributes>Content</FileAttributes>
    </Attribute>
    <Attribute Type="String" Name="Extension">
      <FileAttributes>FileExtension</FileAttributes>
    </Attribute>
    <Attribute Type="String" Name="Size">
      <FileAttributes>Size</FileAttributes>
    </Attribute>
  </Attributes>
  <Process>
    <BaseDir>/Users/thoms/temp</BaseDir>
    <Filter Recursive="true" CaseSensitive="false">
      <Include Name="*.txt"/>
      <Include Name="*.htm"/>
      <Include Name="*.html"/>
      <Include Name="*.xml"/>
    </Filter>
  </Process>
</DataSourceConnectionConfig>

And here the same situation described in the DSL (the box with “caseSensitive” is there because I pressed CTRL+SPACE after the keyword “recursive” and the content assist proposes that “caseSensitive” could be entered here):

I decided that the record attribute name (in XML the Attribute#name property) can be omitted in the case that it matches the File attribute name, which I think often will be the case. Only if the names don’t match a mapping has to be done. Here the example is

FileExtension -> Extension

“FileExtension” is the File attribute name and “Extension” is the name of the Record attribute.

Flags are added in brackets and are optional (key, hash, attachment).

Builder Integration

Since Xtext Helios M4 a builder infrastructure was added to Xtext. I leveraged this infrastructure to generate the resulting configuration files on-the-fly when you save the DSL model. So if you, for example, add an “Include” line to your model the respective crawler config is automatically changed. Even better: When you rename your crawler, let’s say from “file” to “userdir_scanner” the configuration file “file.xml” gets deleted from your workspace and is replaced by “userdir_scanner.xml”!

After renaming the FileSystemCrawler:

Outlook

This is just the start of this project and many things have to be done now. I plan to use this project also as a good example for using the Xtext features properly, of course open sourced. Also I have to learn more about SMILA and the appropriate configuration. I’m in exchange with Sebastian Voigt from brox, co-lead of the SMILA project. With his help I think this project can be a valuable contribution to SMILA later.

Here are some features that I want to add to this project:

  • Complete language for covering the tutorial
    In a first step at least everything that makes up the “5 Minutes to Success” tutorial should be possible to describe in the DSL and the configuration files should be generated from that description.
  • Integrate existing configuration files
    I saw that some configuration files might not be worth to be mapped to a DSL and might be better left just in XML for editing. One example is the QueueWorkerConnectionConfig.xml, where the available brokers are defined. Of course from the DSL I want to refer to brokers at several places and I need to get them from this file. My first idea is here to use generate EMF models from the XSDs using the XSD importer. That makes it possible to reference types from that schema directly in the DSL. It should be like normal integration of existing Ecore models.
  • Validation
    One of the major benefits that the DSL can provide is the ease to add validation on the models. Especially consistency constraints make sense, for example to prove that every queue where records are routed to must have a listener that processes that records further.
  • JDT integration
    In the BPEL configuration files services are invoked. The services are qualified by their class names and parameters that can be passed correspond to properties. At Eclipse Summit 2009 Sven Efftinge and Sebastian Zarnekow showed a nice integration of Xtext with JDT to add content assist and validation on qualified Java classes.
  • Product build
    The complete bundle, SMILA and the Configuration toolkit, should be available as a ready-to-use product. I’m planning to use Maven Tycho for setting up the build process.

These are just some few examples of what I can imagine for the future. I hope that I find or get some time to realize this.

February 15, 2010

Lazy evaluation in Xpand

Filed under: Eclipse, openArchitectureWare, Xpand — kthoms @ 10:36 AM

Usually code generation is a purely sequential process. Since the model does not change during the generation of an artifact all content can be computed in the template where it is needed for the output. But sometimes there is the wish to defer the output to a later point of time during the generation of an artifact.

The typical use case for this is import statements. If for example you want to generate a Java class and want to import all used types then the following alternatives are given:

  • Compute the types that the about-to-be-generated class will use
  • Print out all type names full qualified whenever needed and organize the imports with a postprocessor. For Java code generation the Hybridlabs Beautifier is used widely.

However, both approaches do not seamlessly solve the problem. What really is needed is some kind of lazy evaluation in Xpand. Therefore Jos Warmer wrote a feature proposal once. The feature that he proposed for Xpand is called Insertion Point. The idea was to mark some point in the Xpand template where some code will be inserted at a later point of time. Code is evaluated into this insertion point when the content can be derived easier.

From this feature proposal Feature Request #261607 was created in Eclipse Bugzilla system. In this bug entry, and also offline, a lively discussion arose in the team. The challenges for this feature request were:

  • The Xpand language has a rather small set of keywords. Adding this feature, which is used in some cases only, should not introduce too much changes to the Xpand language
  • An implementation should not break existing code
  • The solution should enable decent evaluation, e.g. avoiding duplicates, sorting

The latest proposal just introduces just one new keyword in Xpand, but requires an implementation pattern with Xtend function. The proposal is to add a keyword ONFILECLOSE to the EXPAND statement. By calling EXPAND with ONFILECLOSE the evaluation of the EXPAND statement is deferred until the FILE statement is closed. Any state that is used by the called definition is computed during the FILE evaluation. The EXPAND statement has to be evaluated with the execution context which is active when reaching the EXPAND statement.

Let’s see this by example. As example we take the project that Xpand’s project wizard creates, with small changes. The entities and types have now an additional ‘packageName’ attribute, and the both entities have been assigned different packages ‘entities1′ and ‘entities2′. Additionally entity ‘Person’ has a feature ‘birthday’ of type Date, which is mapped to java.util.Date. Therefore class Person.java has to import entities2.Address and java.util.Date. The used types should be collected when rendering instance variables and accessor methods, but inserted earlier in the code.

First take a look at the template code:


As you can see the definition ImportBlock is called for each Type instance (e.g. Entity and DataType instances) in the collection returned by the Xtend function UsedType(). In an alternative approach this function would be responsible to compute all the types that will be used by this Entity instance. But for the new implementation it just creates an empty list and returns that one. The implementation of the UsedType() function (in file GeneratorExtensions.ext) is:

create List[Type] UsedType (Entity e) : (List[Type]) {};

So at the time the Xpand engine reaches EXPAND ImportBlock… the list would be empty. Now note the ONFILECLOSE keyword at the end of the EXPAND statement. This one tells the engine now that the evaluation of this Xpand statement should be deferred until the file is about to be closed. During evaluation of the template code, in the FOREACH loop, another extension function addUsedType(f.type) is called. This one adds the type of the current processed feature to the collection returned by UsedTypes(). Therfore it is important that the UsedType() function uses the create keyword, since we want to create the collection on first access for one Entity and return the same instance when UsedTypes() is called later for that Entity again.

The function addUsedType() is used in the template like this:

«addUsedType(f.type)»

Xpand would print out the result of the function as string, but we don’t want to produce any output by calling the function. Therefore we assure that the function adds the type to the collection and returns an empty string:

addUsedType (Entity e, Type t) : UsedType(e).add(t) -> "";

During evaluation it could be that types were used multiple times within the template, but we want just one import statement per type. Further we don’t need imports for types from the java.lang package (here the packageName information for the DataType instances String and Integer is null) or for types that are in the same package like the entity. Therefore we transform the UsedType() collection before finally invoking the ImportBlock definition.

«EXPAND ImportBlock FOREACH UsedType()
 .select(t|t.packageName!=null && t.packageName!=this.packageName).toSet()
 .sortBy(t|t.qualifiedName()) ONFILECLOSE»

Conclusion

Lazy evaluation allows code generated with Xpand in an non-sequential matter. The proposed solution using ONFILECLOSE solves the desired Insertion Point feature by adding just one additional keyword to the Xpand language. It does not break existing template code. When accepted this code will be contributed to Xpand 0.8.0-M6 soon. For those who want to test it I have created a feature patch for the org.eclipse.xpand feature. The example project with the sources listed in this article can be downloaded here.

Feedback is welcome and is best placed in the bugzilla feature request.

October 5, 2009

Speaking at W-JAX 09

Filed under: openArchitectureWare — kthoms @ 9:24 AM

wjax09_button_speaker_enMy colleague Heiko Behrens and I will have a session at the upcoming W-JAX conference.
The session is entitled “Mastering differentiated MDSD Requirements at Deutsche Boerse AG“, and we will share our experience about successfully applying MDSD approaches that we gained through our consulting tasks at the Deutsche Boerse. The project at the Deutsche Boerse is in many ways challenging and a great example how model driven software development really helps to master complexity. The Deutsche Boerse is developing together with the New York ISE a new Global Trading System (GTS), which is scheduled to start operations in 2011. High performance and reliability is crucial for this system, and it would not be possible to deliver both without means of modeling and code generation. Of course we leverageEclipse Modeling and openArchitectureWare for that. We held this session already at the Code Generation 2009 with overwhealming feedback, and in the meantime the story continued and we will include the new experiences into our talk.

September 24, 2009

Xtext 0.7.2 artefacts installed to Maven repository

Filed under: Maven, openArchitectureWare, Xtext — kthoms @ 7:21 PM

Finally I came to deploy the Xtext 0.7.2 artefacts to the Maven repository at openarchitectureware.org. I almost forgot that I did not that so far, but Michael Clay dropped me a mail today.

Maintaining the repository is a hard an error prone task. With each updated plugin the POM files need to be adjusted manually. Therefore each plugin manifest must be opened and the dependencies compared. Fortunately this stayed stable through the Xtext 0.7.x releases and also the underlying Eclipse libraries did not need to be updated after the Galileo release. I hope in future Eclipse plugins will be automatically deployed to a central Maven repository. There is currently some discussion on this in Eclipse Bug#283745.

I have optimized the manual deployment process a bit for myself. In my workspace I set up a project reflecting the repository structure. The jars are placed like they are stored in the repository with pattern {artifactId}-{version}.jar. Slightly different for the POMs. In the repository they are stored with pattern {artifactId}-{version}.pom, but in my workspace slightly different with {artifactId}-{version}.jar.pom.

The reason for is that I configured an External Tool configuration in Eclipse for deployment, which invokes the Maven deployment using “mvn deploy:deploy-file”. The property “pomFile” is computed from the selected resource through the {selected_resource_loc} variable (which ends with .jar), appended with “.pom”.

This way the jar file just needs to be selected and the deployment is executed with the external tool launcher.

[INFO] Scanning for projects...
[INFO] Searching repository for plugin with prefix: 'deploy'.
[INFO] ------------------------------------------------------------------------
[INFO] Building Maven Default Project
[INFO] task-segment: [deploy:deploy-file] (aggregator-style)
[INFO] ------------------------------------------------------------------------
[INFO] [deploy:deploy-file]
Uploading: scp://openarchitectureware.org/home/oaw/m2/repository/org/eclipse/xtext/xtext-core/0.7.2/xtext-core-0.7.2.jar
1495K uploaded
[INFO] Retrieving previous metadata from releases.openarchitectureware.org
[INFO] Uploading repository metadata for: 'artifact org.eclipse.xtext:xtext-core'
[INFO] Uploading project information for xtext-core 0.7.2
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 37 seconds
[INFO] Finished at: Thu Sep 24 21:06:43 CEST 2009
[INFO] Final Memory: 3M/6M
[INFO] ------------------------------------------------------------------------

Of course the repository must be configured in the ~/.m2/settings.xml file. The repository is accessed with SCP, authentication is done with public/private key.

<servers>
    <server>
      <id>releases.openarchitectureware.org</id>
      <username>oaw</username>
	  <passphrase>XXXXXXXXXXXX</passphrase>
      <filePermissions>775</filePermissions>
      <directoryPermissions>775</directoryPermissions>
      <privateKey>/Users/thoms/.ssh/id_rsa</privateKey>
    </server>
</servers>

Posted via email from Karsten’s Blog

July 20, 2009

Building TMF Xtext projects with Maven

Filed under: Maven, openArchitectureWare, Xtext — kthoms @ 2:10 PM

Normally Xtext projects are built within the Eclipse IDE. But what do you do if the requirement is to have server side builds for for Xtext based projects, including the Grammar project itself, and of course without checking in generated sources? Setting up a working build process for Eclipse plugin projects is often a hell. I often face this requirement, and this is where Maven is a suitable alternative. This article explains how you can build Xtext based projects with Maven.

At first this requires that you have Maven installed on your machine and that the mvn command is on your path. Type on your shell

mvn -version

and it will answer something like

Maven version: 2.0.10
Java version: 1.6.0_07
OS name: "mac os x" version: "10.5.7" arch: "x86_64" Family: "mac"

This is all you basically need.

Xtext projects consist of

  1. A grammar project
  2. An UI project
  3. A generator project

The aim is to run the generator, which of course requires that at least the grammar project is built. We will also build the UI project, but it won’t result into a deployable plugin yet. I plan to investigate on this later. For the moment the projects are just compiled, packaged, and sources generated.

For this article I created projects using the Xtext project wizard, which creates by default the projects

  1. org.xtext.example.mydsl
  2. org.xtext.example.mydsl.ui
  3. org.xtext.example.mydsl.generator

Additionaly a further project was added:

  • org.xtext.example.mydsl.parent.

All projects can be downloaded here as Zip. Your project structure will look like this:

Bild 17

Maven Repositories

The most important thing for Maven based builds is that every artifact and plugin required must be available in a Maven Repository.

This is where the story begins, because Eclipse itself has no Maven repository, and thus every artifact (i.e. plugin jar) required within the build must be made available in a Maven Repository. This is at the moment maintained manually. I just deployed all required artifacts to the Maven Repository at openArchitectureWare.org. It contains now the latest 0.7.1 build of TMF Xtext, M2T Xpand and EMFT MWE.

Next repository we need is the one where to get Maven plugins from. Maven itself has just a small kernel and all features are added by plugins. Most of them are quite common and central available (e.g. Compiler Plugin), others are available in project specific repositories. For running Xpand based code generators the Fornax oAW Maven plugin is used, which is available through the Fornax Maven Repository.

In this example we make us of a small plugin (Replacer Plugin) which is available at the JBoss Maven Repository.

Last but not least the central Maven repository will be used to fetch some common artifacts that are not available in both of them. So we need the following repositories:

POM Files

Maven builds are described by Project Object Models (POM). The POMs are stored in pom.xml files for each subproject. These files are basically the build scripts for the projects and this is where all the magic goes in. Mainly it is “just” adding the right POMs to the projects that enable your projects to be built by Maven. This is totally non invasive. For more information about POMs read the Maven POM Reference.

In this article I show just excerpts from the POMs, so make sure to get the sources. Gaps in the POM are marked with “…”.

Parent Project

The org.xtext.example.parent project is the so called “parent project”. The parent’s POM is parent to all POMs from the submodules. The parent pom.xml

  • aggregates all subprojects as modules (Multi-module project)
  • declares common used Maven plugins
  • declares Maven Repositories to use

Module aggregation

We want to build all submodules when running the build on the parent project. This is done by declaring modules by relative paths. Since the projects are organized within an Eclipse workspace all projects including the parent are on the same level.

<modules>
  <module>../org.xtext.example.mydsl</module>
  <module>../org.xtext.example.mydsl.generator</module>
  <module>../org.xtext.example.mydsl.ui</module>
</modules>

Repository configuration

The required repositories are declared within a profile named “RepositoryManagement”, which is activated by default.

<profiles>
 <profile>
  <activation>
   <activeByDefault>true</activeByDefault>
  </activation>
  <id>RepositoryManagement</id>
  <repositories>
   <repository>
    <id>releases.openarchitectureware.org</id>
    <name>openArchitectureWare Release Repository</name>
    <url>http://www.openarchitectureware.org/m2</url>
    <releases>
     <enabled>true</enabled>
    </releases>
    <snapshots>
     <enabled>false</enabled>
    </snapshots>
   </repository>
  </repositories>
  <pluginRepositories>
   <pluginRepository>
    <id>releases.archiva.fornax-platform.org</id>
    <name>Archiva Managed Release Repository</name>
    <url>http://www.fornax-platform.org/archiva/repository/releases</url>
    <releases>
     <enabled>true</enabled>
    </releases>
    <snapshots>
     <enabled>false</enabled>
    </snapshots>
   </pluginRepository>
   ...
  </pluginRepositories>
 </profile>
</profiles>

Plugin configuration

Maven plugins are configured in the build section of a POM. Xtext projects require at least Java 1.5, so the Maven compiler plugin is configured so.

<build>
 ...
 <plugins>
  <plugin>
   <groupId>org.apache.maven.plugins</groupId>
   <artifactId>maven-compiler-plugin</artifactId>
   <configuration>
    <source>1.5</source>
    <target>1.5</target>
   </configuration>
  </plugin>
...
 </plugins>
</build>

Grammar Project

The Grammar project org.xtext.example.mydsl contains a workflow that runs the Xtext generator to create the Xtext implementation artifacts. By default the file is GenerateMyDsl.mwe in package org.xtext.example.

Bild 18

When running a build this generator workflow should be executed, generating the Xtext sources. Afterwards the sources should be compiled.

Project parent and basic descriptor tags

The project’s parent is declared by groupId, artifactId and version in the <parent> section. Since groupId and version should always be the same as the parent they are just inherited. This way the must only declared once.

<parent>
  <groupId>org.xtext.example.mydsl</groupId>
  <artifactId>mydsl-parent</artifactId>
  <version>1.0.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>mydsl-grammar</artifactId>
<name>TMF Xtext Example - Grammar</name>

Dependencies

For generating the sources and compilation the Grammar project needs just 2 dependencies:

<dependencies>
  <dependency>
    <groupId>org.eclipse.xtext</groupId>
    <artifactId>xtext-generator</artifactId>
    <version>0.7.1</version>
  </dependency>
  <dependency>
    <groupId>de.itemis.xtext</groupId>
    <artifactId>antlr</artifactId>
    <version>0.7.1</version>
  </dependency>
</dependencies>

All other required dependencies are included by Maven’s transitive dependency management.

Alternative Source/Resource paths

Xtext projects don’t follow Maven’s default paths for Java sources and resources. By default Maven will compile Java sources only from the src/main/java folder and will add src/main/resources. For Xtext projects there are two source folders, /src and /src-gen. To compile both an additional plugin from Codehaus, the Build Helper Maven Plugin, is needed.

<build>
  ...
  <sourceDirectory>src</sourceDirectory>
  <resources>
    <resource>
      <directory>src</directory>
    </resource>
    <resource>
      <directory>src-gen</directory>
    </resource>
  </resources>
  <plugins>
    <plugin>
      <groupId>org.codehaus.mojo</groupId>
      <artifactId>build-helper-maven-plugin</artifactId>
      <version>1.3</version>
      <executions>
        <execution>
          <id>add-source</id>
          <phase>generate-sources</phase>
          <goals>
            <goal>add-source</goal>
          </goals>
          <configuration>
            <sources>
              <source>src-gen</source>
            </sources>
          </configuration>
        </execution>
      </executions>
    </plugin>
    ...
  </plugins>
</build>

Workflow execution

To execute the MWE workflow we need the Fornax oAW Maven Plugin. Maven builds have a lifecycle, where predefined lifecycles exists. The oAW Maven plugin must be executed in the generate-sources phase. Since version 3.0.0 the Fornax plugin supports both workflow engines, oAW 4 Workflow Engine and oAW 5 MWE. Current version is 3.0.1.

To enable execution of MWE workflows the configuration section must be configured with <workflowEngine>mwe</workflowEngine>. The <workflowDescriptor> parameter specifies the path to the workflow file.

<build>
  ...
  <plugins>
    <plugin>
       <groupId>org.fornax.toolsupport</groupId>
       <artifactId>fornax-oaw-m2-plugin</artifactId>
       <version>3.0.1</version>
       <configuration>
         <outletSrcDir>src-gen</outletSrcDir>
         <outletSrcOnceDir>src</outletSrcOnceDir>
       </configuration>
       <executions>
         <execution>
           <phase>generate-sources</phase>
           <goals>
             <goal>run-workflow</goal>
           </goals>
           <configuration>
             <workflowDescriptor>org/xtext/example/GenerateMyDsl.mwe</workflowDescriptor>
             <workflowEngine>mwe</workflowEngine>
           </configuration>
         </execution>
       </executions>
     </plugin>
   </plugins>
   ...
</build>

UI Project

Configuring the POM for the UI project org.xtext.example.mydsl.ui is pretty similar to the Grammar project. In this project even no workflow execution is required, just source paths must be adjusted to /src and /src-gen (see above, Grammar project).

Dependencies

The UI plugin has 2 dependencies:

  1. The Grammar project
  2. org.eclipse.xtext:xtext-ui-common
<dependencies>
 <dependency>
  <groupId>${project.groupId}</groupId>
  <artifactId>mydsl-grammar</artifactId>
  <version>${project.version}</version>
 </dependency>

 <dependency>
  <groupId>org.eclipse.xtext</groupId>
  <artifactId>xtext-ui-common</artifactId>
  <version>0.7.1</version>
 </dependency>
</dependencies>

Generator Project

The POM for the Generator project again is similar to the Grammar project. This plugin of course executes a workflow during the build, so the Fornax oAW Maven plugin needs to be configured again. Again the source paths must be configured in the build section, and the Build Helper plugin used. See the Grammar project for this configuration.

Dependencies

The Generator project needs also just 2 dependencies:

  1. The Grammar project
  2. org.eclipse.xtext:xtext-util

All other dependencies come in transitive.

<dependencies>
  <dependency>
   <groupId>${pom.parent.groupId}</groupId>
   <artifactId>mydsl-grammar</artifactId>
   <version>${pom.parent.version}</version>
  </dependency>
  <dependency>
   <groupId>org.eclipse.xtext</groupId>
   <artifactId>xtext-util</artifactId>
   <version>0.7.1</version>
  </dependency>
</dependencies>

Workflow execution

The configuration of the oAW Maven plugin is similar to the Grammar project. Of course the path of the workflow file differs. To avoid unneccessary generator execution the plugin can be configured to just run the workflow when specified resources change. This is done through the checkResources section. In this case the generator will only run when the model file src/model/MyModel.dsl changes.

<build>
 ...
 <plugins>
  <plugin>
   <groupId>org.fornax.toolsupport</groupId>
   <artifactId>fornax-oaw-m2-plugin</artifactId>
   <version>3.0.1</version>
   <configuration>
    <checkResources>
     <checkResource>
      src/model/MyModel.mydsl
    </checkResource>
   </checkResources>
   <outletSrcDir>src-gen</outletSrcDir>
  </configuration>
  <executions>
   <execution>
    <phase>generate-sources</phase>
    <goals>
     <goal>run-workflow</goal>
    </goals>
    <configuration>
     <workflowDescriptor>workflow/MyDslGenerator.mwe</workflowDescriptor>
     <workflowEngine>mwe</workflowEngine>
    </configuration>
   </execution>
  </executions>
 </plugin>
 ...
 </plugins>
</build>

Running the build

Now it is time to build the projects with Maven. Note that we only added the described pom.xml files to the projects that the Xtext Wizard created.

Open a shell, go to the org.xtext.example.parent folder and type

mvn clean install

The Maven build starts…

[INFO] Scanning for projects...
[INFO] Reactor build order:
[INFO]   TMF Xtext Example - Parent
[INFO]   TMF Xtext Example - Grammar
[INFO]   TMF Xtext Example - Generator
[INFO]   TMF Xtext Example - UI
[INFO] ------------------------------------------------------------------------
[INFO] Building TMF Xtext Example - Parent
[INFO]    task-segment: [clean, install]
[INFO] ------------------------------------------------------------------------

These are the first log statements that the build prints out. You can see from here in which order Maven will build the projects. First the parent project will be built, followed by Grammar project, Generator project and UI project.

Downloading artifacts

When you start with a clean Maven installation Maven will download all plugins and artifacts required for the build.

...
Downloading: http://www.openarchitectureware.org/m2/org/eclipse/emf/ecore/xmi/2.5.0/xmi-2.5.0.pom
583b downloaded
Downloading: http://www.openarchitectureware.org/m2/org/eclipse/m2t/xpand/m2t-xpand-xpand/0.7.1/m2t-xpand-xpand-0.7.1.pom
1K downloaded
Downloading: http://www.openarchitectureware.org/m2/org/eclipse/m2t/xpand/m2t-xpand/0.7.1/m2t-xpand-0.7.1.pom
3K downloaded
Downloading: http://www.openarchitectureware.org/m2/org/eclipse/m2t/xpand/m2t-xpand-xtend/0.7.0/m2t-xpand-xtend-0.7.0.pom
1K downloaded
Downloading: http://www.openarchitectureware.org/m2/org/eclipse/m2t/xpand/m2t-xpand/0.7.0/m2t-xpand-0.7.0.pom
3K downloaded
Downloading: http://www.openarchitectureware.org/m2/org/eclipse/emf/mwe/emf-mwe/0.7.0/emf-mwe-0.7.0.pom
24K downloaded
Downloading: http://www.openarchitectureware.org/m2/org/eclipse/emf/mwe/emf-mwe-core/0.7.0/emf-mwe-core-0.7.0.pom
2K downloaded
...

This is already a lot! All of the downloaded artifacts will be stored in your local repository, which is usually located at ~/.m2/repository. Once the artifacts are downloaded you don’t need to access the internet again. You can than go to offline mode by adding the -o option:

mvn -o clean install

You may need to configure a proxy to access the public Maven repositories. Read “Configuring a proxy” to get more information.

Generating the Xtext artifacts

When running the build on the Grammar project the artifacts from the Xtext grammar file are generated. You will see that the Fornax oAW Maven plugin starts execution and the Xtext generator is run.

[INFO] [fornax-oaw-m2:run-workflow {execution: default}]
[INFO] Fornax oAW/MWE Maven2 Plugin V3.0.1
0    [main] INFO  eclipse.emf.mwe.core.WorkflowRunner  - --------------------------------------------------------------------------------------
174  [main] INFO  eclipse.emf.mwe.core.WorkflowRunner  - EMF Modeling Workflow Engine 0.7.1, Build v200907170432
174  [main] INFO  eclipse.emf.mwe.core.WorkflowRunner  - (c) 2005-2009 openarchitectureware.org and contributors
174  [main] INFO  eclipse.emf.mwe.core.WorkflowRunner  - --------------------------------------------------------------------------------------
175  [main] INFO  eclipse.emf.mwe.core.WorkflowRunner  - running workflow: org/xtext/example/GenerateMyDsl.mwe
175  [main] INFO  eclipse.emf.mwe.core.WorkflowRunner  -
882  [main] INFO  lipse.emf.mwe.utils.StandaloneSetup  - Registering platform uri '/Users/thoms/Development/workspaces/oaw-v5-test'
2794 [main] INFO  e.core.container.CompositeComponent  - DirectoryCleaner: cleaning directory '../org.xtext.example.mydsl/src-gen'
2795 [main] INFO  ipse.emf.mwe.utils.DirectoryCleaner  - Cleaning /Users/thoms/Development/workspaces/oaw-v5-test/org.xtext.example.mydsl/../org.xtext.example.mydsl/src-gen
2820 [main] INFO  e.core.container.CompositeComponent  - DirectoryCleaner: cleaning directory '../org.xtext.example.mydsl.ui/src-gen'
2820 [main] INFO  ipse.emf.mwe.utils.DirectoryCleaner  - Cleaning /Users/thoms/Development/workspaces/oaw-v5-test/org.xtext.example.mydsl/../org.xtext.example.mydsl.ui/src-gen
2824 [main] INFO  e.core.container.CompositeComponent  - Generator
2929 [main] INFO  ipse.xtext.generator.LanguageConfig  - generating infrastructure for org.xtext.example.MyDsl with fragments : ImplicitRuntimeFragment, ImplicitUiFragment, GrammarAccessFragment, EcoreGeneratorFragment, ParseTreeConstructorFragment, ResourceFactoryFragment, AntlrDelegatingFragment, JavaValidatorFragment, JavaScopingFragment, FormatterFragment, LabelProviderFragment, TransformerFragment, OutlineNodeAdapterFactoryFragment, JavaBasedContentAssistFragment, DelegatingGeneratorFragment

Done!

When the build is through you will get the build summary. All generators are run, sources are compiled, jar files are created and installed into your local repository.

...
...
[INFO] Installing /Users/thoms/Development/workspaces/oaw-v5-test/org.xtext.example.mydsl.ui/target/org.xtext.example.mydsl.mydsl-ui_1.0.0-SNAPSHOT.jar to /tmp/maven-rep/org/xtext/example/mydsl/mydsl-ui/1.0.0-SNAPSHOT/mydsl-ui-1.0.0-SNAPSHOT.jar
[INFO]
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] ------------------------------------------------------------------------
[INFO] TMF Xtext Example - Parent ............................ SUCCESS [4.743s]
[INFO] TMF Xtext Example - Grammar ........................... SUCCESS [24.037s]
[INFO] TMF Xtext Example - Generator ......................... SUCCESS [2.600s]
[INFO] TMF Xtext Example - UI ................................ SUCCESS [2.766s]
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 34 seconds
[INFO] Finished at: Mon Jul 20 15:07:14 CEST 2009
[INFO] Final Memory: 67M/160M
[INFO] ------------------------------------------------------------------------

Of course on first execution this will take much longer, since every needed dependency is downloaded. On second execution no download is needed anymore.

July 16, 2009

oAW 4 migration with cartridge adaption

Filed under: Eclipse, openArchitectureWare — kthoms @ 3:54 AM

In one of my projects we make use of the Fornax oAW 4 cartridges Hibernate and JavaBasic. This week we migrated our generator projects to Eclipse Galileo (we use the EPP Modeling Package at the moment) with MWE, M2T Xpand and TMF Xtext. (Note: I’ll name this package here oAW 5 although it does not exist yet, but this is another topic).

Now these cartridges are not available for oAW 5 yet and will not in the near future. So we have these alternatives:

  • migrate the cartridges
  • replace the cartridges by a equivalent, project specific cartridge which takes the same input and produces the same output
  • adapt the cartridges and leave them as is

We decided to try adapting it. Now the problem is that the cartridges are oAW 4 specific and won’t work without changes. We faced some problems, and luckily we were able to solve them. Some are quite hacks ;-)

Dependencies

The required plugins have changed. With oAW 4.3.1 only the aggregator plugin org.openarchitectureware.dependencies and org.openarchitectureware.uml2.adapter were required dependencies. In our generator project we now have these dependencies (besides project specific ones):
org.eclipse.xtend.typesystem.uml2;bundle-version="0.7.0",
org.eclipse.xtend.util.stdlib;bundle-version="0.7.0",
org.eclipse.xpand;bundle-version="0.7.0",
org.eclipse.xtend;bundle-version="0.7.0",
org.eclipse.jface.text;bundle-version="3.5.0",
org.eclipse.xtend.typesystem.emf;bundle-version="0.7.0",
org.eclipse.core.runtime;bundle-version="3.5.0",
org.eclipse.jdt.core;bundle-version="3.5.0"

Workflow

The standard workflow is packaged into the cartridge jars and cannot be changed. The namespaces of the used components has changed. Looking up the component class names with CTRL+SHIFT+T (Open Type) quickly revealed the package names the workflow components have now.
Screenshot
Configuration of the components did not change.

Tipp: if you want to know which workflow component are available then open the interface org.eclipse.emf.mwe.WorkflowComponent and press F4. This opens the type hierarchy and allows you to browse and inspect all known WorkflowComponent implementations. The most interesting ones are derived from AbstractWorkflowComponent2.

Bild 15

Loading profiles

Formerly profiles were just located by classpath. We had the Persistence.profile.uml in folder model, which was on classpath root. This profile could not be loaded anymore. The path was changed to a platform resource URI (platform:/resource/PROJECT/model/Persistence.profile.uml).

With the UML profile file from Fornax we had the problem that when instantiating the ProfileMetaModel instance with that file our generator process hang. It turned out that it was tried to open a HTTP connection to load “http://www.eclipse.org/uml2/2.1.0/UML“, which is declared as xmlns:uml in the model file. Eclipse did not have a internet connection nor should it require. The profile was loaded in MagicDraw 16 and reexported, and the file changed to xmlns:uml="http://www.eclipse.org/uml2/2.0.0/UML". After doing so the profile could be loaded with ProfileMetaModel. Since we use Galileo, the installed UML2 plugins have version 3.0.0.

Stdlib Extensions

The cartridges make use of the oAW’s stdlib extensions. The extensions exist unchanged, but now can be found in the plugin org.eclipse.xtend.util.stdlib with the same package name. So the namespace of those extensions changed and workflow execution failed as follows:


ERROR - [ERROR]: Unable to load extension file : org::openarchitectureware::util::stdlib::io(Element: EvaluationException : Unable to load extension file : org::openarchitectureware::util::stdlib::io
org::fornax::cartridges::uml2::hibernate::checks::CartridgeProperties.chk[967,28] on line 27 'getHibernatePropertiesFile()'
; Reported by: -UNKNOWN-)
EvaluationException : Unable to load extension file : org::openarchitectureware::util::stdlib::io
org::fornax::cartridges::uml2::hibernate::checks::CartridgeProperties.chk[967,28] on line 27 'getHibernatePropertiesFile()'
at org.eclipse.internal.xtend.expression.ast.Expression.evaluate(Expression.java:56)

To solve this we added extension files with the same name as the old stdlib file and same package, and just reexport the new stdlib extensions. For example, for the IO extensions a file org/openarchitectureware/util/stdlib/io.ext was created with the following content:

extension org::eclipse::xtend::util::stdlib::io reexport;

Dieter Moroff had already the idea to provide a stdlib compatibility package that contain all oAW 4 stdlib extensions mapped this way. I think it would be great for all users to have that, it would make migrations more smooth. Maybe I find some time to realize it. Main question would be where to host this plugin. I think the sources should be kept at Sourceforge in the oAW 4 repository, and the plugin should be available through an update site on openArchitectureWare.org.

In our own templates that we have under development we just replaced the usage of org::openarchitectureware::util::stdlib... by org::eclipse::xtend::util::stdlib...

PLE feature

The cartridges make use of the org.openarchitectureware.util.featureconfig plugin, which has not been migrated yet. We had to replace the behavior. The cartridges use this plugin mainly to conditionally execute workflow components. Since we had to replace the workflow files anyway we changed all <feature isSelected="FEATURENAME">...</feature> occurances by <if cond="${FEATURENAME}">...</if>, which has the same behavior, but configuration of the features is done by workflow properties and <if> is a built-in feature of the workflow engine. The if-components were not documented in the MWE reference yet, so we filed a bug which already has been fixed now.
The cartridge makes also use of the features.ext extensions, which defines a function isFeatureSelected(). We replaced the behavior by using the stdlib properties extensions. Therefore we configured our features in a properties file and read them with the PropertiesReader component.
<component>
<propertiesFile value="featureconfig.properties"/>
</component>

We created a file org/openarchitectureware/util/featureconfig/features.ext:

extension org::eclipse::xtend::util::stdlib::properties;

Boolean isFeatureSelected (String feature) : getProperty(feature).trim()=="true";

Changed namespace of internal types

oAW has a built-in typesystem (with types Type, Operation etc.) whose types were in oAW 4 in the namespace oaw. So if you have the expression uml::Class.metaType.name the result would be oaw::Type. With Eclipse M2T the namespace of those types changed to xpand2, so oaw::Type is now xpand2::Type. Normally you don’t work with the internal types directly until you do some reflection stuff, but extensions in the Fornax JavaBasic cartridge make use of those types. This leads to errors like this already when an extension is loaded and analyzed:

ERROR - Error in Component hibernateCartridge_transformations of type org.eclipse.xtend.XtendComponent:
EvaluationException : Couldn't resolve type for 'oaw::Type'. Did you forget to configure the corresponding metamodel?
org::fornax::cartridges::uml2::javabasic::extensions::DataType.ext[5047,76] on line 135 'boolean typeOf(uml::Type type1,oaw::Type type2)'
nofile[0,26] on line 1 'createDefaultValues(model)'

ERROR – Workflow interrupted. Reason: Couldn’t resolve type for ‘oaw::Type’. Did you forget to configure the corresponding metamodel?

To work around this the extension files need to be replaced. Since they are packaged in a jar it is required to copy them and make them available through the classpath before the cartridge. We use the approach to have a compatibility plugin which contains this .ext file as well as the stdlib compatibility extensions.

Problems with UML2 typesystem

We experienced some severe problems with the org.eclipse.xtend.typesystem.uml2 plugin that lead to EvaluationExceptions through our generator process. Basically the plugin is just a migrated version of the org.openarchitectureware.uml2.adapter plugin, but the behavior of the underlying EMF typesystem changed slightly but significant. It was not possible to handle different typesystems at the same time it (e.g. for M2M transformation), which blocked us. We opened Bug#283469 and hunted it down. Some other issues with the typesystem were detected and fixed. A patched plugin you can download from here. We created a folder dropins/oaw5-patch/plugins and put it there until the official service release will be available.
As a positive side effect it is not necessary anymore to register an UML2MetaModel nor EmfRegistryMetaModel instance in the workflow when a ProfileMetaModel is used, since it will delegate to them internally.

Hybridlabs Beautifier

The cartridges make use of the Hybridlabs Beautifier 1.1.9, which is only compatible with oAW 4. The new version 2.0.0 is now able to work also with M2T Xpand.

Conclusion

Our projects including cartridge adaption and bugfixing were migrated in about 4 days. The scenario is a real-world case, and some issues (esp. the mentioned bugs) we faced required deep knowledge about the implementation. I think most projects can be migrated even easier now, but if you face problems it might be worth ordering expert help before spending too much time. Experienced users of oAW 4 will not have much problems migrating the Xpand based code generators.

Older Posts »

The Silver is the New Black Theme. Create a free website or blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 375 other followers