Xtext 2.2.0 was just released yesterday, and for me the most important new feature is direct support through the xtend-maven-plugin plugin. I could not hesitate to test this feature, and created a small example.

Scenario

I have set up a small Maven project with 2 Java classes and 1 Xtend class that depend on each other.

To translate XtendClass1 to Java code and compile it requires that JavaClass1 is compiled before, and to compile JavaClass2 it is necessary that XtendClass is translated to Java code before.

Maven POM Configuration

I have decided to not use Maven Tycho for this example, and use the typical structure of an Eclipse project with usage of Xtend, i.e. sources in folder /src (instead /src/main/java) and xtend-gen to generate the Java code for Xtend classes to.

Repositories

Since shortly Xtext artifacts are available on maven.eclipse.org. The new Maven plugin is available on http://build.eclipse.org/common/xtend/maven/

	<repositories>
		<repository>
			<id>maven.eclipse.org</id>
			<url>http://maven.eclipse.org/nexus/content/groups/public/</url>
		</repository>
		<repository>
			<id>xtend</id>
			<url>http://build.eclipse.org/common/xtend/maven/</url>
		</repository>
	</repositories>
	<pluginRepositories>
		<pluginRepository>
			<id>xtend</id>
			<url>http://build.eclipse.org/common/xtend/maven/</url>
		</pluginRepository>
	</pluginRepositories>

Dependencies

This small project setup requires just a mimimal set of dependencies to be configured:

	<dependencies>
		<dependency>
			<groupId>org.eclipse.xtend2</groupId>
			<artifactId>org.eclipse.xtend2.lib</artifactId>
			<version>2.2.0</version>
		</dependency>
		<dependency>
			<groupId>org.eclipse.xtext</groupId>
			<artifactId>org.eclipse.xtext.xtend2.lib</artifactId>
			<version>2.2.0.v201112061305</version>
		</dependency>
		<dependency>
			<groupId>com.google.inject</groupId>
			<artifactId>com.google.inject</artifactId>
			<version>2.0.0.v201105231817</version>
		</dependency>
	</dependencies>

Source folders

The main source folder src/ can be configured with the build/sourceDirectory setting, for xtend-gen we need the support of the build-helper-maven-plugin.

	<build>
		<sourceDirectory>src</sourceDirectory>
		<plugins>
			<plugin>
				<groupId>org.codehaus.mojo</groupId>
				<artifactId>build-helper-maven-plugin</artifactId>
				<version>1.7</version>
				<executions>
					<execution>
						<id>add-source</id>
						<phase>generate-sources</phase>
						<goals>
							<goal>add-source</goal>
						</goals>
						<configuration>
							<sources>
								<source>xtend-gen</source>
							</sources>
						</configuration>
					</execution>
				</executions>
			</plugin>
			...

			<plugin>
				<artifactId>maven-clean-plugin</artifactId>
				<version>2.4.1</version>
				<configuration>
					<filesets>
						<fileset>
							<directory>xtend-gen</directory>
							<includes>
								<include>**</include>
							</includes>
						</fileset>
					</filesets>
				</configuration>
			</plugin>

xtend-maven-plugin

Now let’s finally come to the Xtend plugin. The plugin is configured as follows:

			<plugin>
				<groupId>org.eclipse.xtend2</groupId>
				<artifactId>xtend-maven-plugin</artifactId>
				<version>2.2.0</version>
				<executions>
					<execution>
						<goals>
							<goal>compile</goal>
							<goal>testCompile</goal>
						</goals>
						<configuration>
							<outputDirectory>xtend-gen</outputDirectory>
						</configuration>
					</execution>
				</executions>
			</plugin>

The plugin does not define a lifecycle mapping for M2E, which leads to an error marker “Plugin execution not covered by lifecycle configuration”. This is a well known issue when using M2E, and requires some additional work on the plugin. I have opened Bug#366118 for this.

Sample Project

The project described above is shared on Github: https://github.com/kthoms/xtext-experimental

In Eclipse the project looks like this after import (assuming the M2E plugin installed):

Build Execution

The classes in the project can now be built using mvn clean install. This will produce the following output:

[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building xtend-maven-classic 2.2.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ xtend-maven-classic ---
[INFO] Deleting /Users/thoms/git/xtext-experimental/maven/xtend.maven/target
[INFO] Deleting /Users/thoms/git/xtext-experimental/maven/xtend.maven/xtend-gen (includes = [**], excludes = [])
[INFO] 
[INFO] --- build-helper-maven-plugin:1.7:add-source (add-source) @ xtend-maven-classic ---
[INFO] Source directory: /Users/thoms/git/xtext-experimental/maven/xtend.maven/xtend-gen added.
[INFO] 
[INFO] --- xtend-maven-plugin:2.2.0:compile (default) @ xtend-maven-classic ---
[WARNING] 
WARNING: 	XtendClass1.xtend - /Users/thoms/git/xtext-experimental/maven/xtend.maven/src/mypackage/XtendClass1.xtend
7: The value of the field XtendClass1.cls is not used
[INFO] Compiling 1 source file to xtend-gen
[INFO] 
[INFO] --- maven-resources-plugin:2.4.3:resources (default-resources) @ xtend-maven-classic ---
[WARNING] Using platform encoding (MacRoman actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /Users/thoms/git/xtext-experimental/maven/xtend.maven/src/main/resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ xtend-maven-classic ---
[WARNING] File encoding has not been set, using platform encoding MacRoman, i.e. build is platform dependent!
[INFO] Compiling 3 source files to /Users/thoms/git/xtext-experimental/maven/xtend.maven/target/classes
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.923s
[INFO] Finished at: Thu Dec 08 23:01:42 CET 2011
[INFO] Final Memory: 20M/81M
[INFO] ------------------------------------------------------------------------

From the output it can be seen that at the end 3 Java classes are compiled, after the xtend-maven-plugin created the Java code for the Xtend class.

Enabling debug output with -X reveals some more insight, how the plugin works:

[DEBUG] Configuring mojo 'org.eclipse.xtend2:xtend-maven-plugin:2.2.0:compile' with basic configurator -->
[DEBUG]   (f) outputDirectory = xtend-gen
[DEBUG]   (f) project = MavenProject: org.eclipse.xtext.example:xtend-maven-classic:2.2.0-SNAPSHOT @ /Users/thoms/git/xtext-experimental/maven/xtend.maven/pom.xml
[DEBUG]   (f) tempDirectory = /Users/thoms/git/xtext-experimental/maven/xtend.maven/target/xtend
[DEBUG] -- end configuration --
[DEBUG] load xtend file 'file:/Users/thoms/git/xtext-experimental/maven/xtend.maven/src/mypackage/XtendClass1.xtend'
[DEBUG] Parsing took: 42 ms
...
[DEBUG] create java stub 'mypackage/XtendClass1.java'
[DEBUG] invoke batch compiler with '-cp /Users/thoms/git/xtext-experimental/maven/xtend.maven/src:/Users/thoms/.m2/repository/org/eclipse/xtend2/org.eclipse.xtend2.lib/2.2.0/org.eclipse.xtend2.lib-2.2.0.jar:/Users/thoms/.m2/repository/com/google/guava/guava/10.0.1/guava-10.0.1.jar:/Users/thoms/.m2/repository/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar:/Users/thoms/.m2/repository/org/eclipse/xtext/org.eclipse.xtext.xtend2.lib/2.2.0.v201112061305/org.eclipse.xtext.xtend2.lib-2.2.0.v201112061305.jar:/Users/thoms/.m2/repository/org/eclipse/xtext/org.eclipse.xtext.xbase.lib/2.2.0.v201112061305/org.eclipse.xtext.xbase.lib-2.2.0.v201112061305.jar:/Users/thoms/.m2/repository/com/google/inject/com.google.inject/2.0.0.v201105231817/com.google.inject-2.0.0.v201105231817.jar -d /Users/thoms/git/xtext-experimental/maven/xtend.maven/target/xtend/classes -1.5 -proceedOnError /Users/thoms/git/xtext-experimental/maven/xtend.maven/src /Users/thoms/git/xtext-experimental/maven/xtend.maven/target/xtend/stubs'
[DEBUG] classpath used for Xtend compilation : [file:/Users/thoms/git/xtext-experimental/maven/xtend.maven/src/, file:/Users/thoms/.m2/repository/org/eclipse/xtend2/org.eclipse.xtend2.lib/2.2.0/org.eclipse.xtend2.lib-2.2.0.jar, file:/Users/thoms/.m2/repository/com/google/guava/guava/10.0.1/guava-10.0.1.jar, file:/Users/thoms/.m2/repository/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar, file:/Users/thoms/.m2/repository/org/eclipse/xtext/org.eclipse.xtext.xtend2.lib/2.2.0.v201112061305/org.eclipse.xtext.xtend2.lib-2.2.0.v201112061305.jar, file:/Users/thoms/.m2/repository/org/eclipse/xtext/org.eclipse.xtext.xbase.lib/2.2.0.v201112061305/org.eclipse.xtext.xbase.lib-2.2.0.v201112061305.jar, file:/Users/thoms/.m2/repository/com/google/inject/com.google.inject/2.0.0.v201105231817/com.google.inject-2.0.0.v201105231817.jar, file:/Users/thoms/git/xtext-experimental/maven/xtend.maven/src/, file:/Users/thoms/git/xtext-experimental/maven/xtend.maven/target/xtend/classes/]

The plugin invokes the Xtend Batch compiler through a spawned JVM. The batch compiler is realized in class org.eclipse.xtext.xtend2.compiler.batch.Xtend2BatchCompiler from the newly added plugin org.eclipse.xtext.xtend2.standalone.

After the build XtendClass1.java is generated as expected to xtend-gen. When opening the target folder it can be seen that the plugin produces a Java stub class to target/stubs for the Xtend class. Finally, everything is compiled and the project is clean

Limitation

It seems that the plugin only takes one source folder into account, namely the one configured by build/sourceDirectory. This is a problem when you have multiple source folders, which is quite typical for Xtext projects, namely src and src-gen. I will have to test this with an appropriate project and raise a bug is necessary.

Conclusion

I deeply desired this plugin and this finally allows that the xtend-gen folder does not need to be checked in. The sample used here was just simple. Next would be a real life project were I want to apply the plugin, most likely in Project Spray. This project uses Xtend based code generation heavily and Maven Tycho for the build. The project has not been upgraded to Xtext 2.2.0 (of course, it was just released), and I guess we have to upgrade to Xtext 2.2.0 before we can use the plugin. But if this allows us to finally remove the xtend-gen folders from the repository this alone is worth upgrading.

Nice work, Xtext team!

Today Achim Demelt pointed me to a Google+ posting from Dennis Huebner (our Xtext build manager) where he announced that Xtext artifacts are now automatically available on the Nexus repository at maven.eclipse.org.

The given POMs have all dependencies from their bundle manifests. This is the recommended “Maven way”, but you may face that too many dependencies are pulled transitive, and you may have to cut them down by explicit exclusions. I haven’t tested this with the new repository, but this is what I experienced when doing a similar approach earlier with Xtext 2.0.1, whose artifacts are deployed on the Fornax Repository. Use the dependency:tree plugin, or better M2E, to identify possible exclusions.

The project was founded at this year’s Code Generation conference, and Jos contributed an initial state of Spray derived from a customer’s project back in August. Since then the project team refactored the code base quite a bit. We are still in an early project phase. Most of the development is done in spare time yet, like Open Source projects often start. That was quite tough besides my full packed work schedule. Many hours on train or evenings in the hotel were spent to push this project.

At the moment I am just about to release version 0.3.0 of the Spray SDK, which I will use for our demo on thursday. Of course we want to gain a large audience. I am honest enough to say that attendees won’t see a production ready tool yet, but of course it is in working state. Jan announced a pie fight for visual tooling, I am ready to open my mouth wide enough to catch the incoming pies.

For our session we don’t want to start any discussion on graphical editing vs. visualization, whether GMF is better than Graphiti, or DSL / code generation vs. framework approach. There is always a niche for everything, and we are confident that Spray fills one of these gaps.

Besides what an end user actually can do with Spray at this moment there are quite some hidden gems in the project. I am using Spray to have also a non-trivial example for Eclipse Modeling tooling. You will find, for example

• a Maven Tycho based build for Xtext 2 based projects
• SWTBot based UI tests, integrated in the Maven build
• a project wizard with Xtend based code generation
• an advanced example use of Xbase in a DSL
• documentation based on WikiText, integrated in the build
• unit testing of the Xtext DSL with xtext-utils unittesting
• separation of the Xtend code generator from the language project

At EclipseCon I will take the chance to exchange with some experts in the fields of Modeling, especially Xtext and Graphiti. I will lay another focus on Build systems, especially Maven. Besides that there are many other interesting talks, far more than I can attend. And not to forget to meet all the other Eclipse enthusiasts. See you there!

In this article I will show some guidelines that I follow when checking in Xtext projects into source control.

Handling the src-gen folder

The content of src-gen can be produced during a build, so I do not check in the content of it. What needs to be considered is that is no good idea to ignore the folder completely, since it would be missing when fresh checking out the project. This leads to an incomplete build path, and the MWE2 workflow is not executable from Eclipse until the folder was created. This is because the .mwe2 file needs to be on the project’s classpath, which is not the folder src, but the output folder, bin or target/classes. It will be copied automatically with default workspace settings only when the build path is complete. This behavior can be changed in the workspace / project settings in Java / Compiler / Building.

If you rate down build path problems to warning level or just disable build abortion on errors, the .mwe2 file will be copied anyway to the target folder. If using project specific settings you could even check in these settings and share it in the team.

I prefer to check in the src-gen folder, but avoid that any content gets checked in. To do so, add the folder with svn add, but none of its content! Best you do this immediately after the Xtext project was created. After the folder was scheduled for adding, you can set the svn:ignore property to value “*”.

The xtend-gen folder

At the moment it is not possible to compile .xtend files during a Maven build, so the content of xtend-gen must be checked in and handled like handcrafted sources. To cut this story short: Xtend and Java files depend usually on each other. Xtend files cannot be compiled until Java sources from the same project were compiled, and some Java sources will depend on about-to-be-compiled Xtend sources. So the Java source compilation fails, aborting the build.

I hope to solve this problem soon.

Some ignore patterns

On the root of each of the 3 Xtext projects, set the property svn:ignore to this value:

bin
target
.DS_Store
plugin.xml_gen

Configure Maven clean plugin

When running mvn clean on your project you would like to to clean up all derived resources. The target folder will be removed by default, but this is not enough. The content of the src-gen folders must be cleaned up as well. This can be done by configuring the maven-clean-plugin. One thing to consider is that Xtext has 3 projects. It is not correct to configure the clean plugin for all 3 projects due the lifecycle of a Maven build: The projects will be build after each other, executing all lifecycles per project. If you execute a “mvn clean install” this would have the effect that

1. clean is called on the grammar project
2. in phase generate-sources the Xtext implementation classes are produced to all 3 projects
3. the sources for the grammar project are compiled
4. now the build continues with the next module, e.g. the UI project
5. the project will be cleaned, removing the previously generated sources
6. compilation fails, since the generated sources are missing now

For this reason the clean plugin must be configured only in the grammar project, and it should clean up the sources from all 3 projects. Now comes again a detail related to source control: Since we want to have the src-gen checked in, we have to avoid that the SCM related folders and files are removed. Otherwise this would confuse your SCM system. So these files must be excluded from deletion. This leads to the following configuration:

<plugin>
  <artifactId>maven-clean-plugin</artifactId>
  <configuration>
    <filesets>
      <fileset>
        <directory>src-gen</directory>
        <includes>
          <include>**</include>
        </includes>
        <excludes>
          <exclude>.svn/**</exclude>
          <exclude>.gitignore</exclude>
        </excludes>
      </fileset>
      <fileset>
        <directory>../${project.artifactId}.ui/src-gen</directory>
        <includes>
          <include>**</include>
        </includes>
        <excludes>
          <exclude>.svn/**</exclude>
          <exclude>.gitignore</exclude>
        </excludes>
      </fileset>
      <fileset>
        <directory>../${project.artifactId}.tests/src-gen</directory>
        <includes>
          <include>**</include>
        </includes>
        <excludes>
          <exclude>.svn/**</exclude>
          <exclude>.gitignore</exclude>
        </excludes>
      </fileset>
    </filesets>
  </configuration>
</plugin>

As a reference take the sources from project Spray. There you can find an concrete example where the described steps have been applied.

Create the generator plugins

Create two additional plugins: One for the runtime part of the generator, one for the UI contributions.

Generator runtime plugin

Move everything from the .generator subpackage of your DSL runtime project to the generator plugin. (In Spray this is org.eclipselabs.spray.generator.graphiti). The dependencies are the same as in the DSL runtime project, but add the DSL runtime plugin as additional dependency (see this MANIFEST.MF as reference). Don’t forget to add an xtend-gen source folder to the project.

Guice module for runtime plugin

Create a Module class extending AbstractGenericModule in the runtime plugin. In this module at least the IGenerator implementation class must be bound.

public class GraphitiGeneratorModule extends AbstractGenericModule {
    public Class<? extends org.eclipse.xtext.generator.IGenerator> bindIGenerator() {
        return SprayGenerator.class;
    }
    ...
}

Generator UI plugin

The UI plugin must basically bind the JavaProjectBasedBuilderParticipant in its own module and register it through the org.eclipse.xtext.builder.participant extension point. These steps are required:

UI Guice Module

Create a module class extending AbstractGenericModule and bind JavaProjectBasedBuilderParticipant:

public class GraphitiGeneratorUIModule extends AbstractGenericModule {
    private final AbstractUIPlugin plugin;
    
    public GraphitiGeneratorUIModule (AbstractUIPlugin plugin) {
        this.plugin = plugin;
    }

    @Override
    public void configure(Binder binder) {
        super.configure(binder);
        binder.bind(AbstractUIPlugin.class).toInstance(plugin);
        binder.bind(IDialogSettings.class).toInstance(plugin.getDialogSettings());
    }
    /**
     * Bind the JavaProjectBasedBuilderParticipant in order to invoke the generator during the build.
     */
    public Class<? extends org.eclipse.xtext.builder.IXtextBuilderParticipant> bindIXtextBuilderParticipant() {
        return org.eclipse.xtext.builder.JavaProjectBasedBuilderParticipant.class;
    }

    ...
}

Activator class

Create a class Activator which creates a Guice injector from the modules:

1. DSL runtime module
2. org.eclipse.xtext.ui.shared.SharedStateModule
3. DSL UI module
4. Generator runtime module
5. Generator UI module

public class Activator extends AbstractUIPlugin {
	private Injector injector;
	private static Activator INSTANCE;

	public Injector getInjector() {
		return injector;
	}
	
	@Override
	public void start(BundleContext context) throws Exception {
		super.start(context);
		INSTANCE = this;
		try {
		    injector = Guice.createInjector(Modules2.mixin(new SprayRuntimeModule(), new SharedStateModule(), new SprayUiModule(this), new GraphitiRuntimeModule(), new GraphitiGeneratorModule(), new GraphitiGeneratorUIModule(this)));
		} catch (Exception e) {
			Logger.getLogger(getClass()).error(e.getMessage(), e);
			throw e;
		}
	}
	
	@Override
	public void stop(BundleContext context) throws Exception {
		injector = null;
		super.stop(context);
	}
	
	public static Activator getInstance() {
		return INSTANCE;
	}
	
}

ExecutableExtensionFactory

Create a class ExecutableExtensionFactory. Since this class won’t be public API it is a good approach to put it into an internal package.

public class ExecutableExtensionFactory extends AbstractGuiceAwareExecutableExtensionFactory {

	@Override
	protected Bundle getBundle() {
		return Activator.getInstance().getBundle();
	}
	
	@Override
	protected Injector getInjector() {
		return Activator.getInstance().getInjector();
	}
	
}

plugin.xml

Add a plugin.xml with the following content:

<?xml version="1.0" encoding="UTF-8"?>
<?eclipse version="3.4"?>
<plugin>
   <extension
         point="org.eclipse.xtext.builder.participant">
      <participant
            class="org.eclipselabs.spray.generator.graphiti.ui.internal.ExecutableExtensionFactory:org.eclipse.xtext.builder.IXtextBuilderParticipant">
      </participant>
   </extension>
</plugin>

Remove the GeneratorFragment

Open the .mwe2 workflow of your DSL project. Remove the entry for the GeneratorFragment:

// Code generator
fragment = generator.GeneratorFragment {
  generateJavaMain = false 
  generateMwe = false
  generatorStub = true 
}

Add binding for IWorkspaceRoot

The generator fragment contributes a binding for IWorkspaceRoot to the UI Module of your DSL, which is required by the builder participant. Therefore this binding must be added manually to your UI Module.

public class SprayUiModule extends AbstractSprayUiModule {
    ...
    public org.eclipse.core.resources.IWorkspaceRoot bindIWorkspaceRootToInstance() {
        return org.eclipse.core.resources.ResourcesPlugin.getWorkspace().getRoot();
    }
}

Result

As a result your DSL plugins should have no dependencies on your generator plugin. When you save a model file in your Eclipse instance with the deployed plugins the code generator should be invoked. The pattern described here would also allow to create multiple generator plugins for the same DSL which are invoked independently when building the project. Each of them registers its own builder participant and invokes its own generator.

When executing the Xpand generator with MWE it is important that the templates are read with the right encoding. Otherwise you will get an error like this:
org.eclipse.internal.xtend.xtend.parser.ParseException: no viable alternative at character '�' on line 159
Xpand’s Generator component has a property fileEncoding, but this is the encoding used by the outlets to write files. If you need to change the behavior for reading files, you need to set the encoding value of the ResourceManager. The configuration is like this:

MWE1:

    ...

MWE2

  component = org.eclipse.xpand2.Generator {
    ...
    resourceManager = org.eclipse.xtend.expression.ResourceManagerDefaultImpl {
     	fileEncoding = "ISO-8859-1"
    }
  }

InjectableGenerator

I subclassed org.eclipse.xpand2.Generator and called it InjectableGenerator. The intention is to register a global variable for a configurable set of types (e.g. the IScopeProvider). The name of the variable is the class name that should be get from Guice. To add injectors, instances of ISetup are registered to the component, similar to Xtext’s org.eclipse.xtext.mwe.Reader component.

The hook that is used to register the global variables to Xpand’s execution context is the method getGlobalVars(WorkflowContext ctx), which is overridden here.

package org.eclipselabs.xtext.xpand;

import java.util.List;
import java.util.Map;
import java.util.Set;

import org.eclipse.emf.mwe.core.WorkflowContext;
import org.eclipse.xpand2.Generator;
import org.eclipse.xtend.expression.Variable;
import org.eclipse.xtext.ISetup;

import com.google.common.collect.Lists;
import com.google.common.collect.Sets;
import com.google.inject.ConfigurationException;
import com.google.inject.Injector;

/**
 * Allows Xtext based dependency injection. Injected instances are registered as
 * global vars to the Xpand Execution Context.
 */
public class InjectableGenerator extends Generator {
	private List<Injector> injectors = Lists.newArrayList();

	public void addRegister(ISetup setup) {
		injectors.add(setup.createInjectorAndDoEMFRegistration());
	}

	protected List<Injector> getInjectors() {
		return injectors;
	}
	
	private Set<Class<?>> injectedTypes = Sets.newHashSet();

	public void addInjectGlobalVar (String className) {
		try {
			Class<?> clazz = Class.forName(className);
			injectedTypes.add(clazz);
		} catch (Exception e) {
			throw new IllegalArgumentException("Invalid type: "+className);
		}
	}
	
	@Override
	protected Map<String, Variable> getGlobalVars(WorkflowContext ctx) {
		Map<String, Variable> result = super.getGlobalVars(ctx);
		// try to get an instance of each configured type
		for (Class<?> clazz : injectedTypes) {
			boolean configured = false;
			for (Injector injector : injectors) {
				try {
					Object obj = injector.getInstance(clazz);
					result.put(clazz.getName(), new Variable(clazz.getName(), obj));
					configured = true;
				} catch (ConfigurationException e) {
					; // ignore
				}
			}
			if (!configured) {
				throw new IllegalStateException("Could not configure instance of "+clazz.getName());
			}
		}
		return result;
	}
}

Workflow Configuration

In your workflow configuration you would of course now use the specialized Generator component. It could be used like this:

...
component = org.eclipselabs.xtext.xpand.InjectableGenerator {
   register = my.dsl.MyDslStandaloneSetup {}
   injectGlobalVar = "org.eclipse.xtext.scoping.IScopeProvider"
   // injectGlobalVar = ... other types
   
   // further configuration of Xpand Generator
}

Execution context aware Java extensions

It is not possible to directly access the ExecutionContext from within Xpand templates or Xtend functions. We need to delegate to Java extensions. The classes that we delegate to must implement the IExecutionContextAware interface. The setExecutionContext(ExecutionContext ctx) method is the right place to retrieve the IScopeProvider and bind it to an instance variable.

public class MyExtensions implements IExecutionContextAware {
	protected IScopeProvider scopeProvider;

	@Override
	public void setExecutionContext(ExecutionContext ctx) {
		if (ctx.getGlobalVariables().get("scopeProvider")!=null) {
			this.scopeProvider = (IScopeProvider) ctx.getGlobalVariables().get("scopeProvider").getValue();
		} else if (ctx.getGlobalVariables().get(IScopeProvider.class.getName())!=null) {
			this.scopeProvider = (IScopeProvider) ctx.getGlobalVariables().get(IScopeProvider.class.getName()).getValue();
		} else {
			throw new IllegalStateException("No IScopeProvider found as global var in the execution context");
		}
	}

	// add (non-static!) extension methods here!
        public List<EObject> someMethod (org.eclipse.emf.ecore.EObject context) {
          ... do something
        }
}

It might be a good idea to have an abstract base class for these Extension classes.

Now you can use the methods defined in the Java extension class and access the required instances from your language configuration within.

// file: MyExtensions.ext
List[emf::EObject] someMethod (emf::EObject ctx) : JAVA MyExtensions.someMethod(org.eclipse.emf.ecore.EObject);

However, the API does not allow to create a single artifact from multiple input models. But this is a common use case if you do serious code generation. I will show you how you could extend Xtext’s Domainmodel example to generate a simple text file that lists the name of all Entity instances of all .dmodel files that are reachable in the built project.

Adding an extended generator interface

Create an interface IGenerator2 that extends the IGenerator interface:

public interface IGenerator2 extends IGenerator {
	/**
	 * @param input - the input for which to generate resources
	 * @param fsa - file system access to be used to generate files
	 */
	public void doGenerate(ResourceSet input, IFileSystemAccess fsa);
}

The generator class will later implement this interface and generate a single artifact from all Entity instances in all resources.

Extending the JavaProjectBasedBuilderParticipant

The JavaProjectBasedBuilderParticipant is used by default to invoke the IGenerator implementation that was bound to the runtime module. We will extend this class to add additional logic and inject an IGenerator2 instance that will be invoked once per triggered build.

The builder participant is an UI concept, so create the extended class JavaProjectBasedBuilderParticipant2 in the UI plugin:

package org.eclipse.xtext.builder;

import java.util.List;

import org.eclipse.core.runtime.CoreException;
import org.eclipse.core.runtime.IProgressMonitor;
import org.eclipse.emf.ecore.resource.Resource;
import org.eclipse.xtext.builder.JavaProjectBasedBuilderParticipant;
import org.eclipse.xtext.generator.IFileSystemAccess;
import org.eclipse.xtext.generator.IGenerator2;
import org.eclipse.xtext.resource.IContainer;
import org.eclipse.xtext.resource.IResourceDescription;
import org.eclipse.xtext.resource.IResourceDescription.Delta;
import org.eclipse.xtext.resource.IResourceDescriptions;
import org.eclipse.xtext.resource.impl.ResourceDescriptionsProvider;

import com.google.inject.Inject;

public class JavaProjectBasedBuilderParticipant2 extends JavaProjectBasedBuilderParticipant {
  @Inject
  private ResourceDescriptionsProvider resourceDescriptionsProvider;

  @Inject
  private IContainer.Manager containerManager;

  @Inject (optional=true)
  private IGenerator2 generator;

  protected ThreadLocal<Boolean> buildSemaphor = new ThreadLocal<Boolean>();

  @Override
  public void build(IBuildContext context, IProgressMonitor monitor) throws CoreException {
    buildSemaphor.set(false);
    super.build(context, monitor);
  }

  @Override
  protected void handleChangedContents(Delta delta, IBuildContext context, IFileSystemAccess fileSystemAccess) {
    super.handleChangedContents(delta, context, fileSystemAccess);
    if (!buildSemaphor.get() && generator != null) {
      invokeGenerator(delta, context, fileSystemAccess);
    }
  }

  @Override
  protected void handleDeletion(Delta delta, IBuildContext context, IFileSystemAccess fileSystemAccess) {
    super.handleDeletion(delta, context, fileSystemAccess);
    if (!buildSemaphor.get() && generator != null) {
      invokeGenerator(delta, context, fileSystemAccess);
    }
  }

  private void invokeGenerator (Delta delta, IBuildContext context, IFileSystemAccess fileSystemAccess) {
    buildSemaphor.set(true);
    Resource resource = context.getResourceSet().getResource(delta.getUri(), true);
    if (shouldGenerate(resource, context)) {
      IResourceDescriptions index = resourceDescriptionsProvider.createResourceDescriptions();
      IResourceDescription resDesc = index.getResourceDescription(resource.getURI());
      List<IContainer> visibleContainers = containerManager.getVisibleContainers(resDesc, index);
      for (IContainer c : visibleContainers) {
        for (IResourceDescription rd : c.getResourceDescriptions()) {
          context.getResourceSet().getResource(rd.getURI(), true);
        }
      }

      generator.doGenerate(context.getResourceSet(), fileSystemAccess);
    }
  }
}

Note that IGenerator instance will be invoked once per changed resource, so potentially multiple times. We have to make sure that the IGenerator2 instance will be invoked just once. Therefore a thread local semaphor flag will be queried before the generator is called. In this scenario it does not matter whether a change or deletion should be handled. The generator always processes the whole model once.

The most important thing is to populate the BuildContext’s ResourceSet with all relevant model resources. It is important here collect the resources that are in scope of the built project. The right way to do this is to use the language’s IContainer.Manager to collect the resources that are within the visible containers. Each of them is queried for its contained IResourceDescription instances, from which the Resource instances are created.

Extending the generator class

Now open the generator class DomainmodelGenerator and implement IGenerator2. Add an implementation of the method from the extended interface:

...
// makes allContentsIterable available
import static extension org.eclipse.xtext.xtend2.lib.ResourceExtensions.*
...

class DomainmodelGenerator implements IGenerator2 {
  ...
  override void doGenerate(ResourceSet rs, IFileSystemAccess fsa) {
    val Iterable<Entity> entities = rs.resources.map(r|r.allContentsIterable.filter(typeof(Entity))).flatten

    fsa.generateFile("entities.txt", entities.compile)
  }

  def compile (Iterable<Entity> entities) '''
    «FOR e : entities SEPARATOR ", "»«e.name»«ENDFOR»
  '''

  ...
}

Guice configuration

Now bind the extended builder particpant in the UI module:

public class DomainmodelUiModule extends AbstractDomainmodelUiModule {
	@Override
	public Class<? extends IXtextBuilderParticipant> bindIXtextBuilderParticipant() {
		return JavaProjectBasedBuilderParticipant2.class;
	}
}

… and the generator to the Runtime module:

public class DomainmodelRuntimeModule extends AbstractDomainmodelRuntimeModule {
	public Class<? extends IGenerator2> bindIGenerator2 () {
		return DomainmodelGenerator.class;
	}
}

Result & Conclusion

After restarting your runtime workspace the generator will produce an “entities.txt” file to the src-gen folder of a project with .dmodel files, which will simply list the names of all found Entity instances.

This scenario is quite common, so it would be great if the Xtext framework could provide built-in support for that. I have therefore raised a feature request.

Game Development is highly complex, developed in multidisciplinary teams under high time pressure and quality requirements. And the complexity is even growing, whilst time pressure also. Time-to-market is everything there. If your game comes too late, you are out. If you don’t use the latest technologies, you are lame. How could such projects ever be successful just by coding and hacking? I could imagine that game developers are just too busy with developing their games in a traditional way to think how they could gain speed and quality by applying software engineering techniques like MDSD.

I would not wonder if they associate MDSD with drawing UML diagrams and wasting time clicking and drawing useless stuff. Model Driven Software Development is everything else than useless. It helps raising the level of abstraction, speeding up development and gaining quality. If applied correctly, of course. Of course they think their kind of software development is special and completely different than other disciplines. But let me say, it’s not the case. Every piece of software has generic parts, schematic parts and parts that don’t fit into one of the previous sections. And for the schematic parts, MDSD can always help. Don’t tell me that a multi-million, mission-critical enterprise project is less challenging than game development.

One of the most promising things for game development can be the usage of Domain Specific Languages (DSLs), especially textual ones. With Xtext 2.0 the development of textual DSLs with tight integration of expression languages and code generators has become easier than ever before. If you don’t ever tried Xtext, do it!

On October 4th there will be an interesting workshop at the 10th International Conference on Entertaining Computing (ICEC 2011) in Vancouver, Canada. The “1st Workshop on Game Development and MDSD” will bring experts from both worlds together. The Call for Position Papers is running now, deadline is July 30th. If you are in one of those businesses, submit a propopsal or attend the workshop. I think both “worlds” can really win a lot from working together. This workshop could be a good start.

With the help of the osgi-to-maven2 project I was now able with some tweaks to create the necessary POMs and deploy scripts for the plugins. What osgi-to-maven2 basically does is to analyze the plugin manifests and create POMs for them including dependencies. The project is not maintained at the moment and has a problem especially with the parsing of the plugins org.eclipse.ui.views and org.eclipse.ui.workbench, which have some strange directives in the Export-Package section where I could not find documentation about. The underlying manifest parser from Springsource (com.springsource.util.osgi) vomits when parsing those manifests, and also upgrading to the latest release does not help. Time to open a bug.


Export-Package: org.eclipse.ui;mandatory:="ui.workb
ench",org.eclipse.ui.about,org.eclipse.ui.actions;ui.workbench=split;
mandatory:="ui.workbench",org.eclipse.ui.activities,org.eclipse.ui.ap
plication,org.eclipse.ui.branding,org.eclipse.ui.browser;ui.workbench
=split;mandatory:="ui.workbench",org.eclipse.ui.commands,org.eclipse.

Also the osgi-to-maven2 tool does not handle absent optional libraries so far.

The Xtext 2.0, Xpand 1.1 and dependent plugins are available at the Fornax Platform’s repository http://fornax-platform.org/nexus/content/groups/public/ under the groupId org.eclipse.plugins.

Based on the deployed plugins it should be possible to setup a plain Maven build for Xtext 2 based projects.