Xtend – Generating from multiple input models

Xtext’s integration of Xtend based code generators is nice. If you save changed (or add/remove) DSL model files an incremental build will automatically be triggered and even removes derived artifacts if you remove the source from which it is generated. The IGenerator interface takes a Resource, and you can generate 1 or multiple target artifacts from the model elements within that resource.

However, the API does not allow to create a single artifact from multiple input models. But this is a common use case if you do serious code generation. I will show you how you could extend Xtext’s Domainmodel example to generate a simple text file that lists the name of all Entity instances of all .dmodel files that are reachable in the built project.

Adding an extended generator interface

Create an interface IGenerator2 that extends the IGenerator interface:

public interface IGenerator2 extends IGenerator {
	/**
	 * @param input - the input for which to generate resources
	 * @param fsa - file system access to be used to generate files
	 */
	public void doGenerate(ResourceSet input, IFileSystemAccess fsa);
}

The generator class will later implement this interface and generate a single artifact from all Entity instances in all resources.

Extending the JavaProjectBasedBuilderParticipant

The JavaProjectBasedBuilderParticipant is used by default to invoke the IGenerator implementation that was bound to the runtime module. We will extend this class to add additional logic and inject an IGenerator2 instance that will be invoked once per triggered build.

The builder participant is an UI concept, so create the extended class JavaProjectBasedBuilderParticipant2 in the UI plugin:

package org.eclipse.xtext.builder;

import java.util.List;

import org.eclipse.core.runtime.CoreException;
import org.eclipse.core.runtime.IProgressMonitor;
import org.eclipse.emf.ecore.resource.Resource;
import org.eclipse.xtext.builder.JavaProjectBasedBuilderParticipant;
import org.eclipse.xtext.generator.IFileSystemAccess;
import org.eclipse.xtext.generator.IGenerator2;
import org.eclipse.xtext.resource.IContainer;
import org.eclipse.xtext.resource.IResourceDescription;
import org.eclipse.xtext.resource.IResourceDescription.Delta;
import org.eclipse.xtext.resource.IResourceDescriptions;
import org.eclipse.xtext.resource.impl.ResourceDescriptionsProvider;

import com.google.inject.Inject;

public class JavaProjectBasedBuilderParticipant2 extends JavaProjectBasedBuilderParticipant {
  @Inject
  private ResourceDescriptionsProvider resourceDescriptionsProvider;

  @Inject
  private IContainer.Manager containerManager;

  @Inject (optional=true)
  private IGenerator2 generator;

  protected ThreadLocal<Boolean> buildSemaphor = new ThreadLocal<Boolean>();

  @Override
  public void build(IBuildContext context, IProgressMonitor monitor) throws CoreException {
    buildSemaphor.set(false);
    super.build(context, monitor);
  }

  @Override
  protected void handleChangedContents(Delta delta, IBuildContext context, IFileSystemAccess fileSystemAccess) {
    super.handleChangedContents(delta, context, fileSystemAccess);
    if (!buildSemaphor.get() && generator != null) {
      invokeGenerator(delta, context, fileSystemAccess);
    }
  }

  @Override
  protected void handleDeletion(Delta delta, IBuildContext context, IFileSystemAccess fileSystemAccess) {
    super.handleDeletion(delta, context, fileSystemAccess);
    if (!buildSemaphor.get() && generator != null) {
      invokeGenerator(delta, context, fileSystemAccess);
    }
  }

  private void invokeGenerator (Delta delta, IBuildContext context, IFileSystemAccess fileSystemAccess) {
    buildSemaphor.set(true);
    Resource resource = context.getResourceSet().getResource(delta.getUri(), true);
    if (shouldGenerate(resource, context)) {
      IResourceDescriptions index = resourceDescriptionsProvider.createResourceDescriptions();
      IResourceDescription resDesc = index.getResourceDescription(resource.getURI());
      List<IContainer> visibleContainers = containerManager.getVisibleContainers(resDesc, index);
      for (IContainer c : visibleContainers) {
        for (IResourceDescription rd : c.getResourceDescriptions()) {
          context.getResourceSet().getResource(rd.getURI(), true);
        }
      }

      generator.doGenerate(context.getResourceSet(), fileSystemAccess);
    }
  }
}

Note that IGenerator instance will be invoked once per changed resource, so potentially multiple times. We have to make sure that the IGenerator2 instance will be invoked just once. Therefore a thread local semaphor flag will be queried before the generator is called. In this scenario it does not matter whether a change or deletion should be handled. The generator always processes the whole model once.

The most important thing is to populate the BuildContext’s ResourceSet with all relevant model resources. It is important here collect the resources that are in scope of the built project. The right way to do this is to use the language’s IContainer.Manager to collect the resources that are within the visible containers. Each of them is queried for its contained IResourceDescription instances, from which the Resource instances are created.

Extending the generator class

Now open the generator class DomainmodelGenerator and implement IGenerator2. Add an implementation of the method from the extended interface:

...
// makes allContentsIterable available
import static extension org.eclipse.xtext.xtend2.lib.ResourceExtensions.*
...

class DomainmodelGenerator implements IGenerator2 {
  ...
  override void doGenerate(ResourceSet rs, IFileSystemAccess fsa) {
    val Iterable<Entity> entities = rs.resources.map(r|r.allContentsIterable.filter(typeof(Entity))).flatten

    fsa.generateFile("entities.txt", entities.compile)
  }

  def compile (Iterable<Entity> entities) '''
    «FOR e : entities SEPARATOR ", "»«e.name»«ENDFOR»
  '''

  ...
}

Guice configuration

Now bind the extended builder particpant in the UI module:

public class DomainmodelUiModule extends AbstractDomainmodelUiModule {
	@Override
	public Class<? extends IXtextBuilderParticipant> bindIXtextBuilderParticipant() {
		return JavaProjectBasedBuilderParticipant2.class;
	}
}

… and the generator to the Runtime module:

public class DomainmodelRuntimeModule extends AbstractDomainmodelRuntimeModule {
	public Class<? extends IGenerator2> bindIGenerator2 () {
		return DomainmodelGenerator.class;
	}
}

Result & Conclusion

After restarting your runtime workspace the generator will produce an “entities.txt” file to the src-gen folder of a project with .dmodel files, which will simply list the names of all found Entity instances.

This scenario is quite common, so it would be great if the Xtext framework could provide built-in support for that. I have therefore raised a feature request.

Advertisements

Game Development – A whole new area for Eclipse Modeling?

The Eclipse Modeling Project provides the world’s leading set of tools and frameworks that are used for successfully applying model driven software development techniques in various areas. Successful adoption are known in Enterprise Computing, Embedded System Development, Mobile Development etc. But what about Game Development? I have not heard about Game productions that use Eclipse Modeling or Model Driven Software Development in general so far. I cannot know about all projects in the world, but at least it is an indicator that this development technique is at least not wide adopted in the branch of Game Development.

Game Development is highly complex, developed in multidisciplinary teams under high time pressure and quality requirements. And the complexity is even growing, whilst time pressure also. Time-to-market is everything there. If your game comes too late, you are out. If you don’t use the latest technologies, you are lame. How could such projects ever be successful just by coding and hacking? I could imagine that game developers are just too busy with developing their games in a traditional way to think how they could gain speed and quality by applying software engineering techniques like MDSD.

I would not wonder if they associate MDSD with drawing UML diagrams and wasting time clicking and drawing useless stuff. Model Driven Software Development is everything else than useless. It helps raising the level of abstraction, speeding up development and gaining quality. If applied correctly, of course. Of course they think their kind of software development is special and completely different than other disciplines. But let me say, it’s not the case. Every piece of software has generic parts, schematic parts and parts that don’t fit into one of the previous sections. And for the schematic parts, MDSD can always help. Don’t tell me that a multi-million, mission-critical enterprise project is less challenging than game development.

One of the most promising things for game development can be the usage of Domain Specific Languages (DSLs), especially textual ones. With Xtext 2.0 the development of textual DSLs with tight integration of expression languages and code generators has become easier than ever before. If you don’t ever tried Xtext, do it!

On October 4th there will be an interesting workshop at the 10th International Conference on Entertaining Computing (ICEC 2011) in Vancouver, Canada. The “1st Workshop on Game Development and MDSD” will bring experts from both worlds together. The Call for Position Papers is running now, deadline is July 30th. If you are in one of those businesses, submit a propopsal or attend the workshop. I think both “worlds” can really win a lot from working together. This workshop could be a good start.

Xtext 2.0 / Xpand 1.1 plugins available as Maven artifacts

It’s now one week after release of Eclipse Indigo, and with that Xtext 2.0 and Xpand 1.1. For those who use Maven without Tycho to build projects using these frameworks it is of course necessary to have all related plugins available in some Maven repository.

With the help of the osgi-to-maven2 project I was now able with some tweaks to create the necessary POMs and deploy scripts for the plugins. What osgi-to-maven2 basically does is to analyze the plugin manifests and create POMs for them including dependencies. The project is not maintained at the moment and has a problem especially with the parsing of the plugins org.eclipse.ui.views and org.eclipse.ui.workbench, which have some strange directives in the Export-Package section where I could not find documentation about. The underlying manifest parser from Springsource (com.springsource.util.osgi) vomits when parsing those manifests, and also upgrading to the latest release does not help. Time to open a bug.


Export-Package: org.eclipse.ui;mandatory:="ui.workb
ench",org.eclipse.ui.about,org.eclipse.ui.actions;ui.workbench=split;
mandatory:="ui.workbench",org.eclipse.ui.activities,org.eclipse.ui.ap
plication,org.eclipse.ui.branding,org.eclipse.ui.browser;ui.workbench
=split
;mandatory:="ui.workbench",org.eclipse.ui.commands,org.eclipse.

Also the osgi-to-maven2 tool does not handle absent optional libraries so far.

The Xtext 2.0, Xpand 1.1 and dependent plugins are available at the Fornax Platform’s repository http://fornax-platform.org/nexus/content/groups/public/ under the groupId org.eclipse.plugins.

Based on the deployed plugins it should be possible to setup a plain Maven build for Xtext 2 based projects.