Karsten's Blog

September 13, 2011

Setting template file encoding in Xpand workflow

Filed under: Xpand — kthoms @ 2:14 PM

A typical problem when running Xpand is file encoding of the Xpand templates. Xpand uses guillemot brackets («»), which are not available in all character sets. Often, ISO-8859-1 and UTF-8 are good choices for encoding Xpand templates. However, on Mac the default encoding is MacRoman, and I would recommend to change that for interoperability reason. I would recommend changing the resource encoding setting on the root folder containing Xpand templates, or even the project, and checking in the settings. This way the encoding is shared amongst team members.

When executing the Xpand generator with MWE it is important that the templates are read with the right encoding. Otherwise you will get an error like this:
org.eclipse.internal.xtend.xtend.parser.ParseException: no viable alternative at character '�' on line 159
Xpand’s Generator component has a property fileEncoding, but this is the encoding used by the outlets to write files. If you need to change the behavior for reading files, you need to set the encoding value of the ResourceManager. The configuration is like this:

MWE1:


    ...




MWE2

  component = org.eclipse.xpand2.Generator {
    ...
    resourceManager = org.eclipse.xtend.expression.ResourceManagerDefaultImpl {
     	fileEncoding = "ISO-8859-1"
    }
  }

August 17, 2011

Using Xtext injected types in Xpand

Filed under: Xpand, Xtext — kthoms @ 12:57 PM

During a project migration I stumbled over the need to access the IScopeProvider from an Xtext based language within Xtend(1) extensions that are used from Xpand templates. This is not quite how Xpand would work out-of-the-box, so here is an approach to solve it.

InjectableGenerator

I subclassed org.eclipse.xpand2.Generator and called it InjectableGenerator. The intention is to register a global variable for a configurable set of types (e.g. the IScopeProvider). The name of the variable is the class name that should be get from Guice. To add injectors, instances of ISetup are registered to the component, similar to Xtext’s org.eclipse.xtext.mwe.Reader component.

The hook that is used to register the global variables to Xpand’s execution context is the method getGlobalVars(WorkflowContext ctx), which is overridden here.

package org.eclipselabs.xtext.xpand;

import java.util.List;
import java.util.Map;
import java.util.Set;

import org.eclipse.emf.mwe.core.WorkflowContext;
import org.eclipse.xpand2.Generator;
import org.eclipse.xtend.expression.Variable;
import org.eclipse.xtext.ISetup;

import com.google.common.collect.Lists;
import com.google.common.collect.Sets;
import com.google.inject.ConfigurationException;
import com.google.inject.Injector;

/**
 * Allows Xtext based dependency injection. Injected instances are registered as
 * global vars to the Xpand Execution Context.
 */
public class InjectableGenerator extends Generator {
	private List<Injector> injectors = Lists.newArrayList();

	public void addRegister(ISetup setup) {
		injectors.add(setup.createInjectorAndDoEMFRegistration());
	}

	protected List<Injector> getInjectors() {
		return injectors;
	}
	
	private Set<Class<?>> injectedTypes = Sets.newHashSet();

	public void addInjectGlobalVar (String className) {
		try {
			Class<?> clazz = Class.forName(className);
			injectedTypes.add(clazz);
		} catch (Exception e) {
			throw new IllegalArgumentException("Invalid type: "+className);
		}
	}
	
	@Override
	protected Map<String, Variable> getGlobalVars(WorkflowContext ctx) {
		Map<String, Variable> result = super.getGlobalVars(ctx);
		// try to get an instance of each configured type
		for (Class<?> clazz : injectedTypes) {
			boolean configured = false;
			for (Injector injector : injectors) {
				try {
					Object obj = injector.getInstance(clazz);
					result.put(clazz.getName(), new Variable(clazz.getName(), obj));
					configured = true;
				} catch (ConfigurationException e) {
					; // ignore
				}
			}
			if (!configured) {
				throw new IllegalStateException("Could not configure instance of "+clazz.getName());
			}
		}
		return result;
	}
}

Workflow Configuration

In your workflow configuration you would of course now use the specialized Generator component. It could be used like this:

...
component = org.eclipselabs.xtext.xpand.InjectableGenerator {
   register = my.dsl.MyDslStandaloneSetup {}
   injectGlobalVar = "org.eclipse.xtext.scoping.IScopeProvider"
   // injectGlobalVar = ... other types
   
   // further configuration of Xpand Generator
}

Execution context aware Java extensions

It is not possible to directly access the ExecutionContext from within Xpand templates or Xtend functions. We need to delegate to Java extensions. The classes that we delegate to must implement the IExecutionContextAware interface. The setExecutionContext(ExecutionContext ctx) method is the right place to retrieve the IScopeProvider and bind it to an instance variable.

public class MyExtensions implements IExecutionContextAware {
	protected IScopeProvider scopeProvider;

	@Override
	public void setExecutionContext(ExecutionContext ctx) {
		if (ctx.getGlobalVariables().get("scopeProvider")!=null) {
			this.scopeProvider = (IScopeProvider) ctx.getGlobalVariables().get("scopeProvider").getValue();
		} else if (ctx.getGlobalVariables().get(IScopeProvider.class.getName())!=null) {
			this.scopeProvider = (IScopeProvider) ctx.getGlobalVariables().get(IScopeProvider.class.getName()).getValue();
		} else {
			throw new IllegalStateException("No IScopeProvider found as global var in the execution context");
		}
	}

	// add (non-static!) extension methods here!
        public List<EObject> someMethod (org.eclipse.emf.ecore.EObject context) {
          ... do something
        }
}

Note that we also query the global variable named “scopeProvider” here. This is because in another context (i.e. Check based validation) the same extension would be used with another execution context. The InjectableExecutionContext binds the IScopeProvider to that name.

It might be a good idea to have an abstract base class for these Extension classes.

Now you can use the methods defined in the Java extension class and access the required instances from your language configuration within.

// file: MyExtensions.ext
List[emf::EObject] someMethod (emf::EObject ctx) : JAVA MyExtensions.someMethod(org.eclipse.emf.ecore.EObject);

July 5, 2011

Xtext 2.0 / Xpand 1.1 plugins available as Maven artifacts

Filed under: Maven, Xpand, Xtext — Tags: , , , , — kthoms @ 5:22 AM

It’s now one week after release of Eclipse Indigo, and with that Xtext 2.0 and Xpand 1.1. For those who use Maven without Tycho to build projects using these frameworks it is of course necessary to have all related plugins available in some Maven repository.

With the help of the osgi-to-maven2 project I was now able with some tweaks to create the necessary POMs and deploy scripts for the plugins. What osgi-to-maven2 basically does is to analyze the plugin manifests and create POMs for them including dependencies. The project is not maintained at the moment and has a problem especially with the parsing of the plugins org.eclipse.ui.views and org.eclipse.ui.workbench, which have some strange directives in the Export-Package section where I could not find documentation about. The underlying manifest parser from Springsource (com.springsource.util.osgi) vomits when parsing those manifests, and also upgrading to the latest release does not help. Time to open a bug.


Export-Package: org.eclipse.ui;mandatory:="ui.workb
ench",org.eclipse.ui.about,org.eclipse.ui.actions;ui.workbench=split;
mandatory:="ui.workbench",org.eclipse.ui.activities,org.eclipse.ui.ap
plication,org.eclipse.ui.branding,org.eclipse.ui.browser;ui.workbench
=split
;mandatory:="ui.workbench",org.eclipse.ui.commands,org.eclipse.

Also the osgi-to-maven2 tool does not handle absent optional libraries so far.

The Xtext 2.0, Xpand 1.1 and dependent plugins are available at the Fornax Platform’s repository http://fornax-platform.org/nexus/content/groups/public/ under the groupId org.eclipse.plugins.

Based on the deployed plugins it should be possible to setup a plain Maven build for Xtext 2 based projects.

August 19, 2010

Tycho build works also for oAW 4.3 projects

Filed under: Maven, openArchitectureWare, Xpand — kthoms @ 10:16 PM

Yesterday I showed how Xtext 1.0 projects can be build with Maven Tycho. So why shouldn’t the same setup don’t work for ‘old’ openArchitectureWare Xpand projects? This works like a charm. I took the simple project that the Xpand project wizard creates and added the POM, and it worked! You can download the sample project here. See the POM below, a setup for other oAW Xpand projects should be rather similar.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>p2.osgi.bundle</groupId>
	<artifactId>org.openarchitectureware.example.xpand.tycho</artifactId>
	<version>1.0.0</version>
	<packaging>eclipse-plugin</packaging>
	<name>org.openarchitectureware.example.xpand.tycho</name>

	<properties>
		<version.tycho>0.9.0</version.tycho>
	</properties>

	<build>
		<resources>
			<resource>
				<directory>src</directory>
			</resource>
		</resources>
		<plugins>
			<plugin>
				<groupId>org.sonatype.tycho</groupId>
				<artifactId>tycho-maven-plugin</artifactId>
				<version>${version.tycho}</version>
				<extensions>true</extensions>
			</plugin>
			<plugin>
				<groupId>org.sonatype.tycho</groupId>
				<artifactId>target-platform-configuration</artifactId>
				<version>${version.tycho}</version>
				<configuration>
					<resolver>p2</resolver>
				</configuration>
			</plugin>
			<plugin>
				<groupId>org.fornax.toolsupport</groupId>
				<artifactId>fornax-oaw-m2-plugin</artifactId>
				<version>3.1.0-SNAPSHOT</version>
				<configuration>
					<workflowEngine>oaw</workflowEngine>
					<workflowDescriptor>workflow/generator.oaw</workflowDescriptor>
				</configuration>
				<executions>
					<execution>
						<phase>generate-sources</phase>
						<goals>
							<goal>run-workflow</goal>
						</goals>
					</execution>
				</executions>
			</plugin>
			<plugin>
				<artifactId>maven-clean-plugin</artifactId>
				<version>2.4.1</version>
				<configuration>
					<filesets>
						<fileset>
							<directory>src-gen</directory>
							<includes>
								<include>**/**</include>
							</includes>
						</fileset>
					</filesets>
				</configuration>
			</plugin>
		</plugins>
	</build>
	<repositories>

		<repository>
			<id>p2.openarchitectureware</id>
			<url>http://www.openarchitectureware.org/updatesite/milestone/</url>
			<layout>p2</layout>
		</repository>
		<repository>
			<id>p2.eclipse.helios</id>
			<url>http://download.eclipse.org/releases/helios</url>
			<layout>p2</layout>
		</repository>
		<!-- At the moment the Fornax plugin is only available as snapshot -->
		<repository>
			<id>fornax-snapshots</id>
			<url>http://www.fornax-platform.org/archiva/repository/snapshots/</url>
			<releases>
				<enabled>false</enabled>
			</releases>
			<snapshots>
				<enabled>true</enabled>
			</snapshots>
		</repository>
	</repositories>
	<pluginRepositories>
		<pluginRepository>
			<id>fornax-snapshots</id>
			<url>http://www.fornax-platform.org/archiva/repository/snapshots/</url>
			<releases>
				<enabled>false</enabled>
			</releases>
			<snapshots>
				<enabled>true</enabled>
			</snapshots>
		</pluginRepository>
	</pluginRepositories>
</project>

February 15, 2010

Lazy evaluation in Xpand

Filed under: Eclipse, openArchitectureWare, Xpand — kthoms @ 10:36 AM

Usually code generation is a purely sequential process. Since the model does not change during the generation of an artifact all content can be computed in the template where it is needed for the output. But sometimes there is the wish to defer the output to a later point of time during the generation of an artifact.

The typical use case for this is import statements. If for example you want to generate a Java class and want to import all used types then the following alternatives are given:

  • Compute the types that the about-to-be-generated class will use
  • Print out all type names full qualified whenever needed and organize the imports with a postprocessor. For Java code generation the Hybridlabs Beautifier is used widely.

However, both approaches do not seamlessly solve the problem. What really is needed is some kind of lazy evaluation in Xpand. Therefore Jos Warmer wrote a feature proposal once. The feature that he proposed for Xpand is called Insertion Point. The idea was to mark some point in the Xpand template where some code will be inserted at a later point of time. Code is evaluated into this insertion point when the content can be derived easier.

From this feature proposal Feature Request #261607 was created in Eclipse Bugzilla system. In this bug entry, and also offline, a lively discussion arose in the team. The challenges for this feature request were:

  • The Xpand language has a rather small set of keywords. Adding this feature, which is used in some cases only, should not introduce too much changes to the Xpand language
  • An implementation should not break existing code
  • The solution should enable decent evaluation, e.g. avoiding duplicates, sorting

The latest proposal just introduces just one new keyword in Xpand, but requires an implementation pattern with Xtend function. The proposal is to add a keyword ONFILECLOSE to the EXPAND statement. By calling EXPAND with ONFILECLOSE the evaluation of the EXPAND statement is deferred until the FILE statement is closed. Any state that is used by the called definition is computed during the FILE evaluation. The EXPAND statement has to be evaluated with the execution context which is active when reaching the EXPAND statement.

Let’s see this by example. As example we take the project that Xpand’s project wizard creates, with small changes. The entities and types have now an additional ‘packageName’ attribute, and the both entities have been assigned different packages ‘entities1′ and ‘entities2′. Additionally entity ‘Person’ has a feature ‘birthday’ of type Date, which is mapped to java.util.Date. Therefore class Person.java has to import entities2.Address and java.util.Date. The used types should be collected when rendering instance variables and accessor methods, but inserted earlier in the code.

First take a look at the template code:


As you can see the definition ImportBlock is called for each Type instance (e.g. Entity and DataType instances) in the collection returned by the Xtend function UsedType(). In an alternative approach this function would be responsible to compute all the types that will be used by this Entity instance. But for the new implementation it just creates an empty list and returns that one. The implementation of the UsedType() function (in file GeneratorExtensions.ext) is:

create List[Type] UsedType (Entity e) : (List[Type]) {};

So at the time the Xpand engine reaches EXPAND ImportBlock… the list would be empty. Now note the ONFILECLOSE keyword at the end of the EXPAND statement. This one tells the engine now that the evaluation of this Xpand statement should be deferred until the file is about to be closed. During evaluation of the template code, in the FOREACH loop, another extension function addUsedType(f.type) is called. This one adds the type of the current processed feature to the collection returned by UsedTypes(). Therfore it is important that the UsedType() function uses the create keyword, since we want to create the collection on first access for one Entity and return the same instance when UsedTypes() is called later for that Entity again.

The function addUsedType() is used in the template like this:

«addUsedType(f.type)»

Xpand would print out the result of the function as string, but we don’t want to produce any output by calling the function. Therefore we assure that the function adds the type to the collection and returns an empty string:

addUsedType (Entity e, Type t) : UsedType(e).add(t) -> "";

During evaluation it could be that types were used multiple times within the template, but we want just one import statement per type. Further we don’t need imports for types from the java.lang package (here the packageName information for the DataType instances String and Integer is null) or for types that are in the same package like the entity. Therefore we transform the UsedType() collection before finally invoking the ImportBlock definition.

«EXPAND ImportBlock FOREACH UsedType()
 .select(t|t.packageName!=null && t.packageName!=this.packageName).toSet()
 .sortBy(t|t.qualifiedName()) ONFILECLOSE»

Conclusion

Lazy evaluation allows code generated with Xpand in an non-sequential matter. The proposed solution using ONFILECLOSE solves the desired Insertion Point feature by adding just one additional keyword to the Xpand language. It does not break existing template code. When accepted this code will be contributed to Xpand 0.8.0-M6 soon. For those who want to test it I have created a feature patch for the org.eclipse.xpand feature. The example project with the sources listed in this article can be downloaded here.

Feedback is welcome and is best placed in the bugzilla feature request.

The Silver is the New Black Theme Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 375 other followers