Xbase Customization: Redefining operator keywords

If you use Xbase in your Xtext based DSL, you are usually satisfied with the set of operators the expression language defines. They are closely related to what you are used to in Java or similar languages.

However, in a customer’s workshop the customer wished to have custom keywords for some operators. For example, the operator && should be alternatively presented as AND, and the || operator as OR.

To demonstrate this customization we’ll start with Xtext’s famous Domainmodel Example. The Domainmodel.xtext grammar is derived from org.eclipse.xtext.xbase.Xbase and the Operation rule uses an XBlockExpression for the Operation’s body:

grammar org.eclipse.xtext.example.domainmodel.Domainmodel with org.eclipse.xtext.xbase.Xbase

...

Operation:
	'op' name=ValidID '(' (params+=FullJvmFormalParameter (',' params+=FullJvmFormalParameter)*)? ')' (':' type=JvmTypeReference)?
		body=XBlockExpression;

Unit Test

We’ll extend the language test-driven, thus we first create a unit test that uses the new feature, but will fail first until we have successfully implemented it. Fortunately there is already a suitable test class in project org.eclipse.xtext.example.domainmodel.tests.

We extend the class ParserTest.xtend:

	@Test
	def void testOverriddenKeyword() {
		val model = '''
			package example {
			  entity MyEntity {
			    property : String
			    op foo(String s) {
			    	return s!= null && s.length > 0 AND s.startsWith("bar")
			    }
			  }
			}
		'''.parse
		val pack = model.elements.head as PackageDeclaration
		val entity = pack.elements.head as Entity
		val op = entity.features.last as Operation
		val method = op.jvmElements.head as JvmOperation
		model.eResource.assertNoErrors
		Assert::assertEquals("boolean", method.returnType.simpleName)
	}

Note the Operation body, the expression uses both presentations of the And-operator.

return s!= null && s.length > 0 AND s.startsWith("bar")

When the ParserTest is executed it will now fail, of course, but only for the AND keyword:

java.lang.AssertionError: Expected no errors, but got :
ERROR (org.eclipse.xtext.diagnostics.Diagnostic.Linking) 
'The method or field AND is undefined' 
on XFeatureCall, offset 119, length 3
ERROR (org.eclipse.xtext.xbase.validation.IssueCodes.unreachable_code) 'Unreachable expression.' on XFeatureCall, offset 119, length 3

	at org.junit.Assert.fail(Assert.java:88)
	at org.eclipse.xtext.junit4.validation.ValidationTestHelper.assertNoErrors(ValidationTestHelper.java:187)
	at org.eclipse.xtext.example.domainmodel.tests.ParserTest.testOverriddenKeyword(ParserTest.java:268)
...

Overloading the operator rules

Looking at Xbase.xtext shows that Xbase defines separate data type rules for operators:

OpOr:
	'||';
OpAnd:
	'&&';

Xtext’s Grammar Mixin feature allows a redefinition of those rules. The obvious customization is in Domainmodel.xtext:

OpOr:
	'||' | 'OR';
OpAnd:
	'&&' | 'AND';

Regenerating the language’s Xtext implementation makes those keywords available to the syntax. However, the unit test still fails, but now with a different error:

java.lang.AssertionError: Expected no errors, but got :
ERROR (org.eclipse.xtext.diagnostics.Diagnostic.Linking) 
'AND cannot be resolved.' on XBinaryOperation, offset 119, length 3

	at org.junit.Assert.fail(Assert.java:88)
	at org.eclipse.xtext.junit4.validation.ValidationTestHelper.assertNoErrors(ValidationTestHelper.java:187)
	at org.eclipse.xtext.example.domainmodel.tests.ParserTest.testOverriddenKeyword(ParserTest.java:268)

OperatorMapping

The missing piece is a customization of class OperatorMapping. Thus we create a subclass OperatorMappingCustom with constant QualifiedNames for the additional operator keywords and bind it in DomainmodelRuntimeModule:

@Singleton
public class OperatorMappingCustom extends OperatorMapping {
	public static final QualifiedName AND_2 = create("AND");
	public static final QualifiedName OR_2 = create("OR");
}
public class DomainmodelRuntimeModule extends AbstractDomainmodelRuntimeModule {
	[...]
	public Class bindOperatorMapping() {
		return OperatorMappingCustom.class;
	}
}

A naive approach is here to overload the initializeMapping(), as the Javadoc suggests (“Clients may want to override #initializeMapping() to add other operators.“):

But this fails again:

com.google.inject.CreationException: Guice creation errors:

1) Error injecting constructor, java.lang.IllegalArgumentException: value already present: operator_and
  at org.eclipse.xtext.example.domainmodel.OperatorMappingCustom.(Unknown Source)
  at org.eclipse.xtext.example.domainmodel.OperatorMappingCustom.class(Unknown Source)
  while locating org.eclipse.xtext.example.domainmodel.OperatorMappingCustom
  while locating org.eclipse.xtext.xbase.scoping.featurecalls.OperatorMapping
    for field at org.eclipse.xtext.xbase.util.XExpressionHelper.operatorMapping(Unknown Source)
  while locating org.eclipse.xtext.xbase.util.XExpressionHelper
    for field at org.eclipse.xtext.xbase.validation.XbaseValidator.expressionHelper(Unknown Source)
  at org.eclipse.xtext.service.MethodBasedModule.configure(MethodBasedModule.java:57)
  while locating org.eclipse.xtext.example.domainmodel.validation.DomainmodelJavaValidator
Caused by: java.lang.IllegalArgumentException: value already present: operator_and
	at com.google.common.collect.HashBiMap.put(HashBiMap.java:237)
	at com.google.common.collect.HashBiMap.put(HashBiMap.java:214)
	at org.eclipse.xtext.example.domainmodel.OperatorMappingCustom.initializeMapping(OperatorMappingCustom.java:25)
	at org.eclipse.xtext.xbase.scoping.featurecalls.OperatorMapping.(OperatorMapping.java:121)
	at org.eclipse.xtext.example.domainmodel.OperatorMappingCustom.(OperatorMappingCustom.java:18)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	...

Instead, the method getMethodName() must be overridden and delegate the invocations of the new operators to the existing ones. The resulting OperatorMappingCustom is then:

@Singleton
public class OperatorMappingCustom extends OperatorMapping {
	public static final QualifiedName AND_2 = create("AND");
	public static final QualifiedName OR_2 = create("OR");

	@Override
	public QualifiedName getMethodName(QualifiedName operator) {
		if (AND_2.equals(operator)) {
			return getMethodName(AND);
		}
		if (OR_2.equals(operator)) {
			return getMethodName(OR);
		}
		return super.getMethodName(operator);
	}
}

Finally, the unit test will execute successful.

Summary

Xbase allows customization of operators by overriding the operator’s data type rule from Xbase.xtext. This adds the operator keywords to the language, but fails at runtime. Additionaly the class OperatorMapping must be customized and method getMethodName() overloaded.

How to limit proposed Java types to implementors of an interface

When you develop a DSL that derives from Xbase you likely want to refer to Jvm types through a JvmTypeReference.

CustomBehavior :
    'custom' (name = ID | 'ref' realizedBy=JvmTypeReference) (label = STRING);

If you hit CTRL+SPACE at such a position in the editor, by default you will get all visible Java types proposed:

This is not really useful when it only makes sense to refer to specifc types. If you want to limit the proposals to just implementors of a specific interface, you can customize this in the proposal provider by overriding the method completeJvmParameterizedTypeReference_Type(). The following example from Spray shows limiting the types to implementors of Graphiti’s ICustomFeature interface in the context of Spray’s CustomBehavior:

public class SprayProposalProvider extends AbstractSprayProposalProvider {
    @Inject
    private IJvmTypeProvider.Factory      jvmTypeProviderFactory;
    @Inject
    private ITypesProposalProvider        typeProposalProvider;
    
    @Override
    public void completeJvmParameterizedTypeReference_Type(EObject model, Assignment assignment, ContentAssistContext context, ICompletionProposalAcceptor acceptor) {
        if (EcoreUtil2.getContainerOfType(model, CustomBehavior.class) != null) {
            final IJvmTypeProvider jvmTypeProvider = jvmTypeProviderFactory.createTypeProvider(model.eResource().getResourceSet());
            // Graphiti specific
            final JvmType interfaceToImplement = jvmTypeProvider.findTypeByName(ICustomFeature.class.getName());
            typeProposalProvider.createSubTypeProposals(interfaceToImplement, this, context, SprayPackage.Literals.BEHAVIOR__REALIZED_BY, TypeMatchFilters.canInstantiate(), acceptor);
        } else {
            super.completeJvmParameterizedTypeReference_Type(model, assignment, context, acceptor);
        }
    }

Now you will get the desired subtypes as proposals:

Note that this does only limit proposals to certain types. It does not prevent you from actually referring to another type that you don’t get proposed. This needs to be restricted in your scope provider.
 

EclipseCon 2011 warmup

EclipseCon is just about to start and I am glad that I can make it this year. I have the chance to present Project Spray on thursday (Nov 3rd, 11:30 AM Bürgersaal 2) together with Jos Warmer. Spray aims to provide tooling to facilitate the creation of visual editors based on the Graphiti framework by the use of Xtext based DSL and Xtend code generator. You can basically compare it with what GMF Tools does for GMF, just with a DSL based approach.

EclipseCon Europe 2011

The project was founded at this year’s Code Generation conference, and Jos contributed an initial state of Spray derived from a customer’s project back in August. Since then the project team refactored the code base quite a bit. We are still in an early project phase. Most of the development is done in spare time yet, like Open Source projects often start. That was quite tough besides my full packed work schedule. Many hours on train or evenings in the hotel were spent to push this project.

At the moment I am just about to release version 0.3.0 of the Spray SDK, which I will use for our demo on thursday. Of course we want to gain a large audience. I am honest enough to say that attendees won’t see a production ready tool yet, but of course it is in working state. Jan announced a pie fight for visual tooling, I am ready to open my mouth wide enough to catch the incoming pies.

For our session we don’t want to start any discussion on graphical editing vs. visualization, whether GMF is better than Graphiti, or DSL / code generation vs. framework approach. There is always a niche for everything, and we are confident that Spray fills one of these gaps.

Besides what an end user actually can do with Spray at this moment there are quite some hidden gems in the project. I am using Spray to have also a non-trivial example for Eclipse Modeling tooling. You will find, for example

At EclipseCon I will take the chance to exchange with some experts in the fields of Modeling, especially Xtext and Graphiti. I will lay another focus on Build systems, especially Maven. Besides that there are many other interesting talks, far more than I can attend. And not to forget to meet all the other Eclipse enthusiasts. See you there!