Preparations for CodeGeneration 2013

As I am addicted to code generation and DSLs, the CodeGeneration conference in Cambridge is always a must each year. Last year I could not make it, since I had the chance to speak at EclipseCon North America, which was in the same week. This year Mark took EclipseCon into his considerations (it was last week), so me and my colleagues from itemis will be there again. Actually, this year we will be more itemis guys then ever. Mark already assumed in his opening words at CG2011 that almost everyone from itemis would be there, this year we prove itemis is larger. I think our company is so close related to the conference theme that it is natural that we have lots to present, and much interest to hear others about what they are doing and have learned in the past.

screenshot 2013-04-05 um 10.55.19

Before the actual CodeGeneration conference starts on Wednesday, there are some pre-conference activities. My colleagues Holger Schill and Moritz Eysholdt will hold an intensive 2-day Xtext workshop on monday and tuesday.

I will arrive monday noon, since I take the early flight directly from my hometown Dortmund to London Luton. From there, I have to take a 2 hour bus trip to Cambridge. In the evening, I plan to meet Holger, Moritz, Meinte Boersma and hopefully some others in the Castle Inn pub. When you arrive on monday, drop into the Castle Inn roughly at 20 PM (I guess we go to a restaurant before). You can reach me there on my mobile phone: screenshot 2013-04-05 um 12.30.02

DSC00412

On tuesday this year’s Language Workbench Challenge summit takes place. We have 14 submissions (wow!) for the LWC13 assignment. I have been working on the Xtext submission together with 2 colleagues, Johannes Dicks and Thomas Kutz. The results are available as open-source project lwc13-xtext at Eclipselabs. We have prepared a detailed step-by-step tutorial as submission paper. The resulting document LWC13-XtextSubmission.pdf is available for download. On the project homepage I have placed today a quick start tutorial. Oh boy, this project did cost some time. The actual solution is not much code, but as often, it is harder to write less code than more. It could be even less, but we took care of that the code is readable and understandable. And writing the document is at least that much work as the implementation.

screenshot 2013-04-08 um 13.52.06

Every presenter has only 15 minutes to present their approach. 15 minutes presentation for that much work. I guess that the other participants did invest also quite some time. Both of my co-authors got the chance to visit the conference, and Thomas will support me with the presentation. He will demo the resulting JSF application and DSL source code while I do the main talking. We did a test run of the talk yesterday evening, and easily exceeded 20 minutes. I think Angelo will bring his egg timer again, which begs no pardon with the talk time of speakers. But only that way we will be able to run 14 talks on one day. We will have to restrict on the most important aspects only.

Besides my colleagues Thomas and Johannes also Sven Lange will join us then. I have the pleasure to work with him in my long-time project at Deutsche Börse (German Stock Exchange), now since over half a year. Sven is a highly motivated, skilled and smart person. It is still the same project I have reported about at CodeGeneration 2009 together with my former colleague Heiko Behrens. Sven is full-time working on this project, while I am for 20% scheduled. We have migrated here a huge code generator project from Xpand to Xtend. This alone would be worth an experience report session. Sven is working on Xtend support for IntelliJ, which he might present in a Lightning Talk on wednesday.

Wednesday the conference will start. I will have my main talk “Alive and Kicking with Code Generation” together with Dr. Boris Baginski from ATOSS Software AG after lunch at 13:45.

screenshot 2013-04-05 um 10.57.34

Currently we are finalizing our presentation slides. Boris has been ill for some days and busy with a new release of their ASES product, a workforce management suite. This is really an interesting customer and project. They are evolving this product now for 25 years, and they make use of code generation for ages. I think one can say that it helped them to survive in their business, some contestants did not manage to make larger platform shifts and died. Most of them tried a big-bang replacement, but the business is too fast evolving so that the target is moving steadily. Boris and I will speak about this product and how it has been evolved over the years. ATOSS was one of the first major projects using openArchitectureWare 4 (which mainly means Xpand), and now they are currently preparing a shift to Xtend.

I am glad that this talk is already on wednesday, I never come to rest until I finished some talk. After it, I can just relax and enjoy the conference. I am expecting some interesting insights on different approaches. Especially experience reports are interesting for me. I did not finally decide which sessions I will attend. At the moment I plan to see John Hutchinson with “The Use of Model-Driven Development in Industry” in the morning, and Darius Silingas with “Why MDA Fails: Analysis of Unsuccessful Cases” in the afternoon.

In the evening it is again time for the punting boat tour. I already attended three times, but it will be great fun again for sure. Let’s hope the weather is not too bad. I saw a prediction of ~10°C and possibility of light shower. In the past we had luck, and on a warm, sunny day the tour is double fun. However, I’ll better put an umbrella into the suitcase.

On thursday I have again an active part in the hands-on session “Have Your Language Built While You Wait”, which is hosted by Risto Pohjonen from MetaCase. The idea of this session is that attendees can get a DSL with the language workbench of their choice built with the help of experts for this workbench. Of course I will assist on Xtext. If you had no chance to visit the Xtext workshop this might be your chance to get some hands on Xtext. This session was already run last year successfully. Last year my colleague Benjamin Schwertfeger took over the Xtext part, since we were at EclipseCon.

There are also some other talks around Xtext and Xtend. Both have been released in version 2.4 on March 20th, which brings some interesting new features. Most notably in regard to code generation are the Active Annotations. I guess this is also part of what Sven Efftinge will adress as future of code generation in his keynote “The Past, Present and Future of Code Generation” wednesday morning. More details he will present together with Sebastian Zarnekow in the tutorial “Internal DSLs with Xtend” (thursday 10:45-12:00). The last Xtext related talk will be from Moritz Eysholdt, called “Executable Specifications for Xtext Languages” (friday 10:45-12:15). I am actually not sure which of these talks I will attend personally. They are most relevant for my work, and I don’t work close enough with them to catch everything new in Xtext on my own. Thus, I’ll definetely would learn important aspects. On the other side, there are also other interesting talks in parallel.

The coming week will be an intensive experience with lots to learn and interesting persons to meet. Although I will really enjoy this time, I will be glad when I finally come back home. At the moment, my family is ill and I hope that I get not infected these days. I have been looking forward and worked for this event, so I am crossing fingers when I can board monday morning healthy.

I am sure the organizing team around Mark and Jacqui will do again a great job.

DSC00363

See you there and let’s make this event special!

Xtext Content Assist: Filtering keyword proposals

Sometimes when I introduce template proposals in Xtext editors I want them to be a replacement for a keyword proposal. By default you would get a proposal for the template AND for the keyword. In the following screenshot a template for a “class” concept exists, and a proposal for the “class” keyword, which would be inserted as part of the template at the cursor position.

The key to the solution is to overwrite the completeKeyword() method from your proposal provider class, e.g. like this:

public class SprayProposalProvider extends AbstractSprayProposalProvider {
    private static final Set<String>      FILTERED_KEYWORDS = Sets.newHashSet("text", "line", "class", "behavior", "style", "custom");
    @Override
    public void completeKeyword(Keyword keyword, ContentAssistContext contentAssistContext, ICompletionProposalAcceptor acceptor) {
        if (FILTERED_KEYWORDS.contains(keyword.getValue())) {
            // don't propose keyword
            return;
        }
        super.completeKeyword(keyword, contentAssistContext, acceptor);
    }
}

Christian Dietrich added in a comment how to filter a keyword that may occur more than once in your grammar. If you want to filter specific occurances you need to use the IGrammarAccess of your language:

@Inject MyDslGrammarAccess grammarAccess;

@Override
public void completeKeyword(Keyword keyword,ContentAssistContext contentAssistContext,ICompletionProposalAcceptor acceptor) {
  if (!grammarAccess.getGreetingAccess().getHelloKeyword_0().equals(keyword)) {
    super.completeKeyword(keyword, contentAssistContext, acceptor);
  }
}

xtext-utils unittesting 0.9.4 released

I have just released the new version 0.9.4 of the xtext-utils unittesting framework. The release became necessary due to a too restrictive version constraint for Google Guice, which made the framework incompatible with the upcoming Xtext 2.3.0 release.

This little framework is a useful addition to Xtext which makes writing DSL tests quite easy. In my daily work on developing Xtext projects and coaching Xtext in workshops it became a useful tool which complements Xtext’s test support. It is not the only way to write tests for Xtext DSLs, but a real simple one. The main idea is that the best way to develop DSLs is to write prototype models. This framework allows you to read, parse, validate, format, serialize, and compare your models in a single line.

The framework can be installed from its update site.

Unofficial release 2.2.2 of the Xtend Maven Plugin

In my previous post I showed how to use the maven-xtend-plugin to compile Xtend sources within a Maven build. This worked fine in simple cases, but there was a limitation of only one supported source directory (the one configured by build/sourceDirectory property), see Bug#367914. Unfortunately already each Xtext project has already 2 source folders (src and src-gen), so this made the plugin unusable for Xtext projects and forced checking in the generated Java files. This causes terrible problems with version control, especially when working on a team.

In the meantime Bug#367914 was resolved, but only for the upcoming Xtend 2.3 (Eclipse Juno) version. This urgent bugfix was not available on a Maven repository. But I needed it for project Spray to finally get rid of the merge conflicts and delete the generated sources from the repository (issue#94). Spray is still using Xtext/Xtend version 2.2.1 and depends on public available Maven artifacts. Since the bugfix won’t be available from the Xtend project for the 2.2.1 release and Eclipse Juno is still some way to go I decided to backport the current 2.3.0 M6 sources of the Xtend Maven Plugin to 2.2.x and create an unofficial version 2.2.2. This plugin I have deployed to the Fornax Repository to make it public available.

For an usage example refer to the POMs of project Spray: pom.xml, parent pom.xml.

JvmType import check

Several Xtext based DSLs will use import statements that import JvmTypes, be them real Java types, or just inferred types produced with your IJvmModelInferrer. The check for a valid import might be simple, but here it is to just make it even easier for you. I just had the need to realize this to resolve Spray issue#113. Basically you need to get the IJvmTypeProvider, to which you gain access through the IJvmTypeProvider.Factory.

public class SprayJavaValidator extends AbstractSprayJavaValidator implements IssueCodes {
    @Inject
    private IJvmTypeProvider.Factory typeProviderFactory;
    @Check
    public void checkImports(final Import imp) {
        // don't check wildcard imports
        if (imp.getImportedNamespace().endsWith(".*"))
            return;
        IJvmTypeProvider typeProvider = typeProviderFactory.findOrCreateTypeProvider(imp.eResource().getResourceSet());
        JvmType jvmType = typeProvider.findTypeByName(imp.getImportedNamespace());
        if (jvmType == null) {
            error("The import " + imp.getImportedNamespace() + " cannot be resolved", SprayPackage.Literals.IMPORT__IMPORTED_NAMESPACE, IMPORT_NOTEXISTS, new String[0]);
        }
    }
    ...
}

That’s all.

Xtext Content Assist: Escape identifiers conflicting with keywords

I have faced in Spray the situation that a segment of a package name of a qualified Java type conflicts with a keyword of the Spray language. Specifically you can refer to custom features with the custom keyword, and the referred type was in a package containing also the name custom.

When inserting the proposed qualified name this results in an error due to the keyword collision.

The qualified name is usually combined through a sequence of ID rules, which allows escaping an identifier by prefixing it with the ^ character in the case of conflict with a keyword.

For content assist it would now be useful if the inserted string would be automatically escaped. So here is a way to solve this.

My first approach was to override the completeJvmParameterizedTypeReference_Type() method from the proposal provider and modify the proposals through a custom ICompletionProposalAcceptor. Sebastian Zarnekow commented that the proper way would be to register a IValueConverter for QualifiedName. I tried to do so, but was not successful. Actually there is already an QualifiedNameValueConverter registered.

My next solution is now to subclass JdtTypesProposalProvider and pass a custom IValueConverter. The overridden methods just pass null as converter, thus do no conversion. I am reusing the QualifiedNameValueConverter here, since it does the necessary keyword escaping.

public class SprayJdtTypesProposalProvider extends JdtTypesProposalProvider {
    @Inject
    private QualifiedNameValueConverter qnValueConverter;

    /**
     * Overridden to pass a default value converter
     */
    @Override
    public void createSubTypeProposals(JvmType superType, ICompletionProposalFactory proposalFactory, ContentAssistContext context, EReference typeReference, Filter filter, ICompletionProposalAcceptor acceptor) {
        createSubTypeProposals(superType, proposalFactory, context, typeReference, filter, getConverter(), acceptor);
    }

    /**
     * Overridden to pass a default value converter
     */
    @Override
    public void createTypeProposals(ICompletionProposalFactory proposalFactory, ContentAssistContext context, EReference typeReference, Filter filter, ICompletionProposalAcceptor acceptor) {
        createTypeProposals(proposalFactory, context, typeReference, filter, getConverter(), acceptor);
    }

    private AbstractValueConverter<String> getConverter() {
        return new AbstractValueConverter<String>() {
            /**
             * Remove ^ character from escaped segments
             */
            @Override
            public String toValue(String string, INode node) throws ValueConverterException {
                return string.replace("^", "");
            }

            /**
             * Escape segments colliding with keywords with ^ character
             */
            @Override
            public String toString(String value) throws ValueConverterException {
                // this converter will escape keywords with ^
                return qnValueConverter.toString(value);
            }
        };
    }
}

This custom implementation must be bound to the UI module:

public class SprayUiModule extends AbstractSprayUiModule {
    @Override
    public Class<? extends ITypesProposalProvider> bindITypesProposalProvider() {
        return SprayJdtTypesProposalProvider.class;
    }
}

Now the inserted qualified type name will be automatically replaced.

How to limit proposed Java types to implementors of an interface

When you develop a DSL that derives from Xbase you likely want to refer to Jvm types through a JvmTypeReference.

CustomBehavior :
    'custom' (name = ID | 'ref' realizedBy=JvmTypeReference) (label = STRING);

If you hit CTRL+SPACE at such a position in the editor, by default you will get all visible Java types proposed:

This is not really useful when it only makes sense to refer to specifc types. If you want to limit the proposals to just implementors of a specific interface, you can customize this in the proposal provider by overriding the method completeJvmParameterizedTypeReference_Type(). The following example from Spray shows limiting the types to implementors of Graphiti’s ICustomFeature interface in the context of Spray’s CustomBehavior:

public class SprayProposalProvider extends AbstractSprayProposalProvider {
    @Inject
    private IJvmTypeProvider.Factory      jvmTypeProviderFactory;
    @Inject
    private ITypesProposalProvider        typeProposalProvider;
    
    @Override
    public void completeJvmParameterizedTypeReference_Type(EObject model, Assignment assignment, ContentAssistContext context, ICompletionProposalAcceptor acceptor) {
        if (EcoreUtil2.getContainerOfType(model, CustomBehavior.class) != null) {
            final IJvmTypeProvider jvmTypeProvider = jvmTypeProviderFactory.createTypeProvider(model.eResource().getResourceSet());
            // Graphiti specific
            final JvmType interfaceToImplement = jvmTypeProvider.findTypeByName(ICustomFeature.class.getName());
            typeProposalProvider.createSubTypeProposals(interfaceToImplement, this, context, SprayPackage.Literals.BEHAVIOR__REALIZED_BY, TypeMatchFilters.canInstantiate(), acceptor);
        } else {
            super.completeJvmParameterizedTypeReference_Type(model, assignment, context, acceptor);
        }
    }

Now you will get the desired subtypes as proposals:

Note that this does only limit proposals to certain types. It does not prevent you from actually referring to another type that you don’t get proposed. This needs to be restricted in your scope provider.