EclipsePlugin/CodeGeneration

From DPFWiki

Jump to: navigation, search

Contents

Code Generation with DPF Workbench and the Xpand plugin

When using a language workbench, we often want to transform our model into code. Creating code generators for the DPF Workbench is supported through the *codegen.xpand* plugins. This page will describe the features of the tool, how it works, as well as provide tutorials to demonstrate it in action.

This page is based on this master's thesis.

Features

  • Rich editing environment based on the concepts of the domain-specific modeling language (DSML)
  • Code completion
  • Refactoring
  • Syntax highlighting
  • Eclipse integration

The DPF Xpand metamodel

The biggest difference from a simple template engine, is the built-in interpreter that interprets a DSML and provides functionality based on what a Xpand metamodel dictates. In Xpand, the metamodel is the internal mapping from types in the DSML to types that Xpand understands. Xpand comes with metamodels for the most popular modelling facilities today; EMF, UML2, XSD and plain Java classes. Although these metamodels covers the most used languages, it lacks support for DSMLs specified in anything besides what Xpand has to offer.

The framework provides three different languages that has separate functionality; Xpand, Xtend and Check.

Xpand
Xpand is the template language that controls the output of the generator. Apart from the usual control flow statements, it supports lazy evaluation, let statements, aspect oriented programming and extensions created using Xtend and/or Java.
Xtend
Xtend is used as an extension language. It follows the functional paradigm and provides features like type inference, recursion, caching of methods and calling external Java extensions. It also provides ways to perform model-to-model transformations. This language helps to enforce the separation between program logic and template code; it is strongly encouraged to perform logic in an extension, and call the extension from the template.
Check
The Check language handles constraint checking on the model. We will not show the use of this language in this article, as constraints are validated within the DPF Editor (i.e. we assume the model we create the code generator for is valid).

Note that the Xpand framework is no longer under active development (although still supported). The replacement, Xtend, is a programming language which compiles to readable Java code and has the Xpand syntax embedded for template processing. However, as the language is compiled rather than interpreted, the ability to define custom metamodels which is reflected in the template language is not present. More info on this can be found here.

Installation

Update Site -> Code Generation for DPF [Optional]

Plugin Architecture

no.hib.dpf.codegen.xpand.feature
  • Eclipse feature.
no.hib.dpf.codegen.xpand.ui
  • The ui plugin contains the code concerning Eclipse integration. This includes wizards, natures, templates etc. It also supports setting properties on a project with the DPF Code Generation nature. This particular piece of code is not used, but will be useful if the project is expanded and maintained.
no.hib.dpf.codegen.xpand.metamodel
  • This plugin contains the DPF Xpand metamodel, as well as custom Xpand types which corresponds to the DPF types (e.g. NodeType -> dpf.core.Node). The DPF Xpand metamodel is documented using javadoc (DpfMetamodel.java). This file is a mapping from DPF types to Xpand types. Both the *.ui and *.metamodel plugin uses logging (log4j).
  • The typesystem package contains helper classes and supertypes which the DPF Xpand types inherits and uses.
  • typesystem.types contain the types.
  • The workflow package contains the workflow component definition for the DPF metamodel. I.e. the configuration component used in a MWE workflow file.

- Kva gjer dei forskjellige klassane/pakkane

Tutorial: Generating simple Java code

When creating a code generator, it is useful to have both a sample input and output at hand. This ensures that we adapt the generator to our model, and not the other way around.

This example will be very simple. We will only demonstrate the Xpand templates. Another simple (but a tad more feature rich) example can be found in this master's thesis.

Sample input

We create a simple model which describes a Class, a Reference, an Attribute and a Type.

File:M0.jpeg

Each Class can have 0 or more references to other classes. It can also have 0 or more Attributes that is typed by Type.

File:M1.jpeg

  • Create new project
  • Create new DPF Specification
    • File -> New -> Other -> Diagram Predicate Framework -> DPF Specifcation -> name it m0.dpf
  • Create model according to the first picture
  • Create second model from the second picture with the first specification as type
    • File -> New -> Other -> Diagram Predicate Framework -> DPF Specifcation -> name it m1.dpf -> Next -> Select Load File and browse to the location of the "m0" specification.

Sample output

public class Author {
  private String name;
  private List<Book> books;

  public Author() {}
}
public class Book {
  private List<Author> author;
  private String name, isbn;
  
  public Book() {}
}

For simplicity we assume all references are 0 to many, using a lists.

Creating a new code generator project

The first thing we need to do is to create the metamodel:

File -> New -> Other -> Diagram Predicate Framework -> DPF Generator Project

File:genwiz.jpeg

  • Create a new project with the name "no.test.generator", and press Browse to find the "m0" specification.
  • Press Finish

A project structure should now be generated for you with the following hierarchy:

no.test.generator
|-src
| |-template
| | |-templ.xpt
| |-workflow
| | |-workflow.mwe
|-src-gen
|-model

The template package contains a simple template skeleton. The workflow package contains a "standard" workflow setup. For an explanation of what a workflow file is, and how it works, see the Xpand documentation. Our DSML is copied from its location and into the model folder. The location of the model instance, which serves as the generators input data, needs to be specified in the workflow file. If the workflow file has not been changed, the instance location is specified in the "dpf_model" property.

Creating a Xpand template

We are now ready to specify a template.

«IMPORT dpf»

«DEFINE main FOR dpf::Specification»

«ENDDEFINE»

The code above is the generated stub in the generator project.

  • The IMPORT statement lets us import a certain namespace into the template editing environment. However, DPF has no support for namespaces and the statement thus imports the whole DPF specification.
  • The central concept of Xpand is the DEFINE block, also called a template. This is the smallest identifiable unit in a template file. The tag consists of a name, an optional comma-separated parameter list, as well as the name of the metamodel class for which the template is defined. This particular block ("main"), serves as the generators entry point. This is specified through the workflow file. In addition, the main block is defined for a dpf::Specification type; the DPF Xpand metamodel follows the hierarchy of a DPF Specification because the Xpand generator gets a Specification object passed to it containing the model.

Expanding our template

Our template definition is far from completed, as it does nothing. Because we have to follow the hierarchy of the DPF specification, the next step is to create a DEFINE block for a Graph type.

«DEFINE graph FOR dpf::Graph»

«ENDDEFINE»

This block is at the moment unused. We need to reach it from our main block. This is achieved using an EXPAND statement. We alter our main block so it looks like the following:

«DEFINE main FOR dpf::Specification»
	«EXPAND graph FOR this.graph»
«ENDDEFINE»
  • The EXPAND statement expands a block.

In this tutorial we will not do anything in the graph block besides expanding other blocks. However, the DPF Xpand metamodel provides access to most of the DPF Core API for convenience.

We change our graph block and add another DEFINE block:

«DEFINE graph FOR dpf::Graph»
	«EXPAND classes FOREACH this.getClassNodes()»
«ENDDEFINE»

«DEFINE classes FOR dpf::Class»

«ENDDEFINE»

The classes block demonstrates how the tool provides convenience methods based on the (meta)meta-models domain concepts.

We now have a block ("classes") which will run for each of the nodes returned from "this.getClassNodes()".

Let's add a FILE statement within classes:

«FILE this.name + ".java"»
 	
«ENDFILE»
  • The FILE statement outputs a file to the specified output directory in the workflow file. In our case, this is the src-gen source folder.

Finishing the template

Alter the classes block so it looks like the following:

«DEFINE classes FOR dpf::Class»
	«FILE this.name + ".java"»
		«IF this.getReferenceArrows().size != 0»
			import java.util.List;
		«ENDIF»
		public class «this.name.toFirstUpper()» {
			«FOREACH this.getAttributeArrows() AS e»
				private «e.target.name.toFirstUpper()» «e.name»;
			«ENDFOREACH»
			«FOREACH this.getReferenceArrows() AS e»
				private List<«e.target.name.toFirstUpper()»> «e.name»;
			«ENDFOREACH»
			
			public «this.name.toFirstUpper()»() {}
		}
	«ENDFILE»
«ENDDEFINE»

For each instance of a Class type, the classes block is executed. The first thing we check is if we should import the List interface or not. We should only import List if we have a Reference (as we assume each Reference is 0 to many). The next thing we do is iterate over all Attribute arrows, as well as all Reference arrows. We take advantage of the built-in string methods such as toFirstUpper, which uppercases the first letter.

As this example illustrates, the Xpand language provides loop constructs and conditional statements. To much logic in the templates is a bad idea as the template code gets complex and hard to maintain relatively fast. Most logic should reside in extensions which is called from the templates. This is demonstrated in the next tutorial or the previous mentioned thesis. More on the Xpand syntax can be found in the documentation.

Running the generator

  • Make sure the "dpf_model" and "dpf_metamodel" properties in the workflow.mwe file is set
  • Right click the workflow.mwe file -> Run As -> MWE Workflow
    • If the generator has problems with using absolute/relative paths, use platform:/resource URIs
  • Two files Book.java and Author.java is now generated

Note that the textual output is formatted using the Eclipse Java code formatter. This is defined in the postprocessor tag in the workflow file.

Tutorial: Generating medical workflow configuration code

This example will show the process of creating a code generator for a medical workflow engine. This example requires the reader to be familiar with DPF and the basic concepts of the code generation tool and the Xpand language (the previous tutorial covers this). This generator was written without any knowledge of the dve language besides studying the input/output, it will probably not implement everything.

Input and desired output

Input

Our input model looks like the following:

Metamodel

Metamodel

Instance model

Instance model

The meta-metamodel defines a Task and a Flow. The Flow is constrained by the [irref] predicate, meaning that a Task's outgoing Flow cannot point to itself. [irref] is part of the default signature in the DPF Workbench.

The instance model shows a set of tasks, the flow between them, as well as custom predicates.

Output

The instance model should result in the following code:

int f1 = 0;                                                                                                                
int g1 = 0;                                                                                                                
int f2 = 0;                                                                                                                
int g2 = 0;                                                                                                                
int f3 = 0;                                                                                                                
int f4 = 0;                                                                                                                
int g4 = 0;                                                                                                                
int f5 = 0;                                                                                                                
int g5 = 0;                                                                                                                
int f6 = 0;                                                                                                                 

int c = -1;                                                                                                                

process InitialEvaluation{                                                                                                 
  state disabled, enabled, running, finished;                                                                                
  init disabled;                                                                                                             
  trans                                                                                                                      
  disabled -> enabled { },                                                                                                   
  enabled -> running {},                                                                                                     
  running -> finished {effect f1 = 1, g1 = 1;};                                                                              
}                                                                                                                          

process Mri{                                                                                                               
  state disabled, enabled, running, finished;                                                                                
  init disabled;                                                                                                             
  trans                                                                                                                      
  disabled -> enabled { guard f1 > 0  ; },                                                                                   
  enabled -> running { },                                                                                                    
  running -> finished {effect f2 = 1;};                                                                                      
}                                                                                                                          

process BloodTest{                                                                                                         
  state disabled, enabled, running, finished;                                                                                
  init disabled;                                                                                                             
  trans                                                                                                                      
  disabled -> enabled { guard g1 > 0  ; },                                                                                   
  enabled -> running { },                                                                                                    
  running -> finished {effect g2 = 1; };                                                                                     
}                                                                                                                          

process Evaluation{                                                                                                        
  state disabled, enabled, running, finished;                                                                                
  init disabled;                                                                                                             
  trans                                                                                                                      
  disabled -> enabled { guard f2 > 0  && g2 > 0; },                                                                          
  enabled -> running { },                                                                                                    
  running -> finished {effect f3 = 1;};                                                                                      
}                                                                                                                          

process CheckForProcedure{                                                                                                 
  state disabled, enabled, running, finished;                                                                                
  init disabled;                                                                                                             
  trans                                                                                                                      
  disabled -> enabled { guard f3 > 0;},                                                                                      
  enabled -> running {},                                                                                                     
  running -> finished {effect c = 1, f4 = 1;},                                                                               
  running -> finished {effect c = 0, g4 = 1;};                                                                               
}                                                                                                                          

process Procedure_a{                                                                                                       
  state disabled, enabled, running, finished;                                                                                
  init disabled;                                                                                                             
  trans                                                                                                                      
  disabled -> enabled { guard f4 > 0  && c == 1; },                                                                          
  enabled -> running { },                                                                                                    
  running -> finished {effect f5 = 1;};                                                                                      
}                                                                                                                          

process Procedure_b{
  state disabled, enabled, running, finished;
  init disabled;
  trans
  disabled -> enabled { guard g4 > 0  && c == 0; },
  enabled -> running { },
  running -> finished {effect g5 = 1; };
}

process Join{
  state disabled, enabled, running, finished;
  init disabled;
  trans
  disabled -> enabled { guard f5 > 0 && g5 == 0 ;  },
  disabled -> enabled { guard g5 > 0 && f5 == 0; },
  enabled -> running { },
  running -> finished {effect f6 = 1; };
}

process FinalEvaluation{
  state disabled, enabled, running, finished;
  init disabled;
  trans
  disabled -> enabled { guard f6 > 0; },
  enabled -> running { },
  running -> finished { };
}

system async;

A few observations can be done on the desired output and the model:

  • The states are always the same
  • The init state is always set to disabled upon execution
  • The trans keyword is always present
  • system async is always present
  • outgoing Flow arrows affects the running -> finished transition by setting the guard expressions
  • incoming Flow arrows affects the disabled -> enabled transitions by checking the value of the guard expressions
  • ++

Signature

The semantics behind the signature:

Signature

The signature in this tutorial project does not contain the or_merge and or_split predicates.

Getting started

  • Download model/signature from subversion (playground/no.hib.dve.generator) (FIXME: not possible to upload .zip/.tar files directly)
  • Create a new generator project using the defMeta.xmi as metamodel
  • Set both the dpf_model and dpf_metamodel paths. If absolute/relative paths aren't working when executing the generator, use platform:/resource URIs.
  • As we will not generate Java code in this project, remove or comment out the postprocessor tag in the workflow file. It is important to note that the generated code now depends on the formatting in the template. It is possible to create custom postprocessors (check Xpand documentation).

Creating the template and extension

  • Copy/paste the following template code into templ.xpt:
«IMPORT dpf»
«EXTENSION org::eclipse::xtend::util::stdlib::io»
«EXTENSION template::extensions»

«DEFINE main FOR dpf::Specification»
	«EXPAND graph FOR this.graph»
«ENDDEFINE»

«DEFINE graph FOR dpf::Graph»
	«FILE "Test.dve"»
		«EXPAND printFlowStatements FOREACH getFlowStatements(this)»
	«insertConstantDecl(this)»
		«EXPAND task FOREACH this.getTaskNodes()-»
	system async;
	«ENDFILE»
«ENDDEFINE»

«DEFINE printFlowStatements FOR dpf::Arrow»
	int «this.name» = 0; 
«ENDDEFINE»

«DEFINE task FOR dpf::Task»
	process «this.name» {
		state disabled, enabled, running, finished;
		init disabled;
		trans
		«printDisabledEnabled(this)-»
		enabled -> running { },
		«printRunningFinished(this)»		
	}
«ENDDEFINE»


Extensions

The template itself is not very long, but we see some new stuff. First of all we see EXTENSION statements. These are used to import extensions into our template editing environment. Besides the possibility of creating your own templates in the Xtend language or Java, the Xpand framework comes with its own "standard library". It offers functionality for handling IO, properties, counters, UIDs and more. See the Xpand documentation for more information. The EXTENSION statement requires the extension file to be prefixed with its namespace.

There are a few methods used in the template which is defined in a separate extension file. Although that logic probably could have been implemented directly in the template, it is more clean to keep them in its own file.

The «EXTENSION org::eclipse::xtend::util::stdlib::io» statement imports IO functionality which is useful for debugging templates. You can use syserr(obj) to print to the console when running the generator.

  • Create a new file in the template package called extensions.ext


The "graph" block

The formatting of the block looks a bit off, but this is to "format" the output without a postprocessor. As one might observe, we call extensions in two places; getFlowStatements(Graph) and insertConstantDecl().

The getFlowStatements(Graph) extension takes a Graph object as argument and outputs a list of arrows. It looks like this:

import dpf;
extension org::eclipse::xtend::util::stdlib::io;
 
getFlowStatements(Graph g):
	g.arrows.select(e|e.typeArrow.name == "Flow");
  • Copy and paste the example into your extensions.ext file

The first thing we see is that we still have to import our model's namespace. We can also import extensions into our extension. The rest is quite different from the Xpand template language; there are no guillemets («») and the Xtend language also follows the functional paradigm rather than the imperative. Xtend gives us the ability to query collections in a concise manner. The listed example returns a list of arrows which meets the criteria; every arrow which has a type arrow with the name "Flow", i.e. we return every arrow that is typed by Flow. Notice that we don't provide a return value, as this is inferred by Xpand.

insertConstantDecl(Graph):

insertConstantDecl(Graph g) :
	let res = g.nodes.collect(e|e.getConstraints().select(f|f.predicate.symbol == "xor")) :
		syserr(res) ->
		if res.size != 0 then
			"int c = -1;";

This method returns a string if we have a "xor" predicate constraining any of our nodes. This is because the xor predicate requires an extra variable to identify which path to follow (see the signature). This example demonstrates the support for "let" expressions, as well as chaining of methodcalls.

The task block

We iterate over all the model's tasks inserting the proper values as we go. This part requires a bit more logic which isn't as easy to achieve using Xtend. The language is great for querying collections, but it does not provide language features like higher-order functions, which means we dont have built-ins like map. I.e. its harder to manipulate the collections. Fortunately, Xtend supports Java extensions.

The task block has a lot of hardcoded lines due to our realizations in the beginning. Let's take a look at the printEnabledDisabled(Task) method:

printDisabledEnabled(Task t):
	if t.incomings.size == 0 then 
		"disabled -> enabled { },\n"
	else
		mergeStatement(t.incomings);

private String mergeStatement(List[Flow] f) :
	JAVA template.StringUtil.mergeStatement(java.util.List);
  • Copy/paste the example into the extensions.ext file

The method is written in Xtend, but relies on external Java code. As we recall from our realizations, all incoming arrows results in a guard check. If we do not have any incoming arrows, we return an empty statement. If we have incoming arrows, we let some external Java code handle the logic. A Java return type cannot be inferred which means we have to explicitly declare what we return. We also need a fully qualified name for the Java class and the method called. The Java classes are usually static as the classes are stateless. You could use a regular class as well, but Xpand will always create a new object instance. Implementing stateful helper classes can be achieved through the GLOBALVAR extension provided with Xpand.

The Java extension

  • Create StringUtil.java in your template package
  • Insert the following code:
package template;

import java.util.List;

import no.hib.dpf.core.Arrow;
import no.hib.dpf.core.Constraint;
import no.hib.dpf.core.Predicate;

public class StringUtil {
	private enum Constraints {		
		XOR_SPLIT("xor"),
		XOR_MERGE("xor'"),
		AND_SPLIT("and"),
		AND_MERGE("and'"),
		OR_SPLIT("or"),
		OR_MERGE("or'");
		
		private String symbol;
		
		private Constraints(String symbol) {
			this.symbol = symbol;
		}

		public String mergeStatement(List<Arrow> l) {
			if(l.size() == 2) {
				switch(this) {
					case XOR_MERGE:
						return xorMerge(l);
					case AND_MERGE:
						return andMerge(l);
					default:
						return "";
				}
			} else if (l.size() == 1 && this == XOR_SPLIT) {
				//Case where we need to handle the c and !c predicate on a single incoming arrow on a task node
				boolean c = false;
				
				for(Constraint cc : l.get(0).getConstraints()) {
					if(cc.getPredicate().getSymbol().equals("c")) {
						c = true;
					}
				}
				
				return  "disabled -> enabled { guard " + l.get(0).getName() + " > 0 && " + (c ? "c == 1" : "c == 0") + "; },\n";
			} else {
				return  "disabled -> enabled { guard " + l.get(0).getName() + " > 0; },\n";
			}
			
		}
		
		public String splitStatement(List<Arrow> l) {
			if(l.size() == 2) {
				switch(this) {
				case XOR_SPLIT:
					return xorSplit(l);
				case AND_SPLIT:
					return andSplit(l);
				default:
					return "";
				}
			} else {
				return "running -> finished { effect " + l.get(0).getName() + " = 1; };";
			}
		}
		
		public static Constraints getConstraint(String s) {
			for(Constraints c : Constraints.values()) {
				if(s.equals(c.symbol)) {
					return c;
				}
			}
			return null;
		}
		
	}

	public static String mergeStatement(List<Arrow> l) {
		//Assuming the incoming arrows are constrained by the same predicate
		try {
			Predicate p = l.get(0).getConstraints().get(0).getPredicate();
			Constraints c = Constraints.getConstraint(p.getSymbol());
			return c.mergeStatement(l);
		} catch(IndexOutOfBoundsException e) {
			//Case where an arrow is not constrained. Should still result in a guard expression
			return "disabled -> enabled { guard " + l.get(0).getName() + " > 0; },\n";
		}
	}

       public static String splitStatement(List<Arrow> l) {
		//All outgoing arrows should result in an effect statement
		try {
			Predicate p = l.get(0).getConstraints().get(0).getPredicate();
			Constraints c = Constraints.getConstraint(p.getSymbol());
			return c.splitStatement(l);
		} catch(IndexOutOfBoundsException e) {
			//Case where an arrow is not constrained. Should still result in a effect expression
			return "running -> finished { effect " + l.get(0).getName() + " = 1; };";
		}
	}
	
	private static String andSplit(List<Arrow> l) {
		//AND split predicates shape consists of two arrows 
		return "running -> finished { effect " + l.get(0).getName() + " = 1, " + l.get(1).getName() + " = 1; };";
	}
	
       private static String xorSplit(List<Arrow> l) {
		//XOR split predicates shape consists of two arrows 
		StringBuffer ret = new StringBuffer();
		ret.append("running -> finished { effect c = 1, " + l.get(0).getName() + " = 1, " + l.get(1).getName() + " = 0; },\n\t\t");
		ret.append("running -> finished { effect c = 0, " + l.get(1).getName() + " = 1, " + l.get(0).getName() + " = 0; };");
		return ret.toString();
	}
 	
	private static String xorMerge(List<Arrow> l) {
		//XOR merge predicates shape consists of two arrows 
		StringBuffer ret = new StringBuffer();
		ret.append("disabled -> enabled { guard " + l.get(0).getName() + " > 0 && " + l.get(1).getName() + " == 0; },\n\t\t");
		ret.append("disabled -> enabled { guard " + l.get(1).getName() + " > 0 && " + l.get(0).getName() + " == 0; },\n");
		return ret.toString();
	}
	
	private static String andMerge(List<Arrow> l) {
		//AND merge predicates shape consists of two arrows 
		return "disabled -> enabled { guard " + l.get(0).getName() + " > 0 && " + l.get(1).getName() + " > 0; },\n";
	}
}

The code above is the extensions needed for this tutorial. There is some logic involved which is better to separate from the template itself to avoid clutter. The code uses an enum class which defines each of the predicates in our signature. When calling e.g. mergeStatement(List<Arrow>), we assume that we retrieve arrows which are constrained.

  • If we have more than one incoming/outgoing arrow, we know that the arrows have some kind of constraint.
  • If we only have one arrow it might not be constrained.
  • In merge/splitStatement we retrieve the predicate for the first arrow in the list and utilize our enum class to extract it and take the proper action.
  • If the arrow is not constrained (meaning there is only one incoming/outgoing arrow), we get an IndexOutOfBoundsException that returns the proper guard or effect expression.
  • The XOR_SPLIT predicate is handled in a special way. According to the signature we need a separate variable c which helps identifying which route to take.

The rest of the 'task' block

  • Insert the following into your extension.ext file:
printRunningFinished(Task t):
	if t.getOutgoingArrows().size == 0 then
		"running -> finished { },\n" 
	else
		splitStatement(t.getOutgoingArrows());
private String splitStatement(List[Flow] f):
	JAVA template.StringUtil.splitStatement(java.util.List);

Run the generator

We have reached the end of the tutorial; you should now be able to run the generator.

  • Right click workflow.mwe -> Run as -> MWE Workflow

This should result in a file called Test.dve in your src-gen directory. The output should more or less look like what we defined in Output.

Personal tools