SDM Lab 7: Domain-specific languages in xtext (Updated 12/11/19)
This page gives brief notes and instructions on the guided lab session. It
will have omissions and bugs - you need to listen as well as read!
Introduction: setting up xtext
- Go to http://www.eclipse.org/Xtext/download.html
- Copy the link for releases (top blue button).
- Go to eclipse -> help -> install new software.
- Paste the link into the Work with window, and wait while eclipse
fetches the packages (this may take a while...).
- Select Xtext -> Complete Xtext SDK. Click next button until you finish.
- Let it download. Agree to restart Eclipse when prompted, and you're
good to go.
2018 note If you get told your request cannot be completed and end
up at an installation remediation page, pick the first option "Keep my
installation the same...".
Hello Xtext!
Here we will try and build a very basic language in xtext but still we will
be able to realize the power it has.
- First Select new Xtext Project (file->new->project->Xtext->Xtext project).
For the time being do not change the defaults and click on finish.
- You will see that multiple eclipse projects have been created. For the
time being, we are interested in org.xtext.example.mydsl. Expand the
project and find MyDSL.xtext. This is where our grammar is defined. Don't
worry if this does not make too much sense at this stage. The grammar
basically says that greetings start with a keyword 'Hello' and end with a
'!', and that programs in the DSL are lists of greetings.
- Right click in the pane where MyDSL.xtext is displayed, select Run as,
then click on Generate Xtext Artifacts. It might take a couple of minutes
for the generation to run.
- Now we are ready to test our generated language.
- Click on Run-> Run configurations .
- Search for Eclipse, and double-click on Runtime Eclipse Application.
- Name the
configuration myConfig and click on run. This should open a new Eclipse
Window. (2018 note: you can ignore an xmlrpc problem, just click Continue;
and you may see an error about "Builtin LFS support not present/detected"
in the console of your original Eclipse - you can ignore this too.)
- Go through to the Workbench in the new Eclipse.
- Create new General project and in it create a new file. Give it the
extension .mydsl (the file extension that programs in your DSL should have
was one of the options you didn't change when you created the DSL project).
- Say yes when asked whether you want to convert the project to an xtext
project.
- Type Hello world! into the file.
- Try adding a couple more lines of code that you expect to agree with
the grammar.
- Now change a line in different ways so it does not agree with the
grammar. (At this stage, check carefully that you understand the grammar,
i.e., what is and isn't allowed, and how the xtext grammar expresses that.)
- What happens if you have several mistakes in your file?
- What are some desirable things you see in the interface?
- Try adding comments as you would in java (both block and single line)
Does this work?
- Experiment with the content assist (Ctrl-Space).
- What about formatting (Ctrl-Shift-F)?
Everything is configurable, but we won't go far into how in this lab. Have
a browse around the files, though. For example, have a look at the
test that has been generated for this DSL, in ...mydsl.tests, folder src
(not src-gen). Run it by
selecting Run as...JUnit test on the test file. Add a similar test, and run
both tests. The tests use the xtend language. In folder xtend-gen you will
find the plain Java that is generated from your xtend tests: have a look.
Exit the Eclipse containing your DSL
when you're ready to go on; feel free to play, regenerate, try again...
Creating our own language
- Again, create a new Xtext Project. This time set the Project Name to
uk.ac.ed.inf.sdm.paleo, the Language name to uk.ac.ed.inf.sdm.paleo.Diet
and the extension to diet.
- Again, the default grammar has been created. This time, we are going to
modify it. Leave the grammar and generate lines alone,
though.
- The subsequent code defines the grammar. If you're up to date with the
reading you have the basics, but still look at the documentation at https://eclipse.org/Xtext/documentation/
- I suggest at least opening the 15 Minutes Tutorial, and you may like to
follow it in a separate project. All the features of xtext that you will
need in this lab are in that example (including the or rule, which wasn't
in the article you read).
- Design a DSL to let users specify a network of questions to determine
whether a food is OK according to a given diet. Start very simplistic, and
see how far you get! For example, start off by letting your grammar specify
that a file vegan.diet would contain a list of questions or
answers starting Q: or A: respectively, each followed by
a string. Then think about how you can specify the structure that explains
where to go after getting an answer to a question... You'll need to give
each question and result an attribute name, and then you can use that in a
cross-reference, which uses a rule-name in square brackets like
[Question]...
Each time you change something in the grammar, check your change does what
you expect by running the Eclipse configuration. That gets tedious fast: we
aren't going into it here, but you'll notice that xtext has helpfully
created a project uk.ac.ed.inf.sdm.paleo.tests... this is so that
when you do this kind of thing for real you can maintain JUnit tests which
give you a better way to check that your grammar does what you think. (Have
a look inside. There's some skeleton code written in xtend, which was
mentioned in the lecture. Try running it (as JUnit test - if you are told
there is none, try running the generated Java version of the test instead,
but only ever edit the xtend version). It fails - why? Can you fix it, as
simply as possible?)
- Browse the code that has been generated for your grammar in the
src-gen folder. It would take longer than is reasonable now to
fully understand everything that goes on when you "generate xtext
artefacts", but you can see that what is here goes way beyond letting you
generate a cute syntax checker for your language: it also provides you with
a lot of the boring parts of what you'd need to write an application that
took in descriptions of diets written according to your grammar, and did
things with them (as in our original imagined scenario of the start-up
company quickly generating applications for each new fad diet).
- An ecore model, representing the abstract syntax (or "semantic model")
of your DSL has been automatically generated from your xtext grammar -
you'll find it in uk.ac.ed.inf.sdm/paleo/model/generated. You can
open it with the OCLinEcore editor that we used last week, or you might
like to try selecting Initialise ecore_diagram diagram file for a
graphical representation - remember, different concrete syntaxes for the
same abstract syntax!
- Finally let's do a tiny bit of "code generation". Have a look at the
DietGenerator class in uk.ac.ed.inf.sdm.paleo.generator in the
src folder. (Note that the fact this is in src, not
src-gen, means you are expected to edit it: what's there at the
start is just an indication of the kind of thing that can go here. By
contrast, files in src-gen are not to be edited; they are used
as-is, and regenerated by xtext as you change things.)
- By minimally adapting the example that's commented out in that file,
specify that the generated file should contain a list of the questions (but
not the results) that are in the user's .diet file. (2019 note:
you will need to add an appropriate import, e.g. import
uk.ac.ed.inf.sdm.paleo.diet.Question , as well as uncommenting and
modifying some code.)
- Try it out, by restarting the Eclipse configuration. You'll need to
create a folder src-gen in your diet project. Next time you modify the diet
definition, you should see an automatically generated .txt file created in
that folder.
There is an arbitrary amount of exploration you could do, but hopefully
this has given you a small flavour of what it's like to define a language
in xtext. The bits you exercised here are (mostly) a subset of what's covered in the
15 minute (extended) tutorial, so that's the next place to go if you want
to learn more.
This page is maintained by
Perdita Stevens (
perdita@inf.ed.ac.uk)
