This is about the idea to apply agile methods to language engineering. We discuss the agile principle, why it is interesting for language engineering, what is needed to actually realise an agile language engineering process, and what can already be done with the technology at hand.
The agile method is popular in general software engineering. It promises better products, more efficient development, and above all more fun for all participants. Agility basically means to use a lightweight, very adaptive, and dynamic development process that: (1) concentrates on developers and customers, their needs above fix processes; (2) short development cycles that concentrate on the most important changes and maintain a software product that is testable and executable at all times; (3) use a single software description, which is usually the program code, as the only artifact; (4) include the customer into the process, have him use the software extensively during development, and embrace whatever feedback he/she can provide. More about the agile principle and agile methods can be learned from the agile manifesto.
How can we apply this general philosophy for (software) engineering to the engineering of languages; and why would we like to do so? The why first: domain specific languages. These are languages that provide functionality to domain experts in an outfit that is suitable to them: this basically means to provide concepts and notations as they are already used within the domain. The problem with these languages is that they are only used by a small group of people, these language have a short life-cycle, are hard to understand by the computer scientist that design them, and are subject to regular changes. All these problems are taken care of by agile methods: rigorous user/developer exchange, embrace change, short development cycles with fast results.
When you accept that language engineering could benefit from agile methods, you will ask what do I need to conduct agile language engineering? Lets look at a development cycle, typical for an agile process. The whole process usually starts with a user story: he/she needs a whole new language. You sit down with the user and scribble a few example models, which we call reference models. Now, you’re going to implement the language, meaning that you create language tools (editors, simulators, code-generators, etc.; depending on the language). You do implementing until you reach the point where all the reference models can be successfully handled by your tools. You can always test your tools using the reference models. Now you have a working version of the language and can give it to the user. He/she will try the language, while you start refactoring what you have build. After a while the user comes back to you with a new story, requesting a new feature or to change an existing language feature. The whole cycle starts again.
All this sounds like a normal agile process, what is so language specific about it? We identified three technologies that are obviously necessary to realise this process. You need these reference models as a specific form of testing. These reference models are the base for your automated language tests. We would like to know whether the reference models cover the whole language. We summarize these research aspects under the name language testing. The next thing is that we need ways to develop language tools efficient. We propose the term language modelling which uses (meta-)models at a high-level of abstraction to describe certain language aspects rather than implementing language tools manually. The meta bubble is full of examples: OCL can be used for static language semantics, our own MOF Action Semantics can be used to define operational semantics, GMF or TEF can be used to define graphical or textual editors, and so on and so on. These technologies allow you to create language tools fast. In agile language engineering the language often changes. User come up with new ideas or we have to refactor our language. We need a third technology to change meta-models in such away that we can co-evolve all the models and other descriptions that are based on the meta-model. This research aspects is known as meta-model adaptation and model co-adaptation. With all these technologies available and aligned to each other, one could actually exercise agile language development.
We are currently trying to align all our tools to realise an example language project based on the agile method. We have meta-modelling, OCL, and MOF Action Semantics to effecient develop a language with operational semantics. A college of mine, is currently realising a meta-model refactoring and co-adaptation tools suit based on the eclipse refactoring framework. Another college is looking into the reference model test issue. Theoretically, we have everything needed for a first experiment on a toy language. I hope to present some results of this experiment soon.