This document presents the general roadmap for the Xenomai project.
Xenomai was born in 2001, out of a basic idea: support traditional RTOS APIs over a Linux-based real-time framework, so that existing industrial applications coming from the proprietary world could migrate easily to a GNU/Linux-based environment, while keeping stringent real-time guarantees. To this end, the core Xenomai technology exhibits an abstract real-time nucleus, which provides generic building blocks for implementing real-time APIs, aka "skins". This way, a skin can emulate a proprietary API efficiently, based on a reusable real-time core.
Today, Xenomai currently runs over eight architectures (namely powerpc32/64, blackfin, arm, x86_32/64, nios2 and SuperH, a legacy ia64 port was discontinued from Xenomai 2.5.x), a variety of embedded and server platforms, and can be coupled to two major Linux kernel versions (2.4 and 2.6), for MMU-enabled and MMU-less systems. Supported real-time APIs include POSIX 1003.1b, VxWorks, pSOS+, VRTX and uITRON, and also a Xenomai-specific API.
Integrating Hard Real-Time into Linux
Albeit the real-time Linux ecosystem is usually keen on dividing itself between the "co-kernel" and "native real-time preemption" approaches, fact is that such distinction is not relevant when it comes to Xenomai.
Firstly, Xenomai has been designed to separate into distinct layers the generic RTOS features it provides (e.g. basic thread abstraction and real-time scheduling properties), from the arch-dependent real-time enabling capabilities it needs from the hosting hardware to implement them (e.g. context switch support and deterministic interrupt latency). For instance, this separation allows to run the Xenomai core and interfaces over a simulation engine, which in turn provides a fully virtualized real-time environment. In other words, Xenomai is able to use the most appropriate real-time enabling technology available with the underlying target to support its API personalities, or skins.
Secondly, the project keeps investing time and energy to get the highest possible level of integration between the real-time sub-system it provides, and the native Linux environment it is based on. Keeping the usual programming model for real-time applications has always been considered as much as important as guaranteeing the lowest latency figures achievable on any given platform. For instance, even when underlaid by a dual kernel technology, application developers relying on Xenomai can still use native debugging tools such as GDB.
Started during the summer of 2001, this effort has built the foundations of the project. Xenomai 1.1.1, the last release of this branch, already featured most of the generic RTOS abstractions we are still using today. It was able to run various traditional RTOS skins built over the generic core, which were usable by applications embodied into kernel modules. By mid-2002, the system could run over different real-time enabling layers, such as Adeos, or the event-driven simulation engine still available today.
Xenomai 2 has focused on getting a high level of integration with the GNU/Linux environment, so that the regular user-space programming model is preserved as much as possible when writing real-time applications. In parallel, a major refactoring has taken place, in order to ease the porting effort to other architectures, as a result of which, Xenomai 2 is currently running over six architectures with Linux 2.4 and 2.6 kernels, whilst Xenomai 1 would only run over 2.4/x86.
Additionally, Xenomai 2 has extended the set of real-time interfaces available to user-space applications:
- The POSIX 1003.1b skin guarantees ultra-low latencies to user-space applications, and conforms to the ubiquitous standard for real-time programming.
- The RTDM (Real-Time Driver Model) interface provides a skin-independent framework for writing real-time device drivers, and accessing them through a POSIX interface. RTDM helps to follow the well-known design principle of clean separation between hardware interface and application program, also when strict deterministic behaviour is required.
- The native Xenomai skin provides a rich API, which leverages the capabilities of the underlying real-time core, with semantics close to the traditional RTOS APIs.
We also made sure to enable Xenomai for use with key debugging tools:
- KGDB and GDB, respectively for kernel-based and user-space debugging.
- The I-Pipe tracer, used to uncover latency spots or to trace kernel execution at microlevel.
- The ability to run the entire Xenomai core and APIs over an event-driven simulation system, for host-based development, testing and debugging.
- The preliminary support for coupling the Xenomai core to LTTng.
Toward Xenomai 3
While we were working hard on Xenomai 2, two major trends have been emerging. Firstly, the effort for adding native real-time capabilities to the Linux kernel has reached a critical mass with Ingo Molnar's PREEMPT_RT patch series. Secondly, a growing number of industrial applications are being ported to GNU/Linux, which translates into large amounts of legacy source code, projects need to accommodate with.
Preserving a mature option
The dual kernel technology Xenomai 2 uses is based on Adeos/I-pipe, a lightweight and mature real-time enabling layer for the Linux kernel, which is available for various architectures at low engineering and maintenance cost. Porting the I-pipe to a new architecture is fairly simple, and the overhead induced by this simple and straightforward interrupt virtualization layer is small.
A dual kernel based architecture has some distinct advantages we want to keep providing when this is desirable to Xenomai users:
- It decouples Xenomai from the development cycle of standard Linux, thus allowing to pick the desired (or required) kernel version more freely.
- It can simplify the worst-case analysis and keeps the results valid over update cycles of the vanilla kernel.
- It allows to fine-tune the non-real-time part for throughput without negative impact on the real-time job.
- It does not require any complex tuning of the Linux system to guarantee short and bounded latencies to the real-time job.
Evolving with Xenomai 3
The undergoing PREEMPT_RT effort already delivered on the short and bounded latency promise with a single kernel implementation for Linux, for a variety of applications, on selected hardware platforms for which this technology is available and mature. In this respect, PREEMPT_RT creates opportunities to extend the relevance of Xenomai as a migration tool, so that moving an application to a PREEMPT_RT-based system does not necessarily involve porting its code over the POSIX API.
Therefore, Xenomai 3 will focus on supporting both dual kernel and native preemption as real-time enabling technologies. This evolution is going to be three-fold:
- Allow the real-time interfaces (aka skins) to either rely on a dual kernel layer (i.e. Adeos-based) or PREEMPT_RT-enabled kernel. When running over a PREEMPT_RT kernel, Xenomai will not require any interrupt virtualization add-on to the Linux kernel such as Adeos anymore.
- Enhance both the dual kernel and PREEMPT_RT-based RTDM incarnations, and enrich the RTDM-based driver collection hosted by Xenomai.
- Extend the set of Xenomai skins mimicking traditional RTOS APIs, to open more options for migrating applications from legacy RTOS to the GNU/Linux environment.
What Will Change With Xenomai 3
Compared to Xenomai 2, Xenomai 3 is a paradigm shift, because users will not be tied to the dual kernel approach anymore. Users will still be able to run the Xenomai real-time APIs over a dual kernel layer running side-by-side with Linux, but also directly on top of a vanilla Linux kernel, preferably PREEMPT_RT enabled. Their choice will depend on multiple criteria, such as:
- the hardware characteristics of the target platform.
- the availability of a dual kernel and/or PREEMPT_RT support on such platform.
- the level of performances required from the real-time system, in terms of worst-case latency and overhead.
- the application requirements in terms of API, and how clearly real-time and non real-time activities could be segregated within the code.
Years ago, applications based on Linux-based dual kernel systems used to behave as totally foreign code, strictly isolated from the common Linux environment. Sometimes those applications could only live in kernel space because of the lack of user-space support from the real-time Linux variant involved. Xenomai 2 proved that a more common programming model could be followed to implement real-time applications in regular Linux processes, not simply running as "kernel modules in user-space", but truly as real-time capable Linux applications. To this end, significant efforts were made to blur the boundaries between the Xenomai and Linux programming environments, so that common tools such as GDB can be safely used to debug real-time applications for instance.
Xenomai 3 will generalize such orientation, so that applications should be able to run either over a dual kernel layer, or on top of an PREEMPT_RT-enabled kernel, indifferently. This implies that real-time application code shall reside in user-space, regardless of whether the dual kernel or native preemption is used. This makes the common device driver / application code split design most desirable, with RTDM exporting the same interface to both environments. As an added benefit, this common programming model is known to lead to better designed real-time applications, drastically reducing the number of different APIs and mechanisms both used in kernel and user spaces. To this end:
- the Xenomai skins will not export their interface to kernel modules anymore, at the notable exception of the RTDM device driver API, which by essence must be used from kernel space for writing real-time device drivers.
- deprecated uses of the Xenomai APIs from kernel space will cause warning messages to be issued at build time, starting from v2.5-rc2. However, those kernel interfaces will remain fully functional throughout the Xenomai v2.x series, until they are eventually removed in Xenomai 3.
- Xenomai 2 APIs which only provide kernel-based support will not be part of Xenomai 3. As of today, this only concerns the RTAI emulator, which will therefore be discontinued in Xenomai 3. People who want to port from RTAI to Xenomai 3 should consider moving their application over the native Xenomai API in user-space, which has semantics compatible with those of RTAI.
To sum up, the canonical way to develop real-time systems based on Xenomai 3 will involve RTDM-based device drivers, and user-space applications talking to these drivers via the common RTDM interface.
- the native RTDM incarnation will be merged into Xenomai 3, side-by-side with the dual kernel based implementation inherited from Xenomai 2. The project will maintain and evolve this key component as required.
- the dual kernel layer of Xenomai 3 will support recent Linux kernel releases when Adeos/I-pipe patches are already available for the target architecture (e.g. ARM, powerpc and x86_32). As a direct consequence of this, dual kernel support for the Linux 2.4 series will be discontinued in Xenomai 3. PREEMPT_RT-enabled kernels will be better supported over v2.6.24 or later.
Xenomai 3 FAQ
Q: I can run POSIX based applications directly over a PREEMPT_RT kernel on my target system, so what is the point of running Xenomai 3 there?
A: If your application is already fully POSIXish, and the performances requirements are met, then there is likely no point. However, you may want to consider Xenomai 3 in two other situations:
- you want to port a legacy embedded application to Linux without having to switch APIs, i.e. you don't want to rewrite it on top of the POSIX interface. Xenomai may help in this case, since it supports multiple programming interfaces over a common real-time layer, including emulators of traditional RTOS APIs. Xenomai 3 will make those APIs available to a PREEMPT_RT based system as well.
- the target hardware platform has limited horsepower, and/or you want the real-time job to put the smallest possible overhead on your system. This is where dual kernels are usually better than a native preemption system. With the latter, all parts of the Linux system have to run internal code that prevents real-time activities from being delayed in an unacceptable manner (e.g. priority inheritance mechanism, threaded IRQ handlers). In a dual kernel system, there is no need for this, since the real-time co-kernel runs separately from the normal Linux kernel. Therefore, regular Linux activity is not charged for real-time activity, it does not even have to know about it.
In short, there cannot be any pre-canned answer to such a question: it really depends on your performance requirements, and your target hardware capabilities. This has to be evaluated on a case-by-case basis. Telling the world about "we can achieve X microseconds worst-case latency" without specifying the characteristics of the target platform would make no sense.
Q: I want to port a RTLinux/RTAI application running in kernel space to Xenomai. Since Xenomai 3 discontinues kernel space APIs, should I pick Xenomai 2 then?
A: It is one of the options the Xenomai project keeps open. Xenomai 2 will be on a long term maintenance path, and since this is a mature software architecture, it will remain a safe bet in any case. On the other hand, you may want to seize this opportunity to move your code to user-space and fully benefit from the standard programming model Xenomai 3 upholds.
Never felt the pain of debugging a crashing real-time application in kernel space, without any KGDB support due to the dual kernel pecularities? Probably you did, so what about a little help from standard GDB when developing? What about relying on a stable ABI while upgrading to Xenomai maintenance releases, so that you can upgrade applications in the field independently from the kernel side?
Worst-case scheduling latency in user-space is higher than the one you may get for a pure kernel-based application, that is a fact. But in a dual kernel environment, we are talking about an increase of 3 to 20 microseconds, depending on the target hardware (usually, how efficient the MMU is makes the difference). Most of the time, this difference still matches the requirements, and is not worth the burden of living in kernel space.
Q: I want to port a legacy VxWorks, pSOS, VRTX, etc. application to Xenomai. After some analysis, I concluded that I should port my code over the relevant kernel-based APIs. I have a single address space there, permanent supervisor privileges, direct access to device memory and ports, everything looks similar to my original runtime environment. Since Xenomai 3 discontinues kernel space APIs, should I pick Xenomai 2 then?
A: Same answer as previously, but with one major difference: your original code is likely under a proprietary license that does not mix well with the GPL that rules the Linux kernel. Moving all your proprietary code to user-space would probably solve such license issue. Xenomai libraries are licensed under the terms of the LGPL.
Xenomai 3 will support those APIs in user-space, so you may want to reconsider the issue differently:
- a multi-threaded Linux process provides a single address space as well, but with the added bonus of memory protection. So tasks/threads running your application could still share variables and code easily, but would not be able to crash the system by corrupting the kernel memory.
- many legacy RTOS do not provide strong device driver semantics, and in such environments, application code may do device management informally as it sees fit. However, keeping the option of porting your code to a native Linux environment open, will require that you do follow the device driver / application split. So you may want to tackle the issue upfront and only once, and port your driver code over RTDM directly, which will keep your options open afterwards; i.e. between relying on native preemption or a dual kernel system.
Q: Does the more common programming model upheld by Xenomai 3 mean that we will not be able to implement device drivers in user-space?
A: No. It basically means that you will not get any support from Xenomai 3 to implement application code in kernel space. The only API available from kernel space will be RTDM, to implement real-time device drivers.
Q: I am currently running a native API/POSIX based application in user-space over Xenomai 2, what will change for me with Xenomai 3?
A: If your target kernel is >= 2.6.35, basically nothing will change. Otherwise, you may want to upgrade to a more recent kernel in order to port to Xenomai 3.
Q: I am currently running an application based on the VxWorks, pSOS, VRTX etc. emulator in user-space over Xenomai 2, what will change for me with Xenomai 3?
A: Same as previously. Additionally, those emulators will be rebased on an evolution of the Xenomai/SOLO emulation engine, that will also work in a dual kernel environment.
Evolving RTDM in Xenomai
With porting Xenomai over PREEMPT_RT, RTDM will be given the unique chance of providing a driver framework for both native and dual kernel real-time Linux. Whenever this development will demand careful adjustments to the current API, these revisions will be tagged and clearly documented as before.
And while RTDM will go on enhancing its interfaces, also to provide user-space driver support as seamlessly as feasible, the design will always follow the maxim to keep features reduced to the required minimum for real-time device drivers. Straight interfaces and concise core services help to review critical drivers as well as the driver framework itself with respect to functional and temporal correctness.
We will furthermore continue integrating contributed high-quality industrial drivers or driver stacks into the main repository. And we will foster the specification of additional device profiles that provide hardware independent APIs, while keeping the door open for vendor-specific extensions.
To sum up, the way Linux gets real-time capabilities, either by native real-time preemption or a dual kernel approach, is not the most significant issue to us. What really matters are the interfaces Linux will provide on top of those capabilities, especially to ease the migration process from legacy real-time systems to GNU/Linux.
- Xenomai Task Market: list/schedule of enhancements, fixes, refactorings, or wishes to improve Xenomai in future versions
All brand and product names are registered trademarks of their respective owners.