From Xenomai

[edit] Adapting ARM I-pipe patch to a new board.

Linux board X specific code is located in the arch/arm/mach-X and include/asm-arm/arch-X subdirectories of Linux kernel sources.

To start porting the I-pipe patch to a new board, look in these directories to see where the hardware timer and its interrupt are handled (usually arch/arm/mach-X/time.c) and where cascaded interrupt, if any, are decoded (usually arch/arm/mach-X/irq.c). In the rest of this document we will call these files time.c and irq.c, even though they may have another name for your board.

[edit] New variables and functions.

You are going to modify these two files to define the following variables and functions:

[edit] I-pipe and dynamic tick.

The dynamic tick is not supported with the I-pipe patch. So, please include the following snippet in time.c:

#ifdef CONFIG_IPIPE
#ifdef CONFIG_NO_IDLE_HZ
#error "dynamic tick timer not yet supported with IPIPE"
#endif /* CONFIG_NO_IDLE_HZ */
#endif /* CONFIG_IPIPE */

[edit] Modifying the timer management.

The timer interrupt handler should be modified in two ways:

• first, if the timer interrupt needs to be acknowledged in some way (clearing a

flag in a hardware register, reading a status in a hardware register, etc...), this acknowledgement must be put in __ipipe_mach_acktimer, because when the timer is handled by Xenomai, timer interrupts will no longer get handled by Linux timer interrupt handler.

• second, if the timer interrupt runs timer_tick() in a while loop, accounting

for lost ticks by comparing the current value of a free-running counter with a hardware match register, this comparison must be replaced with a comparison of the free-running counter with a variable, because when Xenomai controls the timer, the match register is no longer related to Linux timer tick date.

Note that this paragraph departs from what was in this wiki for versions of the I-pipe patch until version 1.7-06. This because in past versions of the I-pipe a log was maintained for each interruption, which resulted in the interrupt handler being invoked as many times as the interrupt was triggered, so we did not need to account for lost ticks. Starting with I-pipe version 1.8, the log has been "flattened" which means that it now acts as a single bit, if an interrupt is triggered several times before its handler is invoked, the handler will be invoked only once, so we need again to account for lost tick in Linux tick handler.

So the typical pxa_timer_interrupt, which looks like:

pxa_timer_interrupt()
{
        /* (...) */
        do {
                timer_tick();
                OSSR = OSSR_M0;
                next_match = (OSMR0 += LATCH);
        while ((signed long) (next_match - OSCR) <= 8);
        /* (...) */
}

becomes:

static unsigned long last_jiffy_time;

void __ipipe_mach_acktimer(void)
{
        OSSR = OSSR_M0;  /* Clear match on timer 0 */
}

pxa_timer_interrupt()
{
        int next_match;
         /* (...) */
         do {
               timer_tick();
#ifndef CONFIG_IPIPE
               OSSR = OSSR_M0;  /* Clear match on timer 0 */
#else /* CONFIG_IPIPE */
               last_jiffy_time += LATCH;
               if (__ipipe_mach_timerstolen)
                       next_match = last_jiffy_time + LATCH;
               else
#endif /* CONFIG_IPIPE */
               next_match = (OSMR0 += LATCH);
         } while( (signed long)(next_match - OSCR) <= 8 );
         /* (...) */
}

[edit] Modifying struct sys_timer::offset

Another Linux function that the I-pipe patch should modify is the offset member of the sys_timer struct. Here, there are two cases:

• either the platform has a free-running counter, then adding LATCH to a

variable at each Linux timer handler invocation, and substracting this variable from the free-running counter is enough to implement the offset function. If last_jiffy_time is maintained as in the code above, the offset function for PXA could be, for instance:

static unsigned long pxa_gettimeoffset (void)
{
        long elapsed, usec;

        elapsed = OSCR - last_jiffy_time;

        /* Now convert them to usec */
        usec = (unsigned long)(elapsed * (tick_nsec / 1000))/LATCH;

        return usec;
}

• or the only available hardware register is a decrementer, then the

implementation is a bit more complicated, have a look at the integrator implementation (file arch/arm/mach-integrator/core.c).

[edit] Implementing a high-resolution counter.

The I-pipe patch for ARM implements an emulation of a 64 bits high-resolution counter (the one called tsc on x86, we will hence call it tsc for short), using whatever hardware is proposed by the underlying machine. A port of the I-pipe patch for ARM is expected to define:

• __ipipe_mach_get_tsc() which emulates the tsc;
• __ipipe_mach_get_tscinfo() which is used to export the tsc to user-space.

Note that the tsc emulation can not work in user-space in the SMP case, because there is no fast (meaning without a system call) way to get the current processor id.

In order to allow sharing data between kernel and user-space with no cache effect, the Linux kernel uses the vectors page, the I-pipe patch defines an area of 20 bytes in this page to share tsc emulation data between kernel-space and user-space. The symbol __ipipe_tsc_area is a pointer to the beginning of this area.

Currently, there are two kinds of hardware counters which Xenomai can access from user-space in order to emulate a tsc.

[edit] Free-running counters

We will use the PXA as an implementation example in this section.

In this case, only the first 8 bytes of __ipipe_tsc_area are used to store a previous result of the tsc. A typical implementation is as follows:

• a pointer called tsc is initiliazed to __ipipe_tsc_area if not SMP or to a

static array if SMP:

union tsc_reg {
#ifdef __BIG_ENDIAN
	struct {
		unsigned long high;
		unsigned long low;
	};
#else /* __LITTLE_ENDIAN */
	struct {
		unsigned long low;
		unsigned long high;
	};
#endif /* __LITTLE_ENDIAN */
	unsigned long long full;
};

#ifdef CONFIG_SMP
static union tsc_reg tsc[NR_CPUS];

/* (...) */
#else /* !CONFIG_SMP */
static union tsc_reg *tsc;

/* (...) */
#endif

in pxa_timer_init:

#ifdef CONFIG_IPIPE
#ifndef CONFIG_SMP
	tsc = (union tsc_reg *) __ipipe_tsc_area;
	barrier();
#endif /* CONFIG_SMP */

	pxa_timer_initialized = 1;
#endif /* CONFIG_IPIPE */

• a function is expected to update the tsc area from time to time (at least

twice before the underlying counter wraps, in order to avoid losing some wrap arounds), the PXA implementation uses __ipipe_mach_acktimer for this:

void __ipipe_mach_acktimer(void)
{
	OSSR = OSSR_M0;  /* Clear match on timer 0 */
	if (likely(pxa_timer_initialized)) {
		union tsc_reg *local_tsc;
		unsigned long stamp, flags;

		local_irq_save_hw(flags);
		local_tsc = &tsc[ipipe_processor_id()];
		stamp = OSCR;
		if (unlikely(stamp < local_tsc->low))
			/* 32 bit counter wrapped, increment high word. */
			local_tsc->high++;
		local_tsc->low = stamp;
		local_irq_restore_hw(flags);
	}
}

• the function __ipipe_mach_get_tsc reads the tsc area and free-running counter

to get the current tsc value:

notrace unsigned long long __ipipe_mach_get_tsc(void)
{
	if (likely(pxa_timer_initialized)) {
		union tsc_reg *local_tsc, result;
		unsigned long stamp;

		local_tsc = &tsc[ipipe_processor_id()];

		__asm__ ("ldmia %1, %M0\n"
			 : "=r"(result.full), "+&r"(local_tsc)
			 : "m"(*local_tsc));
		barrier();
		stamp = OSCR;
		if (unlikely(stamp < result.low))
			/* 32 bit counter wrapped, increment high word. */
			result.high++;
		result.low = stamp;

		return result.full;
	}

        return 0;
}

Note that we use ldmia to read the tsc area in order to avoid having to shut interrupts off.

Also note that the function has the "notrace" attribute, this because it is called by the I-pipe tracer code, so we would enter an infinite loop if it was traced. As such, this function is only permitted to call other non traced functions, in particular, if interrupts need to be disabled, the "notrace" variants of the interrupts disabling/enabling macros should be used.

• the function __ipipe_mach_get_tscinfo is defined as follows:

#ifdef CONFIG_SMP

/* (...) */
void __ipipe_mach_get_tscinfo(struct __ipipe_tscinfo *info)
{
	info->type = IPIPE_TSC_TYPE_NONE;
}
#else
/* (...) */
void __ipipe_mach_get_tscinfo(struct __ipipe_tscinfo *info)
{
	info->type = IPIPE_TSC_TYPE_FREERUNNING;
	info->u.fr.counter = (unsigned *) 0x40A00010;
	info->u.fr.mask = 0xffffffff;
	info->u.fr.tsc = &tsc->full;
}
#endif

info->type indicates that the tsc is based on a free-running counter,

info->u.fr.counter is set to the PHYSICAL address of the free-running counter,

info->u.fr.mask is a mask indicating which bits in the free-running counter are valid.

info->u.fr.tsc is a pointer to the shared tsc area

[edit] Decrementer

See s3c24xx implementation.

[edit] Tips and tricks.

• When the screen remains blank: activate Kernel low-level debugging functions in

the kernel hacking menu, then add a call to printascii(printk_buf) rigth after the call to vscnprintf(printk_buf, ...) in function vprintk, file kernel/printk.c

• If the kernel locks at "Calibrating delay loop...", it means that the timer

interrupt is not ticking and that the delay calibration routine is running an infinite loop at while (ticks == jiffies) in function calibrate_delay, file init/calibrate.c. To help debugging this situation, you can put printks in the while (ticks == jiffies) loop, printing any hardware timer registers or status register you want.

• If the kernel crashes when I-pipe tracer is enabled, and the function where it

crashes is marked with the "naked" attribute, you should also add the notrace attribute to this function, "naked" functions can not be instrumented.

• If Xenomai seems to run for a while, then locks up. Sometimes, there are minimum

delays below which the hardware timer can not be reprogrammed. For instance the PXA timer hardware can not be programmed for delays shorter than 8 ticks, hence __ipipe_mach_set_dec:

void __ipipe_mach_set_dec(unsigned long delay)
{
        if (delay > 8) {
                unsigned long flags;

                local_irq_save_hw(flags);
                OSMR0 = delay + OSCR;
                local_irq_restore_hw(flags);
        } else
                ipipe_trigger_irq(IRQ_OST0);
}