Monday, January 16, 2012

Developing a multitasking OS for ARM part 3 3/4

So, we looked at how to do the difficult part of task swapping last time, now let's look at how we go about handling interrupts and the syscall layer.

Firstly, interrupts.  We'll not bother with FIQs for the moment, we'll stick with IRQs.

Despite having an ARM1176 core, the Raspberry Pi (or, rather, its Broadcom SoC) doesn't have a vectored interrupt controller - instead it has 3 "standard" ARM interrupt controllers.  That makes things a bit more complex for interrupt handling, but it's not too arduous.  As a quick reminder, here's the IRQ code we had before, with a little bit of meat in the middle:

.global _irq_handler

sub lr, lr, #4 /* Save adjusted LR_IRQ */
srsdb sp!, #SYS_MODE /* save LR_irq and SPSR_irq to system mode stack */

cpsid i,#SYS_MODE /* Go to system mode */

push {r0-r12} /* Save registers */

and r0, sp, #4 /* align the stack and save adjustment with LR_user */
sub sp, sp, r0
push {r0, lr}

/* Identify and clear interrupt source */
/* Should return handler address in r0 */
bl identify_and_clear_irq

blxne r0 /* go handle our interrupt if we have a handler */
/* An interruptible handler should disable / enable irqs */

/* Exit is via context switcher */
b switch_context_do

and the context switcher looks like this:

.global switch_context_do
/* Do we need to switch context? */
mov r3, #0x0c /* offset to fourth word of task block */
ldr r0, =__current_task
ldr r1, [r0]
ldr r0, =__next_task
ldr r2, [r0]
cmp r2, #0 /* If there's no next task, we can't switch */
beq .Lswitch_context_exit
cmp r1, #0 /* In the normal case, we will have a __current_task */
bne .Lnormal_case

/* When we get here, we're either idling in system mode at startup, or we've */
/* just voluntarily terminated a task.  In either case, we need to remove the */
/* return information we just pushed onto the stack, as we're never, ever going */
/* back. */
pop {r0, r1} /* remove any potential stack alignment */
add sp, sp, r0
add sp, sp, #0x3c /* and the other registers that should be there */
/* r0-r12, interrupted pc & spsr */
/* Now we can do our first actual task swap */
ldr r0,  =__next_task /* swap out the task */
ldr r2,  [r0]
ldr r0,  =__current_task
str r2,  [r0]
ldr sp,  [r2, r3] /* and restore stack pointer */
b .Lswitch_context_exit /* bail */
cmp r1, r2 /* otherwise, compare current task to next */
beq .Lswitch_context_exit

clrex /* Clear all mutexes */

/* At this point we have everything we need on the sysmode (user) stack */
/* {stack adjust, lr}_user, {r0-r12}_user, {SPSR, LR}_irq */
/* Save our stack pointer, and swap in the new one before returning */

ldr r0, =__current_task /* save current stack pointer */
ldr r0, [r0]
str sp, [r0, r3] /* stack pointer is second word of task object */
ldr r0,  =__next_task /* swap out the task */
ldr r2,  [r0]
ldr r0,  =__current_task
str r2,  [r0]
ldr sp,  [r2, r3] /* and restore stack pointer */
pop {r0, lr} /* restore LR_user and readjust stack */
add sp, sp, r0
pop {r0-r12} /* and other registers */
rfeia sp! /* before returning */

The context switcher looks complex, but it isn't really.  There's 4 cases to cater for, viz:

  • No 'next' task, don't switch
  • No 'current' task, switch to 'next' task, cleanup stack and switch to 'next' task
  • 'next' task is the same as 'current' task, don't switch
  • 'next' task is different from 'current' task, switch
Obviously, the meat of the interrupt handler is held in 'identify_and_clear_interrupt', which does pretty much what it says on the tin.  In this article, I'll show the handler for the qemu platform, which is significantly simpler than that for the Pi, but the Pi handler looks largely the same modulo having to deal with 3 controllers.
.global identify_and_clear_irq

FUNC identify_and_clear_irq

ldr r4, =.Lirq_base

ldr r4, [r4]
/* read the vector address to indicate we're handling the interrupt */
ldr r0, [r4, #IRQ_HANDLER]
/* which IRQs are asserted? */
ldr r0, [r4, #IRQ_STATUS]
ldr r5, =__irq_handlers

clz r6, r0 /* which IRQ was asserted? */
mov r1, #1 /* make a mask */
bic r0, r0, r1, lsl r6 /* clear flag */
str r0, [r4, #IRQ_ACK] /* Now acknowledge the interrupt */
str r0, [r4, #IRQ_SOFTCLEAR] /* and make sure we clear software irqs too */
ldr r0, [r5, r6, lsl #2] /* load handler address */
.Lret: bx lr /* exit */
.word IRQ_BASE

.global __irq_handlers
__irq_handlers: .skip 32 * 4

and patching in a hander is as simple as setting the address for the interrupt handler into __irq_handlers at the appropriate place.  Simples, as they say in internet-land.

Syscalls are very similar.  Here's the syscall handler:

.global _svc_handler
srsdb sp!, #SYS_MODE /* save LR_svc and SPSR_svc to sys mode stack */
cpsid i,#SYS_MODE
push {r0-r12} /* Save registers */

and r0, sp, #4 /* align the stack and save adjustment with LR_user */
sub sp, sp, r0
push {r0, lr}
ldr r0,[lr,#-4] /* Calculate address of SVC instruction */
/* and load it into R0. */
and r0,r0,#0x000000ff /* Mask off top 24 bits of instruction */
/* to give SVC number. */
ldr r2, =__syscall_table /* get the syscall */
ldr r3, [r2, r0, lsl#2]
cmp r3, #0
beq _syscall_exit
tst r3, #0x01 /* what linkage are we using */
bxeq r3 /* ASM, just run away */
bic r3, r3, #0x01
blx r3 /* C, must come back here */

.global _syscall_exit
pop {r0, lr} /* restore LR_user and readjust stack */
add sp, sp, r0

pop {r0-r12} /* and other registers */
rfeia sp! /* before returning */

.section .bss
.global __syscall_table
/* leave space for 256 syscall addresses */
.skip 2048

The fun bit is how we go about getting the syscall number into the handler.  I've taken the "canonical" approach of using the svc operand, hence the bit where we get the instruction and extract the number.  Other ways include using a register, a global variable, pushing onto the stack, or some combination of these.

The other twist here is that I allow for both C and assembler syscall functions by setting (or not) bit 0 of the function address in the syscall table.  Assembler syscall handlers must, of course, exit via _syscall_exit or equivalent, but that's down to the programmer to get it right.


  1. Hi,
    Would you consider going a little into how the processor can be made to talk to the other parts of the Pi board? Particularly the screen and a USB keyboard since I'm assuming you plan to implement those at the least in order to make the OS at least slightly interactive.
    Although task scheduling is certainly interesting and isn't stuff I'd be able to do this well by myself, it's something I could just about get by without doing well. On the other hand there really isn't any way to interact with the Pi without a screen and keyboard working.

    1. Hello Angus
      For the moment, I'm concentrating on the low level aspects, the underlying bedrock of any given system. As such, you can expect some further gnarly low-level stuff before we get anywhere near being able to plug in a USB keyboard and mouse.

      PArticularly, there's a bunch of stuff that needs to be done regarding memory allocation (and reclamation - remember, this is a Lisp-based OS), and then we'll probably get on to at least making it display something.

      USB drivers promise to be fun. For the moment, it's serial port only...


  2. Hi Simon,

    In your previous post you said that each task has got an absolute maximum time, and they can get capped by a preemptive scheduler.

    After getting 'capped', will the preempted task resume ASAP? Or do we go through the list (round robin) before we start executing it again where we left it?,

    task 5 takes a long time.
    IRQ. It gets preempted.
    then what?

    thank you! I like reading your blog entries :)