An alternative program architecture for embedded Lua
Locked
 
|
Ned Konz-2 |
An alternative program architecture for embedded Lua
|
Hello folks,
I'd like to introduce myself. I'm new to Lua, but have been doing embedded system work since 1975 (on the MC6800 microprocessor). I'm now using Cortex-M3 processors, and have been thinking once more about the usefulness of a VM on such a chip. I'd be interested in hearing your thoughts on the subject. My own interests don't necessarily include a self-hosted language system (though it's nice to have, I don't know how much space the compiler requires); I'd prefer to spend the flash on system capabilities and extended domain-specific vocabularies. The eLua VM looks attractive as a dynamic runtime engine, though. However, the runtime architecture that eLua currently uses (which AFAICT is roughly the same as Lua on the desktop) has a several limitations that I'd like to change: * No way to do interesting work in Lua from interrupt handlers. Unless you want to use the peripherals the way the driver writer had provided for, you're out of luck. * The Big Loop structure. The (Big Loop) + (interrupt handlers) scheme has proven to be inflexible and a poor match for the mostly event-driven needs of embedded systems, especially where concurrency is required. Existing threading schemes and primitives (especially semaphores!) are difficult to write reliable code with. A demonstration of the difficulties of this structure can be seen with the uIP demo code; uIP generates event notifications via callbacks, but the eLua HTTP server application is itself blocking in a Big Loop. So the "impedance mismatch" between the lightweight event notifications from uIP and the blocking socket interface in the server requires extra code and would be hard to multi-thread. What I'd like to do instead is to implement a system using the eLua VM that does this: * Allows for eLua peripheral control at a level equivalent to that currently done in C code. Once you get past the simple configuration of I/O ports, the complexity and richness of existing microcontroller peripherals makes it hard to fully use the capabilities of the chip. For instance, I can set up the STM32F10x peripherals so that the timer's input capture event triggers a DMA from a GPIO port to memory, at the same time that I'm using another DMA channel to transfer bytes from the I2C receive into memory, with interrupts at the beginning of each I2C transfer. It would be hard if not impossible to do that in eLua the way it stands now (I think; I'd love to hear that this would be possible to do). * Allows eLua programs to be structured as one or more hierarchical state machines, each with its own runtime context and event queue. * Responds to interrupts by packaging up the minimal context (at least which interrupt occurred, and perhaps which of several interrupt sources caused it) into an event and delivering it to (subscribed or all) state machines. * Does cooperative multi-tasking by using a run-to-completion scheme: each state machine that is ready to run (i.e. has an event ready to react to) gets its current state's handler routine run with the next event; these are not preempted but rather do their work and return. It seems to me that this would not require any kind of thread support in eLua itself (as the state machine executive is itself a single-threaded program), and that it would better match the needs of embedded systems. For a good example of how clean an equivalent embedded HTTP server demo can be (using the lwIP TCP/IP stack, which also uses callbacks like uIP), look at http://www.state-machine.com/lwip/ where Miro Samek has provided an example program that includes lwIP, an HTTP server with SSI and CGI capabilities, and the "Dining Philosopher Problem" demo (which is a multi-threaded simulation). I have ported the QP architecture (described at http://www.state-machine.com) to Ruby and Squeak Smalltalk, and have used it in C programs in embedded systems; I've found it to be very lightweight and a good way to write embedded programs. I would probably use a very similar scheme for eLua. What do you think? Ned Konz _______________________________________________ eLua-dev mailing list [hidden email] https://lists.berlios.de/mailman/listinfo/elua-dev |
|
|
Ned, excellent discussion topic and ideas. I work on a different vm
than eLua, but I listen in because of the overlapping topics. I've been trying to find a way to use coroutines as interrupt handlers. Coroutines are functions that persist their own state between calls. But, you're right, there's always the issue of notifying the "big loop" when an interrupt has occurred and the lag between the actual event and when the VM is signaled and can respond to the event. Coroutines would also be a good way to do the cooperative multi- tasking and state machines that you referred to. Thanks for sharing the info about QP, I'll look into that. !!Dean On Jun 20, 2010, at 20:04 , Ned Konz wrote: > Hello folks, > > I'd like to introduce myself. I'm new to Lua, but have been doing > embedded system work since 1975 (on the MC6800 microprocessor). I'm > now using Cortex-M3 processors, and have been thinking once more > about the usefulness of a VM on such a chip. > > I'd be interested in hearing your thoughts on the subject. > > My own interests don't necessarily include a self-hosted language > system (though it's nice to have, I don't know how much space the > compiler requires); I'd prefer to spend the flash on system > capabilities and extended domain-specific vocabularies. > > The eLua VM looks attractive as a dynamic runtime engine, though. > > However, the runtime architecture that eLua currently uses (which > AFAICT is roughly the same as Lua on the desktop) has a several > limitations that I'd like to change: > > * No way to do interesting work in Lua from interrupt handlers. > Unless you want to use the peripherals the way the driver writer had > provided for, you're out of luck. > > * The Big Loop structure. The (Big Loop) + (interrupt handlers) > scheme has proven to be inflexible and a poor match for the mostly > event-driven needs of embedded systems, especially where concurrency > is required. Existing threading schemes and primitives (especially > semaphores!) are difficult to write reliable code with. > > A demonstration of the difficulties of this structure can be seen > with the uIP demo code; uIP generates event notifications via > callbacks, but the eLua HTTP server application is itself blocking > in a Big Loop. So the "impedance mismatch" between the lightweight > event notifications from uIP and the blocking socket interface in > the server requires extra code and would be hard to multi-thread. > > What I'd like to do instead is to implement a system using the eLua > VM that does this: > > * Allows for eLua peripheral control at a level equivalent to that > currently done in C code. Once you get past the simple configuration > of I/O ports, the complexity and richness of existing > microcontroller peripherals makes it hard to fully use the > capabilities of the chip. For instance, I can set up the STM32F10x > peripherals so that the timer's input capture event triggers a DMA > from a GPIO port to memory, at the same time that I'm using another > DMA channel to transfer bytes from the I2C receive into memory, with > interrupts at the beginning of each I2C transfer. It would be hard > if not impossible to do that in eLua the way it stands now (I think; > I'd love to hear that this would be possible to do). > > * Allows eLua programs to be structured as one or more hierarchical > state machines, each with its own runtime context and event queue. > > * Responds to interrupts by packaging up the minimal context (at > least which interrupt occurred, and perhaps which of several > interrupt sources caused it) into an event and delivering it to > (subscribed or all) state machines. > > * Does cooperative multi-tasking by using a run-to-completion > scheme: each state machine that is ready to run (i.e. has an event > ready to react to) gets its current state's handler routine run with > the next event; these are not preempted but rather do their work and > return. > > It seems to me that this would not require any kind of thread > support in eLua itself (as the state machine executive is itself a > single-threaded program), and that it would better match the needs > of embedded systems. > > For a good example of how clean an equivalent embedded HTTP server > demo can be (using the lwIP TCP/IP stack, which also uses callbacks > like uIP), look at http://www.state-machine.com/lwip/ where Miro > Samek has provided an example program that includes lwIP, an HTTP > server with SSI and CGI capabilities, and the "Dining Philosopher > Problem" demo (which is a multi-threaded simulation). > > I have ported the QP architecture (described at http://www.state-machine.com > ) to Ruby and Squeak Smalltalk, and have used it in C programs in > embedded systems; I've found it to be very lightweight and a good > way to write embedded programs. I would probably use a very similar > scheme for eLua. > > What do you think? > > Ned Konz > > _______________________________________________ > eLua-dev mailing list > [hidden email] > https://lists.berlios.de/mailman/listinfo/elua-dev _______________________________________________ eLua-dev mailing list [hidden email] https://lists.berlios.de/mailman/listinfo/elua-dev |
|
|
By QP do you mean "Queue Pair (/QP/), the message transfer engine that
lies at the heart of InfiniBand"? Howard Dean Hall wrote: > Ned, excellent discussion topic and ideas. I work on a different vm > than eLua, but I listen in because of the overlapping topics. I've > been trying to find a way to use coroutines as interrupt handlers. > Coroutines are functions that persist their own state between calls. > But, you're right, there's always the issue of notifying the "big > loop" when an interrupt has occurred and the lag between the actual > event and when the VM is signaled and can respond to the event. > > Coroutines would also be a good way to do the cooperative > multi-tasking and state machines that you referred to. > > Thanks for sharing the info about QP, I'll look into that. > > !!Dean > > > On Jun 20, 2010, at 20:04 , Ned Konz wrote: > >> Hello folks, >> >> I'd like to introduce myself. I'm new to Lua, but have been doing >> embedded system work since 1975 (on the MC6800 microprocessor). I'm >> now using Cortex-M3 processors, and have been thinking once more >> about the usefulness of a VM on such a chip. >> >> I'd be interested in hearing your thoughts on the subject. >> >> My own interests don't necessarily include a self-hosted language >> system (though it's nice to have, I don't know how much space the >> compiler requires); I'd prefer to spend the flash on system >> capabilities and extended domain-specific vocabularies. >> >> The eLua VM looks attractive as a dynamic runtime engine, though. >> >> However, the runtime architecture that eLua currently uses (which >> AFAICT is roughly the same as Lua on the desktop) has a several >> limitations that I'd like to change: >> >> * No way to do interesting work in Lua from interrupt handlers. >> Unless you want to use the peripherals the way the driver writer had >> provided for, you're out of luck. >> >> * The Big Loop structure. The (Big Loop) + (interrupt handlers) >> scheme has proven to be inflexible and a poor match for the mostly >> event-driven needs of embedded systems, especially where concurrency >> is required. Existing threading schemes and primitives (especially >> semaphores!) are difficult to write reliable code with. >> >> A demonstration of the difficulties of this structure can be seen >> with the uIP demo code; uIP generates event notifications via >> callbacks, but the eLua HTTP server application is itself blocking in >> a Big Loop. So the "impedance mismatch" between the lightweight event >> notifications from uIP and the blocking socket interface in the >> server requires extra code and would be hard to multi-thread. >> >> What I'd like to do instead is to implement a system using the eLua >> VM that does this: >> >> * Allows for eLua peripheral control at a level equivalent to that >> currently done in C code. Once you get past the simple configuration >> of I/O ports, the complexity and richness of existing microcontroller >> peripherals makes it hard to fully use the capabilities of the chip. >> For instance, I can set up the STM32F10x peripherals so that the >> timer's input capture event triggers a DMA from a GPIO port to >> memory, at the same time that I'm using another DMA channel to >> transfer bytes from the I2C receive into memory, with interrupts at >> the beginning of each I2C transfer. It would be hard if not >> impossible to do that in eLua the way it stands now (I think; I'd >> love to hear that this would be possible to do). >> >> * Allows eLua programs to be structured as one or more hierarchical >> state machines, each with its own runtime context and event queue. >> >> * Responds to interrupts by packaging up the minimal context (at >> least which interrupt occurred, and perhaps which of several >> interrupt sources caused it) into an event and delivering it to >> (subscribed or all) state machines. >> >> * Does cooperative multi-tasking by using a run-to-completion scheme: >> each state machine that is ready to run (i.e. has an event ready to >> react to) gets its current state's handler routine run with the next >> event; these are not preempted but rather do their work and return. >> >> It seems to me that this would not require any kind of thread support >> in eLua itself (as the state machine executive is itself a >> single-threaded program), and that it would better match the needs of >> embedded systems. >> >> For a good example of how clean an equivalent embedded HTTP server >> demo can be (using the lwIP TCP/IP stack, which also uses callbacks >> like uIP), look at http://www.state-machine.com/lwip/ where Miro >> Samek has provided an example program that includes lwIP, an HTTP >> server with SSI and CGI capabilities, and the "Dining Philosopher >> Problem" demo (which is a multi-threaded simulation). >> >> I have ported the QP architecture (described at >> http://www.state-machine.com) to Ruby and Squeak Smalltalk, and have >> used it in C programs in embedded systems; I've found it to be very >> lightweight and a good way to write embedded programs. I would >> probably use a very similar scheme for eLua. >> >> What do you think? >> >> Ned Konz >> >> _______________________________________________ >> eLua-dev mailing list >> [hidden email] >> https://lists.berlios.de/mailman/listinfo/elua-dev > > _______________________________________________ > eLua-dev mailing list > [hidden email] > https://lists.berlios.de/mailman/listinfo/elua-dev > _______________________________________________ eLua-dev mailing list [hidden email] https://lists.berlios.de/mailman/listinfo/elua-dev |
|
|
On Jun 20, 2010, at 7:57 PM, Howard Hansen wrote: > By QP do you mean "Queue Pair (/QP/), the message transfer engine that lies at the heart of InfiniBand"? > Sorry, no... QP in this case stands for "Quantum Programming", which is a term that Dr. Miro Samek came up with to refer to his state machine framework which he described in "Practical Statecharts in C/C++" (subtitled "Quantum Programming for Embedded Systems") (the second edition is called "Practical UML Statecharts in C/C++, Second Edition" (subtitled "Event-driven Programming for Embedded Systems")) and on his web site http://state-machine.com (a/k/a http://quantum-leaps.com). > Howard > > > Dean Hall wrote: >> Ned, excellent discussion topic and ideas. I work on a different vm than eLua, but I listen in because of the overlapping topics. I've been trying to find a way to use coroutines as interrupt handlers. Coroutines are functions that persist their own state between calls. But, you're right, there's always the issue of notifying the "big loop" when an interrupt has occurred and the lag between the actual event and when the VM is signaled and can respond to the event. >> >> Coroutines would also be a good way to do the cooperative multi-tasking and state machines that you referred to. >> >> Thanks for sharing the info about QP, I'll look into that. >> >> !!Dean >> >> >> On Jun 20, 2010, at 20:04 , Ned Konz wrote: >> >>> Hello folks, >>> >>> I'd like to introduce myself. I'm new to Lua, but have been doing embedded system work since 1975 (on the MC6800 microprocessor). I'm now using Cortex-M3 processors, and have been thinking once more about the usefulness of a VM on such a chip. >>> >>> I'd be interested in hearing your thoughts on the subject. >>> >>> My own interests don't necessarily include a self-hosted language system (though it's nice to have, I don't know how much space the compiler requires); I'd prefer to spend the flash on system capabilities and extended domain-specific vocabularies. >>> >>> The eLua VM looks attractive as a dynamic runtime engine, though. >>> >>> However, the runtime architecture that eLua currently uses (which AFAICT is roughly the same as Lua on the desktop) has a several limitations that I'd like to change: >>> >>> * No way to do interesting work in Lua from interrupt handlers. Unless you want to use the peripherals the way the driver writer had provided for, you're out of luck. >>> >>> * The Big Loop structure. The (Big Loop) + (interrupt handlers) scheme has proven to be inflexible and a poor match for the mostly event-driven needs of embedded systems, especially where concurrency is required. Existing threading schemes and primitives (especially semaphores!) are difficult to write reliable code with. >>> >>> A demonstration of the difficulties of this structure can be seen with the uIP demo code; uIP generates event notifications via callbacks, but the eLua HTTP server application is itself blocking in a Big Loop. So the "impedance mismatch" between the lightweight event notifications from uIP and the blocking socket interface in the server requires extra code and would be hard to multi-thread. >>> >>> What I'd like to do instead is to implement a system using the eLua VM that does this: >>> >>> * Allows for eLua peripheral control at a level equivalent to that currently done in C code. Once you get past the simple configuration of I/O ports, the complexity and richness of existing microcontroller peripherals makes it hard to fully use the capabilities of the chip. For instance, I can set up the STM32F10x peripherals so that the timer's input capture event triggers a DMA from a GPIO port to memory, at the same time that I'm using another DMA channel to transfer bytes from the I2C receive into memory, with interrupts at the beginning of each I2C transfer. It would be hard if not impossible to do that in eLua the way it stands now (I think; I'd love to hear that this would be possible to do). >>> >>> * Allows eLua programs to be structured as one or more hierarchical state machines, each with its own runtime context and event queue. >>> >>> * Responds to interrupts by packaging up the minimal context (at least which interrupt occurred, and perhaps which of several interrupt sources caused it) into an event and delivering it to (subscribed or all) state machines. >>> >>> * Does cooperative multi-tasking by using a run-to-completion scheme: each state machine that is ready to run (i.e. has an event ready to react to) gets its current state's handler routine run with the next event; these are not preempted but rather do their work and return. >>> >>> It seems to me that this would not require any kind of thread support in eLua itself (as the state machine executive is itself a single-threaded program), and that it would better match the needs of embedded systems. >>> >>> For a good example of how clean an equivalent embedded HTTP server demo can be (using the lwIP TCP/IP stack, which also uses callbacks like uIP), look at http://www.state-machine.com/lwip/ where Miro Samek has provided an example program that includes lwIP, an HTTP server with SSI and CGI capabilities, and the "Dining Philosopher Problem" demo (which is a multi-threaded simulation). >>> >>> I have ported the QP architecture (described at http://www.state-machine.com) to Ruby and Squeak Smalltalk, and have used it in C programs in embedded systems; I've found it to be very lightweight and a good way to write embedded programs. I would probably use a very similar scheme for eLua. >>> >>> What do you think? >>> >>> Ned Konz >>> >>> _______________________________________________ >>> eLua-dev mailing list >>> [hidden email] >>> https://lists.berlios.de/mailman/listinfo/elua-dev >> >> _______________________________________________ >> eLua-dev mailing list >> [hidden email] >> https://lists.berlios.de/mailman/listinfo/elua-dev >> > > _______________________________________________ > eLua-dev mailing list > [hidden email] > https://lists.berlios.de/mailman/listinfo/elua-dev _______________________________________________ eLua-dev mailing list [hidden email] https://lists.berlios.de/mailman/listinfo/elua-dev |
|
|
In reply to this post by Ned Konz-2
Hi,
On Mon, Jun 21, 2010 at 4:04 AM, Ned Konz <[hidden email]> wrote:
Hello folks, Nice to meet you. My own interests don't necessarily include a self-hosted language system (though it's nice to have, I don't know how much space the compiler requires); I'd prefer to spend the flash on system capabilities and extended domain-specific vocabularies. Roughly, you can do fine with eLua in about 200k of Flash on Cortex-M3, or even less. tt depends on the components you choose to include in your build. I'm much more worried about RAM than Flash most of the time. The eLua VM looks attractive as a dynamic runtime engine, though. It's definitely the most atractive one I found. There is also a very good (and free) BASIC variant for micros, with source code and excellent documentation (the VM itself is documented quite nicely) at http://www.compuphase.com/pawn/pawn.htm. Yes, I know we're talking about BASIC :), but the VM was quite OK from what I can remember, and the documentation is outstanding. However, the runtime architecture that eLua currently uses (which AFAICT is roughly the same as Lua on the desktop) has a several limitations that I'd like to change: We'll have interrupt support in eLua soon, although it won't be without limitations, as the Lua VM is not interruptible. I plan to implement it using standard Lua hooks (I actually have some proof on concept code that works quite nicely), which will be a huge problem if the VM remains stuck in a blocking function (I/O for example). But I don't know of a better way of doing this, at least not without using an OS with threading support. * The Big Loop structure. The (Big Loop) + (interrupt handlers) scheme has proven to be inflexible and a poor match for the mostly event-driven needs of embedded systems, especially where concurrency is required. Existing threading schemes and primitives (especially semaphores!) are difficult to write reliable code with. The eLua HTTP server is something hand-crafted in half a week-end, so not really a good reference :) It can be rewritten quite nicely using coroutines (once the network support in eLua gets better itself, because now it's not really usable for handling multiple connections). What I'd like to do instead is to implement a system using the eLua VM that does this: It's not hard, it's just not imlemented yet. You'd need at least two things to implement the above: 1. interrupt support directly in eLua (and we'll have this soon, as previously stated) 2. a Lua module that implements the CPU specific functions needed for DMA, I2C and whatever else. We don't have that, but they can certainly be written, and they are perfectly inline with eLua's current architecture (they are called "platform specific modules" in the official documentation). We try to focus on the core eLua code + generic modules most of the time, and let people write their own platform specific modules for their platform of choice when there's a need for that. * Allows eLua programs to be structured as one or more hierarchical state machines, each with its own runtime context and event queue. You mean cooperative multi-tasking at coroutine level? Or with an OS? If you mean multi-tasking at coroutine level, what you wrote above can probably be implemented directly in eLua (+interrupt handlers), without any need to modify the VM or any other part of the code. Please let me know if I'm missing something here :) It seems to me that this would not require any kind of thread support in eLua itself (as the state machine executive is itself a single-threaded program), and that it would better match the needs of embedded systems. I'm hesitant in naming a development methodology "better" than another in most cases. An event+state machine based model (as they are used together more often than not) is a great thing for particular classes of applications (protocol implementations being the first example that comes to mind), but it's hard to generalize this. Some applications are perfectly happy with the Big Loop structure (thanks, I didn't know this is the official name :) ), others can do all their work only in interrupt handlers without having to worry much about synchronization, and so on. For a good example of how clean an equivalent embedded HTTP server demo can be (using the lwIP TCP/IP stack, which also uses callbacks like uIP), look at http://www.state-machine.com/lwip/ where Miro Samek has provided an example program that includes lwIP, an HTTP server with SSI and CGI capabilities, and the "Dining Philosopher Problem" demo (which is a multi-threaded simulation).
I think this is a great idea for many applications, but not all of them. All in all, I'd be very interested to see something like this running in eLua :) One note though: QP seems to be GLPed, so not compatible with eLua (which is mostly MIT/BSD). So a direct port might not be possible. Best, Bogdan _______________________________________________ eLua-dev mailing list [hidden email] https://lists.berlios.de/mailman/listinfo/elua-dev |
|
|
Ned Konz-2 |
Re: An alternative program architecture for embedded Lua
|
|
On Jun 21, 2010, at 2:31 AM, Bogdan Marinescu wrote:
> > Roughly, you can do fine with eLua in about 200k of Flash on > Cortex-M3, or even less. tt depends on the components you choose to > include in your build. I'm much more worried about RAM than Flash most > of the time. What have you identified as the minimum RAM requirements? Given that there are 64K RAM Cortex-M3 parts available, it seems like your work on separating flash from RAM objects should help considerably. > There is also a very good (and free) BASIC variant for micros, with > source code and excellent documentation (the VM itself is documented > quite nicely) at http://www.compuphase.com/pawn/pawn.htm. Yes, I know > we're talking about BASIC :), but the VM was quite OK from what I can > remember, and the documentation is outstanding. Thanks for the pointer. > > * No way to do interesting work in Lua from interrupt handlers. > > Unless you want to use the peripherals the way the driver writer had > > provided for, you're out of luck. > > We'll have interrupt support in eLua soon, although it won't be > without limitations, as the Lua VM is not interruptible. I plan to > implement it using standard Lua hooks (I actually have some proof > on concept code that works quite nicely), which will be a huge problem > if the VM remains stuck in a blocking function (I/O for example). But > I don't know of a better way of doing this, at least not without using > an OS with threading support. What I was intending to do, at least as a first attempt, was to keep the interrupt processing in C, and keep it as generic and simple as possible. There is very little that *must* be done in an interrupt handler, typically. So the job of the interrupt handler is to package up into an event object: * the identity of the interrupt itself (including perhaps the identity of the interrupt source if there are multiple possibilities for a single interrupt) * whatever other data (if any) that is *always* relevant to handling that interrupt (perhaps a received character for a UART receive-buffer-not-empty interrupt, or a timer value for a timer input capture interrupt) and then publish or post that event object for consumption by the state machines. That's it. This way, the interrupts are kept fast and short, and won't need to be changed for alternative uses of the various peripherals. The state machines are, of course, welcome to use less of the supplied data (if we're just interested in the rate of received characters, we can discard the characters themselves), or do something with it (like buffering received characters somewhere), or go out and get additional state from the peripheral status registers. > The eLua HTTP server is something hand-crafted in half a week-end, so > not really a good reference :) It can be rewritten quite nicely using > coroutines (once the network support in eLua gets better itself, > because now it's not really usable for handling multiple connections). How are coroutines handled in eLua? Is there explicit support for them, or are they just done using continuations? > > What I'd like to do instead is to implement a system using the eLua > > VM that does this: > > > > * Allows for eLua peripheral control at a level equivalent to that > > currently done in C code. Once you get past the simple configuration > > of I/O ports, the complexity and richness of > > existing microcontroller peripherals makes it hard to fully use the > > capabilities of the chip. For instance, I can set up the STM32F10x > > peripherals so that the timer's input capture event triggers a DMA > > from a GPIO port to memory, at the same time that I'm using another > > DMA channel to transfer bytes from the I2C receive into memory, with > > interrupts at the beginning of each I2C transfer. It would be hard > > if not impossible to do that in eLua the way it stands now (I think; > > I'd love to hear that this would be possible to do). > > It's not hard, it's just not imlemented yet. Is there direct support for (say) peripheral register access? If not, could a kind of boxed pointer be made that allows for access to the various register spaces? I could see doing this in a bit of a condensed form: say 20-24 bits for an address offset, and a few bits to describe the space (flash, RAM, a few different peripheral register spaces). This allows for peripheral handling code to be written in Lua itself, and not have to use code and RAM space for platform-specific modules that may not be needed. > You'd need at least two things to implement the above: > > 1. interrupt support directly in eLua (and we'll have this soon, as > previously stated) > 2. a Lua module that implements the CPU specific functions needed for > DMA, I2C and whatever else. We don't have that, but they can certainly > be written, and they are perfectly inline with eLua's current > architecture (they are called "platform specific modules" in the > official documentation). We try to focus on the core eLua code + > generic modules most of the time, and let people write their own > platform specific modules for their platform of choice when there's a > need for that. Understood. I just don't see why they need to be in C. > > * Does cooperative multi-tasking by using a run-to-completion > > scheme: each state machine that is ready to run (i.e. has an event > > ready to react to) gets its current state's handler routine run > > with the next event; these are not preempted but rather do their > > work and return. > > > You mean cooperative multi-tasking at coroutine level? Or with an OS? > If you mean multi-tasking at coroutine level, what you wrote above can > probably be implemented directly in eLua (+interrupt handlers), > without any need to modify the VM or any other part of the code. > Please let me know if I'm missing something here :) Yes, I don't see that VM alterations would be needed, other than perhaps allowing for a type to represent peripheral registers so that user Lua code can be written to do the bulk of the work. Though it may also be helpful to separately manage event objects (so as not to burden the GC with these transient objects that can be allocated from their own pools). I'm proposing something even simpler than coroutines: just subroutines. Each state machine has one or more state handler routines. There is an implicit "top" handler that is the parent of the topmost user state handlers; it merely ignores all events. When an event is delivered to a state machine, that machine's current state handler routine is called and passed the event. That routine does whatever it wants to (possibly nothing) with the event, returning an indication that: * it handled the event * it ignored the event * it wants its parent state's handler to handle the event (also returning a pointer to that handler) * it has handled the event and has changed the current state (also returning a pointer to the new handler) Typically, the structure of the handler routines is a big switch on the event type. Additionally, there are a few special event types that are used for state machine actions: * state machine initialization * state entry * state exit * probe (which all handlers pass to their superstate handlers, thereby providing dynamic state hierarchy discovery) Roughly, this is the QP implementation from the user point of view: typedef uint8_t QState; typedef QState (*QStateHandler)(void *me, QEvent const *e); #define Q_HANDLED() ((QState)0) #define Q_IGNORED() ((QState)1) #define Q_TRAN(s) ((me->state = (QStateHandler)(s)), ((QState)2)) #define Q_SUPER(s) ((me->state = (QStateHandler)(s)), ((QState)3)) QState MyMachine_SomeTopLevelState(MyMachine *me, QEvent const *e) { switch (e->sig) { case Q_ENTRY_SIG: { /* entry actions */ return Q_HANDLED(); } case Q_EXIT_SIG: { /* exit actions */ return Q_HANDLED(); } case SOME_SIG_I_HANDLE: { /* actions */ return Q_HANDLED(); } case SOME_OTHER_SIG_I_HANDLE: { /* actions */ return Q_TRAN(&MyMachine_SomeOtherState); } case SOME_SIG_I_DONT_HANDLE: { /* let my parent state handle this */ return Q_SUPER(&QHsm_top); } /* ... */ } return Q_SUPER(&QHsm_top); /* default: let parent handle this */ } QState MyMachine_SomeSubState(MyMachine *me, QEvent const *e) { switch (e->sig) { case Q_ENTRY_SIG: { /* entry actions */ return Q_HANDLED(); } case Q_EXIT_SIG: { /* exit actions */ return Q_HANDLED(); } case SOME_SIG_I_DONT_HANDLE: { /* let my parent state handle this */ return Q_SUPER(&MyMachine_SomeTopLevelState); } /* ... */ } return Q_SUPER(&MyMachine_SomeTopLevelState); /* default: let parent handle this */ } > > It seems to me that this would not require any kind of thread > > support in eLua itself (as the state machine executive is itself a > > single-threaded program), and that it would better match the needs > > of embedded systems. > > I'm hesitant in naming a development methodology "better" than another > in most cases. An event+state machine based model (as they are used > together more often than not) is a great thing for particular classes > of applications (protocol implementations being the first example that > comes to mind), but it's hard to generalize this. Some applications > are perfectly happy with the Big Loop structure (thanks, I didn't know > this is the official name :) ), others can do all their work only in > interrupt handlers without having to worry much about synchronization, > and so on. Big Loop is merely my name for that idiom, of course. And I don't think it's original, though I don't recall where I heard it first. Can you give an example of an embedded application whose runtime needs are better served by the Big Loop structure? I am not concerned here with considerations of programmer convenience or aesthetics. I would like to use a more domain-specific language to specify the state machine structure, so the details of the switch statements, etc. would be wrapped in some syntactic sugar (for a text-based language) or would be invisible (being generated as eLua bytecode from a graphical description) in a graphical language. I imagine that the code for actions and condition testing could be specified comfortably in Lua itself. > > For a good example of how clean an equivalent embedded HTTP server > > demo can be (using the lwIP TCP/IP stack, which also uses callbacks > > like uIP), look at http://www.state-machine.com/lwip/ where Miro > > Samek has provided an example program that includes lwIP, an HTTP > > server with SSI and CGI capabilities, and the "Dining Philosopher > > Problem" demo (which is a multi-threaded simulation). > > > > I have ported the QP architecture (described at > > http://www.state-machine.com) to Ruby and Squeak Smalltalk, and have > > used it in C programs in embedded systems; I've found it to be > > very lightweight and a good way to write embedded programs. I would > > probably use a very similar scheme for eLua. > > > > What do you think? > > > I think this is a great idea for many applications, but not all of > them. All in all, I'd be very interested to see something like this > running in eLua :) One note though: QP seems to be GLPed, so > not compatible with eLua (which is mostly MIT/BSD). So a direct port > might not be possible. I would not be making a "derived work" (i.e. not doing a "port"), merely using the ideas and structure of QP. For instance, my Ruby implementation uses a more Ruby-friendly method for finding the least-common ancestor of two states (needed for calling exit and entry handlers during a transition). Ned Konz _______________________________________________ eLua-dev mailing list [hidden email] https://lists.berlios.de/mailman/listinfo/elua-dev |
|
|
In reply to this post by BogdanM
On Mon, Jun 21, 2010 at 4:31 AM, Bogdan Marinescu
<[hidden email]> wrote: > Hi, > > On Mon, Jun 21, 2010 at 4:04 AM, Ned Konz <[hidden email]> > wrote: >> >> Hello folks, >> >> I'd like to introduce myself. I'm new to Lua, but have been doing embedded >> system work since 1975 (on the MC6800 microprocessor). I'm now using >> Cortex-M3 processors, and have been thinking once more about the usefulness >> of a VM on such a chip. > > Nice to meet you. > >> >> My own interests don't necessarily include a self-hosted language system >> (though it's nice to have, I don't know how much space the compiler >> requires); I'd prefer to spend the flash on system capabilities and extended >> domain-specific vocabularies. > > Roughly, you can do fine with eLua in about 200k of Flash on Cortex-M3, or > even less. tt depends on the components you choose to include in your build. > I'm much more worried about RAM than Flash most of the time. > >> The eLua VM looks attractive as a dynamic runtime engine, though. > > It's definitely the most atractive one I found. There is also a very good > (and free) BASIC variant for micros, with source code and excellent > documentation (the VM itself is documented quite nicely) at > http://www.compuphase.com/pawn/pawn.htm. Yes, I know we're talking about > BASIC :), but the VM was quite OK from what I can remember, and the > documentation is outstanding. > >> >> However, the runtime architecture that eLua currently uses (which AFAICT >> is roughly the same as Lua on the desktop) has a several limitations that >> I'd like to change: >> >> * No way to do interesting work in Lua from interrupt handlers. Unless you >> want to use the peripherals the way the driver writer had provided for, >> you're out of luck. > > We'll have interrupt support in eLua soon, although it won't be without > limitations, as the Lua VM is not interruptible. I plan to implement it > using standard Lua hooks (I actually have some proof on concept code that > works quite nicely), which will be a huge problem if the VM remains stuck in > a blocking function (I/O for example). But I don't know of a better way of > doing this, at least not without using an OS with threading support. Well, we can't interrupt the VM mid-instruction, but we can use the hooks to check for pending "interrupts" after each VM instruction. So, as Bogdan says, blocking I/O would be a problem. One would have to take care to avoid blocking operations if an interrupt-driven operation needs to interact with the VM. One certainly has freedom to do whatever one wants outside the VM. So, interrupts could certainly use the platform APIs to do anything else that doesn't require immediately changing the VM state, and some sort of queueing could be used if the VM needs notification. I suspect this sort of thing could get very complicated very fast, but there could be a lot of power in being able to define C or Lua based interrupt handlers from Lua. > >> >> * The Big Loop structure. The (Big Loop) + (interrupt handlers) scheme has >> proven to be inflexible and a poor match for the mostly event-driven needs >> of embedded systems, especially where concurrency is required. Existing >> threading schemes and primitives (especially semaphores!) are difficult to >> write reliable code with. >> >> A demonstration of the difficulties of this structure can be seen with the >> uIP demo code; uIP generates event notifications via callbacks, but the eLua >> HTTP server application is itself blocking in a Big Loop. So the "impedance >> mismatch" between the lightweight event notifications from uIP and the >> blocking socket interface in the server requires extra code and would be >> hard to multi-thread. > > The eLua HTTP server is something hand-crafted in half a week-end, so not > really a good reference :) It can be rewritten quite nicely using coroutines > (once the network support in eLua gets better itself, because now it's not > really usable for handling multiple connections). > >> What I'd like to do instead is to implement a system using the eLua VM >> that does this: >> >> * Allows for eLua peripheral control at a level equivalent to that >> currently done in C code. Once you get past the simple configuration of I/O >> ports, the complexity and richness of existing microcontroller peripherals >> makes it hard to fully use the capabilities of the chip. For instance, I can >> set up the STM32F10x peripherals so that the timer's input capture event >> triggers a DMA from a GPIO port to memory, at the same time that I'm using >> another DMA channel to transfer bytes from the I2C receive into memory, with >> interrupts at the beginning of each I2C transfer. It would be hard if not >> impossible to do that in eLua the way it stands now (I think; I'd love to >> hear that this would be possible to do). > > It's not hard, it's just not imlemented yet. You'd need at least two things > to implement the above: > > 1. interrupt support directly in eLua (and we'll have this soon, as > previously stated) > 2. a Lua module that implements the CPU specific functions needed for DMA, > I2C and whatever else. We don't have that, but they can certainly be > written, and they are perfectly inline with eLua's current architecture > (they are called "platform specific modules" in the official documentation). > We try to focus on the core eLua code + generic modules most of the time, > and let people write their own platform specific modules for their platform > of choice when there's a need for that. If I'm thinking correctly here, this adds another level to the model I was talking about above where in addition there are platform specific types of interrupt responses that might not even require executing code when the interrupt is triggered, since the pre-configured trigger will allow handling entirely by the peripheral. I think in this case, while we could probably provide some generic ability to do things like this, is probably best suited to platform-specific modules. We could always go the route of using hardware supported features like this on some platforms and then falling back to C code on others where they don't have this sort of triggering built-in, but I think there's a lot of potential complexity there as well. > >> >> * Allows eLua programs to be structured as one or more hierarchical state >> machines, each with its own runtime context and event queue. >> >> * Responds to interrupts by packaging up the minimal context (at least >> which interrupt occurred, and perhaps which of several interrupt sources >> caused it) into an event and delivering it to (subscribed or all) state >> machines. >> >> * Does cooperative multi-tasking by using a run-to-completion scheme: each >> state machine that is ready to run (i.e. has an event ready to react to) >> gets its current state's handler routine run with the next event; these are >> not preempted but rather do their work and return. > > You mean cooperative multi-tasking at coroutine level? Or with an OS? If you > mean multi-tasking at coroutine level, what you wrote above can probably be > implemented directly in eLua (+interrupt handlers), without any need to > modify the VM or any other part of the code. Please let me know if I'm > missing something here :) > >> >> It seems to me that this would not require any kind of thread support in >> eLua itself (as the state machine executive is itself a single-threaded >> program), and that it would better match the needs of embedded systems. > > I'm hesitant in naming a development methodology "better" than another in > most cases. An event+state machine based model (as they are used together > more often than not) is a great thing for particular classes of applications > (protocol implementations being the first example that comes to mind), but > it's hard to generalize this. Some applications are perfectly happy with the > Big Loop structure (thanks, I didn't know this is the official name :) ), > others can do all their work only in interrupt handlers without having to > worry much about synchronization, and so on. > >> >> For a good example of how clean an equivalent embedded HTTP server demo >> can be (using the lwIP TCP/IP stack, which also uses callbacks like uIP), >> look at http://www.state-machine.com/lwip/ where Miro Samek has provided an >> example program that includes lwIP, an HTTP server with SSI and CGI >> capabilities, and the "Dining Philosopher Problem" demo (which is a >> multi-threaded simulation). >> >> I have ported the QP architecture (described at >> http://www.state-machine.com) to Ruby and Squeak Smalltalk, and have used it >> in C programs in embedded systems; I've found it to be very lightweight and >> a good way to write embedded programs. I would probably use a very similar >> scheme for eLua. >> >> What do you think? > > I think this is a great idea for many applications, but not all of them. All > in all, I'd be very interested to see something like this running in eLua :) > One note though: QP seems to be GLPed, so not compatible with eLua (which is > mostly MIT/BSD). So a direct port might not be possible. I think we could certainly support this type of development model, the specific implementation may be strongly influenced by proposals and/or code :-) > > Best, > Bogdan > > > _______________________________________________ > eLua-dev mailing list > [hidden email] > https://lists.berlios.de/mailman/listinfo/elua-dev > > -- James Snyder Biomedical Engineering Northwestern University [hidden email] PGP: http://fanplastic.org/key.txt Phone: (847) 448-0386 _______________________________________________ eLua-dev mailing list [hidden email] https://lists.berlios.de/mailman/listinfo/elua-dev |
|
|
In reply to this post by Ned Konz-2
True, and what I had in mind is to put the interrupt ID in a queue (maybe even a heap, which would be a nice and fast way to implement interrupt priorities, although I'm not sure about this yet), set a Lua hook and then let the Lua handler take over :) Of course, a mechanism can (and will) be imagined that allows both C and Lua code to use interrupt handlers in the way described above. So the job of the interrupt handler is to package up into an event Precisely. and then publish or post that event object for consumption by the state ... or simply let the C/Lua interrupt handler take over. This way, the interrupts are kept fast and short, and won't need to be Sure, this is exactly what I had in mind too, save the part with the state machine (which is optional in my design). > The eLua HTTP server is something hand-crafted in half a week-end, so There isn't any difference between eLua coroutines regular Lua coroutines. They are implemented and behave exactly the same.
It is, via the CPU module: http://www.eluaproject.net/en_refman_gen_cpu.html It's actually a quite dangerous module that you can use to access the whole memory space freely :) But this does let you access all of the CPU's memory mapped registers as a side effect. And it has a nice and simple way to define register names (as described in the link above), which makes one's life a bit easier.
Right now, we don't even need boxed pointers. A 32 bit value is enough to represent any address for all the targets currently supported by eLua. What you said makes sense for Harvard architectures, but we're not using these yet. Well, technically speaking the ARM966E-S (STR9) is a Harvard architecture at its core, but it uses some clever tricks to mask this from the rest of the system, effectively exposing an unified memory space. I'll worry about Harvard architectures when I meet one that is a good candidate for eLua :) Also, all of the targets used by eLua have memory mapped registers, so pointers represented as 32 bit values work here too. This allows for peripheral handling code to be written in Lua itself, Sure, you can already do this.
They don't _need_ to be in C, this is purely a matter of aesthetics (and a problem of maintenance as a side effect). This call: stm32.dma.enable_channel( 1 ) is much easier to understand than this (not actual code, but it illustrates my point): cpu.w32( cpu.DMA_CHANNEL1_BASE, xxx ) cpu.w32( cpu.DMA_CHANNEL1_STATE, yyy ) ......................... This is probably the only reason we're writing platform specific modules :) Otherwise one could in theory do absolutely everything from eLua with the cpu module.
Even for architectures without memory mapped registers, I think a 32-bit integer would be enough to represent this. Surely this becomes a question of type safety, but whoever handles low level stuff like this generally knows to be careful :) Though it may also be helpful to separately manage event objects (so as You mean bypassing Lua's GC mechanism? Sure, this is OK as long as you remember to deallocate all your data structures manually. I'm proposing something even simpler than coroutines: just subroutines. Thanks, I'm quite familiar with state machines myself and I fully understand your point. I believe this can be implemented directly in Lua, or perharps in a specialized C state machine module, maybe using a simple object model to expose a nice interface, without any need to modify the core itself.
I wrote several simple applications like this (all on 8 and 16 bit MCUs), such as data loggers, current converters, smart terminals, and even some basic DSP blocks. Basically, they would all block in a while(1) in main(), waiting to process some data. Sure, the CPU spends all the time in this loop, but the point is that sometimes this is exactly what you need it to do :) Adding a state machine in this case would've meant writing unnecessary complex code. Obviously this isn't applicable for more complex applications. I am not concerned here with considerations of programmer convenience or I understand your point now, I think. You want to use eLua for an environment based completely on state machines. This is actually very cool, and you won't hear me complaining about it :) I imagine that the code for actions and condition testing could be Definitely. You don't need any other domain-specific language, Lua is more than capable to do the job.
Then by all means, please go for it. I'm 100% positive that having a fully defined state-machine oriented development model in eLua would make a lot of people happy. Best, Bogdan _______________________________________________ eLua-dev mailing list [hidden email] https://lists.berlios.de/mailman/listinfo/elua-dev |
«
Return to eLua Development
|
1 view|%1 views
| Free forum by Nabble | Edit this page |