Pure Lua multitasking with eLua

[Jorge]

Hi, I announce eLumen, a coroutine based scheduler for multitasking in
eLua: https://github.com/xopxe/eLumen

 From the README:

-------------------------------------------------------------------
This is basically the [Lumen](https://github.com/xopxe/Lumen) scheduler
after being pruned from the less used stuff. There is space for more
pruning, tough.

...

This is tested on a MBED board. To port to another board, the only
change needed is in the sched.get\_time() function. You must provide a
method for reading the time (for example, on MBED it is doing  _return
tmr.read(0)_).

There is a test.lua script (also for MBED). It will blink led1 at 1Hz,
and toggle led2 and led3 with the keys "q" and "w" respectively. It will
also print memory usage, at the script starting time, after loading the
scheduler, after setting up the tasks, and then while running once a second.
-------------------------------------------------------------------

This is not ready yet (must update the docs, for example), but it can be
used already to decide if it is useful for something at all :)


Greetings,

Jorge
_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[gduarte]

Cool :) Congrats!!

2014/1/7 Jorge <[hidden email]>

Hi, I announce eLumen, a coroutine based scheduler for multitasking in eLua: https://github.com/xopxe/eLumen

From the README:

-------------------------------------------------------------------
This is basically the [Lumen](https://github.com/xopxe/Lumen) scheduler after being pruned from the less used stuff. There is space for more pruning, tough.

...

This is tested on a MBED board. To port to another board, the only change needed is in the sched.get\_time() function. You must provide a method for reading the time (for example, on MBED it is doing  _return tmr.read(0)_).

There is a test.lua script (also for MBED). It will blink led1 at 1Hz, and toggle led2 and led3 with the keys "q" and "w" respectively. It will also print memory usage, at the script starting time, after loading the scheduler, after setting up the tasks, and then while running once a second.
-------------------------------------------------------------------

This is not ready yet (must update the docs, for example), but it can be used already to decide if it is useful for something at all :)


Greetings,

Jorge
_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

--

Gabriel Duarte

Linux User #471185
Rio de Janeiro / RJ
http://genericdev.wordpress.com/

_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[BogdanM]

Hi Jorge,

Thanks a lot! That looks indeed very interesting. I'll play with it a bit as soon as I get a chance.

Best,

Bogdan

On Tue, Jan 7, 2014 at 6:48 PM, Jorge <[hidden email]> wrote:

Hi, I announce eLumen, a coroutine based scheduler for multitasking in eLua: https://github.com/xopxe/eLumen

From the README:

-------------------------------------------------------------------
This is basically the [Lumen](https://github.com/xopxe/Lumen) scheduler after being pruned from the less used stuff. There is space for more pruning, tough.

...

This is tested on a MBED board. To port to another board, the only change needed is in the sched.get\_time() function. You must provide a method for reading the time (for example, on MBED it is doing  _return tmr.read(0)_).

There is a test.lua script (also for MBED). It will blink led1 at 1Hz, and toggle led2 and led3 with the keys "q" and "w" respectively. It will also print memory usage, at the script starting time, after loading the scheduler, after setting up the tasks, and then while running once a second.
-------------------------------------------------------------------

This is not ready yet (must update the docs, for example), but it can be used already to decide if it is useful for something at all :)


Greetings,

Jorge
_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[Stefan Heinzmann]

Can you generate signals from within an interrupt handler (written in C)?

 

Cheers

Stefan

 

Gesendet: Dienstag, 07. Januar 2014 um 17:48 Uhr
Von: Jorge <[hidden email]>
An: [hidden email]
Betreff: [eLua-dev] Pure Lua multitasking with eLua

Hi, I announce eLumen, a coroutine based scheduler for multitasking in
eLua: https://github.com/xopxe/eLumen

From the README:

-------------------------------------------------------------------
This is basically the [Lumen](https://github.com/xopxe/Lumen) scheduler
after being pruned from the less used stuff. There is space for more
pruning, tough.

...

This is tested on a MBED board. To port to another board, the only
change needed is in the sched.get\_time() function. You must provide a
method for reading the time (for example, on MBED it is doing _return
tmr.read(0)_).

There is a test.lua script (also for MBED). It will blink led1 at 1Hz,
and toggle led2 and led3 with the keys "q" and "w" respectively. It will
also print memory usage, at the script starting time, after loading the
scheduler, after setting up the tasks, and then while running once a second.
-------------------------------------------------------------------

This is not ready yet (must update the docs, for example), but it can be
used already to decide if it is useful for something at all :)


Greetings,

Jorge
_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[1100F]

Hi Jorge!

It's a big job. Working perfectly in mbed.
need modify the line 56 (sched.lua) => return tmr.read( tmr.SYS_TIMER ) with discovery F407: working perfectly (of course, with modify the pins;-) )

Congratulations Jorge!

Best,
Albert

Le 07/01/2014 17:48, Jorge a écrit :

Hi, I announce eLumen, a coroutine based scheduler for multitasking in eLua: https://github.com/xopxe/eLumen

From the README:

-------------------------------------------------------------------
This is basically the [Lumen](https://github.com/xopxe/Lumen) scheduler after being pruned from the less used stuff. There is space for more pruning, tough.

...

This is tested on a MBED board. To port to another board, the only change needed is in the sched.get\_time() function. You must provide a method for reading the time (for example, on MBED it is doing  _return tmr.read(0)_).

There is a test.lua script (also for MBED). It will blink led1 at 1Hz, and toggle led2 and led3 with the keys "q" and "w" respectively. It will also print memory usage, at the script starting time, after loading the scheduler, after setting up the tasks, and then while running once a second.
-------------------------------------------------------------------

This is not ready yet (must update the docs, for example), but it can be used already to decide if it is useful for something at all :)


Greetings,

Jorge
_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

---
Ce courrier électronique ne contient aucun virus ou logiciel malveillant parce que la protection avast! Antivirus est active.
http://www.avast.com

---

Ce courrier électronique ne contient aucun virus ou logiciel malveillant parce que la protection Antivirus avast! est active.

_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[Jorge]

On 07/01/14 16:38, Stefan Heinzmann wrote:

Can you generate signals from within an interrupt handler (written in C)?

I guess not directly. Only Lumen tasks can emit Lumen signals. IF it is possible for the handler to insert data into a table available in the Lua context, I would have a task periodically iterate such a table and emit signals. For example

sched.run(function()
  while true do
    for event, parameter in pairs(signals_to_emit) do
      sched.emit(event, parameter)
    end
    sched.wait()
  end
end)

As I said, that assuming it is possible to access he Lua environment from a interrupt handler (as must be clear, I go at great lengths to avoid touching C code :) )

The idea for doing IO under Lumen was to use a select/poll like call: something that would block until there's activity, with a setable timeout. If such a glue code can be provided to Lua, that would be another method.

And just in case, Lumen signals aren't well suited for real time stuff. The conext switching is expensive compared with, say, a thread does, as it involves table lookups and traversals and timeout computations and what not. It is ok for "business logic", IO bound, concurrency.

Jorge


_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[Jorge]

On 01/07/2014 05:38 PM, Albert FERNANDES wrote:
> It's a big job. Working perfectly in mbed.

Thanks!

> need modify the line 56 (sched.lua) => return tmr.read( tmr.SYS_TIMER )
> with discovery F407: working perfectly (of course, with modify the pins;-) )

Great, I missed SYS_TIMER because I was looking into 0.8 docs... Updated
to use it, as it works also on my lpc1768.

I also updated and generated the ldoc documentation.

Jorge
_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[Stefan Heinzmann]

The overall architecture I have in mind would work like this:

 

1. Hardware is sitting there waiting for something interesting to happen. This could even be with the CPU stopped for low power consumption, provided that it wakes up when the hardware receives an interrupt.

 

2. Something happens that generates an interrupt and wakes up the CPU. The CPU invokes the interrupt handler, which examines what happened, and decides whether the lua event loop must be signalled.

 

3. If the decision is positive, a Lumen signal is emit, so that after returning from the interrupt handler, the Lumen scheduler runs and finds the signal in the queue.

 

4. The lua code handles the signal, and after finding the signal queue empty it puts the CPU back to sleep mode and goes back to point 1.

 

I pretty much know how to do all this, except for how to create a Lumen signal from within the interrupt handler.

 

Doing this with a periodic task means that I have to wake up the system periodically, even when there's nothing happening. And there would be a handler latency that is dependent on the frequency of the periodic task's invocation. This introduces a tradeoff between frequency and latency. Not very attractive, in my view.

 

Do you think that C library signals could be employed here?

 

I'm not really concerned with extremely short reaction times - I know that Lua has its overheads. I just don't like the idea of having to use active polling, particularly when there can be longer periods of time where nothing is happening, and polling would only waste energy.

 

Cheers

Stefan

 

 

Gesendet: Mittwoch, 08. Januar 2014 um 01:39 Uhr
Von: Jorge <[hidden email]>
An: "eLua Users and Development List (www.eluaproject.net)" <[hidden email]>
Betreff: Re: [eLua-dev] Pure Lua multitasking with eLua

On 07/01/14 16:38, Stefan Heinzmann wrote:

Can you generate signals from within an interrupt handler (written in C)?

I guess not directly. Only Lumen tasks can emit Lumen signals. IF it is possible for the handler to insert data into a table available in the Lua context, I would have a task periodically iterate such a table and emit signals. For example

sched.run(function()
  while true do
    for event, parameter in pairs(signals_to_emit) do
      sched.emit(event, parameter)
    end
    sched.wait()
  end
end)

As I said, that assuming it is possible to access he Lua environment from a interrupt handler (as must be clear, I go at great lengths to avoid touching C code :) )

The idea for doing IO under Lumen was to use a select/poll like call: something that would block until there's activity, with a setable timeout. If such a glue code can be provided to Lua, that would be another method.

And just in case, Lumen signals aren't well suited for real time stuff. The conext switching is expensive compared with, say, a thread does, as it involves table lookups and traversals and timeout computations and what not. It is ok for "business logic", IO bound, concurrency.

Jorge

_______________________________________________ eLua-dev mailing list [hidden email] https://lists.berlios.de/mailman/listinfo/elua-dev

_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[Jorge]

On 01/08/2014 12:43 PM, Stefan Heinzmann wrote:
> The overall architecture I have in mind would work like this:
...
> Doing this with a periodic task means that I have to wake up the system
> periodically, even when there's nothing happening. And there would be a

I see what you are trying to do, and that would be great. Avoid polling
was the reason for writing Lumen in the first place. Under Linux, what
Lua does is to call poll(), which will block until there is something
happening.

The more natural thing to me to implement what you want would be:

1. Have a task as follows:

   sched.run(function ()
     while true do
       local _, timeout = sched.wait()
       -- set a timer to wake me timeout in the future
       -- enable interrupts (timer and anything needed)
       -- go to sleep (low power mode)
       -- i was waked, so check what happened
       --   if on timeout -> do nothing
       --   if some interrupt -> emit signals
     end
   end)

2. have a interrupt handler that would:
        // wake as needed
        // disable interrupts
         // insert events in the lua to be picked by the task above
         // continue on to running lua code

Does that make sense?

Jorge

_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[Fabien Fleutot]

On Tue, Jan 7, 2014 at 7:38 PM, Stefan Heinzmann <[hidden email]> wrote:

Can you generate signals from within an interrupt handler (written in C)?

In Mihini's scheduler, (the original inspiration for [e]lumen), we have a function using Lua's C API to do that. If you want to write your own, beware that you need to manage two different cases, depending on whether the signalling function is called from the OS thread running the lua_State or from another thread.

Of course, it's by no means real-time, but you shouldn't expect otherwise from a collaboratie multithreading system.

--
Fabien.

_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[Stefan Heinzmann]

Yes, I think I understand how your proposal is supposed to work. The problem I'm having with it is that when nothing happens, the timeout will eventually wake up the lua loop, only to find that there's nothing to do, which consumes energy needlessly. In some embedded systems, this can be significant.

 

Furthermore, if you want the lua code to react to the hardware-caused event after some maximum time, you would have to make the timeout value smaller that that maximum time.

 

Effectively, your method uses the hardware interrupt to set a flag, and you use a timer-driven polling loop in lua to check whether the flag has been set. That defeats half of the benefit of an interrupt, and in many cases you could just poll the hardware instead. I think there must be a better way. The interrupt handler needs a way to make the lua code handle the signal immediately after the interrupt handler returns.

 

I acknowledge that you may not be keen on coding this in C, as you want to work in lua. Furthermore, the way how an interrupt handler interacts with the rest of the code is quite system dependent. Linux has its way of dealing with this, but it would be highly useful if something similar were also supported in eLua.

 

Cheers

Stefan

 

 

Gesendet: Mittwoch, 08. Januar 2014 um 16:09 Uhr
Von: Jorge <[hidden email]>
An: [hidden email]
Betreff: Re: [eLua-dev] Pure Lua multitasking with eLua

On 01/08/2014 12:43 PM, Stefan Heinzmann wrote:
> The overall architecture I have in mind would work like this:
...
> Doing this with a periodic task means that I have to wake up the system
> periodically, even when there's nothing happening. And there would be a

I see what you are trying to do, and that would be great. Avoid polling
was the reason for writing Lumen in the first place. Under Linux, what
Lua does is to call poll(), which will block until there is something
happening.

The more natural thing to me to implement what you want would be:

1. Have a task as follows:

sched.run(function ()
while true do
local _, timeout = sched.wait()
-- set a timer to wake me timeout in the future
-- enable interrupts (timer and anything needed)
-- go to sleep (low power mode)
-- i was waked, so check what happened
-- if on timeout -> do nothing
-- if some interrupt -> emit signals
end
end)

2. have a interrupt handler that would:
// wake as needed
// disable interrupts
// insert events in the lua to be picked by the task above
// continue on to running lua code

Does that make sense?

Jorge

_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[Stefan Heinzmann]

Fabien Fleutot wrote:

> In Mihini's scheduler[http://www.eclipse.org/mihini/], (the original inspiration for [e]lumen), we have a function using Lua's C API to do that. If > you want to write your own, beware that you need to manage two different cases, depending on whether the signalling function is called from the OS > thread running the lua_State or from another thread.
 
In the eLua system there wouldn't be any other threads (unless the system was built with FreeRTOS). It would be just the "main thread" running the lua interpreter, and interrupt handlers that are invoked occasionally in reaction to hardware events.
 
> Of course, it's by no means real-time, but you shouldn't expect otherwise from a collaboratie multithreading system.

I am fully aware of that. If the lua code is busy handling some previous event, the reaction to the next event has to wait until the processor is done with it. That is quite acceptable. One could still do the truly time-critical stuff within the interrupt handlers if necessary.

Essentially, a function needs to be provided that can be called safely from within an interrupt handler, which does the following:

1. Put a Lumen signal into the signal queue
2. If the lua code is waiting on the signal queue, release it, so that after returning from the interrupt handler, it will recheck the queue and find the new signal in there. In other words, arrange the scheduler to be run immediately after returning from the interrupt handler.

If this could be done in a way that is compatible with standard APIs, such as select(), even better.

Unfortunately, I am not familiar enough with the innards of the lua interpreter to tell how difficult this is.

Cheers
Stefan
_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[Jorge]

On 01/08/2014 01:33 PM, Stefan Heinzmann wrote:
> Yes, I think I understand how your proposal is supposed to work. The
> problem I'm having with it is that when nothing happens, the timeout
> will eventually wake up the lua loop, only to find that there's nothing
> to do, which consumes energy needlessly. In some embedded systems, this
> can be significant.

The purpose of the timeout is to wake-up tasks that went to sleep for a
set amount of time (like the blinker task in the test.lua), or are
waiting for signals with a timeout set. The call to sched.wait() returns
with a value that is the time you have available for your task before
some other task needs to do something. So if you have some other task
busy-looping this timeout value will be 0, but it they are idle
(sleeping or waiting) the timeout will be a positive number you can
safely use for whatever you want, for example go to low power mode.

> I think there must be a
> better way. The interrupt handler needs a way to make the lua code
> handle the signal immediately after the interrupt handler returns.

That is achieve by waking the lua environment, which will continue on
running and proceed to pick the data and emit signals.

> I acknowledge that you may not be keen on coding this in C, as you want
> to work in lua. Furthermore, the way how an interrupt handler interacts
> with the rest of the code is quite system dependent.

That's what my proposal does: when there are no lumen tasks to run (they
all are waiting for something), go to sleep mode, and when that sleep is
interrupted, immediately pick from lua and signal whatever happened.
Perhaps a more expanded pseudocode can explain it better:

sched.run(function ()
   while true do
     local _, timeout = sched.wait()

     if timeout==0 then
       -- some task is busy-looping, so must return straight away:
       -- read the data of interest, and emit sched.signal() as needed
     else
       -- set a timer to wake me timeout in the future
       -- enable interrupts (timer and anything needed)
       -- go to sleep (low power mode)
       -- i was waken!
       if not waken_on_timeout then -- there was a data interrupt
         -- read the data of interest, and emit sched.signal() as needed
       end
     end

   end
end)


If this works, it will be great! :)

Jorge


_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[Brent Picasso]

Been following this thread a bit.  My recommendation would be to put eLua in the context of an existing RTOS (FreeRTOS or chibiOS/RT) and wrap the threading primitives in Lua statements. That way you can spin up new tasks as needed, getting the benefits of a real underlying preemptive scheduler.

-Brent

On Wed, Jan 8, 2014 at 8:02 AM, Stefan Heinzmann <[hidden email]> wrote:

Fabien Fleutot wrote:

> In Mihini's scheduler[http://www.eclipse.org/mihini/], (the original inspiration for [e]lumen), we have a function using Lua's C API to do that. If > you want to write your own, beware that you need to manage two different cases, depending on whether the signalling function is called from the OS > thread running the lua_State or from another thread.
 
In the eLua system there wouldn't be any other threads (unless the system was built with FreeRTOS). It would be just the "main thread" running the lua interpreter, and interrupt handlers that are invoked occasionally in reaction to hardware events.

 
> Of course, it's by no means real-time, but you shouldn't expect otherwise from a collaboratie multithreading system.

I am fully aware of that. If the lua code is busy handling some previous event, the reaction to the next event has to wait until the processor is done with it. That is quite acceptable. One could still do the truly time-critical stuff within the interrupt handlers if necessary.

Essentially, a function needs to be provided that can be called safely from within an interrupt handler, which does the following:

1. Put a Lumen signal into the signal queue
2. If the lua code is waiting on the signal queue, release it, so that after returning from the interrupt handler, it will recheck the queue and find the new signal in there. In other words, arrange the scheduler to be run immediately after returning from the interrupt handler.

If this could be done in a way that is compatible with standard APIs, such as select(), even better.

Unfortunately, I am not familiar enough with the innards of the lua interpreter to tell how difficult this is.

Cheers
Stefan

_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

--
Brent Picasso

Autosport Labs

Technology for Race and Street

_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev