eLua Development

extreme small

Classic

List

Threaded

9 messages Options

Karoly Molnar

extreme small

Hi,

Did anyone try to port eLua to the XMOS architecture? It would be good to exchange experiences and ideas. I have some thoughts on how to overcome the 64K memory limitation but am not sure at all if this would be finally doable.

For anyone interested but not knowing about XMOS (www.xmos.com): It is running on 4 or 500MHz, single, dual or quad core, 8 real time threads per core, so it is fairly fast. Dev environment is free and low cost EVBs with JTAG debugging are available.

But there is only 64K dedicated memory (RAM, actually) available for each core :-S (256K for quad core). It is possible to extend this with external SRAM but only by software emulation of data and address lines; so the extension would be slow, possibly enough for the script itself but not for the engine.

Regards

Karoly

_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

Martin Guy

Re: extreme small

On 24 July 2012 22:19, Karoly Molnar <[hidden email]> wrote:
> For anyone interested but not knowing about XMOS (www.xmos.com): It is
> running on 4 or 500MHz, single, dual or quad core, 8 real time threads per
> core, so it is fairly fast. Dev environment is free and low cost EVBs with
> JTAG debugging are available.

> But there is only 64K dedicated memory (RAM, actually) available for each
> core :-S (256K for quad core). It is possible to extend this with external
> SRAM but only by software emulation of data and address lines; so the
> extension would be slow, possibly enough for the script itself but not for
> the engine.

Elua is fine in 64KB, low RAM usage being one of its strong points,
but is a single-threaded interpreter, so you would only get 1/8th (or
16th or 32nd) of the available CPU power.

M
_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

Karoly Molnar

Re: extreme small

Hi Martin,

Thanks, this is great news! Do you know of any hints, git repository or any other information to help me to see where to cut the code?
I am evaluating now on the MBED platform first because a port already exists and my goal is to achieve the 64K limit first on this board and then migrate to XMOS. On the MBED with the standard settings I was able to go down to 120K. This is still twice as much as desired.
I am thinking about replacing the ROMFS with a const array and cut the part that is handling stdio first to see how much i would gain. Is it also possible to disable / remove the text parser so only the bytecode execution engine remains in the eLua binary?

Regards
Karoly

> Date: Wed, 25 Jul 2012 11:37:56 +0200

> From: [hidden email]
> To: [hidden email]
> Subject: Re: [eLua-dev] extreme small
>
> On 24 July 2012 22:19, Karoly Molnar <[hidden email]> wrote:
> > For anyone interested but not knowing about XMOS (www.xmos.com): It is
> > running on 4 or 500MHz, single, dual or quad core, 8 real time threads per
> > core, so it is fairly fast. Dev environment is free and low cost EVBs with
> > JTAG debugging are available.
>
> > But there is only 64K dedicated memory (RAM, actually) available for each
> > core :-S (256K for quad core). It is possible to extend this with external
> > SRAM but only by software emulation of data and address lines; so the
> > extension would be slow, possibly enough for the script itself but not for
> > the engine.
>
> Elua is fine in 64KB, low RAM usage being one of its strong points,
> but is a single-threaded interpreter, so you would only get 1/8th (or
> 16th or 32nd) of the available CPU power.
>
> M
> _______________________________________________
> eLua-dev mailing list
> [hidden email]
> https://lists.berlios.de/mailman/listinfo/elua-dev

_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

Martin Guy

Re: extreme small

On 25 July 2012 13:44, Karoly Molnar <[hidden email]> wrote:
> Do you know of any hints, git repository or any
> other information to help me to see where to cut the code?
> On the MBED with the standard settings I was able to go down to
> 120K. This is still twice as much as desired.

I've done a lot of code size reduction hacks, for which the patches
are on the "mizar32" branch.
This was for AVR32 CPU and barely fits in 120K with just uart, pio and
tmr modules.
The code size reduction hacks made enough space to enable all working modules.

On a different CPU, of course. you will get better or worse code size.
Using the original Thumb instructions instead of ARM instructions used
to save about 30% code size with GCC, while I gather that Thumb2,
which the MBED's Cortex-M3 has, does slightly better *and* maintains
the speed of ARM instructions.

It also depends a lot on which compiler you use. For example, using
-Os LLVM for x86 generates *much* smaller code than GCC. When I
measured it, it was something like 170K compared to 250K. For ARM,
however, the situation was the reverse: LLVM generated much larger
code than GCC, again with both compilers tuned for minimum code size.
I haven't measured the proprietary (Keil) compilers, and have no idea
about XMOS's code density of relative compiler performance. I guess
you will have to experiment (and your measured results would be
interesting)

> Is it also
> possible to disable / remove the text parser so only the bytecode execution
> engine remains in the eLua binary?

That is another option, yes, which should also help quite a lot, but
is not one I pursued.

M
_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

Tim Van Der Hulst

Re: extreme small

In reply to this post by Martin Guy

Martin,

Some advise if you please. I am using STM32 (64K SRAM) and my project is built around 5 modules plus one additional module per sensor (in our case its soil moisture + weather station sensors (wind direction,rain,etc) which equates in KB to (20+6+8+2+2+ approx 1 K per sensor), Roughly 40K. With precompiling several of the larger modules I can just manage to load the code from MMC and it runs fine day and night. If I put the modules on the ROM it seems to use about a 1/3 less memory however these modules are still a work in progress so even when we have some units in the field I intend to remotely upgrade them hence the only code (autorun.lua) I want on the ROM is a small bootstrap routine to load some config off the MMC and then kick off a script running on the MMC.

My code is completely unoptimized so I am not worried as I can always give it a good squeeze if needed but I would rather not have to if it could be avoided. RAD, rather than performance being the name of the game at this stage..

Any advice appreciated,

TiM

----- Original Message -----
From: "Martin Guy" <[hidden email]>
To: "eLua Users and Development List (www.eluaproject.net)" <[hidden email]>
Sent: Wednesday, July 25, 2012 9:37:56 PM
Subject: Re: [eLua-dev] extreme small

On 24 July 2012 22:19, Karoly Molnar <[hidden email]> wrote:
> For anyone interested but not knowing about XMOS (www.xmos.com): It is
> running on 4 or 500MHz, single, dual or quad core, 8 real time threads per
> core, so it is fairly fast. Dev environment is free and low cost EVBs with
> JTAG debugging are available.

> But there is only 64K dedicated memory (RAM, actually) available for each
> core :-S (256K for quad core). It is possible to extend this with external
> SRAM but only by software emulation of data and address lines; so the
> extension would be slow, possibly enough for the script itself but not for
> the engine.

Elua is fine in 64KB, low RAM usage being one of its strong points,
but is a single-threaded interpreter, so you would only get 1/8th (or
16th or 32nd) of the available CPU power.

M
_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev
_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

BogdanM

Re: extreme small

On Thu, Jul 26, 2012 at 12:04 AM, Tim Van Der Hulst
<[hidden email]> wrote:
> Martin,
>
> Some advise if you please. I am using STM32 (64K SRAM) and my project is built around 5 modules plus one additional module per sensor (in our case its soil moisture + weather station sensors (wind direction,rain,etc) which equates in KB to (20+6+8+2+2+ approx 1 K per sensor), Roughly 40K.

That sounds like an extremely cool application.

> With precompiling several of the larger modules I can just manage to load the code from MMC and it runs fine day and night. If I put the modules on the ROM it seems to use about a 1/3 less memory however these modules are still a work in progress so even when we have some units in the field I intend to remotely upgrade them hence the only code (autorun.lua) I want on the ROM is a small bootstrap routine to load some config off the MMC and then kick off a script running on the MMC.

Have you tried our new WOFS
(http://www.eluaproject.net/doc/master/en_arch_wofs.html) ? It seems
to cover your needs nd it works currently on the STM32F1 subfamily
(can be easily extended for other STM32 subfamilies)

>
> My code is completely unoptimized so I am not worried as I can always give it a good squeeze if needed but I would rather not have to if it could be avoided. RAD, rather than performance being the name of the game at this stage..

I'm sorry, I'm not sure I understand what exactly you need help for.
What's the problem at the moment? Too much flash? Too much RAM?
Something else?

Thanks,
Bogdan

>
> Any advice appreciated,
>
> TiM
>
>
> ----- Original Message -----
> From: "Martin Guy" <[hidden email]>
> To: "eLua Users and Development List (www.eluaproject.net)" <[hidden email]>
> Sent: Wednesday, July 25, 2012 9:37:56 PM
> Subject: Re: [eLua-dev] extreme small
>
> On 24 July 2012 22:19, Karoly Molnar <[hidden email]> wrote:
>> For anyone interested but not knowing about XMOS (www.xmos.com): It is
>> running on 4 or 500MHz, single, dual or quad core, 8 real time threads per
>> core, so it is fairly fast. Dev environment is free and low cost EVBs with
>> JTAG debugging are available.
>
>> But there is only 64K dedicated memory (RAM, actually) available for each
>> core :-S (256K for quad core). It is possible to extend this with external
>> SRAM but only by software emulation of data and address lines; so the
>> extension would be slow, possibly enough for the script itself but not for
>> the engine.
>
> Elua is fine in 64KB, low RAM usage being one of its strong points,
> but is a single-threaded interpreter, so you would only get 1/8th (or
> 16th or 32nd) of the available CPU power.
>
> M
> _______________________________________________
> eLua-dev mailing list
> [hidden email]
> https://lists.berlios.de/mailman/listinfo/elua-dev
> _______________________________________________
> eLua-dev mailing list
> [hidden email]
> https://lists.berlios.de/mailman/listinfo/elua-dev

_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

Tim Van Der Hulst

Re: extreme small

Hi Bogdan,

Main problem is am getting out of memory errors. I assumed the VM would be smart enough to swap out unused memory so it didnt need to keep all my code in memory but that seems like an oversight on my part? I would happily trade CPU cycles for extra memory. 99% of the time my app is going to be in sleep mode except when logging so speed is not important.

I am also confused by the WOFS as it seems like its a one shot write to Flash and then when you write it again, does it update the same memory space or occupy more memory, what happens if you write too often and run out of space, then you would have to flash the firmware again no?

Cheers,

Martin, apologies, hijacked another/existing thread, sorry old habbits die hard :(

TiM

----- Original Message -----
From: "Bogdan Marinescu" <[hidden email]>
To: "eLua Users and Development List (www.eluaproject.net)" <[hidden email]>
Sent: Friday, July 27, 2012 8:33:01 AM
Subject: Re: [eLua-dev] extreme small

On Thu, Jul 26, 2012 at 12:04 AM, Tim Van Der Hulst
<[hidden email]> wrote:
> Martin,
>
> Some advise if you please. I am using STM32 (64K SRAM) and my project is built around 5 modules plus one additional module per sensor (in our case its soil moisture + weather station sensors (wind direction,rain,etc) which equates in KB to (20+6+8+2+2+ approx 1 K per sensor), Roughly 40K.

That sounds like an extremely cool application.

> With precompiling several of the larger modules I can just manage to load the code from MMC and it runs fine day and night. If I put the modules on the ROM it seems to use about a 1/3 less memory however these modules are still a work in progress so even when we have some units in the field I intend to remotely upgrade them hence the only code (autorun.lua) I want on the ROM is a small bootstrap routine to load some config off the MMC and then kick off a script running on the MMC.

Have you tried our new WOFS
(http://www.eluaproject.net/doc/master/en_arch_wofs.html) ? It seems
to cover your needs nd it works currently on the STM32F1 subfamily
(can be easily extended for other STM32 subfamilies)

>
> My code is completely unoptimized so I am not worried as I can always give it a good squeeze if needed but I would rather not have to if it could be avoided. RAD, rather than performance being the name of the game at this stage..

I'm sorry, I'm not sure I understand what exactly you need help for.
What's the problem at the moment? Too much flash? Too much RAM?
Something else?

Thanks,
Bogdan

_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev
_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

BogdanM

Re: extreme small

Hi,

On Fri, Jul 27, 2012 at 12:30 AM, Tim Van Der Hulst
<[hidden email]> wrote:
> Hi Bogdan,
>
> Main problem is am getting out of memory errors. I assumed the VM would be smart enough to swap out unused memory so it didnt need to keep all my code in memory but that seems like an oversight on my part?

This probably needs some clarifications. What you're getting are out
of memory errors for RAM. On the Lua side, this can come from:

- the code. With the latest eLua patches to Lua, you can execute
_bytecode_ (only bytecode!) directly from the Flash, without having to
load it to RAM. To do that, you need to cross-compile your Lua code on
the host side (http://www.eluaproject.net/doc/master/en_using.html#cross)
and upload the resulting bytecode file to your board using either
ROMFS or WOFS. Please let me know if you need more details on this,
I'm a bit busy at the moment and I can't get into much details.
- the data. Each type in Lua is allocated on the heap (malloc and
friends) (yes, even numbers). Unfortunately, this is something we
can't really control much (although we have some patches here too,
their effect isn't nearly as dramatic as the above "bytecode from
Flash" optimization). I'm afraid the VM is unable to swap anything in
the absence of a hardware MMU, which we don't have. You can also try
to change the EGC patch with aggresive triggering, more about the
subject here:

http://www.eluaproject.net/doc/master/en_elua_egc.html

(the most aggresive mode means enabling EGC_ON_ALLOC_FAILURE +
EGC_ON_MEM_LIMIT + EGC_ALWAYS with a 'limit' parameter that suits your
needs).

> I would happily trade CPU cycles for extra memory. 99% of the time my app is going to be in sleep mode except when logging so speed is not important.
>
> I am also confused by the WOFS as it seems like its a one shot write to Flash and then when you write it again, does it update the same memory space or occupy more memory,

More memory. This is transparent though.

> what happens if you write too often and run out of space, then you would have to flash the firmware again no?

No, you can use "wofmt" from the shell and this will clear the WOFS
(and ONLY the WOFS) to a blank state (extremely similar to a 'format'
(DOS) or 'mkfs' (UNIX) command). This is documented here:

http://www.eluaproject.net/doc/master/en_arch_wofs.html

Best,
Bogdan

>
> Cheers,
>
> Martin, apologies, hijacked another/existing thread, sorry old habbits die hard :(
>
> TiM
>
>
>
> ----- Original Message -----
> From: "Bogdan Marinescu" <[hidden email]>
> To: "eLua Users and Development List (www.eluaproject.net)" <[hidden email]>
> Sent: Friday, July 27, 2012 8:33:01 AM
> Subject: Re: [eLua-dev] extreme small
>
> On Thu, Jul 26, 2012 at 12:04 AM, Tim Van Der Hulst
> <[hidden email]> wrote:
>> Martin,
>>
>> Some advise if you please. I am using STM32 (64K SRAM) and my project is built around 5 modules plus one additional module per sensor (in our case its soil moisture + weather station sensors (wind direction,rain,etc) which equates in KB to (20+6+8+2+2+ approx 1 K per sensor), Roughly 40K.
>
> That sounds like an extremely cool application.
>
>> With precompiling several of the larger modules I can just manage to load the code from MMC and it runs fine day and night. If I put the modules on the ROM it seems to use about a 1/3 less memory however these modules are still a work in progress so even when we have some units in the field I intend to remotely upgrade them hence the only code (autorun.lua) I want on the ROM is a small bootstrap routine to load some config off the MMC and then kick off a script running on the MMC.
>
> Have you tried our new WOFS
> (http://www.eluaproject.net/doc/master/en_arch_wofs.html) ? It seems
> to cover your needs nd it works currently on the STM32F1 subfamily
> (can be easily extended for other STM32 subfamilies)
>
>>
>> My code is completely unoptimized so I am not worried as I can always give it a good squeeze if needed but I would rather not have to if it could be avoided. RAD, rather than performance being the name of the game at this stage..
>
> I'm sorry, I'm not sure I understand what exactly you need help for.
> What's the problem at the moment? Too much flash? Too much RAM?
> Something else?
>
> Thanks,
> Bogdan
>
>>
>> Any advice appreciated,
>>
>> TiM
>>
>>
>> ----- Original Message -----
>> From: "Martin Guy" <[hidden email]>
>> To: "eLua Users and Development List (www.eluaproject.net)" <[hidden email]>
>> Sent: Wednesday, July 25, 2012 9:37:56 PM
>> Subject: Re: [eLua-dev] extreme small
>>
>> On 24 July 2012 22:19, Karoly Molnar <[hidden email]> wrote:
>>> For anyone interested but not knowing about XMOS (www.xmos.com): It is
>>> running on 4 or 500MHz, single, dual or quad core, 8 real time threads per
>>> core, so it is fairly fast. Dev environment is free and low cost EVBs with
>>> JTAG debugging are available.
>>
>>> But there is only 64K dedicated memory (RAM, actually) available for each
>>> core :-S (256K for quad core). It is possible to extend this with external
>>> SRAM but only by software emulation of data and address lines; so the
>>> extension would be slow, possibly enough for the script itself but not for
>>> the engine.
>>
>> Elua is fine in 64KB, low RAM usage being one of its strong points,
>> but is a single-threaded interpreter, so you would only get 1/8th (or
>> 16th or 32nd) of the available CPU power.
>>
>> M
>> _______________________________________________
>> eLua-dev mailing list
>> [hidden email]
>> https://lists.berlios.de/mailman/listinfo/elua-dev
>> _______________________________________________
>> eLua-dev mailing list
>> [hidden email]
>> https://lists.berlios.de/mailman/listinfo/elua-dev
> _______________________________________________
> eLua-dev mailing list
> [hidden email]
> https://lists.berlios.de/mailman/listinfo/elua-dev
> _______________________________________________
> eLua-dev mailing list
> [hidden email]
> https://lists.berlios.de/mailman/listinfo/elua-dev

_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

Martin Guy

Re: extreme small

In reply to this post by Tim Van Der Hulst

On 26 July 2012 23:30, Tim Van Der Hulst <[hidden email]> wrote:
> Main problem is am getting out of memory errors. I assumed the VM would be smart enough to swap out unused memory

Elua doesn't have virtual memory (or do you mean virtual machine?).
The memory in this case is the RAM.
There is a Lua construct "weak tables" which obligingly forget their
contents if the interpreter runs out of RAM, often used for caching
the results of some calculation, so you might optimise your app to use
those.

> I would happily trade CPU cycles for extra memory. 99% of the time my app is going to be in sleep mode except when logging so speed is not important.

Of course, one way to fix that is to use a different base board. Of
course I would suggest the Mizar32, which has 64KB of fast RAM and
32MB of SDRAM, so small applications run a bit faster than those that
overflow into the SDRAM; the difference is about a factor of two since
the SDRAM is on a 16-bit bus with 1 wait state so two accesses per
cell, whereas the internal RAM is 32-bit no wait states. But you will
*never* run out of memory :) and can have a full interpreter in the
two models with larger flash.

> Martin, apologies, hijacked another/existing thread, sorry old habbits die hard :(

No worries

M
_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev