Fwd: [GitHub] Decrease AVR32 stack size from 8192 to 2048, the same as the other small [elua/elua 9a3abb6]

[BogdanM]

---------- Forwarded message ----------
From: martinwguy <[hidden email]>
Date: Tue, Mar 15, 2011 at 9:45 PM
Subject: Re: [GitHub] Decrease AVR32 stack size from 8192 to 2048, the same as the other small [elua/elua 9a3abb6]


Hi
  Can we follow the usual good practise of keeping technical
discussions on public lists?

Sure, my bad. Forwarding to the list.

Best,

Bogdan

On 15 March 2011 20:24, bogdanm

<[hidden email]>
wrote:

> Hi,
>
> On Tue, Mar 15, 2011 at 9:14 PM, martinwguy <

> [hidden email]>wrote:

>
>> On 15 March 2011 08:32, bogdanm
>> <[hidden email]>
>> wrote:
>> > The Lua parser is definitely not going to like this. Setting the stack to
>> anything less than 4K is asking for stack overflow trouble in my experience.
>>
>> Thanks. Do you have an example that demonstrates usage of stack > 2K?
>> I don't have any real figures or is this just a feeling?
>>
>
> The romfs/ directory used to contain a version of life.lua that I used
> precisely for stack-related tests. I'm not sure where it is now :) You can
> probably find it in older commits.
>
>
>> Me, I'm just following the setting used in almost all the other platforms.
>>
>
> Sure. But the reason I'm trying to keep it small in other platforms (even
> with the risk of stack overflows) is the lack of memory. Mizar32 doesn't
> have this problem, this is why I set the stack to a value that gives me
> piece of mind.
> At some point I'd like to take a deeper look into this issue. I don't really
> understand the need for this much stack especially since I manually moved
> many of the Lua parser internal data structures from heap to stack. Or maybe
> the stack problem moved from the parser to some other part of the code. The
> Lua call stack is dynamically allocated just like about everything else
> related to Lua, so the problem must still exist in the C code somewhere.
>
> Best,
> Bogdan
>

> --

> Reply to this email directly or view it on GitHub:

> https://github.com/elua/elua/commit/9a3abb6b1025814875a73753a11ceb1401f67d82#commitcomment-302840

>

--
Reply to this email directly or view it on GitHub:

https://github.com/elua/elua/commit/9a3abb6b1025814875a73753a11ceb1401f67d82#commitcomment-302863

_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[Martin Guy]

On 15 March 2011 20:51, Bogdan Marinescu <[hidden email]> wrote:

>>> > The Lua parser is definitely not going to like this. Setting the stack
>>> > to
>>> anything less than 4K is asking for stack overflow trouble in my
>>> experience.
>>>
>>> Thanks. Do you have an example that demonstrates usage of stack > 2K?
>>> I don't have any real figures or is this just a feeling?
>>
>> The romfs/ directory used to contain a version of life.lua that I used
>> precisely for stack-related tests. I'm not sure where it is now :) You can
>> probably find it in older commits.

Confirmed. With stack of 2048 or 3072, life.lua either doesn't parse
or doesn't run. 4096 seems to work for life.lua
However, if it's a recursive descent parser, the amount of C stack
that a piece of Lua code could require is potentially infinite.

>> I don't really
>> understand the need for this much stack especially since I manually moved
>> many of the Lua parser internal data structures from heap to stack.

From Lua 5 onwards, calls from Lua functions to Lua functions are
"stackless", i.e. do not consume any C stack.  There is just a hard
limit of 20000 nested calls to trap infinite recursion.
C stack is used in parsing (as you say) and, I gather from the
following message, in calls from Lua to C functions that call back
into Lua (sort, pcall...) but those should not be infinitely deep.
http://comments.gmane.org/gmane.comp.lang.lua.general/6376

>> Sure. But the reason I'm trying to keep it small in other platforms (even
>> with the risk of stack overflows) is the lack of memory. Mizar32 doesn't
>> have this problem

With the speed difference of a factor of two between SRAM+SDRAM and
only SDRAM for the Lua heap, I'd say it does.

I confess I am confused by the assertion that simple and newlib can't
use multiple memory blocks.
Experience using SRAM, SRAM+SDRAM and just SDRAM, with simple and newlib is that
- SRAM+SDRAM is markedly faster than just SDRAM: about 10% for 20
generations of life.
- You can allocate and access 100K of junk in a table when using
SRAM+SDRAM in both allocators
and inspection of the code seems to show that the newlib allocator in
newlib/stubs.c, using newlib _sbrk_r is redefined to return memory
from all blocks listed in MEM_{START,END}_ADDRESS, while the simple
allocator also does this explicitly in its code.

    M
_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[BogdanM]

On Wed, Mar 16, 2011 at 1:23 AM, Martin Guy <[hidden email]> wrote:

Confirmed. With stack of 2048 or 3072, life.lua either doesn't parse

or doesn't run. 4096 seems to work for life.lua
However, if it's a recursive descent parser, the amount of C stack
that a piece of Lua code could require is potentially infinite.

>> I don't really
>> understand the need for this much stack especially since I manually moved
>> many of the Lua parser internal data structures from heap to stack.

From Lua 5 onwards, calls from Lua functions to Lua functions are
"stackless", i.e. do not consume any C stack.  There is just a hard
limit of 20000 nested calls to trap infinite recursion.
C stack is used in parsing (as you say) and, I gather from the
following message, in calls from Lua to C functions that call back
into Lua (sort, pcall...) but those should not be infinitely deep.

Exactly, those should not be infinitely deep. So I'm still confused.

http://comments.gmane.org/gmane.comp.lang.lua.general/6376

>> Sure. But the reason I'm trying to keep it small in other platforms (even
>> with the risk of stack overflows) is the lack of memory. Mizar32 doesn't
>> have this problem

With the speed difference of a factor of two between SRAM+SDRAM and
only SDRAM for the Lua heap, I'd say it does.

Of course it depends on your optimization goal. So far it was always space in detriment of speed for me.

 

I confess I am confused by the assertion that simple and newlib can't
use multiple memory blocks.
Experience using SRAM, SRAM+SDRAM and just SDRAM, with simple and newlib is that
- SRAM+SDRAM is markedly faster than just SDRAM: about 10% for 20
generations of life.
- You can allocate and access 100K of junk in a table when using
SRAM+SDRAM in both allocators
and inspection of the code seems to show that the newlib allocator in
newlib/stubs.c, using newlib _sbrk_r is redefined to return memory
from all blocks listed in MEM_{START,END}_ADDRESS, while the simple
allocator also does this explicitly in its code.

'newlib' and 'multiple' are actually the same allocator: dlmalloc (http://g.oswego.edu/dl/html/malloc.html). In the early days, when the first embedded eLua targets were brought to life, Newlib didn't have a dlmalloc version that supported non-contigous memory spaces (or it couldn't be compiled to support non-contigious memory spaces, I can't remember the exact reason) so I added a newever version of dlmalloc (that didn't have this problem) to the source tree. The only problem was that the newer version was also larger so I kept it as a separate option. Quite a bit of time have passed since then though and by your claims it seems that things have changed in Newlib. Thanks! I'll take a look at this when I have some time.

On a somehow related note, I'll integrate your emBlod patch in the eLua mainline as soon as (you guessed it) I have some time, which should me this week. Again, sorry for the delay.

Best,

Bogdan

_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev