Instrumenting stack usage

[Martin Guy]

I've checked to see how much C stack is used in reality and attach a
nasty patch that does the job.
It works by zeroing the C stack at startup and checking to see how
much of it is still zero just before printing the Lua prompt.

The results confirm Bogdan's practical experience: Parsing a
non-trivial program (50KB with lots of upvalues) needs 4096 bytes of C
stack.
This is tested on a 32-bit AVR32 platform, but suggests that the other
platforms with the stack set to 2048 bytes will be unreliable.

> Lua 5.1.4  Copyright (C) 1994-2011 Lua.org, PUC-Rio
2212 bytes of stack have been used
>
2380 bytes of stack have been used
> require "life"
3264 bytes of stack have been used
> require "ed"
3948 bytes of stack have been used

> Lua 5.1.4  Copyright (C) 1994-2011 Lua.org, PUC-Rio
2212 bytes of stack have been used
> require "ed"
3948 bytes of stack have been used
>

     M

_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[BogdanM]

Hi,

On Mon, Mar 28, 2011 at 7:38 AM, Martin Guy <[hidden email]> wrote:

I've checked to see how much C stack is used in reality and attach a
nasty patch that does the job.
It works by zeroing the C stack at startup and checking to see how
much of it is still zero just before printing the Lua prompt.

The results confirm Bogdan's practical experience: Parsing a
non-trivial program (50KB with lots of upvalues) needs 4096 bytes of C
stack.
This is tested on a 32-bit AVR32 platform, but suggests that the other
platforms with the stack set to 2048 bytes will be unreliable.

> Lua 5.1.4  Copyright (C) 1994-2011 Lua.org, PUC-Rio
2212 bytes of stack have been used
>
2380 bytes of stack have been used
> require "life"
3264 bytes of stack have been used
> require "ed"
3948 bytes of stack have been used

> Lua 5.1.4  Copyright (C) 1994-2011 Lua.org, PUC-Rio
2212 bytes of stack have been used
> require "ed"
3948 bytes of stack have been used
>

Thanks a lot for this! I've been meaning to do some experimentation precisely like this for a while now. The next step is to understand why this happens. And if that works, to test if it's possible to make the stack-hungry function(s) consume the heap instead and if that actually helps (my vote would be on "yes"). Ideally, this would be a job for a stack tracer. Need to check if I have the right tools for that.

 

Best,

Bogdan

_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[Dado Sutter]

Hi,

On Mon, Mar 28, 2011 at 03:31, Bogdan Marinescu <[hidden email]> wrote:

Hi,

On Mon, Mar 28, 2011 at 7:38 AM, Martin Guy <[hidden email]> wrote:

I've checked to see how much C stack is used in reality and attach a
nasty patch that does the job.
It works by zeroing the C stack at startup and checking to see how
much of it is still zero just before printing the Lua prompt.

The results confirm Bogdan's practical experience: Parsing a
non-trivial program (50KB with lots of upvalues) needs 4096 bytes of C
stack.
This is tested on a 32-bit AVR32 platform, but suggests that the other
platforms with the stack set to 2048 bytes will be unreliable.

> Lua 5.1.4  Copyright (C) 1994-2011 Lua.org, PUC-Rio
2212 bytes of stack have been used
>
2380 bytes of stack have been used
> require "life"
3264 bytes of stack have been used
> require "ed"
3948 bytes of stack have been used

> Lua 5.1.4  Copyright (C) 1994-2011 Lua.org, PUC-Rio
2212 bytes of stack have been used
> require "ed"
3948 bytes of stack have been used
>

Thanks a lot for this! I've been meaning to do some experimentation precisely like this for a while now. The next step is to understand why this happens. And if that works, to test if it's possible to make the stack-hungry function(s) consume the heap instead and if that actually helps (my vote would be on "yes"). Ideally, this would be a job for a stack tracer. Need to check if I have the right tools for that.

Need any help or guess from Roberto for that ?

Best
Dado


 

 

Best,

Bogdan

_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[BogdanM]

On Mon, Mar 28, 2011 at 4:19 PM, Dado Sutter <[hidden email]> wrote:

Need any help or guess from Roberto for that ?

Well, he could hint us on what part of the code takes so much stack space when running life.lua :)

Best,

Bogdan

_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[Dado Sutter]

On Mon, Mar 28, 2011 at 10:22, Bogdan Marinescu <[hidden email]> wrote:

On Mon, Mar 28, 2011 at 4:19 PM, Dado Sutter <[hidden email]> wrote:

Need any help or guess from Roberto for that ?

Well, he could hint us on what part of the code takes so much stack space when running life.lua :)

OK :)
We'll "play" Life one of these days at lunch then :)

Best,

Bogdan

Best
Dado



 

_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[Martin Guy]

On 28 March 2011 08:31, Bogdan Marinescu <[hidden email]> wrote:
> On Mon, Mar 28, 2011 at 7:38 AM, Martin Guy <[hidden email]> wrote:
>> I've checked to see how much C stack is used in reality and attach a
>> nasty patch that does the job.
>> 2380 bytes of stack have been used
>> > require "life"
>> 3264 bytes of stack have been used
>> > require "ed"
>> 3948 bytes of stack have been used

> The next step is to understand why this happens.

Why what happens? Why it uses 2K of stack during starup? Or why it
uses 3 to 3.9K to parse programs?

As you say, parsing the programs is the big one.  Running the Lua
programs requires very little stack, since the Lua stack is on the
heap and recursive Lua functions use no C stack.

There is an email post from Roberto about what contribures to using
stack. He says the compiled limits on upvalues and local variables
make a difference, but I've experimented increasing LUAI_MAXUPVALUES
from 20 to 250 in eLua and it made absolutely no difference whatsoever
to the stack high water mark. Not a byte!
However, at least now we can play with the settings and see the
difference that it really makes... in all this free time that we
have... ;-)

     M
_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[BogdanM]

On Tue, Mar 29, 2011 at 10:27 AM, Martin Guy <[hidden email]> wrote:

On 28 March 2011 08:31, Bogdan Marinescu <[hidden email]> wrote:
> On Mon, Mar 28, 2011 at 7:38 AM, Martin Guy <[hidden email]> wrote:
>> I've checked to see how much C stack is used in reality and attach a
>> nasty patch that does the job.

>> 2380 bytes of stack have been used
>> > require "life"
>> 3264 bytes of stack have been used
>> > require "ed"
>> 3948 bytes of stack have been used

> The next step is to understand why this happens.

Why what happens? Why it uses 2K of stack during starup? Or why it
uses 3 to 3.9K to parse programs?

Both. 2K on startup happens most likely because of some large automatic variables. Which reminds me of a very nifty little script called checkstack.pl:

http://stackoverflow.com/questions/126036/checking-stack-usage-at-compile-time

I used it once (quite a while ago), I should probably do it again. It does something very simple, yet very effective: it disassembles the object files and looks for instructions that decrement the stack pointer (something like "sub (e)sp,xxxx" on i386 (I remember that it can work with other architectures too)). I was able to somehow sort its output by the ammount of bytes substracted from the stack pointer. Quite an easy way to find the functions that use too much stack space.

 

As you say, parsing the programs is the big one.  Running the Lua
programs requires very little stack, since the Lua stack is on the
heap and recursive Lua functions use no C stack.

There is an email post from Roberto about what contribures to using
stack. He says the compiled limits on upvalues and local variables
make a difference, but I've experimented increasing LUAI_MAXUPVALUES
from 20 to 250 in eLua and it made absolutely no difference whatsoever
to the stack high water mark. Not a byte!

That doesn't really surprise me. Upvalues are not kept on the stack. I don't understand Roberto's statement.

 

However, at least now we can play with the settings and see the
difference that it really makes... in all this free time that we
have... ;-)

... or build an instrumentation framework :D Yes, I have that in mind too. Only for Cortex M3 for now. I should invent the 96 hours day before starting to work on that though :)

Best,

Bogdan

_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[Martin Guy]

> checkstack.pl:
Cool!

>> There is an email post from Roberto about what contribures to using
>> stack.
>
> That doesn't really surprise me. Upvalues are not kept on the stack. I don't
> understand Roberto's statement.

The article http://lua-users.org/lists/lua-l/2005-01/msg00275.html is
from 2005, so maybe it applies to Lua 4 or something similar.

> I should invent the 96 hours day before starting to work on that though :)

An acquaintance at Uni lived 28 hour days for a while, giving 6 days
per (conventional) week.
He had some theory about syncing with the circadian chemical rhythms
of the body that cycle every 28 hours.
In practice it meant he could party with his friends at the weekends
but during the week it made it less likely he would meet his project
supervisor :)

    M
_______________________________________________
eLua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev