eLua Development

reducing rom size

Classic

List

Threaded

9 messages Options

Dean Hall

reducing rom size

The reason I was so happy to see Bogdan reduce the RAM footprint was
that I was hoping to get eLua to run on the Atmel AT91SAM7S64 that has
64 KB Flash and 16 KB SRAM. eLua is small, but I want to see how much
smaller I can make it and still run pre-compiled code. Here's what
I've done:

- Copied Atmel's SAM7S development files into a new platform directory
- Copied the platform.c from the SAM7X platform directory
- Removed Lua's lexer, parser and dump following instructions in
lua-5.1.4/etc/noparser.c
- Removed Lua's libs except base and math
- Removed uip
- Removed demo source in romfs
- Removed all modules except pio and uart
- Build line: scons board=AT91SAM7S-EK target=lualong cpumode=thumb
noparser=true

(aside: I did all of the above without modifying any eLua source code
except SConstruct and adding src/noparser.c. That is a sign of a well-
conceived project and build system.)

The result is "region flash overflowed by 4380 bytes." I'm so close!
Size details are below for those interested. I welcome any ideas to
further reduce flash usage.

!!Dean

text data bss dec hex filename
532 0 0 532 214 src/main.o
572 36 52 660 294 src/romfs.o
592 0 8 600 258 src/xmodem.o
1739 0 4 1743 6cf src/shell.o
544 0 28 572 23c src/term.o
0 0 0 0 0 src/dlmalloc.o
636 1 19 656 290 src/common.o
4615 subtotal

272 0 0 272 110 src/platform/at91sam7s/
board_cstartup.o
248 0 0 248 f8 src/platform/at91sam7s/
board_lowlevel.o
64 0 0 64 40 src/platform/at91sam7s/
board_memories.o
1472 12 0 1484 5cc src/platform/at91sam7s/
platform.o
440 0 0 440 1b8 src/platform/at91sam7s/pmc/
pmc.o
276 0 0 276 114 src/platform/at91sam7s/pio/
pio.o
168 0 0 168 a8 src/platform/at91sam7s/tc/tc.o
64 0 0 64 40 src/platform/at91sam7s/aic/
aic.o
468 0 0 468 1d4 src/platform/at91sam7s/pwmc/
pwmc.o
383 0 0 383 17f src/platform/at91sam7s/usart/
usart.o
3855 subtotal

328 0 580 908 38c src/newlib/devman.o
718 0 8 726 2d6 src/newlib/stubs.o
356 36 8 400 190 src/newlib/genstd.o
0 0 0 0 0 src/newlib/stdtcp.o
1402 subtotal

54 0 0 54 36 src/lua/noparser.o
4048 0 0 4048 fd0 src/lua/lapi.o
4661 0 0 4661 1235 src/lua/lcode.o
3455 0 0 3455 d7f src/lua/ldebug.o
2614 0 0 2614 a36 src/lua/ldo.o
652 0 0 652 28c src/lua/lfunc.o
2916 0 0 2916 b64 src/lua/lgc.o
219 0 0 219 db src/lua/lmem.o
1129 0 0 1129 469 src/lua/lobject.o
436 0 0 436 1b4 src/lua/lopcodes.o
814 0 0 814 32e src/lua/lstate.o
456 0 0 456 1c8 src/lua/lstring.o
2388 0 0 2388 954 src/lua/ltable.o
550 0 0 550 226 src/lua/ltm.o
1288 0 0 1288 508 src/lua/lundump.o
5457 0 0 5457 1551 src/lua/lvm.o
220 0 0 220 dc src/lua/lzio.o
194 0 0 194 c2 src/lua/linit.o
2624 4 4 2632 a48 src/lua/lua.o
508 0 0 508 1fc src/lua/lrotable.o
34683 subtotal

4984 0 0 4984 1378 src/lua/lbaselib.o
2094 0 0 2094 82e src/lua/lmathlib.o
7078 subtotal

1425 0 64 1489 5d1 src/modules/pio.o
695 0 0 695 2b7 src/modules/uart.o
2120 subtotal
=======
53753 TOTAL

(yes I'm under 64 KB here, but I believe arm-elf-gcc is adding in libc
and libm that takes up the remaining space and more)

BogdanM

reducing rom size

Very interesting! I was aiming to try this on the 128k SAM7 at some point,
but you're even more ambitious :) Some other ideas:

- remove the shell if you don't need it, you'd save more than 1.5k of Flash
this way. The code in src/main.c automatically starts the standard Lua
interpreter if the shell isn't compiled in, but I'm guessing this isn't a
solution for you, since you don't have a parser at all. If all you do is
start a precompiled script stored in ROMFS, then name it "autorun.luac" (I
think this should work, although I never tried autorun with precompiled
bytecode, only with source code) and once again you won't have to touch the
code at all, as main.c will start it automatically. If your scritpt runs in
an infinite loop you never return to main(), those you won't start the
interpreter.

- you're compiling in integer mode, so why exactly do you need the mathlib
for ? :) It won't work, and the "lualong" mode doesn't even include the
mathlib, but the rotables implementation doesn't take this into account and
adds the math module map array to flash anyway (which is a bug and I'll fix
it), so removing the mathlib would some even more Flash.

- start drifting away from Newlib. This isn't by any means easy, but there's
already a branch called "elua_with_own_libc" on which I started to replace
Newlib with a modified version of the Minix libc implementation. But the
work on this is in its very early stages, so don't expect anything
functional just yet. Although I think that you can completely replace the
strxxx (strcmp/strcpy...) and memxxxx functions with you can find there, and
not only that. Try to play with it a bit if you have the time.

- use a different compiler. GCC is a great compiler, but there are better
ARM compilers (in terms of optimizations) out there.

- about the RAM now: we all know that LTR is great :D, still 16k isn't much
at all. The default allocator in Lua is Newlib's default allocator, which is
Doug Lea's dlmalloc (http://g.oswego.edu/dl/html/malloc.html). It is a very
good seggregated allocator and it does wonders when it comes to handling
memory fragmentation, but the RAM overhead that the allocator itself adds
might be significant in a system with only 16k of RAM. We might need a
special allocator for this. I wrote a very simple, very small "chained
allocator" to test the RAM requirements of Lua before even starting eLua,
and it works quite well, even though it is slow and it obviously doesn't
handle fragmentation as well as a seggregated allocator, so we might use it
instead. Also Ralph Hempel (the author of pbLua) wrote something quite
similar (
http://www.hempeldesigngroup.com/embedded/stories/memoryManager.html). I
didn't have a chance to benchmark his implementation versus mine yet.

HTH,
Bogdan

On Wed, Jan 14, 2009 at 6:50 AM, Dean Hall <dwhall256 at gmail.com> wrote:

> The reason I was so happy to see Bogdan reduce the RAM footprint was
> that I was hoping to get eLua to run on the Atmel AT91SAM7S64 that has
> 64 KB Flash and 16 KB SRAM. eLua is small, but I want to see how much
> smaller I can make it and still run pre-compiled code. Here's what
> I've done:
>
> - Copied Atmel's SAM7S development files into a new platform directory
> - Copied the platform.c from the SAM7X platform directory
> - Removed Lua's lexer, parser and dump following instructions in
> lua-5.1.4/etc/noparser.c
> - Removed Lua's libs except base and math
> - Removed uip
> - Removed demo source in romfs
> - Removed all modules except pio and uart
> - Build line: scons board=AT91SAM7S-EK target=lualong cpumode=thumb
> noparser=true
>
> (aside: I did all of the above without modifying any eLua source code
> except SConstruct and adding src/noparser.c. That is a sign of a well-
> conceived project and build system.)
>
> The result is "region flash overflowed by 4380 bytes." I'm so close!
> Size details are below for those interested. I welcome any ideas to
> further reduce flash usage.
>
> !!Dean
>
>
>
>
> text data bss dec hex filename
> 532 0 0 532 214 src/main.o
> 572 36 52 660 294 src/romfs.o
> 592 0 8 600 258 src/xmodem.o
> 1739 0 4 1743 6cf src/shell.o
> 544 0 28 572 23c src/term.o
> 0 0 0 0 0 src/dlmalloc.o
> 636 1 19 656 290 src/common.o
> 4615 subtotal
>
> 272 0 0 272 110 src/platform/at91sam7s/
> board_cstartup.o
> 248 0 0 248 f8 src/platform/at91sam7s/
> board_lowlevel.o
> 64 0 0 64 40 src/platform/at91sam7s/
> board_memories.o
> 1472 12 0 1484 5cc src/platform/at91sam7s/
> platform.o
> 440 0 0 440 1b8 src/platform/at91sam7s/pmc/
> pmc.o
> 276 0 0 276 114 src/platform/at91sam7s/pio/
> pio.o
> 168 0 0 168 a8
> src/platform/at91sam7s/tc/tc.o
> 64 0 0 64 40 src/platform/at91sam7s/aic/
> aic.o
> 468 0 0 468 1d4 src/platform/at91sam7s/pwmc/
> pwmc.o
> 383 0 0 383 17f
> src/platform/at91sam7s/usart/
> usart.o
> 3855 subtotal
>
> 328 0 580 908 38c src/newlib/devman.o
> 718 0 8 726 2d6 src/newlib/stubs.o
> 356 36 8 400 190 src/newlib/genstd.o
> 0 0 0 0 0 src/newlib/stdtcp.o
> 1402 subtotal
>
> 54 0 0 54 36 src/lua/noparser.o
> 4048 0 0 4048 fd0 src/lua/lapi.o
> 4661 0 0 4661 1235 src/lua/lcode.o
> 3455 0 0 3455 d7f src/lua/ldebug.o
> 2614 0 0 2614 a36 src/lua/ldo.o
> 652 0 0 652 28c src/lua/lfunc.o
> 2916 0 0 2916 b64 src/lua/lgc.o
> 219 0 0 219 db src/lua/lmem.o
> 1129 0 0 1129 469 src/lua/lobject.o
> 436 0 0 436 1b4 src/lua/lopcodes.o
> 814 0 0 814 32e src/lua/lstate.o
> 456 0 0 456 1c8 src/lua/lstring.o
> 2388 0 0 2388 954 src/lua/ltable.o
> 550 0 0 550 226 src/lua/ltm.o
> 1288 0 0 1288 508 src/lua/lundump.o
> 5457 0 0 5457 1551 src/lua/lvm.o
> 220 0 0 220 dc src/lua/lzio.o
> 194 0 0 194 c2 src/lua/linit.o
> 2624 4 4 2632 a48 src/lua/lua.o
> 508 0 0 508 1fc src/lua/lrotable.o
> 34683 subtotal
>
> 4984 0 0 4984 1378 src/lua/lbaselib.o
> 2094 0 0 2094 82e src/lua/lmathlib.o
> 7078 subtotal
>
> 1425 0 64 1489 5d1 src/modules/pio.o
> 695 0 0 695 2b7 src/modules/uart.o
> 2120 subtotal
> =======
> 53753 TOTAL
>
> (yes I'm under 64 KB here, but I believe arm-elf-gcc is adding in libc
> and libm that takes up the remaining space and more)
>
> _______________________________________________
> Elua-dev mailing list
> Elua-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/elua-dev
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.berlios.de/pipermail/elua-dev/attachments/20090114/bf434bc1/attachment.html

BogdanM

reducing rom size

You might want to try now. After fixing the bug I describe below (mathlib's
rotable not being ignored in lualong mode) the binary image decreased with
about 23k on a LM3S :) I somehow doubt it will have the same dramatic effect
in your case, but it's worth a shot.

Best,
Bogdan

On Wed, Jan 14, 2009 at 11:11 AM, Bogdan Marinescu <
bogdan.marinescu at gmail.com> wrote:

> Very interesting! I was aiming to try this on the 128k SAM7 at some point,
> but you're even more ambitious :) Some other ideas:
>
> - remove the shell if you don't need it, you'd save more than 1.5k of Flash
> this way. The code in src/main.c automatically starts the standard Lua
> interpreter if the shell isn't compiled in, but I'm guessing this isn't a
> solution for you, since you don't have a parser at all. If all you do is
> start a precompiled script stored in ROMFS, then name it "autorun.luac" (I
> think this should work, although I never tried autorun with precompiled
> bytecode, only with source code) and once again you won't have to touch the
> code at all, as main.c will start it automatically. If your scritpt runs in
> an infinite loop you never return to main(), those you won't start the
> interpreter.
>
> - you're compiling in integer mode, so why exactly do you need the mathlib
> for ? :) It won't work, and the "lualong" mode doesn't even include the
> mathlib, but the rotables implementation doesn't take this into account and
> adds the math module map array to flash anyway (which is a bug and I'll fix
> it), so removing the mathlib would some even more Flash.
>
> - start drifting away from Newlib. This isn't by any means easy, but
> there's already a branch called "elua_with_own_libc" on which I started to
> replace Newlib with a modified version of the Minix libc implementation. But
> the work on this is in its very early stages, so don't expect anything
> functional just yet. Although I think that you can completely replace the
> strxxx (strcmp/strcpy...) and memxxxx functions with you can find there, and
> not only that. Try to play with it a bit if you have the time.
>
> - use a different compiler. GCC is a great compiler, but there are better
> ARM compilers (in terms of optimizations) out there.
>
> - about the RAM now: we all know that LTR is great :D, still 16k isn't much
> at all. The default allocator in Lua is Newlib's default allocator, which is
> Doug Lea's dlmalloc (http://g.oswego.edu/dl/html/malloc.html). It is a
> very good seggregated allocator and it does wonders when it comes to
> handling memory fragmentation, but the RAM overhead that the allocator
> itself adds might be significant in a system with only 16k of RAM. We might
> need a special allocator for this. I wrote a very simple, very small
> "chained allocator" to test the RAM requirements of Lua before even starting
> eLua, and it works quite well, even though it is slow and it obviously
> doesn't handle fragmentation as well as a seggregated allocator, so we might
> use it instead. Also Ralph Hempel (the author of pbLua) wrote something
> quite similar (
> http://www.hempeldesigngroup.com/embedded/stories/memoryManager.html). I
> didn't have a chance to benchmark his implementation versus mine yet.
>
> HTH,
> Bogdan
>
>
> On Wed, Jan 14, 2009 at 6:50 AM, Dean Hall <dwhall256 at gmail.com> wrote:
>
>> The reason I was so happy to see Bogdan reduce the RAM footprint was
>> that I was hoping to get eLua to run on the Atmel AT91SAM7S64 that has
>> 64 KB Flash and 16 KB SRAM. eLua is small, but I want to see how much
>> smaller I can make it and still run pre-compiled code. Here's what
>> I've done:
>>
>> - Copied Atmel's SAM7S development files into a new platform directory
>> - Copied the platform.c from the SAM7X platform directory
>> - Removed Lua's lexer, parser and dump following instructions in
>> lua-5.1.4/etc/noparser.c
>> - Removed Lua's libs except base and math
>> - Removed uip
>> - Removed demo source in romfs
>> - Removed all modules except pio and uart
>> - Build line: scons board=AT91SAM7S-EK target=lualong cpumode=thumb
>> noparser=true
>>
>> (aside: I did all of the above without modifying any eLua source code
>> except SConstruct and adding src/noparser.c. That is a sign of a well-
>> conceived project and build system.)
>>
>> The result is "region flash overflowed by 4380 bytes." I'm so close!
>> Size details are below for those interested. I welcome any ideas to
>> further reduce flash usage.
>>
>> !!Dean
>>
>>
>>
>>
>> text data bss dec hex filename
>> 532 0 0 532 214 src/main.o
>> 572 36 52 660 294 src/romfs.o
>> 592 0 8 600 258 src/xmodem.o
>> 1739 0 4 1743 6cf src/shell.o
>> 544 0 28 572 23c src/term.o
>> 0 0 0 0 0 src/dlmalloc.o
>> 636 1 19 656 290 src/common.o
>> 4615 subtotal
>>
>> 272 0 0 272 110 src/platform/at91sam7s/
>> board_cstartup.o
>> 248 0 0 248 f8 src/platform/at91sam7s/
>> board_lowlevel.o
>> 64 0 0 64 40 src/platform/at91sam7s/
>> board_memories.o
>> 1472 12 0 1484 5cc src/platform/at91sam7s/
>> platform.o
>> 440 0 0 440 1b8 src/platform/at91sam7s/pmc/
>> pmc.o
>> 276 0 0 276 114 src/platform/at91sam7s/pio/
>> pio.o
>> 168 0 0 168 a8
>> src/platform/at91sam7s/tc/tc.o
>> 64 0 0 64 40 src/platform/at91sam7s/aic/
>> aic.o
>> 468 0 0 468 1d4
>> src/platform/at91sam7s/pwmc/
>> pwmc.o
>> 383 0 0 383 17f
>> src/platform/at91sam7s/usart/
>> usart.o
>> 3855 subtotal
>>
>> 328 0 580 908 38c src/newlib/devman.o
>> 718 0 8 726 2d6 src/newlib/stubs.o
>> 356 36 8 400 190 src/newlib/genstd.o
>> 0 0 0 0 0 src/newlib/stdtcp.o
>> 1402 subtotal
>>
>> 54 0 0 54 36 src/lua/noparser.o
>> 4048 0 0 4048 fd0 src/lua/lapi.o
>> 4661 0 0 4661 1235 src/lua/lcode.o
>> 3455 0 0 3455 d7f src/lua/ldebug.o
>> 2614 0 0 2614 a36 src/lua/ldo.o
>> 652 0 0 652 28c src/lua/lfunc.o
>> 2916 0 0 2916 b64 src/lua/lgc.o
>> 219 0 0 219 db src/lua/lmem.o
>> 1129 0 0 1129 469 src/lua/lobject.o
>> 436 0 0 436 1b4 src/lua/lopcodes.o
>> 814 0 0 814 32e src/lua/lstate.o
>> 456 0 0 456 1c8 src/lua/lstring.o
>> 2388 0 0 2388 954 src/lua/ltable.o
>> 550 0 0 550 226 src/lua/ltm.o
>> 1288 0 0 1288 508 src/lua/lundump.o
>> 5457 0 0 5457 1551 src/lua/lvm.o
>> 220 0 0 220 dc src/lua/lzio.o
>> 194 0 0 194 c2 src/lua/linit.o
>> 2624 4 4 2632 a48 src/lua/lua.o
>> 508 0 0 508 1fc src/lua/lrotable.o
>> 34683 subtotal
>>
>> 4984 0 0 4984 1378 src/lua/lbaselib.o
>> 2094 0 0 2094 82e src/lua/lmathlib.o
>> 7078 subtotal
>>
>> 1425 0 64 1489 5d1 src/modules/pio.o
>> 695 0 0 695 2b7 src/modules/uart.o
>> 2120 subtotal
>> =======
>> 53753 TOTAL
>>
>> (yes I'm under 64 KB here, but I believe arm-elf-gcc is adding in libc
>> and libm that takes up the remaining space and more)
>>
>> _______________________________________________
>> Elua-dev mailing list
>> Elua-dev at lists.berlios.de
>> https://lists.berlios.de/mailman/listinfo/elua-dev
>>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.berlios.de/pipermail/elua-dev/attachments/20090114/bbc5307a/attachment.html

Dean Hall

reducing rom size

I followed some of Bogdan's advice and have it compiling under 64 KB!
I had to edit lua code this time, so it's not as clean.

- Removed the shell (had to remove calls to shell funcs in src/main.c)
- Updated svn and now mathlib is no longer compiled in; no code
changes needed.
- Restored src/lua/lauxlib.c (I previously overlooked error messages
that said its funcs were needed)
- Added a stub in src/lua/noparser.c for luaX_syntaxerror(LexState
*ls, const char *msg);
- Ifdef'd away unused/uncompiled entries in lualibs[] in linit.c

Result:

$ arm-elf-size elua_lualong_at91sam7s64.elf
text data bss dec hex filename
60088 2265 764 63117 f68d elua_lualong_at91sam7s64.elf

I didn't think I'd make it this far so quickly, so I haven't planned
my next steps in detail yet. I think I'd like to make a hosted
interactive environment. This is where the host PC will accept
interactive input from the programmer, compile to lua bytecode (.luac
packets), send those to the micro for execution, and accept & print
result strings.

I realize I'll run out of RAM quickly, but it's worth a try.

!!Dean

On Jan 14, 2009, at 05:05 , Bogdan Marinescu wrote:

> You might want to try now. After fixing the bug I describe below
> (mathlib's rotable not being ignored in lualong mode) the binary
> image decreased with about 23k on a LM3S :) I somehow doubt it will
> have the same dramatic effect in your case, but it's worth a shot.
>
> Best,
> Bogdan
>
> On Wed, Jan 14, 2009 at 11:11 AM, Bogdan Marinescu <bogdan.marinescu at gmail.com
> > wrote:
> Very interesting! I was aiming to try this on the 128k SAM7 at some
> point, but you're even more ambitious :) Some other ideas:
>
> - remove the shell if you don't need it, you'd save more than 1.5k
> of Flash this way. The code in src/main.c automatically starts the
> standard Lua interpreter if the shell isn't compiled in, but I'm
> guessing this isn't a solution for you, since you don't have a
> parser at all. If all you do is start a precompiled script stored in
> ROMFS, then name it "autorun.luac" (I think this should work,
> although I never tried autorun with precompiled bytecode, only with
> source code) and once again you won't have to touch the code at all,
> as main.c will start it automatically. If your scritpt runs in an
> infinite loop you never return to main(), those you won't start the
> interpreter.
>
> - you're compiling in integer mode, so why exactly do you need the
> mathlib for ? :) It won't work, and the "lualong" mode doesn't even
> include the mathlib, but the rotables implementation doesn't take
> this into account and adds the math module map array to flash anyway
> (which is a bug and I'll fix it), so removing the mathlib would some
> even more Flash.
>
> - start drifting away from Newlib. This isn't by any means easy, but
> there's already a branch called "elua_with_own_libc" on which I
> started to replace Newlib with a modified version of the Minix libc
> implementation. But the work on this is in its very early stages, so
> don't expect anything functional just yet. Although I think that you
> can completely replace the strxxx (strcmp/strcpy...) and memxxxx
> functions with you can find there, and not only that. Try to play
> with it a bit if you have the time.
>
> - use a different compiler. GCC is a great compiler, but there are
> better ARM compilers (in terms of optimizations) out there.
>
> - about the RAM now: we all know that LTR is great :D, still 16k
> isn't much at all. The default allocator in Lua is Newlib's default
> allocator, which is Doug Lea's dlmalloc (http://g.oswego.edu/dl/html/malloc.html
> ). It is a very good seggregated allocator and it does wonders when
> it comes to handling memory fragmentation, but the RAM overhead that
> the allocator itself adds might be significant in a system with only
> 16k of RAM. We might need a special allocator for this. I wrote a
> very simple, very small "chained allocator" to test the RAM
> requirements of Lua before even starting eLua, and it works quite
> well, even though it is slow and it obviously doesn't handle
> fragmentation as well as a seggregated allocator, so we might use it
> instead. Also Ralph Hempel (the author of pbLua) wrote something
> quite similar (http://www.hempeldesigngroup.com/embedded/stories/memoryManager.html
> ). I didn't have a chance to benchmark his implementation versus
> mine yet.
>
> HTH,
> Bogdan
>
>
> On Wed, Jan 14, 2009 at 6:50 AM, Dean Hall <dwhall256 at gmail.com>
> wrote:
> The reason I was so happy to see Bogdan reduce the RAM footprint was
> that I was hoping to get eLua to run on the Atmel AT91SAM7S64 that has
> 64 KB Flash and 16 KB SRAM. eLua is small, but I want to see how much
> smaller I can make it and still run pre-compiled code. Here's what
> I've done:
>
> - Copied Atmel's SAM7S development files into a new platform directory
> - Copied the platform.c from the SAM7X platform directory
> - Removed Lua's lexer, parser and dump following instructions in
> lua-5.1.4/etc/noparser.c
> - Removed Lua's libs except base and math
> - Removed uip
> - Removed demo source in romfs
> - Removed all modules except pio and uart
> - Build line: scons board=AT91SAM7S-EK target=lualong cpumode=thumb
> noparser=true
>
> (aside: I did all of the above without modifying any eLua source code
> except SConstruct and adding src/noparser.c. That is a sign of a
> well-
> conceived project and build system.)
>
> The result is "region flash overflowed by 4380 bytes." I'm so close!
> Size details are below for those interested. I welcome any ideas to
> further reduce flash usage.
>
> !!Dean
>
>
>
>
> text data bss dec hex filename
> 532 0 0 532 214 src/main.o
> 572 36 52 660 294 src/romfs.o
> 592 0 8 600 258 src/xmodem.o
> 1739 0 4 1743 6cf src/shell.o
> 544 0 28 572 23c src/term.o
> 0 0 0 0 0 src/dlmalloc.o
> 636 1 19 656 290 src/common.o
> 4615 subtotal
>
> 272 0 0 272 110 src/platform/at91sam7s/
> board_cstartup.o
> 248 0 0 248 f8 src/platform/at91sam7s/
> board_lowlevel.o
> 64 0 0 64 40 src/platform/at91sam7s/
> board_memories.o
> 1472 12 0 1484 5cc src/platform/at91sam7s/
> platform.o
> 440 0 0 440 1b8 src/platform/
> at91sam7s/pmc/
> pmc.o
> 276 0 0 276 114 src/platform/
> at91sam7s/pio/
> pio.o
> 168 0 0 168 a8 src/platform/
> at91sam7s/tc/tc.o
> 64 0 0 64 40 src/platform/
> at91sam7s/aic/
> aic.o
> 468 0 0 468 1d4 src/platform/
> at91sam7s/pwmc/
> pwmc.o
> 383 0 0 383 17f src/platform/
> at91sam7s/usart/
> usart.o
> 3855 subtotal
>
> 328 0 580 908 38c src/newlib/devman.o
> 718 0 8 726 2d6 src/newlib/stubs.o
> 356 36 8 400 190 src/newlib/genstd.o
> 0 0 0 0 0 src/newlib/stdtcp.o
> 1402 subtotal
>
> 54 0 0 54 36 src/lua/noparser.o
> 4048 0 0 4048 fd0 src/lua/lapi.o
> 4661 0 0 4661 1235 src/lua/lcode.o
> 3455 0 0 3455 d7f src/lua/ldebug.o
> 2614 0 0 2614 a36 src/lua/ldo.o
> 652 0 0 652 28c src/lua/lfunc.o
> 2916 0 0 2916 b64 src/lua/lgc.o
> 219 0 0 219 db src/lua/lmem.o
> 1129 0 0 1129 469 src/lua/lobject.o
> 436 0 0 436 1b4 src/lua/lopcodes.o
> 814 0 0 814 32e src/lua/lstate.o
> 456 0 0 456 1c8 src/lua/lstring.o
> 2388 0 0 2388 954 src/lua/ltable.o
> 550 0 0 550 226 src/lua/ltm.o
> 1288 0 0 1288 508 src/lua/lundump.o
> 5457 0 0 5457 1551 src/lua/lvm.o
> 220 0 0 220 dc src/lua/lzio.o
> 194 0 0 194 c2 src/lua/linit.o
> 2624 4 4 2632 a48 src/lua/lua.o
> 508 0 0 508 1fc src/lua/lrotable.o
> 34683 subtotal
>
> 4984 0 0 4984 1378 src/lua/lbaselib.o
> 2094 0 0 2094 82e src/lua/lmathlib.o
> 7078 subtotal
>
> 1425 0 64 1489 5d1 src/modules/pio.o
> 695 0 0 695 2b7 src/modules/uart.o
> 2120 subtotal
> =======
> 53753 TOTAL
>
> (yes I'm under 64 KB here, but I believe arm-elf-gcc is adding in libc
> and libm that takes up the remaining space and more)
>
> _______________________________________________
> Elua-dev mailing list
> Elua-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/elua-dev
>
>
> _______________________________________________
> Elua-dev mailing list
> Elua-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/elua-dev

BogdanM

reducing rom size

On Wed, Jan 14, 2009 at 6:31 PM, Dean Hall <dwhall256 at gmail.com> wrote:

> I followed some of Bogdan's advice and have it compiling under 64 KB!

Wow! Congrats, I didn't think this was possible with so few changes! Besides
the obvious practical applications, this is very good advertisment for eLua
:)

>
> I had to edit lua code this time, so it's not as clean.
>
> - Removed the shell (had to remove calls to shell funcs in src/main.c)
>

Really? I just did a test: I commented out #define BUILD_SHELL in
src/platform/at91sam7x/platform_conf.h,and the whole thing compiled just
fine.

> - Updated svn and now mathlib is no longer compiled in; no code
> changes needed.
> - Restored src/lua/lauxlib.c (I previously overlooked error messages
> that said its funcs were needed)
> - Added a stub in src/lua/noparser.c for luaX_syntaxerror(LexState
> *ls, const char *msg);
> - Ifdef'd away unused/uncompiled entries in lualibs[] in linit.c

Good point, I forgot to tell you that you should do this : )

> Result:
>
> $ arm-elf-size elua_lualong_at91sam7s64.elf
> text data bss dec hex filename
> 60088 2265 764 63117 f68d
> elua_lualong_at91sam7s64.elf
>
> I didn't think I'd make it this far so quickly, so I haven't planned
> my next steps in detail yet. I think I'd like to make a hosted
> interactive environment. This is where the host PC will accept
> interactive input from the programmer, compile to lua bytecode (.luac
> packets), send those to the micro for execution, and accept & print
> result strings.

Will you use XMODEM, or just simply send the data via UART?
Also, you might to crosscompile your Lua bytecode, although this shouldn't
be a problem for lualong, it's only important for "regular" Lua. I'd give
you a link to a page with more data on the subject, but
eluaproject.netseems to be going very slow right now for some reason.

Best,
Bogdan

On Jan 14, 2009, at 05:05 , Bogdan Marinescu wrote:

>
> > You might want to try now. After fixing the bug I describe below
> > (mathlib's rotable not being ignored in lualong mode) the binary
> > image decreased with about 23k on a LM3S :) I somehow doubt it will
> > have the same dramatic effect in your case, but it's worth a shot.
> >
> > Best,
> > Bogdan
> >
> > On Wed, Jan 14, 2009 at 11:11 AM, Bogdan Marinescu <
> bogdan.marinescu at gmail.com
> > > wrote:
> > Very interesting! I was aiming to try this on the 128k SAM7 at some
> > point, but you're even more ambitious :) Some other ideas:
> >
> > - remove the shell if you don't need it, you'd save more than 1.5k
> > of Flash this way. The code in src/main.c automatically starts the
> > standard Lua interpreter if the shell isn't compiled in, but I'm
> > guessing this isn't a solution for you, since you don't have a
> > parser at all. If all you do is start a precompiled script stored in
> > ROMFS, then name it "autorun.luac" (I think this should work,
> > although I never tried autorun with precompiled bytecode, only with
> > source code) and once again you won't have to touch the code at all,
> > as main.c will start it automatically. If your scritpt runs in an
> > infinite loop you never return to main(), those you won't start the
> > interpreter.
> >
> > - you're compiling in integer mode, so why exactly do you need the
> > mathlib for ? :) It won't work, and the "lualong" mode doesn't even
> > include the mathlib, but the rotables implementation doesn't take
> > this into account and adds the math module map array to flash anyway
> > (which is a bug and I'll fix it), so removing the mathlib would some
> > even more Flash.
> >
> > - start drifting away from Newlib. This isn't by any means easy, but
> > there's already a branch called "elua_with_own_libc" on which I
> > started to replace Newlib with a modified version of the Minix libc
> > implementation. But the work on this is in its very early stages, so
> > don't expect anything functional just yet. Although I think that you
> > can completely replace the strxxx (strcmp/strcpy...) and memxxxx
> > functions with you can find there, and not only that. Try to play
> > with it a bit if you have the time.
> >
> > - use a different compiler. GCC is a great compiler, but there are
> > better ARM compilers (in terms of optimizations) out there.
> >
> > - about the RAM now: we all know that LTR is great :D, still 16k
> > isn't much at all. The default allocator in Lua is Newlib's default
> > allocator, which is Doug Lea's dlmalloc (
> http://g.oswego.edu/dl/html/malloc.html
> > ). It is a very good seggregated allocator and it does wonders when
> > it comes to handling memory fragmentation, but the RAM overhead that
> > the allocator itself adds might be significant in a system with only
> > 16k of RAM. We might need a special allocator for this. I wrote a
> > very simple, very small "chained allocator" to test the RAM
> > requirements of Lua before even starting eLua, and it works quite
> > well, even though it is slow and it obviously doesn't handle
> > fragmentation as well as a seggregated allocator, so we might use it
> > instead. Also Ralph Hempel (the author of pbLua) wrote something
> > quite similar (
> http://www.hempeldesigngroup.com/embedded/stories/memoryManager.html
> > ). I didn't have a chance to benchmark his implementation versus
> > mine yet.
> >
> > HTH,
> > Bogdan
> >
> >
> > On Wed, Jan 14, 2009 at 6:50 AM, Dean Hall <dwhall256 at gmail.com>
> > wrote:
> > The reason I was so happy to see Bogdan reduce the RAM footprint was
> > that I was hoping to get eLua to run on the Atmel AT91SAM7S64 that has
> > 64 KB Flash and 16 KB SRAM. eLua is small, but I want to see how much
> > smaller I can make it and still run pre-compiled code. Here's what
> > I've done:
> >
> > - Copied Atmel's SAM7S development files into a new platform directory
> > - Copied the platform.c from the SAM7X platform directory
> > - Removed Lua's lexer, parser and dump following instructions in
> > lua-5.1.4/etc/noparser.c
> > - Removed Lua's libs except base and math
> > - Removed uip
> > - Removed demo source in romfs
> > - Removed all modules except pio and uart
> > - Build line: scons board=AT91SAM7S-EK target=lualong cpumode=thumb
> > noparser=true
> >
> > (aside: I did all of the above without modifying any eLua source code
> > except SConstruct and adding src/noparser.c. That is a sign of a
> > well-
> > conceived project and build system.)
> >
> > The result is "region flash overflowed by 4380 bytes." I'm so close!
> > Size details are below for those interested. I welcome any ideas to
> > further reduce flash usage.
> >
> > !!Dean
> >
> >
> >
> >
> > text data bss dec hex filename
> > 532 0 0 532 214 src/main.o
> > 572 36 52 660 294 src/romfs.o
> > 592 0 8 600 258 src/xmodem.o
> > 1739 0 4 1743 6cf src/shell.o
> > 544 0 28 572 23c src/term.o
> > 0 0 0 0 0 src/dlmalloc.o
> > 636 1 19 656 290 src/common.o
> > 4615 subtotal
> >
> > 272 0 0 272 110 src/platform/at91sam7s/
> > board_cstartup.o
> > 248 0 0 248 f8 src/platform/at91sam7s/
> > board_lowlevel.o
> > 64 0 0 64 40 src/platform/at91sam7s/
> > board_memories.o
> > 1472 12 0 1484 5cc src/platform/at91sam7s/
> > platform.o
> > 440 0 0 440 1b8 src/platform/
> > at91sam7s/pmc/
> > pmc.o
> > 276 0 0 276 114 src/platform/
> > at91sam7s/pio/
> > pio.o
> > 168 0 0 168 a8 src/platform/
> > at91sam7s/tc/tc.o
> > 64 0 0 64 40 src/platform/
> > at91sam7s/aic/
> > aic.o
> > 468 0 0 468 1d4 src/platform/
> > at91sam7s/pwmc/
> > pwmc.o
> > 383 0 0 383 17f src/platform/
> > at91sam7s/usart/
> > usart.o
> > 3855 subtotal
> >
> > 328 0 580 908 38c src/newlib/devman.o
> > 718 0 8 726 2d6 src/newlib/stubs.o
> > 356 36 8 400 190 src/newlib/genstd.o
> > 0 0 0 0 0 src/newlib/stdtcp.o
> > 1402 subtotal
> >
> > 54 0 0 54 36 src/lua/noparser.o
> > 4048 0 0 4048 fd0 src/lua/lapi.o
> > 4661 0 0 4661 1235 src/lua/lcode.o
> > 3455 0 0 3455 d7f src/lua/ldebug.o
> > 2614 0 0 2614 a36 src/lua/ldo.o
> > 652 0 0 652 28c src/lua/lfunc.o
> > 2916 0 0 2916 b64 src/lua/lgc.o
> > 219 0 0 219 db src/lua/lmem.o
> > 1129 0 0 1129 469 src/lua/lobject.o
> > 436 0 0 436 1b4 src/lua/lopcodes.o
> > 814 0 0 814 32e src/lua/lstate.o
> > 456 0 0 456 1c8 src/lua/lstring.o
> > 2388 0 0 2388 954 src/lua/ltable.o
> > 550 0 0 550 226 src/lua/ltm.o
> > 1288 0 0 1288 508 src/lua/lundump.o
> > 5457 0 0 5457 1551 src/lua/lvm.o
> > 220 0 0 220 dc src/lua/lzio.o
> > 194 0 0 194 c2 src/lua/linit.o
> > 2624 4 4 2632 a48 src/lua/lua.o
> > 508 0 0 508 1fc src/lua/lrotable.o
> > 34683 subtotal
> >
> > 4984 0 0 4984 1378 src/lua/lbaselib.o
> > 2094 0 0 2094 82e src/lua/lmathlib.o
> > 7078 subtotal
> >
> > 1425 0 64 1489 5d1 src/modules/pio.o
> > 695 0 0 695 2b7 src/modules/uart.o
> > 2120 subtotal
> > =======
> > 53753 TOTAL
> >
> > (yes I'm under 64 KB here, but I believe arm-elf-gcc is adding in libc
> > and libm that takes up the remaining space and more)
> >
> > _______________________________________________
> > Elua-dev mailing list
> > Elua-dev at lists.berlios.de
> > https://lists.berlios.de/mailman/listinfo/elua-dev
> >
> >
> > _______________________________________________
> > Elua-dev mailing list
> > Elua-dev at lists.berlios.de
> > https://lists.berlios.de/mailman/listinfo/elua-dev
>
> _______________________________________________
> Elua-dev mailing list
> Elua-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/elua-dev
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.berlios.de/pipermail/elua-dev/attachments/20090114/19732940/attachment-0001.html

Dean Hall

reducing rom size

On Jan 14, 2009, at 11:38 , Bogdan Marinescu wrote:

> On Wed, Jan 14, 2009 at 6:31 PM, Dean Hall <dwhall256 at gmail.com>
> wrote:
>
> - Removed the shell (had to remove calls to shell funcs in src/main.c)
>
> Really? I just did a test: I commented out #define BUILD_SHELL in
> src/platform/at91sam7x/platform_conf.h,and the whole thing compiled
> just fine.

I was working from compiler errors and ignoring source (a.k.a. hatchet
job). Your way is easier and cleaner. I will switch to your way.

>
> Will you use XMODEM, or just simply send the data via UART?

I'm undecided. I don't know the .luac format yet, so I don't know if
the receiver can determine the number of bytes to expect based on the
contents of the data sent. If .luac does indicate its length, I'll do
raw UART; otherwise, I might use a super-simple protocol.

> Also, you might to crosscompile your Lua bytecode, although this
> shouldn't be a problem for lualong, it's only important for
> "regular" Lua. I'd give you a link to a page with more data on the
> subject, but eluaproject.net seems to be going very slow right now
> for some reason.

I think I remember reading something to that effect: that ARM uses a
different encoding for floating point values. I'll look into it if it
becomes a problem, but like you say, I'm using lualong.

I haven't studied how easy it is to insert into the build my own pre-
compiled module without ROMFS. If ROMFS is required for such things,
I will put it back in.

With these items [un]defined:

#define BUILD_XMODEM
//#define BUILD_SHELL
//#define BUILD_ROMFS
//#define BUILD_TERM
#define BUILD_CON_GENERIC

the build is

$ arm-elf-size elua_lualong_at91sam7s64.elf
text data bss dec hex filename
59920 2265 736 62921 f5c9 elua_lualong_at91sam7s64.elf

!!Dean

BogdanM

reducing rom size

> I'm undecided. I don't know the .luac format yet, so I don't know if
> the receiver can determine the number of bytes to expect based on the
> contents of the data sent. If .luac does indicate its length, I'll do
> raw UART; otherwise, I might use a super-simple protocol.

This is a tricky one. All the terminal emulators I used pad the last
XMODEM packet with EOF (0x1A) chars up to its maximum size (128
bytes). Now, if I interpreted the "no frills" document correctly
(http://luaforge.net/docman/view.php/83/98/ANoFrillsIntroToLua51VMInstructions.pdf)
a binary chunk can never end in a 0x1A, so we _should_ be safe (all
you have to do is scan the last received XMODEM packet until no more
0x1A chars are found, like code in src/shell/shell_recv does) . But I
need to check this more. FWIW all the tests I made with compiled
bytecode over XMODEM worked flawlessly.

> I haven't studied how easy it is to insert into the build my own pre-
> compiled module without ROMFS. If ROMFS is required for such things,
> I will put it back in.

Copy it to the ROMFS directory, then add it to your board-to-ROMFS
list mapping. See the definition of "romfs" and "file_list" variables
in SConstruct.

> With these items [un]defined:
>
> #define BUILD_XMODEM
> //#define BUILD_SHELL
> //#define BUILD_ROMFS
> //#define BUILD_TERM
> #define BUILD_CON_GENERIC
>
> the build is
>
> $ arm-elf-size elua_lualong_at91sam7s64.elf
> text data bss dec hex filename
> 59920 2265 736 62921 f5c9 elua_lualong_at91sam7s64.elf

You can comment out BUILD_CON_GENERIC too. It's used only for standard
I/O (stdio/stdout/stderr) over UART, and you don't seem to use that.

Best,
Bogdan

Alex Babkin

reducing rom size

So does this mean i can now try compiling/running it on the STM32 with 20kb
of ram and 128kb flash ?
sweet!

Alex

On Wed, Jan 14, 2009 at 2:54 PM, Bogdan Marinescu <
bogdan.marinescu at gmail.com> wrote:

> > I'm undecided. I don't know the .luac format yet, so I don't know if
> > the receiver can determine the number of bytes to expect based on the
> > contents of the data sent. If .luac does indicate its length, I'll do
> > raw UART; otherwise, I might use a super-simple protocol.
>
> This is a tricky one. All the terminal emulators I used pad the last
> XMODEM packet with EOF (0x1A) chars up to its maximum size (128
> bytes). Now, if I interpreted the "no frills" document correctly
> (
> http://luaforge.net/docman/view.php/83/98/ANoFrillsIntroToLua51VMInstructions.pdf
> )
> a binary chunk can never end in a 0x1A, so we _should_ be safe (all
> you have to do is scan the last received XMODEM packet until no more
> 0x1A chars are found, like code in src/shell/shell_recv does) . But I
> need to check this more. FWIW all the tests I made with compiled
> bytecode over XMODEM worked flawlessly.
>
> > I haven't studied how easy it is to insert into the build my own pre-
> > compiled module without ROMFS. If ROMFS is required for such things,
> > I will put it back in.
>
> Copy it to the ROMFS directory, then add it to your board-to-ROMFS
> list mapping. See the definition of "romfs" and "file_list" variables
> in SConstruct.
>
> > With these items [un]defined:
> >
> > #define BUILD_XMODEM
> > //#define BUILD_SHELL
> > //#define BUILD_ROMFS
> > //#define BUILD_TERM
> > #define BUILD_CON_GENERIC
> >
> > the build is
> >
> > $ arm-elf-size elua_lualong_at91sam7s64.elf
> > text data bss dec hex filename
> > 59920 2265 736 62921 f5c9
> elua_lualong_at91sam7s64.elf
>
> You can comment out BUILD_CON_GENERIC too. It's used only for standard
> I/O (stdio/stdout/stderr) over UART, and you don't seem to use that.
>
> Best,
> Bogdan
> _______________________________________________
> Elua-dev mailing list
> Elua-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/elua-dev
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.berlios.de/pipermail/elua-dev/attachments/20090126/59edc2c2/attachment.html

BogdanM

reducing rom size

You can try, don't make too many hopes though :) I'm actually curious
to know how that goes.

Best,
Bogdan

On Tue, Jan 27, 2009 at 1:34 AM, Alex Babkin <ababkin at gmail.com> wrote:

> So does this mean i can now try compiling/running it on the STM32 with 20kb
> of ram and 128kb flash ?
> sweet!
>
> Alex
>
> On Wed, Jan 14, 2009 at 2:54 PM, Bogdan Marinescu
> <bogdan.marinescu at gmail.com> wrote:
>>
>> > I'm undecided. I don't know the .luac format yet, so I don't know if
>> > the receiver can determine the number of bytes to expect based on the
>> > contents of the data sent. If .luac does indicate its length, I'll do
>> > raw UART; otherwise, I might use a super-simple protocol.
>>
>> This is a tricky one. All the terminal emulators I used pad the last
>> XMODEM packet with EOF (0x1A) chars up to its maximum size (128
>> bytes). Now, if I interpreted the "no frills" document correctly
>>
>> (http://luaforge.net/docman/view.php/83/98/ANoFrillsIntroToLua51VMInstructions.pdf)
>> a binary chunk can never end in a 0x1A, so we _should_ be safe (all
>> you have to do is scan the last received XMODEM packet until no more
>> 0x1A chars are found, like code in src/shell/shell_recv does) . But I
>> need to check this more. FWIW all the tests I made with compiled
>> bytecode over XMODEM worked flawlessly.
>>
>> > I haven't studied how easy it is to insert into the build my own pre-
>> > compiled module without ROMFS. If ROMFS is required for such things,
>> > I will put it back in.
>>
>> Copy it to the ROMFS directory, then add it to your board-to-ROMFS
>> list mapping. See the definition of "romfs" and "file_list" variables
>> in SConstruct.
>>
>> > With these items [un]defined:
>> >
>> > #define BUILD_XMODEM
>> > //#define BUILD_SHELL
>> > //#define BUILD_ROMFS
>> > //#define BUILD_TERM
>> > #define BUILD_CON_GENERIC
>> >
>> > the build is
>> >
>> > $ arm-elf-size elua_lualong_at91sam7s64.elf
>> > text data bss dec hex filename
>> > 59920 2265 736 62921 f5c9
>> > elua_lualong_at91sam7s64.elf
>>
>> You can comment out BUILD_CON_GENERIC too. It's used only for standard
>> I/O (stdio/stdout/stderr) over UART, and you don't seem to use that.
>>
>> Best,
>> Bogdan
>> _______________________________________________
>> Elua-dev mailing list
>> Elua-dev at lists.berlios.de
>> https://lists.berlios.de/mailman/listinfo/elua-dev
>
>
> _______________________________________________
> Elua-dev mailing list
> Elua-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/elua-dev
>
>