Patrick-11 |
Hi Guys
I am sorry to say that since discovering the project I have had almost no time to use it let alone contribute anything at all. I've been working like crazy and now expect to be out of debt in the next two weeks. After months of anxiety driven chest pains and breathing issues I plan to slow my business down and have a bit of "me" time. What better way then to do something with eLua :) Am I correct that the biggest strength of the project is as a rapid development environment? Having Lua right on a device is great but there is of course a cost of this. What if someone started out with Lua scripts but then little by little moved all the Lua-Lua code into Lua C modules and then eventually jettisoned Lua entirely for a significant speed up and footprint reduction for final deployment. """" #include "lua.h" #include "lauxlib.h" #include "lualib.h" lua_State *L = luaL_newstate(); """ In the above code I believe "L" is now a struct with both data and function pointers. What about this: """ #include "nolua.h" lua_State *L = luaL_newstate();""" Where L is now a stripped down struct only containing enough functionality to allow C modules to talk to each other, a stack, a registry the required functions. If this dummy Lua state contained no parsing abilities for source/byte code I would assume there would be a significant reduction in footprint or am I the one in a dummy state! If this is not nuts I could make an attempt in February-March. Thanks-Pat _______________________________________________ eLua-dev mailing list [hidden email] https://lists.berlios.de/mailman/listinfo/elua-dev |
> Hi Guys
> > I am sorry to say that since discovering the project I have had almost no > time to use it let alone contribute anything at all. I've been working like > crazy and now expect to be out of debt in the next two weeks. After months > of anxiety driven chest pains and breathing issues I plan to slow my > business down and have a bit of "me" time. What better way then to do > something with eLua :) Wow, that sounds serious and unpleasant. Glad to hear you're better now. > Am I correct that the biggest strength of the project is as a rapid > development environment? Yes. > Having Lua right on a device is great but there is > of course a cost of this. What if someone started out with Lua scripts but > then little by little moved all the Lua-Lua code into Lua C modules and then > eventually jettisoned Lua entirely for a significant speed up and footprint > reduction for final deployment. > > > """" > #include "lua.h" > #include "lauxlib.h" > #include "lualib.h" > > lua_State *L = luaL_newstate(); > > """ > In the above code I believe "L" is now a struct with both data and function > pointers. What about this: > > """ > #include "nolua.h" > > lua_State *L = luaL_newstate();""" > > Where L is now a stripped down struct only containing enough functionality > to allow C modules to talk to each other, a stack, a registry the required > functions. > > If this dummy Lua state contained no parsing abilities for source/byte code > I would assume there would be a significant reduction in footprint or am I > the one in a dummy state! > > If this is not nuts I could make an attempt in February-March. This isn't nuts for sure but it's not really in sync with eLua's development philosophy. It seems to me that doing what you suggested would eliminate eLua's greatest strength (mentioned earlier) which is rapid development. Moving things to the C side requires both effort and a different methodology (as you won't be able to simply enter your program as you go, you'd have to compile it somewhere and then upload it somehow). Plus, C development on embedded platforms is already very well covered by a number of libraries and tools. All in all I don't see many advantages of such an approach, although the idea is definitely interesting. Best, Bogdan _______________________________________________ eLua-dev mailing list [hidden email] https://lists.berlios.de/mailman/listinfo/elua-dev |
Patrick-11 |
Hi Bogdan
Thanks for your feedback. I am not a professional programmer let alone an embedded one. Would not most manufactures use much of the same code across different models?(i,e ten models of microwave ovens) Would it not be better to write the common components as C modules and then glue them together with eLua? If this statement is true then couldn't someone complete a new model with eLua using it as a rapid development environment and then in the final stages port the glue to C? Nolua would allow the C modules to function as they did before and only the original Lua glue would need porting. It's -20C today in Toronto, hope Bucharest is more merciful-Pat _______________________________________________ eLua-dev mailing list [hidden email] https://lists.berlios.de/mailman/listinfo/elua-dev |
> Hi Bogdan
> > Thanks for your feedback. > > I am not a professional programmer let alone an embedded one. > > Would not most manufactures use much of the same code across different > models?(i,e ten models of microwave ovens) Would it not be better to write > the common components as C modules and then glue them together with eLua? I'm looking for reasons that would make it "better". If you want to write your firmware in C you're likely to find a lot of libraries that would make your job easier that using eLua as a glue layer between a number of modules. > If this statement is true then couldn't someone complete a new model with > eLua using it as a rapid development environment and then in the final > stages port the glue to C? The glue would be Lua, which is already extremely portable and requires almost no porting effort at all (which the exception of some libraries like "io" and "os"). > Nolua would allow the C modules to function as > they did before and only the original Lua glue would need porting. I understand the idea but I don't find it too appealing for the reasons outlined above. If you want to play with the concept you can probably start from here: https://github.com/davidm/lua2c I'm actually curious to find the difference in size/speed after using lua2c. I don't expect major improvements on any front, but I'm still curious to see some actual results. > It's -20C today in Toronto, hope Bucharest is more merciful-Pat Well it's about -6 here but it's been snowing a lot in the last couple of days so not really that much more merciful :) Best, Bogdan _______________________________________________ eLua-dev mailing list [hidden email] https://lists.berlios.de/mailman/listinfo/elua-dev |
On Sun, Jan 23, 2011 at 10:38 AM, Bogdan Marinescu
<[hidden email]> wrote: >> Hi Bogdan >> >> Thanks for your feedback. >> >> I am not a professional programmer let alone an embedded one. >> >> Would not most manufactures use much of the same code across different >> models?(i,e ten models of microwave ovens) Would it not be better to write >> the common components as C modules and then glue them together with eLua? > > I'm looking for reasons that would make it "better". If you want to > write your firmware in C you're likely to find a lot of libraries that > would make your job easier that using eLua as a glue layer between a > number of modules. There is another model here which is that we now have a platform API that could be separated from the main Lua core and used as a cross-platform build environment for C applications. I think we've talked a little about pulling this out before. I don't think it would require a huge amount of effort, but it's not really a core project goal of eLua. > >> If this statement is true then couldn't someone complete a new model with >> eLua using it as a rapid development environment and then in the final >> stages port the glue to C? > > The glue would be Lua, which is already extremely portable and > requires almost no porting effort at all (which the exception of some > libraries like "io" and "os"). > >> Nolua would allow the C modules to function as >> they did before and only the original Lua glue would need porting. > > I understand the idea but I don't find it too appealing for the > reasons outlined above. If you want to play with the concept you can > probably start from here: > > https://github.com/davidm/lua2c > > I'm actually curious to find the difference in size/speed after using > lua2c. I don't expect major improvements on any front, but I'm still > curious to see some actual results. I've been curious about this as well, and I've never given it a try. I'm not sure what potential roadblocks might exist, but there might be some value to providing a compile mode that uses lua2c to convert Lua scripts and then compiles for a specific platform to generate "final" firmware for a device. One thing about lua2c though is that it converts Lua scripts into C code that uses the Lua C API to do the same thing, so you would still need the VM. You wouldn't need the compiler, but we can already skip that by pre-generating Lua bytecode. It might be possible to cut out a few additional components when only using the C API code, but I suspect the size effect might be minimal. > >> It's -20C today in Toronto, hope Bucharest is more merciful-Pat > > Well it's about -6 here but it's been snowing a lot in the last couple > of days so not really that much more merciful :) -7C in chicago, but it was around -20 on Friday. > > Best, > Bogdan > _______________________________________________ > eLua-dev mailing list > [hidden email] > https://lists.berlios.de/mailman/listinfo/elua-dev > -- James Snyder Biomedical Engineering Northwestern University [hidden email] PGP: http://fanplastic.org/key.txt Phone: (847) 448-0386 _______________________________________________ eLua-dev mailing list [hidden email] https://lists.berlios.de/mailman/listinfo/elua-dev |
Patrick-11 |
Hi James
Thanks for offering direction on this. It seems like the response to this is luke warm at best, I wonder if my idea is based on faulty principles. I know these benchmarks are not always accurate but according to this: http://shootout.alioth.debian.org/u32/which-programming-languages-are-fastest.php Vanilla Lua is about 31X slower then C, which in itself does not make total sense as LuaJit is supposed to be 2X to 4X faster then Lua but is ranked on this list as only 2X slower then C, which ought to make it 15X faster then Lua. Having said all this I would have expected that removing Lua from a final deployment would have speed things up at least 2X and offered a similar reduction in memory allow for the final program to be loaded on reduced hardware and in turn saving costs. I have not done much with serialization but I was thinking about serializing the L state struct and trying to reload it back without including the lua files. I am not even sure if this would work at all but if it did it might give me some idea of the savings of just having what is contained in L which I am assuming is not all of Lua itself but just some function pointers, the API stack and a registry, enough to make the C API work. If this is all false could you rescue me from myself? Thanks-Patrick _______________________________________________ eLua-dev mailing list [hidden email] https://lists.berlios.de/mailman/listinfo/elua-dev |
Patrick-11 |
Sorry for answering my own post. I just wanted to mention that I am
thinking that L would contain C API functions and pointers to them but not all of Lua. Thanks On 01/23/2011 05:10 PM, Patrick Mc(avery wrote: > Hi James > > Thanks for offering direction on this. > > It seems like the response to this is luke warm at best, I wonder if my > idea is based on faulty principles. > > I know these benchmarks are not always accurate but according to this: > http://shootout.alioth.debian.org/u32/which-programming-languages-are-fastest.php > > > Vanilla Lua is about 31X slower then C, which in itself does not make > total sense as LuaJit is supposed to be 2X to 4X faster then Lua but is > ranked on this list as only 2X slower then C, which ought to make it 15X > faster then Lua. Having said all this I would have expected that > removing Lua from a final deployment would have speed things up at least > 2X and offered a similar reduction in memory allow for the final program > to be loaded on reduced hardware and in turn saving costs. > > I have not done much with serialization but I was thinking about > serializing the L state struct and trying to reload it back without > including the lua files. I am not even sure if this would work at all > but if it did it might give me some idea of the savings of just having > what is contained in L which I am assuming is not all of Lua itself but > just some function pointers, the API stack and a registry, enough to > make the C API work. > > If this is all false could you rescue me from myself? > > Thanks-Patrick > _______________________________________________ > eLua-dev mailing list > [hidden email] > https://lists.berlios.de/mailman/listinfo/elua-dev > _______________________________________________ eLua-dev mailing list [hidden email] https://lists.berlios.de/mailman/listinfo/elua-dev |
Robert G. Jakabosky |
In reply to this post by jbsnyder
On Sunday 23, James Snyder wrote:
> On Sun, Jan 23, 2011 at 10:38 AM, Bogdan Marinescu > > <[hidden email]> wrote: > >> Hi Bogdan > >> > >> Thanks for your feedback. > >> > >> I am not a professional programmer let alone an embedded one. > >> > >> Would not most manufactures use much of the same code across different > >> models?(i,e ten models of microwave ovens) Would it not be better to > >> write the common components as C modules and then glue them together > >> with eLua? > > > > I'm looking for reasons that would make it "better". If you want to > > write your firmware in C you're likely to find a lot of libraries that > > would make your job easier that using eLua as a glue layer between a > > number of modules. > > There is another model here which is that we now have a platform API > that could be separated from the main Lua core and used as a > cross-platform build environment for C applications. I think we've > talked a little about pulling this out before. I don't think it would > require a huge amount of effort, but it's not really a core project > goal of eLua. > > >> If this statement is true then couldn't someone complete a new model > >> with eLua using it as a rapid development environment and then in the > >> final stages port the glue to C? > > > > The glue would be Lua, which is already extremely portable and > > requires almost no porting effort at all (which the exception of some > > libraries like "io" and "os"). > > > >> Nolua would allow the C modules to function as > >> they did before and only the original Lua glue would need porting. > > > > I understand the idea but I don't find it too appealing for the > > reasons outlined above. If you want to play with the concept you can > > probably start from here: > > > > https://github.com/davidm/lua2c > > > > I'm actually curious to find the difference in size/speed after using > > lua2c. I don't expect major improvements on any front, but I'm still > > curious to see some actual results. > > I've been curious about this as well, and I've never given it a try. > > I'm not sure what potential roadblocks might exist, but there might be > some value to providing a compile mode that uses lua2c to convert Lua > scripts and then compiles for a specific platform to generate "final" > firmware for a device. One thing about lua2c though is that it > converts Lua scripts into C code that uses the Lua C API to do the > same thing, so you would still need the VM. You wouldn't need the > compiler, but we can already skip that by pre-generating Lua bytecode. > It might be possible to cut out a few additional components when only > using the C API code, but I suspect the size effect might be minimal. One performance problem with compiling Lua scripts to C using the C API is the use of C strings. When C code passes a C string to Lua it must be internalized before it can be used. This will slow down all table accesses where the key is a string. The only way around that is to create an upvalue for each const. C string used in the C code. The last time I checked lua2c didn't use upvalues for string constants. Also I think the Lua C API has more checks (overhead) then the VM core has when executing bytecodes. Some things could be made faster like loop counters (using integers instead of doubles). I had started porting my llvm-lua project over to a new slua [1] project that static compiles lua scripts/bytecode into C code that uses the same internal VM opcode instead of the Lua C API. The one big downside of using this method to compile script is that it requires some changes to the Lua VM core (not any major changes, just some hooks for static compiled Lua functions). slua can static compile a Lua script into a C source file that looks like a normal Lua C module, which can then be compiled with a standard C compiler. slua needs a little bit of cleaning up, but it is able to create stand-alone exe or Lua C modules (note the C modules only work with a modified Lua VM core, they can't be loaded into a normal Lua VM). 1. https://github.com/Neopallium/slua -- Robert G. Jakabosky _______________________________________________ eLua-dev mailing list [hidden email] https://lists.berlios.de/mailman/listinfo/elua-dev |
In reply to this post by Patrick-11
On Mon, Jan 24, 2011 at 12:10 AM, Patrick Mc(avery
<[hidden email]> wrote: > Hi James > > Thanks for offering direction on this. > > It seems like the response to this is luke warm at best, I wonder if my idea > is based on faulty principles. Don't let anybody (me included) discourage you. Just because me or others don't find this a good idea it doesn't mean it actually isn't a good idea :) If you'd like to work on something like this do it anyway. After all results are all that matters, so if you get good results you'll be able to prove your detractors wrong :) Just for the record I don't think the idea is bad, just strange in an interesting (but not very useful) way. > I know these benchmarks are not always accurate but according to this: > http://shootout.alioth.debian.org/u32/which-programming-languages-are-fastest.php > > Vanilla Lua is about 31X slower then C, which in itself does not make total > sense as LuaJit is supposed to be 2X to 4X faster then Lua but is ranked on > this list as only 2X slower then C, which ought to make it 15X faster then > Lua. Welcome to the wonderful world of benchmarks-which-many-times-aren't-relevant-in-the-real-world :) > Having said all this I would have expected that removing Lua from a > final deployment would have speed things up at least 2X and offered a > similar reduction in memory allow for the final program to be loaded on > reduced hardware and in turn saving costs. That actually makes sense. Eliminating the interpreter inner loop might have an actual (and observable) performance impact. > I have not done much with serialization but I was thinking about serializing > the L state struct and trying to reload it back without including the lua > files. I can't say that I understood this part very well, but why don't you start with something like lua2c (mentioned on my previous e-mail) and build from there? It does exactly what you want: it turns Lua code into C code. And it doesn't require any modifications to Lua itself. If you're satisfied with the results you can work from there, removing the parser (the Lua distribution already has an example on how to do this) and maybe the bytecode interpreter and so on (note though that you'll have some issues if you do this, for example 'loadstring' won't work anymore). You don't have to start with modifying Lua itself. I can tell you this can become quite a bit challenging (especially something like modifying the Lua state structure which touches almost every part of the code). >I am not even sure if this would work at all but if it did it might > give me some idea of the savings of just having what is contained in L which > I am assuming is not all of Lua itself but just some function pointers, the > API stack and a registry, enough to make the C API work. > > If this is all false could you rescue me from myself? I don't know if modifying L is the first thing to think about. Depending on what you'll end up removing from standard Lua you might be able to come down with a stripped down version of L, although I doubt that this will have a significant impact. Understanding what parts of code you can remove/make smaller, on the other hand, might make a big difference and it might also be easier to implement. Best, Bogdan _______________________________________________ eLua-dev mailing list [hidden email] https://lists.berlios.de/mailman/listinfo/elua-dev |
In reply to this post by Robert G. Jakabosky
Hi,
> One performance problem with compiling Lua scripts to C using the C API is the > use of C strings. When C code passes a C string to Lua it must be > internalized before it can be used. This will slow down all table accesses > where the key is a string. The only way around that is to create an upvalue > for each const. C string used in the C code. The last time I checked lua2c > didn't use upvalues for string constants. Also I think the Lua C API has > more checks (overhead) then the VM core has when executing bytecodes. Thanks for your insight. It always helps to have someone who's actually familiar with the Lua core around. > Some things could be made faster like loop counters (using integers instead of > doubles). I had started porting my llvm-lua project over to a new slua [1] > project that static compiles lua scripts/bytecode into C code that uses the > same internal VM opcode instead of the Lua C API. The one big downside of > using this method to compile script is that it requires some changes to the > Lua VM core (not any major changes, just some hooks for static compiled Lua > functions). slua can static compile a Lua script into a C source file that > looks like a normal Lua C module, which can then be compiled with a standard > C compiler. Thank you very much! This sounds very interesting and useful for eLua. I don't have time right now to look at it but I definitely will. Best, Bogdan _______________________________________________ eLua-dev mailing list [hidden email] https://lists.berlios.de/mailman/listinfo/elua-dev |
On Mon, Jan 24, 2011 at 7:31 AM, Bogdan Marinescu
<[hidden email]> wrote: > Hi, > >> One performance problem with compiling Lua scripts to C using the C API is the >> use of C strings. When C code passes a C string to Lua it must be >> internalized before it can be used. This will slow down all table accesses >> where the key is a string. The only way around that is to create an upvalue >> for each const. C string used in the C code. The last time I checked lua2c >> didn't use upvalues for string constants. Also I think the Lua C API has >> more checks (overhead) then the VM core has when executing bytecodes. > > Thanks for your insight. It always helps to have someone who's > actually familiar with the Lua core around. I had forgotten about the string internalization issue, this is a good point. > >> Some things could be made faster like loop counters (using integers instead of >> doubles). I had started porting my llvm-lua project over to a new slua [1] >> project that static compiles lua scripts/bytecode into C code that uses the >> same internal VM opcode instead of the Lua C API. The one big downside of >> using this method to compile script is that it requires some changes to the >> Lua VM core (not any major changes, just some hooks for static compiled Lua >> functions). slua can static compile a Lua script into a C source file that >> looks like a normal Lua C module, which can then be compiled with a standard >> C compiler. > > Thank you very much! This sounds very interesting and useful for eLua. > I don't have time right now to look at it but I definitely will. Using something like LLVM as a static compiler was one thing I was thinking of when reading through Patrick's proposal. I thought of mentioning llvm-lua, but from my understanding the ARM support for MCUs isn't necessarily usable at this stage, although I believe I saw some Thumb-2 support make it into LLVM during this last year. I suspect that one might get the greatest speed benefit by being able to statically compile Lua into machine code, perhaps using LLVM as an intermediate to get some optimization benefits. One question about what you're describing though: "compiles lua scripts/bytecode into C code that uses the same internal VM opcode instead of the Lua C API" I'm not completely clear on what advantages this has vs just using luac to generate a loadable but pre-compiled Lua module? I could see in the case of eLua that we could maybe use this as another way to use Lua code and keep things in flash rather than in RAM on the heap. Are there other advantages that aren't occurring to me? From the readme it also sounds like it can generate stand-alone executables, which I would presume would allow one to save some space? In any case, I'll be downloading and checking out slua :-) Also, to Patrick: As Bogdan was saying, we don't wish to discourage you from trying things, crazy or not :-) Sometimes the the only way to find out if something is viable is to try. I believe that there should be some solution in the realm of what you're describing that would allow the creation of more compact distributions, it might be that another tack will get there more effectively. Also, whether something works or not, you learn a lot from digging around in the guts of things like Lua, which can help a lot to shape future ideas and approaches. -- James Snyder Biomedical Engineering Northwestern University [hidden email] PGP: http://fanplastic.org/key.txt Phone: (847) 448-0386 _______________________________________________ eLua-dev mailing list [hidden email] https://lists.berlios.de/mailman/listinfo/elua-dev |
Robert G. Jakabosky |
On Monday 24, James Snyder wrote:
> Using something like LLVM as a static compiler was one thing I was > thinking of when reading through Patrick's proposal. I thought of > mentioning llvm-lua, but from my understanding the ARM support for > MCUs isn't necessarily usable at this stage, although I believe I saw > some Thumb-2 support make it into LLVM during this last year. I > suspect that one might get the greatest speed benefit by being able to > statically compile Lua into machine code, perhaps using LLVM as an > intermediate to get some optimization benefits. I don't know the state of LLVM's ARM backend support, but one big benefit of sLua compared to llvm-lua is that it generates pure C code that is as portable as the Lua VM core. LLVM does have a backend for compiling it's IR into C code but by the time the IR gets compiled to C there are alignment/size issues, since the size of pointers/ints/longs and structure member alignment gets baked into the LLVM IR before the C backend codegen runs. With some hacks I was able to fix the alignment issues and do a cross-compile of a Lua script for the iPhone (with the help of someone with a Mac & iPhone). But it was messy, which is one reason I started sLua. > One question about what you're describing though: > "compiles lua scripts/bytecode into C code that uses the same internal > VM opcode instead of the Lua C API" > > I'm not completely clear on what advantages this has vs just using > luac to generate a loadable but pre-compiled Lua module? I could see > in the case of eLua that we could maybe use this as another way to use > Lua code and keep things in flash rather than in RAM on the heap. Are > there other advantages that aren't occurring to me? From the readme > it also sounds like it can generate stand-alone executables, which I > would presume would allow one to save some space? static compiling it. But the native code can be faster for somethings like loops. The static compiler does a little bit of analysis (mostly detecting integer vs floating point numbers) to help the C compiler remove type checks and produce faster code. sLua could do better at optimizing the code it generates, but I haven't had the time to add more code analysis. The way llvm-lua & sLua work is by compiling the Lua bytecodes for a Lua script into one function call for each bytecode (control structures like if/while/for/etc... require a bit more codegen work). Using one function per VM opcode (some opcodes have extra specialized functions) was the simplest way to static compile Lua scripts. For example a simple hello world script has 4 opcodes: GETGLOBAL 0, -1 ; print LOADK 1, -2 ; "hello, world" CALL 0 2 1 RETURN 0 1 those opcodes basically get compiled to: vm_OP_GETGLOBAL(L, constants_list, closure, 0, 0); vm_OP_LOADK(L, constants_list, 1, 1); retval1 = vm_OP_CALL(L, 0, 2, 1); retval2 = vm_OP_RETURN(L, 0, 1); return retval2; Those vm_OP_* functions are marked for inlining and they are included at the top of the generated C file: #include "lua_vm_ops.c" A good C compiler should be able to inline atleast some of the similar opcode functions and apply more optimizations based on the opcode's parameters. At the start of each compiled Lua function is some setup code to get the current function's Lua closure for access to the functions UpValues & constants. Also it seems that I haven't finished implementing support for generating a loadable modules. It looks like I just forgot to add the command line option to the compiler to have it generate the luaopen_<mod_name> function that is needed for a module. I might be able to finish that tonight. This method of static compiling Lua scripts might not be the best in terms of optimizations possibilities, but it was the simplest since it reuses the C code from the Lua VM code for those opcode functions. Another way would be to create a new Lua compiler that either directly generates C code or compiles to a more high-level IR language that provide better information for code analysis & optimization. -- Robert G. Jakabosky _______________________________________________ eLua-dev mailing list [hidden email] https://lists.berlios.de/mailman/listinfo/elua-dev hello.c (891 bytes) Download Attachment |
Free forum by Nabble | Edit this page |