Re-compiling with cross-compiler?

[jbsnyder]

The one last thing I think I need to do before getting LuaRPC into  
eLua itself is to deal with compiling for different platforms so that  
you can serialize functions between nodes.  The only problem is that  
if you string.dump a local function, that bytecode is already compiled  
for the local host.

I'm thinking it should be possible to somehow modify the dump code in  
the cross compiler to convert a pre-existing function into a dump that  
would be correct for a target device to which it is being sent.  I  
assume this would be easier (and more flexible) than somehow caching  
the original source representation and replaying it through the  
compiler later?  It would also be nice to be able to do things full-
circle where a function could be dumped on a host, loaded on a target,  
redumped on the target and reloaded on the host, or another machine  
that doesn't have the same environment parameters.

If anyone has any thoughts on this, I'd certainly appreciate them if  
they might help avoid rabbit holes that don't have an exit :-)

-jsnyder
_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[BogdanM]

eLua already has a patch for "cross-compiling", for details please check this page:

http://elua.berlios.de/beta/en/using.html#cross

I think you can work on this. My suggestion would be to serialize everything into a predefined format (for example "network order" for all numbers) and let the remote target unpack the dump itself (although it would need some kind of information about the machine you're sending the request from, but this kind of stuff can be easily embedded into the dump itself).

Best,
Bogdan

On Mon, Jun 15, 2009 at 8:08 PM, James Snyder <[hidden email]> wrote:

The one last thing I think I need to do before getting LuaRPC into
eLua itself is to deal with compiling for different platforms so that
you can serialize functions between nodes.  The only problem is that
if you string.dump a local function, that bytecode is already compiled
for the local host.

I'm thinking it should be possible to somehow modify the dump code in
the cross compiler to convert a pre-existing function into a dump that
would be correct for a target device to which it is being sent.  I
assume this would be easier (and more flexible) than somehow caching
the original source representation and replaying it through the
compiler later?  It would also be nice to be able to do things full-
circle where a function could be dumped on a host, loaded on a target,
redumped on the target and reloaded on the host, or another machine
that doesn't have the same environment parameters.

If anyone has any thoughts on this, I'd certainly appreciate them if
they might help avoid rabbit holes that don't have an exit :-)

-jsnyder
_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[jbsnyder]

On Jun 15, 2009, at 12:39 PM, Bogdan Marinescu wrote:

eLua already has a patch for "cross-compiling", for details please check this page:

http://elua.berlios.de/beta/en/using.html#cross

Right, I've been taking a look at that to try and figure out how to integrate this into the RPC code.

I think you can work on this. My suggestion would be to serialize everything into a predefined format (for example "network order" for all numbers) and let the remote target unpack the dump itself (although it would need some kind of information about the machine you're sending the request from, but this kind of stuff can be easily embedded into the dump itself).

I was thinking of having the clients exchange information about endianness and sizes when the connection is initiated.  I think I could just mimic the set of bytes used in the headers for a binary dump in order to get the right info exchanged.

Network order sounds like a good idea for parameters that are being exchanged.  I suppose whatever is done there should be consistent with how bytecode is handled as well.  My initial thought was to use header information exchanged when the connection is started to see who has what configuration (endianness, lua_Number type, etc..) and  only do translations if necessary, and have whoever supports more formats be responsible for putting things into alternate formats for less capable nodes (i.e.: a float machine can handle both float and int, an integer only one however would be the only one who can deal with float).  I may not get all of these cases covered for the initial version, but I would like it to at least work between an Intel 32 or 64 bit machine and a Cortex-M3 for starters.  I suppose one of the merits of just going with network ordering or standardized output is that if one were, say, broadcasting code over some network link to multiple machines at once, this would prevent having to do some sort of negotiation process.

Hmm... more experimentation and coding needed...

-jsnyder

--

James Snyder

Biomedical Engineering

Northwestern University

[hidden email]

http://fanplastic.org/key.txt

ph: [phone]

_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[jbsnyder]

Quick question, if someone else has a 64-bit system, could you try the following:

build the cross-compiler:

scons -f cross-lua.py

put the following code into test.lua:

function testfunc(x)

    return x^2

end

./luac -ccn float 64 -cce little -o abc.luac -s test.lua

./luac -ccn float 64 -cce little -o abc2.luac abc.luac

Does the second one blow up?

It works for me if I compile it as 32-bit (-m32), but not by default as a 64-bit binary.

I think that ldump.c doesn't necessary work quite correctly on LP64 systems...

http://www.ibm.com/developerworks/library/l-port64.html

I guess I should just use int32_t, uint32_t etc?

As always, things are more complicated than one might hope :-)

-jsnyder

On Jun 15, 2009, at 1:17 PM, James Snyder wrote:

On Jun 15, 2009, at 12:39 PM, Bogdan Marinescu wrote:

eLua already has a patch for "cross-compiling", for details please check this page:

http://elua.berlios.de/beta/en/using.html#cross

Right, I've been taking a look at that to try and figure out how to integrate this into the RPC code.

I think you can work on this. My suggestion would be to serialize everything into a predefined format (for example "network order" for all numbers) and let the remote target unpack the dump itself (although it would need some kind of information about the machine you're sending the request from, but this kind of stuff can be easily embedded into the dump itself).

I was thinking of having the clients exchange information about endianness and sizes when the connection is initiated.  I think I could just mimic the set of bytes used in the headers for a binary dump in order to get the right info exchanged.

Network order sounds like a good idea for parameters that are being exchanged.  I suppose whatever is done there should be consistent with how bytecode is handled as well.  My initial thought was to use header information exchanged when the connection is started to see who has what configuration (endianness, lua_Number type, etc..) and  only do translations if necessary, and have whoever supports more formats be responsible for putting things into alternate formats for less capable nodes (i.e.: a float machine can handle both float and int, an integer only one however would be the only one who can deal with float).  I may not get all of these cases covered for the initial version, but I would like it to at least work between an Intel 32 or 64 bit machine and a Cortex-M3 for starters.  I suppose one of the merits of just going with network ordering or standardized output is that if one were, say, broadcasting code over some network link to multiple machines at once, this would prevent having to do some sort of negotiation process.

Hmm... more experimentation and coding needed...

-jsnyder

--

James Snyder

Biomedical Engineering

Northwestern University

[hidden email]

http://fanplastic.org/key.txt

ph: [phone]

_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev