New linux platform for eLua

[Robert G. Jakabosky]

Attached is a patch to add a linux platform that allows eLua to be compiled
and tested as a linux app.

eLua will still use it's romfs/newlib/etc.. libraries.  The only linux
syscalls eLua uses are read/write/mmap.  read/write are used to emulate a
keyboard & monitor.  mmap is used to allocate a 64Kbyte block of ram for eLua
as memory.  You can configure the amount of memory in platform_conf.h

This platform is just to make it easier to use gdb for debugging.  It
shouldn't be to hard to add other simulated devices.

One bug I found is that longjmp is broken since it uses the 'cli' intruction
which causes eLua to segfault.  It should be possible to override the
setjmp/longjmp implementations and maybe pull in the one from linux.  If
someone looks into overrideing/replacing those symbols I could write a
setjmp/longjmp in assembly that should work.

Trying to run life.lua in eLua with only 64Kbyte will cause an out of memory
error and crash eLua.  If eLua is also compiled with the EGC patch life.lua
runs fine. :)

--
Robert G. Jakabosky

_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[BogdanM]

On Mon, May 11, 2009 at 3:37 PM, Robert G. Jakabosky <[hidden email]> wrote:

Attached is a patch to add a linux platform that allows eLua to be compiled
and tested as a linux app.

Excellent work, once again! :) I thought about this myself for a while, but I considered it too involved. Little did I know about the syscallx mechanism, although I should've been able to infer it after looking at the source of quite a few libcs. 

eLua will still use it's romfs/newlib/etc.. libraries.  The only linux
syscalls eLua uses are read/write/mmap.  read/write are used to emulate a
keyboard & monitor.  mmap is used to allocate a 64Kbyte block of ram for eLua
as memory.  You can configure the amount of memory in platform_conf.h 

This platform is just to make it easier to use gdb for debugging.  It
shouldn't be to hard to add other simulated devices.

One bug I found is that longjmp is broken since it uses the 'cli' intruction
which causes eLua to segfault.  It should be possible to override the
setjmp/longjmp implementations and maybe pull in the one from linux.  If
someone looks into overrideing/replacing those symbols I could write a
setjmp/longjmp in assembly that should work.

I did that simply by taking Newlib's longjmp source code, removing CLI/STI and converting it to NASM syntax. I also added it to the trunk, it's a very nice start for a full-featured eLua "simulator" on desktops. It seems to work and it can run life.lua now (partially because I have it 1M of memory :), since your patch is not included yet).

Trying to run life.lua in eLua with only 64Kbyte will cause an out of memory
error and crash eLua.  If eLua is also compiled with the EGC patch life.lua
runs fine. :)

And the good news just keep on coming ... :) Can't wait to get the time to play with EGC a bit myself.

Best,
Bogdan

_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[jbsnyder]

Very neat patch :-)  I've been looking at a number of different  
approaches including using qemu for emulating the ARM CPUs we use, but  
having a native linux binary would be easier to debug for sure.

I presume from taking a look at the sources that it is somewhat linux  
specific, so having a similar binary on OS X would require some work?

In addition to GDB if you're interested in using tracing you might  
want to checkout DTrace for linux:
http://www.crisp.demon.co.uk/blog/index.html

And dtrace probes for Lua:
http://lua-users.org/lists/lua-l/2007-06/msg00310.html

Thanks for all the patches Robert :-)

-jsnyder

On May 11, 2009, at 7:37 AM, Robert G. Jakabosky wrote:

>
> Attached is a patch to add a linux platform that allows eLua to be  
> compiled
> and tested as a linux app.
>
> eLua will still use it's romfs/newlib/etc.. libraries.  The only linux
> syscalls eLua uses are read/write/mmap.  read/write are used to  
> emulate a
> keyboard & monitor.  mmap is used to allocate a 64Kbyte block of ram  
> for eLua
> as memory.  You can configure the amount of memory in platform_conf.h
>
> This platform is just to make it easier to use gdb for debugging.  It
> shouldn't be to hard to add other simulated devices.
>
> One bug I found is that longjmp is broken since it uses the 'cli'  
> intruction
> which causes eLua to segfault.  It should be possible to override the
> setjmp/longjmp implementations and maybe pull in the one from  
> linux.  If
> someone looks into overrideing/replacing those symbols I could write a
> setjmp/longjmp in assembly that should work.
>
> Trying to run life.lua in eLua with only 64Kbyte will cause an out  
> of memory
> error and crash eLua.  If eLua is also compiled with the EGC patch  
> life.lua
> runs fine. :)
>
> --
> Robert G. Jakabosky
> <elua_linux.patch>_______________________________________________
> Elua-dev mailing list
> [hidden email]
> https://lists.berlios.de/mailman/listinfo/elua-dev

--
James Snyder
Biomedical Engineering
Northwestern University
[hidden email]
http://fanplastic.org/key.txt
ph: [phone]


_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[Robert G. Jakabosky]

The first copy of this e-mail I sent from the wrong e-mail address.  Please
ignore the other one if it still gets through.

On Monday 11, James Snyder wrote:
> Very neat patch :-)  I've been looking at a number of different
> approaches including using qemu for emulating the ARM CPUs we use, but
> having a native linux binary would be easier to debug for sure.

I haved looked at qemu, since it has a debugger, but I wanted to be able to
use valgrind/strace/etc.. tools for testing.  As a linux app. I can use all
the normal debugging tools that I normally use.

> I presume from taking a look at the sources that it is somewhat linux
> specific, so having a similar binary on OS X would require some work?

It should be possible to port it to other systems.  I was going to call the
platform "hosted" instead of linux but didn't since it includes code for
making linux syscalls.  Only the host.c file has the linux syscalls code, so
if you know how to make syscalls on OS X you could port it over.

I don't know how to pull in the minimal code needed from the host OS's
standard C library and not have it conflict with the newlib C library.

> In addition to GDB if you're interested in using tracing you might
> want to checkout DTrace for linux:
> http://www.crisp.demon.co.uk/blog/index.html
>
> And dtrace probes for Lua:
> http://lua-users.org/lists/lua-l/2007-06/msg00310.html

Thanks I didn't know someone was working on a port of DTrace to linux.  I had
looked arround for one about a year ago.


--
Robert G. Jakabosky
_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[jbsnyder]

Quick question about this again:

Are you using the i686 elf build of the toolchain to make this work?  
I'm trying to get it going on Darwin (Mac OS X), and I'm wondering if  
I can use newlib with a gcc toolchain that was originally set up for  
glibc, or if it needs to be a full toolchain built around newlib...

-jsnyder

On May 11, 2009, at 7:37 AM, Robert G. Jakabosky wrote:

>
> Attached is a patch to add a linux platform that allows eLua to be  
> compiled
> and tested as a linux app.
>
> eLua will still use it's romfs/newlib/etc.. libraries.  The only linux
> syscalls eLua uses are read/write/mmap.  read/write are used to  
> emulate a
> keyboard & monitor.  mmap is used to allocate a 64Kbyte block of ram  
> for eLua
> as memory.  You can configure the amount of memory in platform_conf.h
>
> This platform is just to make it easier to use gdb for debugging.  It
> shouldn't be to hard to add other simulated devices.
>
> One bug I found is that longjmp is broken since it uses the 'cli'  
> intruction
> which causes eLua to segfault.  It should be possible to override the
> setjmp/longjmp implementations and maybe pull in the one from  
> linux.  If
> someone looks into overrideing/replacing those symbols I could write a
> setjmp/longjmp in assembly that should work.
>
> Trying to run life.lua in eLua with only 64Kbyte will cause an out  
> of memory
> error and crash eLua.  If eLua is also compiled with the EGC patch  
> life.lua
> runs fine. :)
>
> --
> Robert G. Jakabosky
> <elua_linux.patch>_______________________________________________
> Elua-dev mailing list
> [hidden email]
> https://lists.berlios.de/mailman/listinfo/elua-dev

--
James Snyder
Biomedical Engineering
Northwestern University
[hidden email]
http://fanplastic.org/key.txt
ph: [phone]

_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[Robert G. Jakabosky]

First make sure you have the latest svn revision, since Bogdan renamed the
platform to "sim" and made it easier to add support for other host OS.  Still
use "cpu=linux" with the build command.

You will need the i686-elf toolchain built with newlib support.  Glibc is not
used when building the "sim" plantform, so the syscalls to the host OS need
to be manually written.  As far as I know you can't use a copy of the gcc
toolchain that was built for glibc.

On Wednesday 20, James Snyder wrote:

> Quick question about this again:
>
> Are you using the i686 elf build of the toolchain to make this work?
> I'm trying to get it going on Darwin (Mac OS X), and I'm wondering if
> I can use newlib with a gcc toolchain that was originally set up for
> glibc, or if it needs to be a full toolchain built around newlib...
>
> -jsnyder
>
> On May 11, 2009, at 7:37 AM, Robert G. Jakabosky wrote:
> > Attached is a patch to add a linux platform that allows eLua to be
> > compiled
> > and tested as a linux app.
> >
> > eLua will still use it's romfs/newlib/etc.. libraries.  The only linux
> > syscalls eLua uses are read/write/mmap.  read/write are used to
> > emulate a
> > keyboard & monitor.  mmap is used to allocate a 64Kbyte block of ram
> > for eLua
> > as memory.  You can configure the amount of memory in platform_conf.h
> >
> > This platform is just to make it easier to use gdb for debugging.  It
> > shouldn't be to hard to add other simulated devices.
> >
> > One bug I found is that longjmp is broken since it uses the 'cli'
> > intruction
> > which causes eLua to segfault.  It should be possible to override the
> > setjmp/longjmp implementations and maybe pull in the one from
> > linux.  If
> > someone looks into overrideing/replacing those symbols I could write a
> > setjmp/longjmp in assembly that should work.
> >
> > Trying to run life.lua in eLua with only 64Kbyte will cause an out
> > of memory
> > error and crash eLua.  If eLua is also compiled with the EGC patch
> > life.lua
> > runs fine. :)
> >
> > --
> > Robert G. Jakabosky
> > <elua_linux.patch>_______________________________________________
> > Elua-dev mailing list
> > [hidden email]
> > https://lists.berlios.de/mailman/listinfo/elua-dev
>
> --
> James Snyder
> Biomedical Engineering
> Northwestern University
> [hidden email]
> http://fanplastic.org/key.txt
> ph: [phone]
>
> _______________________________________________
> Elua-dev mailing list
> [hidden email]
> https://lists.berlios.de/mailman/listinfo/elua-dev

--
Robert G. Jakabosky
_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[BogdanM]

On Wed, May 20, 2009 at 9:58 PM, Robert G. Jakabosky <[hidden email]> wrote:

First make sure you have the latest svn revision, since Bogdan renamed the
platform to "sim" and made it easier to add support for other host OS.  Still
use "cpu=linux" with the build command.

This holds for Linux only, I bet OSX uses a different way to invoke syscalls. He'd need to use "cpu=osx" and implement the  hostif_osx.c file in src/platform/sim. How to do this, I have no idea :)

You will need the i686-elf toolchain built with newlib support.  Glibc is not
used when building the "sim" plantform, so the syscalls to the host OS need
to be manually written.  As far as I know you can't use a copy of the gcc
toolchain that was built for glibc.

I also don't think this is possible, gcc built with gliobc depends on glibc in a number of obscure ways (I'm sure the same holds for Newlib at least to some extent). I tried using the precompiled ncurses library from Linux in our sim executable to set up the terminal, and the linker blessed me with so many errors that I gave up instantly. I'm sure things would've been different if I tried to compile ncurses itself with i686-elf-gcc, but this is already more complex than needed.

Best,
Bogdan
 

On Wednesday 20, James Snyder wrote:
> Quick question about this again:
>
> Are you using the i686 elf build of the toolchain to make this work?
> I'm trying to get it going on Darwin (Mac OS X), and I'm wondering if
> I can use newlib with a gcc toolchain that was originally set up for
> glibc, or if it needs to be a full toolchain built around newlib...
>
> -jsnyder
>
> On May 11, 2009, at 7:37 AM, Robert G. Jakabosky wrote:

> > Attached is a patch to add a linux platform that allows eLua to be
> > compiled
> > and tested as a linux app.
> >
> > eLua will still use it's romfs/newlib/etc.. libraries.  The only linux
> > syscalls eLua uses are read/write/mmap.  read/write are used to
> > emulate a
> > keyboard & monitor.  mmap is used to allocate a 64Kbyte block of ram
> > for eLua
> > as memory.  You can configure the amount of memory in platform_conf.h
> >
> > This platform is just to make it easier to use gdb for debugging.  It
> > shouldn't be to hard to add other simulated devices.
> >
> > One bug I found is that longjmp is broken since it uses the 'cli'
> > intruction
> > which causes eLua to segfault.  It should be possible to override the
> > setjmp/longjmp implementations and maybe pull in the one from
> > linux.  If
> > someone looks into overrideing/replacing those symbols I could write a
> > setjmp/longjmp in assembly that should work.
> >
> > Trying to run life.lua in eLua with only 64Kbyte will cause an out
> > of memory
> > error and crash eLua.  If eLua is also compiled with the EGC patch
> > life.lua
> > runs fine. :)
> >
> > --
> > Robert G. Jakabosky

> > <elua_linux.patch>_______________________________________________
> > Elua-dev mailing list
> > [hidden email]
> > https://lists.berlios.de/mailman/listinfo/elua-dev
>
> --
> James Snyder
> Biomedical Engineering
> Northwestern University
> [hidden email]
> http://fanplastic.org/key.txt
> ph: [phone]
>
> _______________________________________________
> Elua-dev mailing list
> [hidden email]
> https://lists.berlios.de/mailman/listinfo/elua-dev

--
Robert G. Jakabosky

_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[jbsnyder]

On May 20, 2009, at 2:51 PM, Bogdan Marinescu wrote:

On Wed, May 20, 2009 at 9:58 PM, Robert G. Jakabosky <[hidden email]> wrote:

First make sure you have the latest svn revision, since Bogdan renamed the
platform to "sim" and made it easier to add support for other host OS.  Still
use "cpu=linux" with the build command.

This holds for Linux only, I bet OSX uses a different way to invoke syscalls. He'd need to use "cpu=osx" and implement the  hostif_osx.c file in src/platform/sim. How to do this, I have no idea :)

Yeah, I was making a _darwin one (since OS X is built on top of a BSD-like set of sources called Darwin)

You will need the i686-elf toolchain built with newlib support.  Glibc is not
used when building the "sim" plantform, so the syscalls to the host OS need
to be manually written.  As far as I know you can't use a copy of the gcc
toolchain that was built for glibc.

I also don't think this is possible, gcc built with gliobc depends on glibc in a number of obscure ways (I'm sure the same holds for Newlib at least to some extent). I tried using the precompiled ncurses library from Linux in our sim executable to set up the terminal, and the linker blessed me with so many errors that I gave up instantly. I'm sure things would've been different if I tried to compile ncurses itself with i686-elf-gcc, but this is already more complex than needed.

OK.  The reason I was asking was because I did manage to get the i686-elf toolchain to build on OS X, but I then realized that OS X works with Mach-O binaries, not ELF.  I tried building a normal GCC for OS X w/ the only difference being using newlib as opposed to glibc, but newlib's build process died on me, so I'll have to figure out what went wrong there.  I can't find reference to anyone trying to use newlib as a C library on OS X (not terribly surprising, I guess).

There are elf loading environments for OS X like vx32 (http://pdos.csail.mit.edu/~baford/vm/) and Google's Native Client (http://code.google.com/p/nativeclient/), but those might be more of a pain than its worth.

Two things that Native Client has going for it at the moment are that it uses Newlib as its C library, and that I suppose you could use it to make eLua run in a web browser (pointless, but neat?)

Why can't things be simple? :-) (and why no ELF support, Apple?)

--

James Snyder

Biomedical Engineering

Northwestern University

[hidden email]

http://fanplastic.org/key.txt

ph: [phone]

_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

[Robert G. Jakabosky]

On Wednesday 20, Bogdan Marinescu wrote:
> I also don't think this is possible, gcc built with gliobc depends on glibc
> in a number of obscure ways (I'm sure the same holds for Newlib at least to
> some extent). I tried using the precompiled ncurses library from Linux in
> our sim executable to set up the terminal, and the linker blessed me with
> so many errors that I gave up instantly. I'm sure things would've been
> different if I tried to compile ncurses itself with i686-elf-gcc, but this
> is already more complex than needed.

You could build a wrapper program that is built with the normal host gcc
toolchain.  It would setup the terminal then run eLua in a child process that
shares stdin/stdout/stderr with the wrapper process.  If you don't need to
reset the terminal back to some normal settings on exit, then exec() could be
used to replace the wrapper process with eLua.

Also here is a program that can convert ELF objects to Mach-O objects:
http://www.agner.org/optimize/objconv.zip

The OSX syscalls would still need to be written, but atleast objconv might
make the ELF eLua binary runnable.  As a test compile (with i686-elf-gcc) a
simple program that only has a long loop in main().  If you can convert that
elf binary to a runnable Mach-O binary, then it should work for eLua once the
syscalls are written.

--
Robert G. Jakabosky
_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev