USB - HID & MSD

classic Classic list List threaded Threaded
15 messages Options
John Hind John Hind
Reply | Threaded
Open this post in threaded view
|

USB - HID & MSD

Having reluctantly concluded that Lua on microcontrollers (in real projects) is still a year or two off, I have been looking again at the 8-bit market accepting I will have to code in C.

 

I was just blown away by what you can do with microcontrollers and the USB interface these days. I got a small microchip development board using their PIC18F46J50 controller which unfortunately has only 64k program and 3k data memory, so hopeless for Lua. However it came with a simple test app loaded which presented a composite HID and MSD USB device interface (no external components required) and it had a full file system implementation (including folders and long file names) using spare program Flash memory on the chip. This is presented over the MSD interface and appears just like a USB flash drive on the computer. As the icing on the cake, there is also a HID based bootloader program so new firmware can be loaded over USB. All the software (PC and PIC) is free and open source and the development system is also free (express editions of MS Visual Studio for the PC end, free MPLAB and C18 for the PIC) except that the PIC C compiler loses some optimisations after 60 days. Rather wonderfully, the PC sample software for the HID side was supplied in the USB flash drive so you could run it directly from the device itself! I was able to modify both PC and PIC software really easily and complete a full build cycle for both machines within less than an hour. (I cannot help contrasting this with the two days it took for my first eLua build on Linux, mostly admittedly the problems of compiling the development tools).

 

Once the memory sizes increase sufficiently this would make a really cool eLua platform as you could just develop the scripts directly in the “USB Flash Drive”. The use of MSD and HID is really cool because it eliminates the need for custom USB drivers on the PC – drivers for these protocols are already in every modern OS. Also using HID means you can emulate keyboards, mice and game controllers directly, so producing devices that work with existing software.

 


_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev
Andreas Krinke Andreas Krinke
Reply | Threaded
Open this post in threaded view
|

Re: USB - HID & MSD

John Hind wrote:
[snip]

This reminds me of a nice platform I found the other day:

http://mbed.org

It uses a NXP LPC1786 ARM Cortex-M3 at 100 Mhz with 512kB of flash and
64kB of SRAM. It behaves like an USB stick as well and you just have to
save the firmware to it.

The development tools are available online. So you can use it whereever
you are (your code is online as well).

At the moment, it's available for $60 or €60.

Regards,
Andreas
_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev
Dado Sutter Dado Sutter
Reply | Threaded
Open this post in threaded view
|

Re: USB - HID & MSD

In reply to this post by John Hind
Hello eLuers :)
   Thanks for the good topics and comments John.
   We are indeed using eLua on some real-projects already but yes, still with some extra custom code. I think we will have a good speed-up again on the development cycle, after the release of v0.6, expected for soon now.
   Toolchain building has indeed shown to be a barrier for most entry-level users. We expect to help a bit with this in several fronts, like better tutorials, use of really-easy-to-install toolchains (like CodeSourcery), "transplants" of already-built toolchains (some may violate licence aspects though) and with an "eLua Web Builder", a web service that will allow one to configure your build options, upload (and maintain on the server) ROM File System files, select targets, toolchains, generate an eLua image and download it thru a regular browser.
   It is actually under development here, the initial tests are promissing and we hope to make it available for evaluation soon.
   Bogdan still dreams on bringing eLua to the 8bits world too but unfortunately, our days are still limited to 24hrs too :)

Best
Dado



On Fri, Sep 25, 2009 at 11:45, John Hind <[hidden email]> wrote:

Having reluctantly concluded that Lua on microcontrollers (in real projects) is still a year or two off, I have been looking again at the 8-bit market accepting I will have to code in C.

 

I was just blown away by what you can do with microcontrollers and the USB interface these days. I got a small microchip development board using their PIC18F46J50 controller which unfortunately has only 64k program and 3k data memory, so hopeless for Lua. However it came with a simple test app loaded which presented a composite HID and MSD USB device interface (no external components required) and it had a full file system implementation (including folders and long file names) using spare program Flash memory on the chip. This is presented over the MSD interface and appears just like a USB flash drive on the computer. As the icing on the cake, there is also a HID based bootloader program so new firmware can be loaded over USB. All the software (PC and PIC) is free and open source and the development system is also free (express editions of MS Visual Studio for the PC end, free MPLAB and C18 for the PIC) except that the PIC C compiler loses some optimisations after 60 days. Rather wonderfully, the PC sample software for the HID side was supplied in the USB flash drive so you could run it directly from the device itself! I was able to modify both PC and PIC software really easily and complete a full build cycle for both machines within less than an hour. (I cannot help contrasting this with the two days it took for my first eLua build on Linux, mostly admittedly the problems of compiling the development tools).

 

Once the memory sizes increase sufficiently this would make a really cool eLua platform as you could just develop the scripts directly in the “USB Flash Drive”. The use of MSD and HID is really cool because it eliminates the need for custom USB drivers on the PC – drivers for these protocols are already in every modern OS. Also using HID means you can emulate keyboards, mice and game controllers directly, so producing devices that work with existing software.

 


_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev



_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev
BogdanM BogdanM
Reply | Threaded
Open this post in threaded view
|

Re: USB - HID & MSD


   Thanks for the good topics and comments John.
   We are indeed using eLua on some real-projects already but yes, still with some extra custom code. I think we will have a good speed-up again on the development cycle, after the release of v0.6, expected for soon now.
   Toolchain building has indeed shown to be a barrier for most entry-level users. We expect to help a bit with this in several fronts, like better tutorials, use of really-easy-to-install toolchains (like CodeSourcery), "transplants" of already-built toolchains (some may violate licence aspects though)

I doubt we'll ever have a problem with toolchains and our license, but I might be wrong.
 
and with an "eLua Web Builder", a web service that will allow one to configure your build options, upload (and maintain on the server) ROM File System files, select targets, toolchains, generate an eLua image and download it thru a regular browser.
   It is actually under development here, the initial tests are promissing and we hope to make it available for evaluation soon. 
   Bogdan still dreams on bringing eLua to the 8bits world too but unfortunately, our days are still limited to 24hrs too :)

Ah, so the days in America have the same structure as the day in Europe? Weird :)
Lua on 8bit is possible, and I'm getting more and more into this. However, a complete rewrite of the VM is needed for this, and it's not an easy task. That's not to say that it can't be done, though :)

Best,
Bogdan



On Fri, Sep 25, 2009 at 11:45, John Hind <[hidden email]> wrote:

Having reluctantly concluded that Lua on microcontrollers (in real projects) is still a year or two off, I have been looking again at the 8-bit market accepting I will have to code in C.

 

I was just blown away by what you can do with microcontrollers and the USB interface these days. I got a small microchip development board using their PIC18F46J50 controller which unfortunately has only 64k program and 3k data memory, so hopeless for Lua. However it came with a simple test app loaded which presented a composite HID and MSD USB device interface (no external components required) and it had a full file system implementation (including folders and long file names) using spare program Flash memory on the chip. This is presented over the MSD interface and appears just like a USB flash drive on the computer. As the icing on the cake, there is also a HID based bootloader program so new firmware can be loaded over USB. All the software (PC and PIC) is free and open source and the development system is also free (express editions of MS Visual Studio for the PC end, free MPLAB and C18 for the PIC) except that the PIC C compiler loses some optimisations after 60 days. Rather wonderfully, the PC sample software for the HID side was supplied in the USB flash drive so you could run it directly from the device itself! I was able to modify both PC and PIC software really easily and complete a full build cycle for both machines within less than an hour. (I cannot help contrasting this with the two days it took for my first eLua build on Linux, mostly admittedly the problems of compiling the development tools).

 

Once the memory sizes increase sufficiently this would make a really cool eLua platform as you could just develop the scripts directly in the “USB Flash Drive”. The use of MSD and HID is really cool because it eliminates the need for custom USB drivers on the PC – drivers for these protocols are already in every modern OS. Also using HID means you can emulate keyboards, mice and game controllers directly, so producing devices that work with existing software.

 


_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev



_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev



_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev
Dean Hall Dean Hall
Reply | Threaded
Open this post in threaded view
|

eLua on 8-bit

Bogdan, Is this something you want to do individually or as an open  
process?  Because I would be *very* interested in discussing eLua on 8-
bit with you.  I may even be able to contribute after I release PyMite  
r10 this fall.

!!Dean

On Sep 25, 2009, at 14:21 , Bogdan Marinescu wrote:

> Lua on 8bit is possible, and I'm getting more and more into this.  
> However, a complete rewrite of the VM is needed for this, and it's  
> not an easy task. That's not to say that it can't be done, though :)

_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev
BogdanM BogdanM
Reply | Threaded
Open this post in threaded view
|

Re: eLua on 8-bit

I'm fully open to new ideas and code contributions. Plus, you already rewrote a large part of the Python VM if i'm not mistaken, so we can only benefit from your experience. We can discuss the ideas part right after 0.6, which will happen soon (for real this time!). Right now, everything is still very much undefined. The idea in itself is very simple: rewrite the Lua VM, probably leaving out some of its features (garbage collection is the first that comes to mind, most likely others too) but adding some new features which would benefit 8-bit architectures greatly (like the possibility to use external serial SRAM/FRAM memories to compensate for the limited memory of the host MCU, or the possibility to execute Lua bytecode directly from an external serial EEPROM, without having to load it to RAM first). Optimize everything for space instead of speed.
This sounds very simplistic, and might be so in the real world. Other more "radical" thoughts include more functional limitations (like limited operations on a table), recompiling the Lua bytecode to a form more suitable for 8-bit systems, and other things that sound w

On Fri, Sep 25, 2009 at 10:54 PM, Dean Hall <[hidden email]> wrote:
Bogdan, Is this something you want to do individually or as an open
process?  Because I would be *very* interested in discussing eLua on 8-
bit with you.  I may even be able to contribute after I release PyMite
r10 this fall.

!!Dean

On Sep 25, 2009, at 14:21 , Bogdan Marinescu wrote:

> Lua on 8bit is possible, and I'm getting more and more into this.
> However, a complete rewrite of the VM is needed for this, and it's
> not an easy task. That's not to say that it can't be done, though :)

_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev


_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev
BogdanM BogdanM
Reply | Threaded
Open this post in threaded view
|

Re: eLua on 8-bit

...that sound way too dumb to share :) (sorry about that).
So yes, there's much to talk about here, and I can't wait to finish with 0.6 and get the opportunity to put my ideas in order.

Best,
Bogdan

On Fri, Sep 25, 2009 at 11:55 PM, Bogdan Marinescu <[hidden email]> wrote:
I'm fully open to new ideas and code contributions. Plus, you already rewrote a large part of the Python VM if i'm not mistaken, so we can only benefit from your experience. We can discuss the ideas part right after 0.6, which will happen soon (for real this time!). Right now, everything is still very much undefined. The idea in itself is very simple: rewrite the Lua VM, probably leaving out some of its features (garbage collection is the first that comes to mind, most likely others too) but adding some new features which would benefit 8-bit architectures greatly (like the possibility to use external serial SRAM/FRAM memories to compensate for the limited memory of the host MCU, or the possibility to execute Lua bytecode directly from an external serial EEPROM, without having to load it to RAM first). Optimize everything for space instead of speed.
This sounds very simplistic, and might be so in the real world. Other more "radical" thoughts include more functional limitations (like limited operations on a table), recompiling the Lua bytecode to a form more suitable for 8-bit systems, and other things that sound w


On Fri, Sep 25, 2009 at 10:54 PM, Dean Hall <[hidden email]> wrote:
Bogdan, Is this something you want to do individually or as an open
process?  Because I would be *very* interested in discussing eLua on 8-
bit with you.  I may even be able to contribute after I release PyMite
r10 this fall.

!!Dean

On Sep 25, 2009, at 14:21 , Bogdan Marinescu wrote:

> Lua on 8bit is possible, and I'm getting more and more into this.
> However, a complete rewrite of the VM is needed for this, and it's
> not an easy task. That's not to say that it can't be done, though :)

_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev



_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev
John Hind John Hind
Reply | Threaded
Open this post in threaded view
|

Re: eLua on 8-bit

In reply to this post by Dean Hall
What would be the objective of going 8 bit? As I see it, the roadblock at
the moment is the memory footprint and if anything 8-bit would make that
harder because there is generally fewer 8-bit chips with large memory (and
large memory 8-bit devices are likely to be just as expensive as 16 or 32
bit ones). On the other hand a port to Microchip's 16 bit devices would be
nice - they are about the only manufacturer still selling modern low pin
count devices in PDIP packages. Trouble is they are not ARM at 32-bit and
their memory sizes tend to top out too low.

For me, the real breakpoint comes when you can sell chips with the firmware
already loaded. If you could just plug one of these into any computer and
have it present as an MSD flash drive you could simply copy Lua scripts onto
it and even edit them in place (and you get cross-platform capability for
free). If we could lose the need to bootstrap the chips from source code,
there would be a much larger market for them! Pre-cooked toolchains would
help, but ideally this is a Lua system so users should not have to mess
about with C at all!

Still seems to me we are either waiting for chips with more memory or
seriously going the cross-development route, stripping back on the chip to
bytecode runtime only and parsing on the host computer. But even there, lack
of RAM is likely to be a constraint. (As we have discussed before, there is
no point going multi-chip because if you go that route you may as well put
Linux and standard Lua on it.)

> -----Original Message-----
> From: [hidden email] [mailto:elua-dev-
> [hidden email]] On Behalf Of Dean Hall
> Sent: 25 September 2009 20:55
> To: eLua Users and Development List
> Subject: [eLua-dev] eLua on 8-bit
>
> Bogdan, Is this something you want to do individually or as an open
> process?  Because I would be *very* interested in discussing eLua on 8-
> bit with you.  I may even be able to contribute after I release PyMite
> r10 this fall.
>
> !!Dean
>
> On Sep 25, 2009, at 14:21 , Bogdan Marinescu wrote:
>
> > Lua on 8bit is possible, and I'm getting more and more into this.
> > However, a complete rewrite of the VM is needed for this, and it's
> > not an easy task. That's not to say that it can't be done, though :)
>
> _______________________________________________
> Elua-dev mailing list
> [hidden email]
> https://lists.berlios.de/mailman/listinfo/elua-dev

_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev
Dean Hall Dean Hall
Reply | Threaded
Open this post in threaded view
|

Re: eLua on 8-bit

John, the goal isn't to go 8-bit just for the sake of 8-bits.  It's to  
reduce the memory requirements of the VM so it can run on  
architectures with less RAM (such as 8-bit devices).  I have made a  
reduced Python VM (www.pythononachip.org) and was able to get it to  
run in 4 KB with features equivalent to a BASIC language.  I've added  
features expected in a Python VM (like classes and threads) and now  
the minimum footprint is 8 KB.  Considering the footprint of the  
standard desktop Python can be reduced by this much, I believe a  
language such as Lua--designed to be petite from the very beginning--
stands a good chance of being reduced to a 4-8 KB footprint as well.

!!Dean

On Sep 26, 2009, at 05:01 , John Hind wrote:

> What would be the objective of going 8 bit? As I see it, the  
> roadblock at
> the moment is the memory footprint and if anything 8-bit would make  
> that
> harder because there is generally fewer 8-bit chips with large  
> memory (and
> large memory 8-bit devices are likely to be just as expensive as 16  
> or 32
> bit ones). On the other hand a port to Microchip's 16 bit devices  
> would be
> nice - they are about the only manufacturer still selling modern low  
> pin
> count devices in PDIP packages. Trouble is they are not ARM at 32-
> bit and
> their memory sizes tend to top out too low.
>
> For me, the real breakpoint comes when you can sell chips with the  
> firmware
> already loaded. If you could just plug one of these into any  
> computer and
> have it present as an MSD flash drive you could simply copy Lua  
> scripts onto
> it and even edit them in place (and you get cross-platform  
> capability for
> free). If we could lose the need to bootstrap the chips from source  
> code,
> there would be a much larger market for them! Pre-cooked toolchains  
> would
> help, but ideally this is a Lua system so users should not have to  
> mess
> about with C at all!
>
> Still seems to me we are either waiting for chips with more memory or
> seriously going the cross-development route, stripping back on the  
> chip to
> bytecode runtime only and parsing on the host computer. But even  
> there, lack
> of RAM is likely to be a constraint. (As we have discussed before,  
> there is
> no point going multi-chip because if you go that route you may as  
> well put
> Linux and standard Lua on it.)

_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev
BogdanM BogdanM
Reply | Threaded
Open this post in threaded view
|

Re: eLua on 8-bit

On Sun, Sep 27, 2009 at 7:41 AM, Dean Hall <[hidden email]> wrote:
John, the goal isn't to go 8-bit just for the sake of 8-bits.  It's to
reduce the memory requirements of the VM so it can run on
architectures with less RAM (such as 8-bit devices).  

Yes, this is pretty much the point. Plus, with LuaRPC, you can do a lot of interesting stuff on small devices too :) You probably can't reduce the code size significantly without reducing the functionality quite a bit too, but then again, most of the time you'd be fine with reduced functionality in the real world. One of the ideas I had is to define a (small) number of different "profiles", or, in other words, different levels of conformance with "regular" Lua. Then, obviously, a smaller device would implement a simpler profile, and more feature-rich devices would naturally go towards a more complex profile.
 
I have made a
reduced Python VM (www.pythononachip.org) and was able to get it to
run in 4 KB with features equivalent to a BASIC language.  I've added
features expected in a Python VM (like classes and threads) and now
the minimum footprint is 8 KB.  Considering the footprint of the
standard desktop Python can be reduced by this much, I believe a
language such as Lua--designed to be petite from the very beginning--
stands a good chance of being reduced to a 4-8 KB footprint as well.

I don't know anything about the Python VM, but I can see a Lua VM running in 8k (quite limited though) or maybe even 4k (severely limited :) ). Given that the code density on a 8-bit CPU is generally better than what you can find on an ARM, we can expect surprises in this area.
One of the things that I really want to do is to let the VM execute code from arbitrary memories (for example SPI/I2C EEPROMs) and use different types of RAMs (for example SPI/I2C SRAMs :) ). Or simply use a serial FRAM that can work both as a SRAM and an EEPROM. These are small, relatively inexpensive devices, and having a VM that can take advantage of them can greatly expand the MCU capabilities.

Best,
Bogdan

On Sep 26, 2009, at 05:01 , John Hind wrote:

> What would be the objective of going 8 bit? As I see it, the
> roadblock at
> the moment is the memory footprint and if anything 8-bit would make
> that
> harder because there is generally fewer 8-bit chips with large
> memory (and
> large memory 8-bit devices are likely to be just as expensive as 16
> or 32
> bit ones). On the other hand a port to Microchip's 16 bit devices
> would be
> nice - they are about the only manufacturer still selling modern low
> pin
> count devices in PDIP packages. Trouble is they are not ARM at 32-
> bit and
> their memory sizes tend to top out too low.
>
> For me, the real breakpoint comes when you can sell chips with the
> firmware
> already loaded. If you could just plug one of these into any
> computer and
> have it present as an MSD flash drive you could simply copy Lua
> scripts onto
> it and even edit them in place (and you get cross-platform
> capability for
> free). If we could lose the need to bootstrap the chips from source
> code,
> there would be a much larger market for them! Pre-cooked toolchains
> would
> help, but ideally this is a Lua system so users should not have to
> mess
> about with C at all!
>
> Still seems to me we are either waiting for chips with more memory or
> seriously going the cross-development route, stripping back on the
> chip to
> bytecode runtime only and parsing on the host computer. But even
> there, lack
> of RAM is likely to be a constraint. (As we have discussed before,
> there is
> no point going multi-chip because if you go that route you may as
> well put
> Linux and standard Lua on it.)

_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev


_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev
Dean Hall Dean Hall
Reply | Threaded
Open this post in threaded view
|

Re: eLua on 8-bit

I achieved this in the PyMite VM by defining a "memory space" type/
enum.  I did this because my initial target was AVR (a Harvard  
architecture) and I had to be able to tell if the static bytecode-
compiled code image was in Program Memory (Flash) or RAM.  I  
abstracted this to other memory spaces like EEPROM and SEEPROM, but I  
haven't implemented them yet.  PyMite users are wanting access to a  
file system, so MEMSPACE_FS may be my next memory space to implement.

As an implementation note: when a PyMite function executes, the  
execution frame is in RAM, of course, but the function's bytecodes  
remain in its original memory space.  That is, I don't copy any  
function's bytecode into RAM.  I just read it from the memory space on-
demand.  Considering most functions have about 1-2 KB of bytecode, and  
there can be a dozen functions stacked up in the call-tree; leaving  
bytecode in its memory space saves considerable amounts of RAM.

!!Dean

On Sep 27, 2009, at 13:17 , Bogdan Marinescu wrote:

> One of the things that I really want to do is to let the VM execute  
> code from arbitrary memories (for example SPI/I2C EEPROMs) and use  
> different types of RAMs (for example SPI/I2C SRAMs :) ). Or simply  
> use a serial FRAM that can work both as a SRAM and an EEPROM. These  
> are small, relatively inexpensive devices, and having a VM that can  
> take advantage of them can greatly expand the MCU capabilities.

_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev
John Hind John Hind
Reply | Threaded
Open this post in threaded view
|

Re: eLua on 8-bit

In reply to this post by BogdanM

I bow to your superior experience (and intellect!) because I do not even really understand what we mean by “8-bit” or “16-bit” anymore. I suppose in Lua it is the size of the Number type, but Microchip’s “8-Bit” devices actually have 16-bit instruction words and 20-bit memory addressing! Most ARM devices also have the 16-Bit instruction set even though they are called “32-Bit”. As far as I know, most code density comparisons give tens of percent differences not hundreds or thousands. Also why would it save RAM – my understanding of the large RAM requirement of Lua (relatively speaking) comes from its reliance on tables? I suppose if your Boolean data type is 8-bits rather than 16 it does save some wastage though!

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Bogdan Marinescu
Sent: 27 September 2009 19:17
To: eLua Users and Development List
Subject: Re: [eLua-dev] eLua on 8-bit

 

On Sun, Sep 27, 2009 at 7:41 AM, Dean Hall <[hidden email]> wrote:

John, the goal isn't to go 8-bit just for the sake of 8-bits.  It's to
reduce the memory requirements of the VM so it can run on
architectures with less RAM (such as 8-bit devices).  


Yes, this is pretty much the point. Plus, with LuaRPC, you can do a lot of interesting stuff on small devices too :) You probably can't reduce the code size significantly without reducing the functionality quite a bit too, but then again, most of the time you'd be fine with reduced functionality in the real world. One of the ideas I had is to define a (small) number of different "profiles", or, in other words, different levels of conformance with "regular" Lua. Then, obviously, a smaller device would implement a simpler profile, and more feature-rich devices would naturally go towards a more complex profile.
 

I have made a
reduced Python VM (www.pythononachip.org) and was able to get it to
run in 4 KB with features equivalent to a BASIC language.  I've added
features expected in a Python VM (like classes and threads) and now
the minimum footprint is 8 KB.  Considering the footprint of the
standard desktop Python can be reduced by this much, I believe a
language such as Lua--designed to be petite from the very beginning--
stands a good chance of being reduced to a 4-8 KB footprint as well.


I don't know anything about the Python VM, but I can see a Lua VM running in 8k (quite limited though) or maybe even 4k (severely limited :) ). Given that the code density on a 8-bit CPU is generally better than what you can find on an ARM, we can expect surprises in this area.
One of the things that I really want to do is to let the VM execute code from arbitrary memories (for example SPI/I2C EEPROMs) and use different types of RAMs (for example SPI/I2C SRAMs :) ). Or simply use a serial FRAM that can work both as a SRAM and an EEPROM. These are small, relatively inexpensive devices, and having a VM that can take advantage of them can greatly expand the MCU capabilities.

Best,
Bogdan

On Sep 26, 2009, at 05:01 , John Hind wrote:

> What would be the objective of going 8 bit? As I see it, the
> roadblock at
> the moment is the memory footprint and if anything 8-bit would make
> that
> harder because there is generally fewer 8-bit chips with large
> memory (and
> large memory 8-bit devices are likely to be just as expensive as 16
> or 32
> bit ones). On the other hand a port to Microchip's 16 bit devices
> would be
> nice - they are about the only manufacturer still selling modern low
> pin
> count devices in PDIP packages. Trouble is they are not ARM at 32-
> bit and
> their memory sizes tend to top out too low.
>
> For me, the real breakpoint comes when you can sell chips with the
> firmware
> already loaded. If you could just plug one of these into any
> computer and
> have it present as an MSD flash drive you could simply copy Lua
> scripts onto
> it and even edit them in place (and you get cross-platform
> capability for
> free). If we could lose the need to bootstrap the chips from source
> code,
> there would be a much larger market for them! Pre-cooked toolchains
> would
> help, but ideally this is a Lua system so users should not have to
> mess
> about with C at all!
>
> Still seems to me we are either waiting for chips with more memory or
> seriously going the cross-development route, stripping back on the
> chip to
> bytecode runtime only and parsing on the host computer. But even
> there, lack
> of RAM is likely to be a constraint. (As we have discussed before,
> there is
> no point going multi-chip because if you go that route you may as
> well put
> Linux and standard Lua on it.)

_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

 


_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev
Robert G. Jakabosky Robert G. Jakabosky
Reply | Threaded
Open this post in threaded view
|

Re: eLua on 8-bit

In reply to this post by BogdanM

I would also be interested in helping with the development of a low memory
footprint LuaVM.  I have been thinking about writting out the rules/logic of
the LuaVM in Lua code, then using plugable backends that generate C code to
implement the VM.  This would allow turning features on/off, or switch
between a Mark-and-Sweep collector to a reference count system.

I have written a Lua bindings generator that allows me to easily create binds
for all my C-style objects.  Below is an example of how the bindings to the
GD graphics library would be described.  I am thinking about releasing the
bindings generator as open source.

A similar code generator could be used to generate different sized LuaVMs.
Using the same description of the LuaVM logic would help keep the different
VMs compatible (minus any features that are turned off).

Also if an 16/8bit LuaVM needed a different C API, a bindings generator like
this would help bind the different VMs to the same library code.

-- file "gd_api.lua"
c_module "gd" {
use_globals = true,
hide_meta_info = true,
include "gd.h",
subfiles {
  "gdImage.lua"
}
}

-- file "gdImage.lua"
object "gdImage" {
  include "gd.h",
  method_new {
    c_call "gdImage *" "gdImageCreate" { "int", "sx", "int", "sy" }
  },
  method_delete {
    c_call "void" "gdImageDestroy" {}
  },
  method "color_allocate" {
    c_call "void" "gdImageColorAllocate"
      { "int", "r", "int", "g", "int", "b" }
  },
  method "line" {
    c_call "void" "gdImageLine"
      { "int", "x1", "int", "y1", "int", "x2", "int", "y2", "int", "colour" }
  },
  method "toPNG" {
    var_in { "const char *", "name" },
    c_source [[
  FILE *pngout = fopen( ${name}, "wb");
  gdImagePng(${this}, pngout);
  fclose(pngout);
]]
  },
}

Note: c_module, include, object, method, method_new, method_delete, subfiles,
c_source, var_in, c_call, c_source  are all functions that either return
another function or return a table.  When processed by the bindings generator
the above code generates a tree of records (tables with record type that
matches those functions).

--
Robert G. Jakabosky
_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev
BogdanM BogdanM
Reply | Threaded
Open this post in threaded view
|

Re: eLua on 8-bit

In reply to this post by BogdanM
On Sun, Sep 27, 2009 at 9:32 PM, John Hind <[hidden email]> wrote:

I bow to your superior experience (and intellect!)

I suspect you're overestimating me :)

because I do not even really understand what we mean by “8-bit” or “16-bit” anymore.

It's more a problem of symbols for me. Saying that something "works on 8-bit" is largely equivalent in my mind to saying "we have this piece of software that requires so little memory that it can run even on 8-bit micros". Other that that, you're definitely right; with an ANSI C compiler, you can compile standard Lua on any 8-bit architecture with sufficient resources.

I suppose in Lua it is the size of the Number type, but Microchip’s “8-Bit” devices actually have 16-bit instruction words and 20-bit memory addressing!

Yes, but:

- very few of them have an external memory interface.
- if they do, you still can't run code from an external memory, thus the limitation becomes program memory space instead of data memory space (it's kinda hard to fit Lua in the limited memory space of most 8-bit MCUs)
- they have a Harvard architecture, so you'll have lots of problems with "const". Which sometimes means that the "const" data ends up in RAM, which is a huge penalty.

Most ARM devices also have the 16-Bit instruction set even though they are called “32-Bit”. As far as I know, most code density comparisons give tens of percent differences not hundreds or thousands. Also why would it save RAM – my understanding of the large RAM requirement of Lua (relatively speaking) comes from its reliance on tables?

Exactly. This is why I was saying that a complete rewrite with the VM is needed to make Lua less RAM and Flash hungry. That's not to say that I have a good idea on how to replace tables at the moment :), even with accepting reduced functionality, but I'm working on this. And this isn't by far the only thing an 8-bit oriented VM must be aware of.

I suppose if your Boolean data type is 8-bits rather than 16 it does save some wastage though!

Having different storage for different data types is definitely a good idea, but from my observations this is quite hard to do with the Lua VM (including the bytecode, which dosesn't make provisions for different data sizes) . Still, something to think about :)

Best,
Bogdan

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Bogdan Marinescu
Sent: 27 September 2009 19:17


To: eLua Users and Development List
Subject: Re: [eLua-dev] eLua on 8-bit

 

On Sun, Sep 27, 2009 at 7:41 AM, Dean Hall <[hidden email]> wrote:

John, the goal isn't to go 8-bit just for the sake of 8-bits.  It's to
reduce the memory requirements of the VM so it can run on
architectures with less RAM (such as 8-bit devices).  


Yes, this is pretty much the point. Plus, with LuaRPC, you can do a lot of interesting stuff on small devices too :) You probably can't reduce the code size significantly without reducing the functionality quite a bit too, but then again, most of the time you'd be fine with reduced functionality in the real world. One of the ideas I had is to define a (small) number of different "profiles", or, in other words, different levels of conformance with "regular" Lua. Then, obviously, a smaller device would implement a simpler profile, and more feature-rich devices would naturally go towards a more complex profile.
 

I have made a
reduced Python VM (www.pythononachip.org) and was able to get it to
run in 4 KB with features equivalent to a BASIC language.  I've added
features expected in a Python VM (like classes and threads) and now
the minimum footprint is 8 KB.  Considering the footprint of the
standard desktop Python can be reduced by this much, I believe a
language such as Lua--designed to be petite from the very beginning--
stands a good chance of being reduced to a 4-8 KB footprint as well.


I don't know anything about the Python VM, but I can see a Lua VM running in 8k (quite limited though) or maybe even 4k (severely limited :) ). Given that the code density on a 8-bit CPU is generally better than what you can find on an ARM, we can expect surprises in this area.
One of the things that I really want to do is to let the VM execute code from arbitrary memories (for example SPI/I2C EEPROMs) and use different types of RAMs (for example SPI/I2C SRAMs :) ). Or simply use a serial FRAM that can work both as a SRAM and an EEPROM. These are small, relatively inexpensive devices, and having a VM that can take advantage of them can greatly expand the MCU capabilities.

Best,
Bogdan

On Sep 26, 2009, at 05:01 , John Hind wrote:

> What would be the objective of going 8 bit? As I see it, the
> roadblock at
> the moment is the memory footprint and if anything 8-bit would make
> that
> harder because there is generally fewer 8-bit chips with large
> memory (and
> large memory 8-bit devices are likely to be just as expensive as 16
> or 32
> bit ones). On the other hand a port to Microchip's 16 bit devices
> would be
> nice - they are about the only manufacturer still selling modern low
> pin
> count devices in PDIP packages. Trouble is they are not ARM at 32-
> bit and
> their memory sizes tend to top out too low.
>
> For me, the real breakpoint comes when you can sell chips with the
> firmware
> already loaded. If you could just plug one of these into any
> computer and
> have it present as an MSD flash drive you could simply copy Lua
> scripts onto
> it and even edit them in place (and you get cross-platform
> capability for
> free). If we could lose the need to bootstrap the chips from source
> code,
> there would be a much larger market for them! Pre-cooked toolchains
> would
> help, but ideally this is a Lua system so users should not have to
> mess
> about with C at all!
>
> Still seems to me we are either waiting for chips with more memory or
> seriously going the cross-development route, stripping back on the
> chip to
> bytecode runtime only and parsing on the host computer. But even
> there, lack
> of RAM is likely to be a constraint. (As we have discussed before,
> there is
> no point going multi-chip because if you go that route you may as
> well put
> Linux and standard Lua on it.)

_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

 


_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev



_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev
jbsnyder jbsnyder
Reply | Threaded
Open this post in threaded view
|

Re: USB - HID & MSD

In reply to this post by Andreas Krinke
It is quite a nice device :-)

I've actually got one, although it is a pre-release version that sues  
an LPC2368.  The only thing is, I've yet to finish porting to it since  
the build environment provided is an online compiler (armcc, no code  
size limit that  I know of?) and I haven't worked out all of the  
newlib -> ARM C library hooks.

It's possible to use a GCC toolchain with the device, because first  
UART is exposed as a CDC, but the abstraction is provided by using a  
secondary MCU on the board (which also does the flash filesystem).  
The main downside if you don't use their library is that you use read/
write for the onboard flash and you lose near-free portability if they  
change microcontroller devices (which you get if you write to their  
API).

FYI, PyMite (Dean Hall's project: http://code.google.com/p/python-on-a-chip/ 
) does already run on this device.

I WILL finish the port since it's a neat platform, and I think it  
would also be good to have an example of how to make eLua work without  
Newlib :-)

-jsnyder

On Sep 25, 2009, at 1:00 PM, Andreas Krinke wrote:

> John Hind wrote:
> [snip]
>
> This reminds me of a nice platform I found the other day:
>
> http://mbed.org
>
> It uses a NXP LPC1786 ARM Cortex-M3 at 100 Mhz with 512kB of flash and
> 64kB of SRAM. It behaves like an USB stick as well and you just have  
> to
> save the firmware to it.
>
> The development tools are available online. So you can use it  
> whereever
> you are (your code is online as well).
>
> At the moment, it's available for $60 or €60.
>
> Regards,
> Andreas
> _______________________________________________
> Elua-dev mailing list
> [hidden email]
> https://lists.berlios.de/mailman/listinfo/elua-dev
--
James Snyder
Biomedical Engineering
Northwestern University
[hidden email]
http://fanplastic.org/key.txt
ph: 847.448.0386





_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev

smime.p7s (5K) Download Attachment