Re: SVN access to eLua

classic Classic list List threaded Threaded
3 messages Options
mpthompson mpthompson
Reply | Threaded
Open this post in threaded view
|

Re: SVN access to eLua

In getting developer SVN access the eLua, Bogdan and I wanted to move a
portion of our discussion to the developer list.  Portions of the thread
are copied below for reference.

Bogdan Marinescu wrote:
> ... some more thoughts that I forgot about :)
>
> - embedded flash controller and file system: generally speaking, we like
> to make eLua components as generic as possible, so let's discuss your
> file system first. Ideally we'll separate the generic parts from the
> platform-specific parts, thus making it really portable.

The code is split into two parts -- the top level file system code is
fairly generic and the bottom level code in a separate file which deals
directly with the AT91 embedded flash controller.  The code is really
green (just written a few days ago) so there is no urgent need to get it
in or anything.  I'll post a link to the files where others can examine
them (and make suggestions).  Over time, if others think the file system
would be useful it can be integrated in a way that makes it fairly
portable between platforms that may implement embedded nand flash memory
in different ways.

> - USB drivers? what for? I mean, I know what they are :), but do you
> have a specific usage scenario for USB+eLua ?

With regards to USB, I think it's useful for the AT91 chips in that the
dev boards for those chips are commonly plugged into the PC via the USB
port.  Serial ports are harder to come by on PCs these days and almost
non-existent on laptops (OK, USB to serial cables aren't too hard to
find).  I believe enabling a build switch to make the eLua shell console
active through the USB port rather than the serial port is indeed
useful.  I would suggest making USB shell access a compile time switch
that can be set for those platforms which support a USB controller.  It
should be easy to make portable in that from eLua perspective it looks
pretty much just like a serial port -- only the underlying driver kept
in the platform specific code is much different.

> - zmodem: nice to have, but I see it happening only on platforms with
> LOTS of RAM, as it is not at all a resource-friendly protocol (talking
> from an embedded programmer's point of view).

For zmodem, the implementation I ported from the AVRFreaks site was
intended for the AVR ATmega128 (16k SRAM) and uses very little RAM --
less than the current eLua xmodem code in fact.  It does a good job of
implementing the protocol byte-by-byte rather than buffering entire
frames.  That said, zmodem doesn't offer much over xmodem for the way
eLua is using it.  I needed it for bulk transfer of files to the file
system on the chip.

These are just things that I need for my specific use and I would be
happy to make any pieces available to others who would find them useful
as well.

Mike

> It might be wise to move this discussion to the eLua dev list.
>
> Best,
> Bogdan
>
> On Thu, Apr 9, 2009 at 12:19 AM, Bogdan Marinescu
> <[hidden email] <mailto:[hidden email]>> wrote:
>
>     Hi Mike,
>
>     You should have developer access now, please check.
>
>     Best,
>     Bogdan
 >

>     On Thu, Apr 9, 2009 at 12:03 AM, Mike Thompson <[hidden email]
>     <mailto:[hidden email]>> wrote:
>
>         Bogdan and Dado,
>
>         Can I be given developer access to the eLua SVN repository?
>
>         I now have a number of things I would like to contribute to eLua
>         including platform support for AT91SAM7S, embedded flash
>         controller and file system, USB drivers and zmodem code.
>
>         Thanks,
>
>         Mike Thompson

_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev
BogdanM BogdanM
Reply | Threaded
Open this post in threaded view
|

Re: SVN access to eLua


The code is split into two parts -- the top level file system code is
fairly generic and the bottom level code in a separate file which deals
directly with the AT91 embedded flash controller.  The code is really
green (just written a few days ago) so there is no urgent need to get it
in or anything.  I'll post a link to the files where others can examine
them (and make suggestions).  Over time, if others think the file system
would be useful it can be integrated in a way that makes it fairly
portable between platforms that may implement embedded nand flash memory
in different ways.

Excellent, this is exactly what we're looking for.
 
> - USB drivers? what for? I mean, I know what they are :), but do you
> have a specific usage scenario for USB+eLua ?

With regards to USB, I think it's useful for the AT91 chips in that the
dev boards for those chips are commonly plugged into the PC via the USB
port.  Serial ports are harder to come by on PCs these days and almost
non-existent on laptops (OK, USB to serial cables aren't too hard to
find).  I believe enabling a build switch to make the eLua shell console
active through the USB port rather than the serial port is indeed
useful.  I would suggest making USB shell access a compile time switch
that can be set for those platforms which support a USB controller.  It
should be easy to make portable in that from eLua perspective it looks
pretty much just like a serial port -- only the underlying driver kept
in the platform specific code is much different.

Sure, console over serial port over USB sounds just fine to me :) About the build switch, we should probably keep it platform-specific, don't know if we can include it in the platform interface.
 
For zmodem, the implementation I ported from the AVRFreaks site was
intended for the AVR ATmega128 (16k SRAM) and uses very little RAM --
less than the current eLua xmodem code in fact.  It does a good job of
implementing the protocol byte-by-byte rather than buffering entire
frames.  That said, zmodem doesn't offer much over xmodem for the way
eLua is using it.  I needed it for bulk transfer of files to the file
system on the chip.

Oh. In this case, by all means, let us have it :) ZMODEM would be better than XMODEM for eLua, as it contains information such as the file name being transfered and (very important) its size.

Thanks,
Bogdan
 
These are just things that I need for my specific use and I would be
happy to make any pieces available to others who would find them useful
as well.

Mike

> It might be wise to move this discussion to the eLua dev list.
>
> Best,
> Bogdan
>
> On Thu, Apr 9, 2009 at 12:19 AM, Bogdan Marinescu
> <[hidden email] <mailto:[hidden email]>> wrote:
>
>     Hi Mike,
>
>     You should have developer access now, please check.
>
>     Best,
>     Bogdan
 >
>     On Thu, Apr 9, 2009 at 12:03 AM, Mike Thompson <[hidden email]
>     <mailto:[hidden email]>> wrote:
>
>         Bogdan and Dado,
>
>         Can I be given developer access to the eLua SVN repository?
>
>         I now have a number of things I would like to contribute to eLua
>         including platform support for AT91SAM7S, embedded flash
>         controller and file system, USB drivers and zmodem code.
>
>         Thanks,
>
>         Mike Thompson

_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev


_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev
mpthompson mpthompson
Reply | Threaded
Open this post in threaded view
|

Re: SVN access to eLua

Bogdan Marinescu wrote:

>
>     The code is split into two parts -- the top level file system code is
>     fairly generic and the bottom level code in a separate file which deals
>     directly with the AT91 embedded flash controller.  The code is really
>     green (just written a few days ago) so there is no urgent need to get it
>     in or anything.  I'll post a link to the files where others can examine
>     them (and make suggestions).  Over time, if others think the file system
>     would be useful it can be integrated in a way that makes it fairly
>     portable between platforms that may implement embedded nand flash memory
>     in different ways.
>
>
> Excellent, this is exactly what we're looking for.

To make this discussion a little less theoretical, the code for my eLua
file system for embedded flash controller and NAND flash on the AT91 Arm
chips is included in the link below:

http://home.comcast.net/~michael.p.thompson/elua/elua_efcfs.zip

This is something I wrote out of necessity fairly quickly after reading
about how other NAND file systems such as YAFFs work at a theoretical
level.  The top level portion in the file 'efcfs.c' implements the file
system and calls the five functions in the lower level file 'efcio.c'
which in turn use the efc/flashd library calls provided by Atmel for the
AT91 Arm MCUs (that code is not included in the zip file above).

In a nutshell, the driver works as follows.  It uses the existing linker
symbols _efixed, _erelocate and _srelocate to determine the unused
portion of on-board flash memory.  These pages should be initialized to
all 0xFF -- something I do by adding the '--pad-to 0x00140000 --gap-fill
0xff' options to the arm-elf-objcopy command that creates the final
binary from the elf image.  For each 256 byte flash page I reserve 12
bytes for page usage including the page sequence, file id, file block
and file block size information -- 244 bytes for file data.  Writes to a
file handle are buffered in SRAM until the block needs to be flushed to
a page in flash.  Algorithms then kick in make sure that the write
occurs in a round robin fashion to the underlying pages in flash memory
so pages get used evenly.

One aspect of this file system is that it currently exercises flash wear
leveling to the dynamic (unused) portion of the file system only -- not
to the static portion of the file system.  This means that if a file
consumes 50% of the file system only the pages in the other 50% of file
system be recycled in a round robin manner as other files are written
and deleted.  Changing things so that both the static and dynamic parts
of the file system are wear leveled at the cost of some additional I/O
shouldn't be hard, but I just haven't done it yet.

Also, unlike other NAND file systems such as YAFFS, this file system
consumes very little SRAM overhead other than the required page buffers.
  Instead of keeping file system booking structures in SRAM my functions
simply scan the addressed NAND to determine the next free page to write
and such.  This doesn't make it fast, but with the limited flash on most
Arm processors making only a few hundred pages (128K to 256K typically)
at most available to the file system it seems plenty fast enough.
Finally, the file system only supports a root level directory and
filenames are limited to 32 characters with null termination (this can
be changed easily in a #define).

I'm not familiar with how flash works on non-AT91SAM7 Arm processors,
but hopefully just changes to 'efcio.c' would be needed to support other
families of Arm processors with on-board flash.  External NAND chips
could possibly be supported, but that is beyond the scope of the
intentions for this simple file system.  In that case, YAFFs would be
much more suited as it covers error correction and will work with the
out-of-band data on each flash page.

This code is pretty green right now and I would love to have feedback on
improvements that could be made.

Mike Thompson
_______________________________________________
Elua-dev mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/elua-dev