A Few Models For Dealing With ADC Samples

[BogdanM]

Alright, a word of warning first: I'm not by any means a specialist in ADCs,
and I only used them in a buch of applications. That said, I do have some
ideas about how they work and what they're supposed to do, and a few ideas
of my own about the design of the module itself. The basic principle here is
KISS. I want the modules to cover all the needed functionality, but no more
than that. I definitely don't want a full blown API, but rather to keep
everything as lightweight as possible. For example, you might have noticed
that none of the modules currently exported by eLua use the "object
notation". This is intentional, as the "standard" notation is still more
widely used in different instances of embedded programming, plus it keeps
the implementation simpler. Of course, keeping a good balance between a
minimal API and a good functionality is an art in itself, something that I
don't pretend to master (but I try anyway :) ).
With that in mind, I think that our current ADC module does enough to
qualify as functional

- single sampling
- multiple sampling at a given frequency (which is very deterministic, being
handled in the backend with interrupts which are quite accurate).
- moving average filtering (quite important in real-life applications).

In the future, once interrupt handlers are implemented in eLua, we'll also
support double buffering, in that we'll have a circular sample buffer, and
an "interrupt" will be fired when that buffer is half full. This would allow
certain kind of applications (like data loggers) to be written very easily.
I think this is a very powerful model.

Please see below my other comment.


Single Function to Request and Return:
> 1. Simple Model
> call function to start sampling
> function waits for sampling to finish
> function returns samples
>

Not acceptable if this is the only model, perfectly OK if it coexists with
other (asynchronous) models.\

2. Non-blocking Model (this was present in an earlier version of the ADC
> module)
> call function to start sampling, function returns
> do some other stuff
> call function again to get samples from the sampling period, and start next
> set of sampling
>

Interesting, but let's see the next one.

Separate Request & Return Functions:
> 3. Separated Request/Return Model
> call function to start sampling, function returns
> do other work
> use another function to get samples in the buffer (this can be done while
> sampling is still happening)
>

I assume you're talking about a circular buffer here, and this is perfectly
OK with me. Add an overflow flag for those situations when you don't collect
the samples on time and you're all set (and it also matches (to some extent)
the interrupt model I was talking about earlier. Maybe even add a function
that would return "true" when half (or an user-specified fraction of the
buffer) is full).
This is the kind of funtionality I get from an Advantech acquisition board
on a PC (well, from its library API actually), and it was all I needed to
write a very precise acquisition application that archives data over very
long periods of time, plot real time graphs, thinks like this. I'm sure the
model would work equally well on embedded.

- samples collected in the background, other work can be done while samples
> are collected
> - samples collected using fairly deterministic timer-based interrupt system
> - longer deterministic periods can be used where one can request 100
> samples, but only pull in 5 at a time while sampling is happening, while
> still getting low timing jitter
>

For this situations, it might be a good idea to add a "poll" function that
would return 'true' each time a predefined number of samples (6 in your
example) are available. Kinda like the "return true when half of the buffer
is full" idea, but more generic.

- could be extended to support indefinite periods of sampling with buffer
> wrapping if samples aren't pulled out fast enough, but when you do need
> samples you can pull out up to the length of the buffer and they're still
> pretty well-timed
>

Of course you can. You can't escape circular buffers. If your process is
slow enough, it's all you need for continuous acquisition. If it's too fast,
it doesn't matter how long your buffers are, you will get into overflows at
some point.


> - more complicated, including dealing with buffer sizes
>

Why is that? We already know how to specify buffer sizes.


> - multiple commands needed to initiate/get samples
>

That's not really a problem.
In conclusion, this (3) would be my preffered model, with a catch: let's not
make it more complex than it needs to be.

> One alternative that I've not mentioned here is that there be two
> functional models for getting samples.  If one wants, one could have a
> simple getsamples method that follows model 1, and a second pair that
> follows model 3. i.e.:
>
> channels = adc.channel(0, 3, 17, 21)
>
> -- Style 1: Simple Blocking Model
> a = channels:getsamples(count)
>
> -- Style 2: Separated Model
> channels:requestsamples(count, timer, frequency)
> channels:returnsamples(count)
>

Let's go for option 3, the one that doesn't use an object notation at all
(for reasons I mentioned at the beginning of this thread).

Best,
Bogdan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.berlios.de/pipermail/elua-dev/attachments/20090219/24768332/attachment.html 

[jbsnyder]

On Feb 19, 2009, at 4:48 AM, Bogdan Marinescu wrote:

> Alright, a word of warning first: I'm not by any means a specialist  
> in ADCs, and I only used them in a buch of applications. That said,  
> I do have some ideas about how they work and what they're supposed  
> to do, and a few ideas of my own about the design of the module  
> itself. The basic principle here is KISS. I want the modules to  
> cover all the needed functionality, but no more than that. I  
> definitely don't want a full blown API, but rather to keep  
> everything as lightweight as possible. For example, you might have  
> noticed that none of the modules currently exported by eLua use the  
> "object notation". This is intentional, as the "standard" notation  
> is still more widely used in different instances of embedded  
> programming, plus it keeps the implementation simpler. Of course,  
> keeping a good balance between a minimal API and a good  
> functionality is an art in itself, something that I don't pretend to  
> master (but I try anyway :) ).
> With that in mind, I think that our current ADC module does enough  
> to qualify as functional
>
> - single sampling
> - multiple sampling at a given frequency (which is very  
> deterministic, being handled in the backend with interrupts which  
> are quite accurate).
> - moving average filtering (quite important in real-life  
> applications).
>
> In the future, once interrupt handlers are implemented in eLua,  
> we'll also support double buffering, in that we'll have a circular  
> sample buffer, and an "interrupt" will be fired when that buffer is  
> half full. This would allow certain kind of applications (like data  
> loggers) to be written very easily. I think this is a very powerful  
> model.
>
> Please see below my other comment.
>
>
> Single Function to Request and Return:
> 1. Simple Model
> call function to start sampling
> function waits for sampling to finish
> function returns samples
>
> Not acceptable if this is the only model, perfectly OK if it  
> coexists with other (asynchronous) models.\
>
> 2. Non-blocking Model (this was present in an earlier version of the  
> ADC module)
> call function to start sampling, function returns
> do some other stuff
> call function again to get samples from the sampling period, and  
> start next set of sampling
>
> Interesting, but let's see the next one.
>
> Separate Request & Return Functions:
> 3. Separated Request/Return Model
> call function to start sampling, function returns
> do other work
> use another function to get samples in the buffer (this can be done  
> while sampling is still happening)
>
> I assume you're talking about a circular buffer here, and this is  
> perfectly OK with me. Add an overflow flag for those situations when  
> you don't collect the samples on time and you're all set (and it  
> also matches (to some extent) the interrupt model I was talking  
> about earlier. Maybe even add a function that would return "true"  
> when half (or an user-specified fraction of the buffer) is full).

This is, by far, the most common model I've seen for a lot of higher  
level APIs (initiate sampling, samples buffer, pull samples out with  
another function).  As an example, since I've worked with it before,  
and the docs are online, here's an overview of how MATLAB deals with  
the wide range of  ADC/DAC/GPIO devices it talks to:
http://www.mathworks.com/access/helpdesk/help/toolbox/daq/index.html?/access/helpdesk/help/toolbox/daq/f14-17602.html&http://www.mathworks.com/products/daq/

An overflow flag would be a good idea.  The partial filling flag is  
interesting as well.  Currently we've got a function that will return  
the number of samples available, so adding something to poll for a  
specified number would be trivial.


>
> This is the kind of funtionality I get from an Advantech acquisition  
> board on a PC (well, from its library API actually), and it was all  
> I needed to write a very precise acquisition application that  
> archives data over very long periods of time, plot real time graphs,  
> thinks like this. I'm sure the model would work equally well on  
> embedded.

Do you mean the flag/partial filling mechanism, or the two function  
request/return model?  I'd be interested in hearing some general usage  
case information from your application so that I can keep that in mind  
while doing the implementation.

>
>
> - samples collected in the background, other work can be done while  
> samples are collected
> - samples collected using fairly deterministic timer-based interrupt  
> system
> - longer deterministic periods can be used where one can request 100  
> samples, but only pull in 5 at a time while sampling is happening,  
> while still getting low timing jitter
>
> For this situations, it might be a good idea to add a "poll"  
> function that would return 'true' each time a predefined number of  
> samples (6 in your example) are available. Kinda like the "return  
> true when half of the buffer is full" idea, but more generic.

Right, I was actually thinking of implementing the "half-full" thing  
this way anyways.

>
>
> - could be extended to support indefinite periods of sampling with  
> buffer wrapping if samples aren't pulled out fast enough, but when  
> you do need samples you can pull out up to the length of the buffer  
> and they're still pretty well-timed
>
> Of course you can. You can't escape circular buffers. If your  
> process is slow enough, it's all you need for continuous  
> acquisition. If it's too fast, it doesn't matter how long your  
> buffers are, you will get into overflows at some point.

Right. Honestly to make it run continuously all I have to do is turn  
off the function that tells the sequencer to stop once the number of  
requested samples are collected.

>
>
> - more complicated, including dealing with buffer sizes
>
> Why is that? We already know how to specify buffer sizes.

I didn't mean much more complicated :-)  The complication here, at  
least in terms of the way I'm thinking is if we want to minimize  
buffer size as much as possible but be flexible if more samples are  
coming in and we want to grow the buffer without destroying  
uncollected samples.  I know this isn't essential, though :-) Right  
now, buffers can be resized whenever all samples are pulled out, or  
when a request is made that exceeds current buffer size (any samples  
remaining in the buffer at these times will get nuked though).

>
>
> - multiple commands needed to initiate/get samples
>
> That's not really a problem.
> In conclusion, this (3) would be my preffered model, with a catch:  
> let's not make it more complex than it needs to be.

Right, I'd be open to suggestions of where to pare things down.  But  
before I do that, here are a few things that people have brought up or  
have come up that might be useful to consider:

1. Have a function that returns only one sample at a time from the  
buffer (if available). Depending on whether we keep the "amend/adjust  
the table" approach, this does make it so that if all I want is one  
sample, I can do that without the overhead of creating a new table.  
This could also be a function that could request the sample itself as  
well.

2. Locking multiple channels to a clock was discussed during the  
object model discourse, so that one could have a bunch of channels  
grab each sample in lock step.  Doing this without the object model is  
less attractive, but I could see doing something like the following:

adc.chain(0,3,8)

Where any property modification or sampling request you make of one is  
made of all of them, so if I then do:

adc.setsmoothing(0,16)

it is actually equivalent to:

adc.setsmoothing(0,16)
adc.setsmoothing(3,16)
adc.setsmoothing(8,16)

but more importantly, when I do adc.sample(0)

they all fire off conversions sequentially on the C side where the  
delay between conversions would be much more minimal than doing a loop  
in lua.  I think I would keep the reads from each channel separate  
though (not return all channels' results with one adc.getsamples(),  
though that could be done too)

If you do the same for burst requests, they all get triggered by the  
same clock.

If the idea of chaining seems a little unsavory, since it could make  
code hard to interpret later if you miss the chaining command, we  
could also accept a table of channel_ids as opposed to just single  
channel ids for at least burst and sample.  So you might call  
something like this:

adc.burst({0, 3},16,0,5000)

this would initiate sampling on channels 0 and 3 to collect 16  
samples, using timer 0, at 5000 Hz...

That doesn't even require a new function :-)

3. Do we keep the modify-the-table approach? I think having number 1  
above that returns single samples at least removes this need for  
something that samples like adcscope.lua does.

Table modification is a bit ugly in the API, but it could be done in a  
limited fashion.  Either only appending could be allowed, or going  
slightly more flexible, modifications could be made starting at an  
offset, but only if the number of elements to modify were included.  I  
think of this as sort a syntactically ugly version of using slices  
(like in Python) to modify data :-)

Or, we can just return new tables for now :-)  Any thoughts about what  
to do in the immediate future with this?  We could have global tables  
to represent returned samples, but I don't like that a huge amount.

>
>
> One alternative that I've not mentioned here is that there be two  
> functional models for getting samples.  If one wants, one could have  
> a simple getsamples method that follows model 1, and a second pair  
> that follows model 3. i.e.:
>
> channels = adc.channel(0, 3, 17, 21)
>
> -- Style 1: Simple Blocking Model
> a = channels:getsamples(count)
>
> -- Style 2: Separated Model
> channels:requestsamples(count, timer, frequency)
> channels:returnsamples(count)
>
> Let's go for option 3, the one that doesn't use an object notation  
> at all (for reasons I mentioned at the beginning of this thread).

I do like the syntax of the object model approach, but consistency is  
a pretty good argument.  The other thing is that if one wanted to, one  
could certainly build a lua-only object model implementation on top of  
the simple API. :-)

Best.

-jsnyder

--
James Snyder
Biomedical Engineering
Northwestern University
jbsnyder at fanplastic.org
http://fanplastic.org/key.txt
ph: [phone]

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.berlios.de/pipermail/elua-dev/attachments/20090219/13dfeecf/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
Url : https://lists.berlios.de/pipermail/elua-dev/attachments/20090219/13dfeecf/attachment-0001.pgp 

[Andre Carregal-2]

(changing gears... screech...)

> On Feb 19, 2009, at 4:48 AM, Bogdan Marinescu wrote:
> Alright, a word of warning first: I'm not by any means a specialist in ADCs,
> and I only used them in a buch of applications. That said, I do have some
> ideas about how they work and what they're supposed to do, and a few ideas
> of my own about the design of the module itself. The basic principle here is
> KISS. I want the modules to cover all the needed functionality, but no more
> than that. I definitely don't want a full blown API, but rather to keep
> everything as lightweight as possible. For example, you might have noticed
> that none of the modules currently exported by eLua use the "object
> notation". This is intentional, as the "standard" notation is still more
> widely used in different instances of embedded programming, plus it keeps
> the implementation simpler. Of course, keeping a good balance between a
> minimal API and a good functionality is an art in itself, something that I
> don't pretend to master (but I try anyway :) ).
> With that in mind, I think that our current ADC module does enough to
> qualify as functional

I hear you, crystal clear. I really came to the list with a very
different point of view and this thread has been a good lesson of
scale and politics for me... :o)

As a side note, it's interesting how "small" is a subjective concept.
For an astronomer, the moon is probably small... for me, something
around 5k was small, but once you find that people are fighting for
bytes suddenly the scale changes.

I apologize for not spending time enough to get a feeling of the
project bias and culture before manifesting my ideas and I promise to
be more careful in the future.

> Let's go for option 3, the one that doesn't use an object notation at all
> (for reasons I mentioned at the beginning of this thread).
>
> On Thu, Feb 19, 2009 at 2:58 PM, James Snyder <jbsnyder at fanplastic.org> wrote:
> I do like the syntax of the object model approach, but consistency is a
> pretty good argument.  The other thing is that if one wanted to, one could
> certainly build a lua-only object model implementation on top of the simple
> API. :-)

Consistency is enough for me too, let's leave the higher level APIs
for those with extra space.

Andr?

[jbsnyder]

On Feb 21, 2009, at 3:55 PM, Andre Carregal wrote:

> (changing gears... screech...)
>
>> On Feb 19, 2009, at 4:48 AM, Bogdan Marinescu wrote:
>> Alright, a word of warning first: I'm not by any means a specialist  
>> in ADCs,
>> and I only used them in a buch of applications. That said, I do  
>> have some
>> ideas about how they work and what they're supposed to do, and a  
>> few ideas
>> of my own about the design of the module itself. The basic  
>> principle here is
>> KISS. I want the modules to cover all the needed functionality, but  
>> no more
>> than that. I definitely don't want a full blown API, but rather to  
>> keep
>> everything as lightweight as possible. For example, you might have  
>> noticed
>> that none of the modules currently exported by eLua use the "object
>> notation". This is intentional, as the "standard" notation is still  
>> more
>> widely used in different instances of embedded programming, plus it  
>> keeps
>> the implementation simpler. Of course, keeping a good balance  
>> between a
>> minimal API and a good functionality is an art in itself, something  
>> that I
>> don't pretend to master (but I try anyway :) ).
>> With that in mind, I think that our current ADC module does enough to
>> qualify as functional
>
> I hear you, crystal clear. I really came to the list with a very
> different point of view and this thread has been a good lesson of
> scale and politics for me... :o)
>
> As a side note, it's interesting how "small" is a subjective concept.
> For an astronomer, the moon is probably small... for me, something
> around 5k was small, but once you find that people are fighting for
> bytes suddenly the scale changes.
>
> I apologize for not spending time enough to get a feeling of the
> project bias and culture before manifesting my ideas and I promise to
> be more careful in the future.

There's no problem with discussing these things though (with some  
limitations) :-)

Even if the lower level APIs don't adopt an object style of notation,  
there were a couple of things that came out there that were  
interesting.  I think that the grouping or atomizing of channels is  
something that may prove to be a very useful feature, which may not  
have been communicated had we not gone down that path.  Also, I  
personally have only started recently working with Lua, so getting  
perspectives how others would want to implement something like this is  
quite valuable.

KISS is a difficult philosophy to adhere to when there are innumerable  
ways to do even simple things.  Given that, I think that brainstorming  
a bit about the different ways to do things can help in figuring out  
what does and doesn't make sense about any given approach.  It also  
should help with preventing cases where someone looks at this, many  
months down the road, and wonders why some particularly braindead  
decision was made :-)

Also, giving good reasons to rip out useless or bizarre functionality  
should help keep this smaller.

All of that said, things do need to be balanced.  We don't want  
something that needs even a megabyte of RAM to run in, so some ideas  
will generally always be off the table :-)

--
James Snyder
Biomedical Engineering
Northwestern University
jbsnyder at fanplastic.org
http://fanplastic.org/key.txt
ph: [phone]

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.berlios.de/pipermail/elua-dev/attachments/20090221/cc2b8cbe/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
Url : https://lists.berlios.de/pipermail/elua-dev/attachments/20090221/cc2b8cbe/attachment-0001.pgp 

[BogdanM]

On Sat, Feb 21, 2009 at 11:55 PM, Andre Carregal <carregal at pobox.com> wrote:

> (changing gears... screech...)


You really should take care of that gearbox of yours :)

> I hear you, crystal clear. I really came to the list with a very
> different point of view and this thread has been a good lesson of
> scale and politics for me... :o)
>
> As a side note, it's interesting how "small" is a subjective concept.
> For an astronomer, the moon is probably small... for me, something
> around 5k was small, but once you find that people are fighting for
> bytes suddenly the scale changes.
>
> I apologize for not spending time enough to get a feeling of the
> project bias and culture before manifesting my ideas and I promise to
> be more careful in the future.

Oh, there's nothing at all to apologize here, and your contribution is most
welcome. There is a lot to learn from a constructive critique (or simply a
debate on a given subject), even when the discussion happens between people
that come from very different backgrounds. And BTW, to that extent, your
solution is excellent. I didn't think of it as "wrong" for a single moment,
because it's not. But I believe that a project must follow a certain design
philosophy, otherwise it won't get very far. This is why I have to assume my
tiranic posture at times and mercilessly dictate the evolution of the
project :) But other than that, this kind of discussions are strongly
encouraged on this list.

Best,
Bogdan

>
> > Let's go for option 3, the one that doesn't use an object notation at all
> > (for reasons I mentioned at the beginning of this thread).
> >
> > On Thu, Feb 19, 2009 at 2:58 PM, James Snyder <jbsnyder at fanplastic.org>
> wrote:
> > I do like the syntax of the object model approach, but consistency is a
> > pretty good argument.  The other thing is that if one wanted to, one
> could
> > certainly build a lua-only object model implementation on top of the
> simple
> > API. :-)
>
> Consistency is enough for me too, let's leave the higher level APIs
> for those with extra space.
>
> Andr?
> _______________________________________________
> Elua-dev mailing list
> Elua-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/elua-dev
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.berlios.de/pipermail/elua-dev/attachments/20090222/7710db88/attachment.html