A Few Models For Dealing With ADC Samples

[Andre Carregal-2]

On Wed, Feb 18, 2009 at 1:08 AM, James Snyder <jbsnyder at fanplastic.org> wrote:

> One alternative that I've not mentioned here is that there be two functional
> models for getting samples.  If one wants, one could have a simple
> getsamples method that follows model 1, and a second pair that follows model
> 3. i.e.:
>
> channels = adc.channel(0, 3, 17, 21)
>
> -- Style 1: Simple Blocking Model
> a = channels:getsamples(count)
>
> -- Style 2: Separated Model
> channels:requestsamples(count, timer, frequency)
> channels:returnsamples(count)

I'd go with this last suggestion, but let me check if I'm following you.

Apparently you need a way to define channels, samplers and consumers.
A sampler collects data from a channel and may or not buffer it for
future use. A consumer asks for samples from the sampler, possibly in
an asynchronous mode.

Assuming this terminology, the proposed API could be slightly changed
so instead of having channels as the first order objects, we have
samplers now. In this scenario channels are just numbers again.

s1 = adc.sampler(1)
others = adc.sampler(3, 4, 6) -- no channel 21 this time hehehe

Your proposed style 1 could be used like this, where the sampler
blocks, reads 10 samples at the rate of 1000/s and returns

-- defines the sampler mode and do the one time initialization you need
s1:setmode{asyn = false, rate=100, buffer=10}

while cond do
   mybuffer = s1:read() -- only reads, no initialization
   dosomething(mybuffer)
end

Note here that the buffer defined is in fact used to pre allocate
space for each read. If you need to vary the number of samples read,
just pass this number to s1:read(), but this may generate some timing
issues.

Your proposed style 2 could be used as this, where the samplers don't
block and you cand decide how much to pull from the buffers at any
time:

s1:setmode{rate = 300, buffer = 1000} -- asyn is true by default

s1:start() -- starts background sampling
...
while cond do
   mybuffer = s1:read(5) -- pulls 5 samples from the buffer
   dosomething(mybuffer)
end
...
s1:stop()

Apparently the differences between the second option and the third are
mostly related to flexibility and buffer types.

Buffer types could be defined in the setmode() function. For example
we could use buffertype="circular" (probably the default) or some
other type.

In the same way, if you need to specify more sampler parameters, just
add them to the sampler() constructor.

You may want to consider adding a timeout parameter too, for the case
where you want to read 10 samples but the buffer contains less than
that.

Andr?

[jbsnyder]

On Feb 18, 2009, at 8:28 AM, Andre Carregal wrote:

> On Wed, Feb 18, 2009 at 1:08 AM, James Snyder  
> <jbsnyder at fanplastic.org> wrote:
>> One alternative that I've not mentioned here is that there be two  
>> functional
>> models for getting samples.  If one wants, one could have a simple
>> getsamples method that follows model 1, and a second pair that  
>> follows model
>> 3. i.e.:
>>
>> channels = adc.channel(0, 3, 17, 21)
>>
>> -- Style 1: Simple Blocking Model
>> a = channels:getsamples(count)
>>
>> -- Style 2: Separated Model
>> channels:requestsamples(count, timer, frequency)
>> channels:returnsamples(count)
>
> I'd go with this last suggestion, but let me check if I'm following  
> you.
>
> Apparently you need a way to define channels, samplers and consumers.
> A sampler collects data from a channel and may or not buffer it for
> future use. A consumer asks for samples from the sampler, possibly in
> an asynchronous mode.

Yes.

>
>
> Assuming this terminology, the proposed API could be slightly changed
> so instead of having channels as the first order objects, we have
> samplers now. In this scenario channels are just numbers again.
>
> s1 = adc.sampler(1)
> others = adc.sampler(3, 4, 6) -- no channel 21 this time hehehe

I like this concept.  Especially because it somewhat mirrors how the  
hardware is implemented on LM boards.

I also like it because it makes for a nice conceptual way to think  
about grouping channels.  Channels act as sources individual sources,  
which pool into a given sampler.  The sampler then has additional  
properties such as when samples are collected, how many, etc..

This will make the backend somewhat more complicated, but it should  
all be doable.

>
> Your proposed style 1 could be used like this, where the sampler
> blocks, reads 10 samples at the rate of 1000/s and returns
>
> -- defines the sampler mode and do the one time initialization you  
> need
> s1:setmode{asyn = false, rate=100, buffer=10}
>
> while cond do
>   mybuffer = s1:read() -- only reads, no initialization
>   dosomething(mybuffer)
> end
>
> Note here that the buffer defined is in fact used to pre allocate
> space for each read. If you need to vary the number of samples read,
> just pass this number to s1:read(), but this may generate some timing
> issues.

OK, I like that.

One modification is that we would have to pass a timer number to  
setmode as well, because eLua doesn't have any resource management  
system that abstracts this away.

>
>
> Your proposed style 2 could be used as this, where the samplers don't
> block and you cand decide how much to pull from the buffers at any
> time:
>
> s1:setmode{rate = 300, buffer = 1000} -- asyn is true by default
>
> s1:start() -- starts background sampling
> ...
> while cond do
>   mybuffer = s1:read(5) -- pulls 5 samples from the buffer
>   dosomething(mybuffer)
> end
> ...
> s1:stop()

I like this too.

One interesting thing here is what to do if a user calls start when  
asyn=false. Under your model would you propose that it return an error?

I suppose you could use the start method with asyn=false for a fixed-
length pre-collection of samples before read?  If this were done  
though, perhaps the expectation would be that as soon as you had  
cleared out samples, it would fill things up again?

If asyn is false, does this mean:  Samples will only be collected when  
read is called, and they will be returned via read?
vs the following (asyn = true): Sampling is never initiated by read,  
read can only pull out samples that are in the buffer or arriving in  
the buffer?



>
>
> Apparently the differences between the second option and the third are
> mostly related to flexibility and buffer types.
>
> Buffer types could be defined in the setmode() function. For example
> we could use buffertype="circular" (probably the default) or some
> other type.

Right, I think this relates to what I was saying above.  You could  
have a circular buffer that wraps around when full, or one that fills  
and then stops until more space is available.

In this respect, you could describe a circular or linear buffer as  
controlling whether or not once you start sampling it will stop at the  
end of the buffer, or be free-running.

Hmm...

>
>
> In the same way, if you need to specify more sampler parameters, just
> add them to the sampler() constructor.
>
> You may want to consider adding a timeout parameter too, for the case
> where you want to read 10 samples but the buffer contains less than
> that.

This is a good idea.  At the moment there is no timeout, but I think  
this might get complicated unless we say make a dedicated virtual  
timer or something.

The whole timer usage issue is one of the reasons why we have a simple  
sample function, and then a burst function.  One needs a timer (burst,  
for evenly spaced samples lower than running at the max rate), the  
other just samples whenever called.

One other thing is that it sounds as if this module will end up being  
a mixture of a C interface, and some lua code to handle sampler  
instances? If this is the case, then both the C interface and the end-
user interface might be rather different-looking...

>
>
> Andr?
> _______________________________________________
> Elua-dev mailing list
> Elua-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/elua-dev

--
James Snyder
Biomedical Engineering
Northwestern University
jbsnyder at fanplastic.org
http://fanplastic.org/key.txt
ph: [phone]

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.berlios.de/pipermail/elua-dev/attachments/20090218/3285c9d3/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
Url : https://lists.berlios.de/pipermail/elua-dev/attachments/20090218/3285c9d3/attachment-0001.pgp 

[Andre Carregal-2]

On Wed, Feb 18, 2009 at 5:09 PM, James Snyder <jbsnyder at fanplastic.org> wrote:
> One modification is that we would have to pass a timer number to setmode as
> well, because eLua doesn't have any resource management system that
> abstracts this away.

No problem.

> One interesting thing here is what to do if a user calls start when
> asyn=false. Under your model would you propose that it return an error?

I wouldn't mind having both start() and stop() being equivalent to NOP
when asyn=false, but that depends on how user friendly you want to be
to programmers... :o)

> I suppose you could use the start method with asyn=false for a fixed-length
> pre-collection of samples before read?  If this were done though, perhaps
> the expectation would be that as soon as you had cleared out samples, it
> would fill things up again?

I'm afraid this could be confusing. The proposed start() and stop()
just do that. From the programmer point of view, all the reading
happens on the read() part.

> If asyn is false, does this mean:  Samples will only be collected when read
> is called, and they will be returned via read?

Well, that seems reasonable for me since we are talking about a
synchronous read. But I may be missing some timing details here.

> vs the following (asyn = true): Sampling is never initiated by read, read
> can only pull out samples that are in the buffer or arriving in the buffer?

Exactly. Iniatilization and termination would be delegated to start()
and stop(), no matter the asyn mode.

Again, let me know if this violate some timing requirement.

> Right, I think this relates to what I was saying above.  You could have a
> circular buffer that wraps around when full, or one that fills and then
> stops until more space is available.

Exactly.

> In this respect, you could describe a circular or linear buffer as
> controlling whether or not once you start sampling it will stop at the end
> of the buffer, or be free-running.
> Hmm...

Call me naive, but I'm not seeing the problem with this approach.
Would you mind giving more details?

> This is a good idea.  At the moment there is no timeout, but I think this
> might get complicated unless we say make a dedicated virtual timer or
> something.

Timeout can be left for future versions, the point was that the
constructor uses named parameters, which facilitates extensions.

> The whole timer usage issue is one of the reasons why we have a simple
> sample function, and then a burst function.  One needs a timer (burst, for
> evenly spaced samples lower than running at the max rate), the other just
> samples whenever called.

Here a few use cases would help determining what way to go. I think it
may be possible to set timer types (and behaviors) on the constructor,
but I'd need to understand better your needs.

> One other thing is that it sounds as if this module will end up being a
> mixture of a C interface, and some lua code to handle sampler instances? If
> this is the case, then both the C interface and the end-user interface might
> be rather different-looking...

Well, once you have the API defined, you can call it from both sides.
It would depend more on which you would prefer.

Andr?

[jbsnyder]

On Feb 18, 2009, at 3:03 PM, Andre Carregal wrote:

> On Wed, Feb 18, 2009 at 5:09 PM, James Snyder  
> <jbsnyder at fanplastic.org> wrote:
>> One modification is that we would have to pass a timer number to  
>> setmode as
>> well, because eLua doesn't have any resource management system that
>> abstracts this away.
>
> No problem.
>
>> One interesting thing here is what to do if a user calls start when
>> asyn=false. Under your model would you propose that it return an  
>> error?
>
> I wouldn't mind having both start() and stop() being equivalent to NOP
> when asyn=false, but that depends on how user friendly you want to be
> to programmers... :o)
>
>> I suppose you could use the start method with asyn=false for a  
>> fixed-length
>> pre-collection of samples before read?  If this were done though,  
>> perhaps
>> the expectation would be that as soon as you had cleared out  
>> samples, it
>> would fill things up again?
>
> I'm afraid this could be confusing. The proposed start() and stop()
> just do that. From the programmer point of view, all the reading
> happens on the read() part.

OK, fair enough.

>
>
>> If asyn is false, does this mean:  Samples will only be collected  
>> when read
>> is called, and they will be returned via read?
>
> Well, that seems reasonable for me since we are talking about a
> synchronous read. But I may be missing some timing details here.
>
>> vs the following (asyn = true): Sampling is never initiated by  
>> read, read
>> can only pull out samples that are in the buffer or arriving in the  
>> buffer?
>
> Exactly. Iniatilization and termination would be delegated to start()
> and stop(), no matter the asyn mode.
>
> Again, let me know if this violate some timing requirement.

No violations, I just wanted to make sure that I was understanding the  
intent correctly :-)

>
>
>> Right, I think this relates to what I was saying above.  You could  
>> have a
>> circular buffer that wraps around when full, or one that fills and  
>> then
>> stops until more space is available.
>
> Exactly.
>
>> In this respect, you could describe a circular or linear buffer as
>> controlling whether or not once you start sampling it will stop at  
>> the end
>> of the buffer, or be free-running.
>> Hmm...
>
> Call me naive, but I'm not seeing the problem with this approach.
> Would you mind giving more details?

I didn't mean to imply that there was a problem with this part of the  
approach.  I think this would be useful for situations where you know  
you want to collect samples about an event, but you're not sure if  
you'd necessarily pick up the results before a ring implementation  
might result in samples being overwritten.

>
>
>> This is a good idea.  At the moment there is no timeout, but I  
>> think this
>> might get complicated unless we say make a dedicated virtual timer or
>> something.
>
> Timeout can be left for future versions, the point was that the
> constructor uses named parameters, which facilitates extensions.

Right.  I'm not exactly sure what the best route is for this now.  The  
approach I'm inclined to take for now is to return nil values or  
something similar if there are no pending operations which could  
provide the requested set of samples.  This should be fairly robust  
for built-in ADCs, but might have some ugly situations if you had an  
external ADC that just got unplugged :-)

>
>
>> The whole timer usage issue is one of the reasons why we have a  
>> simple
>> sample function, and then a burst function.  One needs a timer  
>> (burst, for
>> evenly spaced samples lower than running at the max rate), the  
>> other just
>> samples whenever called.
>
> Here a few use cases would help determining what way to go. I think it
> may be possible to set timer types (and behaviors) on the constructor,
> but I'd need to understand better your needs.

I think this is one of the critical things here.  I know ways in which  
I would use this personally, and have built in a fair amount of  
infrastructure for that.  The problem is that I can think of  
innumerable features that _could_ be useful in hypothetical  
situations :-)  It would be nice to have a rundown of common usage  
scenarios and see how well whatever implementation is chosen fulfills  
that range.  This would allow us to draw the line on features.  I'm  
certainly not aiming to reproduce MATLAB's data acquisition toolbox or  
anything :-)

>
>> One other thing is that it sounds as if this module will end up  
>> being a
>> mixture of a C interface, and some lua code to handle sampler  
>> instances? If
>> this is the case, then both the C interface and the end-user  
>> interface might
>> be rather different-looking...
>
> Well, once you have the API defined, you can call it from both sides.
> It would depend more on which you would prefer.

Sorry, what I meant here was that the implementation of the API itself  
might be a mixture of Lua and C.  Looking a bit at some things in PIL  
though, suggest to me that you could still do the module  
implementation on the C side.

OT: Speaking of which I should just break down and get a copy of PIL  
2, I'm mainly using the online version of the first edition :-)  Is  
there any way to get an electronic version of the second edition or is  
there just the paper version?

--
James Snyder
Biomedical Engineering
Northwestern University
jbsnyder at fanplastic.org
http://fanplastic.org/key.txt
ph: [phone]

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.berlios.de/pipermail/elua-dev/attachments/20090218/81afc19a/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
Url : https://lists.berlios.de/pipermail/elua-dev/attachments/20090218/81afc19a/attachment-0001.pgp 

[Andre Carregal-2]

On Wed, Feb 18, 2009 at 7:00 PM, James Snyder <jbsnyder at fanplastic.org> wrote:
> I didn't mean to imply that there was a problem with this part of the
> approach.  I think this would be useful for situations where you know you
> want to collect samples about an event, but you're not sure if you'd
> necessarily pick up the results before a ring implementation might result in
> samples being overwritten.

But assuming that the event or generator is continuous, this could
lead to the program reading really old data, no? Wouldn't this be
somewhat strange since the data flow would tend to be very unrelated
to the real time flow?

You can look at this as something similar to the choices offered by
TCP and UDP. You can offer continuity or real time, but not both at
the same time if the reading loop is not fast enough to follow the
event flow. :o)

>> Timeout can be left for future versions, the point was that the
>> constructor uses named parameters, which facilitates extensions.
>
> Right.  I'm not exactly sure what the best route is for this now.  The
> approach I'm inclined to take for now is to return nil values or something
> similar if there are no pending operations which could provide the requested
> set of samples.  This should be fairly robust for built-in ADCs, but might
> have some ugly situations if you had an external ADC that just got unplugged
> :-)

I'd suggest returning an empty table when there were no samples and a
nil+errmsg when there was an error.

>> Here a few use cases would help determining what way to go. I think it
>> may be possible to set timer types (and behaviors) on the constructor,
>> but I'd need to understand better your needs.
>
> I think this is one of the critical things here.  I know ways in which I
> would use this personally, and have built in a fair amount of infrastructure
> for that.  The problem is that I can think of innumerable features that
> _could_ be useful in hypothetical situations :-)  It would be nice to have a
> rundown of common usage scenarios and see how well whatever implementation
> is chosen fulfills that range.  This would allow us to draw the line on
> features.  I'm certainly not aiming to reproduce MATLAB's data acquisition
> toolbox or anything :-)

Agreed, but for that a wiki page is better than a mail thread. :o)

>> Well, once you have the API defined, you can call it from both sides.
>> It would depend more on which you would prefer.
>
> Sorry, what I meant here was that the implementation of the API itself might
> be a mixture of Lua and C.  Looking a bit at some things in PIL though,
> suggest to me that you could still do the module implementation on the C
> side.

Exactly, but that choice usually depends more on system restrictions
than on style.

> OT: Speaking of which I should just break down and get a copy of PIL 2, I'm
> mainly using the online version of the first edition :-)  Is there any way
> to get an electronic version of the second edition or is there just the
> paper version?

No PiL 2 electronic version yet.

<plug>
If you decide to get the cellulose version, keep in mind that buying
it from lua.org or luaforge.net helps each project respectively, it's
your choice. hehe
</plug>

Andr?

[jbsnyder]

On Feb 18, 2009, at 6:35 PM, Andre Carregal wrote:

> On Wed, Feb 18, 2009 at 7:00 PM, James Snyder  
> <jbsnyder at fanplastic.org> wrote:
>> I didn't mean to imply that there was a problem with this part of the
>> approach.  I think this would be useful for situations where you  
>> know you
>> want to collect samples about an event, but you're not sure if you'd
>> necessarily pick up the results before a ring implementation might  
>> result in
>> samples being overwritten.
>
> But assuming that the event or generator is continuous, this could
> lead to the program reading really old data, no? Wouldn't this be
> somewhat strange since the data flow would tend to be very unrelated
> to the real time flow?

That may be true.  The problem with hypothetical situations is that  
their number is infinite :-)

> You can look at this as something similar to the choices offered by
> TCP and UDP. You can offer continuity or real time, but not both at
> the same time if the reading loop is not fast enough to follow the
> event flow. :o)

Right, although at least we likely don't have to worry about the  
security issues that appear with things coming over TCP/UDP :-)

>
>>> Timeout can be left for future versions, the point was that the
>>> constructor uses named parameters, which facilitates extensions.
>>
>> Right.  I'm not exactly sure what the best route is for this now.  
>> The
>> approach I'm inclined to take for now is to return nil values or  
>> something
>> similar if there are no pending operations which could provide the  
>> requested
>> set of samples.  This should be fairly robust for built-in ADCs,  
>> but might
>> have some ugly situations if you had an external ADC that just got  
>> unplugged
>> :-)
>
> I'd suggest returning an empty table when there were no samples and a
> nil+errmsg when there was an error.

OK, that seems reasonable.  In the latter case we actually return a  
luaL_error, although those are mainly done in cases where buffer  
allocations fail.  For something more recoverable, it seems reasonable  
to complain but not bail out completely.

>
>
>>> Here a few use cases would help determining what way to go. I  
>>> think it
>>> may be possible to set timer types (and behaviors) on the  
>>> constructor,
>>> but I'd need to understand better your needs.
>>
>> I think this is one of the critical things here.  I know ways in  
>> which I
>> would use this personally, and have built in a fair amount of  
>> infrastructure
>> for that.  The problem is that I can think of innumerable features  
>> that
>> _could_ be useful in hypothetical situations :-)  It would be nice  
>> to have a
>> rundown of common usage scenarios and see how well whatever  
>> implementation
>> is chosen fulfills that range.  This would allow us to draw the  
>> line on
>> features.  I'm certainly not aiming to reproduce MATLAB's data  
>> acquisition
>> toolbox or anything :-)
>
> Agreed, but for that a wiki page is better than a mail thread. :o)

Yeah.  Maybe we should consider that.  I can post some of this up on  
the eLua wiki, so that it is in one place while I'm working on  
implementation and people can make changes when needed.

>
>
>>> Well, once you have the API defined, you can call it from both  
>>> sides.
>>> It would depend more on which you would prefer.
>>
>> Sorry, what I meant here was that the implementation of the API  
>> itself might
>> be a mixture of Lua and C.  Looking a bit at some things in PIL  
>> though,
>> suggest to me that you could still do the module implementation on  
>> the C
>> side.
>
> Exactly, but that choice usually depends more on system restrictions
> than on style.
>
>> OT: Speaking of which I should just break down and get a copy of  
>> PIL 2, I'm
>> mainly using the online version of the first edition :-)  Is there  
>> any way
>> to get an electronic version of the second edition or is there just  
>> the
>> paper version?
>
> No PiL 2 electronic version yet.
>
> <plug>
> If you decide to get the cellulose version, keep in mind that buying
> it from lua.org or luaforge.net helps each project respectively, it's
> your choice. hehe
> </plug>

Hmm.. If I had more spare funds, I might do both.  Perhaps I'll go  
with lua.org for now since I've not yet thanked them for such a lovely  
and simple but flexible language :-)

(That said, I also have to say that I'm also a large fan certain  
language written by a Dutch man, who may happen to currently work for  
Google.  Different tools for different situations.  This other  
language could definitely learn a thing or two about VM  
implementation, if not some other things, from Lua.)


--
James Snyder
Biomedical Engineering
Northwestern University
jbsnyder at fanplastic.org
http://fanplastic.org/key.txt
ph: [phone]

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.berlios.de/pipermail/elua-dev/attachments/20090219/dcf59e83/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
Url : https://lists.berlios.de/pipermail/elua-dev/attachments/20090219/dcf59e83/attachment.pgp