How to manipulate a receiving buffer

On Apr 3, 4:01=EF=BF=BDpm, "Steven Woody" <narkewo...@gmail.com> wrote:
> i am writting problem to receive data from a interrupt-driven serial
> port, and i think my code will always have change of lossing data.
>
> in the receving ISR, i get a byte from a register and put it into a
> buffer,
>
> static unsigned char buff[ BUFF_SIZE ];
> static buff_len =3D 0;
>
> __interrupt =A0on_byte_in( void )
> {
> =A0 =A0 =A0 =A0if ( buffer_len < sizeof( buff ) )
> =A0 =A0 =A0 =A0 =A0 =A0buffer[ buffer_len++ ] =3D the_byte_in_the_registe=
r;
>
> }
>
> another function used to fetch the buffer data,
>
> flush_receiving_buff( void )
> {
> =A0 =A0for ( size_t i =3D 0; i < buff_len; ++i ) {
> =A0 =A0 =A0 process the buff[ i ];
> =A0 =A0buff_len =3D 0; =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 --=
----------- (a)
>
> }
>
> my question is that, during the execution of statement (a), one or
> more new bytes might come in and code above will lost them. =A0if you
> folks see the problem, can you please suggest me a solution? i believe
> the question is very basic and must have been well resolved.
>
> thanks in advance.
>
> --
> woody

First set up your buffer, setup 2 pointers to it, the serial interrupt
uses one of them to write each byte, incrementing each time (ie a
circular pointer). Ditto for your reading routine (use the other
pointer). To flush the buffer set both pointers to the same number.
You will not need to disable the serial interrupt. Your read routine
must check the pointers are not the same before reading (buffer empty)
the write routine must check they are not the same after writing
(buffer overflow).

On 2007-04-04, cbarn24050@aol.com <cbarn24050@aol.com> wrote:

> First set up your buffer, setup 2 pointers to it, the serial interrupt
> uses one of them to write each byte, incrementing each time (ie a
> circular pointer). Ditto for your reading routine (use the other
> pointer). To flush the buffer set both pointers to the same number.
> You will not need to disable the serial interrupt.

That's true as long as you know that reading/writing a pointer
is an atomic operation.  That's not true for all
microprocessors. For example, on the AVR, reading/writing a
pointer isn't atomic -- a MAJOR flaw in any processor that's
intended for use with C -- one of the reasons I dislice the AVR
ISA.  

For such processors use of an index variable that's fewer bits
than a pointer will often work -- I don't know of any popular
microprocessors where reading/writing an 8-bit variable isn't
an atomic operation (of course that limits your buffer size to
256 bytes).

> Your read routine must check the pointers are not the same
> before reading (buffer empty) the write routine must check
> they are not the same after writing (buffer overflow).

-- 
Grant Edwards                   grante             Yow!  What PROGRAM are
                                  at               they watching?
                               visi.com            

Grant Edwards wrote:
> On 2007-04-04, cbarn24050@aol.com <cbarn24050@aol.com> wrote:
> 
>> First set up your buffer, setup 2 pointers to it, the serial
>> interrupt uses one of them to write each byte, incrementing each
>> time (ie a circular pointer). Ditto for your reading routine (use
>> the other pointer). To flush the buffer set both pointers to the
>> same number. You will not need to disable the serial interrupt.
> 
> That's true as long as you know that reading/writing a pointer
> is an atomic operation.  That's not true for all microprocessors.
> For example, on the AVR, reading/writing a pointer isn't atomic --
> a MAJOR flaw in any processor that's intended for use with C --
> one of the reasons I dislice the AVR ISA.
> 
> For such processors use of an index variable that's fewer bits
> than a pointer will often work -- I don't know of any popular
> microprocessors where reading/writing an 8-bit variable isn't
> an atomic operation (of course that limits your buffer size to
> 256 bytes).

This works provided you carefully control the order of operations,
and ensure that the read index is advanced only by the read
process, and the write index by the write process.  Advancing those
indices must be modulo buffersize, and so must the comparisons that
establish "bufferNotEmpty" and "bufferNotFull".  Always keep in
mind that one routine may be interrupted to execute the other at
any time.  Only the read routine can alter the NotFull condition,
and only the write routine can alter the NotEmpty condition.

If the buffer size is under 256 bytes, you can usually ensure that
the indices (not pointers) read/writes are atomic.  Unsigned char
are useful here.  A buffer of exactly UCHAR_MAX+1 length makes the
modulo operations automatic.  When these indices get larger you
will need to resort to critical sections.

-- 
Please do not top-post.  Your answer belongs after (or intermixed
with) the quoted material to which you reply, after snipping all
irrelevant material.  See the following links:

  <http://www.catb.org/~esr/faqs/smart-questions.html>
  <http://www.caliburn.nl/topposting.html>
  <http://www.netmeister.org/news/learn2quote.html>
  <http://cfaj.freeshell.org/google/>  (taming google)
  <http://members.fortunecity.com/nnqweb/>  (newusers)



-- 
Posted via a free Usenet account from http://www.teranews.com

On Apr 4, 11:47 pm, "Dave Pollum" <vze24...@verizon.net> wrote:
> On Apr 3, 10:46 pm, "Steven Woody" <narkewo...@gmail.com> wrote:
>
>
>
> > On Apr 4, 3:26 am, David R Brooks <daveb...@iinet.net.au> wrote:
>
> > > Steven Woody wrote:
> > > > On 4=E6=9C=884=E6=97=A5, =E4=B8=8A=E5=8D=881=E6=97=B637=E5=88=86, "=
Arlet" <usene...@c-scape.nl> wrote:
> > > >> On Apr 3, 7:13 pm, "Steven Woody" <narkewo...@gmail.com> wrote:
>
> > > >>> On 4=E6=9C=884=E6=97=A5, =E4=B8=8A=E5=8D=8812=E6=97=B606=E5=88=86=
, "Arlet" <usene...@c-scape.nl> wrote:
> > > >>>> On Apr 3, 5:11 pm, "John Speth" <johnsp...@yahoo.com> wrote:
> > > >>>>>> __interrupt  on_byte_in( void )
> > > >>>>>> {
> > > >>>>>>       if ( buffer_len < sizeof( buff ) )
> > > >>>>>>           buffer[ buffer_len++ ] =3D the_byte_in_the_register;
> > > >>>>>> }
> > > >>>>>> flush_receiving_buff( void )
> > > >>>>>> {
> > > >>>>>>   for ( size_t i =3D 0; i < buff_len; ++i ) {
> > > >>>>>>      process the buff[ i ];
> > > >>>>>>   buff_len =3D 0;                         ------------- (a)
> > > >>>>>> }
> > > >>>>>> my question is that, during the execution of statement (a), on=
e or
> > > >>>>>> more new bytes might come in and code above will lost them.  i=
f you
> > > >>>>>> folks see the problem, can you please suggest me a solution?
> > > >>>>> Yes, it's basic.  You'll need to disable your receive interrupt=
 during
> > > >>>>> statement (a) execution.  Doing so will prevent the ISR from ex=
ecuting and
> > > >>>>> changing buff_len while your flush function is also changing it.
> > > >>>> It would also be a good idea to declare 'buff_len' as volatile.
> > > >>>> Alternatively, you could use something similar to this, and leave
> > > >>>> interrupts enabled:
> > > >>>> static unsigned char buff[ BUFF_SIZE ];
> > > >>>> static volatile unsigned char buff_head;
> > > >>>> static unsigned char buff_tail;
> > > >>>> __interrupt  on_byte_in( void )
> > > >>>> {
> > > >>>>     if ( (unsigned char) (buff_head - buff_tail) < BUFF_SIZE )
> > > >>>>         buff[ buff_head++  % BUFF_SIZE ] =3D the_byte_in_the_reg=
ister;
> > > >>>> }
> > > >>>> flush_receiving_buff( void )
> > > >>>> {
> > > >>>>     while( buff_head !=3D buff_tail )
> > > >>>>         process the buff[ buff_tail++ % BUFF_SIZE  ];
> > > >>>> }
> > > >>>> This works only if BUFF_SIZE < 256. Powers of two are recommende=
d to
> > > >>>> avoid modulo operation. If your compiler doesn't optimize away t=
he
> > > >>>> modulo, rewrite it as '& (BUFF_SIZE-1)'. For larger BUFF_SIZE, u=
se a
> > > >>>> bigger type.
> > > >>> Arlet,
> > > >>> 1, does you ment to say, by letting the on_byte_in wrte only one
> > > >>> variable and letting the flush_receiving_buff write only another
> > > >>> variable, hence the code above don't need to be pretected into a
> > > >>> disable/enable interrupt section?
> > > >> True. Instead of having a single 'length', this code uses two
> > > >> variables, and the 'length' is the difference between them. The
> > > >> interrupt handler only changes head, and the main code only changes
> > > >> tail, so there's no conflict. It does require atomic reads, though=
. On
> > > >> an 8-bit architecture, where the CPU doesn't have a 16-bit read,
> > > >> you'll be restricted to 8-bit head/tail counters. For most
> > > >> applications on an 8-bit platform this is sufficient for a UART
> > > >> buffer.
>
> > > > thanks for you answer. and, even  in a 8-bit architecture, one need=
 to
> > > > receive a network packet which may longer than 255 bytes :-(  in the
> > > > case, data might be lost. am i right?
>
> > > >>> 2, what's the exact functionality of keyword 'volatile', what's t=
he
> > > >>> different if don't use it?
> > > >> 'volatile' tells the compiler not to optimize away memory referenc=
es.
> > > >> For example in the while loop:
>
> > > >>     while( buff_head !=3D buff_tail )
>
> > > >> Without the volatile, the compiler could assume that buff_head nev=
er
> > > >> changes, and may therefore load the value in a register before the
> > > >> while loop, and never read it again. With the 'volatile' keyword, =
the
> > > >> buff_head variable will be read from memory before every compariso=
n=2E
> > > >> Since buff_tail is not modified in the interrupt handler, it does =
not
> > > >> need to be volatile.
>
> > > > thank you very much!
>
> > > >>> 3, can you please answer me another question: will a disabled
> > > >>> interrupt pop up again after it is enabled?
> > > >> On any sane architecture, yes.
>
> > > > so, if in a disable/enable interrupt secion, there are two bytes co=
mes
> > > > in, the first byte will be lost since the interrupt will appear only
> > > > once after the enable interrupt instruction. am i righ? ( supposing=
 i
> > > > am talking about a UART with only one byte FIFO ).
>
> > > One thing wasn't clear from the original problem description: will the
> > > fn. flush_receiving_buff() always clear the buffer to empty (for
> > > example, if a complete packet is been received?
> > > If so, you might consider using two receiving buffers, controlled by a
> > > pointer. __interrupt  on_byte_in() will load the buffer indicated by =
the
> > > pointer. Once a full packet is received, the pointer is switched to t=
he
> > > second buffer, & the first is processed at your leisure.
> > > This is the classic double buffer setup. Of course, the actual switch=
ing
> > > of the pointer may need protecting as a critical section (eg by
> > > disabling interrupts).
>
> >  i understand this kind of double buffer setup. but for some reasons i
> > dont want to put packet recognizing logical into the ISR.
>
> > -
> > woody
>
> I've used double buffering with packet/protocol logic in the ISR, and
> that works well.  However, if you don't want the packet logic in the
> ISR, you could still use double buffering.  The ISR could mark a
> buffer as ready for the upper level code, and then store more
> characters in the _other_ buffer.  Ready could be set if the buffer is
> full, or if a certain amount of time has passed and no new characters
> have been received.  The ISR can save the numbers of bytes received in
> a variable associated with that buffer (use a struct - see below).
> After the upper level code has read all of the chars from the buffer,
> it as available.  The ISR can then use that buffer again.  Disabling
> and restoring interrupts aren't needed.  This eliminates the need for
> both the ISR code and the upper level code to write to the count
> (buff_len) variable.
>
>   typedef struct {
>     char  nbr_bytes_read;     /* 0..255 */
>     char  OK_to_read_buffer;  /* T/F    */
>     char  buffer[ BUFF_SZ ];
>     } buffer_stuff;
>   buffer_stuff  *buffer_1;
>   buffer_stuff  *buffer_2;
>
> -Dave Pollum

thank you David.

On Apr 5, 1:03 am, clubley@remove_me.eisner.decus.org-Earth.UFP (Simon
Clubley) wrote:
> In article <1175692210.770492.61...@n59g2000hsh.googlegroups.com>, "Steven Woody" <narkewo...@gmail.com> writes:
>
> > On 4=D4=C24=C8=D5, =CF=C2=CE=E78=CA=B104=B7=D6, clubley@remove_me.eisner.de=
> > cus.org-Earth.UFP (Simon
> > Clubley) wrote:
>
> >> What I don't think I've seen anyone mention is the possibility of using
> >> RTS/CTS hardware flow control if you are worried about interrupts been
> >> disabled for an extended period.
>
> > when and where to set the RTS/CTS pins?
>
> You should drop CTS before disabling the interrupt, or if that's not
> possible, as the very first thing that you do in the interrupt handler.
>
> Raise CTS after enabling the interrupt, or if that's not possible, as
> the last thing before leaving the interrupt handler.
>
> You should also check if your setup requires you to invert the CTS signal.
>
> However, make sure that your device will deliver an interrupt for an
> action (in this case, receipt of a character) that occured while interrupts
> were disabled.
>
> Of course, the best solution would be to not have interrupts off for that
> long.
>
> Also, if you are driving the pin directly, don't forget to make sure that
> the voltages on the two devices are compatible so that you don't fry
> something. :-)
>
> I also recommend you try and find some online tutorials on this, so that
> you can see what the various issues are. Unfortunately, I don't have any
> pointers for you.
>
> BTW, does anyone here disagree with any of the above, or do this in some
> better way ?
>

only a little confusing: in the Linux-Serial-HOWTO, i saw that
negating RTS is used to stop incoming flow and raising RTS is used to
resume the incoming flow.

Arlet wrote:

> Alternatively, you could use something similar to this, and leave
> interrupts enabled:
> 
> static unsigned char buff[ BUFF_SIZE ];
> static volatile unsigned char buff_head;
> static unsigned char buff_tail;
> 
> __interrupt  on_byte_in( void )
> {
>     if ( (unsigned char) (buff_head - buff_tail) < BUFF_SIZE )
>         buff[ buff_head++  % BUFF_SIZE ] = the_byte_in_the_register;
> 
> }
> 
> flush_receiving_buff( void )
> {
>     while( buff_head != buff_tail )
>         process the buff[ buff_tail++ % BUFF_SIZE  ];
> }
> 
> This works only if BUFF_SIZE < 256. 

It only works if BUFF_SIZE divides evenly into UCHAR_MAX+1.  In order to 
fix this, store the incremented indexes after mod reduction (which 
normally works faster by comparison, rather than mod operator).  Also, 
you have BUF_SIZE+1 states to distinguish: 0 through BUF_SIZE stored 
characters.  They can't be uniquely distinguished by the comparisons 
shown in the code.  The standard approach is to either use a separate 
state variable or to eliminate the all full state by storing a max of 
BUF_SIZE-1 characters.

-- 
Thad

On 2007-04-05, Steven Woody <narkewoody@gmail.com> wrote:

[150 lines of crap deleted]

> thank you David.

For kibo's sake trim the quoted material.  Was it really
necessary to quote 150 lines of old material to add that one
single line of new text?

-- 
Grant Edwards                   grante             Yow!  "THE LITTLE PINK
                                  at               FLESH SISTERS," I saw them
                               visi.com            at th' FLUROESCENT BULB
                                                   MAKERS CONVENTION...

Steven Woody wrote:
> 
... snip ...
> 
> only a little confusing: in the Linux-Serial-HOWTO, i saw that
> negating RTS is used to stop incoming flow and raising RTS is
> used to resume the incoming flow.

Bear in mind that the effect of RTS (request-to-send) is not
immediate.  The sender may well have some number of characters
buffered and ready to go.  So the receiver has to be able to handle
that.

-- 
 <http://www.cs.auckland.ac.nz/~pgut001/pubs/vista_cost.txt>
 <http://www.securityfocus.com/columnists/423>
 <http://www.aaxnet.com/editor/edit043.html>
 
 "A man who is right every time is not likely to do very much."
                           -- Francis Crick, co-discover of DNA
 "There is nothing more amazing than stupidity in action."
                                             -- Thomas Matthews



-- 
Posted via a free Usenet account from http://www.teranews.com

On Apr 5, 4:45 am, Thad Smith <ThadSm...@acm.org> wrote:
> Arlet wrote:
> > Alternatively, you could use something similar to this, and leave
> > interrupts enabled:
>
> > static unsigned char buff[ BUFF_SIZE ];
> > static volatile unsigned char buff_head;
> > static unsigned char buff_tail;
>
> > __interrupt  on_byte_in( void )
> > {
> >     if ( (unsigned char) (buff_head - buff_tail) < BUFF_SIZE )
> >         buff[ buff_head++  % BUFF_SIZE ] = the_byte_in_the_register;
>
> > }
>
> > flush_receiving_buff( void )
> > {
> >     while( buff_head != buff_tail )
> >         process the buff[ buff_tail++ % BUFF_SIZE  ];
> > }
>
> > This works only if BUFF_SIZE < 256.
>
> It only works if BUFF_SIZE divides evenly into UCHAR_MAX+1.  In order to
> fix this, store the incremented indexes after mod reduction (which
> normally works faster by comparison, rather than mod operator).  Also,
> you have BUF_SIZE+1 states to distinguish: 0 through BUF_SIZE stored
> characters.  They can't be uniquely distinguished by the comparisons
> shown in the code.  The standard approach is to either use a separate
> state variable or to eliminate the all full state by storing a max of
> BUF_SIZE-1 characters.

Actually, it works as long as the number of possible states can be
represented by an unsigned char, or (BUFF_SIZE+1) <= (UCHAR_MAX+1). In
theory, BUFF_SIZE could be as high as 255. The fact that the
comparison is done _before_ the mod reduction makes sure that we still
have the complete state information available. If you do the mod
reduction first, you lose information, so then you would need
additional state, which could be undesirable if it needs to be shared
between the two contexts.

Of course, the mod may be unacceptably slow, so in that case BUFF_SIZE
needs to be restricted to a constant power of two (with 128 being the
highest number that meets the requirements), so it can be implemented
by a bitwise 'and'. Any decent compiler will do the optimization
automatically. Using the mod is still possible if it meets the timing
requirements, and if there's a need for special buffer sizes.

>>
>> when and where to set the RTS/CTS pins?
>>
>
> You should drop CTS before disabling the interrupt, or if that's not
> possible, as the very first thing that you do in the interrupt handler.
>
> Raise CTS after enabling the interrupt, or if that's not possible, as
> the last thing before leaving the interrupt handler.
>


Hope everyone is aware that the RTS/CTS is to be considered
a recommendation to the other end to try to stop its babbling.

You must expect to get AT LEAST 16 more incoming characters
if you want to be PC compliant.

The NS16550 UART chip, was an improvement of the NS16450
UART, adding a 16 level FIFO.
If something is written to the FIFO before the hardware flow control is 
deasserted,
the data gets sent without regards to the flow control.


-- 
Best Regards,
Ulf Samuelsson
This is intended to be my personal opinion which may,
or may not be shared by my employer Atmel Nordic AB