EmbeddedRelated.com
Forums

How to manipulate a receiving buffer

Started by Steven Woody April 3, 2007
"Steven Woody" <narkewoody@gmail.com> wrote in
news:1175615743.906257.103310@w1g2000hsg.googlegroups.com: 

> On 4&#4294967295;&#4294967295;3&#4294967295;&#4294967295;, &#4294967295;&#4294967295;&#4294967295;&#4294967295;11&#689;11&#4294967295;&#4294967295;, "John Speth" <johnsp...@yahoo.com> wrote: >> > __interrupt on_byte_in( void ) >> > { >> > if ( buffer_len < sizeof( buff ) ) >> > buffer[ buffer_len++ ] = the_byte_in_the_register; >> > } >> >> > flush_receiving_buff( void ) >> > { >> > for ( size_t i = 0; i < buff_len; ++i ) { >> > process the buff[ i ]; >> > buff_len = 0; ------------- (a) >> > } >> >> > my question is that, during the execution of statement (a), one or >> > more new bytes might come in and code above will lost them. if you >> > folks see the problem, can you please suggest me a solution? >> >> Yes, it's basic. You'll need to disable your receive interrupt >> during statement (a) execution. Doing so will prevent the ISR from >> executing and changing buff_len while your flush function is also >> changing it. >> >> It falls under the topic of "critical sections" of code execution. >> >> JJS > > after an interrupt was disabled during the execuation of (a) by the > means you described, will it pop up again after the enable-interrupt > statement? if not, i think the problem is still not resolved. > > thanks. >
I don't get it. Presumably, flushing is a really drastic act. Otherwise you would just be fetching stuff as it comes in. If you really want to flush the buffer, why is it a problem if a few "late bytes" come in? If it truly is a problem, maybe the caller to flush should pass in a pointer/index to the "current" end of buffer and flush should go to there? Regards ~Steve
"Steven Woody" <narkewoody@gmail.com> skrev i meddelandet
news:1175612480.940672.274520@w1g2000hsg.googlegroups.com...
>i am writting problem to receive data from a interrupt-driven serial > port, and i think my code will always have change of lossing data. > > in the receving ISR, i get a byte from a register and put it into a > buffer, > > static unsigned char buff[ BUFF_SIZE ]; > static buff_len = 0; > > __interrupt on_byte_in( void ) > { > if ( buffer_len < sizeof( buff ) ) > buffer[ buffer_len++ ] = the_byte_in_the_register; > } > > another function used to fetch the buffer data, > > flush_receiving_buff( void ) > { > for ( size_t i = 0; i < buff_len; ++i ) { > process the buff[ i ]; > buff_len = 0; ------------- (a) > } > > my question is that, during the execution of statement (a), one or > more new bytes might come in and code above will lost them. if you > folks see the problem, can you please suggest me a solution? i believe > the question is very basic and must have been well resolved. > > thanks in advance. > > -- > woody >
I would read from the buffer using an access routine. Note that UART data does not come in burst so you have to disable interrupts for some time to miss an interrupt. You have almost two character times to process the interrupt. You can put a while loop in your interrupt routine to allow processing of data coming in at high speed, but that assumes that you receive short packets at high speed because you will not leave CPU cycles to the background task. unsigned short buffer[BUF_SIZE]; __interrupt on_byte_in( void ) { while(!UART.Receive_Holding_Register_Empty) { error=0; if(UART.OVERRUN) { uart_ovr_error++; error=1; } else if (UART.PARITY) { uart_par_error++; error=1; } else if (buffer_len >= BUF_SIZE) { uart_buf_error++; error=1; } if(error) { buffer[write_p] = 0xFFFF; // Lose a character uart_error_count ++; } else { buffer[write_p] = UART.DATA & 0xFF; buffer_len++; } write_p = (write_p + 1) % BUF_SIZE; } } char get_char(unsigned int *byte) { unsigned int c; __disable_interrupt(); c = buffer[read_p]; buffer_len--; __enable_interrupt(); read_p = (read_p + 1) % BUF_SIZE; *byte = c & 0xFF; if(c == 0xFFFF) { return ERROR; } else { return SUCCESS; } } flush_receiving_buff( void ) { for ( size_t i = 0; i < buff_len; ++i ) { sts = getchar(*b); if(sts == SUCCESS) { process(b); } else { error_handling(); } } } -- Best Regards, Ulf Samuelsson This is intended to be my personal opinion which may, or may not be shared by my employer Atmel Nordic AB
On Apr 4, 3:26 am, David R Brooks <daveb...@iinet.net.au> wrote:
> Steven Woody wrote: > > On 4=E6=9C=884=E6=97=A5, =E4=B8=8A=E5=8D=881=E6=97=B637=E5=88=86, "Arle=
t" <usene...@c-scape.nl> wrote:
> >> On Apr 3, 7:13 pm, "Steven Woody" <narkewo...@gmail.com> wrote: > > >>> On 4=E6=9C=884=E6=97=A5, =E4=B8=8A=E5=8D=8812=E6=97=B606=E5=88=86, "A=
rlet" <usene...@c-scape.nl> wrote:
> >>>> On Apr 3, 5:11 pm, "John Speth" <johnsp...@yahoo.com> wrote: > >>>>>> __interrupt on_byte_in( void ) > >>>>>> { > >>>>>> if ( buffer_len < sizeof( buff ) ) > >>>>>> buffer[ buffer_len++ ] =3D the_byte_in_the_register; > >>>>>> } > >>>>>> flush_receiving_buff( void ) > >>>>>> { > >>>>>> for ( size_t i =3D 0; i < buff_len; ++i ) { > >>>>>> process the buff[ i ]; > >>>>>> buff_len =3D 0; ------------- (a) > >>>>>> } > >>>>>> my question is that, during the execution of statement (a), one or > >>>>>> more new bytes might come in and code above will lost them. if you > >>>>>> folks see the problem, can you please suggest me a solution? > >>>>> Yes, it's basic. You'll need to disable your receive interrupt dur=
ing
> >>>>> statement (a) execution. Doing so will prevent the ISR from execut=
ing and
> >>>>> changing buff_len while your flush function is also changing it. > >>>> It would also be a good idea to declare 'buff_len' as volatile. > >>>> Alternatively, you could use something similar to this, and leave > >>>> interrupts enabled: > >>>> static unsigned char buff[ BUFF_SIZE ]; > >>>> static volatile unsigned char buff_head; > >>>> static unsigned char buff_tail; > >>>> __interrupt on_byte_in( void ) > >>>> { > >>>> if ( (unsigned char) (buff_head - buff_tail) < BUFF_SIZE ) > >>>> buff[ buff_head++ % BUFF_SIZE ] =3D the_byte_in_the_registe=
r;
> >>>> } > >>>> flush_receiving_buff( void ) > >>>> { > >>>> while( buff_head !=3D buff_tail ) > >>>> process the buff[ buff_tail++ % BUFF_SIZE ]; > >>>> } > >>>> This works only if BUFF_SIZE < 256. Powers of two are recommended to > >>>> avoid modulo operation. If your compiler doesn't optimize away the > >>>> modulo, rewrite it as '& (BUFF_SIZE-1)'. For larger BUFF_SIZE, use a > >>>> bigger type. > >>> Arlet, > >>> 1, does you ment to say, by letting the on_byte_in wrte only one > >>> variable and letting the flush_receiving_buff write only another > >>> variable, hence the code above don't need to be pretected into a > >>> disable/enable interrupt section? > >> True. Instead of having a single 'length', this code uses two > >> variables, and the 'length' is the difference between them. The > >> interrupt handler only changes head, and the main code only changes > >> tail, so there's no conflict. It does require atomic reads, though. On > >> an 8-bit architecture, where the CPU doesn't have a 16-bit read, > >> you'll be restricted to 8-bit head/tail counters. For most > >> applications on an 8-bit platform this is sufficient for a UART > >> buffer. > > > thanks for you answer. and, even in a 8-bit architecture, one need to > > receive a network packet which may longer than 255 bytes :-( in the > > case, data might be lost. am i right? > > >>> 2, what's the exact functionality of keyword 'volatile', what's the > >>> different if don't use it? > >> 'volatile' tells the compiler not to optimize away memory references. > >> For example in the while loop: > > >> while( buff_head !=3D buff_tail ) > > >> Without the volatile, the compiler could assume that buff_head never > >> changes, and may therefore load the value in a register before the > >> while loop, and never read it again. With the 'volatile' keyword, the > >> buff_head variable will be read from memory before every comparison. > >> Since buff_tail is not modified in the interrupt handler, it does not > >> need to be volatile. > > > thank you very much! > > >>> 3, can you please answer me another question: will a disabled > >>> interrupt pop up again after it is enabled? > >> On any sane architecture, yes. > > > so, if in a disable/enable interrupt secion, there are two bytes comes > > in, the first byte will be lost since the interrupt will appear only > > once after the enable interrupt instruction. am i righ? ( supposing i > > am talking about a UART with only one byte FIFO ). > > One thing wasn't clear from the original problem description: will the > fn. flush_receiving_buff() always clear the buffer to empty (for > example, if a complete packet is been received? > If so, you might consider using two receiving buffers, controlled by a > pointer. __interrupt on_byte_in() will load the buffer indicated by the > pointer. Once a full packet is received, the pointer is switched to the > second buffer, & the first is processed at your leisure. > This is the classic double buffer setup. Of course, the actual switching > of the pointer may need protecting as a critical section (eg by > disabling interrupts).
i understand this kind of double buffer setup. but for some reasons i dont want to put packet recognizing logical into the ISR. - woody
On Apr 4, 3:23 am, "Arlet" <usene...@c-scape.nl> wrote:
> On Apr 3, 8:45 pm, "Steven Woody" <narkewo...@gmail.com> wrote: > > > > > > > > Alternatively, you could use something similar to this, and leave > > > > > interrupts enabled: > > > > > > static unsigned char buff[ BUFF_SIZE ]; > > > > > static volatile unsigned char buff_head; > > > > > static unsigned char buff_tail; > > > > > > __interrupt on_byte_in( void ) > > > > > { > > > > > if ( (unsigned char) (buff_head - buff_tail) < BUFF_SIZE ) > > > > > buff[ buff_head++ % BUFF_SIZE ] = the_byte_in_the_register; > > > > > > } > > > > > > flush_receiving_buff( void ) > > > > > { > > > > > while( buff_head != buff_tail ) > > > > > process the buff[ buff_tail++ % BUFF_SIZE ]; > > > > > > } > > > > > > This works only if BUFF_SIZE < 256. Powers of two are recommended to > > > > > avoid modulo operation. If your compiler doesn't optimize away the > > > > > modulo, rewrite it as '& (BUFF_SIZE-1)'. For larger BUFF_SIZE, use a > > > > > bigger type. > > > > > Arlet, > > > > > 1, does you ment to say, by letting the on_byte_in wrte only one > > > > variable and letting the flush_receiving_buff write only another > > > > variable, hence the code above don't need to be pretected into a > > > > disable/enable interrupt section? > > > > True. Instead of having a single 'length', this code uses two > > > variables, and the 'length' is the difference between them. The > > > interrupt handler only changes head, and the main code only changes > > > tail, so there's no conflict. It does require atomic reads, though. On > > > an 8-bit architecture, where the CPU doesn't have a 16-bit read, > > > you'll be restricted to 8-bit head/tail counters. For most > > > applications on an 8-bit platform this is sufficient for a UART > > > buffer. > > > thanks for you answer. and, even in a 8-bit architecture, one need to > > receive a network packet which may longer than 255 bytes :-( in the > > case, data might be lost. am i right? > > You could have a 2-part solution. The first would be a smallish FIFO, > like above, big enough to store all the data that comes in while your > main code is busy with something else. In a second step, you pull all > the data out of the FIFO, and put it in a larger buffer for > processing. Only the first FIFO needs to worry about critical sections > and interrupts. > > If you don't want to do that, you'll need to disable interrupts in > some way, yes. > > > > > 3, can you please answer me another question: will a disabled > > > > interrupt pop up again after it is enabled? > > > > On any sane architecture, yes. > > > so, if in a disable/enable interrupt secion, there are two bytes comes > > in, the first byte will be lost since the interrupt will appear only > > once after the enable interrupt instruction. am i righ? ( supposing i > > am talking about a UART with only one byte FIFO ). > > Yes, if interrupts are disabled for an extended time, you may lose > data from your peripheral.
thank you very much, Arlet
In article <1175658659.811532.292920@y80g2000hsf.googlegroups.com>, "Steven Woody" <narkewoody@gmail.com> writes:
> On Apr 4, 3:23 am, "Arlet" <usene...@c-scape.nl> wrote: >> >> Yes, if interrupts are disabled for an extended time, you may lose >> data from your peripheral. > > > thank you very much, Arlet >
What I don't think I've seen anyone mention is the possibility of using RTS/CTS hardware flow control if you are worried about interrupts been disabled for an extended period. Simon. -- Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP Microsoft: Bringing you 1980's technology to a 21st century world
On 4=D4=C24=C8=D5, =CF=C2=CE=E78=CA=B104=B7=D6, clubley@remove_me.eisner.de=
cus.org-Earth.UFP (Simon
Clubley) wrote:
> In article <1175658659.811532.292...@y80g2000hsf.googlegroups.com>, "Stev=
en Woody" <narkewo...@gmail.com> writes:
> > > On Apr 4, 3:23 am, "Arlet" <usene...@c-scape.nl> wrote: > > >> Yes, if interrupts are disabled for an extended time, you may lose > >> data from your peripheral. > > > thank you very much, Arlet > > What I don't think I've seen anyone mention is the possibility of using > RTS/CTS hardware flow control if you are worried about interrupts been > disabled for an extended period. > > Simon. > > -- > Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP > Microsoft: Bringing you 1980's technology to a 21st century world
when and where to set the RTS/CTS pins? - woody
On Apr 3, 10:46 pm, "Steven Woody" <narkewo...@gmail.com> wrote:
> On Apr 4, 3:26 am, David R Brooks <daveb...@iinet.net.au> wrote: > > > > > Steven Woody wrote: > > > On 4=E6=9C=884=E6=97=A5, =E4=B8=8A=E5=8D=881=E6=97=B637=E5=88=86, "Ar=
let" <usene...@c-scape.nl> wrote:
> > >> On Apr 3, 7:13 pm, "Steven Woody" <narkewo...@gmail.com> wrote: > > > >>> On 4=E6=9C=884=E6=97=A5, =E4=B8=8A=E5=8D=8812=E6=97=B606=E5=88=86, =
"Arlet" <usene...@c-scape.nl> wrote:
> > >>>> On Apr 3, 5:11 pm, "John Speth" <johnsp...@yahoo.com> wrote: > > >>>>>> __interrupt on_byte_in( void ) > > >>>>>> { > > >>>>>> if ( buffer_len < sizeof( buff ) ) > > >>>>>> buffer[ buffer_len++ ] =3D the_byte_in_the_register; > > >>>>>> } > > >>>>>> flush_receiving_buff( void ) > > >>>>>> { > > >>>>>> for ( size_t i =3D 0; i < buff_len; ++i ) { > > >>>>>> process the buff[ i ]; > > >>>>>> buff_len =3D 0; ------------- (a) > > >>>>>> } > > >>>>>> my question is that, during the execution of statement (a), one =
or
> > >>>>>> more new bytes might come in and code above will lost them. if =
you
> > >>>>>> folks see the problem, can you please suggest me a solution? > > >>>>> Yes, it's basic. You'll need to disable your receive interrupt d=
uring
> > >>>>> statement (a) execution. Doing so will prevent the ISR from exec=
uting and
> > >>>>> changing buff_len while your flush function is also changing it. > > >>>> It would also be a good idea to declare 'buff_len' as volatile. > > >>>> Alternatively, you could use something similar to this, and leave > > >>>> interrupts enabled: > > >>>> static unsigned char buff[ BUFF_SIZE ]; > > >>>> static volatile unsigned char buff_head; > > >>>> static unsigned char buff_tail; > > >>>> __interrupt on_byte_in( void ) > > >>>> { > > >>>> if ( (unsigned char) (buff_head - buff_tail) < BUFF_SIZE ) > > >>>> buff[ buff_head++ % BUFF_SIZE ] =3D the_byte_in_the_regis=
ter;
> > >>>> } > > >>>> flush_receiving_buff( void ) > > >>>> { > > >>>> while( buff_head !=3D buff_tail ) > > >>>> process the buff[ buff_tail++ % BUFF_SIZE ]; > > >>>> } > > >>>> This works only if BUFF_SIZE < 256. Powers of two are recommended =
to
> > >>>> avoid modulo operation. If your compiler doesn't optimize away the > > >>>> modulo, rewrite it as '& (BUFF_SIZE-1)'. For larger BUFF_SIZE, use=
a
> > >>>> bigger type. > > >>> Arlet, > > >>> 1, does you ment to say, by letting the on_byte_in wrte only one > > >>> variable and letting the flush_receiving_buff write only another > > >>> variable, hence the code above don't need to be pretected into a > > >>> disable/enable interrupt section? > > >> True. Instead of having a single 'length', this code uses two > > >> variables, and the 'length' is the difference between them. The > > >> interrupt handler only changes head, and the main code only changes > > >> tail, so there's no conflict. It does require atomic reads, though. =
On
> > >> an 8-bit architecture, where the CPU doesn't have a 16-bit read, > > >> you'll be restricted to 8-bit head/tail counters. For most > > >> applications on an 8-bit platform this is sufficient for a UART > > >> buffer. > > > > thanks for you answer. and, even in a 8-bit architecture, one need to > > > receive a network packet which may longer than 255 bytes :-( in the > > > case, data might be lost. am i right? > > > >>> 2, what's the exact functionality of keyword 'volatile', what's the > > >>> different if don't use it? > > >> 'volatile' tells the compiler not to optimize away memory references. > > >> For example in the while loop: > > > >> while( buff_head !=3D buff_tail ) > > > >> Without the volatile, the compiler could assume that buff_head never > > >> changes, and may therefore load the value in a register before the > > >> while loop, and never read it again. With the 'volatile' keyword, the > > >> buff_head variable will be read from memory before every comparison. > > >> Since buff_tail is not modified in the interrupt handler, it does not > > >> need to be volatile. > > > > thank you very much! > > > >>> 3, can you please answer me another question: will a disabled > > >>> interrupt pop up again after it is enabled? > > >> On any sane architecture, yes. > > > > so, if in a disable/enable interrupt secion, there are two bytes comes > > > in, the first byte will be lost since the interrupt will appear only > > > once after the enable interrupt instruction. am i righ? ( supposing i > > > am talking about a UART with only one byte FIFO ). > > > One thing wasn't clear from the original problem description: will the > > fn. flush_receiving_buff() always clear the buffer to empty (for > > example, if a complete packet is been received? > > If so, you might consider using two receiving buffers, controlled by a > > pointer. __interrupt on_byte_in() will load the buffer indicated by the > > pointer. Once a full packet is received, the pointer is switched to the > > second buffer, & the first is processed at your leisure. > > This is the classic double buffer setup. Of course, the actual switching > > of the pointer may need protecting as a critical section (eg by > > disabling interrupts). > > i understand this kind of double buffer setup. but for some reasons i > dont want to put packet recognizing logical into the ISR. > > - > woody
I've used double buffering with packet/protocol logic in the ISR, and that works well. However, if you don't want the packet logic in the ISR, you could still use double buffering. The ISR could mark a buffer as ready for the upper level code, and then store more characters in the _other_ buffer. Ready could be set if the buffer is full, or if a certain amount of time has passed and no new characters have been received. The ISR can save the numbers of bytes received in a variable associated with that buffer (use a struct - see below). After the upper level code has read all of the chars from the buffer, it as available. The ISR can then use that buffer again. Disabling and restoring interrupts aren't needed. This eliminates the need for both the ISR code and the upper level code to write to the count (buff_len) variable. typedef struct { char nbr_bytes_read; /* 0..255 */ char OK_to_read_buffer; /* T/F */ char buffer[ BUFF_SZ ]; } buffer_stuff; buffer_stuff *buffer_1; buffer_stuff *buffer_2; -Dave Pollum
In article <1175692210.770492.61820@n59g2000hsh.googlegroups.com>, "Steven Woody" <narkewoody@gmail.com> writes:
> On 4=D4=C24=C8=D5, =CF=C2=CE=E78=CA=B104=B7=D6, clubley@remove_me.eisner.de= > cus.org-Earth.UFP (Simon > Clubley) wrote: >> >> What I don't think I've seen anyone mention is the possibility of using >> RTS/CTS hardware flow control if you are worried about interrupts been >> disabled for an extended period. >> > > when and where to set the RTS/CTS pins? >
You should drop CTS before disabling the interrupt, or if that's not possible, as the very first thing that you do in the interrupt handler. Raise CTS after enabling the interrupt, or if that's not possible, as the last thing before leaving the interrupt handler. You should also check if your setup requires you to invert the CTS signal. However, make sure that your device will deliver an interrupt for an action (in this case, receipt of a character) that occured while interrupts were disabled. Of course, the best solution would be to not have interrupts off for that long. Also, if you are driving the pin directly, don't forget to make sure that the voltages on the two devices are compatible so that you don't fry something. :-) I also recommend you try and find some online tutorials on this, so that you can see what the various issues are. Unfortunately, I don't have any pointers for you. BTW, does anyone here disagree with any of the above, or do this in some better way ? Simon. -- Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP Microsoft: Bringing you 1980's technology to a 21st century world
On 2007-04-04, Simon Clubley <clubley@remove_me.eisner.decus.org-Earth.UFP> wrote:

>>> What I don't think I've seen anyone mention is the possibility >>> of using RTS/CTS hardware flow control if you are worried >>> about interrupts been disabled for an extended period. >> >> when and where to set the RTS/CTS pins? > > You should drop CTS before disabling the interrupt, or if > that's not possible, as the very first thing that you do in > the interrupt handler. > > Raise CTS after enabling the interrupt, or if that's not > possible, as the last thing before leaving the interrupt > handler.
It's way simpler to just use an algorithm that doesn't have a race condition. Why not just use a circular buffer where the read/write index operation is atomic should work fine. -- Grant Edwards grante Yow! Someone in DAYTON, at Ohio is selling USED visi.com CARPETS to a SERBO-CROATIAN
On Apr 4, 1:03 pm, clubley@remove_me.eisner.decus.org-Earth.UFP (Simon
Clubley) wrote:
> In article <1175692210.770492.61...@n59g2000hsh.googlegroups.com>, "Steven Woody" <narkewo...@gmail.com> writes: > > > On 4=D4=C24=C8=D5, =CF=C2=CE=E78=CA=B104=B7=D6, clubley@remove_me.eisner.de= > > cus.org-Earth.UFP (Simon > > Clubley) wrote: > > >> What I don't think I've seen anyone mention is the possibility of using > >> RTS/CTS hardware flow control if you are worried about interrupts been > >> disabled for an extended period. > > > when and where to set the RTS/CTS pins? > > You should drop CTS before disabling the interrupt, or if that's not > possible, as the very first thing that you do in the interrupt handler. > > Raise CTS after enabling the interrupt, or if that's not possible, as > the last thing before leaving the interrupt handler. > > You should also check if your setup requires you to invert the CTS signal. > > However, make sure that your device will deliver an interrupt for an > action (in this case, receipt of a character) that occured while interrupts > were disabled. > > Of course, the best solution would be to not have interrupts off for that > long. > > Also, if you are driving the pin directly, don't forget to make sure that > the voltages on the two devices are compatible so that you don't fry > something. :-) > > I also recommend you try and find some online tutorials on this, so that > you can see what the various issues are. Unfortunately, I don't have any > pointers for you. > > BTW, does anyone here disagree with any of the above, or do this in some > better way ?
On CTS a caution at least. Using it in this fashion depends on - The serial interface on the other end actually having something to connect it to. - The serial interface on the other end responding more or less immediately. The first is questionable. On the second I think the worst I've seen is on the order of 20 characters or so sent after CTS is cleared. Robert