Linux serial port dropping bytes

David Brown wrote:
> Didi wrote:
>> CBFalconer wrote:
>>> ....
>>> That depends on your CPU speed.  Within the interrupt, you have to
>>> handle something like:
>>>
>>>   REPEAT
>>>    test for more in FIFO
>>>    take one, stuff in buffer, while checking for buffer full.
>>>    test for overflow or other errors.
>>
>> So far OK, although it sounds more complex than it actually is -
>> 128 bytes will be processed within 1000 CPU cycles easily, which
>> would be taking all the time on a 1 MHz CPU...
>>
>>>    if any, call appropriate handler
>>
>> You don't call any "handlers" from an IRQ service routine. You
>> set a flag bit somewhere to indicate what happened and let the
>> non-time critical code deal with it.

That action is a 'handler'.  Bear in mind that errors need to be
tied to the appropriate point in the buffered stream.  
> 
> In this case (a fast UART on a Linux system), you don't want to do
> anything with the incoming data except buffer it - processing is
> done in a different process/thread, as you suggest.  But it's worth
> noting that in some embedded systems, it makes a lot of sense to do
> more specific handling of data during interrupt routines - interrupt
> handlers do not necessarily need to be as fast as possible, only as
> fast as necessary. If you have a system where you have better
> knowledge of the interrupts, the response times, and the required
> times, then you are free to do all the work you want during an
> interrupt routine.

This is fine IF you have adequate processing time left after the
interrupts.  You always have to keep an eye on the available
throughput for the system.


-- 
 [mail]: Chuck F (cbfalconer at maineline dot net) 
 [page]: <http://cbfalconer.home.att.net>
            Try the download section.



-- 
Posted via a free Usenet account from http://www.teranews.com

Didi wrote:
> David Brown wrote:
>> ...
>> If you have a system where you have better knowledge of the
>> interrupts, the response times, and the required times, then you
>> are free to do all the work you want during an interrupt routine.
> 
> Why would you ant to do it there?  Interrupts are meant to be as
> short as possible and do only what cannot be done outside their
> handlers - this is fundamental to programming. I know it can be
> done otherwise, and I know they do such a mess to no direct
> consequences because most of the hardware nowadays is 10x to
> 1000+x overkill, but why want to do it so?

The ONLY reason for that attack is to preserve a maximum of cpu
time for the 'unknown' projects.  If you know precisely what the
system has to do, the most efficient mechanism is desireable.  It
may also make the system better partitioned.  For example, in the
serial input scheme, nothing outside the interrupt system needs to
know anything about the i/o ports, etc.

-- 
 [mail]: Chuck F (cbfalconer at maineline dot net) 
 [page]: <http://cbfalconer.home.att.net>
            Try the download section.



-- 
Posted via a free Usenet account from http://www.teranews.com

Grant Edwards wrote:
> CBFalconer <cbfalconer@yahoo.com> wrote:
> 
... snip ...
> 
>> Note that some operations will require several accesses to the
>> UART.  Those will eat up time.  They will be much slower than
>> on-chip memory access.
> 
> People have been supporting 921K bps serial links for ages. You
> do have to pay attention to what you're doing, but it's really
> not that hard with a sufficiently large FIFO.  However, IMO a
> 128 FIFO is getting close to being insufficiently large.  I
> wouldn't want to try to support it on a Linux system with
> interrupt latencies imposed by a bunch of randomly chosen
> device drivers. If it's an embedded system and you've got
> control over what other ISRs are running, it should be doable.

Well, I am not familiar with the particular hardware the OP is
using, and having started out with a ridiculous misapprehension
doesn't help my reliability reputation.  However, I have just been
trying to point out the sort of things to consider.

-- 
 [mail]: Chuck F (cbfalconer at maineline dot net) 
 [page]: <http://cbfalconer.home.att.net>
            Try the download section.



-- 
Posted via a free Usenet account from http://www.teranews.com

Hans-Bernhard Br=F6ker wrote:
> For just one example, let's consider that you're running XON/XOFF flow
> control on a plain old RS232 link.  That would mean even an interrupt
> handler that would normally just stuff each byte received by the UART
> into some software FIFO, had better look at the actual character, too,
> to check if it's XOFF.

Yes, XON/XOFF processing obviosly falls in the category of things
you cannot do outside of the IRQ handler in a reasonable way.

Another example in that line, which demonstrates to a further extent
how the work within the ISR must be minimised, it related to sending
the XOFF, when your ISR sees that the FIFO is nearly full.
 Generally you cannot send the XOFF from within the receive ISR
because you may have to wait almost an entire serial character time
for the UART to be able to take it - this would be about 1 mS at 9600
bpS.
 The way it is done (OK, the way I do it) is to flag the fact that
XOFF is due, and make sure the transmit IRQ will be asserted as soon
as possible (i.e. after the current character has gone out or
immediately);
then, within the imminent transmit IRQ handler the flag is detected
and
the first thing which is sent - overriding the output queue and
possibly some
Tx FIFO - is a XOFF.

 I believe this illustrates pretty well what I mean by saying:

> > Interrupts are meant to be as short as possible and do only what
> > cannot
> > be done outside their handlers - this is fundamental to programming.

I wish I could remember when I first did it this way - was it on a
6800
or on a 6809, I really don't know. Perhaps a 6800, I did a terminal
with
it around 1985, may be it has been then.
Not such a demand for XON/XOFF things nowadays, UARTs, however
popular still, seem to be in the "phase out" phase - which may take
a few decades, that is :-).

Dimiter

------------------------------------------------------
Dimiter Popoff               Transgalactic Instruments

http://www.tgi-sci.com
------------------------------------------------------
http://www.flickr.com/photos/didi_tgi/sets/72157600228621276/


Hans-Bernhard Br=F6ker wrote:
> Didi wrote:
>
> > Interrupts are meant to be as short as possible and do only what
> > cannot
> > be done outside their handlers - this is fundamental to programming.
>
> Of course.  But sometimes "what cannot be done outside" does include
> some, or even all the processing of that incoming data.   Resources or
> response time constraints might not allow any other approach.
>
> For just one example, let's consider that you're running XON/XOFF flow
> control on a plain old RS232 link.  That would mean even an interrupt
> handler that would normally just stuff each byte received by the UART
> into some software FIFO, had better look at the actual character, too,
> to check if it's XOFF.

In article <66adnW6htpJRI27aRVnygQA@lyse.net>, David Brown says...
> CBFalconer wrote:
> <snip>
> > That depends on your CPU speed.  Within the interrupt, you have to
> > handle something like:
> > 
> >   REPEAT
> >    test for more in FIFO
> >    take one, stuff in buffer, while checking for buffer full.
> >    test for overflow or other errors.
> >    if any, call appropriate handler
> >   UNTIL FIFO empty
> >   clear interrupt system
> >   rearm interrupt
> >   exit
> > 
> > Note that some operations will require several accesses to the
> > UART.  Those will eat up time.  They will be much slower than
> > on-chip memory access.
> > 
> 
> This stuff is not magic - it's standard fare for embedded developers. 
> You seem determined to view the problem from the worst possible angle, 
> and pick the worst possible solution.  You do *not* have to check for 
> overflows or other receive errors for each byte (buffered uarts provide 
> summary flags, and you would normally use higher level constructs, such 
> as crc checks, to check correctness on a fast link).  You do *not* have 
> to check for space in your buffer for each byte.  At the start of the 
> ISR, you ask the UART how many bytes are in the FIFO buffer, and you 
> check how much space you have in the memory buffer.  That tells you how 
> often to execute your loop.

Of course this assumes the UART can tell you how many characters are in 
the FIFO.  I don't remember any of the ones I've used having that 
capability (not to suggest they don't esist, just that it's not rare for 
it not to be there).

Robert

-- 
Posted via a free Usenet account from http://www.teranews.com

CBFalconer wrote:
> Didi wrote:
> > David Brown wrote:
> >> ...
> >> If you have a system where you have better knowledge of the
> >> interrupts, the response times, and the required times, then you
> >> are free to do all the work you want during an interrupt routine.
> >
> > Why would you ant to do it there?  Interrupts are meant to be as
> > short as possible and do only what cannot be done outside their
> > handlers - this is fundamental to programming. I know it can be
> > done otherwise, and I know they do such a mess to no direct
> > consequences because most of the hardware nowadays is 10x to
> > 1000+x overkill, but why want to do it so?
>
> The ONLY reason for that attack is to preserve a maximum of cpu
> time for the 'unknown' projects.  If you know precisely what the
> system has to do, the most efficient mechanism is desireable.  It
> may also make the system better partitioned.  For example, in the
> serial input scheme, nothing outside the interrupt system needs to
> know anything about the i/o ports, etc.

Well, the thing is, doing it this way typically one wastes more
resources.
Let us stick to the UART example and PPP.
If you do all within the ISR, this means you will have to add
at least the following overhead for each incoming character:
-retrieve/update the current state (flag $7e seen/not etc.)
-retrieve/update the pointer where to put the character
-retrieve some CRC table pointer
-retrieve/update the CRC value itself
-save/restore some more registers needed to do the above.

If you do this on a larger  block of characters (tens or hundreds),
you will do the above tens or hundreds of times less.

As usual, things must be considered on a per case basis, but
the general rule stands - do in the ISR only what you cannot do
elsewhere. If you think doing more there will save you time or effort,
think again :-).
I don't claim there can be no exceptions to this rule, of course,
but in the vast majority of cases this would be both the best and
likely the simplest way to do it (although it takes some less
straight forward/layered thinking, it is in effect the simpler way).

Dimiter

------------------------------------------------------
Dimiter Popoff               Transgalactic Instruments

http://www.tgi-sci.com
------------------------------------------------------
http://www.flickr.com/photos/didi_tgi/sets/72157600228621276/

Didi wrote:
> CBFalconer wrote:
>> Didi wrote:
>>> David Brown wrote:
>>>> ...
>>>> If you have a system where you have better knowledge of the
>>>> interrupts, the response times, and the required times, then you
>>>> are free to do all the work you want during an interrupt routine.
>>> Why would you ant to do it there?  Interrupts are meant to be as
>>> short as possible and do only what cannot be done outside their
>>> handlers - this is fundamental to programming. I know it can be
>>> done otherwise, and I know they do such a mess to no direct
>>> consequences because most of the hardware nowadays is 10x to
>>> 1000+x overkill, but why want to do it so?
>> The ONLY reason for that attack is to preserve a maximum of cpu
>> time for the 'unknown' projects.  If you know precisely what the
>> system has to do, the most efficient mechanism is desireable.  It
>> may also make the system better partitioned.  For example, in the
>> serial input scheme, nothing outside the interrupt system needs to
>> know anything about the i/o ports, etc.
> 
> Well, the thing is, doing it this way typically one wastes more
> resources.
> Let us stick to the UART example and PPP.
> If you do all within the ISR, this means you will have to add
> at least the following overhead for each incoming character:
> -retrieve/update the current state (flag $7e seen/not etc.)
> -retrieve/update the pointer where to put the character
> -retrieve some CRC table pointer
> -retrieve/update the CRC value itself
> -save/restore some more registers needed to do the above.
> 
> If you do this on a larger  block of characters (tens or hundreds),
> you will do the above tens or hundreds of times less.
> 
 > As usual, things must be considered on a per case basis

Particular cases can be pretty overwhelming.  We have an application 
where serial data comes in containing a distinguished end-of-frame 
value, and some escapes that ensure that data bytes are never mistaken 
for end-of-frame markers.  Looking at generated code for PIC18s or 
AVRs shows that:
  - retrieving a state flag (e.g. in-escape/not-in-escape) is one of 
the cheapest things we do -- single byte fetch from a known location
  - subscripted access to a data buffer is one of the most expensive 
things we do
So it makes sense to un-escape our data and detect frame boundaries 
while the incoming bytes are in our hands -- in the interrupt routine. 
  Less time wasted on laborious address arithmetic.  It helps that 
processing this serial input is the critical task for this 
application.  There is not a more important task that would be kept 
waiting.

	Mel.

Didi wrote:
> David Brown wrote:
>> ...
>> If you have a system where you have better knowledge of the interrupts,
>> the response times, and the required times, then you are free to do all
>> the work you want during an interrupt routine.
> 
> Why would you ant to do it there?
> Interrupts are meant to be as short as possible and do only what
> cannot
> be done outside their handlers - this is fundamental to programming.
> I know it can be done otherwise, and I know they do such a mess to
> no direct consequences because most of the hardware nowadays
> is 10x to 1000+x overkill, but why want to do it so?
> 

No, interrupts are not "meant to be as short as possible" - they are a 
feature of the processor, to be used as appropriate in the given system. 
  If you have an OS handling multiple threads or processes, then 
normally you want your interrupts to be as short as possible.  But there 
are many different ways to structure the software in an embedded system, 
and doing real work during interrupt routines is a perfectly good way to 
do it - as long as you are aware of the issues.

Interrupts can give you many of the benefits of a multi-threading RTOS 
while keeping the system as simple as possible - they let you do things 
in the background.  I've written systems where the communication system 
is handled entirely within the interrupt routines - telegrams are 
checked as they come in, handled when the packet ends, and replies sent 
out again from within the interrupt routines.  In fact, I've written 
systems where the *entire* running program is handled by interrupts - 
the "main loop" is nothing more than a "sleep()" function.

Clearly, you need to think about response times, sharing data, nested 
interrupts, and many other issues with interrupts - the more work you do 
during the interrupt routines, the more relevant these issues become. 
But interrupt routines can give you a convenient event-response 
structure that is fast and built into the hardware - why bother doing 
things indirectly (interrupt routines setting flags to be read by other 
threads) or using extra RTOS software if it's not actually necessary?

> No, interrupts are not "meant to be as short as possible"

Yes they are. This is fundamental.
They can be misused and make lengthier - many people do it - and
up to a point this may even be reasonable.

> ...
> out again from within the interrupt routines.  In fact, I've written
> systems where the *entire* running program is handled by interrupts -
> the "main loop" is nothing more than a "sleep()" function.

What advantages did this approach buy you compared to polling?

Dimiter

------------------------------------------------------
Dimiter Popoff               Transgalactic Instruments

http://www.tgi-sci.com
------------------------------------------------------
http://www.flickr.com/photos/didi_tgi/sets/72157600228621276/

David Brown wrote:
> Didi wrote:
> > David Brown wrote:
> >> ...
> >> If you have a system where you have better knowledge of the interrupts,
> >> the response times, and the required times, then you are free to do all
> >> the work you want during an interrupt routine.
> >
> > Why would you ant to do it there?
> > Interrupts are meant to be as short as possible and do only what
> > cannot
> > be done outside their handlers - this is fundamental to programming.
> > I know it can be done otherwise, and I know they do such a mess to
> > no direct consequences because most of the hardware nowadays
> > is 10x to 1000+x overkill, but why want to do it so?
> >
>
> No, interrupts are not "meant to be as short as possible" - they are a
> feature of the processor, to be used as appropriate in the given system.
>   If you have an OS handling multiple threads or processes, then
> normally you want your interrupts to be as short as possible.  But there
> are many different ways to structure the software in an embedded system,
> and doing real work during interrupt routines is a perfectly good way to
> do it - as long as you are aware of the issues.
>
> Interrupts can give you many of the benefits of a multi-threading RTOS
> while keeping the system as simple as possible - they let you do things
> in the background.  I've written systems where the communication system
> is handled entirely within the interrupt routines - telegrams are
> checked as they come in, handled when the packet ends, and replies sent
> out again from within the interrupt routines.  In fact, I've written
> systems where the *entire* running program is handled by interrupts -
> the "main loop" is nothing more than a "sleep()" function.
>
> Clearly, you need to think about response times, sharing data, nested
> interrupts, and many other issues with interrupts - the more work you do
> during the interrupt routines, the more relevant these issues become.
> But interrupt routines can give you a convenient event-response
> structure that is fast and built into the hardware - why bother doing
> things indirectly (interrupt routines setting flags to be read by other
> threads) or using extra RTOS software if it's not actually necessary?

Didi wrote:
>> No, interrupts are not "meant to be as short as possible"
> 
> Yes they are. This is fundamental.

No, this is not "fundamental".  The only thing "fundamental" in embedded 
programming is that the best solution depends on the system you are 
working on.

> They can be misused and make lengthier - many people do it - and
> up to a point this may even be reasonable.
> 

They can be used in different ways for different purposes - just because 
a particular tool is used in a way you are not familiar with, does not 
make it "misuse".

>> ...
>> out again from within the interrupt routines.  In fact, I've written
>> systems where the *entire* running program is handled by interrupts -
>> the "main loop" is nothing more than a "sleep()" function.
> 
> What advantages did this approach buy you compared to polling?
> 

The code was smaller, faster, neater, clearer, and spent more of the 
time asleep.