Forums

What is a Good way to Check Buffer not Empty in Windows?

Started by Rangarajan Varadan July 9, 2008
Hi All,
This question is related to COMM Port, When I make a Read Request or a Write Request to a Device through Serial port,
I get a response back from the board. Now the device is running an RTOS and the task with which it responds to my request runs when its turn comes.... it delivers a few bytes to the PC....
Now on a Windows OS(XP) what is a good condition to check for to see that those bytes have arrived.
i.e. I want to read based on a condition, since my data chunks vary insize....
It will perhaps be similar to !buffer_Empty condition on a Micro....
Please suggest....
Best regards
Ranga



An Engineer's Guide to the LPC2100 Series

--- In l..., Rangarajan Varadan wrote:
>
> Hi All,
> This question is related to COMM Port, When I make a Read Request or
a Write Request to a Device through Serial port,
> I get a response back from the board. Now the device is running an
RTOS and the task with which it responds to my request runs when its
turn comes.... it delivers a few bytes to the PC....
> Now on a Windows OS(XP) what is a good condition to check for to see
that those bytes have arrived.

ReadFile(), ReadFileEx(), WaitCommEvent(),.....

> i.e. I want to read based on a condition, since my data chunks vary
insize....

Windows is a multiThreaded, multiTasking OS. Your read thread will be
made ready/running, (eventually), when the driver gets bytes from the
UART.

Like TCP sockets, RS232 just transfers a byte stream, (though there is
an option in the driver to generate an event on one particular
character received - occasionally useful for 'line-based' protocols).

If you want to receive protocol units more complex than one byte at a
time, (or, at best, one chunk terminated by a single byte), then you
must implement a protocol on top of RS232 to parse the incoming stream
and so build up your protocol units.

> It will perhaps be similar to !buffer_Empty condition on a Micro....

Windows is not designed to 'uC RTOS' principles. Its I/O driver system
runs most I/O-centric apps tolerably well - those that can tolerate
relatively large I/O latencies. BitTorrent apps run fine, apps that
depend upon short, predictable I/O latencies to defice small packets
will not.

Using timers/timeouts to assemble protocol units is usually buggy and
generates poor performance at best since large latencies must be built
in to ensure that the APUs are, (mostly), recovered correctly.

Most Windows COM port components for high-level languages on Windows
will generate an 'rxData' event in your 'main thread' with a buffer
parameter of some length between 1 and bytes. You
can build APUs from this data, byte-by-byte, perhaps with a
case-switch. Alternatively, you can read the COM port in a separate
read thread and assemble/process APUs there, perhaps PostMessaging
them to other threads/main thread.

If you are using an RTOS on your uC, then presumably your
'!buffer_Empty' is only triggered in a CTI/RXD interrupt-handler to
empty the FIFO and you have the same issue - you do not know how many
bytes have been received until the FIFO is empty, just that there are
between 1 and 16.

Whatever.. there is no high-performance/error-free alternative to
reassembling your APUs from
byte-streams/single-character-terminated-lines.

Rgds,
Martin

You can use the SetCommTimeouts Win32 API function to let the ReadFile
function return as many bytes as there are available.
If the buffer is empty the ReadFile will block until the first byte is
available.

The first read will usually return only a single byte and the next read
will return the rest of the UART fifo.
This depends on the COM port driver you are using, but be prepared to
receive the data in multiple reads.

Regards,
Richard.
Rangarajan Varadan wrote:
> Hi All,
> This question is related to COMM Port, When I make a Read Request or a Write Request to a Device through Serial port,
> I get a response back from the board. Now the device is running an RTOS and the task with which it responds to my request runs when its turn comes.... it delivers a few bytes to the PC....
> Now on a Windows OS(XP) what is a good condition to check for to see that those bytes have arrived.
> i.e. I want to read based on a condition, since my data chunks vary in size ....
> It will perhaps be similar to !buffer_Empty condition on a Micro....
> Please suggest....
> Best regards
> Ranga
>
>
>
>
Hi Martin, Richard and David,

I currently do a

Serial.Senddata(&out_buff, byte_count);
Serial.Readdata(&in_buffer, byte_count);

If I were to break at the line above...wait for a few seconds and then let it run... my code works, but if I were to let it run without breaking the Read happens too quickly... so nothing is read.....I have tried to create a slight delay....but it hasn't worked.....

I was thinking that I will do a read based on a condition which will allow me to get the bytes. The Condition being "There is Data present which needs to be read"....

There is a line of code inside the Readdata which is
bReadStatus = ReadFile( m_hIDComDev, buffer, dwBytesRead, &dwBytesRead, &m_OverlappedRead );

This gives me at least 2 conditions Viz., bReadStatus is set or dwBytes has something in it... but both of these conditions will be useful after I make a first read...I don't want to make any reads until there is Data... when there is data I want to read till I've read all. This is my goal.
I wanted to check onwhat you suggested .....

"You can use the SetCommTimeouts Win32 API function to let the ReadFile function return as many bytes as there are available.
If the buffer is empty the ReadFile will block until the first byte is available."
is this how this is supposed to be?
if
{
Serial.Readdata(&buff, byte-count);
}
I never enter this if... seems like I might have something wrong in the parameters passed... but I am doing some Trial and error with it right now.....
"Most Windows COM port components for high-level languages on Windows will generate an 'rxData' event in your 'main thread' with a buffer parameter of some length between 1 and bytes. You can build APUs from this data, byte-by-byte, perhaps with a case-switch."
This is perhaps what I am looking for..... how can I do this without doing a read?
Thanks
Ranga

(!(Serial.SetCommunicationTimeouts(0,100,0,0,0)))
----- Original Message ----
From: mjames_doveridge
To: l...
Sent: Wednesday, July 9, 2008 9:25:01 PM
Subject: [lpc2000] Re: What is a Good way to Check Buffer not Empty in Windows?
--- In lpc2000@yahoogroups .com, Rangarajan Varadan wrote:
>
> Hi All,
> This question is related to COMM Port, When I make a Read Request or
a Write Request to a Device through Serial port,
> I get a response back from the board. Now the device is running an
RTOS and the task with which it responds to my request runs when its
turn comes.... it delivers a few bytes to the PC....
> Now on a Windows OS(XP) what is a good condition to check for to see
that those bytes have arrived.

ReadFile(), ReadFileEx() , WaitCommEvent( ),.....

> i.e. I want to read based on a condition, since my data chunks vary
insize....

Windows is a multiThreaded, multiTasking OS. Your read thread will be
made ready/running, (eventually) , when the driver gets bytes from the
UART.

Like TCP sockets, RS232 just transfers a byte stream, (though there is
an option in the driver to generate an event on one particular
character received - occasionally useful for 'line-based' protocols).

If you want to receive protocol units more complex than one byte at a
time, (or, at best, one chunk terminated by a single byte), then you
must implement a protocol on top of RS232 to parse the incoming stream
and so build up your protocol units.

> It will perhaps be similar to !buffer_Empty condition on a Micro....

Windows is not designed to 'uC RTOS' principles. Its I/O driver system
runs most I/O-centric apps tolerably well - those that can tolerate
relatively large I/O latencies. BitTorrent apps run fine, apps that
depend upon short, predictable I/O latencies to defice small packets
will not.

Using timers/timeouts to assemble protocol units is usually buggy and
generates poor performance at best since large latencies must be built
in to ensure that the APUs are, (mostly), recovered correctly.

Most Windows COM port components for high-level languages on Windows
will generate an 'rxData' event in your 'main thread' with a buffer
parameter of some length between 1 and bytes. You
can build APUs from this data, byte-by-byte, perhaps with a
case-switch. Alternatively, you can read the COM port in a separate
read thread and assemble/process APUs there, perhaps PostMessaging
them to other threads/main thread.

If you are using an RTOS on your uC, then presumably your
'!buffer_Empty' is only triggered in a CTI/RXD interrupt-handler to
empty the FIFO and you have the same issue - you do not know how many
bytes have been received until the FIFO is empty, just that there are
between 1 and 16.

Whatever.. there is no high-performance/ error-free alternative to
reassembling your APUs from
byte-streams/ single-character -terminated- lines.

Rgds,
Martin





Under .net do something like

if( Form1 :: com->IsOpen )
{
if( Form1 :: com->BytesToRead )
{
/* Grab one byte at a time */
try
{
* byte = Form1 :: com->ReadByte( );

res = ( u_int8 )1;
}
catch( ... )
{
/* Print system exception message */;;
}
}
}

Cheers
--------
Jim Stevens
Embedded Software Engineer

Mobile: 07747189863
E-mail: j...@pogogps.com
E-mail: j...@eircom.net

Origin Technologies Ltd.
Unit 5, The Piper Building, Peterborough Road, London SW6 3EF UK
London, UK (HQ)
Tel: ( +44 ) ( 0 )203 010 1000 Reading,
Tel: ( +44 ) ( 0 )118 9312975
www.pogogps.com
--------

for whatever reason the yahoo editor formatted my message weird... my questions again
"You can use the SetCommTimeouts Win32 API function to let the ReadFile function return as many bytes as there are available.
If the buffer is empty the ReadFile will block until the first byte is available."
is this how this is supposed to be?
if(!(Serial.SetCommun icationTimeouts( 0,100,0,0, 0))){
Serial.Readdata( &buff, byte-count);
}

I never enter this if... seems like I might have something wrong in the parameters passed... but I am doing some Trial and error with it right now.....
"Most Windows COM port components for high-level languages on Windows will generate an 'rxData' event in your 'main thread' with a buffer parameter of some length between 1 and bytes. You can build APUs from this data, byte-by-byte, perhaps with a case-switch. "
This is perhaps what I am looking for..... how can I do this without doing

----- Original Message ----
From: Rangarajan Varadan
To: l...
Sent: Thursday, July 10, 2008 10:10:43 AM
Subject: Re: [lpc2000] Re: What is a Good way to Check Buffer not Empty in Windows?
Hi Martin, Richard and David,

I currently do a

Serial.Senddata( &out_buff, byte_count);
Serial.Readdata( &in_buffer, byte_count);

If I were to break at the line above...wait for a few seconds and then let it run... my code works, but if I were to let it run without breaking the Read happens too quickly... so nothing is read.....I have tried to create a slight delay....but it hasn't worked.....

I was thinking that I will do a read based on a condition which will allow me to get the bytes. The Condition being "There is Data present which needs to be read"....

There is a line of code inside the Readdata which is
bReadStatus = ReadFile( m_hIDComDev, buffer, dwBytesRead, &dwBytesRead, &m_OverlappedRead );

This gives me at least 2 conditions Viz., bReadStatus is set or dwBytes has something in it... but both of these conditions will be useful after I make a first read...I don't want to make any reads until there is Data... when there is data I want to read till I've read all. This is my goal.
I wanted to check onwhat you suggested .....

"You can use the SetCommTimeouts Win32 API function to let the ReadFile function return as many bytes as there are available.
If the buffer is empty the ReadFile will block until the first byte is available."
is this how this is supposed to be?
if(!(Serial.SetCommun icationTimeouts( 0,100,0,0, 0)))

{
Serial.Readdata( &buff, byte-count);
}
I never enter this if... seems like I might have something wrong in the parameters passed... but I am doing some Trial and error with it right now.....
"Most Windows COM port components for high-level languages on Windows will generate an 'rxData' event in your 'main thread' with a buffer parameter of some length between 1 and bytes. You can build APUs from this data, byte-by-byte, perhaps with a case-switch. "
This is perhaps what I am looking for..... how can I do this without doing a read?
Thanks
Ranga
----- Original Message ----
From: mjames_doveridge
To: lpc2000@yahoogroups .com
Sent: Wednesday, July 9, 2008 9:25:01 PM
Subject: [lpc2000] Re: What is a Good way to Check Buffer not Empty in Windows?

--- In lpc2000@yahoogroups .com, Rangarajan Varadan wrote:
>
> Hi All,
> This question is related to COMM Port, When I make a Read Request or
a Write Request to a Device through Serial port,
> I get a response back from the board. Now the device is running an
RTOS and the task with which it responds to my request runs when its
turn comes.... it delivers a few bytes to the PC....
> Now on a Windows OS(XP) what is a good condition to check for to see
that those bytes have arrived.

ReadFile(), ReadFileEx() , WaitCommEvent( ),.....

> i.e. I want to read based on a condition, since my data chunks vary
insize....

Windows is a multiThreaded, multiTasking OS. Your read thread will be
made ready/running, (eventually) , when the driver gets bytes from the
UART.

Like TCP sockets, RS232 just transfers a byte stream, (though there is
an option in the driver to generate an event on one particular
character received - occasionally useful for 'line-based' protocols).

If you want to receive protocol units more complex than one byte at a
time, (or, at best, one chunk terminated by a single byte), then you
must implement a protocol on top of RS232 to parse the incoming stream
and so build up your protocol units.

> It will perhaps be similar to !buffer_Empty condition on a Micro....

Windows is not designed to 'uC RTOS' principles. Its I/O driver system
runs most I/O-centric apps tolerably well - those that can tolerate
relatively large I/O latencies. BitTorrent apps run fine, apps that
depend upon short, predictable I/O latencies to defice small packets
will not.

Using timers/timeouts to assemble protocol units is usually buggy and
generates poor performance at best since large latencies must be built
in to ensure that the APUs are, (mostly), recovered correctly.

Most Windows COM port components for high-level languages on Windows
will generate an 'rxData' event in your 'main thread' with a buffer
parameter of some length between 1 and bytes. You
can build APUs from this data, byte-by-byte, perhaps with a
case-switch. Alternatively, you can read the COM port in a separate
read thread and assemble/process APUs there, perhaps PostMessaging
them to other threads/main thread.

If you are using an RTOS on your uC, then presumably your
'!buffer_Empty' is only triggered in a CTI/RXD interrupt-handler to
empty the FIFO and you have the same issue - you do not know how many
bytes have been received until the FIFO is empty, just that there are
between 1 and 16.

Whatever.. there is no high-performance/ error-free alternative to
reassembling your APUs from
byte-streams/ single-character -terminated- lines.

Rgds,
Martin







Don't really see a way around blocking, that's why I always check to see if any data is there before I try to get it.

If you want to set up an event listener...that should work...there are some examples on the net

I have polling, as described in my last post...and it works fine

let me know how you get on i'm interested

Regards for now
Jim

--------
Jim Stevens
Embedded Software Engineer

Mobile: 07747189863
E-mail: j...@pogogps.com
E-mail: j...@eircom.net

Origin Technologies Ltd.
Unit 5, The Piper Building, Peterborough Road, London SW6 3EF UK
London, UK (HQ)
Tel: ( +44 ) ( 0 )203 010 1000 Reading,
Tel: ( +44 ) ( 0 )118 9312975
www.pogogps.com
--------

> This question is related to COMM Port, When I make a
> Read Request or a Write Request to a Device through Serial port,
> I get a response back from the board. Now the device is
> running an RTOS and the task with which it responds to my
> request runs when its turn comes.... it delivers a few bytes
> to the PC....
> Now on a Windows OS(XP) what is a good condition to check for
> to see that those bytes have arrived. i.e. I want to read based
> on a condition, since my data chunks vary in size ....
> It will perhaps be similar to !buffer_Empty condition on a Micro....
> Please suggest....

Here's your key comment:

"I want to read based on a condition, since my data chunks
vary in size ...."

Your data chunks vary, but you must be able to identify which
is coming over the serial port. The identification of the
data type should be a *fixed* number of bytes, eg. a 1-byte
or 4-byte value that can be used as the argument to a switch
statement, that is then used to read the variable number of
data packet bytes.

So in pseudo code:

while (1) {
status = read(fd, &packet_type, PACKET_TYPE_SIZE);
if (status ...

switch (packet_type) {

case TEMPERATURE_DATA:
status = read(fd, &temperature_data, TEMPERATURE_DATA_SIZE);
if (status ...)
break;

case ADC_DATA:
status = read(fd, &adc_data, ADC_DATA_SIZE);
if (status ...)
break;
...
}

The code first reads the packet type identifier using its known
size, and then reads the variable length data based on the
packet type.

I think that is a solution to what you were asking ... but
I may have misinterpreted your question ... if so, try to
be clearer.

Cheers,
Dave

> Don't really see a way around blocking

Under Linux, if I wanted to avoid blocking, I'd open the
file descriptors non-blocking (or set them to non-blocking)
and then use a select() call to monitor the descriptors.
For a serial port application, the select() call would return
with the file descriptor readable in the case where the
serial port had read data.

Most of the stuff I write uses the ACE C++ library, which
wraps the select() call as its ACE_Reactor object. Under
Windows I believe this code wraps the WaitForMultipleObjects()
call (or something like that).

Generally this type of code is use to react to external
events from multiple sources, eg. mouse clicks, keyboard
events, serial events, and multiple sockets handling.
However, it also works just fine for single handles.
But in that case, you could just use blocking calls.

Cheers,
Dave

> I currently do a
>
> Serial.Senddata(&out_buff, byte_count);
> Serial.Readdata(&in_buffer, byte_count);
>
> If I were to break at the line above...wait for a few seconds
> and then let it run... my code works, but if I were to let
> it run without breaking the Read happens too quickly... so
> nothing is read.....I have tried to create a slight delay....
> but it hasn't worked.....

If the serial port was set in blocking mode, then read
should block. So take a look at your Serial object's
initialization sequence. If its set in non-blocking mode,
then change it, or deal with the reads via non-blocking
methods, eg. WaitForMultipleObjects() (I think that is
what it is under Windows, its the analog to Linux's
select() call).

> I was thinking that I will do a read based on a condition
> which will allow me to get the bytes. The Condition being
> "There is Data present which needs to be read"....

For serial ports, you can probably ask that question.
For file descriptors in general, this gets covered by
blocking until data exists, or setting things up as
non-blocking, and using the select() user-space call.
The kernel driver implements the poll() routine under
Linux, which the OS uses to determine whether a device
is readable or not.

> There is a line of code inside the Readdata which is
> bReadStatus = ReadFile( m_hIDComDev, buffer, dwBytesRead,
> &dwBytesRead, &m_OverlappedRead );

Check into that last argument; its for asychronous I/O,
and can be used for 'future' events. I haven't used
Windows for a while, so can't comment further.

Another way to attack this problem is to write two processes
that use Windows pipes to emulate your serial interface.
Get the pipe working between two processes, and that'll
help you get a handle on what the OS wants, eg. with regards
to WaitForMultipleObjects. (VMWare has some example code
for using Windows pipes to act as the serial interface to
a VMWare machine).

Cheers,
Dave