Serial (rs232 etc.) to IP

On 2005-10-27, Ignacio G.T. <igtorque.remove@evomer.yahoo.es> wrote:

>> If you are using a protocol that was initially written for serial line
>> communication with normal CRC checks and timeout controls, why bother
>> with TCP, just use simple UDP. If the UDP frame is lost, let the
>> original serial line protocol timeout mechanism handle any missing
>> data.
>
> This has at least one problem: these serial line protocols usually 
> assume that packets arrive in order.

And unduplicated.  I see duplicate packets not infrequently on
my home wireless network, but I don't ever remember seeing
out-of-order packets (though I've never specifically looked for
them either).

> This makes sense for these protocols (I cannot imagine a
> message jumping over the previous one and reaching its destiny
> sooner than the other), but not for IP packets.

Any decent serial protocol would deal with out-of-order or
duplicated packets, but there are a lot of indecent protocols
in use.

-- 
Grant Edwards                   grante             Yow!  I'm rated PG-34!!
                                  at               
                               visi.com            

I'm not sure if I am missing something, but this seems like a stupid 
conversation :)

TCP is a connection based protocol and as such it ensures that all data is 
received in the correct order and without errors.

There is no need to 'use' another protocol like Telnet if all you wish to do 
is transfer a serial stream of characters from one place to another over IP. 
That is what TCP/IP does.

Open a socket on one end and listen.

Start sending characters to the remote socket.

TCP uses a sliding window mechanism, which is effectively a buffer of data 
that has been sent that has not yet been acknowledged. The sender will 
continue to send until this sliding window is full. As data is acknowledged 
from the remote end, the window opens up some more and more data can be 
sent.

The size of the window dictates how much data you can send before you must 
receive an ACK from the other end.

Every time a SYN is sent, the other end must ACK, unless the remote end has 
implemented delayed acknowledgement. You can normally decline delayed 
acknowledgement when the socket is first negotiated.

Job done!

Hope this helps.


"Ignacio G.T." <igtorque.remove@evomer.yahoo.es> wrote in message 
news:4360df29$0$26325$892e7fe2@authen.white.readfreenews.net...
> Paul Keinanen wrote:
>> On 26 Oct 2005 10:57:55 -0700, cs_posting@hotmail.com wrote:
>>
>>
>>>>..and by doing this also ruin the serial line throughput when a half
>>>>duplex protocol is used with short message frames. Some converters
>>>>wait 10-100 ms after the last serial character received, before the
>>>>(TCP or UDP) IP-frame is sent. For instance at 115200 bit/s, this
>>>>corresponds to 100-1000 character times.
>>>
>>>This is a valid concern, however there's a bit of a problem with single
>>>character packets when many common embedded TCP devices try to talk to
>>>common desktop operating systems.
>>>
>>>TCP requires that you hang onto all outgoing data until it has been
>>>acknowledged, because if it's not acknowledged you will have to resend
>>>it.  Most embedded implementations handle this by sending a packet, and
>>>then being unable to send another until the first has been
>>>acknowledged.  The Windows TCP stack often takes as much as 200 ms to
>>>acknowledge a packet when only one is outstanding, so single character
>>>packets can be slowed down to the rate of only five per second!
>>
>>
>> If you are using a protocol that was initially written for serial line
>> communication with normal CRC checks and timeout controls, why bother
>> with TCP, just use simple UDP. If the UDP frame is lost, let the
>> original serial line protocol timeout mechanism handle any missing
>> data.
>>
>
> This has at least one problem: these serial line protocols usually assume 
> that packets arrive in order. This makes sense for these protocols (I 
> cannot imagine a message jumping over the previous one and reaching its 
> destiny sooner than the other), but not for IP packets. 

Sonic wrote:
> I'm not sure if I am missing something, but this seems like a stupid
> conversation :)
>
> TCP is a connection based protocol and as such it ensures that all data is
> received in the correct order and without errors.

Yes, but in this corner of the thread they are talking about a proposal
to not use TCP, but rather use something like UDP to carry some legacy
serial protocol that does it's own reliability checking - which leads
to comments about how that task is different for different sorts of
connection.

Using TCP is a good alternative... it's just not the only one.

On Thu, 27 Oct 2005 16:55:49 GMT, "Sonic" <spam@spam.com> wrote:

>There is no need to 'use' another protocol like Telnet if all you wish to do 
>is transfer a serial stream of characters from one place to another over IP. 
>That is what TCP/IP does.

TCP/IP is useful when you just want to transfer error-free data but
without any realtime constraints. 

However, in many realtime systems, the TCP/IP is directly harmful due
to the unpredictable propagation delay, which sometimes delivers data
(say 10 ms) too late, in which case it is useless and the resends just
loads the network by sending this outdated data. 

Many realtime systems work much better with direct serial or raw
Ethernet or UDP systems. If the data frame does not arrive within the
expected time, it is simply discarded and the sample is marked as
invalid at the receiver end and processed accordingly. 

Paul
 

Two manufactures that I know of XPOR and DIgi make ehternet modules
that have serial port interfaces.  THey both have methods to connect
that serial port out and across the internet.

So if you look at their data sheets I'm sure you'll find how that
approach this problem.

George

On 2005-10-27, Sonic <spam@spam.com> wrote:

> I'm not sure if I am missing something, but this seems like a
> stupid conversation :)
>
> TCP is a connection based protocol and as such it ensures that
> all data is received in the correct order and without errors.
>
> There is no need to 'use' another protocol like Telnet if all
> you wish to do is transfer a serial stream of characters from
> one place to another over IP.  That is what TCP/IP does.

Right, but how do you set the baud rate and parity?  How do you
get notified when a break or parity error was received?

> Open a socket on one end and listen.
>
> Start sending characters to the remote socket.

You apparently have never used a UART or serial port.  There's
a lot more to it that "start sending characters".

> TCP uses a sliding window mechanism, which is effectively a
> buffer of data that has been sent that has not yet been
> acknowledged. The sender will continue to send until this
> sliding window is full. As data is acknowledged from the
> remote end, the window opens up some more and more data can be
> sent.
>
> The size of the window dictates how much data you can send
> before you must receive an ACK from the other end.
>
> Every time a SYN is sent, the other end must ACK, unless the
> remote end has implemented delayed acknowledgement. You can
> normally decline delayed acknowledgement when the socket is
> first negotiated.
>
> Job done!

Hardly.

-- 
Grant Edwards                   grante             Yow!  YOW!!! I am having
                                  at               fun!!!
                               visi.com            

The number of the port that you use is generally irrelevant other than both 
ends of the connection need to know which port they are both using.

Specific protocols tend to use specific port numbers and these are referred 
to as "Well Known Ports"

As you may have seen when using a web browser (HTTP Client), you can 
actually connect to a web server (HTTP server) using any port that the 
server is listening on. I think it goes something like 
http://192.168.0.123:1234, where 1234 is the port you are choosing to 
connect to the HTTP server on. For this to work, the HTTP server must be 
listening on 1234 instead of (or as well as) port 80.

Key reasons why you would use a well known port number are:
. So your request from the client arrives on a port that the server is 
listening on
. So your traffic is able to break through firewall configurations
. So your traffic can be proxied
. So your traffic can be routed with quality of service applied

Other than that, you could have a server listening on port 80 (HTTP) for any 
type of connection, including your stream of serial data.

For socket communication, you create a socket, 'bind' the socket to a port, 
and then listen on the port.

When a client connects to the port that you are listening on, you can then 
start reading from the port. You can read one character at a time if you 
wish. In this way your server (that is receiving data) can receive as much 
data as it can at a speed that it is able.

The easiest way to choose an 'arbitrary' port is to choose one outside of 
the well known numbers or choose a well known port for an obscure protocol.

Even if you only send one character, the nature of Ethernet is that your 
single character might not get to its destination. TCP will ensure that the 
character is resent until it is either acknowledged or the connection times 
out.

The telnet protocol does much more than just transferring characters. TCP 
does that without requiring telnet.




<packer44@hotmail.com> wrote in message 
news:1130335823.986982.133040@g49g2000cwa.googlegroups.com...
> Frieder,
>   Thanks again.  The reason I mention using singel character mode is
> that if the system is to be "blind" (to the serial protocol) through
> the TCP transport, then you have to transmit a single character at a
> time, since some messages may be a single character, but I suppose a
> good bet for better efficiency would be to send what data you have
> after something like 2-3 character times elapse since the last recieved
> character at the serial end.
>   One last thought ... how does one choose an arbitrary TCP or UDP
> port?  I'll take a look through the RFC's
>
> Thanks Again,
> -D
> 

packer44@hotmail.com wrote:
> Hi,
>
> Could anyone lend some light into which protocol or technique is used
> to actually send I/O from a legacy serial port device onto a TCP/IP
> network?  I see there are many products out there, but none of the
> product sites describe the actual protocol used on the IP network side.
>  An RFC search also came up with nothing, though I could have missed it
> as well...  I know that on the client side (such as a Windblows
> machine) the IP packets can also then be accessed as a COM port from
> the software side...
>
> TIA,
> -D

There is no standar protocol defined for such a thing. The Serial to
TCP/IP bridge (given an Ethernet or Wifi based network) just forwards
the serial data to a TCP or UDP port using proprietary protocols.

In the case of TCP, the bridge simply waits for incomming connections
from a remote host (it acts like a server) or it gets connected to a
remote host before sending any data (it acts like a client) on either
case there must be another host for the bridge to send/receive any COM
data.

In the case of UDP, the same can be implemented plus the ability to
"broadcast" or "multicast" serial data. A Windows driver simply mimics
a serial COM port as a descriptor for the TCP/IP socket required by the
driver pretty similar on how it is done for the USB-to-Serial dongles
out there.

A simple serial to TCP/IP bridge can be implemented in the now old
Visual Basic 6 with no more than 20 lines of code to send/receive pure
raw serial data. I can post the code here if some one is interested.

"Grant Edwards" <grante@visi.com> wrote in message 
news:11m273c4bu05484@corp.supernews.com...
> On 2005-10-27, Sonic <spam@spam.com> wrote:
>
>> I'm not sure if I am missing something, but this seems like a
>> stupid conversation :)
>>
<SNIP>
>
> Right, but how do you set the baud rate and parity?  How do you
> get notified when a break or parity error was received?

I can not find any mention in this thread of baud rate or parity needing to 
be maintained over this TCP/IP connection.

If you picture the primitve technology between a PC and a modem, the parity, 
baud, stop bits, etc. are configured between the serial port and the modem. 
The modem then negotiates settings with the remote peer. The remote peer has 
no notion of the fact that the PC is running at 115,200 with 1 stop bit and 
no parity, or whatever. It is irrelevant. The remote peer has configured its 
own method of communicating with the telephone line. This might be an 
Ethernet aggregator, in which case, there is no baud rate unless you 
consider 100Mbit Ethernet to have a baud rate.

>
>> Open a socket on one end and listen.
>>
>> Start sending characters to the remote socket.
>
> You apparently have never used a UART or serial port.  There's
> a lot more to it that "start sending characters".

I try not to make those type of judgements about people that I speak with in 
Usenet, more often than not those judgements would be wrong.

The essence of this thread is that UART is history...how do we get rid of 
them?

The original author's question was regarding how to transport the serial 
data over a TCP/IP network. I agree with you that clearly the data being 
received from the legacy device would be obtained at a particular rate, but 
I think it is safe to assume that that part of the system has been working 
for some time...hence the legacy.

What I believe we are dealing with here is that data is arriving at a UART 
at a given baud rate. We don't care what rate or parity, other than to 
ensure that we receive it all and don't lose any. Once the data is in the 
memory of our PC or embedded device, we are then trying to figure out how to 
transport it over the Internet.

At this point comes my statement...

On the remote peer:
Open a socket on the remote peer.
Bind that socket to a port
Listen on the port for a connection
While (not terminated)
{
Receive data;
do your thang;
}

At the sender:
While (Carrier detect)
{
Receive serial data (having negotiated all that tedious UART stuff long ago)
Send TCP data (to remote port)
}

If you use TCP then you could make the window size extremely small or make 
it larger. You could use delayed ACK or not. You could use selective ACK or 
NOT.

The problem is just a tuning issue to deal with the environment...which is 
also something that TCP is extremely good at.

>> Job done!
>
> Hardly.
>

'tis so! :) 

Sonic wrote:

> I can not find any mention in this thread of baud rate or parity needing to
> be maintained over this TCP/IP connection.

No, but both are things we could reasonably expect might be needed.

It's no the maintenance of baud rate that is important, but the serial
side of the converter does need to be configured with the right baud
rate (and word length and stop bits and parity setting) somehow.  Your
options broadly speaking are hard coding, dip switches, auto detect, or
a configuration command going over the ethernet side of the link.
Similarly, the ethernet side needs to be configured for IP address,
port number, protocol etc with similar options for how to do it.

Parity is a possible serial error condition, which if present could
need to be reported to the device on the ethernet side.  You have to
figure out how you will signal that.  Similarly hardware flow control.
You also may have status and errors from any protocol on the ethernet
side, for which it may be necessary to pass through to the serial
device somehow.

Hard coding or dip switch configuration and ignoring error conditions
can make this all very simple... but that may not be acceptable in the
(undescribed) appliction.