Donald wrote:
> Randall Nortman wrote:
>
> > That explanation should serve as an indication that you need to do
> > some serious reading if you're going to wrap your mind around all
> > this. There are many books on the subject -- TCP/IP Illustrated comes
> > to mind. Google might also turn some stuff up, and try wikipedia.org
> > too.
> >
>
> Randall,
>
> Thank you for a starter. I have been studing tcp/ip already.
>
> But, in my embedded app, I have a choice of protocols.
>
> I think I understand the 'packets', but a telenet session is byte based.
> I thought that this would be a good place to start. one byte for one
> byte, like a serial port.
>
> Is there any projects, PC based that would show this protocol in operation ?
>
> I have lots of PCs around, so getting some code that I can compile and
> step thru so I can see the sequence of events would help explain what is
> happening.
>
> So if anyone has such a code example, please post here.
First, let's get some terminology straight. Ethernet has frames. You
put an IP packet (sometimes called a datagram) in an Ethernet frame.
You put a TCP segment inside an IP packet. You put chunks of your data
inside the TCP segments. Of course if your application is using UDP
instead of then you have UPD datagrams inside the IP packet, and your
data inside the UDP datagram.
The terminology is not that big a deal, and the words frame and packet
are quite commonly misused. Once you get a handle on this stuff you'll
pretty automatically figure out which one is really meant, but as your
just getting started, knowing the real terms will probably help.
Second, don't try to understand TCP/IP in terms of the seven-layer OSI
model. TCP predates the OSI model, and despite repeated attempts to
mash it into that model, fits rather poorly. Nor do any of the specs
actually match up with the OSI model. In short, TCP/IP has about three
layers in the protocol, plus your application on top, plus the physical
layer underneath.
>From the bottom up, a TCP/IP stack will consist of the link layer,
where an IP packet is somehow moved across a communications link. Each
type of LAN (Ethernet, token-ring, FDDI, etc.) for example, has a
specification for how that takes place. In the case of Ethernet, the
specification spells out how to set up the Ethernet headers, maximum
frame sizes, and how one IP host can find another on the same LAN. A
quite similar specification exists for Token-Ring LANs (although there
are some differences). When you use a dial-up connection, the
link-layer protocol is usually something called PPP - which is the
specification of how to move an IP packet across a serial link.
Obviously and two nodes that want to talk across some communications
link need to use the same mapping.
Now below the link layer is the physical layer. Somewhat loosely, your
application will care little about the physical layer, except, of
course, that the correct hardware has to be there. For example,
Ethernet comes in 1*, 10, 100, 1000, and 10,000Mbps versions, and runs
over several types of coaxial cable, twisted pair, several varieties of
optical fiber, and even various wireless links at times. The layer
exists, but doesn't usually impact your software stack very much.
The link layer in a real implementation is usually split into an upper
and lower half. The upper half does the specified mapping (say IP
packet into an Ethernet frame), and the lower part drives the physical
I/O device (the Ethernet chip, for example). An OS like Windows or
Linux typically separates the two pieces (eg. the IP-to-Ethernet layer
comes from Microsoft, and the lower part is the "device driver" you get
from the network card manufacturer). An embedded system often has
those two pieces closely coupled.
The middle layer is the IP packet. The IP packet has a relatively
simple header that most notably specifies the source and destination IP
addresses, as well as the type of protocol contained within the IP
packet. While there are some complexities (options and fragmentation
for example), most of those can be ignored and then there's really darn
little in the IP header. All TCP/IP hosts (any machine with a TCP/IP
stack is a "host") exchange IP packets across the Internet. That *all*
they do. All data communications breaks down to a host building an IP
packet, and then sending to either a directly connected neighbor, or a
directly connected router. It never sees it again.
The upper layer is the transport protocol. In TCP/IP the two most
common transport protocols are TCP and UDP. UDP is a simple datagram
based protocol, which provides no delivery guarantees. In fact, UPD
does little but add a couple of port numbers in the UDP header so that
traffic to multiple applications on a single host can be sorted out.
UDP is very simple. You put some number of bytes (up to about 64KB) in
a UDP packet, toss it out the back door, and it may show up at the
destination application. Usually UDP applications limit themselves to
much smaller chucks of data (512 bytes is a good starting limit). If
your application can tolerate packet loss, or has some sort of built in
retry mechanism (for example DNS), then UDP is quite simple to use.
If, OTOH, you need reliability, and want a connection oriented view of
the world, TCP is usually a better choice. TCP again lives inside IP
packets, but the two applications set up a connection (virtual circuit)
between them by sending some packets containing TCP handshaking and
negotiation stuff back and forth (of course the TCP software in the OS
does all the real work here). Once established, the applications send
two streams of bytes over the link, one in each direction. The two
streams are largely unrelated to each other, and TCPs job is to package
chunks of those streams into TCP segments and send those off in IP
packets. TCP is also responsible for making sure the stream arrives in
the same condition it was sent, and it handles missing packets,
duplicated packets, reordered packet, etc., so the receiving
application just sees the same stream of bytes as the sending
application sent. The exact details of how the application data is
packaged is largely hidden from the application, and the TCP software
might send a stream of 10,000 bytes in 10,000 individual Ethernet
frames containing one byte of application data each, or might send
seven ~1500 byte frames. Typically you can trust TCP to send large
(and more efficient) segments whenever possible.
Now on top of that is the application. That's sometime called a layer,
and sometimes not (BTW, folks who count both the physical and
application layers usually call TCP/IP a five layer protocol). This is
where the end user stars to see stuff. For example, HTTP (the protocol
use for transferring pages between a web server and a web browser) plus
HTML (how those pages are coded) make up what we all know as the web.
FTP is another common application, as is Telnet.
Telnet is a good example for your application. In short Telnet is a
specification of how you stuff a serial terminal datastream inside a
TCP connection. There's some negotiation (the terminal tells the host
what type of terminal it is), and some minor encapsulation (escapes for
the telnet commands), but one the session is running, you're pretty
much looking at the conventional terminal datastream running over the
TCP connection, essentially identical to what you'd see over a serial
attached terminal. In fact if you're running a TTY style device, you
can skip (or just say "no") to most of the negotiations.
Assuming, for example, that your device supports attaching a VT-100
terminal (obviously emulated these days) via a serial port for whatever
control purposes you have, making your device a Telnet server might be
just the ticket. Then anyone with a Telnet client (that supports
VT-100), will be able to access your device a do whatever it is they'd
do with your device.
On the flip side, if your device is generating some data which you're
sending over a serial link to some PC that's processing it in some way,
then it might make more sense to just set up a TCP connection, and then
just send the bytes as-is. That's not all that much different than
what a Telnet connection would do, but you'd get to avoid some of the
negotiation that's required in Telnet.
But the critical point is that that you can treat a TCP connection like
a stream of bytes, very, very similar to what you'd see between two
devices connected via their serial ports. What happens underneath is
mostly invisible to your application.
On the other hand, TCP is a good sized chuck of code, and perhaps
stuffing your data into UDP datagrams would work. It's especially
attractive if you have messages of at least several bytes (one byte at
a time is possible, but somewhat painful), and you can tolerate losing
messages on occasion. Then you'd just set up the target PC to listen
for incoming UDP datagrams on some predefined port, and as you got
them, you'd process the data (messages) in them.
If you want to get more modern about it, you might want to set up your
device as a web server. As an aside, if you're doing this over the
Internet, most people's firewall policies make it easiest to use HTTP
(web service), and TCP connections are easier than UDP.
OK, so now how do you get started? First, get either the TCP/IP
"Illustrated" series (first two volumes) by Stevens, or the
"Internetworking with TCP/IP" (three volumes) by Comer and Stevens. No
need to get both. That'll teach you everything you need to know to
both write basic applications using TCP/IP, and enough about the
internals of a TCP/IP implementation, so that you'd probably be able to
dig into the RFCs and actually implement a stack if you were so moved.
Second, get "TCP/IP Lean" by Bentham. He's got an entire TCP/IP stack,
including a web server, running on a PIC!
Third, get a network monitor type (sometimes called sniffers**) package
and *watch* some of this traffic. Fire up your web browser, your FTP
client, Telnet to a host. There are several shareware and demo
versions available for Windows (if you've got Windows Server, MS's
"Network Monitor" comes with it), and several network monitors are
included in your favorite Linux distribution.
I cannot emphasize this enough - get a network monitor. Not only is it
a wonderful learning tool, it's an invaluable debugging tool for when
you're trying to figure out what your device is actually doing when
your PC application isn't seeing anything. All of these will do a nice
job decoding the Ethernet and TCP/IP headers, so it's pretty easy to
follow what's going on.
Finally, look at some of the packaged solutions. If you already have a
serial port, there are several serial-to-TCP/IP converters on the
market. You establish a TCP connection from the PC to that (some are
set up as Telnet servers), and you're all set. The down-side is that
it's an extra part. Most of the embedded CPU vendors also have
versions of their CPUs with embedded Ethernet controllers, and have
available TCP/IP stacks so you don't have to write your own.
To give you an idea of the scope, you can probably do a functional UDP
based device, from bare metal (eg. the Ethernet controller chip) on up,
in about 3000 lines of C code. A full TCP/IP stack is probably
10-20KLOCS.
-Robert
*OK, so no one's seen StarNet in a couple of decades...
**"Sniffer" is actually a trademark of a high end network monitor
that's been around for a very long time.