How to snif full-duplex UART protocol between two nodes

Many times I need to snif and log on a file the activity between two 
nodes that talk over a full-duplex UART (separate TX and RX lines).

Just an example, consider a host MCU that talks with a modem through AT 
commands. It's a half-duplex protocol because the MCU sends the AT 
command and the modem replies. However the modem is able to send URC 
messages that can be emitted at any time.

I think the better approach is to use two different UART/USB converters, 
one for each signal (of course, both signals goes to the RX inputs of 
the converters).

What about the sofware on the PC? It should open two serial ports and 
monitors both of them, joining together the messages on a single file. A 
great feature would be to have a timestamp for each message, where a 
message is "\r\n" delimited.

Is there something already available that I can use?

On 2/20/2023 3:16 AM, pozz wrote:
> Many times I need to snif and log on a file the activity between two nodes that 
> talk over a full-duplex UART (separate TX and RX lines).

To be clear, neither of the endpoints will be a PC (which could
host the monitoring software).

> What about the sofware on the PC? It should open two serial ports and monitors 
> both of them, joining together the messages on a single file. A great feature 
> would be to have a timestamp for each message, where a message is "\r\n" 
> delimited.
> 
> Is there something already available that I can use?

There are several PC-hosted "serial port monitors" that are available.
Some for no dollars.

I've used <https://www.232analyzer.com/>, in the past -- it has a "monitor"
mode that can provide timestamps.

If timing is super critical, you may need to explore hardware solutions,
instead.

The simple way of doing this is to spawn two copies of the tool, on
the same hosting PC (so their notion of time is similar).  Capture
each log to a separate file.  Then, glue the two files together
and sort(1) based on the timestamps.

This doesn't rule out ambiguities where the precision of the
timestamp makes two transactions appear concurrent when, in
fact, they likely occurred in a particular order (but too
close in time for the tool to differentiate with its timestamp).

The big downside is tat it isn't a real-time solution; the two files
have to be processed afterwards in a "batch" operation.

[I doubt *this* tool would behave well if both instances were
directed to the same log file!  Who knows how the data is cached
within the app, how often the fd is flushed to disk, etc.]

But, it's a cheap starting point to identify other issues that
you may find significant!

pozz <pozzugno@gmail.com> wrote:
> Many times I need to snif and log on a file the activity between two 
> nodes that talk over a full-duplex UART (separate TX and RX lines).
> 
> Just an example, consider a host MCU that talks with a modem through AT 
> commands. It's a half-duplex protocol because the MCU sends the AT 
> command and the modem replies. However the modem is able to send URC 
> messages that can be emitted at any time.
> 
> I think the better approach is to use two different UART/USB converters, 
> one for each signal (of course, both signals goes to the RX inputs of 
> the converters).
> 
> What about the sofware on the PC? It should open two serial ports and 
> monitors both of them, joining together the messages on a single file. A 
> great feature would be to have a timestamp for each message, where a 
> message is "\r\n" delimited.
> 
> Is there something already available that I can use?

Not quite what you asked for, but I've used:
https://sigrok.org/wiki/Main_Page
as logic analyser software, which will decode UART signals.  You can get
some very cheap modules that it works with, eg:
https://sigrok.org/wiki/Lcsoft_Mini_Board
(about $10 from the usual marketplaces, search for 'CY7C68013A',
'EZ-USB' or 'FX2LP')
if you aren't fussy about buffering.  Sigrok also supports a wide range of
other logic analysers - perhaps you already have something.

The nice thing about the CY7C68013A chip over regular logic analysers is
that it's capable of realtime streaming.  Assuming your PC's USB stack is up
to it and doesn't randomly pause (or have some other chatty USB device on
the bus stealing bandwidth) you can record for as long as your memory and
storage subsystem can keep up.  Given it's 480Mbps USB2.0, that is probably
quite a long time.

Being a logic analyser the inputs are by definition synchronised and
timestamped relative to each other.

Theo

On 2/20/23 5:16 AM, pozz wrote:
> Many times I need to snif and log on a file the activity between two 
> nodes that talk over a full-duplex UART (separate TX and RX lines).
> 
> Just an example, consider a host MCU that talks with a modem through AT 
> commands. It's a half-duplex protocol because the MCU sends the AT 
> command and the modem replies. However the modem is able to send URC 
> messages that can be emitted at any time.
> 
> I think the better approach is to use two different UART/USB converters, 
> one for each signal (of course, both signals goes to the RX inputs of 
> the converters).
> 
> What about the sofware on the PC? It should open two serial ports and 
> monitors both of them, joining together the messages on a single file. A 
> great feature would be to have a timestamp for each message, where a 
> message is "\r\n" delimited.
> 
> Is there something already available that I can use?

As Theo says, there are a number of different "Protocal Analyszers" that 
you just insert into the communications link and it records all of the 
data sent between the units, with time stamps, for display on a PC.

They are very handy for debuging such links.

The main thing you need to make sure of is that the module understands 
the voltage format of your link, be it "TTL", RS-232, RS-422, or RS-485, 
or whatever you have. And that it can capture at the baud rate of your link.

If the link might be variable baud, you need a unit that captures like a 
logic analyizer and then lets you process later, which is a bit more 
complicated. You also want something like that if you need to check if 
there are bit-level timing issues with the link.

pozz <pozzugno@gmail.com> writes:
> Is there something already available that I can use?

There is a traditional hack where you make or buy a Y cable that lets
you tap the serial i/o and listen to both directions from a PC.  Then
just log the data going in and out.  When I did it I used a simple
Python script and it was able to keep up with the device we were dealing
with (might have been 1200 bps, I don't remember).  It's possible
something like that already exists for Wireshark.

On 2023-02-20, pozz <pozzugno@gmail.com> wrote:

> Many times I need to snif and log on a file the activity between two 
> nodes that talk over a full-duplex UART (separate TX and RX lines).

If you have to do that "many times", then buy a serial protocol
analyzer widget that has hardware that has two capture ports and
timestamps every bit/byte.

I have and older version of this and it works brilliantly:

    https://www.iftools.com/analyzer/msb-rs232/index.en.php

There are similar but cheaper products from other vendors, but if it's
something you need to do often, then it's worth a few hundred
dollars/euros to get a decent one.

> Just an example, consider a host MCU that talks with a modem through AT 
> commands. It's a half-duplex protocol because the MCU sends the AT 
> command and the modem replies. However the modem is able to send URC 
> messages that can be emitted at any time.
>
> I think the better approach is to use two different UART/USB converters, 
> one for each signal (of course, both signals goes to the RX inputs of 
> the converters).

The latency/buffer in the USB adpaters is awful. Spend some money and
by a real serial analyzer.

> What about the sofware on the PC? It should open two serial ports and 
> monitors both of them, joining together the messages on a single file. A 
> great feature would be to have a timestamp for each message, where a 
> message is "\r\n" delimited.

On 20/02/23 21:16, pozz wrote:
> Many times I need to snif and log on a file the activity between two 
> nodes that talk over a full-duplex UART (separate TX and RX lines).
> 
> Just an example, consider a host MCU that talks with a modem through AT 
> commands. It's a half-duplex protocol because the MCU sends the AT 
> command and the modem replies. However the modem is able to send URC 
> messages that can be emitted at any time.
> 
> I think the better approach is to use two different UART/USB converters, 
> one for each signal (of course, both signals goes to the RX inputs of 
> the converters).
> 
> What about the sofware on the PC? It should open two serial ports and 
> monitors both of them, joining together the messages on a single file. A 
> great feature would be to have a timestamp for each message, where a 
> message is "\r\n" delimited.
> 
> Is there something already available that I can use?

You could fairly easily modify my "connect.c" to do this, works in Linux 
or OS/X. It does two-way communication between a serial port and the 
terminal console you run it from, similar to `cu`, minicom` and similar 
programs - but this is very simple, you don't have to navigate a wealth 
of features to make changes.

Change it to configure two serial ports and copy data between them, 
logging data to a file in the same way the Unix program `script` does. 
Then plug in two USB/RS232 dongles and connect your two devices.

<https://www.dropbox.com/s/q0tqsszqx0q5hf4/connect.c>

Clifford Heath.

On 2023-02-20, pozz <pozzugno@gmail.com> wrote:
> Many times I need to snif and log on a file the activity between two 
> nodes that talk over a full-duplex UART (separate TX and RX lines).
>
> Is there something already available that I can use?

Try searching for "RS232 analyser", "RS232 datascope" and similar.
A quick look found a number of possible candidates.  I won't make
any recommendation myself as last time I needed something like this
I went back to an old DOS tool.

-- 
Andrew Smallshaw
andrews@sdf.org

On 2023-02-20, pozz <pozzugno@gmail.com> wrote:
> Many times I need to snif and log on a file the activity between two 
> nodes that talk over a full-duplex UART (separate TX and RX lines).
>
> Is there something already available that I can use?

Try searching for "RS232 analyser", "RS232 datascope" and similar.
A quick look found a number of possible candidates.  I won't make
any recommendation myself as last time I needed something like this
I went back to an old DOS tool.

-- 
Andrew Smallshaw
andrews@sdf.org

Il 20/02/2023 22:52, Paul Rubin ha scritto:
> pozz <pozzugno@gmail.com> writes:
>> Is there something already available that I can use?
> 
> There is a traditional hack where you make or buy a Y cable that lets
> you tap the serial i/o and listen to both directions from a PC.  Then
> just log the data going in and out.  When I did it I used a simple
> Python script and it was able to keep up with the device we were dealing
> with (might have been 1200 bps, I don't remember).  It's possible
> something like that already exists for Wireshark.

Do you mean "joining electrically" (AND for UART TTL levels) the two 
signals and connect the result to the RX signal of a single serial port 
on the PC?

It couldn't work in my case, because the protocol could be full-duplex, 
so both nodes could transmit at the same time. The AND result would be 
corrupted.