Anyone know the quickest or easiest way of recording a serial data stream and timestamping each byte with a resolution of at least 1ms (preferably finer)? I can only find cheap or free programs with less resolution, or very expensive solutions, often involving special hardware. I only need to monitor one port, and I don't need to send any characters (actually I'm snooping on an RS485 bus). I'm prepared to use an old PC with a "real" serial port, run MSDOS, Linux or whatever other OS will do the job, use some sort of microprocessor to capture the data (preferably one I'm familiar with, such as 8051 family or PIC) or anything else so long as it's quick and cheap. I just need to get the job done, preferably tomorrow! If possible I'd like it to run at a non-standard baud rate (62,500 baud). Thanks - Rowan --------------------------------------- Posted through http://www.EmbeddedRelated.com
Monitor RS232 comms with millisecond resolution
Started by ●June 20, 2010
Reply by ●June 20, 20102010-06-20
On 2010-06-20, rowan.bradley <rowan@n_o_s_p_a_m.sylvester-bradley.org> wrote:> Anyone know the quickest or easiest way of recording a serial data > stream and timestamping each byte with a resolution of at least 1ms > (preferably finer)? I can only find cheap or free programs with less > resolution, or very expensive solutions, often involving special > hardware. > > I only need to monitor one port, and I don't need to send any > characters (actually I'm snooping on an RS485 bus). > > I'm prepared to use an old PC with a "real" serial port, run MSDOS, > Linux or whatever other OS will do the job, use some sort of > microprocessor to capture the data (preferably one I'm familiar with, > such as 8051 family or PIC) or anything else so long as it's quick > and cheap. I just need to get the job done, preferably tomorrow!Greeleaf used to sell a serial data analyzer program for DOS. It even came with a Y-cable so that you could monitor two data lines. In my experience, it's either that or something hardware-based. If you find a good solution let us know.> If possible I'd like it to run at a non-standard baud rate (62,500 baud).-- Grant
Reply by ●June 20, 20102010-06-20
On 06/20/2010 04:34 PM, rowan.bradley wrote:> Anyone know the quickest or easiest way of recording a serial data stream > and timestamping each byte with a resolution of at least 1ms (preferably > finer)? I can only find cheap or free programs with less resolution, or > very expensive solutions, often involving special hardware.The problem is in the limitations of Windows (and most flavors of Linux as well). The underlying assumption when talking to a serial port is that (a) the hardware will queue up some of the data for you, and (b) the user just wants to download porn from the internet, and doesn't particularly care about the exact timing of the bits. Even users that are nutcases and want to download stuff from the internet that _isn't_ port _still_ just want to check out the latest political flame on their technical newsgroups, and don't much care exactly what millisecond the period at the end of the sentence arrived. So the driver is optimized to catch every bit of those jpegs that your wife would rather you didn't download, but has no effort whatever put into actually timing the arrival of the latest byte. In fact, the best that you can do without extraordinary effort is for whatever user-space application is running to log the time that it processes the bytes -- and that's not very precise at all. Hence the special hardware...> I only need to monitor one port, and I don't need to send any characters > (actually I'm snooping on an RS485 bus). > > I'm prepared to use an old PC with a "real" serial port, run MSDOS, Linux > or whatever other OS will do the job,MSDOS and a custom program would give you your best bet on a PC, but it'd have to be dedicated hardware, not running in a window or a virtual machine or anything.> use some sort of microprocessor to > capture the data (preferably one I'm familiar with, such as 8051 family or > PIC) or anything else so long as it's quick and cheap.With something like a PIC or an 8051 it's much easier -- don't have any other software running, have a _really fast_ loop that polls the serial port, and log the time (I'm assuming it's a processor with at timer) that the data arrives. Note that this is basically what you'd have to do on the PC, as well.> I just need to get > the job done, preferably tomorrow!The technically best solution would probably be an FPGA that logs the arrival time of each byte, and stores a queue of bytes and timestamps -- good gawd, what a pain that would be! But in the mean time, you either need to luck out big time, or start flogging PIC software.> If possible I'd like it to run at a non-standard baud rate (62,500 baud).Depending on the UART, a PIC may be a help here, too. -- Tim Wescott Control system and signal processing consulting www.wescottdesign.com
Reply by ●June 20, 20102010-06-20
On 06/20/2010 06:34 PM, rowan.bradley wrote:> Anyone know the quickest or easiest way of recording a serial data stream > and timestamping each byte with a resolution of at least 1ms (preferably > finer)? I can only find cheap or free programs with less resolution, or > very expensive solutions, often involving special hardware. > > I only need to monitor one port, and I don't need to send any characters > (actually I'm snooping on an RS485 bus). > > I'm prepared to use an old PC with a "real" serial port, run MSDOS, Linux > or whatever other OS will do the job, use some sort of microprocessor to > capture the data (preferably one I'm familiar with, such as 8051 family or > PIC) or anything else so long as it's quick and cheap. I just need to get > the job done, preferably tomorrow! > > If possible I'd like it to run at a non-standard baud rate (62,500 baud). > > Thanks - Rowan > > --------------------------------------- > Posted through http://www.EmbeddedRelated.comIf I had to do it I would grab a serial to USB board such as DLP Designs sells (has an ftdi chip on it). Then get a dev board with an atmega128 (or any micro with 2 serial ports). Hook up the rs485 to a transceiver such as a 75176, then to one of the micro's serial ports. The other port you connect to your usb board. Write a small program that receives the bytes, adds the timestamp, then writes it out over the second port (which you can run at a few hundred kilobaud to keep up). On the pc side, my favorite way to do it is to write a python program, using pyserial to read the port. This can easily keep up with 1 megabaud on a reasonable PC. This system would be able to generate the timestamp within a few microseconds of the byte's arrival, limited by your micro's (and C compiler's, if you use C) interrupt latency. Paul Probert
Reply by ●June 21, 20102010-06-21
"rowan.bradley" <rowan@n_o_s_p_a_m.sylvester-bradley.org> wrote in message news:EpmdnWB348YFOIPRnZ2dnUVZ_qGdnZ2d@giganews.com...> Anyone know the quickest or easiest way of recording a serial data stream > and timestamping each byte with a resolution of at least 1ms (preferably > finer)?How much data do you need to store at a time?>, use some sort of microprocessor to > capture the data (preferably one I'm familiar with, such as 8051 family or > PIC)If I was doing it I would use a low-cost (< $100) off-the-shelf ARM development board - but that is because I could program it much more quickly and easily than an 8051 or a PIC as I already have most of the code written.> or anything else so long as it's quick and cheap.define 'cheap'> I just need to get > the job done, preferably tomorrow! >Doesn't sound too difficult - unless I'm missing something?> If possible I'd like it to run at a non-standard baud rate (62,500 baud). >I'm not certain that is possible with what I have in mind but am optimistic. -- Chris Burrows CFB Software Astrobe: ARM Oberon-07 Development System http://www.astrobe.com
Reply by ●June 21, 20102010-06-21
On Sun, 20 Jun 2010 18:34:48 -0500, "rowan.bradley" <rowan@n_o_s_p_a_m.sylvester-bradley.org> wrote:> >I'm prepared to use an old PC with a "real" serial port, run MSDOS, Linux >or whatever other OS will do the job, use some sort of microprocessor to >capture the data (preferably one I'm familiar with, such as 8051 family or >PIC) or anything else so long as it's quick and cheap. I just need to get >the job done, preferably tomorrow!You can get quite accurate time stamps by disabling the UART FIFO and assuming you are on a Pentium or newer processor with the RDTSC instruction, read the Time Stamp Counter in the Rx interrupt routine, first put the received byte in the software Rx queue, followed by 8 bytes from the TSC. On the user mode side, read 9 bytes at a time, the first being the actual byte received, the rest the time stamp. To convert the TSC to milliseconds, you have to know (or measure) the CPU clock frequency. With the UART FIFO disabled, the maximum baud rate depends on the interrupt latency of the system, e.g. for how long the interrupts are disabled. Also some peripherals (such as graphic cards, ethernet or IDE controllers) may effectively block interrupts for a long time, however, the situation has improved a lot since the 1990's.
Reply by ●June 21, 20102010-06-21
Hi Rowan, rowan.bradley wrote:> Anyone know the quickest or easiest way of recording a serial data stream > and timestamping each byte with a resolution of at least 1ms (preferably > finer)? I can only find cheap or free programs with less resolution, or > very expensive solutions, often involving special hardware. > > I only need to monitor one port, and I don't need to send any characters > (actually I'm snooping on an RS485 bus).Are you actually interested in *every* byte or just "certain" bytes (e.g., are you looking at, perhaps, the timing of *messages*?)> I'm prepared to use an old PC with a "real" serial port, run MSDOS, Linux > or whatever other OS will do the job, use some sort of microprocessor to > capture the data (preferably one I'm familiar with, such as 8051 family or > PIC) or anything else so long as it's quick and cheap. I just need to get > the job done, preferably tomorrow!If you can find something that you can easily write / deploy code on, this is a one page program.> If possible I'd like it to run at a non-standard baud rate (62,500 baud).IIRC, a "real PC" will give you problems hitting 62,500. I think the baud rate clock is fed from too low of a frequency to give you enough precision to hit "any" baudrate you want (esp at higher rates, obviously)
Reply by ●June 21, 20102010-06-21
On Sun, 20 Jun 2010 18:34:48 -0500, "rowan.bradley" <rowan@n_o_s_p_a_m.sylvester-bradley.org> wrote:>Anyone know the quickest or easiest way of recording a serial data stream >and timestamping each byte with a resolution of at least 1ms (preferably >finer)? I can only find cheap or free programs with less resolution, or >very expensive solutions, often involving special hardware. > >I only need to monitor one port, and I don't need to send any characters >(actually I'm snooping on an RS485 bus). > >I'm prepared to use an old PC with a "real" serial port, run MSDOS, Linux >or whatever other OS will do the job, use some sort of microprocessor to >capture the data (preferably one I'm familiar with, such as 8051 family or >PIC) or anything else so long as it's quick and cheap. I just need to get >the job done, preferably tomorrow! > >If possible I'd like it to run at a non-standard baud rate (62,500 baud).I'm not sure an IBM PC uart can run at that rate. I think 57600 is doable, though. Something that bothers me about your request is this 1ms resolution. At 62500 bps, one stop and one start, this is 6250 cps. That works out to 160us per character. Yet you don't seem to __require__ better than 1ms resolution? (I do read you saying that you'd like better.) Okay. None of that really helps. The IBM PC is everywhere, so I understand that desire. But I don't think you can nail the rate at 62500. If you can use 57600, I think that works on the PC. There is a 16-bit timer on the PC. It doesn't interrupt fast enough for you, but you can read it when the characters arrive. So if you use DOS, write in assembly code, can accept the 57600 bps rate, then it's doable. I keep new boxes of DOS 5.0 laying about and 80386 and 80486 machines here with nice ISA buses, just for doing stuff like this. But you may not have them. And new PCs seem to be avoiding RS-232 ports more and more. USB would barely get you by (one of those USB to RS-232 things), as I believe the max 'pace' is 1ms there. (I don't know USB that well, but I disagree with Paul on this subject unless you dig _really_ deep into the USB VxD level of stuff.) In general, I don't know how closely you can track it through through the HID driver and all of the Windows layers. I'd probably avoid it, imagining the work there to be way too much to bite off for too little potential. Which gets to either someone's device you can buy -- you've already looked -- or else something you pony up. I'd wire something up from crap I have around here. Some don't have the level shifters, but you are only looking to receive, not send, so you can easily use a BJT and a few resistors there (cheap and easy.) Software is extra, of course. If you did have one of those USB-RS232 units and could wire up to it (I got one included in the $24 MSP430 Chronos kit I bought, for example), you might choose a micro with two UARTS and wire one of them to the USB-RS232 connector and send serial data in that direction that includes the received characters and their time stamps for logging onto the PC and wire the other one to monitor your 62500 bps line. Add time stamps and send to the USB side. Not too much software that way and you have a huge data logger, then. (You didn't mention how much data must be logged, which is yet another potential problem.) Jon
Reply by ●June 21, 20102010-06-21
In article <EpmdnWB348YFOIPRnZ2dnUVZ_qGdnZ2d@giganews.com>, rowan@n_o_s_p_a_m.sylvester-bradley.org says...> Anyone know the quickest or easiest way of recording a serial data stream > and timestamping each byte with a resolution of at least 1ms (preferably > finer)? I can only find cheap or free programs with less resolution, or > very expensive solutions, often involving special hardware.Everything depends on many factors some of which are 1/ Do you want relative timestamp from power up or first byte received? 2/ Do you want linked timestamp (actual time) to some time stamp reference (even one system's time clock)? 3/ With later mentioned 62500bps, your accuracy could end up +/-1ms 4/ Are you timestamping each byte, or a block of data? 5/ Are you timestamping the - line start bit falling edge, middle or effective end - line Stop bit effective rising edge, middle or effective end - Byte coming out of a UART (with hardware latency from start bit) effectively Data Ready flag, which becomes skewed even with two or more bytes from double buffering or FIFOing and software/interupt latency. - Start/middle/end of any message block 6/ How much data are you expecting to capture per second/min 7/ How large a time stamp range do you need, i.e. do you need one that runs for minutes, hours, days or longer. This affects the overhead of any timestamp, the bigger it is the more to process. Also the more bytes that have to be timestamped the larger the dataset.> I only need to monitor one port, and I don't need to send any characters > (actually I'm snooping on an RS485 bus). > > I'm prepared to use an old PC with a "real" serial port, run MSDOS, Linux > or whatever other OS will do the job, use some sort of microprocessor to > capture the data (preferably one I'm familiar with, such as 8051 family or > PIC) or anything else so long as it's quick and cheap. I just need to get > the job done, preferably tomorrow! > > If possible I'd like it to run at a non-standard baud rate (62,500 baud).Using and external device will make getting that baud rate easy, how big your data set is, the time stamp data has to be affects how much storage buffering of data and in turn potentially latency for timing as recording gets bigger. If on a PC the act of updating screen or saving some of the data to disk could impact on recording timing on a heavily loaded link. If you must timestamp every byte, I would consider a seven bit relative time stamp between bytes, with MSB set as 1. Full time stamp always to have MSB set to 0, thus huge amount of data set can saved if you need a 4 byte or larger timestamp for the whole dataset. If time delta greater than 127 steps, send full time stamp. Even at 62500 bps, you are potentially storing 6250 bytes/second, with a four byte timestamp on ech byte you are then storing 5 x as much data EVERY second. This of course increases to 9x as much for 8 byte timestamp. So your data storage, buffering and processing requirements increase drastically for an external unit. Mean while in a PC storing data to disk could cause gaps in time stamp accuracy, depending on how good the disk drivers and hardware actually are.> Thanks - Rowan > > --------------------------------------- > Posted through http://www.EmbeddedRelated.com >-- Paul Carpenter | paul@pcserviceselectronics.co.uk <http://www.pcserviceselectronics.co.uk/> PC Services <http://www.pcserviceselectronics.co.uk/fonts/> Timing Diagram Font <http://www.gnuh8.org.uk/> GNU H8 - compiler & Renesas H8/H8S/H8 Tiny <http://www.badweb.org.uk/> For those web sites you hate
Reply by ●June 21, 20102010-06-21
"rowan.bradley" wrote:>Anyone know the quickest or easiest way of recording a serial data stream >and timestamping each byte with a resolution of at least 1ms (preferably >finer)? I can only find cheap or free programs with less resolution, or >very expensive solutions, often involving special hardware.So you already checked all solutions listed on http://wiki.oliverbetz.de/owiki.php/TerminalPrograms ? Another possibility is http://www.tech-tools.com/dv_main.htm - it's able to record and decode serial data and of course the resolution is several orders of magnitude better than you require. Not cheap but a good logic analyzer is often useful. Oliver -- Oliver Betz, Munich despammed.com is broken, use Reply-To: