Parser, again!

On 12/15/2013 7:58 AM, Tauno Voipio wrote:
> On 15.12.13 02:10, Vladimir Vassilevsky wrote:
>> On 12/14/2013 3:08 PM, jmariano wrote:
>>>
>>> The box is under the command of a PC, using RS232 or USB, in a
>>> master-slave model, the PC being the master. I want to use a message
>>> based command language, similar to SCPI, but not so complicated (no
>>> tree structure). I was thinking in something like START, STOP SETADC
>>> 1000, REAADC 1, etc.
>>
>> MODBUS protocol?
>>
> Before he start on it - DO NOT!
>
> Modbus has been designed with far too many networking blunders.

What do you think is so wrong with MODBUS ?

Vladimir Vassilevsky
DSP and Mixed Signal Designs
www.abvolt.com

On Sun, 15 Dec 2013 15:58:55 +0200, Tauno Voipio
<tauno.voipio@notused.fi.invalid> wrote:

>On 15.12.13 02:10, Vladimir Vassilevsky wrote:
>> On 12/14/2013 3:08 PM, jmariano wrote:
>>>
>>> The box is under the command of a PC, using RS232 or USB, in a
>>> master-slave model, the PC being the master. I want to use a message
>>> based command language, similar to SCPI, but not so complicated (no
>>> tree structure). I was thinking in something like START, STOP SETADC
>>> 1000, REAADC 1, etc.
>>
>> MODBUS protocol?
>>
>>
>> Vladimir Vassilevsky
>> DSP and Mixed Signal Designs
>> www.abvolt.com
>
>Before he start on it - DO NOT!
>
>Modbus has been designed with far too many networking blunders.

Modbus ASCII is OK (start character, end characters) as well as
Modbus/TCP is OK (frame size in header) but Modbus RTU is not, since
it depends on very critical timing (which is a no no for any PC based
systems). 

While you might be able to work around the timing issues on RTU master
(but not as a terminal emulator), things gets quite ugly on the slave
side, especially in multidrop circuits.

"Tom Gardner" <spamjunk@blueyonder.co.uk> wrote in message 
news:o6eru.53596$kK1.42403@fx12.am4...
>>
>> I have to define the syntax and program the parser (in C) on the uc side. 
>> Since I don't have very strict specification on the syntax, I can define 
>> it in such a way that makes it more easy to analyse, or more robust or 
>> etc.
>>
>
> [snip] embed a Forth interpreter
> from the outset.
>

I second this, a fairly simple forth interpreter can create a very powerful 
command language.  I've used one inspired by jonesforth on avr chips.

jonesforth: 
http://git.annexia.org/?p=jonesforth.git;a=blob;f=jonesforth.S;h=45e6e854a5d2a4c3f26af264dfce56379d401425;hb=66c56998125f3ac265a3a1df9821fd52cfeee8cc 

On 15.12.13 17:34, upsidedown@downunder.com wrote:
> On Sun, 15 Dec 2013 15:58:55 +0200, Tauno Voipio
> <tauno.voipio@notused.fi.invalid> wrote:
>
>> On 15.12.13 02:10, Vladimir Vassilevsky wrote:
>>> On 12/14/2013 3:08 PM, jmariano wrote:
>>>>
>>>> The box is under the command of a PC, using RS232 or USB, in a
>>>> master-slave model, the PC being the master. I want to use a message
>>>> based command language, similar to SCPI, but not so complicated (no
>>>> tree structure). I was thinking in something like START, STOP SETADC
>>>> 1000, REAADC 1, etc.
>>>
>>> MODBUS protocol?
>>>
>>>
>>> Vladimir Vassilevsky
>>> DSP and Mixed Signal Designs
>>> www.abvolt.com
>>
>> Before he start on it - DO NOT!
>>
>> Modbus has been designed with far too many networking blunders.
>
> Modbus ASCII is OK (start character, end characters) as well as
> Modbus/TCP is OK (frame size in header) but Modbus RTU is not, since
> it depends on very critical timing (which is a no no for any PC based
> systems).
>
> While you might be able to work around the timing issues on RTU master
> (but not as a terminal emulator), things gets quite ugly on the slave
> side, especially in multidrop circuits.

Agreed. The problem is in finding the framing in the binary
Modbus RTU format. The message boundaries should be findable
without parsing the whole message. This breaks the protocol
layering badly and complicates the data link layer code.
The addition of timing constraints to framing is poison
to sensible line drivers, e.g. a FIFO -buffered interface
cannot be used, as it destroys the inter-character timing
information. The same applies to delays caused by the other
tasks of the operating system. I have not yet met a Modbus
handler in a PC which did obey the timing constraints.

Modbus/TCP is wrong because it uses TCP. The Modbus messages are,
by definition, datagram messages, and the only correct transport
in the TCP/IP suite is UDP. Using TCP ports the frame boundary
problem to the receiving program. Contrary to popular belief,
TCP does not preserve record boundaries. The only transport
guarantee is that all octets sent will arrive in the same order
as sent, but they may be packed to entirely different set of
TCP segments.

-- 

-T.

On Sun, 15 Dec 2013 08:53:39 +0000, Tom Gardner
<spamjunk@blueyonder.co.uk> wrote:

>If that is a possibility, it is probably much cleaner
>simpler, faster (speed and soon) to embed a Forth interpreter
>from the outset.

There are plenty of Forth interpreters available in C, including
a commercial one from MPE.

Stephen

-- 
Stephen Pelc, stephenXXX@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeforth.com - free VFX Forth downloads

upsidedown@downunder.com wrote:
> On Sun, 15 Dec 2013 15:58:55 +0200, Tauno Voipio
> <tauno.voipio@notused.fi.invalid> wrote:
>
>> On 15.12.13 02:10, Vladimir Vassilevsky wrote:
>>> On 12/14/2013 3:08 PM, jmariano wrote:
>>>>
>>>> The box is under the command of a PC, using RS232 or USB, in a
>>>> master-slave model, the PC being the master. I want to use a message
>>>> based command language, similar to SCPI, but not so complicated (no
>>>> tree structure). I was thinking in something like START, STOP SETADC
>>>> 1000, REAADC 1, etc.
>>>
>>> MODBUS protocol?
>>>
>>>
>>> Vladimir Vassilevsky
>>> DSP and Mixed Signal Designs
>>> www.abvolt.com
>>
>> Before he start on it - DO NOT!
>>
>> Modbus has been designed with far too many networking blunders.
>
> Modbus ASCII is OK (start character, end characters) as well as
> Modbus/TCP is OK (frame size in header) but Modbus RTU is not, since
> it depends on very critical timing (which is a no no for any PC based
> systems).
>
> While you might be able to work around the timing issues on RTU master
> (but not as a terminal emulator), things gets quite ugly on the slave
> side, especially in multidrop circuits.
>

I don't believe the OP's topology is multidrop. It's possible to have 
RS232 multidrop, but it's not advisable - if you must have multidrop,
use 485 or 422. There are lots of converters available.

I still think a simple ASCII protocol is easier than any of those.
MODBUS is for when you're interfacing devices made by two seperate
teams or vendors and there would be gain from having a standard;
if you own both ends, a simple ASCII protocol is easier.

The OP also mentioned SCPI; that's less like MODBUS.

-- 
Les Cargill

Tom Gardner wrote:
> On 14/12/13 21:08, jmariano wrote:
>> Dear All,
<snip>
>
> If that is a possibility, it is probably much cleaner
> simpler, faster (speed and soon) to embed a Forth interpreter
> from the outset.
>

Great idea, although it depends on how much trouble that is
to actually do in cases.

> Yes, I know the XP/agile fraternity will frown on that.
> Tough; some of that brigade doesn't know their limits!
>

I'd think Agile would *support* that, since it should be
easier to test.

<snip>

-- 
Les Cargill

On 15/12/13 18:41, Les Cargill wrote:
> Tom Gardner wrote:
>> On 14/12/13 21:08, jmariano wrote:
>>> Dear All,
> <snip>
>>
>> If that is a possibility, it is probably much cleaner
>> simpler, faster (speed and soon) to embed a Forth interpreter
>> from the outset.
>>
>
> Great idea, although it depends on how much trouble that is
> to actually do in cases.
>
>> Yes, I know the XP/agile fraternity will frown on that.
>> Tough; some of that brigade doesn't know their limits!
>>
>
> I'd think Agile would *support* that, since it should be
> easier to test.

It tends to go against the holy commandments of "don't
do big up-front design" to "do the simplest thing", because
"you can always refactor it later".

Too many XPers/agilistas, IMNSHO, treat XP/agile as a
religion, i.e. something in which there are (12 IIRC)
Commandments To Be Obeyed in order that the magic
recipe works.

That's ridiculous, of course, since XP/agile is immensely
valuable when deployed intelligently in appropriate
circumstances.

I've even seen Them strip out all comments from a
well-documented library on the /principal/ that "comments
get out of sync with code" and "good code doesn't need
comments". Often valid, but not when the comments
describe the subtleties of /why/ the library is
implemented that way and /how/ to use it.

Hi Mariano,

On 12/14/2013 2:08 PM, jmariano wrote:

> I'm a part-time prototype developer at my university. My latest project is a
> box with a microcontroller that measures and actuates on stuff. The box is
> under the command of a PC, using RS232 or USB, in a master-slave model, the
> PC being the master.

I.e., the "commands" are going to be algorithmically generated.  Not
subject to the sorts of errors humans would make!

For example, you don't (??) have to support the possibiliy of a human
typing "STAG" (instead of "STAR"), followed by a backspace or delete
character (to "erase the erroneous preceding 'G') and then "RT" (for
"START")

> I want to use a message based command language, similar to SCPI, but not so
> complicated (no tree structure). I was thinking in something like START, STOP
> SETADC 1000, REAADC 1, etc.

Does the message stream need to be recognizable by mere mortals?
I.e., if USB connected, is anyone snooping the line to see that you
sent "START" and not "GO" or "XZ", for that matter?

What sort of integrity do you require in the communications?  I.e.,
are you likely to encounter transmission errors (noise) that you would
like to safeguard against?
    "LAUNCH THE ROCKET"
(NO!!!  I meant "LUNCH AT THE ROCKET CAFE!")

> I have to define the syntax and program the parser (in C) on the uc side.
> Since I don't have very strict specification on the syntax, I can define it
> in such a way that makes it more easy to analyse, or more robust or etc.

How large is the command set?  If you just have a handful of commands
(message formats), why not just pick a single letter for each command?
If possible, relate them to natural language terms IF YOU ARE OBSESSIVE
ABOUT THESE SORTS OF THINGS (i.e., so you *could* examine a message
"unaided" and suss out what it means).  Otherwise, just pick letters
that are far enough apart to not easily be "corrupted" (e.g., if you
have the baudrate, character size, etc. set incorrectly on the EIA232
version of your device AND/OR encounter transmission errors -- unduly
long serial cables, etc)

Then, the message can be something like:
    <start> <command> <args> <end>
and "parsing" can be purely ad hoc:
     switch (command) {
     case 'A':
         // parse <args> in the context that A requires
         // do whatever A means
     case 'B':
         // parse <args> in the context that B requires
         // do whatever B means
     default:
         panic();
     }

> So, my questions are:
>
> 1 - Regarding the language definition: Are there god examples of such language

God's got nothing to do with it...  :>

> that I can get inspiration from? And references? I'm sure someone as already
> thought about this in a formal way. I'm looking for practical advice like shall
> I use fixed length commands (6 character, for example),

Depends.  Do your commands all seem to need "more or less" 6 charcters
to encode their intent?  If one need 3 and another needs 8, then there
may still be advantages to packing everything into an 8 character frame.

If, for example, many messages have similar argument requirements, then 
you could factor out these requirement and have a first pass deliver
<command>, <arg1> and <arg2> (for example) for interpretation by each
"action routine"

> start-of-message character (#)? Why?

You either have an end of message, start of message or both.
Otherwise, you have no way of putting the contents of the message
in context.

What 3 digit numbers am I trying to send from A to B in the following
*series* of messages:
    12830395834953
Are you sure the first one is "128" and not actually "283" with the
"1" being the last digit of the preceding message (that you didn't see
because the cable wasn't yet attached, because the first two digits of
that message had parity errors, because...

Read a book that has no capitalization or punctuation and see how
much sense you can make out of it WITHOUT BACKING UP!

> And the arguments, separated by commas, spaces? etc.

Who's your audience?  If its just a machine, is there enough
information present without these extra delimiters to RELIABLY
understand the content of the message?   E.g., if you always
send numerics as fixed width fields (with leading and/or trailing
zeroes), then 00000123 can reliably be interpreted as:
00, 0, 001, 23 without the benefit of additional delimiters!

IME, when talking between machines, you want as rigid a
structure as possible so the recipient can reassure itself that
nothing has been "lost in transmission".  OTOH, when interacting with
people, you tend to want convenience -- don't force me to type
six digits for a single value if they're all going to be 0!

Tom Gardner wrote:
> On 15/12/13 18:41, Les Cargill wrote:
>> Tom Gardner wrote:
>>> On 14/12/13 21:08, jmariano wrote:
>>>> Dear All,
>> <snip>
>>>
>>> If that is a possibility, it is probably much cleaner
>>> simpler, faster (speed and soon) to embed a Forth interpreter
>>> from the outset.
>>>
>>
>> Great idea, although it depends on how much trouble that is
>> to actually do in cases.
>>
>>> Yes, I know the XP/agile fraternity will frown on that.
>>> Tough; some of that brigade doesn't know their limits!
>>>
>>
>> I'd think Agile would *support* that, since it should be
>> easier to test.
>
> It tends to go against the holy commandments of "don't
> do big up-front design" to "do the simplest thing", because
> "you can always refactor it later".
>

How is Forth anything *but* the simplest thing? My own
uncertainty is simply the actions necessary to install the
interpreter for a given environment - the various ones
linked to look pretty easy to use.

This being said, if you just have a simple line-oriented
thing, a table-driven "parser" is pretty easy. I posted
some actual partial-code in the thread...

Forth is *extremely* hackerish, and therefore ( IMO )
consonant with XP. Flexibility ot the goal, right?

> Too many XPers/agilistas, IMNSHO, treat XP/agile as a
> religion, i.e. something in which there are (12 IIRC)
> Commandments To Be Obeyed in order that the magic
> recipe works.
>

We can't help that; what my takeaway from Agile/XP is
test-first, in cases pairs and otherwise eschewing big waterfall.

All programming paradigms uncover the latent "Spanish Inquisitor"
in people.


> That's ridiculous, of course, since XP/agile is immensely
> valuable when deployed intelligently in appropriate
> circumstances.
>

Of course. I'd say I first used it in the late '80s,
although it wasn't called that. We got stuff done.

> I've even seen Them strip out all comments from a
> well-documented library on the /principal/ that "comments
> get out of sync with code" and "good code doesn't need
> comments". Often valid,

Not if you have good shop culture to support comments.
reviews help, too - if you can manage them well.

> but not when the comments
> describe the subtleties of /why/ the library is
> implemented that way and /how/ to use it.

-- 
Les Cargill