Parser, again!

If I were you, I would stop searching at SCPI and continue with that 
choice.  A simple implementation of SCPI is dirt simple to implement in 
a parser (that's the right word).  You don't need to support the 
instrument models, multiple command, or hierarchy functionality.  It's 
got human readable keyword/data statements that makes debugging easy. 
Search no more.

I have one complaint about SCPI.  There is no defined inline 
success/failure mechanism for commands, probably because if its long 
history of being married to GPIB, which has error handling defined via 
some other means.  In all of my SCPI implementations, I defined a 
success/failure status response that has served me (and my customers) 
very well.  If the command succeeded, "OK\r\n" was returned.  If it 
failed, "ERR<n>\r\n", was returned (<n> is an error code).  That 
response mode was switchable for compatibility with official SCPI.

You'll find your SCPI parser code will be reusable for future projects 
so design it with that in mind.

JJS

On Monday, 16 December 2013 21:35:45 UTC, Paul Rubin  wrote:

> 
> > - For my application, using Forth or Lisp is a little like using a
> 
> > sledgehammer to crack a nut. I don't think will need that. A simple
> 
> > ascii protocol will do the job. Thanks for the suggestion anyway. ...
> 
> 
> 
> One thing that should have been asked: what is the target processor?
> 
> Why are you even writing the code in C?  Yes, using something like Lisp
> 
> (or these days Python) might be "overkill", but just in the sense that
> 
> it's overkill to drive a 2000 pound car to pick up a quart of milk from
> 
> a store 2 km away, especially if the "car" burns no fuel.  If it's a
> 
> small MCU then you kind of have to use C, but if it's something like a
> 
> Raspberry Pi then you can use almost anything.  Simplest practice with
> 
> such a system might be to use XML or Json, depending.  The parsing and
> 
> encoding libraries are already there.

I'm using a Arduino Due board and the main reason for using C is that... I now C and Lisp (or Forth, or Python, or ...) I don't. I only know C, Fortran, and a couple more and I'm to old to switch dialects unless there is a very good reason for it. In this case, I think that it will take me more time to learn a new dialect than to implement the damn thing.  But I might be wrong.....

On Monday, 16 December 2013 22:35:16 UTC, dp  wrote:
> On Monday, December 16, 2013 10:55:00 PM UTC+2, jmariano wrote:
> 
> > ...
> 
> > To make things a little more clear, my gizmo behaves more like an
> 
> > programmable instrument, an oscilloscope or a multimeter, than a PLC.
> 
> > It measures ph, conductivity and temp and was to activate relays (pumps).
> 
> > For those who like chemistry, this is a titration experiment and my
> 
> > device is an automatic titrator.
> 
> 
> 
> A few years ago I designed for a guy a PH & conductivity meter,
> 
> ensuring the conductivity meter did not interfere with the PH
> 
> electrode was why this came my way. But I only did the analog
> 
> front end which delivered the two analog values, that's where help
> 
> was needed.
> 
> 

Sorry, I was not clear: ph and temperature (for ph compensation) or conductivity, not both at the same time. I also have to design the ph and conductivity front ends, but your case was much more tricky. 

> 
> > The only important things for the machine is the existence of SOM or
> 
> > EOM or both. I think I'll go for the :xxxx>CRLF> format.
> 
> 
> 
> Since your application will likely involve long times from host command
> 
> to device knowing it has succeeded (pumps staying active etc.) you might
> 
> want to do something similar to the command-ack-outcome_notification
> 
> sequence in my protocol which I posted earlier. 
> 
> [ the text is at http://tgi-sci.com/misc/sdvctl.txt ]
> 
> Makes life a lot easier; host issues a command, gets immediately
> 
> an ACK reply which also carries information about how long (max) will
> 
> it take for the command to finish; upon finishing, the device notifies
> 
> the host how it worked out. This can take seconds if not minutes
> 
> so in the meantime the host can keep on issuing other commands to
> 
> the device, like read status, various measured values, make some
> 
> light blink etc.
> 
> 

I have read your specification carefully and I will be using some of your ideas (hope you don't mind) but I'm not so sure of the ACK reply. I have to think a little about it. I think that with SCPI instruments, the ACK is a flag on a status register that the PC reads when we feels necessary. How this two approaches compare?  

> 
> Dimiter
> 
> 
> 
> ------------------------------------------------------
> 
> Dimiter Popoff, TGI             http://www.tgi-sci.com
> 
> ------------------------------------------------------
> 
> http://www.flickr.com/photos/didi_tgi/sets/72157600228621276/

On Monday, 16 December 2013 22:41:09 UTC, John Speth  wrote:
> If I were you, I would stop searching at SCPI and continue with that 
> 
> choice.  A simple implementation of SCPI is dirt simple to implement in 
> 
> a parser (that's the right word).  You don't need to support the 
> 
> instrument models, multiple command, or hierarchy functionality.  It's 
> 
> got human readable keyword/data statements that makes debugging easy. 
> 
> Search no more.
> 
> 
> 
> I have one complaint about SCPI.  There is no defined inline 
> 
> success/failure mechanism for commands, probably because if its long 
> 
> history of being married to GPIB, which has error handling defined via 
> 
> some other means.  In all of my SCPI implementations, I defined a 
> 
> success/failure status response that has served me (and my customers) 
> 
> very well.  If the command succeeded, "OK\r\n" was returned.  If it 
> 
> failed, "ERR<n>\r\n", was returned (<n> is an error code).  That 
> 
> response mode was switchable for compatibility with official SCPI.
> 
> 
> 
> You'll find your SCPI parser code will be reusable for future projects 
> 
> so design it with that in mind.
> 
> 
> 
> JJS


My idea too. The OK/ERR mechanism seems a good idea. Thanks 

On Tuesday, December 17, 2013 1:21:56 AM UTC+2, jmariano wrote:
> ...
> I have read your specification carefully and I will be using some of
> your ideas (hope you don't mind) ...

Of course I don't mind, I would not have made it public if I did :-).

>  ...but I'm not so sure of the ACK reply.I have to think a little about it.
>  I think that with SCPI instruments, the ACK is a flag on a status
> register that the PC reads when we feels necessary. How this two
> approaches compare?  

The main difference which the ack_with_time_to_outcome_report then
the (perhaps much) delayed outcome report makes is that your
device is less dependent on the PC latency - which is impredictable,
can be from milliseconds to ages.
You may or may not need that - you know what you want to do, if
you do not need that sort of thing of course it makes no sense
to add the complexity it will take.
I defined - and implemented it - when I did it without giving it that
much thought, cost me a few hours to draft it. By then I had
enough experience to know how to do it - perhaps with some bearable
overkill - so I would not run into problems later(someone else
was supposed to do the PC side of things so I wanted to have
control over what I was responsible for :-) ).

Dimiter

------------------------------------------------------
Dimiter Popoff, TGI             http://www.tgi-sci.com
------------------------------------------------------
http://www.flickr.com/photos/didi_tgi/sets/72157600228621276/

"jmariano" <jmariano65@gmail.com> wrote in message 
news:845dd624-5685-408a-bf1d-cb4e3bc8a14b@googlegroups.com...

> I'm a part-time prototype developer at my university. My latest project is 
> a box with a microcontroller that measures and actuates on stuff. The box 
> is under the command of a PC, using RS232 or USB, in a master-slave model, 
> the PC being the master. I want to use a message based command language, 
> similar to SCPI, but not so complicated (no tree structure). I was 
> thinking in something like START, STOP SETADC 1000, REAADC 1, etc.
>
> I have to define the syntax and program the parser (in C) on the uc side. 
> Since I don't have very strict specification on the syntax, I can define 
> it in such a way that makes it more easy to analyse, or more robust or 
> etc.

> 2 - Regarding the parser: Is it really a parser that I need or is it 
> something else? Where can I read about this? I just don't what to read the 
> full dragon book just to get to the conclusion that it was the wrong book!

You don't really a parser, assuming the syntax is always going to be 
something like:

command [parameter] ...

Presumably the parameter will either numeric or alphanumeric; it makes it 
easier if the command dictates how many parameters there will be, and what 
type each will be.

And, which is sending the commands, the PC or microcontroller? Decoding will 
be harder work than sending the commands.

With certain languages, you could process this line-oriented kind of input 
just by using its Read statements, reading the command, and each parameter, 
as individual tokens, but is otherwise not difficult to do.

-- 
Bartc



 

On Sat, 14 Dec 2013, jmariano wrote:

> Date: Sat, 14 Dec 2013 13:08:50 -0800 (PST)
> From: jmariano <jmariano65@gmail.com>
> Newsgroups: comp.arch.embedded
> Subject: Parser, again!
> 
> Dear All,
>
> I'm in need of some advice! (not the full solution). A word of caution: 
> i'm not an computer science guy, so probably i'm not using the correct 
> therms!
>
> I'm a part-time prototype developer at my university. My latest project 
> is a box with a microcontroller that measures and actuates on stuff. The 
> box is under the command of a PC, using RS232 or USB, in a master-slave 
> model, the PC being the master. I want to use a message based command 
> language, similar to SCPI, but not so complicated (no tree structure). I 
> was thinking in something like START, STOP SETADC 1000, REAADC 1, etc.
>
> I have to define the syntax and program the parser (in C) on the uc 
> side. Since I don't have very strict specification on the syntax, I can 
> define it in such a way that makes it more easy to analyse, or more 
> robust or etc.
>
> So, my questions are:
>
> 1 - Regarding the language definition: Are there god examples of such 
> language that I can get inspiration from? And references? I'm sure 
> someone as already thought about this in a formal way. I'm looking for 
> practical advice like shall I use fixed length commands (6 character, 
> for example), start-of-message character (#)? Why? And the arguments, 
> separated by commas, spaces? etc.
>
> 2 - Regarding the parser: Is it really a parser that I need or is it 
> something else? Where can I read about this? I just don't what to read 
> the full dragon book just to get to the conclusion that it was the wrong 
> book!
>
> Any thoughts are welcome.
>
> Regards
>
> Mariano
>

HINT:

Here is a snippet of standard C code which takes a null terminated string 
and returns a list of strings as delimited by the Sep characters;

 	whiteSpaceDelimitedList = s_parse( myBuf, " \t\n\r" );

For your problem, a white space delimited token stream is all you need to 
implement an extensible command line interface.  Note that the Str input 
is destroyed in the process (nulls are inserted into the buffer -- a side 
effect of using strsep()), but you can work around that if necessary -- 
and usually it isnt.

The Lst_t* functions are left as an exercise for student.

Lst_t *s_parse( char *Str, char *Sep ){

   char *p, *q ;
   Lst_t  *x ;

   if( !Str )
     return( (Lst_t *) NULL ) ;

   p = Str ;
   x = ls_crt( 1 ) ;
   while( (q = (char *) strsep( (char **) &p, Sep )) ){
     if( !isNul( q ) && strlen( q ) )
       ls_add( x, q ) ;
   }
   return x ;
}


HTH,
Cheers,
Rob.

Tom Gardner wrote:
> On 15/12/13 20:47, Les Cargill wrote:
>> Tom Gardner wrote:
>>> On 15/12/13 18:41, Les Cargill wrote:
>>>> Tom Gardner wrote:
>>>>> On 14/12/13 21:08, jmariano wrote:
>>>>>> Dear All,
>>>> <snip>
>>>>>
>>>>> If that is a possibility, it is probably much cleaner
>>>>> simpler, faster (speed and soon) to embed a Forth interpreter
>>>>> from the outset.
>>>>>
>>>>
>>>> Great idea, although it depends on how much trouble that is
>>>> to actually do in cases.
>>>>
>>>>> Yes, I know the XP/agile fraternity will frown on that.
>>>>> Tough; some of that brigade doesn't know their limits!
>>>>>
>>>>
>>>> I'd think Agile would *support* that, since it should be
>>>> easier to test.
>>>
>>> It tends to go against the holy commandments of "don't
>>> do big up-front design" to "do the simplest thing", because
>>> "you can always refactor it later".
>>>
>>
>> How is Forth anything *but* the simplest thing? My own
>> uncertainty is simply the actions necessary to install the
>> interpreter for a given environment - the various ones
>> linked to look pretty easy to use.
>
> Very rational, which is quite irrelevant in the face
> of the fashionable religion du jour.
>
>

*Blink*  :)

>> This being said, if you just have a simple line-oriented
>> thing, a table-driven "parser" is pretty easy. I posted
>> some actual partial-code in the thread...
>
> That's the rationale! If I've written it it must be
> easier/simpler/better than something I can't be bothered
> to understand.
>

No, it's more a matter of pointing at a thing that solves the
problem that has reasonable cost.

I've done this both ways, with relative-COTS parsers or
100 liner roll-yer-own and it all comes out in the wash.

The trouble with COTS is there is lots of portability and
generic-ity overhead. So for small enough
problems, DIY frequently wins. Now your mandate is to
break big problems into small ones with good interfaces
between them.

COTS may also involve more risk and validation/verification
overhead. So it's pure trade space.

> Me a cynic? Shurely shome mishtake.
>
>
>> Forth is *extremely* hackerish, and therefore ( IMO )
>> consonant with XP. Flexibility ot the goal, right?
>
> NIH. A common complaint.
>
>

Feh. :)

>>> Too many XPers/agilistas, IMNSHO, treat XP/agile as a
>>> religion, i.e. something in which there are (12 IIRC)
>>> Commandments To Be Obeyed in order that the magic
>>> recipe works.
>>>
>>
>> We can't help that; what my takeaway from Agile/XP is
>> test-first, in cases pairs and otherwise eschewing big waterfall.
>
> You clearly have applied common sense, and haven't
> read the religious texts nor derived religious texts
> written by wannabe acolytes or "XP trainers".
>
>

Can't help that.

>> All programming paradigms uncover the latent "Spanish Inquisitor"
>> in people.
>
> Yup.
>
>
>>> That's ridiculous, of course, since XP/agile is immensely
>>> valuable when deployed intelligently in appropriate
>>> circumstances.
>>>
>> Of course. I'd say I first used it in the late '80s,
>> although it wasn't called that. We got stuff done.
>
> Yup. Earlier in my case!
>
>
>>> I've even seen Them strip out all comments from a
>>> well-documented library on the /principal/ that "comments
>>> get out of sync with code" and "good code doesn't need
>>> comments". Often valid,
>>
>> Not if you have good shop culture to support comments.
>> reviews help, too - if you can manage them well.
>
> Ah, but the religion dictates that the comments
> /always/ get out of sync with the code. The code is
> The One Executable Truth (fair enough), therefore
> comments (which will become wrong) are a heresy
> that should be eliminated in case they mislead.
>

I kinda tend to think that, too. Hell is other
people's code :)

> Remarkable, but I've heard it. The next sound was
> that of me going ballistic.
>
>
>>> but not when the comments
>>> describe the subtleties of /why/ the library is
>>> implemented that way and /how/ to use it.
>
>

-- 
Les Cargill

On 20/12/13 19:39, Les Cargill wrote:
> Tom Gardner wrote:
>> On 15/12/13 20:47, Les Cargill wrote:
>>> Tom Gardner wrote:
>>>> I've even seen Them strip out all comments from a
>>>> well-documented library on the /principal/ that "comments
>>>> get out of sync with code" and "good code doesn't need
>>>> comments". Often valid,
>>>
>>> Not if you have good shop culture to support comments.
>>> reviews help, too - if you can manage them well.
>>
>> Ah, but the religion dictates that the comments
>> /always/ get out of sync with the code. The code is
>> The One Executable Truth (fair enough), therefore
>> comments (which will become wrong) are a heresy
>> that should be eliminated in case they mislead.
>>
>
> I kinda tend to think that, too. Hell is other
> people's code :)

I'll disagree, somewhat: "hell is having to *look at*
other people's code".

E.g. I don't think a networking stack is hell; I just use it.

Good comments *reduce* the amount of code I have to
look at because:
   - the problem I'm debugging doesn't match what their
     code is doing
   - they give me sufficient information for me to judge
     accurately the pros and cons of using their code,
     and/or to select between different approaches
all without my having to make guesses based on speed reading
their code and misunderstanding the subtleties and rationale
behind it.

Good commenting isn't difficult, provided the code doesn't
smell and provided time is allotted for comments.

On Friday, December 20, 2013 9:53:43 PM UTC+2, Tom Gardner wrote:
> ...
> Good commenting isn't difficult, provided the code doesn't
> smell and provided time is allotted for comments.

I'll go even further, good commenting saves programming time.
Over the years it saves a lot of programming time and over many
years it makes things possible which otherwise would simply be
impractical if not impossible.

I can speak for myself only of course but while I am writing
and commenting practically each line  (here is a small utility
as an example: http://tgi-sci.com/vpaex/ntp.sa ) I often
discover that I have to stop coding and (re)think something.

Saving time to type in a few words is a huge resource waste.
If one does not have the comment words ready to type in he has even
less code statements ready for typing.

Dimiter

------------------------------------------------------
Dimiter Popoff, TGI             http://www.tgi-sci.com
------------------------------------------------------
http://www.flickr.com/photos/didi_tgi/sets/72157600228621276/