Reply by November 7, 20052005-11-07
Steve Calfee wrote:
...
> It is a shame that there is no universal server finding technique, but > the original arpanet had everyone with a fixed IP and passed around a > hosts.allow file with everyone on the arpanet named. This technique > worked for awhile, but is not scalable and of course windows is not > compatible with it!
... I am not recommending it's use but Windows does still support a host file. In Win2K it is in C:\WINNT\system32\drivers\etc\hosts kevin
Reply by Steve Calfee November 6, 20052005-11-06
On Mon, 31 Oct 2005 07:06:51 -0700, "Don" <None@Given> wrote:

>Hi, David, > >"David Brown" <david@westcontrol.removethisbit.com> wrote in message >news:435f66da@news.wineasy.se... >> If I build an embedded system with an ethernet port and a little >> embedded web server, what is the best way of identifying its IP address? >> It is easy enough to get an address on most networks with DHCP, but >> how does the user know so that they can type in the address in their >> browser? Any ideas would be appreciated. > >You don't state what resources (services) you expect to be available >in the user's environment. :-( (e.g., a home, office, university, etc.) > >Traditionally, RARP, BOOTP and DHCP have been used to >assign ("allocate" may be a better choice of words in the case >of DHCP) an IP to a MAC. Each protocol maps a MAC to >an IP -- on request from the device *having* the MAC (i.e. the >device essentially asks "What is *my* IP address?") > >But, each of these require the respective service to be operational >on a host (server) somewhere on the network in question. AND, >each requires the service to have been *configured* by some >entity (i.e. someone has to build the static map -- bootptab(5), >ethers(5), etc. -- that defines the IP<->MAC relationships OR >the dynamic map that allows DHCP leases to be let). > >While this approach works well in an "organization", it won't >fly in the home market. Instead, the typical approach is to have >your device boot with an RFC 1918 IP -- which can be conveyed >to the user in printed documentation (i.e. EVERY one of your devices >uses the same default IP, typically). >
I think the solution depends on your target network. If your embedded server device will only work on one flavor, say winxp or Linux you can use their default capabilities. When I did the Kitty USB analyzer I wanted universal access. Anything with a browser should be able to access it. This eliminates special dhcp hooks or network clients that try to find a server. Servers are special and need to be found by many different clients/browsers on many different operating systems. The only universal way with an ip server is to have a fixed IP address. All other "finding" techniques will fail if the actual network or clients differs from yours. So I copied what Linksys routers and HP printers do as servers on an arbitrary network. 1) All of the devices start at a known default IP address. (There needs to be some "reset" button for the user to press to reset that device to the default IP address, so when the user forgets what he set it to he can find your server!) 2) The user needs to be able (via a web interface!) to change its IP address, IP netmask and IP gateway router address. It needs to permanently save the settings in flash or eeprom so power fails do not lose its IP address. This does mean that a network administrator (AKA home user) has to change the IP address from the default to work on his network. But it means multiple devices can be on the same network and coexist happily. It is a shame that there is no universal server finding technique, but the original arpanet had everyone with a fixed IP and passed around a hosts.allow file with everyone on the arpanet named. This technique worked for awhile, but is not scalable and of course windows is not compatible with it! There is no "x" in my email address.
Reply by Don November 4, 20052005-11-04
"Stephen Pelc" <stephenXXX@mpeforth.com> wrote.
> On Thu, 03 Nov 2005 20:35:06 -0700, "Richard H." <rh86@no.spam> wrote: > > >If one can assume that a) The environment supports Windows systems, and > >b) that DHCP is supported, you're a good distance there. While not > >always true, this will meet "98%" of your target environments. > > In a Windows network without a router (they still exist), DHCP may not > be available. Nearly all of these can be handled using a protocol > named APIPA, which used to be documented at least on the MSDN site. > It's been a while, but AFAIR it doesn't help with naing problems.
APIPA constrains the choice of IP addresses available to your device *and* the other devices on your subnet. I.e. *this* device "wags the dog". --don
Reply by Stephen Pelc November 4, 20052005-11-04
On Thu, 03 Nov 2005 20:35:06 -0700, "Richard H." <rh86@no.spam> wrote:

>If one can assume that a) The environment supports Windows systems, and >b) that DHCP is supported, you're a good distance there. While not >always true, this will meet "98%" of your target environments.
In a Windows network without a router (they still exist), DHCP may not be available. Nearly all of these can be handled using a protocol named APIPA, which used to be documented at least on the MSDN site. It's been a while, but AFAIR it doesn't help with naing problems. Stephen -- Stephen Pelc, stephenXXX@mpeforth.com MicroProcessor Engineering Ltd - More Real, Less Time 133 Hill Lane, Southampton SO15 5AF, England tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691 web: http://www.mpeforth.com - free VFX Forth downloads
Reply by Rocky November 4, 20052005-11-04
Hi Richard,

Do you have any good references to WINS or Windows local subnet name
resolution mechanism ?

Regards
Robert

Reply by Richard H. November 3, 20052005-11-03
David Brown wrote:
> If I build an embedded system with an ethernet port and a little > embedded web server, what is the best way of identifying its IP address? > It is easy enough to get an address on most networks with DHCP, but how > does the user know so that they can type in the address in their > browser? Any ideas would be appreciated.
David, Someone else alluded to NetBEUI / NetBIOS. This is on the right track for a good solution in a Windows environment. If one can assume that a) The environment supports Windows systems, and b) that DHCP is supported, you're a good distance there. While not always true, this will meet "98%" of your target environments. 1) Boot, get a DHCP, learn the WINS server address. 2) Register with WINS using a pre-defined name 2b) If WINS isn't supported, then use the Windows local subnet name resolution mechanism (local subnet broadcasts, I believe) 3) First device on the network wins the name. Additional devices register themselves with the winner. 4) Users address the pre-defined WINS/NetBEUI name in their ubiquitous Windows-based browser, the name resolves, and the user connects. 4b) User is presented with a list of devices currently online. Hyperlinks point the the current DHCP IP of the device. 5) Configure statically from there. Or - label each widget with a unique random device name less than 16 characters long. That's the WINS name. (Hint: the MAC address is less than 16 characters.) Between ZeroConf and local-subnet Windows name resolution, you should be able to handle just about any environment. Only the pure non-Windows shops would be an issue, and I'll bet there aren't many. Richard
Reply by Don November 3, 20052005-11-03
"Tauno Voipio" <tauno.voipio@INVALIDiki.fi> wrote
> Don wrote:
(sigh) Actually, *you* wrote:
> >>the embedded device is a server and the terminal is a client.
To which I replied:
> > Since when are embedded devices "servers"? There is nothing > > in the nature of an embedded device that makes it a server *or* > > a client. > > The common reason for using TCP/IP in an embedded device > is to use a WWW browser as the control panel for the device. > > This has to be a server.
You're stuck thinking *in* the box! I can (and do!) design boxes that are configured via web interfaces yet are *pure* clients -- by storing the configuration *elsewhere* and letting my client SIMPLY RETRIEVE it!
> Please provide an example of an embedded device which > acts as a client.
I provided an example in the previous post, to which my comments below will refer.
> Yes, I know that TCP is symmetrical as soon as the > connection is made. For the current discussion, a > client is the host initiating the connection and a > server is the host responding to the initiation. This > is the relevant point for the visibility of the > IP addresses: to initiate the connection, the IP > of the responding host has to be known.
First, the IP address needs to be known (*) REGARDLESS of whether a human user needs it in order to type a URL into their browser. The *device* nees to know it in order to carry on the "conversation". So, even devices that don't require any "configuration" *still* need to know their assigned IP. The issues raised by the OP ertain *regardless* of whether or not the IP needs to be (humanly) visible. So, address discovery methods (as brought up by the OP) are important. [* there are games you can play on small, private networks where only MAC addresses are used but that's *too* far "outside the box" for this discussion] Second, *you* are assuming that the "user" is the "client" and the embedded device is the server. That's an *assumption* and not a *requirement*! I've been redesigning the appliances *here* as pure clients -- in a manner similar to the device described in my previous post. Specifically: - all are clients - none store persistent state (or configuration) internally - all are *extremely* simple to design and dirt cheap to build - all are maintained remotely - none are reliant on particular hardware for communication (i.e. I could run SLIP over a length of CAT1) Without knowledge of particular applications/markets, it's hard to come up with specific criteria. There's no way to know if the devices in question are costly or dirt cheap. But, *my* reasoning tries to accommodate the sorts of designs that I have to produce. In no particular order: - By NOT storing state in the device(s), there is less memory required. And, most importantly, no *batteries* to maintain! (Sure, you may get several years out of a lithium cell... but tell a customer that every 5 years they have to replace all of the batteries in every room of their HOTEL and you'll find this "hidden cost" exceeds the purchase price of your device!) Also, removing the battery requirement make the device hardier in terms of the environmental extremes it can safely/practically be exposed to. For example, the garage at my residence sees temperatures in excess of 140F (60C?) daily. Other places where I have resided saw outdoor temps below -20F. Locating a battery in either extreme means short life for the battery! (Or, special instructions telling the user he *can't* locate the device in those extremes). Of course, the use of a battery is suggestive of some nonvolatile *operation* (e.g., keeping track of time of day). Static *data* can be stored in FLASH -- but then you are required to pick a device that *has* flash onboard (or add something externally). Regardless, if you have state preserved onboard, you also need to be able to verify that it hasn't been corrupted. *AND*, if it *has* been corrupted, how do you get an *update*? What do you do *until* you have that update? How do you tell the user "please reconnect your browser to me and tell me what I am supposed to be doing..."? How much error in a ToD clock is "acceptable"? Do you come up with different recovery strategies for different devices based on how critical their stored data is for their continued operation? - Placing the user interface *in* the embedded device significantly increases the complexity of that device. Increased complexity means greater propensiy for bugs. Embedded devices don't readily lend themselves to software upgrades (unless you specifically *add* facilities and mechanisms to do so!) How do you tell the user that he has made an error in specifying a partiular parameter? Is there any "on-line help" available to him? What NLS support do you include in your served pages? Or, do you just tie it to a particular locale? E.g., is today 11/03/05? Or, 20051103? Is it a 24hour clock or 12hour? etc. Do you enforce any mechanisms to regulate access to the configuration "pages"? Perhaps a *password*? Is the user's "session" limited to a single "page"? How do you track a valid session (store a cookie on the user's browser)? What if the user fails to explicitly terminate that "session"? How do you deal with multiple clients wanting to access the HTTP server concurrently? How do you resolve the readers/writers issue? How do you deal with multiple clients wanting to *update* the configuration concurrently (perhaps one is accessing configuration page 1 while nother is accessing page 3)? If you enforce a mutex on the resource, how do you recover from a user failing to release that resource? (remember, the "reset" button on the device may be located a few thousand miles from the user!) - The suggested approach of embedding the web server in the device means you need to have an HTTP server, some sort of "file system" (even if it is a crude collection of char *[]'s referencing the text of the embedded "pages"), some way of parsing the forms data submitted, etc. Invariably, there will be restrictions on the HTML (et al.) features that can be embedded in those pages. Most due to resource requirements. Some due to software support (e.g., serving up streaming audio). Do you track when changes are made to the configuration data (web pages)? Is any sort of audit trail required? Where will *that* reside? How will it be accessed for audits?? (what if your device is a security system for managing access to commercial real estate -- surely you want to know who tampered with the access codes, etc.) - But, embedding a web server implicitly requires support for a full TCP/IP stack! This is more expensive in terms of resources. TCP *connections* consume more resources. Since it is a connection-oriented protocol, there will be *some* limit on the number of connections it can support (how many sockets?). And timers to release sockets from dropped connections, etc. Lot more complexity that has to be managed just to accommodate a server *in* the device. Since resources are never (?) as plentiful in an embedded device as a desktop device, you end up making lots of compromises in the design and implementation. And, many of those cmpromises are directly visible to the user -- since this is the user interface that we are discussing! :> In short, you end up putting a feature-poor interface because a full-featured interface increase the cost/complexity significantly. OTOH, consider the approach I have taken: Devices are pure clients. They rely on server(s) to provide *services* that they need to perform their respective functions. Again, they *RELY* on server(s). Just like your device *RELIES* on having power available. Or having functioning memory. Or an RTC battery that isn't stale, etc. This is key! I can now proceed with the design *knowing* that the services I need are available (with special consideration for those few cases where a resource failure could be costly/invonvenient). Sure, you can claim that you expect your battery backed ToD clock to be accurate and *rely* on that. But, for you to *ensure* that to be true, you must ensure the btteries are replaced regularly, etc. For *me* to be able to rely on services being available, all I need to do is dsign my server(s) to be reliable (this is actually pretty easily accomplished). This can be located in a nice, warm, dry, etc. place (whereas a sprinkler system control box may be located outside, and it may be RAINING when I need to change the battery in it's onboard RTC!) Since the device *knows* that its server(s) *are* always available, there is no need for it to store any persistent state -- RELY on a file server to store that state on its behalf. And, rely on that server to *reliably* store the state (instead of having to design that reliability into the embedded device!) Likewise, no need to maintain a non-volatile ToD clock. Instead, *rely* on a time server to tell you what time it is. No need to provide the user with a means for setting the time on *this* embedded device... and for ensuring that the time set on all of the other time-aware embedded devices are all "in agreement" (ignoring, for the time being, the possibility that you may actually need for those devices to be *precisely* synchronized to sub-second resolution!) Since the device doesn't have to serve web pages, there is no need for an HTTP server, pseudo "file system", etc. The *requirement* to support TCP is now gone. You can opt for a simpler UDP stack. You can use connectionless protocols thereby simplifying the code inside the embedded device. Memory requirements can shrink *dramatically*! You can now deal with ~512 byte packets instead of long datastreams. And, by defining those packets cleverly, you can strip the data you need from the packet *as* you receive it (so you don't even need to store the whole packet and parse it later). Since you *know* you have these services available, you can define the operation of an embedded device (e.g., my sprinkler system described previously), for example, to be: -power up -broadcast BOOTPc request ("Who am I?") using UDP -receive BOOTPs reply ("You are a.b.c.d/m; your name servers are located at e.f.g.h and i.j.k.l; your time server is located at TimeOfDay;your TFTP server is BootstrapServer.MyPlace.com; your bootstrap code is located in the file MyBootstrapCode located in directory /firmware on that server; etc.") - configure your IP address & netmask to a.b.c.d/m - set your resolver library to use e.f.g.h and i.j.k.l - resolve "BootstrapServer.MyPlace.com" using DNS over UDP -- which *should* happen immediately since you *know* these services are available! :> - use the IP returned for BootstrapServer to issue TFTP requests to that server for /firmware/MyBootstrapCode - receive each packet from the TFTP server and build your executable file (in RAM, FLASH, etc.) -- assuming you actually *need* to perform this step (it gives you a *free* mechanism for software updates! :>) - resolve TimeOfDay to get the IP address of the NTP server. - contact the NTP server (over UDP) to get the current time of day. This can be used to initialize a software-based RTC which can be periodically reinitialized (every *minute*, hour, day, etc.). Or, you could implement NTP on the embedded device to synchronize the time more "gracefully". *Or* you could just repeatedly ask "what time is it" and NEVER keep track of the current time! - fetch your configuration file (application specific) from (yet another?) TFTP server and use it to configure your application. - run your application -- consulting servers as necessary to perform those duties ("What time is it? Do I have to *do* anything at *this* time? OK, I'll do it. Otherwise I'll go back and check again...") EXACTLY like the X Terminals (I guess they are NOT so different from "embedded devices" after all, eh? :> ) Now, the user interface can reside in the *server*! The user interfaces for *all* of these devices are just web pages (if you opt for that sort of interface) served up by that *one* web server. Server side scripts can parse the forms submitted by the user and build the "configuration files" used by the embedded device CLIENTS. The server can have a much richer set of resources (than any of the dumb, inexpensive, *plentiful* clients). It can provide "on-line help", more extensive diagnostics, support for different locales, etc. Instead of trying to put all this in *each* embedded device (which makes them *all* more complicated and expensive and, inevitably, causes them all to be kludgy because you can't afford all of those "niceties"), you put it in one place and let them all *share* it. It also makes development of that user interface easier since you aren't recompiling code to embed your newest char *[] of web pages, etc. And, the web pages for each "Model 127 Widget" are guaranteed to be the same -- without you having to remember to update them in each "unit". There are lots of other "free fetures" that come with this sort of approach: For example, if you need to replace a defective device, you simply replace the *device* -- no need to reconfigure it! (the configuration resides on the server(s). Everything on the server(s) can be BACKED UP... how do you backup the configuration of *your* sprinkler system, firewall, router, etc.? :> Maintaining lots of devices *remotely* can be a lot easier -- since some *other* server can talk to *your* server and exchange status/configuration data/updates to a centralized remote host. Since the server(s) are "always available" (i.e. available whenever the devices that depend on them require), you can migrate portions of the application(s) into the server (!). This can be used to "dumb down" the "clients" (for example, keeping track of the date to update the watering schedule for hot/dry months vs. wet/cool ones; *or* to predict sunrise/sunset so that the irrigation schedule can be updated to ensure watering is done at night to minimize evaporative losses) Or, the server's "smarts" can be used to add additionally capabilities that span multiple devices. For example, knowing (from the HVAC control system) that the "swamp cooler" is in use *today* (based on temperature, humidity, etc.) might allow you to modify the irrigation system's schedule since the "bleed off" water from the swamp cooler is being used to irrigate the rose bed (so DON'T run the "roses" circuit in the irrigation system today!) (sigh) Apologies for the long-winded description. :-( It was intended to explain why an embedded device need *not* be a "server". And, to show why this approach can have significant merit -- especially as more and more things become "aware" and more opportunities for integration *among* those devices arise. At least, it hopefully shows *one* way to think outside the box :-/ --don
Reply by Tauno Voipio November 3, 20052005-11-03
Don wrote:
> >>the embedded device is a server and the terminal is a client. > > > Since when are embedded devices "servers"? There is nothing > in the nature of an embedded device that makes it a server *or* > a client.
The common reason for using TCP/IP in an embedded device is to use a WWW browser as the control panel for the device. This has to be a server. Please provide an example of an embedded device which acts as a client. Yes, I know that TCP is symmetrical as soon as the connection is made. For the current discussion, a client is the host initiating the connection and a server is the host responding to the initiation. This is the relevant point for the visibility of the IP addresses: to initiate the connection, the IP of the responding host has to be known. -- Tauno Voipio tauno voipio (at) iki fi
Reply by Don November 2, 20052005-11-02
"Tauno Voipio" <tauno.voipio@INVALIDiki.fi> wrote
> David Brown wrote:
Actally, if you are a bit more careful checking your attributions, you'd realize that *I* wrote (the following):
> >> For example, the X Terminals that I use have *dozens* of > >> configuration options related to basic configuration (address > >> discovery, name resolution and bootstrap protocols) as > >> they tried to be "everything to everyone". A similar approach > >> could let *you* be everything to everyone WITHOUT > >> intimidating them with that long list of options... > > Please note that there is a basic difference between > a X-terminal and a typical embedded device:
Sure!
>the embedded device is a server and the terminal is a client.
Since when are embedded devices "servers"? There is nothing in the nature of an embedded device that makes it a server *or* a client. I have a box that sits in the yard and turns the sprinklers on. It is a *pure* client -- it fetches the time of day AND the watering schedule from a regular UNIX server sitting in the house. Essentially, it just knows how to parse the "schedule file" and compare the current time (via NTP) to the times recorded in the schedule file -- and activate particular solenoids based on the results of those comparisons. It doesn't have to have any sense of (real) time (other than standard timers to implement timeouts on network services, etc.) so it doesn't need a "real time clock". Nor a battery. Nor a means of *setting* that clock. It doesn't need any sort of configuration utility to allow the user to specify a watering schedule -- that is done on the server side. I.e. it *provides* no service (to network clients). OTOH, I have another box that interfaces to the household HVAC. I chose to implement it as a freestanding *server* (i.e. if my UNIX servers die, I don't worry that the yard will dry up overnight and die; but if the thermostat was unable to fetch ToD and configuration information due to an outage, the house could get *quite* uncomfortable in just a few *hours*!)
> For a client it's far less important to know the IP than for > a server.
The IP stack needs to know the IP *regardless* of whether or not the "client" wants to be conciously aware of it. So, getting the IP *to* the box is equally important regardless of the role the box plays. --don
Reply by Don November 2, 20052005-11-02
"David Brown" <david@westcontrol.removethisbit.com> wrote
> Don wrote: > > <snip to save space>
Agreed! :>
> > Again, I think you should try to treat each application domain > > separately. Develop solutions for each independantly. If you > > later want to come up with a common code base for *all* of > > them, then you can bury some configuration switches in the > > code (possibly not even user accessible) to decide which > > srategies the device adopts for address discovery. > > > > For example, the X Terminals that I use have *dozens* of > > configuration options related to basic configuration (address > > discovery, name resolution and bootstrap protocols) as > > they tried to be "everything to everyone". A similar approach > > could let *you* be everything to everyone WITHOUT > > intimidating them with that long list of options... > > > > --don > > Thanks (again) for the information - I'll be keeping a copy of it for > future reference. I fully agree that we'll need to use different > techniques for different markets - with this thread, I've been trying to > gather some general information and ideas so that I know what the > possibilities are.
Understood.
> > P.S. Look into NetBEUI also. > > I was under the impression that NetBEUI was very much on the way out, > and not even supported on newer windows (at least, not by default).
That's quite possible. I haven't used a MS product since W98 so I'm not the right one to comment. As with every OTHER technology, I'm sure MS will *invent* TCP/IP any day now!! :>
> NetBios over TCP/IP is an option for a local naming services, but I > don't know much about its implementation. Is it a straightforward task? > I'm only thinking about the naming protocol, not file or printer > sharing or any sort of authentication or directory services.
NetBIOS doesn't scale well, can be easily abused, etc. I just mentioned it to get you thinking in other directions (again, not knowing what your specific applications and markets are...) It is often beneficial to look at a variety of technologies even if they don't *seem* to fit your needs. Epiphanies come in many colors! :> --don