EmbeddedRelated.com
Forums
Memfault Beyond the Launch

Configure network of an embedded device

Started by pozz September 5, 2023
Nowadays many Internet connected embedded devices are controlled by an 
external proprietary cloud service (for example Sonoff[1] or Shelly[2]). 
The installation in the final destination is very simple to the end user 
that ignores completely all the technical issues:

1. connect the device to the local Ethernet network (WiFi is slightly 
more complex)
2. pair his mobile app to the device (QR code or serial number)
3. start configuring and using

Most probably the device works with DHCP client enabled, hoping to find 
a DHCP server on the network and receive a suitable IP address to make 
outcoming connection to the Cloud server. This happens in 99.99% of 
cases and the user is super happy.

In other situations, if the gadget features a small/big display, the 
advanced user could enter a network configuration menu and set the 
desired configuration (DHCP or fixed IP address). This happens for 
desktop PCs or tablets or similar gadgets (even if the user keeps most 
of the time the DHCP default configuration).

I'm designian't beng a small embedded device that hasn't a display and c 
controlled by a cloud system. It will be controlled on the local network 
through a simple web browser pointed to the IP address of the gadget. 
Indeed, a web server runs in the device.

I'm thinking to start with a fixed IP address, for example 
192.168.1.123. In 90% of cases the IP address can be used immediately 
(written on a label on the gadget or in the quick start guide) and the 
user could access the web page pointing the web browser to 
192.168.1.123. In cases where the local network isn't 192.168.1.x, the 
user should use a PC configured temporarily with a compatible IP address 
(192.168.1.124), access the web page and change the network configuration.

The problem happens when the user wants to install several devices on 
the same network, for example 10 devices. Even if the network is 
compatible (192.168.1.x) with the default fixed IP address 
(192.168.1.123), he should connect one device at a time and change its 
configuration before connecting another device to avoid IP addresses 
conflicts.

So I'm wondering if there's a standard or quasi-standard way to manage 
network configuration of devices on the same LAN.

The situation isn't so uncommon. IPCams are network oriented devices 
that can be controller by a Cloud service from the manufacturer, but 
mainly from a web page. It usually happens that more than one IPCam are 
installed on the same LAN.

For example, Dahua IPCams usually start with the fixed IP address 
192.168.1.108. They have a software (Config Tool) that can be used[3] to 
manage network configuration of multiple IPCams, even if they are 
installed at the same time with the default IP address 192.168.1.108.

Is it a proprietary solution that uses only Ethernet frames (MAC 
addresses) and not IP packets? Is it a well known protocol that I don't 
know?



[1] https://sonoff.tech/
[2] https://www.shelly.com/en-gb
[3] https://www.youtube.com/watch?v=NIiI1BIHfms
On 9/5/2023 11:37, pozz wrote:
> Nowadays many Internet connected embedded devices are controlled by an > external proprietary cloud service (for example Sonoff[1] or Shelly[2]). > The installation in the final destination is very simple to the end user > that ignores completely all the technical issues: > > 1. connect the device to the local Ethernet network (WiFi is slightly > more complex) > 2. pair his mobile app to the device (QR code or serial number) > 3. start configuring and using > > Most probably the device works with DHCP client enabled, hoping to find > a DHCP server on the network and receive a suitable IP address to make > outcoming connection to the Cloud server. This happens in 99.99% of > cases and the user is super happy. > > In other situations, if the gadget features a small/big display, the > advanced user could enter a network configuration menu and set the > desired configuration (DHCP or fixed IP address). This happens for > desktop PCs or tablets or similar gadgets (even if the user keeps most > of the time the DHCP default configuration). > > I'm designian't beng a small embedded device that hasn't a display and c > controlled by a cloud system. It will be controlled on the local network > through a simple web browser pointed to the IP address of the gadget. > Indeed, a web server runs in the device. > > I'm thinking to start with a fixed IP address, for example > 192.168.1.123. In 90% of cases the IP address can be used immediately > (written on a label on the gadget or in the quick start guide) and the > user could access the web page pointing the web browser to > 192.168.1.123. In cases where the local network isn't 192.168.1.x, the > user should use a PC configured temporarily with a compatible IP address > (192.168.1.124), access the web page and change the network configuration. > > The problem happens when the user wants to install several devices on > the same network, for example 10 devices. Even if the network is > compatible (192.168.1.x) with the default fixed IP address > (192.168.1.123), he should connect one device at a time and change its > configuration before connecting another device to avoid IP addresses > conflicts. > > So I'm wondering if there's a standard or quasi-standard way to manage > network configuration of devices on the same LAN. > > The situation isn't so uncommon. IPCams are network oriented devices > that can be controller by a Cloud service from the manufacturer, but > mainly from a web page. It usually happens that more than one IPCam are > installed on the same LAN. > > For example, Dahua IPCams usually start with the fixed IP address > 192.168.1.108. They have a software (Config Tool) that can be used[3] to > manage network configuration of multiple IPCams, even if they are > installed at the same time with the default IP address 192.168.1.108. > > Is it a proprietary solution that uses only Ethernet frames (MAC > addresses) and not IP packets? Is it a well known protocol that I don't > know? > > > > [1] https://sonoff.tech/ > [2] https://www.shelly.com/en-gb > [3] https://www.youtube.com/watch?v=NIiI1BIHfms
I had the same issue some 15 years ago and I think I posted a similar question here. Back then the "IoT" phrase was not invented, so I am not sure if our netmca belongs... :) ( http://tgi-sci.com/tgi/nmca3.htm ) Anyway, the world has not changed much since, same issue same possible remedies. My initial one - which is still active on our devices - was to post the IP address on a specific web address under our domain once the device boots (it does so via ftp to a dedicated account we have for that); the address contains the device serial number which makes it specific, the user can locate it using a browser. Nobody uses it. They all call before they even try to use that. So we started shipping a router with each device; the router is set to give fixed IP addresses to each of the devices we ship to that customer (sometimes they are more than one). This has worked fine. Even if they try to stray from the router we have shipped they can verify things work with it and dig further themselves, look for help locally etc. ====================================================== Dimiter Popoff, TGI http://www.tgi-sci.com ====================================================== http://www.flickr.com/photos/didi_tgi/
pozz <pozzugno@gmail.com> wrote:
> So I'm wondering if there's a standard or quasi-standard way to manage > network configuration of devices on the same LAN. > > The situation isn't so uncommon. IPCams are network oriented devices > that can be controller by a Cloud service from the manufacturer, but > mainly from a web page. It usually happens that more than one IPCam are > installed on the same LAN.
There are several ways to do this. If there's ethernet, it's not too complicated. When the device boots, make a DHCP request and provide your hostname as the device name plus a serial number. For example, supposing your product is called 'HomeBox' and the device's serial number (MAC address?) is 12:34:56:78:90:ab. Then make a DHCP request and specify your hostname as 'homebox90ab'. Use enough digits to minimise the changes of name collisions. A user can either look on the router for the 'homebox' device, or you print instructions telling the user to go to http://homebox90ab/ on a label. At that site they can configure the network settings if they want a fixed IP or whatever. For wifi, it's harder because you need to preconfigure wifi settings before the device appears on the network. One way is to include Bluetooth functionality, which is included on many wifi chipsets. An app is told the wifi config by the user, connects to the Bluetooth and sends the details that way. The device then connects to the wifi and may shut down the Bluetooth interface. To reconnect, a factory reset via a pinhole reset button will clear the wifi settings and re-enable Bluetooth. If you don't have Bluetooth or a mobile app, another option is to power up in factory mode offering an access point with a default SSID, say 'homebox90ab', and no password. The user connects to the SSID and goes to a web page as the ethernet example. There they configure their wifi credentials and, once saved, the device reboots and tries to connect to that wifi network. If it fails, the user must do the pinhole reset and try again.
> For example, Dahua IPCams usually start with the fixed IP address > 192.168.1.108. They have a software (Config Tool) that can be used[3] to > manage network configuration of multiple IPCams, even if they are > installed at the same time with the default IP address 192.168.1.108.
On an enterprise network, maybe expecting any kind of DHCP is too much. In which case the initial setup could be its own DHCP server: you plug in a laptop which gets an IP address handed out by the device, and go to http://192.168.1.1/ (or http://homebox90ab/) which is the device. From there you configure its IP address and other settings. Next time it reboots it uses the new settings. If it is no longer accessible, there's the pinhole reset procedure. While it's possible for the device to answer at a specific hardcoded IP, it's a bit awkward from the user perspective - you have to configure your laptop with a particular static IP, netmask and so on. Many OSes make that annoying, and you can't easily switch to another network without going in and deleting all the settings (eg you want to search for help while configuring the device). On IPv6, a lot of these problems go away if SLAAC is enabled: you know your network prefix, you know what the device's MAC address, so it's trivial to work out the device's static IP, or at least one IP that it'll respond to if you try to connect to it. Theo
Il 06/09/2023 22:56, Theo ha scritto:
> pozz <pozzugno@gmail.com> wrote: >> So I'm wondering if there's a standard or quasi-standard way to manage >> network configuration of devices on the same LAN. >> >> The situation isn't so uncommon. IPCams are network oriented devices >> that can be controller by a Cloud service from the manufacturer, but >> mainly from a web page. It usually happens that more than one IPCam are >> installed on the same LAN. > > There are several ways to do this. > > If there's ethernet, it's not too complicated. When the device boots, make a > DHCP request and provide your hostname as the device name plus a serial > number. For example, supposing your product is called 'HomeBox' and the > device's serial number (MAC address?) is 12:34:56:78:90:ab. Then make a > DHCP request and specify your hostname as 'homebox90ab'. Use enough digits > to minimise the changes of name collisions. A user can either look on the > router for the 'homebox' device,
This isn't an option for typical users without technical knowledge.
> or you print instructions telling the user > to go to http://homebox90ab/ on a label.
Interesting. That URL has a simple hostname that must be resolved by the router. Is it guaranteed that the router is able to resolve the host in URL in the device with the same hostname? What happens if more than one device on the LAN make DHCP requests with the same hostname?
> At that site they can configure > the network settings if they want a fixed IP or whatever. > > For wifi, it's harder because you need to preconfigure wifi settings before > the device appears on the network. > > One way is to include Bluetooth functionality, which is included on many > wifi chipsets. An app is told the wifi config by the user, connects to the > Bluetooth and sends the details that way. The device then connects to the > wifi and may shut down the Bluetooth interface. To reconnect, a factory > reset via a pinhole reset button will clear the wifi settings and re-enable > Bluetooth. > > If you don't have Bluetooth or a mobile app, another option is to power up > in factory mode offering an access point with a default SSID, say > 'homebox90ab', and no password. The user connects to the SSID and goes to a > web page as the ethernet example. There they configure their wifi > credentials and, once saved, the device reboots and tries to connect to that > wifi network. If it fails, the user must do the pinhole reset and try > again. > >> For example, Dahua IPCams usually start with the fixed IP address >> 192.168.1.108. They have a software (Config Tool) that can be used[3] to >> manage network configuration of multiple IPCams, even if they are >> installed at the same time with the default IP address 192.168.1.108. > > On an enterprise network, maybe expecting any kind of DHCP is too much. In > which case the initial setup could be its own DHCP server: you plug in a > laptop which gets an IP address handed out by the device, and go to > http://192.168.1.1/ (or http://homebox90ab/) which is the device.
Are you thinking to have a DHCP server running *on* the embedded device? How does the device know if send a DHCP request (to receive an IP address from an external DHCP server) or launch a DHCP server? Maybe it can try a DHCP request a revert back to internal DHCP server if that fails.
> From > there you configure its IP address and other settings. Next time it reboots > it uses the new settings. If it is no longer accessible, there's the > pinhole reset procedure.
We can say that in professional environment (enterprise network) it's simple to find experts with sufficient knowledge to set network configuration of a device.
> While it's possible for the device to answer at a specific hardcoded IP, > it's a bit awkward from the user perspective - you have to configure your > laptop with a particular static IP, netmask and so on. Many OSes make that > annoying, and you can't easily switch to another network without going in > and deleting all the settings (eg you want to search for help while > configuring the device).
It's very strange there isn't any protocol at MAC level that manages IP configuration. MAC addresses are unique in the LAN and they could be discovered or printed on a label. With this protocol, you could connect and power up all the devices and use a simple software to manage network IP configurations.
> On IPv6, a lot of these problems go away if SLAAC is enabled: you know your > network prefix, you know what the device's MAC address, so it's trivial to > work out the device's static IP, or at least one IP that it'll respond to if > you try to connect to it. > > Theo
On 9/5/2023 1:37 AM, pozz wrote:
> Is it a proprietary solution that uses only Ethernet frames (MAC addresses) and > not IP packets? Is it a well known protocol that I don't know?
RARP gave way to BOOTP which gave way to DHCP for exactly this reason. They all run *below* the IP layer so can be implemented (client-side) relatively easily. Assigning IP, hostname, gateway, nameserver, timeserver, boot server, boot image, etc. are all done, there. (You can operate an ethernet without IP at all!) The problem is: - having a suitable server present on the network (not all will have this -- though most SOHOs will) - conveying the parameters that were assigned by the service to the *human* user (without requiring special knowledge of a special tool which would require more knowledge of the user's operating environment *or* having a UI on the device)
On 9/5/2023 3:43 AM, Dimiter_Popoff wrote:
> I had the same issue some 15 years ago and I think I posted a similar > question here. Back then the "IoT" phrase was not invented, so I am not > sure if our netmca belongs... :) ( http://tgi-sci.com/tgi/nmca3.htm ) > > Anyway, the world has not changed much since, same issue same possible > remedies. > My initial one - which is still active on our devices - was to post > the IP address on a specific web address under our domain once the > device boots (it does so via ftp to a dedicated account we have > for that); the address contains the device serial number which > makes it specific, the user can locate it using a browser.
Essentially creating a (remote) UI to serve in place of the "limited" UI on the instrument. You rely on the user to have something ELSE that can provide the interface by leveraging other popular protocols.
> Nobody uses it. They all call before they even try to use that.
As the device must then be routed, is there a risk of some "outside agent" accessing the device?
> So we started shipping a router with each device; the router is set > to give fixed IP addresses to each of the devices we ship to > that customer (sometimes they are more than one).
Can the user alter the "default" name to something friendlier? Possibly an alias? Presumably, the MAC (or serial number as that is likely friendlier) is easily accessible on the device (and, the user is in proximity to the device)?
> This has worked fine. Even if they try to stray from the router > we have shipped they can verify things work with it and dig > further themselves, look for help locally etc.
pozz <pozzugno@gmail.com> wrote:
> Il 06/09/2023 22:56, Theo ha scritto: > > pozz <pozzugno@gmail.com> wrote: > >> So I'm wondering if there's a standard or quasi-standard way to manage > >> network configuration of devices on the same LAN. > >> > >> The situation isn't so uncommon. IPCams are network oriented devices > >> that can be controller by a Cloud service from the manufacturer, but > >> mainly from a web page. It usually happens that more than one IPCam are > >> installed on the same LAN. > > > > There are several ways to do this. > > > > If there's ethernet, it's not too complicated. When the device boots, make a > > DHCP request and provide your hostname as the device name plus a serial > > number. For example, supposing your product is called 'HomeBox' and the > > device's serial number (MAC address?) is 12:34:56:78:90:ab. Then make a > > DHCP request and specify your hostname as 'homebox90ab'. Use enough digits > > to minimise the changes of name collisions. A user can either look on the > > router for the 'homebox' device, > > This isn't an option for typical users without technical knowledge. > > > > or you print instructions telling the user > > to go to http://homebox90ab/ on a label. > > Interesting. That URL has a simple hostname that must be resolved by the > router. Is it guaranteed that the router is able to resolve the host in > URL in the device with the same hostname?
It's the way most consumer routers work, there's no guarantees that somebody doesn't have a different setup though. It is possible to use multicast DNS (aka mDNS aka Bonjour) in which case http://homebox90ab.local/ will be resolvable (since any .local address is resolved by sending out multicasts asking who has it, rather than asking a central DHCP server).
> What happens if more than one > device on the LAN make DHCP requests with the same hostname?
That's why you include a probably-unique hostname with a serial number. If they still clash, I imagine the DHCP server hands out a different address for device #2 and refuses to assign it a hostname already allocated.
> > On an enterprise network, maybe expecting any kind of DHCP is too much. In > > which case the initial setup could be its own DHCP server: you plug in a > > laptop which gets an IP address handed out by the device, and go to > > http://192.168.1.1/ (or http://homebox90ab/) which is the device. > > Are you thinking to have a DHCP server running *on* the embedded device?
Yes.
> How does the device know if send a DHCP request (to receive an IP > address from an external DHCP server) or launch a DHCP server? > Maybe it can try a DHCP request a revert back to internal DHCP server if > that fails.
The device out of the box is a DHCP server. Once somebody logs into it and configures it, it reboots to become a DHCP client. To revert back to being a DHCP server again somebody has to hold down the factory reset button. (or equivalent physical signal, eg smart lightbulbs you have to turn them on and off several times in a predefined sequence of flashes)
> > From > > there you configure its IP address and other settings. Next time it reboots > > it uses the new settings. If it is no longer accessible, there's the > > pinhole reset procedure. > > We can say that in professional environment (enterprise network) it's > simple to find experts with sufficient knowledge to set network > configuration of a device.
That applies to pretty much any device you want local-only access: somebody has to configure it, whether via a web interface or a mobile app. It can't just work by being plugged in, unless it talks to the cloud to pick up its configuration. If the admins have decided the devices on their network all need static configuration, they will need to use the device's config interface to set that up.
> > While it's possible for the device to answer at a specific hardcoded IP, > > it's a bit awkward from the user perspective - you have to configure your > > laptop with a particular static IP, netmask and so on. Many OSes make that > > annoying, and you can't easily switch to another network without going in > > and deleting all the settings (eg you want to search for help while > > configuring the device). > > It's very strange there isn't any protocol at MAC level that manages IP > configuration. MAC addresses are unique in the LAN and they could be > discovered or printed on a label. > With this protocol, you could connect and power up all the devices and > use a simple software to manage network IP configurations.
There is, it's called DHCP. Or IPv6 SLAAC, as below. As Don says, once you've done DHCP everything is configured, but you either need to have a UI to either the DHCP server or the device if you want to find out what happened. If it is specifically DNS you're interested in, there's mDNS, which also does service discovery - connect a printer to the network and it'll become available to devices like phones without any config; this is because the printer responds to mDNS queries from phones, laptops, etc saying what services it offers and the devices configure themselves automatically (the printer also says it accepts certain standard formats like raster or PDF so no need for drivers).
> > On IPv6, a lot of these problems go away if SLAAC is enabled: you know your > > network prefix, you know what the device's MAC address, so it's trivial to > > work out the device's static IP, or at least one IP that it'll respond to if > > you try to connect to it.
Theo
On 9/7/2023 11:40 AM, Theo wrote:
>> It's very strange there isn't any protocol at MAC level that manages IP >> configuration. MAC addresses are unique in the LAN and they could be >> discovered or printed on a label. >> With this protocol, you could connect and power up all the devices and >> use a simple software to manage network IP configurations. > > There is, it's called DHCP. Or IPv6 SLAAC, as below. As Don says, once > you've done DHCP everything is configured, but you either need to have a UI > to either the DHCP server or the device if you want to find out what > happened.
For SOHO applications, one can likely gain access to the "clients list" in the DHCP server. But, that may not be the case in other scenarios (e.g., enterprise). It also ASSUMES you have a server running, locally (also common in SOHO). I've seen applications that would listen for broadcasts from "their" devices (recognized by OUI or content of the message packet) and present lists of these devices to the user. The user could then contact the device directly (through the application) to query (or modify) its configuration. But, this (IMO) *raises* the bar for casual users. You can also add some other "dedicated" point-to-point port on the device that you use to talk to *it* (before it sits on the wire). E.g., Cisco APs, APC UPSs, etc. all have such a "console" function. But, again, you now expect the user to be more tech savvy. A clever way of doing this for "one-off" units is to have a USB interface and present to the host as a mass storage device with an "autorun" that causes a *portable* app on the device to be invoked so the software is part of the device instead of needing to be "installed" on a phone, PC, etc. The app then presents a configuration interface for the user writing the configuration data back into the device (before being disconnected). [Perhaps a JS app could be more portable -- think Macs -- than a binary] The app could also give you a beachhead through which to install updates in the device (if over-the-wire updates become a problem). [I use an ethernet based version of this in my current design. A "dedicated" port allows for secure configuration of "new" devices prior to deployment. It lets me install firmware, secrets, configuration information, etc. -- as well as inventory the devices encountered, to date]
> If it is specifically DNS you're interested in, there's mDNS, > which also does service discovery - connect a printer to the network and > it'll become available to devices like phones without any config; this is > because the printer responds to mDNS queries from phones, laptops, etc > saying what services it offers and the devices configure themselves > automatically (the printer also says it accepts certain standard formats > like raster or PDF so no need for drivers).
On 9/7/2023 11:40 AM, Theo wrote:
>> How does the device know if send a DHCP request (to receive an IP >> address from an external DHCP server) or launch a DHCP server? >> Maybe it can try a DHCP request a revert back to internal DHCP server if >> that fails. > > The device out of the box is a DHCP server. Once somebody logs into it and > configures it, it reboots to become a DHCP client. To revert back to being > a DHCP server again somebody has to hold down the factory reset button. > (or equivalent physical signal, eg smart lightbulbs you have to turn them on > and off several times in a predefined sequence of flashes)
How do you address the case of your (GUI) client discovering some other DHCP service running on the network? If you could modify the *client* code, you could use something like a different vendor magic to ensure only the server in the device would be recognized. You could flip this around; distribute an app that acts as a special DHCP server. Have the device only issue requests to services offered by *that* server (even if another was competing -- see above). In addition to servicing DHCP requests, it could query any *existing* DHCP service to obtain a lease FOR THE DEVICE (acting as a proxy). In that way, it learns the subnet that the user's network would LIKE to be used (resolving RFC1918 ambiguities as well as *assigned* IP ranges) This covers the case for no DHCP server present as well as a competing DHCP service. And, gives the user a handle into the configuration process as a desired side effect. But, it's yet another app -- hosted OUTSIDE the device -- that has to be maintained :< And, you can't lose sight of the fact that the user has to arrange for any sort of "persistence" that he may require... in light of a network that may not inherently want to offer it!
On 9/7/2023 23:30, Don Y wrote:
> On 9/7/2023 11:40 AM, Theo wrote: >>> How does the device know if send a DHCP request (to receive an IP >>> address from an external DHCP server) or launch a DHCP server? >>> Maybe it can try a DHCP request a revert back to internal DHCP server if >>> that fails. >> >> The device out of the box is a DHCP server.&nbsp; Once somebody logs into >> it and >> configures it, it reboots to become a DHCP client.&nbsp; To revert back to >> being >> a DHCP server again somebody has to hold down the factory reset button. >> (or equivalent physical signal, eg smart lightbulbs you have to turn >> them on >> and off several times in a predefined sequence of flashes) > > How do you address the case of your (GUI) client discovering some other > DHCP service running on the network?
This is another point added to my approach "sell them a router you have set up". In fact I had exactly this at some point, a customer for our TLD readers decided to stray from the router we had delivered with the units they had purchased. The people who had to do the measurements (sometimes 1000+ a day) had no access to the corporate router so someone there had set some fixed IP addresses for our devices on their network, not behind the router we supplied. Things worked - until they did not. They started to call "your device won't boot at times". "Does the LED always indicate the unit did get an IP address?" "Yes it does." Well does it always boot behind the router we supplied? "Hmm let us test... Yes it does" [Most likely this was yet another sabotage against our devices at this customer (they have no legit second dhcp server there so someone must have set one up, the problems started some time after they changed routers and well, they have a history of sabotage attempts there, there was physical evidence for that).] If a design can afford a cheap router just include one and save yourself all the headache (that to the OP). Other than that, you are stuck to my other solution, which works - just technically, I saw no evidence anyone used it ever. People either can do what it takes with their router/network or they use the router we supply.

Memfault Beyond the Launch