Forums

Plug&Play solution to reach an embedded HTTP server under a router

Started by pozz April 6, 2018
Il 09/04/2018 01:39, Clifford Heath ha scritto:
> On 07/04/18 01:51, pozz wrote: >> I'm designing a electronic device running an embedded HTTP server. I'm >> using lwip TCP/IP stack and the httpd application that comes with lwip. >> >> Because the device is under an ADSL router and the public IP is >> dynamic, I need to register to a DDNS service, configure it in the >> router and open one port (NAT). These steps are too tricky for the >> most of my users. >> So I'm thinking to a simpler solution. >> >> I know I can create a cloud on the Internet that runs a standard HTTP >> server to the users, and a custom "IOT" protocol to the remote >> devices. However this way is too difficult for me and I have to change >> almost everything that now works well. >> >> I'm thinking of a public HTTP tunnel that limits to forward the users >> HTTP requests to the HTTP server. Of course, the tunnel is made by two >> ends: one on the public server (and I could use everything, because I >> have to start from zero), the other on the embedded device (where I >> have many restrictions on memory size, computational power, no-OS and >> so on). >> >> Any experience on implementing a HTTP tunnel on lwip? > > You want UPnP, specifically > <https://en.wikipedia.org/wiki/Universal_Plug_and_Play#NAT_traversal>. > > This is supported by basically all domestic routers, to > provide access from the Internet to e.g. gaming consoles. > > It allows your device to find the router and ask it to > open a port from the outside.
This is interesting, even if I understood there are many similar technologies: UPnP, NAT-PMP, Internet Gateway Device Protocol, PCP (Port Control Protocol). I don't know which one is the right choice. It also depends if those are implemented in popular home routers. Anyway, even if the NAT traversal problem is solved, there's another issue: the *dynamic* public IP address that could be solved with a Dynamic DNS account. Anyway free DDNS accounts require manual confirmation every 30 days and this could be a problem for the end user.
On Sunday, April 8, 2018 at 7:39:47 PM UTC-4, Clifford Heath wrote:
> You want UPnP, specifically > <https://en.wikipedia.org/wiki/Universal_Plug_and_Play#NAT_traversal>.
Very interesting, Thanks Clifford. Some concerns: The article cited above says: "UPnP is generally regarded as unsuitable for deployment in business settings..." "... As a result, some UPnP devices ship with UPnP turned off by default as a security measure." Have you used UPnP successfully for commercial applications? Thanks! Best Regards, Dave
Il 09/04/2018 14:36, Dave Nadler ha scritto:
> On Sunday, April 8, 2018 at 7:39:47 PM UTC-4, Clifford Heath wrote: >> You want UPnP, specifically >> <https://en.wikipedia.org/wiki/Universal_Plug_and_Play#NAT_traversal>. > > Very interesting, Thanks Clifford. > Some concerns: The article cited above says: > "UPnP is generally regarded as unsuitable for deployment in business settings..." > "... As a result, some UPnP devices ship with UPnP turned off by default as a security measure." > > Have you used UPnP successfully for commercial applications?
I made just some tests on a couple of home routers and both don't support NAT-PMP, that is a *different* protocol than UPnP/IGD. Both routers replied with ICMP Port Unreachable. I think they support UPnP/IGD because I see something in their web pages. However it's not simple to test in a short time. Of course you can't expect all the routers support UPnP and that this functionality is enabled by the user. I think UPnP is enabled by default, if supported. But the user can disable it, mostly for security concerns.
On Mon, 9 Apr 2018 14:54:03 +0200, pozz <pozzugno@gmail.com> wrote:

>Il 09/04/2018 14:36, Dave Nadler ha scritto: >> On Sunday, April 8, 2018 at 7:39:47 PM UTC-4, Clifford Heath wrote: >>> You want UPnP, specifically >>> <https://en.wikipedia.org/wiki/Universal_Plug_and_Play#NAT_traversal>. >> >> Very interesting, Thanks Clifford. >> Some concerns: The article cited above says: >> "UPnP is generally regarded as unsuitable for deployment in business settings..." >> "... As a result, some UPnP devices ship with UPnP turned off by default as a security measure." >> >> Have you used UPnP successfully for commercial applications? > >I made just some tests on a couple of home routers and both don't >support NAT-PMP, that is a *different* protocol than UPnP/IGD. >Both routers replied with ICMP Port Unreachable.
Most commercial (business/enterprise) routers filter most ICMP traffic - by default some won't even allow pings. Client applications are expected to be using TCP or UDP.
>I think they support UPnP/IGD because I see something in their web >pages. However it's not simple to test in a short time. > >Of course you can't expect all the routers support UPnP and that this >functionality is enabled by the user. I think UPnP is enabled by >default, if supported. But the user can disable it, mostly for security >concerns.
Yes. UPnP can be a major security hole. It's disabled by default in most (all?) commercial units, and now even in some better SOHO units. George
On Fri, 6 Apr 2018 17:51:11 +0200, pozz <pozzugno@gmail.com> wrote:

>I'm designing a electronic device running an embedded HTTP server. I'm >using lwip TCP/IP stack and the httpd application that comes with lwip. > >Because the device is under an ADSL router and the public IP is dynamic, >I need to register to a DDNS service, configure it in the router and >open one port (NAT). These steps are too tricky for the most of my users. >So I'm thinking to a simpler solution.
You don't need any of that if the devices make outgoing connections.
>I know I can create a cloud on the Internet that runs a standard HTTP >server to the users, and a custom "IOT" protocol to the remote devices. >However this way is too difficult for me and I have to change almost >everything that now works well.
HTTPS is the best choice for something that has to work in a security environment because (unless an IP is blocked) all routers will pass an outgoing HTTP(S) request. Other protocols tend to be limited or disabled and may require user/IT intervention to configure. But if your <whatever> is something they want, the customers certainly will arrange to pass your cloud traffic even if it's not HTTP.
>I'm thinking of a public HTTP tunnel that limits to forward the users >HTTP requests to the HTTP server. Of course, the tunnel is made by two >ends: one on the public server (and I could use everything, because I >have to start from zero), the other on the embedded device (where I have >many restrictions on memory size, computational power, no-OS and so on).
I'm assuming here that by "tunnel", you mean you need to pass binary data in the HTTP request/response. Assuming your server IP isn't blocked, your devices should be able to connect directly from behind a router. You shouldn't need any kind of tunneling proxy. If you need to send a lot of data you may want to implement HTTP stream transfers rather than having to ROT64 (or whatever) to embed your binary data into the requests/responses.
>Any experience on implementing a HTTP tunnel on lwip?
Sorry - never used lwip. But HTTP is just a data protocol over TCP ... it can be implemented on any TCP stack. George
On 09.4.2018 &#1075;. 20:43, George Neuner wrote:
> On Fri, 6 Apr 2018 17:51:11 +0200, pozz <pozzugno@gmail.com> wrote: > >> I'm designing a electronic device running an embedded HTTP server. I'm >> using lwip TCP/IP stack and the httpd application that comes with lwip. >> >> Because the device is under an ADSL router and the public IP is dynamic, >> I need to register to a DDNS service, configure it in the router and >> open one port (NAT). These steps are too tricky for the most of my users. >> So I'm thinking to a simpler solution. > > You don't need any of that if the devices make outgoing connections.
Exactly, this is how I have implemented the "online support" on our netmca series (they can be set to call "home" every 20 seconds or so). The issue one might have is that there may be no universal listening application at the "home" side. In our case realVNC can be set to listen and this works fine. Users also get the ability to have their netmca call a set of addresses of their choice (and they can switch off the "support" call of course). Dimiter
On 09/04/18 22:36, Dave Nadler wrote:
> On Sunday, April 8, 2018 at 7:39:47 PM UTC-4, Clifford Heath wrote: >> You want UPnP, specifically >> <https://en.wikipedia.org/wiki/Universal_Plug_and_Play#NAT_traversal>. > > Very interesting, Thanks Clifford. > Some concerns: The article cited above says: > "UPnP is generally regarded as unsuitable for deployment in business settings..." > "... As a result, some UPnP devices ship with UPnP turned off by default as a security measure." > > Have you used UPnP successfully for commercial applications?
No, I have not needed to. But I know enough about networking to have been curious how things like Skype, multi-player games, and bit-torrent arrange inbound connections. If customers can't reconfigure their networks, they need to borrow a teenage boy for the afternoon :)
On Tue, 10 Apr 2018 07:50:48 +1000, Clifford Heath
<no.spam@please.net> wrote:

>On 09/04/18 22:36, Dave Nadler wrote: >> On Sunday, April 8, 2018 at 7:39:47 PM UTC-4, Clifford Heath wrote: >>> You want UPnP, specifically >>> <https://en.wikipedia.org/wiki/Universal_Plug_and_Play#NAT_traversal>. >> >> Very interesting, Thanks Clifford. >> Some concerns: The article cited above says: >> "UPnP is generally regarded as unsuitable for deployment in business settings..." >> "... As a result, some UPnP devices ship with UPnP turned off by default as a security measure." >> >> Have you used UPnP successfully for commercial applications? > >No, I have not needed to. But I know enough about networking >to have been curious how things like Skype, multi-player games, >and bit-torrent arrange inbound connections. > >If customers can't reconfigure their networks, they need to >borrow a teenage boy for the afternoon :)
In ye olden days, Skype classified machines into two groups. Machines which could accept an inbound connection, and ones which could not. The former were used as relay nodes for the latter. That's a bit oversimplified, as you actually could configure your Skype client to not permit the relaying). After MS took it over, they changed it so that all Skype clients relay through a set of central servers. Many multi-player games work the same way.
On 10/04/18 10:19, Robert Wessel wrote:
> On Tue, 10 Apr 2018 07:50:48 +1000, Clifford Heath > <no.spam@please.net> wrote: > >> On 09/04/18 22:36, Dave Nadler wrote: >>> On Sunday, April 8, 2018 at 7:39:47 PM UTC-4, Clifford Heath wrote: >>>> You want UPnP, specifically >>>> <https://en.wikipedia.org/wiki/Universal_Plug_and_Play#NAT_traversal>. >>> >>> Very interesting, Thanks Clifford. >>> Some concerns: The article cited above says: >>> "UPnP is generally regarded as unsuitable for deployment in business settings..." >>> "... As a result, some UPnP devices ship with UPnP turned off by default as a security measure." >>> >>> Have you used UPnP successfully for commercial applications? >> >> No, I have not needed to. But I know enough about networking >> to have been curious how things like Skype, multi-player games, >> and bit-torrent arrange inbound connections. >> >> If customers can't reconfigure their networks, they need to >> borrow a teenage boy for the afternoon :) > > In ye olden days, Skype classified machines into two groups. Machines > which could accept an inbound connection, and ones which could not. > The former were used as relay nodes for the latter.
Yes, but from very early on, it also supported hole-punching, where you're not supposed to be able to accept an incoming request, but with some forgery by a third-party server, many NAT firewalls could be fooled into thinking that you'd originated the request. I think it was that which made Skype special. I started a project to build an API service for gaming companies to do the same, but they're pretty keen to be middle-men, even though the gaming experience is worse. Same goes for Microsoft. Clifford Heath.
Il 09/04/2018 19:06, George Neuner ha scritto:
> On Mon, 9 Apr 2018 14:54:03 +0200, pozz <pozzugno@gmail.com> wrote: > >> Il 09/04/2018 14:36, Dave Nadler ha scritto: >>> On Sunday, April 8, 2018 at 7:39:47 PM UTC-4, Clifford Heath wrote: >>>> You want UPnP, specifically >>>> <https://en.wikipedia.org/wiki/Universal_Plug_and_Play#NAT_traversal>. >>> >>> Very interesting, Thanks Clifford. >>> Some concerns: The article cited above says: >>> "UPnP is generally regarded as unsuitable for deployment in business settings..." >>> "... As a result, some UPnP devices ship with UPnP turned off by default as a security measure." >>> >>> Have you used UPnP successfully for commercial applications? >> >> I made just some tests on a couple of home routers and both don't >> support NAT-PMP, that is a *different* protocol than UPnP/IGD. >> Both routers replied with ICMP Port Unreachable. > > Most commercial (business/enterprise) routers filter most ICMP traffic > - by default some won't even allow pings. Client applications are > expected to be using TCP or UDP.
I was refering to ICMP packets *created* by the router as the reply of my NAT-PMP request (UDP port 5351).
>> I think they support UPnP/IGD because I see something in their web >> pages. However it's not simple to test in a short time. >> >> Of course you can't expect all the routers support UPnP and that this >> functionality is enabled by the user. I think UPnP is enabled by >> default, if supported. But the user can disable it, mostly for security >> concerns. > > Yes. UPnP can be a major security hole. It's disabled by default in > most (all?) commercial units, and now even in some better SOHO units.
I think UPnP is *enabled* by default on most of home routers.