Plug&Play solution to reach an embedded HTTP server under a router

Il 09/04/2018 01:39, Clifford Heath ha scritto:
> On 07/04/18 01:51, pozz wrote:
>> I'm designing a electronic device running an embedded HTTP server. I'm 
>> using lwip TCP/IP stack and the httpd application that comes with lwip.
>>
>> Because the device is under an ADSL router and the public IP is 
>> dynamic, I need to register to a DDNS service, configure it in the 
>> router and open one port (NAT). These steps are too tricky for the 
>> most of my users.
>> So I'm thinking to a simpler solution.
>>
>> I know I can create a cloud on the Internet that runs a standard HTTP 
>> server to the users, and a custom "IOT" protocol to the remote 
>> devices. However this way is too difficult for me and I have to change 
>> almost everything that now works well.
>>
>> I'm thinking of a public HTTP tunnel that limits to forward the users 
>> HTTP requests to the HTTP server. Of course, the tunnel is made by two 
>> ends: one on the public server (and I could use everything, because I 
>> have to start from zero), the other on the embedded device (where I 
>> have many restrictions on memory size, computational power, no-OS and 
>> so on).
>>
>> Any experience on implementing a HTTP tunnel on lwip?
> 
> You want UPnP, specifically 
> <https://en.wikipedia.org/wiki/Universal_Plug_and_Play#NAT_traversal>.
> 
> This is supported by basically all domestic routers, to
> provide access from the Internet to e.g. gaming consoles.
> 
> It allows your device to find the router and ask it to
> open a port from the outside.

This is interesting, even if I understood there are many similar 
technologies: UPnP, NAT-PMP, Internet Gateway Device Protocol, PCP (Port 
Control Protocol).  I don't know which one is the right choice.  It also 
depends if those are implemented in popular home routers.

Anyway, even if the NAT traversal problem is solved, there's another 
issue: the *dynamic* public IP address that could be solved with a 
Dynamic DNS account.  Anyway free DDNS accounts require manual 
confirmation every 30 days and this could be a problem for the end user.

On Sunday, April 8, 2018 at 7:39:47 PM UTC-4, Clifford Heath wrote:
> You want UPnP, specifically 
> <https://en.wikipedia.org/wiki/Universal_Plug_and_Play#NAT_traversal>.

Very interesting, Thanks Clifford.
Some concerns: The article cited above says:
"UPnP is generally regarded as unsuitable for deployment in business settings..."
"... As a result, some UPnP devices ship with UPnP turned off by default as a security measure."

Have you used UPnP successfully for commercial applications?

Thanks!
Best Regards, Dave

Il 09/04/2018 14:36, Dave Nadler ha scritto:
> On Sunday, April 8, 2018 at 7:39:47 PM UTC-4, Clifford Heath wrote:
>> You want UPnP, specifically
>> <https://en.wikipedia.org/wiki/Universal_Plug_and_Play#NAT_traversal>.
> 
> Very interesting, Thanks Clifford.
> Some concerns: The article cited above says:
> "UPnP is generally regarded as unsuitable for deployment in business settings..."
> "... As a result, some UPnP devices ship with UPnP turned off by default as a security measure."
> 
> Have you used UPnP successfully for commercial applications?

I made just some tests on a couple of home routers and both don't 
support NAT-PMP, that is a *different* protocol than UPnP/IGD.
Both routers replied with ICMP Port Unreachable.

I think they support UPnP/IGD because I see something in their web 
pages. However it's not simple to test in a short time.

Of course you can't expect all the routers support UPnP and that this 
functionality is enabled by the user.  I think UPnP is enabled by 
default, if supported. But the user can disable it, mostly for security 
concerns.

On Mon, 9 Apr 2018 14:54:03 +0200, pozz <pozzugno@gmail.com> wrote:

>Il 09/04/2018 14:36, Dave Nadler ha scritto:
>> On Sunday, April 8, 2018 at 7:39:47 PM UTC-4, Clifford Heath wrote:
>>> You want UPnP, specifically
>>> <https://en.wikipedia.org/wiki/Universal_Plug_and_Play#NAT_traversal>.
>> 
>> Very interesting, Thanks Clifford.
>> Some concerns: The article cited above says:
>> "UPnP is generally regarded as unsuitable for deployment in business settings..."
>> "... As a result, some UPnP devices ship with UPnP turned off by default as a security measure."
>> 
>> Have you used UPnP successfully for commercial applications?
>
>I made just some tests on a couple of home routers and both don't 
>support NAT-PMP, that is a *different* protocol than UPnP/IGD.
>Both routers replied with ICMP Port Unreachable.

Most commercial (business/enterprise) routers filter most ICMP traffic
- by default some won't even allow pings.  Client applications are
expected to be using TCP or UDP.


>I think they support UPnP/IGD because I see something in their web 
>pages. However it's not simple to test in a short time.
>
>Of course you can't expect all the routers support UPnP and that this 
>functionality is enabled by the user.  I think UPnP is enabled by 
>default, if supported. But the user can disable it, mostly for security 
>concerns.

Yes.  UPnP can be a major security hole.  It's disabled by default in
most (all?) commercial units, and now even in some better SOHO units.

George

On Fri, 6 Apr 2018 17:51:11 +0200, pozz <pozzugno@gmail.com> wrote:

>I'm designing a electronic device running an embedded HTTP server. I'm 
>using lwip TCP/IP stack and the httpd application that comes with lwip.
>
>Because the device is under an ADSL router and the public IP is dynamic, 
>I need to register to a DDNS service, configure it in the router and 
>open one port (NAT). These steps are too tricky for the most of my users.
>So I'm thinking to a simpler solution.

You don't need any of that if the devices make outgoing connections.


>I know I can create a cloud on the Internet that runs a standard HTTP 
>server to the users, and a custom "IOT" protocol to the remote devices. 
>However this way is too difficult for me and I have to change almost 
>everything that now works well.

HTTPS is the best choice for something that has to work in a security
environment because (unless an IP is blocked) all routers will pass an
outgoing HTTP(S) request.  Other protocols tend to be limited or
disabled and may require user/IT intervention to configure.

But if your <whatever> is something they want, the customers certainly
will arrange to pass your cloud traffic even if it's not HTTP.
 

>I'm thinking of a public HTTP tunnel that limits to forward the users 
>HTTP requests to the HTTP server. Of course, the tunnel is made by two 
>ends: one on the public server (and I could use everything, because I 
>have to start from zero), the other on the embedded device (where I have 
>many restrictions on memory size, computational power, no-OS and so on).

I'm assuming here that by "tunnel", you mean you need to pass binary
data in the HTTP request/response.  

Assuming your server IP isn't blocked, your devices should be able to
connect directly from behind a router.  You shouldn't need any kind of
tunneling proxy.

If you need to send a lot of data you may want to implement HTTP
stream transfers rather than having to ROT64 (or whatever) to embed
your binary data into the requests/responses.  


>Any experience on implementing a HTTP tunnel on lwip?

Sorry - never used lwip.  But HTTP is just a data protocol over TCP
... it can be implemented on any TCP stack.


George

On 09.4.2018 г. 20:43, George Neuner wrote:
> On Fri, 6 Apr 2018 17:51:11 +0200, pozz <pozzugno@gmail.com> wrote:
>
>> I'm designing a electronic device running an embedded HTTP server. I'm
>> using lwip TCP/IP stack and the httpd application that comes with lwip.
>>
>> Because the device is under an ADSL router and the public IP is dynamic,
>> I need to register to a DDNS service, configure it in the router and
>> open one port (NAT). These steps are too tricky for the most of my users.
>> So I'm thinking to a simpler solution.
>
> You don't need any of that if the devices make outgoing connections.

Exactly, this is how I have implemented the "online support" on our
netmca series (they can be set to call "home" every 20 seconds or so).

The issue one might have is that there may be no universal listening
application at the "home" side. In our case realVNC can be set to listen
and this works fine. Users also get the ability to have their netmca
call a set of addresses of their choice (and they can switch off
the "support" call of course).

Dimiter

On 09/04/18 22:36, Dave Nadler wrote:
> On Sunday, April 8, 2018 at 7:39:47 PM UTC-4, Clifford Heath wrote:
>> You want UPnP, specifically
>> <https://en.wikipedia.org/wiki/Universal_Plug_and_Play#NAT_traversal>.
> 
> Very interesting, Thanks Clifford.
> Some concerns: The article cited above says:
> "UPnP is generally regarded as unsuitable for deployment in business settings..."
> "... As a result, some UPnP devices ship with UPnP turned off by default as a security measure."
> 
> Have you used UPnP successfully for commercial applications?

No, I have not needed to. But I know enough about networking
to have been curious how things like Skype, multi-player games,
and bit-torrent arrange inbound connections.

If customers can't reconfigure their networks, they need to
borrow a teenage boy for the afternoon :)

On Tue, 10 Apr 2018 07:50:48 +1000, Clifford Heath
<no.spam@please.net> wrote:

>On 09/04/18 22:36, Dave Nadler wrote:
>> On Sunday, April 8, 2018 at 7:39:47 PM UTC-4, Clifford Heath wrote:
>>> You want UPnP, specifically
>>> <https://en.wikipedia.org/wiki/Universal_Plug_and_Play#NAT_traversal>.
>> 
>> Very interesting, Thanks Clifford.
>> Some concerns: The article cited above says:
>> "UPnP is generally regarded as unsuitable for deployment in business settings..."
>> "... As a result, some UPnP devices ship with UPnP turned off by default as a security measure."
>> 
>> Have you used UPnP successfully for commercial applications?
>
>No, I have not needed to. But I know enough about networking
>to have been curious how things like Skype, multi-player games,
>and bit-torrent arrange inbound connections.
>
>If customers can't reconfigure their networks, they need to
>borrow a teenage boy for the afternoon :)

In ye olden days, Skype classified machines into two groups.  Machines
which could accept an inbound connection, and ones which could not.
The former were used as relay nodes for the latter.  That's a bit
oversimplified, as you actually could configure your Skype client to
not permit the relaying).  After MS took it over, they changed it so
that all Skype clients relay through a set of central servers.  Many
multi-player games work the same way.

On 10/04/18 10:19, Robert Wessel wrote:
> On Tue, 10 Apr 2018 07:50:48 +1000, Clifford Heath
> <no.spam@please.net> wrote:
> 
>> On 09/04/18 22:36, Dave Nadler wrote:
>>> On Sunday, April 8, 2018 at 7:39:47 PM UTC-4, Clifford Heath wrote:
>>>> You want UPnP, specifically
>>>> <https://en.wikipedia.org/wiki/Universal_Plug_and_Play#NAT_traversal>.
>>>
>>> Very interesting, Thanks Clifford.
>>> Some concerns: The article cited above says:
>>> "UPnP is generally regarded as unsuitable for deployment in business settings..."
>>> "... As a result, some UPnP devices ship with UPnP turned off by default as a security measure."
>>>
>>> Have you used UPnP successfully for commercial applications?
>>
>> No, I have not needed to. But I know enough about networking
>> to have been curious how things like Skype, multi-player games,
>> and bit-torrent arrange inbound connections.
>>
>> If customers can't reconfigure their networks, they need to
>> borrow a teenage boy for the afternoon :)
> 
> In ye olden days, Skype classified machines into two groups.  Machines
> which could accept an inbound connection, and ones which could not.
> The former were used as relay nodes for the latter.

Yes, but from very early on, it also supported hole-punching,
where you're not supposed to be able to accept an incoming
request, but with some forgery by a third-party server, many
NAT firewalls could be fooled into thinking that you'd originated
the request. I think it was that which made Skype special.

I started a project to build an API service for gaming companies
to do the same, but they're pretty keen to be middle-men, even
though the gaming experience is worse. Same goes for Microsoft.

Clifford Heath.

Il 09/04/2018 19:06, George Neuner ha scritto:
> On Mon, 9 Apr 2018 14:54:03 +0200, pozz <pozzugno@gmail.com> wrote:
> 
>> Il 09/04/2018 14:36, Dave Nadler ha scritto:
>>> On Sunday, April 8, 2018 at 7:39:47 PM UTC-4, Clifford Heath wrote:
>>>> You want UPnP, specifically
>>>> <https://en.wikipedia.org/wiki/Universal_Plug_and_Play#NAT_traversal>.
>>>
>>> Very interesting, Thanks Clifford.
>>> Some concerns: The article cited above says:
>>> "UPnP is generally regarded as unsuitable for deployment in business settings..."
>>> "... As a result, some UPnP devices ship with UPnP turned off by default as a security measure."
>>>
>>> Have you used UPnP successfully for commercial applications?
>>
>> I made just some tests on a couple of home routers and both don't
>> support NAT-PMP, that is a *different* protocol than UPnP/IGD.
>> Both routers replied with ICMP Port Unreachable.
> 
> Most commercial (business/enterprise) routers filter most ICMP traffic
> - by default some won't even allow pings.  Client applications are
> expected to be using TCP or UDP.

I was refering to ICMP packets *created* by the router as the reply of 
my NAT-PMP request (UDP port 5351).


>> I think they support UPnP/IGD because I see something in their web
>> pages. However it's not simple to test in a short time.
>>
>> Of course you can't expect all the routers support UPnP and that this
>> functionality is enabled by the user.  I think UPnP is enabled by
>> default, if supported. But the user can disable it, mostly for security
>> concerns.
> 
> Yes.  UPnP can be a major security hole.  It's disabled by default in
> most (all?) commercial units, and now even in some better SOHO units.

I think UPnP is *enabled* by default on most of home routers.