Plug&Play solution to reach an embedded HTTP server under a router

I'm designing a electronic device running an embedded HTTP server. I'm 
using lwip TCP/IP stack and the httpd application that comes with lwip.

Because the device is under an ADSL router and the public IP is dynamic, 
I need to register to a DDNS service, configure it in the router and 
open one port (NAT). These steps are too tricky for the most of my users.
So I'm thinking to a simpler solution.

I know I can create a cloud on the Internet that runs a standard HTTP 
server to the users, and a custom "IOT" protocol to the remote devices. 
However this way is too difficult for me and I have to change almost 
everything that now works well.

I'm thinking of a public HTTP tunnel that limits to forward the users 
HTTP requests to the HTTP server. Of course, the tunnel is made by two 
ends: one on the public server (and I could use everything, because I 
have to start from zero), the other on the embedded device (where I have 
many restrictions on memory size, computational power, no-OS and so on).

Any experience on implementing a HTTP tunnel on lwip?

On Friday, April 6, 2018 at 11:51:17 AM UTC-4, pozz wrote:
> Any experience on implementing a HTTP tunnel on lwip?

Did you ask on the lwip forum?
Looks like I'll have the same issue for an upcoming product...

pozz <pozzugno@gmail.com> writes:

[...]

I can't help you with the lwip TCP/IP stack, but consider to let the
owner of the device have the freedom to configure if a cloud server
should be used and/or which cloud server to connect to.

It will give the product a longer potential lifetime (in case you for
some reason have to take your cloud server down), and allow more
paranoid customers to avoid passing their data through a third-party
service.

Greetings,

Jacob
-- 
"... while the C compiler will happily generate code for
 almost anything produced by leaning on the keyboard."

On Fri, 6 Apr 2018 17:51:11 +0200, pozz <pozzugno@gmail.com> wrote:

>I'm designing a electronic device running an embedded HTTP server. I'm 
>using lwip TCP/IP stack and the httpd application that comes with lwip.
>
>Because the device is under an ADSL router and the public IP is dynamic, 
>I need to register to a DDNS service, configure it in the router and 
>open one port (NAT). These steps are too tricky for the most of my users.
>So I'm thinking to a simpler solution.
>
>I know I can create a cloud on the Internet that runs a standard HTTP 
>server to the users, and a custom "IOT" protocol to the remote devices. 
>However this way is too difficult for me and I have to change almost 
>everything that now works well.
>
>I'm thinking of a public HTTP tunnel that limits to forward the users 
>HTTP requests to the HTTP server. Of course, the tunnel is made by two 
>ends: one on the public server (and I could use everything, because I 
>have to start from zero), the other on the embedded device (where I have 
>many restrictions on memory size, computational power, no-OS and so on).
>
>Any experience on implementing a HTTP tunnel on lwip?


Can you use something like SOCKS5 from the device?  Put the SOCKS
server on your public host.  You might be able to do that with some
implementations of SSH.

On 07.4.2018 г. 17:47, Dave Nadler wrote:
> On Friday, April 6, 2018 at 11:51:17 AM UTC-4, pozz wrote:
>> Any experience on implementing a HTTP tunnel on lwip?
>
> Did you ask on the lwip forum?
> Looks like I'll have the same issue for an upcoming product...
>

Don't know what lwip is but I did have this sort of issue 8-9
years ago when we introduced our netMCA series.
My problem was "how does the user know which IP address or host
name to enter in the VNC viewer and/or the browser" in order to
talk to the device. I think I posted here and got some suggestions,
none of which could work "out of the box" in an unknown environment
(i.e. the user might be using a wintel machine or a linux PC or
a phone etc.).

So I ended up leaving my original solution: when the netMCA boots
it does an ftp session and writes a text line (well, file) containing
its IP address. The file name contains the device serial number
which can be read on the box, so the user has to enter an URL
in their browser and see the IP address (which may be behind a router,
yet the user might be unable/unwilling to learn to see the router
dhcp list etc.).

Works OK - and I am not sure anybody uses it. I don;t think we had
a first time user manage that on their own, after all it takes
reading a paragraph in the manual...

So nowadays we ship a small router with the first netMCA for a new
user, the router being set to give fixed IP address(es) to the unit(s,
these being printed on a "quick start" paper. This generally works,
though typically new users still need some initial online
support.

Dimiter

======================================================
Dimiter Popoff, TGI             http://www.tgi-sci.com
======================================================
http://www.flickr.com/photos/didi_tgi/

Il 08/04/2018 06:27, Robert Wessel ha scritto:
 > On Fri, 6 Apr 2018 17:51:11 +0200, pozz <pozzugno@gmail.com> wrote:
 >
 >> I'm designing a electronic device running an embedded HTTP server. I'm
 >> using lwip TCP/IP stack and the httpd application that comes with lwip.
 >>
 >> Because the device is under an ADSL router and the public IP is dynamic,
 >> I need to register to a DDNS service, configure it in the router and
 >> open one port (NAT). These steps are too tricky for the most of my 
users.
 >> So I'm thinking to a simpler solution.
 >>
 >> I know I can create a cloud on the Internet that runs a standard HTTP
 >> server to the users, and a custom "IOT" protocol to the remote devices.
 >> However this way is too difficult for me and I have to change almost
 >> everything that now works well.
 >>
 >> I'm thinking of a public HTTP tunnel that limits to forward the users
 >> HTTP requests to the HTTP server. Of course, the tunnel is made by two
 >> ends: one on the public server (and I could use everything, because I
 >> have to start from zero), the other on the embedded device (where I have
 >> many restrictions on memory size, computational power, no-OS and so on).
 >>
 >> Any experience on implementing a HTTP tunnel on lwip?
 >
 > Can you use something like SOCKS5 from the device?  Put the SOCKS
 > server on your public host.  You might be able to do that with some
 > implementations of SSH.

It seems SOCKS is a "simple" TCP proxy between a client and a server, 
but the client stays client and server stays server.

In my application the user, through its Internet-connected smartphone, 
wants to connect to the embedded HTTP server behind a NAT. I think an 
intermediate SOCKS server doesn't help to reach the server behind the NAT.

Il 07/04/2018 17:20, Jacob Sparre Andersen ha scritto:
 > pozz <pozzugno@gmail.com> writes:
 >
 > [...]
 >
 > I can't help you with the lwip TCP/IP stack, but consider to let the
 > owner of the device have the freedom to configure if a cloud server
 > should be used and/or which cloud server to connect to.

Ok for allowing the user to choose between a direct user<->device 
connection and a "cloud" connected, but I think it's very difficult to 
allow choosing a "cloud server". Here there isn't a standard and the 
implementation is often proprietary. So the cloud server should be mine.


 > It will give the product a longer potential lifetime (in case you for
 > some reason have to take your cloud server down), and allow more
 > paranoid customers to avoid passing their data through a third-party
 > service.

I know, but nowadays users are more interested in the simplicity and 
"plug&play" functionalities instead of "paranoic" thinking.

On Sunday, April 8, 2018 at 4:48:46 PM UTC-4, pozz wrote:
> In my application the user, through its Internet-connected smartphone, 
> wants to connect to the embedded HTTP server behind a NAT. I think an 
> intermediate SOCKS server doesn't help to reach the server behind the NAT.

Last time I had to do this I used OpenVPN from the device to a publicly visible
server. The user connected to the other side of the tunnel on the public server.
This would be overkill for lwip and a small device...

Can't you similarly use SOCKS to initiate a bi-directional connection
to your publicly visible server???

On Sun, 8 Apr 2018 22:48:42 +0200, pozz <pozzugno@gmail.com> wrote:

>Il 08/04/2018 06:27, Robert Wessel ha scritto:
> > On Fri, 6 Apr 2018 17:51:11 +0200, pozz <pozzugno@gmail.com> wrote:
> >
> >> I'm designing a electronic device running an embedded HTTP server. I'm
> >> using lwip TCP/IP stack and the httpd application that comes with lwip.
> >>
> >> Because the device is under an ADSL router and the public IP is dynamic,
> >> I need to register to a DDNS service, configure it in the router and
> >> open one port (NAT). These steps are too tricky for the most of my 
>users.
> >> So I'm thinking to a simpler solution.
> >>
> >> I know I can create a cloud on the Internet that runs a standard HTTP
> >> server to the users, and a custom "IOT" protocol to the remote devices.
> >> However this way is too difficult for me and I have to change almost
> >> everything that now works well.
> >>
> >> I'm thinking of a public HTTP tunnel that limits to forward the users
> >> HTTP requests to the HTTP server. Of course, the tunnel is made by two
> >> ends: one on the public server (and I could use everything, because I
> >> have to start from zero), the other on the embedded device (where I have
> >> many restrictions on memory size, computational power, no-OS and so on).
> >>
> >> Any experience on implementing a HTTP tunnel on lwip?
> >
> > Can you use something like SOCKS5 from the device?  Put the SOCKS
> > server on your public host.  You might be able to do that with some
> > implementations of SSH.
>
>It seems SOCKS is a "simple" TCP proxy between a client and a server, 
>but the client stays client and server stays server.
>
>In my application the user, through its Internet-connected smartphone, 
>wants to connect to the embedded HTTP server behind a NAT. I think an 
>intermediate SOCKS server doesn't help to reach the server behind the NAT.


Sorry, my mistake - it's clearly been too many years since I looked at
it.  SOCKS5 does allow a "client" to establish a "bind" to a port on
the SOCKS server, but it's really only meant for use after an outgoing
connection is established to the other side - for example, it would
allow you to handle the data connection coming back from an FTP
server.

On 07/04/18 01:51, pozz wrote:
> I'm designing a electronic device running an embedded HTTP server. I'm 
> using lwip TCP/IP stack and the httpd application that comes with lwip.
> 
> Because the device is under an ADSL router and the public IP is dynamic, 
> I need to register to a DDNS service, configure it in the router and 
> open one port (NAT). These steps are too tricky for the most of my users.
> So I'm thinking to a simpler solution.
> 
> I know I can create a cloud on the Internet that runs a standard HTTP 
> server to the users, and a custom "IOT" protocol to the remote devices. 
> However this way is too difficult for me and I have to change almost 
> everything that now works well.
> 
> I'm thinking of a public HTTP tunnel that limits to forward the users 
> HTTP requests to the HTTP server. Of course, the tunnel is made by two 
> ends: one on the public server (and I could use everything, because I 
> have to start from zero), the other on the embedded device (where I have 
> many restrictions on memory size, computational power, no-OS and so on).
> 
> Any experience on implementing a HTTP tunnel on lwip?

You want UPnP, specifically 
<https://en.wikipedia.org/wiki/Universal_Plug_and_Play#NAT_traversal>.

This is supported by basically all domestic routers, to
provide access from the Internet to e.g. gaming consoles.

It allows your device to find the router and ask it to
open a port from the outside.

Clifford Heath