Forums

Plug&Play solution to reach an embedded HTTP server under a router

Started by pozz April 6, 2018
Il 09/04/2018 19:43, George Neuner ha scritto:
> On Fri, 6 Apr 2018 17:51:11 +0200, pozz <pozzugno@gmail.com> wrote: > >> I'm designing a electronic device running an embedded HTTP server. I'm >> using lwip TCP/IP stack and the httpd application that comes with lwip. >> >> Because the device is under an ADSL router and the public IP is dynamic, >> I need to register to a DDNS service, configure it in the router and >> open one port (NAT). These steps are too tricky for the most of my users. >> So I'm thinking to a simpler solution. > > You don't need any of that if the devices make outgoing connections.
I know, but I would need a public server, a man-in-the-middle, a cloud server that I have to mantain, manage, support... this isn't free and is over my knowledge.
>> I know I can create a cloud on the Internet that runs a standard HTTP >> server to the users, and a custom "IOT" protocol to the remote devices. >> However this way is too difficult for me and I have to change almost >> everything that now works well. > > HTTPS is the best choice for something that has to work in a security > environment because (unless an IP is blocked) all routers will pass an > outgoing HTTP(S) request. Other protocols tend to be limited or > disabled and may require user/IT intervention to configure. > > But if your <whatever> is something they want, the customers certainly > will arrange to pass your cloud traffic even if it's not HTTP.
MQTT is a popular choice for IoT and it can run over TLS. As before, I would need a MQTT broker or a HTTP(S) server in the public Internet.
>> I'm thinking of a public HTTP tunnel that limits to forward the users >> HTTP requests to the HTTP server. Of course, the tunnel is made by two >> ends: one on the public server (and I could use everything, because I >> have to start from zero), the other on the embedded device (where I have >> many restrictions on memory size, computational power, no-OS and so on). > > I'm assuming here that by "tunnel", you mean you need to pass binary > data in the HTTP request/response. > > Assuming your server IP isn't blocked, your devices should be able to > connect directly from behind a router. You shouldn't need any kind of > tunneling proxy.
Let me explain my idea. I have a HTTP server that runs behind a NAT. The user can connect to it after forwarding the port to the internal private IP address of the device and setting a DDNS account. Those two steps are too complex for "standard user". I know *the solution* is to have only *outgoing* connections and this means a public server. What I want to achieve is to have the simplest server I can. I don't need to save persistent data of remote device or other data from the users. I want to only allow the user connect to his remote device behind his router when he wants. Nothing more. My idea is to create a TCP tunnel between the devices (clients) and a public server. Over this "transparent" TCP tunnel I could transfer any data, such as HTTP requests/response. The end user will connect to a public "non standard" HTTP server. Every request from the browser will be forward to the correct tunnel (of course, I need a mechanism to link the user to the right tunnel). The device-side of the tunnel will forward any data to local(host) HTTP server in both directions. There are many solutions available and I think a similar solution already exists, for the public server and the tunnel protocol.
> If you need to send a lot of data you may want to implement HTTP > stream transfers rather than having to ROT64 (or whatever) to embed > your binary data into the requests/responses. > > >> Any experience on implementing a HTTP tunnel on lwip? > > Sorry - never used lwip. But HTTP is just a data protocol over TCP > ... it can be implemented on any TCP stack. > > > George >
pozz <pozzugno@gmail.com> writes:

> Let me explain my idea. I have a HTTP server that runs behind a > NAT. The user can connect to it after forwarding the port to the > internal private IP address of the device and setting a DDNS account. > Those two steps are too complex for "standard user". > > I know *the solution* is to have only *outgoing* connections and this > means a public server. What I want to achieve is to have the simplest > server I can. I don't need to save persistent data of remote device > or other data from the users. I want to only allow the user connect > to his remote device behind his router when he wants. Nothing more. > > My idea is to create a TCP tunnel between the devices (clients) and a > public server. Over this "transparent" TCP tunnel I could transfer > any data, such as HTTP requests/response. > > The end user will connect to a public "non standard" HTTP > server. Every request from the browser will be forward to the correct > tunnel (of course, I need a mechanism to link the user to the right > tunnel). > > The device-side of the tunnel will forward any data to local(host) > HTTP server in both directions. > > There are many solutions available and I think a similar solution > already exists, for the public server and the tunnel protocol.
It sounds like what you actually need is something like STUN, which helps establish a peer-to-peer connection between two applications (potentially) both being behind NAT gateways. If I remember correctly, STUN is intended for UDP traffic, so that means that you're going to need a custom client for communicating with the devices. Another option is to use UDP (which automatically punches a nice hole for incoming traffic through a NAT gateway, if sent from from the inside) between the devices and your cloud server. If the devices regularly send your cloud server a keep-alive telegram, your cloud server can forward data requests to the devices over UDP. - But this isn't much of an improvement over having the device keeping a connection based on some other protocol open to your cloud server. I'm not fully versed in STUN, but you might be able to buy access to a STUN server run by somebody else, so you don't have to worry about server operations. As some of my work involves writing server frameworks, and operating servers, I consider a cloud server the easy solution. Using STUN, and routing the actual data directly from the device to the client is the proper, privacy-friendly solution. Greetings, Jacob -- "Look! Ugly one-horned mule."
On Monday, April 9, 2018 at 5:50:55 PM UTC-4, Clifford Heath wrote:
> If customers can't reconfigure their networks, they need to > borrow a teenage boy for the afternoon :)
I can't wait for the reaction from IT staff at fabs using our products ;-)
On Tuesday, April 10, 2018 at 8:33:47 AM UTC-4, Jacob Sparre Andersen wrote:
> It sounds like what you actually need is something like STUN, which > helps establish a peer-to-peer connection between two applications > (potentially) both being behind NAT gateways.
Here's an excellent overview of the how-to-escape-NAT problem: https://www.ingate.com/files/Solving_Firewall-NAT_Traversal.pdf Shows pros/cons of several approaches discussed in this thread.
Il 06/04/2018 17:51, pozz ha scritto:
> I'm designing a electronic device running an embedded HTTP server. I'm > using lwip TCP/IP stack and the httpd application that comes with lwip. > > Because the device is under an ADSL router and the public IP is dynamic, > I need to register to a DDNS service, configure it in the router and > open one port (NAT). These steps are too tricky for the most of my users. > So I'm thinking to a simpler solution. > > I know I can create a cloud on the Internet that runs a standard HTTP > server to the users, and a custom "IOT" protocol to the remote devices. > However this way is too difficult for me and I have to change almost > everything that now works well. > > I'm thinking of a public HTTP tunnel that limits to forward the users > HTTP requests to the HTTP server. Of course, the tunnel is made by two > ends: one on the public server (and I could use everything, because I > have to start from zero), the other on the embedded device (where I have > many restrictions on memory size, computational power, no-OS and so on). > > Any experience on implementing a HTTP tunnel on lwip?
What about reverse SSH tunnel? I think it is what I need. A public "relay" server runs a "standard" ssh server, so it will be very simple to delivery. On my device should be running a ssh client that creates, with an outgoing connection, a permanent SSH tunnel with the "relay" server. Now the end user can connect to the device by using the public "relay" server. I think it should work, even if I didn't test this configuration. However the only problem is to implement a simple SSH client on lwip device... I think it will be a very complex task. Anyway I don't need encryption provided by SSH... maybe a simple telnet tunnel should be sufficient. Is it possible?
My ISP was flaky/down for a bit ... catching up.


On Tue, 10 Apr 2018 09:55:40 +0200, pozz <pozzugno@gmail.com> wrote:

>Il 09/04/2018 19:43, George Neuner ha scritto: >> On Fri, 6 Apr 2018 17:51:11 +0200, pozz <pozzugno@gmail.com> wrote: >> >>> I'm designing a electronic device running an embedded HTTP server. I'm >>> using lwip TCP/IP stack and the httpd application that comes with lwip. >>> >>> Because the device is under an ADSL router and the public IP is dynamic, >>> I need to register to a DDNS service, configure it in the router and >>> open one port (NAT). These steps are too tricky for the most of my users. >>> So I'm thinking to a simpler solution. >> >> You don't need any of that if the devices make outgoing connections. > >I know, but I would need a public server, a man-in-the-middle, a cloud >server that I have to mantain, manage, support... this isn't free and is >over my knowledge.
:
>Let me explain my idea. I have a HTTP server that runs behind a NAT. The >user can connect to it after forwarding the port to the internal private >IP address of the device and setting a DDNS account. Those two steps >are too complex for "standard user". > >I know *the solution* is to have only *outgoing* connections and this >means a public server. What I want to achieve is to have the simplest >server I can. I don't need to save persistent data of remote device or >other data from the users. I want to only allow the user connect to his >remote device behind his router when he wants. Nothing more. > >My idea is to create a TCP tunnel between the devices (clients) and a >public server. Over this "transparent" TCP tunnel I could transfer any >data, such as HTTP requests/response. > >The end user will connect to a public "non standard" HTTP server. Every >request from the browser will be forward to the correct tunnel (of >course, I need a mechanism to link the user to the right tunnel). > >The device-side of the tunnel will forward any data to local(host) HTTP >server in both directions. > >There are many solutions available and I think a similar solution >already exists, for the public server and the tunnel protocol.
You want/need a public server, but (per above) you don't have the skills or funds to maintain one. Hmmm. I saw your other message re: public relay servers. Relay servers are a forward proxy: they take an incoming call, make an outgoing call and patch the two together. What you seem to want is more kin to a load balancer: but one that allows the "targets" (your devices) to call in rather than it calling out to them. I'm not aware of any OSS software that does that. I think you are going to have to do some work on the server side. There certainly are OSS proxy and load balancer programs that you could modify to do what you want. George
On Friday, April 6, 2018 at 11:51:17 AM UTC-4, pozz wrote:
> I'm designing a electronic device running an embedded HTTP server. I'm > using lwip TCP/IP stack and the httpd application that comes with lwip. > > Because the device is under an ADSL router and the public IP is dynamic, > I need to register to a DDNS service, configure it in the router and > open one port (NAT). These steps are too tricky for the most of my users. > So I'm thinking to a simpler solution. > > I know I can create a cloud on the Internet that runs a standard HTTP > server to the users, and a custom "IOT" protocol to the remote devices. > However this way is too difficult for me and I have to change almost > everything that now works well. > > I'm thinking of a public HTTP tunnel that limits to forward the users > HTTP requests to the HTTP server. Of course, the tunnel is made by two > ends: one on the public server (and I could use everything, because I > have to start from zero), the other on the embedded device (where I have > many restrictions on memory size, computational power, no-OS and so on). > > Any experience on implementing a HTTP tunnel on lwip?
Pozz, what did you end up doing to solve your "hidden behind NAT" problem?