On Friday, April 6, 2018 at 11:51:17 AM UTC-4, pozz wrote:
> I'm designing a electronic device running an embedded HTTP server. I'm
> using lwip TCP/IP stack and the httpd application that comes with lwip.
>
> Because the device is under an ADSL router and the public IP is dynamic,
> I need to register to a DDNS service, configure it in the router and
> open one port (NAT). These steps are too tricky for the most of my users.
> So I'm thinking to a simpler solution.
>
> I know I can create a cloud on the Internet that runs a standard HTTP
> server to the users, and a custom "IOT" protocol to the remote devices.
> However this way is too difficult for me and I have to change almost
> everything that now works well.
>
> I'm thinking of a public HTTP tunnel that limits to forward the users
> HTTP requests to the HTTP server. Of course, the tunnel is made by two
> ends: one on the public server (and I could use everything, because I
> have to start from zero), the other on the embedded device (where I have
> many restrictions on memory size, computational power, no-OS and so on).
>
> Any experience on implementing a HTTP tunnel on lwip?
Pozz, what did you end up doing to solve your "hidden behind NAT" problem?
Reply by George Neuner●April 12, 20182018-04-12
My ISP was flaky/down for a bit ... catching up.
On Tue, 10 Apr 2018 09:55:40 +0200, pozz <pozzugno@gmail.com> wrote:
>Il 09/04/2018 19:43, George Neuner ha scritto:
>> On Fri, 6 Apr 2018 17:51:11 +0200, pozz <pozzugno@gmail.com> wrote:
>>
>>> I'm designing a electronic device running an embedded HTTP server. I'm
>>> using lwip TCP/IP stack and the httpd application that comes with lwip.
>>>
>>> Because the device is under an ADSL router and the public IP is dynamic,
>>> I need to register to a DDNS service, configure it in the router and
>>> open one port (NAT). These steps are too tricky for the most of my users.
>>> So I'm thinking to a simpler solution.
>>
>> You don't need any of that if the devices make outgoing connections.
>
>I know, but I would need a public server, a man-in-the-middle, a cloud
>server that I have to mantain, manage, support... this isn't free and is
>over my knowledge.
:
>Let me explain my idea. I have a HTTP server that runs behind a NAT. The
>user can connect to it after forwarding the port to the internal private
>IP address of the device and setting a DDNS account. Those two steps
>are too complex for "standard user".
>
>I know *the solution* is to have only *outgoing* connections and this
>means a public server. What I want to achieve is to have the simplest
>server I can. I don't need to save persistent data of remote device or
>other data from the users. I want to only allow the user connect to his
>remote device behind his router when he wants. Nothing more.
>
>My idea is to create a TCP tunnel between the devices (clients) and a
>public server. Over this "transparent" TCP tunnel I could transfer any
>data, such as HTTP requests/response.
>
>The end user will connect to a public "non standard" HTTP server. Every
>request from the browser will be forward to the correct tunnel (of
>course, I need a mechanism to link the user to the right tunnel).
>
>The device-side of the tunnel will forward any data to local(host) HTTP
>server in both directions.
>
>There are many solutions available and I think a similar solution
>already exists, for the public server and the tunnel protocol.
You want/need a public server, but (per above) you don't have the
skills or funds to maintain one. Hmmm.
I saw your other message re: public relay servers. Relay servers are a
forward proxy: they take an incoming call, make an outgoing call and
patch the two together.
What you seem to want is more kin to a load balancer: but one that
allows the "targets" (your devices) to call in rather than it calling
out to them.
I'm not aware of any OSS software that does that. I think you are
going to have to do some work on the server side. There certainly are
OSS proxy and load balancer programs that you could modify to do what
you want.
George
Reply by pozz●April 11, 20182018-04-11
Il 06/04/2018 17:51, pozz ha scritto:
> I'm designing a electronic device running an embedded HTTP server. I'm
> using lwip TCP/IP stack and the httpd application that comes with lwip.
>
> Because the device is under an ADSL router and the public IP is dynamic,
> I need to register to a DDNS service, configure it in the router and
> open one port (NAT). These steps are too tricky for the most of my users.
> So I'm thinking to a simpler solution.
>
> I know I can create a cloud on the Internet that runs a standard HTTP
> server to the users, and a custom "IOT" protocol to the remote devices.
> However this way is too difficult for me and I have to change almost
> everything that now works well.
>
> I'm thinking of a public HTTP tunnel that limits to forward the users
> HTTP requests to the HTTP server. Of course, the tunnel is made by two
> ends: one on the public server (and I could use everything, because I
> have to start from zero), the other on the embedded device (where I have
> many restrictions on memory size, computational power, no-OS and so on).
>
> Any experience on implementing a HTTP tunnel on lwip?
What about reverse SSH tunnel? I think it is what I need. A public
"relay" server runs a "standard" ssh server, so it will be very simple
to delivery.
On my device should be running a ssh client that creates, with an
outgoing connection, a permanent SSH tunnel with the "relay" server.
Now the end user can connect to the device by using the public "relay"
server.
I think it should work, even if I didn't test this configuration.
However the only problem is to implement a simple SSH client on lwip
device... I think it will be a very complex task.
Anyway I don't need encryption provided by SSH... maybe a simple telnet
tunnel should be sufficient. Is it possible?
Reply by Dave Nadler●April 10, 20182018-04-10
On Tuesday, April 10, 2018 at 8:33:47 AM UTC-4, Jacob Sparre Andersen wrote:
> It sounds like what you actually need is something like STUN, which
> helps establish a peer-to-peer connection between two applications
> (potentially) both being behind NAT gateways.
On Monday, April 9, 2018 at 5:50:55 PM UTC-4, Clifford Heath wrote:
> If customers can't reconfigure their networks, they need to
> borrow a teenage boy for the afternoon :)
I can't wait for the reaction from IT staff at fabs using our products ;-)
Reply by Jacob Sparre Andersen●April 10, 20182018-04-10
pozz <pozzugno@gmail.com> writes:
> Let me explain my idea. I have a HTTP server that runs behind a
> NAT. The user can connect to it after forwarding the port to the
> internal private IP address of the device and setting a DDNS account.
> Those two steps are too complex for "standard user".
>
> I know *the solution* is to have only *outgoing* connections and this
> means a public server. What I want to achieve is to have the simplest
> server I can. I don't need to save persistent data of remote device
> or other data from the users. I want to only allow the user connect
> to his remote device behind his router when he wants. Nothing more.
>
> My idea is to create a TCP tunnel between the devices (clients) and a
> public server. Over this "transparent" TCP tunnel I could transfer
> any data, such as HTTP requests/response.
>
> The end user will connect to a public "non standard" HTTP
> server. Every request from the browser will be forward to the correct
> tunnel (of course, I need a mechanism to link the user to the right
> tunnel).
>
> The device-side of the tunnel will forward any data to local(host)
> HTTP server in both directions.
>
> There are many solutions available and I think a similar solution
> already exists, for the public server and the tunnel protocol.
It sounds like what you actually need is something like STUN, which
helps establish a peer-to-peer connection between two applications
(potentially) both being behind NAT gateways. If I remember correctly,
STUN is intended for UDP traffic, so that means that you're going to
need a custom client for communicating with the devices.
Another option is to use UDP (which automatically punches a nice hole
for incoming traffic through a NAT gateway, if sent from from the
inside) between the devices and your cloud server. If the devices
regularly send your cloud server a keep-alive telegram, your cloud
server can forward data requests to the devices over UDP. - But this
isn't much of an improvement over having the device keeping a connection
based on some other protocol open to your cloud server.
I'm not fully versed in STUN, but you might be able to buy access to a
STUN server run by somebody else, so you don't have to worry about
server operations.
As some of my work involves writing server frameworks, and operating
servers, I consider a cloud server the easy solution. Using STUN, and
routing the actual data directly from the device to the client is the
proper, privacy-friendly solution.
Greetings,
Jacob
--
"Look! Ugly one-horned mule."
Reply by pozz●April 10, 20182018-04-10
Il 09/04/2018 19:43, George Neuner ha scritto:
> On Fri, 6 Apr 2018 17:51:11 +0200, pozz <pozzugno@gmail.com> wrote:
>
>> I'm designing a electronic device running an embedded HTTP server. I'm
>> using lwip TCP/IP stack and the httpd application that comes with lwip.
>>
>> Because the device is under an ADSL router and the public IP is dynamic,
>> I need to register to a DDNS service, configure it in the router and
>> open one port (NAT). These steps are too tricky for the most of my users.
>> So I'm thinking to a simpler solution.
>
> You don't need any of that if the devices make outgoing connections.
I know, but I would need a public server, a man-in-the-middle, a cloud
server that I have to mantain, manage, support... this isn't free and is
over my knowledge.
>> I know I can create a cloud on the Internet that runs a standard HTTP
>> server to the users, and a custom "IOT" protocol to the remote devices.
>> However this way is too difficult for me and I have to change almost
>> everything that now works well.
>
> HTTPS is the best choice for something that has to work in a security
> environment because (unless an IP is blocked) all routers will pass an
> outgoing HTTP(S) request. Other protocols tend to be limited or
> disabled and may require user/IT intervention to configure.
>
> But if your <whatever> is something they want, the customers certainly
> will arrange to pass your cloud traffic even if it's not HTTP.
MQTT is a popular choice for IoT and it can run over TLS.
As before, I would need a MQTT broker or a HTTP(S) server in the public
Internet.
>> I'm thinking of a public HTTP tunnel that limits to forward the users
>> HTTP requests to the HTTP server. Of course, the tunnel is made by two
>> ends: one on the public server (and I could use everything, because I
>> have to start from zero), the other on the embedded device (where I have
>> many restrictions on memory size, computational power, no-OS and so on).
>
> I'm assuming here that by "tunnel", you mean you need to pass binary
> data in the HTTP request/response.
>
> Assuming your server IP isn't blocked, your devices should be able to
> connect directly from behind a router. You shouldn't need any kind of
> tunneling proxy.
Let me explain my idea. I have a HTTP server that runs behind a NAT. The
user can connect to it after forwarding the port to the internal private
IP address of the device and setting a DDNS account. Those two steps
are too complex for "standard user".
I know *the solution* is to have only *outgoing* connections and this
means a public server. What I want to achieve is to have the simplest
server I can. I don't need to save persistent data of remote device or
other data from the users. I want to only allow the user connect to his
remote device behind his router when he wants. Nothing more.
My idea is to create a TCP tunnel between the devices (clients) and a
public server. Over this "transparent" TCP tunnel I could transfer any
data, such as HTTP requests/response.
The end user will connect to a public "non standard" HTTP server. Every
request from the browser will be forward to the correct tunnel (of
course, I need a mechanism to link the user to the right tunnel).
The device-side of the tunnel will forward any data to local(host) HTTP
server in both directions.
There are many solutions available and I think a similar solution
already exists, for the public server and the tunnel protocol.
> If you need to send a lot of data you may want to implement HTTP
> stream transfers rather than having to ROT64 (or whatever) to embed
> your binary data into the requests/responses.
>
>
>> Any experience on implementing a HTTP tunnel on lwip?
>
> Sorry - never used lwip. But HTTP is just a data protocol over TCP
> ... it can be implemented on any TCP stack.
>
>
> George
>
Reply by pozz●April 10, 20182018-04-10
Il 09/04/2018 19:06, George Neuner ha scritto:
> On Mon, 9 Apr 2018 14:54:03 +0200, pozz <pozzugno@gmail.com> wrote:
>
>> Il 09/04/2018 14:36, Dave Nadler ha scritto:
>>> On Sunday, April 8, 2018 at 7:39:47 PM UTC-4, Clifford Heath wrote:
>>>> You want UPnP, specifically
>>>> <https://en.wikipedia.org/wiki/Universal_Plug_and_Play#NAT_traversal>.
>>>
>>> Very interesting, Thanks Clifford.
>>> Some concerns: The article cited above says:
>>> "UPnP is generally regarded as unsuitable for deployment in business settings..."
>>> "... As a result, some UPnP devices ship with UPnP turned off by default as a security measure."
>>>
>>> Have you used UPnP successfully for commercial applications?
>>
>> I made just some tests on a couple of home routers and both don't
>> support NAT-PMP, that is a *different* protocol than UPnP/IGD.
>> Both routers replied with ICMP Port Unreachable.
>
> Most commercial (business/enterprise) routers filter most ICMP traffic
> - by default some won't even allow pings. Client applications are
> expected to be using TCP or UDP.
I was refering to ICMP packets *created* by the router as the reply of
my NAT-PMP request (UDP port 5351).
>> I think they support UPnP/IGD because I see something in their web
>> pages. However it's not simple to test in a short time.
>>
>> Of course you can't expect all the routers support UPnP and that this
>> functionality is enabled by the user. I think UPnP is enabled by
>> default, if supported. But the user can disable it, mostly for security
>> concerns.
>
> Yes. UPnP can be a major security hole. It's disabled by default in
> most (all?) commercial units, and now even in some better SOHO units.
I think UPnP is *enabled* by default on most of home routers.
Reply by Clifford Heath●April 10, 20182018-04-10
On 10/04/18 10:19, Robert Wessel wrote:
> On Tue, 10 Apr 2018 07:50:48 +1000, Clifford Heath
> <no.spam@please.net> wrote:
>
>> On 09/04/18 22:36, Dave Nadler wrote:
>>> On Sunday, April 8, 2018 at 7:39:47 PM UTC-4, Clifford Heath wrote:
>>>> You want UPnP, specifically
>>>> <https://en.wikipedia.org/wiki/Universal_Plug_and_Play#NAT_traversal>.
>>>
>>> Very interesting, Thanks Clifford.
>>> Some concerns: The article cited above says:
>>> "UPnP is generally regarded as unsuitable for deployment in business settings..."
>>> "... As a result, some UPnP devices ship with UPnP turned off by default as a security measure."
>>>
>>> Have you used UPnP successfully for commercial applications?
>>
>> No, I have not needed to. But I know enough about networking
>> to have been curious how things like Skype, multi-player games,
>> and bit-torrent arrange inbound connections.
>>
>> If customers can't reconfigure their networks, they need to
>> borrow a teenage boy for the afternoon :)
>
> In ye olden days, Skype classified machines into two groups. Machines
> which could accept an inbound connection, and ones which could not.
> The former were used as relay nodes for the latter.
Yes, but from very early on, it also supported hole-punching,
where you're not supposed to be able to accept an incoming
request, but with some forgery by a third-party server, many
NAT firewalls could be fooled into thinking that you'd originated
the request. I think it was that which made Skype special.
I started a project to build an API service for gaming companies
to do the same, but they're pretty keen to be middle-men, even
though the gaming experience is worse. Same goes for Microsoft.
Clifford Heath.
>On 09/04/18 22:36, Dave Nadler wrote:
>> On Sunday, April 8, 2018 at 7:39:47 PM UTC-4, Clifford Heath wrote:
>>> You want UPnP, specifically
>>> <https://en.wikipedia.org/wiki/Universal_Plug_and_Play#NAT_traversal>.
>>
>> Very interesting, Thanks Clifford.
>> Some concerns: The article cited above says:
>> "UPnP is generally regarded as unsuitable for deployment in business settings..."
>> "... As a result, some UPnP devices ship with UPnP turned off by default as a security measure."
>>
>> Have you used UPnP successfully for commercial applications?
>
>No, I have not needed to. But I know enough about networking
>to have been curious how things like Skype, multi-player games,
>and bit-torrent arrange inbound connections.
>
>If customers can't reconfigure their networks, they need to
>borrow a teenage boy for the afternoon :)
In ye olden days, Skype classified machines into two groups. Machines
which could accept an inbound connection, and ones which could not.
The former were used as relay nodes for the latter. That's a bit
oversimplified, as you actually could configure your Skype client to
not permit the relaying). After MS took it over, they changed it so
that all Skype clients relay through a set of central servers. Many
multi-player games work the same way.