Reply by Mikko OH2HVJ November 18, 20172017-11-18
pozz <pozzugno@gmail.com> writes:

> Now the problem is how to increase the security of communication over > Internet. This isn't a simple task. In my application, it means adding > HTTP authorization, strong encryption and so on. > Unfortunately lwip HTTP server doesn't support authentication (even > the basic method) and encryption, so it must be added. > > I think authentication and encryption is not a problem only with HTTP > applications, but for all communication protocols over public networks > (so MQTT, SMTP, SNMP and so on). > > I understood *the solution* is TLS/SSL (I think HTTPS means HTTP over > TLS). And TLS/SSL means strong encryption that needs some crypto hw > engine (that LPC1769 doesn't have).
We're using TLS with client side (and server side) certificates to do authentication (i.e. knowing who the other party is and create a secure channel). Keys can be kept on a separate TPM chip. Authorization (who is allowed to do what) is more complex issue, basically our server is allowed to do almost anything, except update firmware. Firmware is signed with separate key. We're running Linux on AM3352, though. I found references of mbedTLS being able to use client side certificates, but setting up the whole system may not be that easy. On Linux stunnel is a nice way to tunnel traffic if you don't want to do TLS connections from your application. Separate crypto engine is not necessarily needed, it just gives you more security (key storage) or performance (crypto functionality). -- mikko
Reply by David Brown November 17, 20172017-11-17
On 17/11/17 10:25, pozz wrote:
> Il 22/09/2017 20:27, David Brown ha scritto: >> On 22/09/17 17:52, pozz wrote: >>> There are many MCUs with embedded Ethernet (MAC). It seems most of >>> the open-source examples are lwip-based. >> >> Yes - LWIP is very popular, and works well. > > Other questions on Embedded Ethernet based on lwip stack solution. > > Actually I'm using a demo board with LPC1769 (with integrated MAC) and > LAN8720 (Ethernet PHY). I arranged a simple HTTP server and it works (I > started from example in LPCOpen libraries). > > Now the problem is how to increase the security of communication over > Internet. This isn't a simple task. In my application, it means adding > HTTP authorization, strong encryption and so on. > Unfortunately lwip HTTP server doesn't support authentication (even the > basic method) and encryption, so it must be added. > > I think authentication and encryption is not a problem only with HTTP > applications, but for all communication protocols over public networks > (so MQTT, SMTP, SNMP and so on). > > I understood *the solution* is TLS/SSL (I think HTTPS means HTTP over > TLS). And TLS/SSL means strong encryption that needs some crypto hw > engine (that LPC1769 doesn't have).
I recommend you read a bit about how these work (start with wikipedia). SSL is the older standard - TLS 1.0 was effectively SSL 3.0. But the names are often used interchangeably. TLS provides encryption, but only a limited form of authentication. You /can/ use TLS certificates for identification, but usually they are only used for the client to check that the server has a signed certificate that is valid for its DNS name. If you want usernames, passwords, etc., you need to do that in other ways - and TLS just means you can transfer them securely. TLS is big and complicated. It takes a good deal of code, and a good deal of processor resources. (It does not /need/ crypto hardware, but some TLS implementations can take advantage of it.) The implementations that I know of are either very big and flexible and under licenses such as the GPL, or efficient for small systems but have paid commercial licenses. I don't know if that is an issue for you.
> > I know I can build the system from low-level blocks (lwip, ssl, > http...), but it isn't simple for me. > > So the final question is: is there some Silicon vendors that show secure > communication examples on their demo boards with a ready-to-use example? > Or some examples from other sources that shows how to achieve what I want? > > Do you solve those problems in some way?
We have simply not used TLS/SSL. For some systems, the networking is within a closed local network - encryption and authentication is not an issue. Any remote access will be via gateways with VPN's and the like. For other systems, we do it independently of the connection. For example, one system downloads firmware updates from a server using normal http. The package itself is encrypted and digitally signed, so the receiving system can check that it is valid. That way there is no need to support the vast monstrosity of TLS and all its options and certificates - we can have one simple encryption and authentication routine in the embedded system. I don't remember off-hand what it is - probably AES.
Reply by pozz November 17, 20172017-11-17
Il 17/11/2017 11:36, Boudewijn Dijkstra ha scritto:
 >> [...]
>> And TLS/SSL means strong encryption that needs some crypto hw engine >> (that LPC1769 doesn't have). > > All it *needs* is extra memory. It *benefits from* hw-crypto.
So the problem without an hw crypto engine is only RAM space?
>> I know I can build the system from low-level blocks (lwip, ssl, >> http...), but it isn't simple for me. > > Well-written APIs and good documentation usually helps. But embedded > software is rarely simple.
Another good thing that helps is a ready-to-use example from MCU vendor that works on a demo board.
>> So the final question is: is there some Silicon vendors that show >> secure communication examples on their demo boards with a ready-to-use >> example? Or some examples from other sources that shows how to achieve >> what I want? > > Why silicon vendors?
MCU (Silicon) vendors usually sells demo boards to show all the feaures of its products. And usually give many ready-to-use examples that works without any change on the demo boards.
Reply by pozz November 17, 20172017-11-17
Il 17/11/2017 11:39, Theo Markettos ha scritto:
> pozz <pozzugno@gmail.com> wrote: >> So the final question is: is there some Silicon vendors that show secure >> communication examples on their demo boards with a ready-to-use example? >> Or some examples from other sources that shows how to achieve what I want? > > ARM owns/manages/encourages (or whatever, it's open source) mbedTLS, so > there are presumably examples for running it on a Cortex M.
I found a ready-to-use example[1] from ST. I don't think I have found other similar examples from MCU vendors. However I'll have to search better.
> I don't know whether that would be the right form factor for you, if you're > expecting to buy a module that 'does HTTPS', rather than placing a > microcontroller, MAC, PHY, etc yourself. > > There might be an example for a microcontroller with integrated MAC, which > would simplify the hardware.
I will use MCU with integrated MAC. Only the PHY (and RJ45) will be external. However the problem here isn't hw, but only sw (the hw issue is only related to a crypto engine useful in encryption used by TLS). [1] https://my.st.com/content/my_st_com/en/products/embedded-software/mcus-embedded-software/stm32-embedded-software/stm32-standard-peripheral-library-expansion/stsw-stm32057.license%3d1510913831055.product%3dSTSW-STM32057.html
Reply by Theo Markettos November 17, 20172017-11-17
pozz <pozzugno@gmail.com> wrote:
> So the final question is: is there some Silicon vendors that show secure > communication examples on their demo boards with a ready-to-use example? > Or some examples from other sources that shows how to achieve what I want?
ARM owns/manages/encourages (or whatever, it's open source) mbedTLS, so there are presumably examples for running it on a Cortex M. I don't know whether that would be the right form factor for you, if you're expecting to buy a module that 'does HTTPS', rather than placing a microcontroller, MAC, PHY, etc yourself. There might be an example for a microcontroller with integrated MAC, which would simplify the hardware. Theo
Reply by pozz November 17, 20172017-11-17
Il 22/09/2017 20:27, David Brown ha scritto:
> On 22/09/17 17:52, pozz wrote: >> There are many MCUs with embedded Ethernet (MAC). It seems most of the >> open-source examples are lwip-based. > > Yes - LWIP is very popular, and works well.
Other questions on Embedded Ethernet based on lwip stack solution. Actually I'm using a demo board with LPC1769 (with integrated MAC) and LAN8720 (Ethernet PHY). I arranged a simple HTTP server and it works (I started from example in LPCOpen libraries). Now the problem is how to increase the security of communication over Internet. This isn't a simple task. In my application, it means adding HTTP authorization, strong encryption and so on. Unfortunately lwip HTTP server doesn't support authentication (even the basic method) and encryption, so it must be added. I think authentication and encryption is not a problem only with HTTP applications, but for all communication protocols over public networks (so MQTT, SMTP, SNMP and so on). I understood *the solution* is TLS/SSL (I think HTTPS means HTTP over TLS). And TLS/SSL means strong encryption that needs some crypto hw engine (that LPC1769 doesn't have). I know I can build the system from low-level blocks (lwip, ssl, http...), but it isn't simple for me. So the final question is: is there some Silicon vendors that show secure communication examples on their demo boards with a ready-to-use example? Or some examples from other sources that shows how to achieve what I want? Do you solve those problems in some way?
Reply by David Brown November 13, 20172017-11-13
On 13/11/17 12:11, pozz wrote:
> Il 16/10/2017 23:45, David Brown ha scritto: > [...] >>> The other question. What about dynamic content? The example seems >>> show only static content. >>> >> >> I handled that with JSON files. So my static index.html file has >> (static) javascript, using jQuery, that regularly polls the board >> webserver for a file "data.json". This is generated dynamically >> (basically just a big snprintf statement with the required data >> added), and the javascript running on the client then puts the data in >> the right places in the html. > > I'm trying to do exactly this: jQuery, polling, JSON data requests. > However I'm a little confused how to manage dynamic content with lwip > and its httpd server. >
<snip>
> Any help?
Sorry - I wrote my own (simple and limited) httpd implementation.
Reply by pozz November 13, 20172017-11-13
Il 16/10/2017 23:45, David Brown ha scritto:
[...]
>> The other question. What about dynamic content? The example seems show >> only static content. >> > > I handled that with JSON files.&#4294967295; So my static index.html file has > (static) javascript, using jQuery, that regularly polls the board > webserver for a file "data.json".&#4294967295; This is generated dynamically > (basically just a big snprintf statement with the required data added), > and the javascript running on the client then puts the data in the right > places in the html.
I'm trying to do exactly this: jQuery, polling, JSON data requests. However I'm a little confused how to manage dynamic content with lwip and its httpd server. I think dynamic content can be generated in a simple way by setting LWIP_HTTPD_CUSTOM_FILES and defining the function fs_open_custom() that snprintf() to a static string and set file content to that string. int fs_open_custom(struct fs_file *file, const char *name) { if (!strcmp(name, "/data.json")) { static char json_data[1024]; unsigned int idx = 0; idx += snprintf(&json_data[idx], sizeof(json_data) - idx, "{"); idx += snprintf(&json_data[idx], sizeof(json_data) - idx, ...); idx += snprintf(&json_data[idx], sizeof(json_data) - idx, "}"); return 1; } return 0; } The only problem is I can't access to query string params in fs_open_custom(). For example, I could have "/data.json?array=1" or "/data.json?array=23" or similar things. Another problem is with CGI. After setting LWIP_HTTPD_CGI_SSI, I can define the CGI callback: void httpd_cgi_handler(const char* uri, int iNumParams, char **pcParam, char **pcValue) { if (!strcmp(uri, "/led.cgi")) { if ((iNumParams >= 1) && !strcmp(pcParam[0], "status")) { if (!strcmp(pcValue[0], "on") led_on(); if (!strcmp(pcValue[0], "off") led_off(); } } Here there's another problem. How to return dynamic data to a CGI request, for example a result code, maybe in JSON format? httpd_cgi_handler() doesn't return any data. I could leave "/led.cgi" outside the static files and manage it as a custom file. However fs_open_custom() is called *before* httpd_cgi_handler(), so it's impossible to return a status code that depends on the CGI processing result. Any help?
Reply by David Brown October 16, 20172017-10-16
On 16/10/17 17:54, pozz wrote:
> Il 16/10/2017 15:10, David Brown ha scritto: >> On 16/10/17 14:10, pozz wrote: >>> Il 22/09/2017 17:52, pozz ha scritto: >>>> There are many MCUs with embedded Ethernet (MAC). It seems most of the >>>> open-source examples are lwip-based. >>>> >>>> I have to design a board with one RS485 port and one Ethernet port. It >>>> could be seen as a serial-to-Ethernet converter, where the protocol on >>>> RS485 is proprietary. On Ethernet side I will contact a server (cloud) >>>> with a protocol that must be defined. >>>> >>>> At the moment, I don't need to implement a server, only a simple >>>> client. >>>> >>>> Do you suggest some hw/fw platforms that you think are better than >>>> others? >>> >>> Another question related to embedded Ethernet. >>> >>> If I implement a simple Web Server to remotely control the device from a >>> standard Web Browser, I need some non volatile memory space to store web >>> pages (html, css, images...). >>> >>> It's difficult to estimate the amount of data I will need for "web >>> server filesystem", however I think it's highly probable the internal >>> Flash memory of MCU will not be sufficient. >>> >> >> That depends entirely on the MCU in question, and the complexity of the >> web page.&#4294967295; I have no problem with webpages in the MCU I am using (512 KB >> flash). > > Thanks for sharing your experience. > > >> Note also that static data, such as images, javascript libraries, html >> files, can easily be taken from an independent webserver as long as the >> client has internet access - maybe you need nothing more than a "loader" >> index.html and dynamic json "files" on your internal webserver. > > Good suggestion. > > > I have other two doubts. > > RTOS (Free-RTOS) or not? I usually don't use RTOS and I never found it > is necessary in my applications. However TCP/IP, HTTP server and a > filesystem management could be tricky without an OS. > > I studied the contrib example httpserver_raw in lwip project (an HTTP > server without an OS). It uses a script (makefsdata) to create a virtual > filesystem as a C source code. Is it ok for simple web pages? >
An RTOS is not necessary for a web server. Use one if you think it makes things easier for you, or not if you don't. FreeRTOS is probably the most popular RTOS around, and is a good choice. You can use LWIP with or without FreeRTOS. In my most recent project with a webserver, I used LWIP but no RTOS. I used a similar principle to makefsdata, but with my own scripts for putting the static content files into C const arrays of data. I am quite happy making such scripts (in Python), so that suited me - but I am sure the LWIP demo and makefsdata will work fine too.
> > The other question. What about dynamic content? The example seems show > only static content. >
I handled that with JSON files. So my static index.html file has (static) javascript, using jQuery, that regularly polls the board webserver for a file "data.json". This is generated dynamically (basically just a big snprintf statement with the required data added), and the javascript running on the client then puts the data in the right places in the html.
Reply by pozz October 16, 20172017-10-16
Il 16/10/2017 15:10, David Brown ha scritto:
> On 16/10/17 14:10, pozz wrote: >> Il 22/09/2017 17:52, pozz ha scritto: >>> There are many MCUs with embedded Ethernet (MAC). It seems most of the >>> open-source examples are lwip-based. >>> >>> I have to design a board with one RS485 port and one Ethernet port. It >>> could be seen as a serial-to-Ethernet converter, where the protocol on >>> RS485 is proprietary. On Ethernet side I will contact a server (cloud) >>> with a protocol that must be defined. >>> >>> At the moment, I don't need to implement a server, only a simple client. >>> >>> Do you suggest some hw/fw platforms that you think are better than >>> others? >> >> Another question related to embedded Ethernet. >> >> If I implement a simple Web Server to remotely control the device from a >> standard Web Browser, I need some non volatile memory space to store web >> pages (html, css, images...). >> >> It's difficult to estimate the amount of data I will need for "web >> server filesystem", however I think it's highly probable the internal >> Flash memory of MCU will not be sufficient. >> > > That depends entirely on the MCU in question, and the complexity of the > web page. I have no problem with webpages in the MCU I am using (512 KB > flash).
Thanks for sharing your experience.
> Note also that static data, such as images, javascript libraries, html > files, can easily be taken from an independent webserver as long as the > client has internet access - maybe you need nothing more than a "loader" > index.html and dynamic json "files" on your internal webserver.
Good suggestion. I have other two doubts. RTOS (Free-RTOS) or not? I usually don't use RTOS and I never found it is necessary in my applications. However TCP/IP, HTTP server and a filesystem management could be tricky without an OS. I studied the contrib example httpserver_raw in lwip project (an HTTP server without an OS). It uses a script (makefsdata) to create a virtual filesystem as a C source code. Is it ok for simple web pages? The other question. What about dynamic content? The example seems show only static content.