I'm setting up a household water-tank level sensor that I'd like to be able to monitor from (practically) anywhere to a mobile device or remote PCusing the above-mentioned ESP8266-based setup. The device is sampling at 5min intervals, each sample is a single integer range 0 - 230 (cm of water height). Also toggling a 'heartbeat' boolean variable to indicate it's still alive. Just getting stated with the ESP8266 and Arduino IoT Cloud via WiFi which was easy & convenient as it only took a few hours to get a solution up & running... However, the free Arduino IoT cloud is limited to 1 day of historical data (I'd like months to a year). It also hides the bulk of the cloud API (which is convenient) though I'm not sure yet how to handle recovering from WiFi outages etc. Internet searches uncover a ton of options many of which are far to up-market for my purpose. It also appears there's another route using a local server (eg RPi) that receives the data (via NodeRed or similar) but then to access outside the local WiFi network means exposing a web server or similar - sounds like an extra tier or development and complication/maintenance. Any advice would be helpful... -- Cheers, Chris.
Recommendation: Cheap (free!) IoT cloud service
Started by ●February 2, 2022
Reply by ●February 3, 20222022-02-03
Il 03/02/2022 03:38, Chris ha scritto:> I'm setting up a household water-tank level sensor that I'd like to be able to monitor from (practically) anywhere to a mobile device or remote PCusing the above-mentioned ESP8266-based setup. The device is sampling at 5min intervals, each sample is a single integer range 0 - 230 (cm of water height). Also toggling a 'heartbeat' boolean variable to indicate it's still alive. > > Just getting stated with the ESP8266 and Arduino IoT Cloud via WiFi which was easy & convenient as it only took a few hours to get a solution up & running... > > However, the free Arduino IoT cloud is limited to 1 day of historical data (I'd like months to a year). It also hides the bulk of the cloud API (which is convenient) though I'm not sure yet how to handle recovering from WiFi outages etc. > > Internet searches uncover a ton of options many of which are far to up-market for my purpose. > > It also appears there's another route using a local server (eg RPi) that receives the data (via NodeRed or similar) but then to access outside the local WiFi network means exposing a web server or similar - sounds like an extra tier or development and complication/maintenance. > > Any advice would be helpful...This is a big issue for our poor embedded developers. We are used to create "low-level" software that are near the hardware, but now the big part of the software is composed by "high-level": software that runs on 24h 7/7 connected servers (backend) and software that runs on powerful usually connected device, mostly mobile (smartphones, tables, desktop PCs). It's difficult to have in a single person all the skills needed to create a global IoT system, from the embedded devices to mobile frontends, passing by backend services. Moreover, if 90% of embedded systems are similar (for example, most of them run software written in the C language), high-level software can be created with a multitude of technologies and commercial/free services. If this isn't a commercial consumer product, I think I will end up with your last solution: a local web server exposed to the Internet. A stanrd ubiquitous web browser running on a desktop or mobile device can be used as the frontend. If you are creating a commercial product, you can't use a local web-server, you need a Cloud system and a modern branded mobile app. There are many technologies for this kind of systems, both for Cloud (backend) and for mobile apps. Anyway I don't think you can find all these skills in a single person, mainly if you want to create a reliable system.
Reply by ●February 3, 20222022-02-03
On 2022-02-03 Chris wrote in comp.arch.embedded:> I'm setting up a household water-tank level sensor that I'd like to be > able to monitor from (practically) anywhere to a mobile device or remote > PCusing the above-mentioned ESP8266-based setup. The device is sampling > at 5min intervals, each sample is a single integer range 0 - 230 (cm of > water height). Also toggling a 'heartbeat' boolean variable to indicate > it's still alive. > > Just getting stated with the ESP8266 and Arduino IoT Cloud via WiFi > which was easy & convenient as it only took a few hours to get a > solution up & running... > > However, the free Arduino IoT cloud is limited to 1 day of historical > data (I'd like months to a year). It also hides the bulk of the cloud > API (which is convenient) though I'm not sure yet how to handle > recovering from WiFi outages etc. > > Internet searches uncover a ton of options many of which are far to > up-market for my purpose.The free version of https://io.adafruit.com/ has 30 days of data storage. And it has options for an easy to build simple data display. Microsoft Azure has a free hub version: https://azure.microsoft.com/en-us/pricing/details/iot-hub/ I think this was called an S0 hub, but I don't see that name on the page. It is far more limited than the payed options and does not provide device provisioning. Not sure about the storage however. You may have to pay for a storage service, but with limited data, hot storage starts at $0.018/GB/month. You do have to provide a credit card number when signing up to Azure, so they can charge you as soon as you enable a paid service. Not sure how to get data out of the Azure cloud (for free), my limited Azure experience was mainly on the device side, sending data into the cloud. Amazon AWS only seems to provide time limited (12 months) free options. Google has some free options, but it is not clear to me you can do anything useful with only free stuff. -- Stef Almost nothing in Perl serves a single purpose. -- Larry Wall in <199712040054.QAA13811@wall.org>
Reply by ●February 3, 20222022-02-03
Chris <chris.hoffmann@gmail.com> wrote:> It also appears there's another route using a local server (eg RPi) that > receives the data (via NodeRed or similar) but then to access outside the > local WiFi network means exposing a web server or similar - sounds like an > extra tier or development and complication/maintenance.To add to the options mentioned, you can also get a small VPS for maybe $10-15 a year. You would of course need to configure it yourself, and there's no resiliency in case of outages. So not what I'd do for a commercial product (although I have seen fancy products with effectively this level of backend - and steered clear of them) but could be an option if this is a DIY project. It prevents all the local network hassles, but does mean you have a box on the internet that needs updating. Another option is a hosted Docker container - Google Cloud Run, Azure and IBM have free offerings, according to: https://www.reddit.com/r/docker/comments/c3urip/what_is_a_cheap_or_even_free_way_of_hosting_a/ Theo
Reply by ●February 3, 20222022-02-03
On 02/03/22 02:38, Chris wrote:> I'm setting up a household water-tank level sensor that I'd like to be able to monitor from (practically) anywhere to a mobile device or remote PCusing the above-mentioned ESP8266-based setup. The device is sampling at 5min intervals, each sample is a single integer range 0 - 230 (cm of water height). Also toggling a 'heartbeat' boolean variable to indicate it's still alive. > > Just getting stated with the ESP8266 and Arduino IoT Cloud via WiFi which was easy& convenient as it only took a few hours to get a solution up& running... > > However, the free Arduino IoT cloud is limited to 1 day of historical data (I'd like months to a year). It also hides the bulk of the cloud API (which is convenient) though I'm not sure yet how to handle recovering from WiFi outages etc. > > Internet searches uncover a ton of options many of which are far to up-market for my purpose. > > It also appears there's another route using a local server (eg RPi) that receives the data (via NodeRed or similar) but then to access outside the local WiFi network means exposing a web server or similar - sounds like an extra tier or development and complication/maintenance. > > Any advice would be helpful... > > -- > Cheers, > Chris.How about one of the 3G or 4G mobile data modules. Just call into it to get the results, which can come from an rpi or similar board with a serial or network interface. Only downside is the need for a sim card but not expensive for low usage or bandwidth. Or, a 3 or 4G router with network at one side could be made to work as well, where you could just access an internal host directly via the mobile network. Easy to secure as well... Chris
Reply by ●February 4, 20222022-02-04
On Friday, 4 February 2022 at 10:57:22 UTC+11, chris wrote:> On 02/03/22 02:38, Chris wrote: > > I'm setting up a household water-tank level sensor that I'd like to be able to monitor from (practically) anywhere to a mobile device or remote PCusing the above-mentioned ESP8266-based setup. The device is sampling at 5min intervals, each sample is a single integer range 0 - 230 (cm of water height). Also toggling a 'heartbeat' boolean variable to indicate it's still alive. > > > > Just getting stated with the ESP8266 and Arduino IoT Cloud via WiFi which was easy& convenient as it only took a few hours to get a solution up& running... > > > > However, the free Arduino IoT cloud is limited to 1 day of historical data (I'd like months to a year). It also hides the bulk of the cloud API (which is convenient) though I'm not sure yet how to handle recovering from WiFi outages etc. > > > > Internet searches uncover a ton of options many of which are far to up-market for my purpose. > > > > It also appears there's another route using a local server (eg RPi) that receives the data (via NodeRed or similar) but then to access outside the local WiFi network means exposing a web server or similar - sounds like an extra tier or development and complication/maintenance. > > > > Any advice would be helpful... > > > > -- > > Cheers, > > Chris. > How about one of the 3G or 4G mobile data modules. Just call into it to > get the results, which can come from an rpi or similar board with a > serial or network interface. Only downside is the need for a sim card > but not expensive for low usage or bandwidth. > > Or, a 3 or 4G router with network at one side could be made to work as > well, where you could just access an internal host directly via the > mobile network. Easy to secure as well... > > ChrisThanks to all for the tips. This is not a commercial product, just an initial foray into this space and hopefully a more convenient way of keeping tabs on my water level without having to resort to uncovering the manhole & using a dipstick. We recently build a deck over the tank as an outdoor area and even though I have a (screwed down) hatch in the deck over the manhole, it's an extra level of annoyance to get to it. I'll look into the Adafruit offering and possibly the Azure one depending on what it offers. Thanks for taking the time to reply. -- Cheers, Chris.
Reply by ●February 4, 20222022-02-04
Il 04/02/2022 00:57, chris ha scritto:> On 02/03/22 02:38, Chris wrote: >> I'm setting up a household water-tank level sensor that I'd like to be >> able to monitor from (practically) anywhere to a mobile device or >> remote PCusing the above-mentioned ESP8266-based setup. The device is >> sampling at 5min intervals, each sample is a single integer range 0 - >> 230 (cm of water height). Also toggling a 'heartbeat' boolean variable >> to indicate it's still alive. >> >> Just getting stated with the ESP8266 and Arduino IoT Cloud via WiFi >> which was easy& convenient as it only took a few hours to get a >> solution up& running... >> >> However, the free Arduino IoT cloud is limited to 1 day of historical >> data (I'd like months to a year). It also hides the bulk of the cloud >> API (which is convenient) though I'm not sure yet how to handle >> recovering from WiFi outages etc. >> >> Internet searches uncover a ton of options many of which are far to >> up-market for my purpose. >> >> It also appears there's another route using a local server (eg RPi) >> that receives the data (via NodeRed or similar) but then to access >> outside the local WiFi network means exposing a web server or similar >> - sounds like an extra tier or development and complication/maintenance. >> >> Any advice would be helpful... >> >> -- >> Cheers, >> Chris. > > > How about one of the 3G or 4G mobile data modules. Just call into it to > get the results, which can come from an rpi or similar board with a > serial or network interface. Only downside is the need for a sim card > but not expensive for low usage or bandwidth.What do you mean with "just call into it"?> Or, a 3 or 4G router with network at one side could be made to work as > well, where you could just access an internal host directly via the > mobile network. Easy to secure as well...In my experience, it's not always possible to connect to a server that is connected to a mobile 3G/4G network. Many mobile operators give connectivity that is guaranteed to work for "standard web browsing". Dynamic IP addresses, private IP addresses, firewall, NAT and similar "filters" are enemis of servers. This is the one of the reasons why IoT devices aren't server, but they usually run a client (HTTP, MQTT or others similar) that connects to a Cloud system.
Reply by ●February 4, 20222022-02-04
On 02/04/22 07:34, pozz wrote:> Il 04/02/2022 00:57, chris ha scritto: >> On 02/03/22 02:38, Chris wrote: >>> I'm setting up a household water-tank level sensor that I'd like to >>> be able to monitor from (practically) anywhere to a mobile device or >>> remote PCusing the above-mentioned ESP8266-based setup. The device is >>> sampling at 5min intervals, each sample is a single integer range 0 - >>> 230 (cm of water height). Also toggling a 'heartbeat' boolean >>> variable to indicate it's still alive. >>> >>> Just getting stated with the ESP8266 and Arduino IoT Cloud via WiFi >>> which was easy& convenient as it only took a few hours to get a >>> solution up& running... >>> >>> However, the free Arduino IoT cloud is limited to 1 day of historical >>> data (I'd like months to a year). It also hides the bulk of the cloud >>> API (which is convenient) though I'm not sure yet how to handle >>> recovering from WiFi outages etc. >>> >>> Internet searches uncover a ton of options many of which are far to >>> up-market for my purpose. >>> >>> It also appears there's another route using a local server (eg RPi) >>> that receives the data (via NodeRed or similar) but then to access >>> outside the local WiFi network means exposing a web server or similar >>> - sounds like an extra tier or development and complication/maintenance. >>> >>> Any advice would be helpful... >>> >>> -- >>> Cheers, >>> Chris. >> >> >> How about one of the 3G or 4G mobile data modules. Just call into it >> to get the results, which can come from an rpi or similar board with a >> serial or network interface. Only downside is the need for a sim card >> but not expensive for low usage or bandwidth. > > What do you mean with "just call into it"?I bought a Netgear 4G router for evaluation and also to use as a backup for the isp. That provides standard internet access just by fitting a sim card. Really cheap for low bandwidth applications and it's always on. Haven't tried it, but assume external access would be possible by opening an incoming port to the local network. IP address is dynamic, so you would need dynamic dns.org, whatever, but it should possible. On the local subnet, a couple of pages of C on an rpi, to open a file on the sd card and log tank values as needed. Login via ssh or run a web server to access the stored data. Advantage is that it's a self contained solution dependent only on a mobile account. Solutions like cloud etc really are overthinking what should be a simple problem to solve... Chris> >> Or, a 3 or 4G router with network at one side could be made to work as >> well, where you could just access an internal host directly via the >> mobile network. Easy to secure as well... > > In my experience, it's not always possible to connect to a server that > is connected to a mobile 3G/4G network. Many mobile operators give > connectivity that is guaranteed to work for "standard web browsing". > Dynamic IP addresses, private IP addresses, firewall, NAT and similar > "filters" are enemis of servers. > > This is the one of the reasons why IoT devices aren't server, but they > usually run a client (HTTP, MQTT or others similar) that connects to a > Cloud system.
Reply by ●February 4, 20222022-02-04
Il 04/02/2022 12:19, chris ha scritto:> On 02/04/22 07:34, pozz wrote: >> Il 04/02/2022 00:57, chris ha scritto: >>> On 02/03/22 02:38, Chris wrote: >>>> I'm setting up a household water-tank level sensor that I'd like to >>>> be able to monitor from (practically) anywhere to a mobile device or >>>> remote PCusing the above-mentioned ESP8266-based setup. The device is >>>> sampling at 5min intervals, each sample is a single integer range 0 - >>>> 230 (cm of water height). Also toggling a 'heartbeat' boolean >>>> variable to indicate it's still alive. >>>> >>>> Just getting stated with the ESP8266 and Arduino IoT Cloud via WiFi >>>> which was easy& convenient as it only took a few hours to get a >>>> solution up& running... >>>> >>>> However, the free Arduino IoT cloud is limited to 1 day of historical >>>> data (I'd like months to a year). It also hides the bulk of the cloud >>>> API (which is convenient) though I'm not sure yet how to handle >>>> recovering from WiFi outages etc. >>>> >>>> Internet searches uncover a ton of options many of which are far to >>>> up-market for my purpose. >>>> >>>> It also appears there's another route using a local server (eg RPi) >>>> that receives the data (via NodeRed or similar) but then to access >>>> outside the local WiFi network means exposing a web server or similar >>>> - sounds like an extra tier or development and >>>> complication/maintenance. >>>> >>>> Any advice would be helpful... >>>> >>>> -- >>>> Cheers, >>>> Chris. >>> >>> >>> How about one of the 3G or 4G mobile data modules. Just call into it >>> to get the results, which can come from an rpi or similar board with a >>> serial or network interface. Only downside is the need for a sim card >>> but not expensive for low usage or bandwidth. >> >> What do you mean with "just call into it"? > > I bought a Netgear 4G router for evaluation and also to use as a backup > for the isp. That provides standard internet access just by fitting a > sim card. Really cheap for low bandwidth applications and it's always > on. Haven't tried it, but assume external access would be possible by > opening an incoming port to the local network. IP address is dynamic, > so you would need dynamic dns.org, whatever, but it should possible. > On the local subnet, a couple of pages of C on an rpi, to open a file > on the sd card and log tank values as needed. Login via ssh or run a > web server to access the stored data. Advantage is that it's a self > contained solution dependent only on a mobile account.If it worked, I would agree with you. Howevere in this case you need a server behind the router (what you say "opening an incoming port to the local network"), you need NAT and so on. It could work, but it depends on your mobile operator. I know there are situations where this scenario is impossible, for example if the mobile operator gives your device a *private* IP address.> Solutions like cloud etc really are overthinking what should be a simple > problem to solve...Sure.> > Chris > >> >>> Or, a 3 or 4G router with network at one side could be made to work as >>> well, where you could just access an internal host directly via the >>> mobile network. Easy to secure as well... >> >> In my experience, it's not always possible to connect to a server that >> is connected to a mobile 3G/4G network. Many mobile operators give >> connectivity that is guaranteed to work for "standard web browsing". >> Dynamic IP addresses, private IP addresses, firewall, NAT and similar >> "filters" are enemis of servers. >> >> This is the one of the reasons why IoT devices aren't server, but they >> usually run a client (HTTP, MQTT or others similar) that connects to a >> Cloud system. >
Reply by ●February 4, 20222022-02-04
Chris <chris.hoffmann@gmail.com> wrote:> This is not a commercial product, just an initial foray into this space > and hopefully a more convenient way of keeping tabs on my water level > without having to resort to uncovering the manhole & using a dipstick. We > recently build a deck over the tank as an outdoor area and even though I > have a (screwed down) hatch in the deck over the manhole, it's an extra > level of annoyance to get to it.Depends how much this is a 'project' rather than an end goal, but Home Assistant will cover most of that. HA on a Pi to do the web front end / phone app / etc ESPHome on the ESP8266 to run the sensor node (just needs a config file if the sensor type is already supported). HA auto-detects the ESP8266 and fetches/logs the sensor data. Nabu Casa to provide the cloud connection so you can link back into your HA instance from outside your network. It also provides various cloud service interfacing that can't be done locally, eg Alexa and Google Assistant. The first two are free, Nabu Casa is $6.50 per month. That's not super cheap, but feeds back into funding Home Assistant. You are free to do your own port-forwarding/dynamic DNS/etc setup if you'd rather not pay for that. Theo
