My little weather station setup - looking for improvement ideas

[Psyt0s]

Hey everyone! I know this is a german forum, hope its ok to write in english. If you prefer I can translate the text next time. Thought I'd share my current weather monitoring setup and see if anyone has suggestions for improvements.

Hardware:

Raspberry Pi Zero with a Weather HAT (temperature, humidity, pressure, light sensors)
Been running continuously since January 2024 and collecting data every minute

Data Collection:
The Pi runs a Python script that reads the sensors and handles a few things:

Collects sensor data
Calculates dew point and "feels like" temperature
Stores everything in a local PostgreSQL database

Data Storage & Sharing:

Local PostgreSQL database on the Pi
Set up replication to another PostgreSQL instance on a VM in my home-server
Also uploads readings to Weather Underground and OpenWeatherMap every minute.

Data Visualization:

Integration with Grafana and PostgreSQL as a data source.

Anyone running similar setups? Any suggestions for improvements or additional metrics worth tracking?
Also any other services i should integrate with?

[ProfMobilux]

We created a project very similar to your approach, but focus was sensor selection: https://wiki.loxberry.de/howtos_knowled ... tion/start

I would prefer to seperate the sensors from the data storage. Reading the sensors can be done easily with an ESP32 whch sends data to a MQTT Broker on a NAS or separate Raspberry (located protected inside). From the Broker to the database. The databse on the sdcard will destroy the card sooner or later. SDCards have limited write cycles. 100% Data loss will be the result. Use a SSD for storage.

As database I would use Influx instead of PostgreSQL. SQL isn‘t needed for storing timeseries data. Influx is perfect optimized for that. And you can use data aggregation for older data.

[Gyvate]

We created a project very similar to your approach, but focus was sensor selection: https://wiki.loxberry.de/howtos_knowled ... tion/start

I would prefer to seperate the sensors from the data storage. Reading the sensors can be done easily with an ESP32 whch sends data to a MQTT Broker on a NAS or separate Raspberry (located protected inside). From the Broker to the database. The databse on the sdcard will destroy the card sooner or later. SDCards have limited write cycles. 100% Data loss will be the result. Use a SSD for storage.

As database I would use Influx instead of PostgreSQL. SQL isn‘t needed for storing timeseries data. Influx is perfect optimized for that. And you can use data aggregation for older data.

only cheap SD cards will be destroyed by often repeated read/write processes.
Of course one can use a SSD, but a proper professional industrial SLC SD card will also do the job.
That's why e.g. the Meteobridge datalogger/weather server makes industrial SLC SD cards mandatory for use in the appliances or RaspberryPi installations !

[ProfMobilux]

only cheap SD cards will be destroyed by often repeated read/write processes.
Of course one can use a SSD, but a proper professional industrial SLC SD card will also do the job.

No, it doesn‘t. It‘s far better - right. But it cannot resist database applications. Beside the speed problem SDCards have with small write accesses. The only option is to use a SSD.

[Gyvate]

only cheap SD cards will be destroyed by often repeated read/write processes.
Of course one can use a SSD, but a proper professional industrial SLC SD card will also do the job.

No, it doesn‘t. It‘s far better - right. But it cannot resist database applications. Beside the speed problem SDCards have with small write accesses. The only option is to use a SSD.

Of course, you know it better than the experts who are using it and have tested it in different scenarios ....
Such apodictic statements don't really show expertise. Especially when practical examples show the opposite.

But, of course, a SSD can be used and may have a better I/O performance - but it's definitely NOT the only option. Maybe your only personal option. And if a high performance is needed in the first place or is just hardware overkill is still another question.
Plus, an average user may have difficulties to get a RPi boot from a SSD as it doesn't do it out of the box - not everyone is the top expert in this area.

I'm not saying that a SSD cannot be a good and feasible solution, but the only one ....??? Simply not true as many cases in practice show - and without significant performance issues.

[ProfMobilux]

Apperently you are not one of those experts. So you shouldn‘t write down your Google knowledge here and tell everyone that this is what experts say. You apperently do not know anything about Databases and their functionallity.

There‘s absolutely no need to boot a Pi from SSD. You may google for that, too.

[Gyvate]

I hold it with the Roman philosopher Boethius "o si tacuisses philosophus mansisses".
With my professional background I don't need Google nor a self-proclaimed database expert who tells me what works in this context and what doesn't ... We are not talking SAP HANA here, even though that's one of the fields of my expertise ...
Hence, I don't know anything about databases and DB applications as per you. Interesting perception.
Your apodictic wording and terms speak for themselves.
You want to know it better, so be it - I can live with that.
Discussion closed from my end

[ProfMobilux]

Discussion closed from my end

Thanks for that.

[icedeerreindeer]

I am running a similar setup software side (Node-RED/InfluxDB/Grafana). Additionally, I am pulling API data (open data) from the national official weather service and writing it also into InfluxDB via Node-RED for sensors that my personal weather station doesn't have (e.g. sunshine duration, cloud coverage, weather alerts...). Luckily this weather station is located very close to me. All this data tied together, results in a pretty complete weather dashboard in my opinion.