My Backyard Weather beta1

It has been over a year and I feel like I have a decently tested version to post. It has been a learning experience to be sure and I am already working on some changes for the official version 1 of this monstrosity. In the meantime, here is My Backyard Weather, beta 1.

Overview

While building a weather station has been a "back of the head" interest for me since I was a kid, it wasn't until a few years ago when I started thinking seriously about practical home automation that I revisited the idea. If you want to automate common tasks that are easily forgotten, tedious, or need to happen when no one is home, you need data. A weather station can give you loads of useful data about the outside environment that can drive decisions in your system like "should I water the lawn?" or "someone should shut the kitchen window, it just started raining." This project is one in a series of Home Automation projects I have in the works and this article is the first step to building a weather system for use by other systems.

What data does it provide?

As outlined in my project description, this station collects most of the basic data you would expect from a typical station and is outlined below. I decided that since "Air Quality" is a substantially different dataset, I would separate it into an independent project. I also originally thought that I would produce the calculated "feels like" (heat index, wind chill, etc. data on the station but after considering how intensive the calculations can be, I decided that the scripts that rendered the data to the user would be a better place for that work. This would reduce the compute cycles of the code on the station, reducing the likelihood of error.  

• Temperature - What is the current temperature?
• Humidity - What is relative humidity?
• Barometric Pressure - What is the current pressure in kPa?
• Wind Speed / Direction - How fast is the wind blowing in mph and from what direction(deg)?
• Rainfall (inches per interval) - How much has it rained in the last two minutes?
• Rainfall (Is it raining) - Is it raining or not right now?
• Lightning (counts per interval) - Is there lightning in the area and how intense is it?
• LUX - How bright is it outside?

Why reinvent the wheel? What did I gain?

There are hundreds of home brewed weather stations outlined on the Internet and dozens of well-made commercial stations available so why build one? Simply, none of them gave me control of the data in a way that I could depend on. This station reliably provides a dataset on a useful interval that is stored in a local database. I was able to choose what I wanted to measure against and while this station wasn't cheap, this system does have more options than most commercial systems of the same cost. Since this station doesn't rely on batteries, and is wired directly to my home network, I have been able to operate this station for more than 6 months uninterrupted with no errors, dropped packets or missing data.

Since the data is stored in a database, any number of scripts and applications can poke and prod the information for everything from "it's dark outside" to "based on the current conditions, the crawl vents should be closed." This data can also be used by the homeowner to find trends in energy consumption, irrigation patterns, and behavior to make decisions that can lower costs or improve the overall automation experience. 

Lastly, I control the code of the station which, allows me to adapt to changes in my home automation system and add new features as needed. For example, I recently settled on an open home automation platform called Home Assistant. I'll go into more detail about HA in a separate post but for brevity, HA allows a myriad of inputs to control thousands of devices. One supported type is MQTT which is a fast publisher/subscriber system for managing messages. I added MQTT publishing of my data to the posting code of the weather station in an afternoon and had all of my weather data available to HA in minutes. 

Methodology

I've been working on a station now for a little less than 4 years. The first year, though was mostly spent learning about the weather and what people were doing with stations. I built several test platforms that helped me understand various communications schemes, power types, database updating methods and so on. As it turns out, there are loads of ways to make a station and not all of them are equal. Along the way I learned that solar and directly powered devices are about as equally reliable but wireless networking was not at all as reliable as wired. Some of the devices I chose were cheap and reliable and some were not. Ultimately, the design I wound up with may not be the final version but it is working very well so far.

The platform is powered by an Arduino Mega. I chose this platform for simplicity since I preferred the dedicated nature of a MCU over a single board computer like a Raspberry Pi. I didn't want to have to troubleshoot the operating system in case I had a problem. 

As stated above, I found that wireless networking would regularly fail or require a restart from time to time. Plus, I wound up burning up a couple of wireless network cards for reasons I still don't understand. So, I opted for a wired approach which wasn't my preferred option. Since I had to run a cable anyway (and trying to find the "upside" of committing all of that sweat equity to burying a cable) I also opted to use power over ethernet injectors to power the device over the much more expensive solar power option.

The Mega collects data from the following devices and transmits it directly to a MySQL database as a single insert. I didn't use a real-time clock on the Mega, opting instead to allow the database insert a timestamp on update, again keeping things as simple as possible.

• DHT22 Temp and Humidity Sensor (https://www.adafruit.com/product/385)
  This sensor is nexpensive and accurate enough for my use. There are other options that are arguably more precise but, considering the conditions of my back yard, would not yield any significant improvement over this sensor. 
• SEN-08942 Rain Gauge / Anemometer / Wind Vane (https://www.sparkfun.com/products/8942)
  Inexpensive and accurate enough for my use although a larger, more robust tipping gauge would improve accuracy and longevity. I did make some modifications that helped accuracy but I expect to replace this device in a year or two. More on that below.
• RG-11 Rain Sensor (http://rainsensors.com/)
  Expensive but easily the most reliable rain sensor for the cost. I chose this sensor over a cheaper contact sensor because I wanted something that could detect mist quickly and didn't need to dry off before "resetting" to a "not raining" state. 
• MPL3115A2 - Barometric Pressure/Altitude/Temperature Sensor (https://www.adafruit.com/products/1893)
  This sensor is inexpensive and accurate enough for my use.
• TSL2561 - Luminosity Sensor (LUX) Sensor (https://www.adafruit.com/product/439)
  Inexpensive and accurate.
• MOD-1016 - Lightning Detector (http://www.embeddedadventures.com/as3935_lightning_sensor_module_mod-1016.html)
  Expensive and somewhat difficult to get configured but there are limited options for sensing lightning reliably. I've read articles about home-made circuits to detect lightning and may test out some of those in the future. This device has the potential to be very useful but configuring it has proven to be a pain. More on that below.

I opted to not custom build an enclosure as I simply didn't trust a porus 3D printed enclosure to prevent moisture problems. I bought standard outdoor enclosures as shown below. I also bought a standard solar shield that was properly tested. in the end, This build wasn't the cheapest way to go but in paying a bit more, I knew there were some things I didn't have to worry about. 

Parts List

Part	Qty	Notes
Arduino Mega 2560	1	The primary processor
DHT22	1	Temp and Humidity Sensor
SEN-08942	1	Rain Gauge / Anemometer / Wind Vane
RG-11	1	Optical Rain Sensor
MPL3115A2	1	Barometric Pressure/Altitude/Temperature Sensor
TSL2561	1	Luminosity Sensor (LUX) Sensor
MOD-1016	1	Lightning Detector
E989NNJL	1	Weatherproof PVC New Work/Old Work Junction Box
14252	1	1-Gang Gray Metal Weatherproof Standard Electrical Box
US-SA-AJD-198018	1	3.2"x4.3"x1.8" ABS Universal Project Enclosure w PC Transparent Cover
RHRS	1	Solar Radiation Shield
B00NRHNPUA	1	Power Over Ethernet injection kit
B019Q3U72M	1	12V DC power supply
LM7809CV	1	12VDC to 9VDC regulator
	1	330Ω resistor
	1	RGB LED
	2	RJ-45 Connectors
	3	RJ-11 Connectors
	1	6-port Keystone Jack wall plate
	1	Mega Proto Shield
		Ethernet Cable for data/power and for short runs for component connections

Modifications

Tipping Bucket Rain Gauge

Interesting things you'll learn about measuring rainfall when you build a weather station.

1. Cheap tipping buckets are well, cheap. Don't expect that they were well designed or accurate. My bucket had a too shallow lip around the edge of the collection area. I was getting errors of 25% - 40% depending on how hard it rained or how hard the wind was blowing. I printed a riser and it stabilized the error.
2. Don't trust the paperwork. When the specifications say that each tip of the bucket will be 0.011 inches of rain, the real world results will be something different. In my case, the more accurate reading is .0138. This may be due to internal resistance of the center bar as it turns over the plastic notches.
3. Bigger is better. When it stops raining there is always a certain amount of rainfall that isn't heavy enough to tip the bucket so that amount isn't counted and evaporates. Larger buckets have smaller amounts of rainfall per tip so are more accurate in very light rain and missed buckets at the end of a shower.

With changes 1 and 2 listed above, rainfall measurements through this gauge is around 90% what is in my manual gauge. I'll take that.

Lightning Detector

I've had a lot of trouble out of the lightning detector. First it wouldn't read at all and this was due to a known manufacturer defect. Embedded Adventures, the vendor for this particular breakout board replaced it and for a while it seemed to work using a code library for the AS3935 chip. Then there was an update for the code in the Arduino IDE that broke it. I found a library that was built by Embedded Adventures and it seemed to work but once I put it back in the box, it stopped registering events. I may have a wiring issue but I just haven't made troubleshooting it a priority. 

However, while it was working, I discovered that it can be very touchy and incredibly sensitive to noise. For a while I was getting a lot of false positives and after a bit of crude analysis, I determined that it was too close to the wiring of the anemometer and picked up the events it was sending. I then moved it out to a separatebox with the LUX sensor but haven't been able to see anything since...

More to come on that one.

Power and Heat

My rain detector requires a solid 12 volts to work properly. Period. 
If you've worked with wall warts at all, you have probably noticed by now that they hardly ever supply the rated voltage under load. In my case, a 12 volt supply I was using was actually supplying 8.5 volts under load. The Arduino and all of the other parts were happy with this result but my rain detector was not. It would trip and stay tripped until reset. So, I found one in my junk box that supplied 12.2volts underload reliably and used that instead. Problem solved, or so I thought - the mega would lockup - a lot.

In increasing the voltage I introduced a new problem - heat. The onboard voltage regulator on the mega was good for voltages ranging from 6-20v but at 12volts, the onboard regulator was burning off that excess energy in the form of heat - a lot of it. When combined with a sealed enclosure, the system would overheat and lockup. My solution was to get a 12v to 9v power regulator and secure it with heat transfer adhesive to the lid of an external metal enclosure. The rain detector had its 12 volts and the mega had the 9 volts that liked better. Heat problem solved. 

System Integration

One of the primary goals for building this station was an effort to use the data for home automation. When I started building, I had not yet settled on a home automation management system. The reason was simple; there isn't one that unites all of the devices out there although some get pretty close. I had seriously considered SmartThings and Wink but in both of those cases you must have an internet connection for those things to fully work. Plus, I'm tired of everything I do being used for marketing purposes and I didn't want to give anyone a new dataset to sell me stuff. So, I deferred. 

I read up on some locally managed (as in no internet connection required) systems and found that some were really promising but didn't check off all of the boxes like Open HAB. While being a very active and mature project, it was written in java and since I am learning Python, I wanted something I can use and contribute to. A friend at work referred me to Home Assistant and this one checked off all of the boxes - active community, python based, used open standards and had API integrations for a very large device base AND, not least of all, it worked even if my internet was down. 

One of the niceties here was that it has a plugin for a MQTT broker that was installed in minutes and ready to go. All I needed to do was add the publishing bits to the weather station data upload section. 

Currently, I am using light level data to automatically turn on the den lights when it gets dark enough outside. 

What's Next?

I'm going to let some of these changes bake in for a while and see how well it does in hotter weather. When dealing with an outdoor device, time seems to be the best tester but in the meantime there are some things I'm going to go ahead and change:

• Make a dedicated shield for version 1 - I've wanted to develop a custom PCB for a while now and this project would be a good one to start with. It doesn't require a lot of components and the traces are few. Mostly this shield will be a place to mount RJ connectors and an RGB LED. 
• Troubleshoot / Replace Lightning Detector - As stated, I've had a lot of trouble out of the lightning detector so, I'm going to give it an earnest attempt at troubleshooting and correction but if that doesn't work, I am considering replacing it. At minimum, I need to validate the library.
• Update MySQL Connector Library - I'm using an older version of the software used to communicate with my database. This needs to be updated.
• Create Weather Underground update script - The weather station wasn't intended to talk to the world. That's why I setup a server and database. So, I didn't design it with a realtime clock and other items that would make sending data to external repositories easier. I'm working on a python script to send the latest stored data from the station to weather underground via their API. 

After that, I'll consider it finished and ready to go.