The recent announcement of the end of WeatherUnderground’s API (due to their purchase by IBM) has spurred me to get involved.
I would like to develop a way to get the forecast and radar from the NOAA and NWS sites, local area data from a PWS network like Aeris, and then my local PWS from my own site. I also have a friend with a cabin in another part of the state and he wants to be able to see his data from there.
I have some developer expertise (ancient) and am trying to climb the knowledge hill to be able to contribute to HASS. Unfortunately I am not sure where to start the effort. Any tips would be helpful. I am very comfortable with how to get the weather and climate data we need from NWS/NOAA/NCDC. Just not where to “plug-in” to the HASS architecture.
I have studied the architecture documentation, created a local git repo, and set up a virtual environment for development. I then located the darksky.py weather component in the source. That is where I got confused. I see a number of test directories, files that are obviously not production related, etc. These weather files look like they are part of the automation component. Where are the core and entity files that tie to them?
As you can tell, I am an open source noob. Most of my prior development was for one-off industrial applications. Any help focusing my efforts will be appreciated.
Good clues nickrout, thanks. I guess I could just grab one of the scripts for another service (say openweathermap.py) and then modify it to fit data from the National Weather Service and then see what it does from there. It would just be a stab in the dark though.
Great suggestions. The init.py has a lot of decorators showing properties that are inherited by other modules. The bug list has some interesting information about modules that are giving problems. I will dig into both as well as keep looking at the darksky and openweather component modules.
Wish I could figure out who and DM a developer who works on the WeatherUnderground component that I want to replace.
It is in the components/sensor/ directory and is called wunderground.py
Hopefully I am smart enough (or learn enough) to use that as a starting point template to develop my own module that does the multiple data source gathering that I mentioned above.
Are you still trying to get NWS/NOAA data into HA? I have a working nws platform to the weather component (right now only for daily forecasts), but I’m at a bit of a crossroads with how to proceed. To make it fully compatible with the current state of the component requires a lot of choices and overcoming lots of incompatibilities that have stalled my efforts to integrate it into HA.
For example, the observation data coming from an NWS station contains quality control information. Which level of quality control should used? If the current observation is below this threshold, should a previous time point be used, or a nearby station? This one isn’t too hard.
Also, the NWS forecast is provided separately as a day and a night, yet in HA, it seems that you need to provide one forecast for both combined (although it supports having a low temperature). Right now, my platform outputs two forecasts per day, but it then it is a bit confusing in the frontend as it is reported as ‘Feb 7 6AM’ and ‘Feb 7 6PM’, or something like that. Which is day and which is night? If we conform to the typical weather platform usage, which forecast do we throw out, or how do we merge them?
Furthermore, the precipitation state in HA is in inches, whereas in the NWS data and as is typical in US forecasts, the data is in %. There are some conditions that cannot be mapped from NWS to HA, like “cold”.
If you’d like to help me refine this, we can collaborate on it. You can drop it in as a custom component. I can upload my version to Github at some point and link it here. If someone with frontend skills could help add features like precipitation_probability, it would be really helpful. I can’t make heads or tails of the frontend code, but the backend isn’t too bad for me.
I would definitely enjoy working with you on that!
FYI, separate to HA I wrote a clumsy, but working python script to go into the NWS data and pull down the most recent forecast maps, then go to the National Hurricane Center and pull the Atlantic map. The script then calls ffmpeg and groups the maps into an mp4 video loop that I add to my media servers.
Even though I just did a “quick and dirty” hard code of the folder for my arrangement, the core process for getting the right morning vs evening file is coded into the script. It would be very straightforward for someone to clean it up to meet HA specification.
Are you getting the NWS forecast from their ftp server, by web scraping, or via some other way?
Regarding your % vs inches puzzle: they are not related in any way. The forecasts are based on the probability (or likelihood) that it will rain, not the expected amounts. For example, their 30% number means that over the entire forecast area there is about 1 chance in 3 that you will see rain in that whole area.
I have seen maps occasionally where they predict the likely inches of rainfall, but I have never seen a place where they do that regularly.
Full disclaimer, I am not great with frontend skills. I can muddle my way through, but not easily. My thought would be to grab the weatherunderground component from HA and see what would need to be moved or changed into our new code.
I am getting it from the web API. My code for that is here:
Is the ftp server a better way? I don’t see any reliable info on that service.
The problem is that HA does not support % in the weather component. The weather underground component with sensors itself may have supported it, but apparently HA no longer accepts this kind of arrangement for weather, only platforms for the default component. This means to fix these things, the default component must be fixed. See screen shot below, where you can see how it displays inches of rain. Currently my platform exposes the % number, but it can’t be displayed, hence my request for frontend skills.
This is what happens using the default day/night forecasts in HA using my platform (top) vs one that produces a single day forecast (dark sky). I would say the latter seems like the intended output. Of course again, if I knew how to change the frontend, I could suggest using Mon and Mon Night for example.
Note that HA also has horrible units handling (pressure on dark sky), so the weather platforms only work well in one unit system.
Great, thanks for the info. You caused me to go back to the docs and brush up on Components. When doing that I also noticed that the use of sensors for weather is no longer the recommended path - components are. I need to dig deeper into the docs and see if I can learn how a “weather component” is supposed to be structured. If we are going to spend time on this, it might as well be conforming.
I have been working on the Openweathermap.py component. The bug there is simple, it is hard coded for a 3h rain number, but many stations report a 1h instead.
As for FTP vs API, no difference except that I could not find the map gifs available in the API. They actually come from National Center for Environmental Prediction. If there is an API for them, I couldn’t find it.
I did some digging and found that the new Lovelace (now default) front end has a reasonable way to set up custom cards. There is a LENGTHY discussion here Lovelace Custom Animated Weather Card. The basic code for the front end is in the file ha-weather-card.js. The steps are apparently:
Save, the amcharts icon under www\icons\weather_icons\animated
Add the references to your ui-lovelace.yaml .
I know this does not address how to merge the two forecasts, but I will look into that once I figure out how all these pieces fit together. In the meantime I’m dropping my Openweathermap stuff because getting the card right is the highest priority.
I already have the nws platform for the weather component setup and it conforms to the component as far as it allows (as far as I can tell).
That custom card you posted seems to have a very different structure then the files I can find on the polymer repository for weather. The one that needs to be modified is the internal HA weather card, but as I said, I can’t make any sense out of JavaScript or the structure of the front end.
It is not something I am very good at, but I can see a few items that make sense:
The first 240-ish lines are setting up the style sheet and building the card elements
The function getUnit(measure) is where it picks up the unit basis from the basic config. This is a problem in my opinion because some parts of the world use mixed units. For example, many of the displays and stations here in the US will mix hPa pressure and inches of rain.
Wind speed is the same. The units are not specifiable and you are stuck with whatever is in the getUnit(measure).
Lines 149-236 look like regular HTML and would be where you would need to add the percent precipitation item to the display. The lines like showValue(stateObj.attributes.humidity) are where the data comes in from the component.
Sadly, I have never programmed with a DOM or stylesheet based dynamic web page. So modifying this card looks well beyond my current abilities. I will spend some time tomorrow seeing what I can find on Polymer and determine if any of that will help. Otherwise I will be stuck just using other examples and doing incremental changes.
I hope a “front end” polymer person sees this thread and offers some direction.
It is a very common forecast in the US to have “50% chance of rain on Tuesday” for example. For NWS, they are clear that this is the probability of rain anywhere in the forecast area times the expected percentage of area affected.
In my experience, we almost never get inches of rain forecast unless it is noteworthy, e.g. 4 inches of rain, or it is wintery precipitation like snow or sleet.
This is what those in the US are used to getting in the forecast. So seeing 0.1 inches of rain is just as hard to understand for us, or maybe just me, because it isn’t part of our normal weather experience.
I think the percentage probability here in the US arose due to our size. For example, the Houston metro area covers roughly the same number of square miles/kilometers as Belgium. So to say it will rain 0.25 inches in Houston means nothing.
By the magic of copy paste, I’ve got precipitation probability working. Just need to address all the other points. I think we should request a day/night mode in the default component in addition to daily and hourly. Then, it would be as simple as setting a time of day attribute for each forecast, which is already available in the NWS data.
