Weather Service API feeds to HA Weather platform

Development
7

Found it!

It is in the components/sensor/ directory and is called wunderground.py

Hopefully I am smart enough (or learn enough) to use that as a starting point template to develop my own module that does the multiple data source gathering that I mentioned above.

8

Are you still trying to get NWS/NOAA data into HA? I have a working nws platform to the weather component (right now only for daily forecasts), but I’m at a bit of a crossroads with how to proceed. To make it fully compatible with the current state of the component requires a lot of choices and overcoming lots of incompatibilities that have stalled my efforts to integrate it into HA.

For example, the observation data coming from an NWS station contains quality control information. Which level of quality control should used? If the current observation is below this threshold, should a previous time point be used, or a nearby station? This one isn’t too hard.

Also, the NWS forecast is provided separately as a day and a night, yet in HA, it seems that you need to provide one forecast for both combined (although it supports having a low temperature). Right now, my platform outputs two forecasts per day, but it then it is a bit confusing in the frontend as it is reported as ‘Feb 7 6AM’ and ‘Feb 7 6PM’, or something like that. Which is day and which is night? If we conform to the typical weather platform usage, which forecast do we throw out, or how do we merge them?

Furthermore, the precipitation state in HA is in inches, whereas in the NWS data and as is typical in US forecasts, the data is in %. There are some conditions that cannot be mapped from NWS to HA, like “cold”.

If you’d like to help me refine this, we can collaborate on it. You can drop it in as a custom component. I can upload my version to Github at some point and link it here. If someone with frontend skills could help add features like precipitation_probability, it would be really helpful. I can’t make heads or tails of the frontend code, but the backend isn’t too bad for me.

1 Like
9

I would definitely enjoy working with you on that!
FYI, separate to HA I wrote a clumsy, but working python script to go into the NWS data and pull down the most recent forecast maps, then go to the National Hurricane Center and pull the Atlantic map. The script then calls ffmpeg and groups the maps into an mp4 video loop that I add to my media servers.

Even though I just did a “quick and dirty” hard code of the folder for my arrangement, the core process for getting the right morning vs evening file is coded into the script. It would be very straightforward for someone to clean it up to meet HA specification.

Are you getting the NWS forecast from their ftp server, by web scraping, or via some other way?

Regarding your % vs inches puzzle: they are not related in any way. The forecasts are based on the probability (or likelihood) that it will rain, not the expected amounts. For example, their 30% number means that over the entire forecast area there is about 1 chance in 3 that you will see rain in that whole area.
I have seen maps occasionally where they predict the likely inches of rainfall, but I have never seen a place where they do that regularly.

Full disclaimer, I am not great with frontend skills. I can muddle my way through, but not easily. My thought would be to grab the weatherunderground component from HA and see what would need to be moved or changed into our new code.

10

I am getting it from the web API. My code for that is here:

Is the ftp server a better way? I don’t see any reliable info on that service.

The problem is that HA does not support % in the weather component. The weather underground component with sensors itself may have supported it, but apparently HA no longer accepts this kind of arrangement for weather, only platforms for the default component. This means to fix these things, the default component must be fixed. See screen shot below, where you can see how it displays inches of rain. Currently my platform exposes the % number, but it can’t be displayed, hence my request for frontend skills.

This is what happens using the default day/night forecasts in HA using my platform (top) vs one that produces a single day forecast (dark sky). I would say the latter seems like the intended output. Of course again, if I knew how to change the frontend, I could suggest using Mon and Mon Night for example.

Note that HA also has horrible units handling (pressure on dark sky), so the weather platforms only work well in one unit system.

Screenshot_20190211-071537
Screenshot_20190211-0715371080×1920 183 KB

I’ll post up my ha platform soon.

11

Here it is:

https://github.com/MatthewFlamm/home-assistant/blob/nws/homeassistant/components/weather/nws.py

You can drop the nws.py file into /path-to-config/custom_components/weather/.

My configuration looks like this The only required entry should be userid. Right now they accept any random string but suggest an email address.

weather:
  - platform: nws
    userid: !secret email
    name: home
    station: !secret local_nws_station
12

Great, thanks for the info. You caused me to go back to the docs and brush up on Components. When doing that I also noticed that the use of sensors for weather is no longer the recommended path - components are. I need to dig deeper into the docs and see if I can learn how a “weather component” is supposed to be structured. If we are going to spend time on this, it might as well be conforming.

BTW, I am looking at Creating Components - developer

I have been working on the Openweathermap.py component. The bug there is simple, it is hard coded for a 3h rain number, but many stations report a 1h instead.

As for FTP vs API, no difference except that I could not find the map gifs available in the API. They actually come from National Center for Environmental Prediction. If there is an API for them, I couldn’t find it.

I did some digging and found that the new Lovelace (now default) front end has a reasonable way to set up custom cards. There is a LENGTHY discussion here Lovelace Custom Animated Weather Card. The basic code for the front end is in the file ha-weather-card.js. The steps are apparently:

  1. Place the weather-card.js and weather-card.css files in www\custom_ui\ .
  2. Save, the amcharts icon under www\icons\weather_icons\animated
  3. Add the references to your ui-lovelace.yaml .

I know this does not address how to merge the two forecasts, but I will look into that once I figure out how all these pieces fit together. In the meantime I’m dropping my Openweathermap stuff because getting the card right is the highest priority.

13

I already have the nws platform for the weather component setup and it conforms to the component as far as it allows (as far as I can tell).

That custom card you posted seems to have a very different structure then the files I can find on the polymer repository for weather. The one that needs to be modified is the internal HA weather card, but as I said, I can’t make any sense out of JavaScript or the structure of the front end.

14

I assume it is this one

https://github.com/home-assistant/home-assistant-polymer/blob/dev/src/cards/ha-weather-card.js

15

It is not something I am very good at, but I can see a few items that make sense:

  1. The first 240-ish lines are setting up the style sheet and building the card elements
  2. The function getUnit(measure) is where it picks up the unit basis from the basic config. This is a problem in my opinion because some parts of the world use mixed units. For example, many of the displays and stations here in the US will mix hPa pressure and inches of rain.
  3. Wind speed is the same. The units are not specifiable and you are stuck with whatever is in the getUnit(measure).
  4. Lines 149-236 look like regular HTML and would be where you would need to add the percent precipitation item to the display. The lines like showValue(stateObj.attributes.humidity) are where the data comes in from the component.

Sadly, I have never programmed with a DOM or stylesheet based dynamic web page. So modifying this card looks well beyond my current abilities. I will spend some time tomorrow seeing what I can find on Polymer and determine if any of that will help. Otherwise I will be stuck just using other examples and doing incremental changes.

I hope a “front end” polymer person sees this thread and offers some direction.

16

I have been watching this thread, no useful input as far as development is concerned.

But how do you express rainfall as a percentage? Percentage of what? Annual rainfall? Monthly rainfall?

17

It is a very common forecast in the US to have “50% chance of rain on Tuesday” for example. For NWS, they are clear that this is the probability of rain anywhere in the forecast area times the expected percentage of area affected.

In my experience, we almost never get inches of rain forecast unless it is noteworthy, e.g. 4 inches of rain, or it is wintery precipitation like snow or sleet.

This is what those in the US are used to getting in the forecast. So seeing 0.1 inches of rain is just as hard to understand for us, or maybe just me, because it isn’t part of our normal weather experience.

18

I think the percentage probability here in the US arose due to our size. For example, the Houston metro area covers roughly the same number of square miles/kilometers as Belgium. So to say it will rain 0.25 inches in Houston means nothing.

19

OK I didn’t realise it was a forecast not a record. That makes more sense now :slight_smile:

20

By the magic of copy paste, I’ve got precipitation probability working. Just need to address all the other points. I think we should request a day/night mode in the default component in addition to daily and hourly. Then, it would be as simple as setting a time of day attribute for each forecast, which is already available in the NWS data.

Screenshot_20190213-063445
Screenshot_20190213-0634451080×1920 114 KB

21

I now have a working version for the weather card. I need to look at the more-info card to see if that needs any tweaking as well.

Besides more-info card, I think the only other big TODO on my list is to decide on how to utilize the quality control info. My suggestion is to proceed as it is now with no QC check and no fall back to retrieve data from previous time point. We can always add this more detailed functionality later. Thoughts on this approach?

image

I commented about adding features to the weather component here, although maybe I should have put it in a separate issue.

https://github.com/home-assistant/architecture/issues/75

1 Like
22

Should have posted the update here. @txNgineer, see this link:

23

The web page I use to find various locations is at Weather.gov/MapClick with search

This starts with our area (Houston, Tx), but you can enter any location information in the search box. The formats for searching are detailed under the “View Location Examples” link below the search box.

Have you made any progress on the other timestamps since your posting in April?

24

Thanks. I’ll take a look.

If you mean in terms of modifying the built in weather card to show day/night, I did modify the card successfully, but then my frontend development environment broke with some version update. I can’t figure out how to get it working again to test it with the latest version.

I have a PR for a weather platform and someone else submitted a PR for a nws sensor. The other PR incorporated the metar code as a fallback, so I implemented this as well. Both PRs aren’t getting any attention however.

25

Oops, I didn’t realize you may have meant the api location information. If so, it is a two step process:

  1. Enter an api query (I just used my browser) to your desired location with Lat/Lon coordinates in decimal degrees. Here is an example link: https://api.weather.gov/points/29.62194,-95.49278
  2. It will show several links and a grid x and grid y. In the example that will be 59,89 Put those into a second link such as https://api.weather.gov/gridpoints/HGX/59,89/stations the result will show a list of stations nearest to the desired location.

From the example the nearest station is KSGR and it shows to have the API link of https://api.weather.gov/stations/KSGR.

Note also that the results from step 1 above will return links to both a daily and hourly forecast for the grid block. It looks like you can use the same link but just replace the “stations/KSGR” with either “gridpoints/HGX/59,89/forecast” or “gridpoints/HGX/59,89/forecast/hourly” to get the 7 day and hourly forecasts.

A good programmer could probably automate the above queries, but I didn’t see a need because my desire location(s) don’t change often.

26

Right now, it does the same process, but in a slightly different manner. You can specify no station, and it will find the closest one by lat/lon.

We need the ability to hard code a station as well, as some stations have really bad quality data, like the one by me which never has rainy days.

Since I made the PR, the NWS API changed to deprecate either the forecast or observation, I forget which now. So your method probably makes more sense as it isn’t deprecated. It just means I have to rewrite a lot of code :frowning:.