REST API command/platform fails on POST to external URL (solcast)

Apologies for my ignorance, but I’ve installed this manually in custom components and seems ok.

But is there anyway of changing the schedule for it to pull tomorrows forecast? to say 7pm instead of midnight?

it would be useful to know what solar is going to look like tomorrow so can plan if I need to charge PV battery overnight. :slight_smile:

Thanks for you tremendous work @dannerph

My solcast integration includes a service call to push values to solcast API. My plan is to include an automatic pushing of values in the solcast integration, however I still need to figure out the best way to get the average of a certain period …

push_measurement:
  desciption: >
    Pushes PV measurements to Solcast for model fine tuning.
  fields:
    total_power:
      desciption: >
        The total power of the PV system for the given period.
      example: 1.23456
    period:
      desciption: >
        The period of the total power (e.g. PT5M, PT15M, PT30M, ...)
      example: PT5M
    period_end:
      desciption: >
        The end period of the total power in UTC timezone
      example: 2018-02-02T03:30:00.0000000Z

Good point, at the moment it is hard coded but you can create an automation and use the service call to force pull the current forecast (be aware that the forecast for today will only contain the remaining energy until midnight, the forecast of tomorrow and the day after tomorrow will be for the whole day starting from midnight local timezone).

The json response for the forecast API call only contains future time so I decided to fetch the forecast at midnight to include the whole day (might be more accurate at sunrise actually). I might introduce a parameter for that at next release.

update_forecast:
  description: >
    Fetches the forecasts from Solcast.

1 Like

Hello!

First of all I wanted to thank you for your work and to give it to everybody :slight_smile: I just received my solar panels a few days ago and started using home assistant for it, so I’m relatively new to all this.

I’ve been able to run your script and I can get the history data and forecast. However, there is 2 small issues:

  • I had to change line 112 by adding ssl=False otherwise I wasn’t able to connect
  • I can’t really use the forecast since I’m using the data in Grafana to display everything I need. Is there any way to get the forecast 30 minutes by 30 minutes as with the history data?

Thanks in advance!

Matthieu

@dannerph Just got this installed, also trying to do what @wills106 was saying, set my Tesla Powerwall to charge overnight (2am) if the next day will be a bad day for solar. The powerwall does sort of do this for you, but it doesn’t seem very accurate and over fills quite often.

Do you have an example of a service call for:

solcast.push_measurement

Or your automation you use to do this?

Hi @ParalaX,
nice to see people using my code, so it was a good investment of time :wink:

I had to change line 112 by adding ssl=False otherwise I wasn’t able to connect

I would avoid ssl=False, as it neglects the SSL certificate check and thus renders transport encryption broken. It might be related to your systems certificate store, maybe the root CA (from Amazon) is missing?

I can’t really use the forecast since I’m using the data in Grafana to display everything I need. Is there any way to get the forecast 30 minutes by 30 minutes as with the history data?

I need to check if I can also publish future events on the event bus and what the integrated recorder component, which is compatible with multiple backends, is doing with them. But i remember something that it was no possible that’s why I created another solution. I will check again when I find some time.

Hi @james_hiscott,

actually, I have not yet setup the pushing of measurements by myself (had still to fix reading from the smart meter, which is working now). In addition, I still need to figure out how to aggregate the measurement values on the time dimension (my meter is pushing on 2-3 seconds resolution) before I can forward them to solcast.

Hi,

Thanks a lot for your answer :slight_smile:

Regarding the SSL what do you mean by root CA from amazon? I didn’t know there was something to add. On the other end, I run Home assistant on a NAS, so maybe there is some stuff to do?

For the future event, if Hass doesn’t support future events, one possibility would be to put all events 3 days early, so you don’t have dates in the future, and make the change in the graph? You would however need to update the values as the predictions changes, I don’t know if it is possible? I’m sorry I’m no help here, I can program in C and C++ but know nothing about python :frowning:

Regards,

Matthieu

Hi @ParalaX,
there is no need to manually enter the root CA, it (or any cross signed certificate) should already be in you OS certificate store. I am running home assistant with the official docker image on a Synology DS918+ without issues.

Updating past values in home assistant is not possible as it is an event based system, so the older events a stored when the previous event was pushed an cannot referenced thus removed anymore. Home sssistant cannot show future values in a graph as far as I know, thus storing values to future times does not make sense as well. I am currentlich of adapting the idea of weatherforecast, where predictions are stored in attributes of the forecast sensor, maybe one sensor per forecast day with total energy sum an half hour prediction attributs if someone wants to calculate with more accurate data.

But from your initial request, I like the idea of setting the focus more on the forecast fetch interval. I might add an option in configuration for that to change the behaviour from once a day forecast and as much as possible past values to either equal fetching rate or once a day past values and forecast as much as possible (all by considering the free API limit).
BR
Philipp

About the future values: I don’t know about home assistant, but in my case (and I think I’m not the only one), I use Grafana to plot all the values I need. Doing so allows me to do what I want with my database, such as plotting in the future indeed, or for instance, calculate my self consumption. So I think having an option for this use case would be nice :slight_smile: Of course if it’s too much work I don’t want to bother you either ^^
Using Grafana and SQL commands, it is also very difficult to recover information from the attributes as there is limited options for parsing it.

If it is not possible to overwrite a value, maybe it is possible to set the time_created in the future (so the timestamp from Solcast), and the time_updated to the time where the prediction was taken? So it would be possible to always take the latest prediction?

Regards,

Matthieu

I am also running influxdb as long time datastore :wink:
However when I create a module for home assistant it should be compatible with the features in home assistant. I will check again if it is possible to add future events to the event queue which is processed by the recorder module. Of course one could write a new database connection that is handling this specific use case but it is always good practise to reuse existing components.
I suppose most of us are running automations with the home assistant automation plattform, thus values need to be provided as good as possible for this plattform.
I have not yet had a look on the influxdb module, but I suppose it is, similar to the recorder module, watching the global event queue and posting relevant values to influxdb when they occur. We need to to test both modules (recorder and influxdb) how they react on future values on the event bus

No probs @dannerph look forward to your updates when you have time. Will try to work out how to do it manually for now using the other methods posted above.

ps. thanks for your work on this :grinning:

Hi AussieByrd, I have the same type of setup – 2 sets of panels, the first facing north-east and the second north-west.

From what I read you started with two accounts (and set the Azimuth correctly for each set of panels), right? But when you ended up with one account – what did you set up the azimuth? The “center line” between the two individual ones?

Second, with regards to tuning, I can’t push back into the grid, so my system can’t run at 100% PV yield all the time. I get the most while charging batteries (I have 10.5 kWh batteries) and afterwards the MPPTs basically gives whatever my house is consuming. So I typically get graphs like this (batteries were fully charged around 13:15):

Would you recommend that I only feed data when batteries are charging, i.e. when I get maximum yield?

Hi Eben
When I switched to a single account I just set the azimuth according the where most of my panels aligned (i.e 16 out of my 22 point north). As for tuning, for it to be as representative as possible, I’d say you’ve already come up with the only viable solution–only upload data for periods when the battery is charging.

Good luck!

1 Like

Hi dannerph, thank you very much for the custom component!

Just in terms of doing a manual refresh – will it optimise the “smart fetching interval” afterwards and prevent API limit breach?

I have a PV system with batteries, but can’t feed back into the grid, so I have to try and optimise energy consumption to get the most out of my system. My plan is to use your component and Solcast to estimate how much PV energy will be available for the rest of the day once my batteries are fully charged, thereby deciding if I can run a hot-water system or pool pump or a bitcoin miner or something similar without going into my batteries too much :slight_smile: This can of course happen at different times during the day, hence the idea to manually refresh the sensor.

Hi AussieByrd,

Thanks once again - I am using your YAML to send back data to Solcast.

One thing I noticed is that the Solcast site says “Last measurement: 2 hours ago”, which makes me concerned that I messed up timezones.

AFAIK you need to provide UTC times, right? And your template sensor produces this correctly, since this was the value at 9:13 AM local time (GMT+2):

{"measurement": {"period_end": "2020-07-03T07:13:00Z", "period": "PT5M","total_power": 1443.62}}

However, on the Solcast website it shows:

More concerning is that the graph seems wrong:

Screenshot 2020-07-03 at 09.16.50

Am I missing something?

OK turns out I was missing something (I emailed Solcast’s support) – you have to submit it in KW, not W, so the 1.11 measurement was probably the first measurement picked up at 7:00 AM which was just 1.11W, but Solcast regarded it as 1.11kW. Then other measurements were rejected since it surpassed my system max of 3.8kW, and thus the last measurement was actually 2 hours ago at that point.

You can also view the uploaded values to Solcast here:

https://api.solcast.com.au/rooftop_sites/<resource_ID>/measurements

Hi ebendl,
this feature of reducing the manual fetches from the API count of the smart fetching is not yet implemented but a good idea! I will consider it, when I have some time to work in it again.

Glad you got it sorted @ebendl!

Regarding automating your uploads… if you have a sensor that indicates whether your battery in charging, you could probably just add a condition to the data upload automation to only upload if the battery is in the “charging” state.

1 Like

Yep, this is exactly what I did.

Automation:

#Run data upload every 5 minutes
- alias: solcast-upload
  trigger:
    platform: time_pattern
    minutes: "/5"
  condition:
    condition: and
    conditions:
      - condition: sun
        before: sunset
        before_offset: "+00:30:00"
      - condition: sun
        after: sunrise
        after_offset: "-00:30:00"
      - condition: state
        entity_id: sensor.victron_inverter_state
        state: 'Bulk'
        for:
          minutes: 5
  action:
    service: rest_command.solcast_tune

I had to change your sensor too to convert W to kW:

##5 minute average for production
  - platform: statistics
    entity_id: sensor.victron_pv_power
    name: pv_stats
    max_age:
      minutes: 5
      
##Template sensor for POST string
  - platform: template
    sensors:
      solcast_post:
        entity_id:
          - sensor.date_time_iso
          - sensor.pv_stats
        value_template: >
          {{ '{"measurement": {"period_end": "' ~ as_timestamp(states.sensor.date_time_iso.state)|timestamp_custom ('%Y-%m-%dT%H:%M:%SZ', false) ~ '", "period": "PT5M","total_power": ' ~ state_attr('sensor.pv_stats', 'mean')|float / 1000 ~ '}}' }}

But other than that it is working great – thanks! I also extracted a couple of days worth of history and manually formatted/inserted it into Solcast.

Question: how long does it take before tuning is “attempted”?