Scrape Sensor: "scrape: Error on device update!"

Configuration
1

First time user of the Scrape sensor, but when I try it out I got this error in the log:

Log Details (ERROR)
Wed Aug 29 2018 07:31:46 GMT+0200 (sentraleuropeisk sommertid)

scrape: Error on device update!
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/homeassistant/helpers/entity_platform.py", line 248, in _async_add_entity
    await entity.async_device_update(warning=False)
  File "/usr/local/lib/python3.6/site-packages/homeassistant/helpers/entity.py", line 322, in async_device_update
    yield from self.hass.async_add_job(self.update)
  File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.6/site-packages/homeassistant/components/sensor/scrape.py", line 120, in update
    value = raw_data.select(self._select)[0].text
IndexError: list index out of range

This is My config:

sensor:

  • platform: scrape
    name: jotta
    resource: https://www.jotta.no/jfs/myusername
    select: “.usage”
    unit_of_measurement: Byte
    authentication: basic
    username: myusername
    password: mypassword

This is the top of the xml I will scrape:

<user time=“2018-08-29-T05:13:42Z” host=“Backup2-backup2-get-oldgluster-dp1-5”>
<username>myusername</username>
<account-type>unlimited</account-type>
<locked>false</locked>
<capacity>-1</capacity>
<max-devices>-1</max-devices>
<max-mobile-devices>-1</max-mobile-devices>
<usage>11645076007480</usage> <–this
_<read-locked>fals_e</read-locked>
<write-locked>false</write-locked>
<quota-write-locked>false</quota-write-locked>
<enable-sync>true</enable-sync>
<enable-foldershare>true</enable-foldershare>
<devices>

This is my setup:

Hass.io supervisor
Version 127
Latest version 127
Host System
Hostname Home-Assistant
System Ubuntu 16.04.5 LTS
Home Assistant 0.76.2

2

Could you format your code using the </> button in the editor please? It makes it easier to see whether there are obvious syntax/indentation issues
image

Also without the full xml it’s a bit difficult to be 100% sure. Is it possible that there is another tag somewhere?
For your example you could try select: “usage:nth-of-type(1)”
replace (1) with the actual position of usage in the xml in case there’s more than 1

I personally struggled to make sense of most of my scrape sensors and went through many iterations before finding the ones that work.

3

well, looks like it is: https://pastebin.com/AWmgtZEb

Formated:

sensor:
  - platform: scrape
    name: jotta
    resource: https://www.jotta.no/jfs/myusername
    select: ".usage"
    unit_of_measurement: Byte
    authentication: basic
    username: myusername
    password: mypassword
4

thanks. Not sure if has an impact, but you’re getting data from XML file whereas scrape sensor appears set up for HTML pages. Not sure if this needs to be handled any differently.
Worth trialing a few different ways. Have you tried the suggestion above?

5

hmm maybe. And yes, I tested your suggestion

6

Sorry i’m not home so can’t run tests… Will give it a try later