The following makes sense in the context of this guide.
My configuration.yaml includes:
shell_command:
browserless_scraper: "bash ./scripts/browserless_scraper.sh {{function}} {{output}}"
multiscrape:
- name: Scraper McScrapeface
resource: "http://localhost:8123/local/browserless/my_output.html"
scan_interval: 0
timeout: 30
log_response: true
method: GET
sensor:
- name: Calories Remaining
unique_id: calories_remaining
select: "div > table > tbody > tr:nth-child(3) > td > table > tbody > tr > td:nth-child(2) > table > tbody > tr:nth-child(1) > td > table > tbody > tr:nth-child(2) > td > div > div > table > tbody > tr:nth-child(3) > td > table > tbody > tr > td > div"
My browserless_scraper.sh looks like:
#!/bin/sh
curl -X POST 'http://homeassistant.local:3000/function' --data-binary '@/config/js_scrapers/'$1 -H 'Content-Type: application/javascript' > /config/www/browserless/$2
My automation looks like:
alias: When triggered, update Calories remaining
description: ""
triggers: []
conditions: []
actions:
- data:
function: my_scraper.js
output: my_output.html
action: shell_command.browserless_scraper
alias: Scrapes HTML into /homeassistant/www/browserless/my_output.html
- delay:
hours: 0
minutes: 0
seconds: 2
milliseconds: 0
- data: {}
action: multiscrape.trigger_scraper_mcscrapeface
alias: >-
Reads my_output.html and updates sensor.calories_left in
/homeassistant/configuration.yaml
mode: single
My my_scraper.js looks like this (I use http://homeassistant.local:3000/debugger/ for running in-browser debugging):
export default async ({ page }) => {
await page.goto("https://my.loseit.com/login");
await page.type('input[id*="email"]', "YOUR EMAIL GOES HERE");
await page.type(
'input[id*="password"]',
"YOUR PASSWORD GOES HERE"
);
await Promise.all([
page.keyboard.press('Enter'),
page.waitForNavigation(),
]);
await Promise.all([
page.goto("https://www.loseit.com/#Home"),
page.waitForSelector("div > table > tbody > tr:nth-child(3) > td > table > tbody > tr > td:nth-child(2) > table > tbody > tr:nth-child(1) > td > table > tbody > tr:nth-child(2) > td > div > div > table > tbody > tr:nth-child(3) > td > table > tbody > tr > td > div"),
]);
const html = await page.content();
return html;
};