Using the Scrape integration with IP:PORT

Hi,

I’m trying to set up the Scrape integration to read the version number of my Unifi Controller (it’s on the login page)

The page’s IP is 192.168.1.11:8443

This is the code I’m using:

  - platform: scrape
    resource: https://192.168.1.11:8443
    select: "appNameAndVersion"

It’s not working, the sensor entity is not created and the log says that the platform “is not ready yet”.

However, if I use the example code:

  - platform: scrape
    resource: https://www.home-assistant.io
    select: ".current-version h1"

It’s working perfectly.

I think the problem is the port number in my IP.

Is there a way around this, maybe format it in some way?

Someone please help me with this :confused:

I am using the scrape sensor with ip:port without trouble.
I think your issue is your select statement
Can you post an extract of the html content of the page you’re trying to scrape?

Oh okay, this is the number I want to extract (5.12.35):

And this is the code, I didn’t really know what to use so I just tried it out.

Try the class for the div unifiAccount_appNameAndVersion.
if you can give the the HTML (not the image) we could help try with our scraper

<html lang="en" ng-controller="ManageController as manageCtrl" ng-strict-di=""><head><style type="text/css">@charset "UTF-8";[ng\:cloak],[ng-cloak],[data-ng-cloak],[x-ng-cloak],.ng-cloak,.x-ng-cloak,.ng-hide:not(.ng-hide-animate){display:none !important;}ng\:form{display:block;}.ng-animate-shim{visibility:hidden;}.ng-anchor{position:absolute;}</style><meta charset="utf-8"><title ng-bind="manageCtrl.system.getName() || manageCtrl.system.getHostname() || 'UniFi Network'">UniFi Network</title><meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no" unifi-prevent-focus-zoom=""><meta name="apple-itunes-app" content="app-id=1057750338"><base href="/manage/"><link rel="apple-touch-icon-precomposed" href="images/favicons/favicon-152.png?v=2"><meta name="msapplication-TileColor" content="#0193d7"><meta name="msapplication-TileImage" content="images/favicons/favicon-144.png?v=2"><link rel="apple-touch-icon-precomposed" sizes="152x152" href="images/favicons/favicon-152.png?v=2"><link rel="apple-touch-icon-precomposed" sizes="144x144" href="images/favicons/favicon-144.png?v=2"><link rel="apple-touch-icon-precomposed" sizes="120x120" href="images/favicons/favicon-120.png?v=2"><link rel="apple-touch-icon-precomposed" sizes="72x72" href="images/favicons/favicon-72.png?v=2"><link rel="apple-touch-icon-precomposed" href="images/favicons/favicon-57.png?v=2"><link rel="icon" href="images/favicons/favicon-32.png?v=2" sizes="32x32"><script src="config/config.v5.12.35.0.js"></script><script type="text/javascript">window.unifiConfig.version = '5.12.35.0';
</script><script src="js/initial.v5.12.35.0.js"></script><script src="js/components.v5.12.35.0.js" defer=""></script><script src="js/base.v5.12.35.0.js" defer=""></script><script src="js/main.manage.v5.12.35.0.js" defer=""></script><style>.whiteout{position:fixed;top:0;right:0;bottom:0;left:0;z-index:30;display:flex;flex-direction:column;justify-content:center;align-items:center}.type-word-break{word-break:break-all}[ng-cloak]{display:none}.app{visibility:hidden}*,*:before,*:after{-webkit-box-sizing:border-box;box-sizing:border-box}html,body{font-size:14px !important}body{font-family:"Lato", "Helvetica Neue", "Helvetica", Helvetica, Arial, sans-serif;font-weight:400;height:100%;background:#ffffff;color:#000000;line-height:1}div{display:block}.mt-3{margin-top:12px !important}.mb-3{margin-bottom:12px !important}.stackCenter{display:flex;flex-direction:column;align-items:center}.ubntStatusBar{background-color:#d6d9e2;display:inline-block;height:2px;width:100%}.ubntStatusBar--success{--status-bar-color: #39cc64}.ubntStatusBar__status{background-color:#006fff;display:block;height:100%}.appSpinner.appSpinner--medium{background-size:cover;width:3.5714285714rem;height:3.5714285714rem}
</style><link href="fonts/5.12.35.0/ubnt-icon/style.css" rel="stylesheet" media="all" onload="if(media!=='all') media='all'"><link href="css/styles.bundle.v5.12.35.0.css" rel="stylesheet" media="all" onload="if(media!=='all') media='all'"><link href="css/app.v5.12.35.0.css" rel="stylesheet" media="all" onload="if(media!=='all') media='all'"><link href="https://chrome.google.com/webstore/detail/hmpigflbjeapnknladcfphgkemopofig" rel="chrome-webstore-item"><link href="fonts/5.12.35.0/aura/fonts.css" rel="stylesheet" media="all" onload="if(media!=='all') media='all'"><script src="js/dynamic.settings.v5.12.35.0.js" async=""></script><script src="js/dynamic.dpi.v5.12.35.0.js" async=""></script><script src="js/dynamic.maps.v5.12.35.0.js" async=""></script><style id="angular-vs-repeat-style">
	  	.vs-repeat-debug-element {
        top: 50%;
        left: 0;
        right: 0;
        height: 1px;
        background: red;
        z-index: 99999999;
        box-shadow: 0 0 20px red;
      }

      .vs-repeat-debug-element + .vs-repeat-debug-element {
        display: none;
      }

      .vs-repeat-before-content,
      .vs-repeat-after-content {
        border: none !important;
        padding: 0 !important;
      }
    </style></head><body aura-theme="" class="ubnt-mod-dark"><svg class="pos-absolute" height="0" width="0"><defs><linearGradient id="experienceGraphGradient" x1="0" y1="0" x2="0" y2="100%" patternUnits="userSpaceOnUse"><stop offset="0%" stop-color="#0046cc" stop-opacity="0.3"></stop><stop offset="100%" stop-color="#ffffff" stop-opacity="0"></stop></linearGradient></defs></svg><route-status route-status-fatal-state="manageCtrl.ENV.FATAL_STATE" route-status-immediate="manageCtrl.site.isSiteChanging"></route-status><div class="app is-chrome is-desktop" unifi-bowser=""><unifi-key-events></unifi-key-events><!--[if lte IE 9]><div class="appOutdatedBrowserWarning is-warning-visible ubntToastContainer"><![endif]--><!--[if gt IE 9]><!--><div class="appOutdatedBrowserWarning ubntToastContainer"><!--<![endif]--><div class="ubntToast"><div class="ubntToast__image"><div class="ubntIconContainer ubntIconContainer--round ubntIconContainer--danger"><div class="type-larger ubnt-icon--alert" style="margin-top: -4px;"></div></div></div><div class="ubntToast__body"><div class="ubntToast__title">Outdated Browser</div><div class="ubntToast__details">You are using an <strong>outdated</strong> browser. Please <a href="http://browsehappy.com/" class="ubntLink">upgrade your browser</a> to improve your experience.</div></div></div></div><div class="ubntToastContainer ubntToastContainer--topRight" hey-toaster="" unifi-position-alerts=""></div><!----><ui-view><!----><ui-view><div class="unifiAccountContainer"><div class="stack spreadApart fullHeight"><div class="flexible shelfCenter"><div class="unifiAccount"><div class="unifiAccount__unifiLogo icon ubnt-icon--unifi"></div><div class="unifiAccount__appNameAndVersion"><span class="appDisplaySpaceAfter ng-hide" unifi-if-app="'app-unifi-hotspot'" translate="ACCOUNT_TITLE_HOTSPOT_MANAGER">Hotspot Manager</span><span>5.12.35</span></div><div class="unifiAccount__content"><!----><ui-view><div class="unifiLogin ubnt-mod-dark"><form class="appForm appForm--dark ng-pristine ng-invalid ng-invalid-required" novalidate="" name="loginCtrl.form" ng-class="{'is-form-validation-visible': loginCtrl.showValidation}" ng-submit="loginCtrl.form.$valid &amp;&amp; loginCtrl.login()"><!----><!----><div class="appFormGroup"><label class="appLabel appLabel--primary unweighted" translate="ACCOUNT_LOGIN_LABEL_USERNAME">Username</label><input class="input--full ng-pristine ng-empty ng-invalid ng-invalid-required ng-touched" type="text" name="username" ng-model="loginCtrl.username" required="" autocapitalize="off" autocorrect="off" autofocus=""></div><div class="appFormGroup"><label class="appLabel appLabel--primary unweighted" translate="ACCOUNT_LOGIN_LABEL_PASSWORD">Password</label><input class="input--full ng-pristine ng-untouched ng-empty ng-invalid ng-invalid-required" type="password" name="password" ng-model="loginCtrl.password" required=""></div><div class="verticalRhythmTop--cozy"><div class="appFormGroup alignCenter"><div class="appInputGroup appInputGroup--cozy"><input id="loginRemember" type="checkbox" name="remember" ng-model="loginCtrl.remember" class="ng-pristine ng-untouched ng-valid ng-empty"><label class="unweighted" for="loginRemember" translate="ACCOUNT_LOGIN_LABEL_REMEMBER_ME">Remember me</label><unifi-tooltip unifi-tooltip-class="'ubntTooltip--small'"><div class="ubntTooltipContainer ubntTooltipContainer--hoverable" ng-mouseover="!$ctrl.clickable ? $ctrl.showTooltip = true : angular.noop" ng-mouseleave="!$ctrl.clickable ? $ctrl.showTooltip = false : angular.noop" ng-class="::[    $ctrl.clickable ? 'clickable' : 'ubntTooltipContainer--hoverable',    $ctrl.containerClass  ]" ng-init="$ctrl.clickable ? $ctrl.showTooltip = false : angular.noop" ng-click="$ctrl.clickable ? $ctrl.showTooltip = !$ctrl.showTooltip : angular.noop" on-outside-element-click="$ctrl.clickable ? $ctrl.showTooltip = false : angular.noop"><div ng-transclude="prompt"><div class="unifiAuraIcon" ng-switch="auraIconCtrl.icon" unifi-aura-icon="'info'" unifi-aura-icon-class="['ubntIcon', 'color-primary']"><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><svg ng-switch-when="info" ng-class="{&quot;ubntIcon&quot;:true,&quot;color-primary&quot;:true}" viewBox="0 0 16 16" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" class="ubntIcon color-primary"><g stroke="none" stroke-width="1" fill="none" fill-rule="evenodd"><circle stroke="currentColor" cx="8" cy="8" r="7.5"></circle><rect fill="currentColor" x="7.5" y="7" width="1" height="5"></rect><rect fill="currentColor" x="7.5" y="5" width="1" height="1"></rect></g></svg><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----><!----></div></div><!----></div></unifi-tooltip></div></div></div><div class="verticalRhythmTop--cozy"><button class="appMainButton appMainButton--primary appMainButton--full busyToggle" id="loginButton" ng-click="loginCtrl.showValidation = true" busy-toggle="loginCtrl.isSubmitting" busy-toggle-spinner-classes="['appSpinner', 'appSpinner--light', 'appSpinner--centered']" translate="ACCOUNT_LOGIN_ACTIONS_SIGN_IN">Sign In</button></div><div class="verticalRhythmTop--cozy"><a class="appMainButton appMainButton--transparentDark appMainButton--full" ui-sref="manage.account.forgotpassword" unifi-if-app="['app-unifi-manage', 'app-unifi-overview']" translate="ACCOUNT_LOGIN_LINK_FORGOT_PASSWORD" href="/manage/account/forgotpassword">Forgot password?</a></div></form></div></ui-view></div><!----></div></div><!----></div></div></ui-view><unifi-browser-warning></unifi-browser-warning></ui-view><div class="blackout unifiDebugPanel ng-hide" ng-show="debugPanelCtrl.isEnabled"><!----></div><div class="unifiModalLayoutContainer"><div unifi-blackout-modal-layout="addClientLayout" unifi-blackout-modal-disable-esc="true"><!----></div><div unifi-blackout-modal-layout="autoChannelsModalLayout" unifi-blackout-modal-disable-esc="true"><!----></div><div unifi-blackout-modal-layout="addDpiCategoryGraphLayout"><!----></div><div unifi-blackout-modal-layout="analyticsAcceptanceLayout" unifi-blackout-modal-disable-esc="true"><!----></div><div unifi-blackout-modal-layout="addSiteLayout"><!----></div><div unifi-blackout-modal-layout="addSshKeyLayout"><!----></div><div unifi-blackout-modal-layout="cloudAccess2faTokenLayout"><!----></div><div unifi-blackout-modal-layout="cloudAccessUserLayout" unifi-blackout-modal-reject-on-esc="true"><!----></div><div unifi-blackout-modal-layout="cloudKeySdCardCheckLayout" unifi-blackout-modal-disable-esc="true"><!----></div><div unifi-blackout-modal-layout="configureMapsLayout"><!----></div><div unifi-blackout-modal-layout="confirmLayout"><!----></div><div unifi-blackout-modal-layout="dashboardFormLayout"><!----></div><div unifi-blackout-modal-layout="deviceUlteSetupLayout"><!----></div><div unifi-blackout-modal-layout="discoveryToolChromeRequiredLayout"><!----></div><div unifi-blackout-modal-layout="discoveryToolInstallLayout"><!----></div><div unifi-blackout-modal-layout="discoveryToolPortAlreadyInUseLayout"><!----></div><div unifi-blackout-modal-layout="dpiRestrictionFormLayout"><!----></div><div unifi-blackout-modal-layout="dpiRestrictionGroupFormLayout"><!----></div><div unifi-blackout-modal-layout="dpiRestrictionNetworkAssignmentFormLayout"><!----></div><div unifi-blackout-modal-layout="editAccountLayout"><!----></div><div unifi-blackout-modal-layout="adminPasswordChangeSuggestionLayout" unifi-blackout-modal-disable-esc="true"><!----></div><div unifi-blackout-modal-layout="editMapLayout"><!----></div><div unifi-blackout-modal-layout="exportSiteLayout" unifi-blackout-modal-disable-esc="true"><!----></div><div unifi-blackout-modal-layout="eliteDeviceOfflineLayout" unifi-blackout-modal-disable-esc="true"><!----></div><div unifi-blackout-modal-layout="firewallGroupsCreateLayout"><!----></div><div unifi-blackout-modal-layout="fallbackVlanLayout"><!----></div><div unifi-blackout-modal-layout="hotspot2NaiRealmListEntryLayout"><!----></div><div unifi-blackout-modal-layout="hotspotPackageLayout"><!----></div><div unifi-blackout-modal-layout="hotspot2CapabListEntryLayout"><!----></div><div unifi-blackout-modal-layout="hotspot2IconListEntryLayout"><!----></div><div unifi-blackout-modal-layout="hotspot2OsuListEntryLayout"><!----></div><div unifi-blackout-modal-layout="hotspot2QosListEntryLayout"><!----></div><div unifi-blackout-modal-layout="importSiteLayout"><!----></div><div unifi-blackout-modal-layout="ipsSuppressionFormLayout"><!----></div><div unifi-blackout-modal-layout="ipsWhitelistFormLayout"><!----></div><div unifi-blackout-modal-layout="instantRmaLayout" unifi-blackout-modal-disable-esc="true"><!----></div><div unifi-blackout-modal-layout="ipsThreatDetailLayout"><!----></div><div unifi-blackout-modal-layout="ispThroughputLayout"><!----></div><div unifi-blackout-modal-layout="preferencesLayout"><!----></div><div unifi-blackout-modal-layout="radiusProfileLayout"><!----></div><div unifi-blackout-modal-layout="remoteLoginLayout" unifi-blackout-modal-reject-on-esc="true"><!----></div><div unifi-blackout-modal-layout="reportClientFingerprintLayout"><!----></div><div unifi-blackout-modal-layout="simpleTextEditorLayout"><!----></div><div unifi-blackout-modal-layout="sitesOverviewLayout"><!----></div><div unifi-blackout-modal-layout="speedTestLayout"><!----></div><div unifi-blackout-modal-layout="statisticsDpiFiltersLayout"><!----></div><div unifi-blackout-modal-layout="systemConfigLayout"><!----></div><div unifi-blackout-modal-layout="reportTermsOfServiceLayout" unifi-blackout-modal-disable-esc="true"><!----></div><div unifi-blackout-modal-layout="trainingLoginLayout"><!----></div><div unifi-blackout-modal-layout="warningLayout"><!----></div><div unifi-blackout-modal-layout="webrtcErrorLayout"><!----></div><div unifi-blackout-modal-layout="webrtcFailedLayout"><!----></div><div unifi-blackout-modal-layout="wirelessNetworkMacFilterAddBatchLayout"><!----></div><div unifi-blackout-modal-layout="wirelessNetworkMacFilterAddClientsLayout"><!----></div><div unifi-blackout-modal-layout="wirelessUplinkDfsWarningLayout"><!----></div><div unifi-blackout-modal-layout="workingLayout"><!----></div><div unifi-blackout-modal-layout="webrtcReconnectingLayout" unifi-blackout-modal-disable-esc="true"><!----></div><div unifi-app-takeover-layout="whatsNewTakeoverLayout" unifi-app-takeover-show-header="true"><!----></div></div></div><!-- Native app loading status--></body></html>

You are using https with an IP, so certificate validation will fail.
Set verify_ssl to False.

https://www.home-assistant.io/integrations/scrape/#verify_ssl

as per @ludeeus about https / ssl
Your select config should be something like:
select: ".unifiAccount__appNameAndVersion span:nth-of-type(2)"
(tested only with HTML export you provided)

I added verify_ssl: false and now it loads like it should, however I’m not getting any results.

Tried all of these 3 without any luck:

select: ".unifiAccount__appNameAndVersion"
select: "#unifiAccount__appNameAndVersion"
select: ".unifiAccount__appNameAndVersion span:nth-of-type(2)"

Is there a way to test it out (like the template editor) without restarting HA every single time to see if it works?

No other way to treat I’m afraid unless you write your own python code
I seem to remember there was a limitation with tags having an underscore in it. This could be why is not working…