New AVM Fritz!box Tools problems

Fabietto_Abete · July 10, 2021, 5:49am

Good morning, I have Fritzbox 7590 plus 1750e repeater plus 3000 repeater, all in a mesh setup. I need to track with router source 2 smartphones, I did all the things required passing from old custom integration to the new official one, i.e. removing old yaml config, adding new discovered integrations, enabling smartphones device tracker found, setting a 15 sec period to consider home, BUT my phones that are connected to my wifi ARE ALWAYS SEEN NOT HOME!!!
EDIT: after abiliting specific log, I don’t see any error or warning about fritzbox device_tracker, it simply doesn’t work in my setup and I can’t figure how to solve

chemelli · July 10, 2021, 10:48am

I have switches for all my ports, not only for https.
IIRC, there is not yet support for port-ranges.
I don’t have independent port sharing enable for that device.

Simone

chemelli · July 10, 2021, 10:49am

Open a issue on GitHub and attach debug log.

Simone

DavidFW1960 · July 10, 2021, 12:35pm

Are they only for IPv4 forwards though? I only have ports opened to the IPv6 address of HA.

chemelli · July 10, 2021, 12:41pm

We can try to check if something appears in debug logs, add the following to configuration.yaml , restart HA, and post the logs near the error:

logger:
  default: warning
  logs:
    homeassistant.components.fritz: debug
    fritzconnection: debug
    fritzprofiles: debug

Note: logs may be hard to read here, it is better to use a service such as https://pastebin.com/ for uploading the log.

paddy0174 · July 10, 2021, 2:18pm

About what issues are we even talking about here? Or is this for all “new” problems?

If so, I already answered in the Github issue for the “connection pool error”. But to bring it up here as well, here is the link to pastebin:

DarrenHill · July 10, 2021, 9:09pm

@chemelli - one other question/request - would it be possible to get something equivalent to the old device_tracker_new_device event that we used to have previously?

Basically something that we can use as a trigger for when a new device is tracked by the integration. So that a notification can be made that it has happened, and to also act as a cue that there may be a new device entity that needs to be enabled?

Ideal case would be to pass the new entity’s id as part of the event (as an attribute or something) to make things clearer and more specific in the notification.

chemelli · July 11, 2021, 11:28am

I, @AaronDavidSchneider and @mammuth have those features in our todo list:

Fire an event for new detected devices
Add info about the “location” of the device: to which AP is connected, to which WIFI is connected
Mesh topology: avoid adding repeaters if in mesh
Merge fritzbox_callmonitor integration

Simone

DarrenHill · July 11, 2021, 11:58am

Sounds good to me (although personally I don’t need the last one)

paddy0174 · July 11, 2021, 12:08pm

Exactly the opposite here. I don’t need the first three, as I’m not tracking anything with the FB (to unreliable and slow, ping works better), but the call_monitor is desperately needed. Without it, I can’t let HA react to phone calls (active and missed).

I love this system, there’s something for everybody!

@chemelli
I’m still testing the number of switches, I’ll report back if my currently running uploads are finished and I can finally restart the FritzBox.

And thank you for all your work, much appreciated!

chemelli · July 11, 2021, 12:19pm

I’m not tracking anything with the FB (to unreliable and slow, ping works better)

Here it takes no more that 3 secs to react to home/not_home changes.
You know you have a option to lower the delay even to 0secs ?

Simone

Fabietto_Abete · July 11, 2021, 2:41pm

Ok after removing all avm integrations (I have 1 router and 2 repeater) and readding only the router, my phones device trackers started working as supposed so for me, for now, is better to ignore configuration of the avm repeaters because it creates problems in my setup with my phone device trackers. Anyway good work with this integration.

e-raser · July 11, 2021, 5:19pm

Removed custom integration from HACS → HA restart
Removed custom integration from HA integrations section → HA restart
Checked and changed (renamed) all (new) entities to fit the former IDs/names

Was a bit of work but migration worked pretty flawlessly. Someone just needs to know where to start (steps 1 and 2), to thanks for this topic.

e-raser · July 12, 2021, 9:27pm

New issue coming with 2021.7 and the integrated FritzBox Tools:

With 2021.6 when restarting HA I only received ONE eMail from my FritzBox telling me “homeassistant has logged in”.
Now with 2021.7 I receive this eMail 7 times on every HA restart. What the…?!?

As I changed the user to test it now I´m sure this is from FritzBox Tools integration (see core.config_entries where also the UI integrations are listed):

            {
                "entry_id": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
                "version": 1,
                "domain": "fritz",
                "title": "FRITZ!Box XXXX",
                "data": {
                    "host": "xxx.xxx.xxx.xxx",
                    "password": "plaintextpasswordohmygosh",
                    "port": 49000,
                    "username": "username"
                },
                "options": {},
                "pref_disable_new_entities": false,
                "pref_disable_polling": false,
                "source": "import",
                "unique_id": null,
                "disabled_by": null
            },

Why does the integration need to log on to my FritzBox 7 times?!?

DavidFW1960 · July 13, 2021, 3:35am

Any plans on adding port forward switches for IPv6 only port forwards?

chemelli · July 13, 2021, 10:40am

Looking forward for your debug log to see if those are detected or not

Simone

DavidFW1960 · July 13, 2021, 11:52am

No worries. I didn’t realise that request was meant for me. I will do that tomorrow. Thanks.

DavidFW1960 · July 14, 2021, 12:27am

Ok so here are some forwards for my dev ha instance

I set 44444 as a forward for IPv4 and IPv6 and 33333 as an IPv6 open port only. I also have port 80 as an IPv4 only port forward. debian-pm is my HA dev instance.

Removed and added back the Fritz Tools:

So that will be forwarding for IPv4 only.
Switch off IPv4 forwards with switches in HA:

So not seeing or controlling IPv6 port opens.
Also note I never setup profiles in the old custom_component but those are showing in the integration albeit disabled.
Logs:

2021-07-14 10:22:21 DEBUG (MainThread) [homeassistant.components.fritz] Setting up FRITZ!Box Tools component
2021-07-14 10:22:22 WARNING (MainThread) [homeassistant.config_entries] Config entry 'FRITZ!Box 7590' for upnp integration not ready yet: None; Retrying in background
2021-07-14 10:22:22 WARNING (MainThread) [homeassistant.config_entries] Config entry 'InternetGatewayDeviceV2 - FRITZ!Box 7590' for upnp integration not ready yet: None; Retrying in background
2021-07-14 10:22:23 DEBUG (SyncWorker_9) [fritzprofiles.fritzprofiles] FETCHING AVAILABLE PROFILES...
2021-07-14 10:22:23 DEBUG (SyncWorker_9) [fritzprofiles.fritzprofiles] LOGGING IN TO FRITZ!BOX AT http://10.90.11.1...
2021-07-14 10:22:24 DEBUG (SyncWorker_9) [fritzprofiles.fritzprofiles] LOGGING IN TO FRITZ!BOX AT http://10.90.11.1...
2021-07-14 10:22:24 DEBUG (SyncWorker_9) [fritzprofiles.fritzprofiles] FETCHING THE PROFILE ID...
2021-07-14 10:22:25 DEBUG (SyncWorker_9) [fritzprofiles.fritzprofiles] LOGGING IN TO FRITZ!BOX AT http://10.90.11.1...
2021-07-14 10:22:26 DEBUG (SyncWorker_9) [fritzprofiles.fritzprofiles] FETCHING THE PROFILE ID...
2021-07-14 10:22:27 DEBUG (SyncWorker_9) [fritzprofiles.fritzprofiles] LOGGING IN TO FRITZ!BOX AT http://10.90.11.1...
2021-07-14 10:22:27 DEBUG (SyncWorker_9) [fritzprofiles.fritzprofiles] FETCHING THE PROFILE ID...
2021-07-14 10:22:28 DEBUG (SyncWorker_3) [homeassistant.components.fritz.common] Checking devices for FRITZ!Box router 10.90.11.1
2021-07-14 10:22:32 DEBUG (MainThread) [homeassistant.components.fritz.binary_sensor] Setting up FRITZ!Box binary sensors
2021-07-14 10:22:32 DEBUG (MainThread) [homeassistant.components.fritz.sensor] Setting up FRITZ!Box sensors
2021-07-14 10:22:32 DEBUG (MainThread) [homeassistant.components.fritz.switch] Setting up switches
2021-07-14 10:22:32 DEBUG (MainThread) [homeassistant.components.fritz.switch] Fritzbox services: {'any1': <fritzconnection.core.processor.Service object at 0x7fae024b49d0>, 'WANCommonIFC1': <fritzconnection.core.processor.Service object at 0x7fae024b4940>, 'WANDSLLinkC1': <fritzconnection.core.processor.Service object at 0x7fae02468370>, 'WANIPConn1': <fritzconnection.core.processor.Service object at 0x7fae02468430>, 'WANIPv6Firewall1': <fritzconnection.core.processor.Service object at 0x7fae02468460>, 'DeviceInfo1': <fritzconnection.core.processor.Service object at 0x7fae03098940>, 'DeviceConfig1': <fritzconnection.core.processor.Service object at 0x7fae0a4c1100>, 'Layer3Forwarding1': <fritzconnection.core.processor.Service object at 0x7fae0309d5b0>, 'LANConfigSecurity1': <fritzconnection.core.processor.Service object at 0x7fae0309d3d0>, 'ManagementServer1': <fritzconnection.core.processor.Service object at 0x7fae0309d1f0>, 'Time1': <fritzconnection.core.processor.Service object at 0x7fae0309d9a0>, 'UserInterface1': <fritzconnection.core.processor.Service object at 0x7fae0243f670>, 'X_AVM-DE_Storage1': <fritzconnection.core.processor.Service object at 0x7fae0243f6a0>, 'X_AVM-DE_WebDAVClient1': <fritzconnection.core.processor.Service object at 0x7fae0243f730>, 'X_AVM-DE_UPnP1': <fritzconnection.core.processor.Service object at 0x7fae0243f790>, 'X_AVM-DE_Speedtest1': <fritzconnection.core.processor.Service object at 0x7fae0243f760>, 'X_AVM-DE_RemoteAccess1': <fritzconnection.core.processor.Service object at 0x7fae0243f640>, 'X_AVM-DE_MyFritz1': <fritzconnection.core.processor.Service object at 0x7fae0243f6d0>, 'X_VoIP1': <fritzconnection.core.processor.Service object at 0x7fae0243f850>, 'X_AVM-DE_OnTel1': <fritzconnection.core.processor.Service object at 0x7fae0243f820>, 'X_AVM-DE_Dect1': <fritzconnection.core.processor.Service object at 0x7fae0243f700>, 'X_AVM-DE_TAM1': <fritzconnection.core.processor.Service object at 0x7fae0243f8b0>, 'X_AVM-DE_AppSetup1': <fritzconnection.core.processor.Service object at 0x7fae0243f880>, 'X_AVM-DE_Homeauto1': <fritzconnection.core.processor.Service object at 0x7fae0243f610>, 'X_AVM-DE_Homeplug1': <fritzconnection.core.processor.Service object at 0x7fae0243f7f0>, 'X_AVM-DE_Filelinks1': <fritzconnection.core.processor.Service object at 0x7fae0243f5e0>, 'X_AVM-DE_Auth1': <fritzconnection.core.processor.Service object at 0x7fae0243f7c0>, 'X_AVM-DE_HostFilter1': <fritzconnection.core.processor.Service object at 0x7fae0243f910>, 'WLANConfiguration1': <fritzconnection.core.processor.Service object at 0x7fae02471ee0>, 'WLANConfiguration2': <fritzconnection.core.processor.Service object at 0x7fae02471f10>, 'WLANConfiguration3': <fritzconnection.core.processor.Service object at 0x7fae02471f40>, 'Hosts1': <fritzconnection.core.processor.Service object at 0x7fae02471f70>, 'LANEthernetInterfaceConfig1': <fritzconnection.core.processor.Service object at 0x7fae02471fa0>, 'LANHostConfigManagement1': <fritzconnection.core.processor.Service object at 0x7fae02471fd0>, 'WANCommonInterfaceConfig1': <fritzconnection.core.processor.Service object at 0x7fae01ed8130>, 'WANDSLInterfaceConfig1': <fritzconnection.core.processor.Service object at 0x7fae01ed8160>, 'WANDSLLinkConfig1': <fritzconnection.core.processor.Service object at 0x7fae01ed82e0>, 'WANEthernetLinkConfig1': <fritzconnection.core.processor.Service object at 0x7fae01ed8310>, 'WANPPPConnection1': <fritzconnection.core.processor.Service object at 0x7fae01ed8340>, 'WANIPConnection1': <fritzconnection.core.processor.Service object at 0x7fae01ed8370>}
2021-07-14 10:22:32 DEBUG (SyncWorker_0) [homeassistant.components.fritz.switch] Setting up CallDeflection switches
2021-07-14 10:22:32 DEBUG (SyncWorker_1) [homeassistant.components.fritz.binary_sensor] Updating FRITZ!Box binary sensors
2021-07-14 10:22:32 DEBUG (SyncWorker_7) [homeassistant.components.fritz.sensor] Updating FRITZ!Box sensors
2021-07-14 10:22:33 DEBUG (MainThread) [homeassistant.components.fritz.device_tracker] Starting FRITZ!Box device tracker
2021-07-14 10:22:33 DEBUG (SyncWorker_9) [homeassistant.components.fritz.sensor] Updating FRITZ!Box sensors
2021-07-14 10:22:33 DEBUG (SyncWorker_6) [homeassistant.components.fritz.sensor] Updating FRITZ!Box sensors
2021-07-14 10:22:33 DEBUG (SyncWorker_0) [homeassistant.components.fritz.switch] Specific CallDeflection response: GetNumberOfDeflections={'NewNumberOfDeflections': 3}
2021-07-14 10:22:33 DEBUG (SyncWorker_5) [homeassistant.components.fritz.sensor] Updating FRITZ!Box sensors
2021-07-14 10:22:33 DEBUG (SyncWorker_7) [homeassistant.components.fritz.sensor] Updating FRITZ!Box sensors
2021-07-14 10:22:33 DEBUG (SyncWorker_2) [homeassistant.components.fritz.sensor] Updating FRITZ!Box sensors
2021-07-14 10:22:33 DEBUG (SyncWorker_0) [homeassistant.components.fritz.switch] Setting up PortForward switches
2021-07-14 10:22:33 DEBUG (SyncWorker_3) [homeassistant.components.fritz.sensor] Updating FRITZ!Box sensors
2021-07-14 10:22:33 DEBUG (SyncWorker_8) [homeassistant.components.fritz.sensor] Updating FRITZ!Box sensors
2021-07-14 10:22:33 DEBUG (SyncWorker_0) [homeassistant.components.fritz.switch] Specific PortForward response: GetPortMappingNumberOfEntries=2
2021-07-14 10:22:33 DEBUG (SyncWorker_0) [homeassistant.components.fritz.switch] IP source for 10.90.11.1 is 10.90.11.153
2021-07-14 10:22:33 DEBUG (SyncWorker_0) [homeassistant.components.fritz.switch] Specific PortForward response: GetGenericPortMappingEntry={'NewRemoteHost': '0.0.0.0', 'NewExternalPort': 80, 'NewProtocol': 'TCP', 'NewInternalPort': 80, 'NewInternalClient': '10.90.11.153', 'NewEnabled': False, 'NewPortMappingDescription': 'HTTP-Server', 'NewLeaseDuration': 0}
2021-07-14 10:22:33 DEBUG (SyncWorker_0) [homeassistant.components.fritz.switch] Specific PortForward response: GetGenericPortMappingEntry={'NewRemoteHost': '0.0.0.0', 'NewExternalPort': 44444, 'NewProtocol': 'TCP', 'NewInternalPort': 44444, 'NewInternalClient': '10.90.11.153', 'NewEnabled': False, 'NewPortMappingDescription': 'HTTPS-Server', 'NewLeaseDuration': 0}
2021-07-14 10:22:33 DEBUG (SyncWorker_0) [homeassistant.components.fritz.switch] Setting up DeviceProfile switches
2021-07-14 10:22:33 DEBUG (SyncWorker_0) [homeassistant.components.fritz.switch] Setting up WiFiNetwork switches
2021-07-14 10:22:42 DEBUG (MainThread) [homeassistant.components.fritz.common] Unloading FRITZ!Box router integration
2021-07-14 10:22:43 DEBUG (MainThread) [homeassistant.components.fritz] Setting up FRITZ!Box Tools component
2021-07-14 10:22:45 DEBUG (SyncWorker_8) [fritzprofiles.fritzprofiles] FETCHING AVAILABLE PROFILES...
2021-07-14 10:22:45 DEBUG (SyncWorker_8) [fritzprofiles.fritzprofiles] LOGGING IN TO FRITZ!BOX AT http://10.90.11.1...
2021-07-14 10:22:46 DEBUG (SyncWorker_8) [fritzprofiles.fritzprofiles] LOGGING IN TO FRITZ!BOX AT http://10.90.11.1...
2021-07-14 10:22:46 DEBUG (SyncWorker_8) [fritzprofiles.fritzprofiles] FETCHING THE PROFILE ID...
2021-07-14 10:22:48 DEBUG (SyncWorker_8) [fritzprofiles.fritzprofiles] LOGGING IN TO FRITZ!BOX AT http://10.90.11.1...
2021-07-14 10:22:48 DEBUG (SyncWorker_8) [fritzprofiles.fritzprofiles] FETCHING THE PROFILE ID...
2021-07-14 10:22:49 DEBUG (SyncWorker_8) [fritzprofiles.fritzprofiles] LOGGING IN TO FRITZ!BOX AT http://10.90.11.1...
2021-07-14 10:22:49 DEBUG (SyncWorker_8) [fritzprofiles.fritzprofiles] FETCHING THE PROFILE ID...
2021-07-14 10:22:51 DEBUG (SyncWorker_1) [homeassistant.components.fritz.common] Checking devices for FRITZ!Box router 10.90.11.1

Hope that helps.

chemelli · July 14, 2021, 5:39pm

2021-07-14 10:22:33 DEBUG (SyncWorker_0) [homeassistant.components.fritz.switch] Specific PortForward response: GetPortMappingNumberOfEntries=2

Seems that the APIs, “GetPortMappingNumberOfEntries” and “GetGenericPortMappingEntry”, will list only ipv4 ports. Unfortunately.

I’m also not able to find any other API that will allow ipv6.

Simone

DavidFW1960 · July 14, 2021, 11:54pm

Thanks Simone.
I logged a ticket with AVM 4406785. They have been responsive in the past so will be interesting to see what they say.
Is the API used for this an official or unofficial API? Is it documented anywhere?
Thanks.