Mitsubishi Kumo Cloud Integration

geoff_lowery · July 31, 2021, 11:05pm

Gotcha. Bummer. I was hoping there was a magic fix

parkercat · August 1, 2021, 11:45am

My only advice for you two: start in the middle. Open up pykumo in an interactive Python shell as follows:

$ python3
>>> import pykumo
>>> account = pykumo.KumoCloudAccount.Factory()
(enter your KumoCloud username & password)
>>> account.try_setup()

If it prints True then pykumo is able to log in to your KumoCloud account; False and it can’t.
If it works, try:

account.get_indoor_units()
account.get_address('one-of-your-serial-numbers')

You should get the serial numbers of your unit(s) and an IP address for the one you chose.

If this all works, there’s some problem with how Kumo is configured in HomeAssistant. If it doesn’t, then Mitsubishi has changed or broken something for you (but not for me; I just tried the steps above and they’re fine).

geoff_lowery · August 1, 2021, 6:51pm

Thanks for the reply. I’m a bit confused as to which directory to run the interactive shell from. Should I be in the custom component/kumo directory? Or does it even matter? When I run the interactive shell from various different directories, including inside the Homeassistant Docker container, its unable to import the pykumo module. I just get the error "no module named ‘pykumo’.

My kumo_cache.json file gets populated with the names of my units and has correct serial numbers, mac addresses, and labels, it just doesn’t create any entities for anything. To me, it appears to be contacting the kumo cloud service, just not doing the rest of what it seems like its supposed to. Sorry if this is pretty vague, i’m still working on wrapping my brain around everything that the integration is doing. Thanks for all your work on this!

geoff_lowery · August 1, 2021, 7:05pm

Disregard my comments about where to run python from, I figured that part out. When I run it from the interactive Python shell, it returns false after calling account.try_setup(). Seems as though the connection from Homeassistant to the Kumo Cloud servers is very inconsistent as it clearly was able to contact the servers at some point in order to pull all of the information into my kumo_cache.json file… hooray for computering!

geoff_lowery · August 2, 2021, 9:36pm

Well after lots of trial and error and using some hints from running pykumo in the interactive shell, I was finally able to get the integration working. It turned out to be the absence of IP addresses for all my units as others have stated before. Here’s the steps I used to finally get it to work, including the best way that I found to get around the bad username, password, or information error that I kept running into when attempting to add the integration.

As others have said before, if you’ve already tried installing the integration before and its created the kumo_cache.json, make sure to delete this, delete the integration from integrations screen, and then reinstall from HACS. Reboot HA.
Add the integration again, make sure to select “Prefer_cache” option in the dialog box, and off you go. This, at least for me, has always created the kumo_config.json file for me assuming i’ve done step 1.
In the kumo_config.json file, I then added the static ip address of each unit as a new entry after the line “port”: 80,
I’m not sure if the address entry needs to be there or not, but thats what worked for me.
Once i added the static IP for each unit, I once again rebooted HA, and finally they showed up right away as expected. The local response time is blazingly fast when compared to the Kumo cloud app, plus you get all the HA goodness to go with it to automate all the things.

It could just be that all these steps are only what worked to get my instance and integration up and running, but hopefully not and it helps someone else too

Thank you to everyone whose posted about issues (and suggested answers to those issues) they’ve run into here as well as the issues opened (and closed) in the GitHub repo for the project, it really helped me piece it all together. And a big thank you to @parkercat , this is an awesome integration and I’m very much looking forward to integrating with everything HA and making it even more useful.

spurpura · August 4, 2021, 4:05am

I started working on this again, and I took a clean Raspberry Pi install of the latest Home Assistant OS. I wanted to validate and clarify @geoff_lowery instructions for others.

As others have said before, if you’ve already tried installing the integration before and its created the kumo_cache.json, make sure to delete this, delete the integration from integrations screen, and then reinstall from HACS. Reboot HA. (NOTE: since I installed from clean, I simply installed HACs first, waited a few hours for it to complete its installation fully and not be rate limited, and then started in on the rest of the instructions.)
Add the Kumo integration (again), make sure to select “Prefer_cache” option in the dialog box, and off you go. This, at least for me, has always created the kumo_config.json file for me assuming i’ve done step 1. (NOTE: the full path name is /config/kumo_config.json on the HASS Raspberry Pi install.)

2a. Open up an SSH shell:
cd /config
grep “address” kumo_cache.json

If no address is found or the address is incorrect, continue …

In the kumo_config.json file, I then added the static ip address of each unit as a new entry after the line “port”: 80, I’m not sure if the address entry needs to be there or not, but thats what worked for me. (NOTE: the correct format for the address line is:

“address”: “10.0.1.16”,

It matters where you place this line in the file. Placing it on the immediately after the “port” line worked for me. I also used spaces to push the “address” to be lined up with “port”. Unknown if it wants that it or not but it helps readability.)

Once i added the static IP for each unit, I once again rebooted HA, and finally they showed up right away as expected. The local response time is blazingly fast when compared to the Kumo cloud app, plus you get all the HA goodness to go with it to automate all the things.

geoff_lowery · August 4, 2021, 8:21pm

Thanks for the additions @spurpura , I should have included that information as well. Hopefully it helps others too!

omriasta · August 23, 2021, 6:49pm

@parkercat it appears that they now have a webapp:
https://app.kumocloud.com/#/login
Looked a bit through the network tab of browser dev tools and seeing the json responses. Might be helpful in troubleshooting and maybe nailing down the issues with ip address and the conversion table of C vs F

dgoepp · August 23, 2021, 9:42pm

@omriasta and @parkercat - I’ve looked over the console for that app quite a few times but have not been able to find anything yet regarding the ip address problem:( Not saying it’s not there, I’ve just not been able to find it. I would also add that the API call to get that initial data still doesn’t return the ip addresses for me, months later. I was trying to be patient, but nothing. I believe perhaps it’s because I don’t use the app at all any more to adjust my temps, I just use HA. This might mean that the app is not being given a chance to push back the local address to kumo cloud. I keep trying to run in on occasion just to see if it will update, but it hasn’t. I fear that other people (new users) might fall into this same situation. I have no idea what triggers that data push, but I’m confident that it is the app providing this to the cloud. I honestly think the only solution here is to emulate what the app itself must do, which is to scan the local network then save the address matched to MAC address. ARP lookups won’t work, the ARP table entry won’t exist until a communication happens. Since most home networks are Class C (/24, 255.255.255.0, 256 addresses, whatever you want to say) networks, a quick ping scan should actually be pretty easy and fast. Then follow up HTTP requests to identify/verify. I started doing some work this last weekend on this, just locally here to play around, and I’m hoping to be able to come up with something. My python is not the best, but I think I can hack something together for review. Just in case you were consider the same, or working on something, I didn’t want to conflict or duplicate your efforts and thoughts on this. I do think this is affecting and frustrating people, and the manual work around right now is kinda annoying. Granted, HA users tend to be on the tech side by nature, but no need to make it harder than it needs to be. Thoughts?

omriasta · August 23, 2021, 9:57pm

If you look at the json returned from the WebApps, does it have the local ip address? I don’t have it open right now but when using the WebApp remotely it did return the IP address for me. Wondering if maybe the endpoint used by the WebApp behaves differently from what pykumo uses.

dgoepp · August 23, 2021, 10:21pm

It doesn’t for me. I think the difference between the people that this works for, and those that it doesn’t is if the kumo app (phone or web) has pushed an update up to the service. For example, I bet if you run the manual API check, you get the IP addresses in the output

curl 'https://geo-c.kumocloud.com/login' \
  -H 'Connection: keep-alive' \
  -H 'sec-ch-ua: " Not;A Brand";v="99", "Google Chrome";v="91", "Chromium";v="91"' \
  -H 'Accept: application/json, text/plain, */*' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36' \
  -H 'Content-Type: application/json;charset=UTF-8' \
  -H 'Origin: https://app.kumocloud.com' \
  -H 'Sec-Fetch-Site: same-site' \
  -H 'Sec-Fetch-Mode: cors' \
  -H 'Sec-Fetch-Dest: empty' \
  -H 'Referer: https://app.kumocloud.com/' \
  -H 'Accept-Language: en-US,en;q=0.9,es;q=0.8' \
  --data-raw '{"username":"REDACT","password":"REDACT","appVersion":"2.2.0"}' \
  --compressed | python -m json.tool > ./kumo.txt

I use this to test about once a week, I’m certain at one point this response included the IP address, and was stored in my kumo cache, but it doesn’t any more (so I had to manually add it like many others). I can only conclude that kumo no longer knows my IP addresses.

spurpura · August 23, 2021, 10:38pm

My limited testing (n = 3 sites) suggests that the HA integration is returning the local IP address that was active for each head unit during initial configuration of the head unit.

The test that I’ve never been willing to do is to redo setup for the kumo devices to see if this resolves the problem of finding the local IP addresses for the HA integration.

omriasta · August 23, 2021, 11:15pm

I remember a while back I tried to look into this by running the app on genymotion with an old Android version using a proxy to see what was being sent to the cloud servers. Maybe running it with Wireshark might show how it scans the local network if you guys are willing to try. I couldn’t see it sending anything to the cloud servers and noticed that in general the app always prefers the cloud over local communication which is why it’s so slow for most people.

dgoepp · August 24, 2021, 12:40am

@spurpura when you say the HA integration is returning, do you mean discovering? To my understanding the only awareness that this integration has about the local address is what it gets from kumo cloud. So yes, I think I agree with your statement which I read as “during initial setup, if the IP address is sent to the cloud, then this integration has access to it.” Are you comfortable with command lines at all? Would you be willing to run the command I listed a few comments back and confirm if address is a returned value for your account?

@omriasta / @parkercat is using third party libraries allowed? If so I have a quick code sample here that might be a start to doing a local scan. This works on my machine to find the IP address of one of my units, and is really pretty quick.

import socket
import ipaddress
import scapy.all as scapy

# get my ip address
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.connect(('8.8.8.8', 1))
local_ip_address = s.getsockname()[0]
#print(local_ip_address)

# get my network
interface = ipaddress.IPv4Interface(local_ip_address+'/24')
#print(interface.network)

# arp ping my network
ans, unans = scapy.srp(scapy.Ether(dst="ff:ff:ff:ff:ff:ff")/scapy.ARP(pdst=str(interface.network)),timeout=1)

# load my scan
arp_table=dict()
for i in ans:
    arp_table[i.answer.hwsrc] = i.answer.psrc 
    # print(i.answer.hwsrc+" - "+i.answer.psrc)

# here is the ip address for this device
print(arp_table['dc:ef:ca:10:8e:f0'])

Before I work on this further, I’d like to get your thoughts. The library used is:

spurpura · August 24, 2021, 2:42am

Do you mean the curl command?

After inserting the right username/password, it returns an empty kumo.txt and the error message “No JSON object could be decoded”

spurpura · August 24, 2021, 2:55am

@dgoepp the curl statement returns Bad Request if you cut out the pipe to the json tool.

dgoepp · August 24, 2021, 9:33am

@spurpura - Odd, I wonder what’s happening there. No worries though, thanks for trying.

parkercat · August 24, 2021, 10:59am

Hi Daniel,
Thanks for your continued poking at this issue. My day job has been keeping me busy (basically since the start of the pandemic) and I don’t have a whole lot of time to devote to Kumo these days.

I just ran the curl command you provided (having to replace python with python3 on my system) and now, as it has for years, the data returned for me from the KumoCloud servers contains my IP addresses. I 100% believe you that you’ve never had it work, but without being able to repro I’m not much help for debugging. My memory is fuzzy, but as far as I know the Kumo adapter itself reports its current IP address to the service, which then reports it to the phone app, which then is able to do local communication. The phone app is not running any sort of discovery protocol on its own.

So what could be going wrong? It could be the indoor unit is not being asked for its address by the service, could be not reporting it for some reason (bug, configuration, failing some internal check), or the service is failing to report the address for its own reasons.

Regardless, my preferred fix for this issue is to provide a way in the UI for the user to provide or override the IP address (perhaps even a hostname) for each of their indoor units. Yes, this requires a degree of network savvy by the user, but it’s more friendly than making them edit a JSON file. If the user is configuring static-lease DHCP (recommended), they already know the addresses. I’ve just had zero time or motivation to work on this feature.

A tool like you’re working on might help users discover their units’ IP addresses if unknown though. So I’d be happy to host your code on the pykumo or hass-kumo repos (or provide a link in the documentation if you prefer). I think using something like that for setup-time discovery would only be a good idea if most or all new installations are seeing the issue – and I have a sample size of 1 for a 2021 install that’s working just fine.

Update: one theory came to mind as I was typing this: are your indoor units and your phone on the same WiFi access point? Same subnet? (Same /24 in case Kumo code is being stupid?) Is your router configured with any sort of “coffeehouse” mode that blocks local clients from one another?

omriasta · August 24, 2021, 12:56pm

Hey guys, I actually initially wrote the config flow for the integration. Would it be helpful to add a “configure” option to enter an ip address if not detected using the homeassistant ui?
Can one of you without the ip in json post what you have in the zone table part of the json:

                "zoneTable": {
                    "MY_SERIAL_NUM": {
                        "$$hashKey": "object:592",
                        "_isRespondingLocally": false,
                        "_requestRescan": 19,
                        "address": "MY_IP_ADDRESS",
                        "autoModeEnabled": true,
                        "cryptoKeySet": "F",
                        "cryptoSerial": "CRYPTO_SERIAL",
                        "desiredConditionStack": [],
                        "eqcStageThreeBacksupChannels": {
                            "1": 0,
                            "2": 0,
                            "3": 0,
                            "4": 0
                        },

Also, if you go through the setup and there is no ip, does the integration actually complete and get setup? Meaning do you see it in the integrations and can you hit the “configure” button?

spurpura · August 24, 2021, 3:36pm

@parkercat my 2 head units are located on a different VLAN and network than the raspberry pie running home assistant and different yet than a cell phone or client computer configuring them. I was only able to map their name (eg Media Room, Bed Room) in the partial kumo config file using their MAC address … which I obtained from ubiquiti’s controller client list by searching for the MAC address.

It is quite possible that your code will work perfectly fine with just the IP addresses. But the initial configuration during setup (where most of the kumo config file was populated) couldn’t be fixed without the MAC address to complete the mapping (for me).

I’m trying to tell you that my installer setup the multiple kumos in my house without recording which IP was for which head unit. And the installer also had no idea how to deal with a static dhcp address. So, the state of my system was a few head units with IP addresses and MAC addresses but no indication which unit was which.