How to hack an API

I thought that title would get your attention. I have a device, let’s say a small round thing that runs around and cleans the floors in my house. Today about an hour into it’s cleaning cycle, it got full. This caused it to go back to it’s house and wait there patiently for someone to come by and empty it. Here is the problem. It does not tell anyone except by a little beep, and a message on it’s mobile app. So when it ran while I was at work, I had no idea it had filled up. Had I known, I could have woken up one of my kids and they could have cleaned it out and let it continue on it’s merry way. The device connects to my house wifi to get commands and share information with a cloud server somewhere that then makes it available to my phone.
So after my long winded explanation my question is, how do we determine the api calls when the vendor doesn’t publish them? And I tried to keep this somewhat vague since I’m after the how to do it to learn, and not the here is the hacked API for you, answer.

First guess would be traffic sniffing to find out which server and what is being sent.

Or find a firmware file and try to ‘open’ it

Do you have a particular sniffer you recommend (preferably free, spending all my money on switches lately).

The reverse engineering of the harmony API for that component was done using Wireshark. Not sure if it does all network traffic, or if it only handles the device it’s run on, though.

Wireshark captures every package of your NIC. Since the package from your small round thing doesn’t come to your NIC, you won’t see it in the wireshark capture.
If you have an external Access-Point, you can configure port-mirroring on your switch/router to have all the traffic of the wifi mirrored to a port and capture it there.
If you have a wifi-router with integrated switch, you can try to mirror the wifi-traffic to a port of the integrated switch. Another option is to put a hub or a switch with configured port mirroring between your Wifi-Router and your dsl-modem to capture all the traffic with the internet.
There is also the option to put the wlan-nic of a pc/laptop into monitor mode and capture all wifi packets directly. I have never done this by myself, it seems to be tricky and nic/driver/os dependent.

You can’t sniff packets going to other machines using a switch. However you can if you can find a hub (which sends packets to every connected device). Hubs went out of fashion when switches became cheap, as switches are far more efficient. However if you can put your computer and the device you want to sniff on a hub, you could sniff properly.

I am not sure how this wisdom works in a wifi environment though.

Shure you can, if you have a managed switch. You only have to configure to mirror the traffic from a port to another port, connect your wireshark-machine to the mirrored port and you get all the packages going through the original port.

Port mirroring - Wikipedia

you are right of course, I meant “out of the box”. Cheers.

Another important factor to keep in mind: the device may (and should) communicate to it’s cloud-service with SSL/TLS. If that’s the case, all you can sniff are encrypted packets.
What you could try then is to use a proxy like mitmproxy. But that may fail too depending on how your vacumm does SSL.