La Marzocco GS/3 & Linea Mini support

More interesting tidbits:

Message that periodically reports number of drinks dispensed. I’ve figured out most, but some of these values are vexing. I’ll probably create a sensor for this.

Message=0020002C
0000014A: Total # Key 1 (1 espresso)
00000096: Total # Key 2 (2 espressos) * 2 = 0x12C
000001B0: Total # Key 3 (1 coffee)
00000021: Total # Key 4 (2 coffees) * 2 = 0x42
00000563: Total # Flushing
00000914: Total # from Coffee boiler (total of above)
00000010: Tea, increments every button press
0000000A: ?? Never changes
00000ADE: ?? Increments every button press. Close to adding everything else up, but not quite
00000010: Also Tea? Increments every button press
00000032: # Tea, if you let it finish

Whenever the pump is running, either for espresso or flushing, I get a constant stream of these:

Z (whoa)
Message=60000016
Snxxxxxxxxxx (serial number of the module, I believe)
0501: Started at 0401, changed to 0500, then to 0501 for some reason that I can’t discern. No idea what this is.
003A: Current pulse count (starts at zero when the button is pressed)
0054: seconds that the pump has been running, where the first 12 bits are the whole seconds and the last 4 bits are fractional. Starts at zero when the button is pressed.
03B6: Current coffee boiler temp
04D6: Ciurrent steam boiler temp

I’m not sure what to do with this message, since the connection needs to be open to get it. Might be cool to have a sensor for it, but I would need to leave the connection open all the time for it to be reliable. No bueno.

Edit:

I’m not sure that changing the polling number will help here - I have noticed the device is showing the wrong status in the Apple Home app a long time after last use, by which point the status should have updated.

Also I’ve just found that the Linea Mini is no longer responding - checking in home assistant I see:

“This entity is currently unavailable and is an orphan to a removed, changed or dysfunctional integration or device.”

It is still working via the La Marzocco app

Okay, a couple of things would be helpful, if you don’t mind:

  • In HACS, click the 3 dots in the lower-right corner of the La Marzocco card and select “Reinstall”. Select “Show beta versions” and version 0.7 should now show up in the list (you may have to exit that screen and re-enter). Please install that version. I’ve been making changes to the config data, so you’ll probably have to delete the integration from Configuration->Integrations and set it up again.
  • Add the following to configuration.yaml:
logger:                                                              
  default: info          
  logs:                     
    custom_components.lamarzocco: debug          
    lmdirect: debug

If you already have the logger: section, just add the last two lines to it. It’s pretty chatty, but will be really helpful.

It appears that the drink stats in the app come from the gateway and mysteriously differ from the values returned by the machine. For now, I’m querying the gateway at startup and calculating offsets that I use to adjust the data returned by the machine so that it matches the app. It’s ugly and I don’t understand the delta, but it works.

I’ve also added the version number to the device page and added an attribute indicating if updates are available. The latter currently returns “[]”, so I don’t know what it will look like if updates are available.

The API endpoints for the drink states and update status are:
https://gw.lamarzocco.io/v1/home/machines/{serial_number}/statistics/counters
https://gw.lamarzocco.io/v1/home/machines/updates-available?device=machine

The daily drink stats come from here (with appropriate timestamps):
GET https://gw.lamarzocco.io/v1/home/machines/{serial_number}/statistics/daily?endDate=2021-01-03T08%3A00%3A00.000Z&startDate=2020-12-02T08%3A00%3A00.000Z&timezoneOffset=480

I would that add to the drink stats sensor, but I don’t want to keep querying the gateway and I can’t the info locally.

Ok I am wondering if my Linea Mini changed IP address which could have made it stop responding. I’ve given it a reserved IP now.

I have deleted the integration and reinstalled using v0.7. I now have the brew temp, steam temp, prebrew switch and auto off switch appearing but have not tested them yet.

I noticed that when I first set it up, it was shown as being turned on in the Home Assistant dashboard (and in the Apple Home app), though the machine had not been turned on for some time.

I turned it on and off using Siri and it correctly identified as being turned off.

I then turned it on using the paddle and waited a small amount of time and it still showed as being turned off in the Apple Home app.

I’ll try this again tomorrow, waiting for longer to see if the Home app or Home Assistant dashboard update after 1-2 minutes.

The brew temp seems to work and I am pretty sure shows the current temperature and not the target temperature (which is nice, because the official app doesn’t show this).

The steam temp doesn’t seem to work, perhaps the Linea Mini isn’t equipped with one.

Yes, it should show the current brew/steam boiler temps and not just the set points. Seeing a dump of your data via the debug options that I mentioned would be very helpful. The power info would be in a line like this, with the power being the second 01 from the right.

Message=40000020, Data=017802536E313931323030303439320100000000000000010100003003B604D8

You could also try running the standalone test.py harness at https://github.com/rccoleman/lmdirect to cut out of the “middleman” and see how the data is actually being decoded. Whether the machine is on or off will be reflected as "POWER": 0 or "POWER": 1 in the dictionary dump (option 2). You would need to set up a Python environment and install a few dependencies for that via pip install -r requirements.txt, though.

Great work guys. I got myself a Linea Mini a couple of weeks ago and have been following this thread with some interest.

I ran into a number of issues getting @rccoleman’s test.py script going, but this is due to some missing components in my python install (getting snagged on Crypto). Gave up on this after a couple of days and just went at it with the APIs provided by @plonx

Have managed use the APIs with Apple Shortcuts so that I can get the machine status, and turn the machine ON and into STANDBY using Siri (or Shortcuts on iPhone and Watch). I also note that the Linea Mini Status API always shows TEMP_STEAM as 0.

This morning, I’ve been trying to set the K1-K4 pre-brew on and off to 10 secs to help automate a backflush routine (rather than jockeying the brew paddle back and forth), but it seems that there is no equivalent PUT API for the Configuration GET call. I then looked again at what the App was sending as I modified App settings and saw the https://api.mixpanel.com/track/ and /engage/ calls with the R/W AES encrypted requests. Re-read the thread above to see the challenges in deciphering. :confused:

btw, I’m not seeing direct calls from the App to the Linea Mini on the local network (on mitmproxy). Only via api.mixpanel.com. How are you guys seeing these requests?

Next step for me is to get HACS installed and try @rccoleman’s integration.

pip install -r requirements.txt should install all the requirements, but I haven’t tried it in a while myself. The cloud API is pretty limited and I believe that the only “set” functionality is turning the machine on and off. You’ll see the same thing in the app, where most of the settings are grayed out when connected remotely. That’s one of the reasons that I was so interested in getting the local API working.

I only see the HTTPS requests through mitmproxy, but I haven’t played with it much and I use Wireshark and my parse.py script to decode/decrypt the packets.

I just ran a quick test of test.py and it worked. Here’s what I did:

  • $ git clone https://github.com/rccoleman/lmdirect
  • $ cd lmdirect
  • $ python3 -m venv venv
  • $ source venv/bin/activate
  • $ pip install -r requirements.txt
  • Create a config.json in the directory with content like this:
{
    "host": "ip_addr",
    "port": "1774",
    "client_id": "long_string",
    "client_secret": "another_long_string",
    "username": "email_address",
    "password": "password"
}
  • $ python test.py

Watch it request and receive info, and hit “2” to have it spit out a populated dict. Exact same thing works on both MacOS and Linux for me.

I suspect that the LM uses a pressurestat or similar to control steam boiler temp and it isn’t PID-controlled like the GS/3. That would explain why the steam team isn’t showing up.

Uploading: 231047BD-7179-43DF-9180-C6319F49B3C9.JPG…
Yep, you’re right. A pressurestat controls LMLM steam.

Thanks for your clear instructions - I’m still relatively new to managing my own python environment, so still getting snagged on a few things (but I don’t want to bog down this discussion with that).

  • Mac BigSur python3 uses version 3.6.3 - had issues with venv so instead using python3.9 (I think I may have installed this some other time while bashing the keyboard);
  • needed to install wheel to get pip install working correctly.

I was able to run python3.9 test.py but about the only thing working for me (or at least returning values) was 2:Status.

(venv) CMM-C02T722QGTFM:lmdirect ccq$ python3.9 test.py

1=Power, 2=Status, 3=Coffee Temp, 4=Steam Temp, 5=PB on/off, 6=Auto on/off enable/disable, 7=Dose, 8=Tea Dose, 8=PB on/off:
2
{}

1=Power, 2=Status, 3=Coffee Temp, 4=Steam Temp, 5=PB on/off, 6=Auto on/off enable/disable, 7=Dose, 8=Tea Dose, 8=PB on/off:
2
{'FIRMWARE': '2.07', 'MODULE_SER_NUM': 'Sn2010009957', 'POWER': 1, 'TSET_COFFEE': 93.0, 'TSET_STEAM': 0.0, 'ENABLE_PREBREWING': 0, 'TON_PREBREWING_K1': 2.1, 'TON_PREBREWING_K2': 0.0, 'TON_PREBREWING_K3': 0.0, 'TON_PREBREWING_K4': 0.0, 'TOFF_PREBREWING_K1': 2.0, 'TOFF_PREBREWING_K2': 0.0, 'TOFF_PREBREWING_K3': 0.0, 'TOFF_PREBREWING_K4': 0.0, 'DOSE_K1': 0, 'DOSE_K2': 0, 'DOSE_K3': 0, 'DOSE_K4': 0, 'DOSE_K5': 0, 'DOSE_TEA': 0, 'GLOBAL_AUTO': 'Enabled', 'MON_AUTO': 'Enabled', 'TUE_AUTO': 'Enabled', 'WED_AUTO': 'Enabled', 'THU_AUTO': 'Enabled', 'FRI_AUTO': 'Enabled', 'SAT_AUTO': 'Enabled', 'SUN_AUTO': 'Enabled', 'SUN_ON': 5, 'SUN_OFF': 12, 'MON_ON': 5, 'MON_OFF': 12, 'TUE_ON': 5, 'TUE_OFF': 12, 'WED_ON': 5, 'WED_OFF': 12, 'THU_ON': 5, 'THU_OFF': 12, 'FRI_ON': 5, 'FRI_OFF': 12, 'SAT_ON': 5, 'SAT_OFF': 12}

1=Power, 2=Status, 3=Coffee Temp, 4=Steam Temp, 5=PB on/off, 6=Auto on/off enable/disable, 7=Dose, 8=Tea Dose, 8=PB on/off:
1
Traceback (most recent call last):
  File "/Users/ccq/Documents/GitHub/lmdirect/test.py", line 134, in <module>
    asyncio.run(lm.main())
  File "/usr/local/Cellar/[email protected]/3.9.1_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/Cellar/[email protected]/3.9.1_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/Users/ccq/Documents/GitHub/lmdirect/test.py", line 98, in main
    await self.lmdirect.set_power(args[1] == "on")
IndexError: list index out of range
(venv) CMM-C02T722QGTFM:lmdirect ccq$

My repo was a few days old, so did a git pull, and now test.py is erroring. FYI:

(venv) CMM-C02T722QGTFM:lmdirect ccq$ python3.9 test.py
Traceback (most recent call last):
  File "/Users/ccq/Documents/GitHub/lmdirect/test.py", line 142, in <module>
    asyncio.run(lm.main())
  File "/usr/local/Cellar/[email protected]/3.9.1_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/Cellar/[email protected]/3.9.1_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/Users/ccq/Documents/GitHub/lmdirect/test.py", line 64, in main
    creds = await loop.run_in_executor(None, self.read_config)
  File "/usr/local/Cellar/[email protected]/3.9.1_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/ccq/Documents/GitHub/lmdirect/test.py", line 30, in read_config
    HOST: data["host"],
KeyError: 'host'
(venv) CMM-C02T722QGTFM:lmdirect ccq$

I’m not sure if these errors are specific to the Linea Mini or something else I need to fix in my python environment. I’m not looking for answers - just thought the output may be of interest to you.

The last error appears to indicate that you’re missing ‘host’ from config.json, so make sure you have everything I have above including host and port. I’m using Python 3.8.6, but I’m not aware of anything that would break with 3.9.

I’ll check on the commands - I’ve been playing with the library and may have broken some functionality there. I’m glad that you’re getting some data, and that’ll be very helpful. Thanks for that. Interesting that your firmware version is completely different from mine (Feb 2020 GS/3, 1.20).

Otherwise, the data looks fine. Are you seeing responses to all the commands scroll by?

Yes, firmware level is completely different between the LM and GS3 it seems.

At the moment, I don’t have Wireshark running, so haven’t looked closely at the responses.

host definition is definitely there and as you can see I got a response to Status when using an older version of test.py.

I have to go and #parent now, so won’t get a chance to play for a while.

That error was simple enough to fix - #1 takes 2 arguments (on or off) and I was only checking for 1. There were several others missing a similar check all should be fixed if you pull again.

If you want to turn the machine on or off, do this:

1 on - turn on
1 off - turn off

The other commands have similar syntax - # space-separated-arguments:

1=Power <on/off>, 2=Status, 3=Coffee Temp <temp>, 4=Steam Temp <temp>, 5=PB <on/off>, 6=Auto on/off <0=global or day> <on/off>, 7=Dose <sec>, 8=Tea Dose <sec>, 8=PB times <on off>: 

I have no idea why it’s not picking up the host. It’s pretty straightforward JSON:

        try:
            with open("config.json") as config_file:
                data = json.load(config_file)

        except Exception as err:
            print(err)
            exit(1)

        creds = {
            HOST: data["host"],
            PORT: data["port"],
            CLIENT_ID: data["client_id"],
            CLIENT_SECRET: data["client_secret"],
            USERNAME: data["username"],
            PASSWORD: data["password"],
            KEY: data.get("key", None),
        }

I did rename “ip_addr” to “host” on Dec 27th, so depending on how old that version was, that could be causing the problem: https://github.com/rccoleman/lmdirect/commit/948fb130e6a07f9cdb03ee6832b951ddf72c7daf#diff-3665d65394f4f58a56a256ad6dd8621c68118d90fe56a19387e251c19cec2d2e

I see that I forgot to update the documentation on github to reflect that change and the need for the port field. I updated it.

@rccoleman I noticed these showing up in my logs. Is this something you want to know about? I’m have a Linea Mini with v0.7 of your code.

Logger: lmdirect.connection
Source: /usr/local/lib/python3.8/site-packages/lmdirect/connection.py:229
First occurred: 2:53:00 PM (202 occurrences)
Last logged: 2:54:44 PM

Unexpected response: Z60000016536E32303037303035353138010000110000038E0000A7
Unexpected response: Z60000016536E32303037303035353138010000120000038E0000A8
Unexpected response: Z60000016536E32303037303035353138010000130000038E0000A9
Unexpected response: Z60000016536E32303037303035353138000000130000038E0000A8
Unexpected response: Z60000016536E32303037303035353138000000130000039800009C

Yeah, those are the streaming flow meter/timer messages that are sent while the pump is running that I discovered a few days ago and mentioned here.. The latest dev branch decodes it and eliminates that log message, but doesn’t do anything more with it because I can’t think of a good use for it. You can safely ignore the messages until I release a new version. I’m not polling for that, so you’ll only see the messages if they come in while I’m already polling for other things.

I have changes ready that customize the attributes and exposed sensors based on the model (GS/3 AV, MP, or LM), but it relies on the string that I get from the gateway. What does your model name look like here?

Based on the data above and model features, I’m only showing attributes for “key 1” for the LM and GS/3 MP, not showing the steam boiler sensor for the LM, and not showing the prebrewing switch for the GS/3 MP. I can adjust as needed.

btw @rccoleman, you were right. I was previously using ip_addr in the config.json file with an old version of test.py. I’ve updated to host and all is good. I also see what you mean about messages flying past with all the debug messages.

Mine looks like this:

Good to hear that it’s working. I’m forcing the logging level to debug in the main lmdirect library files, so I get a lot of logging.

You can remove this line from __init__.py and connection.py to remove the debug. I didn’t intend to check it in like that :slight_smile:

_LOGGER.setLevel(logging.DEBUG)

Excellent, thanks. I guessed “LM” and will udpate.

Edit: I have to say, having a good test suite is fantastic for testing configurations that I don’t have access to :slight_smile:

New lmdirect (0.6.1) and integration (0.7.1) versions released. Key changes are drink stats and model awareness. If you’re just using the integration, update in HACS and you’ll get everything.

If any of you guys with Linea Minis see one of these, could please post the content? I’ve only seen 2 of them and it looks like some sort of usage counter, but I can’t figure out what the numbers represent. I’ll start actively interrogating that command, but seeing some Linea Mini data would tell me if it’s related to the buttons on my GS/3 AV (like pulse count for water flow or similar).

Unexpected response: R00500018000001FD000001880000972800001CA4000094BF0000006BAC

It’s just being ignored for now and the error is just cosmetic, but I’d like to decode and integrate the date if I can.

Edit: Got some of it:

00500018 - Message
000001FE - Increased by 1 since this morning, and 8 since 12/31, can’t change it on demand
00000189 - Increased by 1 since this morning and 7 since 12/31, can’t change it on demand
00009772 - Number of seconds that the machine has been running, either coffee or hot water
00001CA9 - Total hours of on-time
00009500 - Number of seconds that the pump has been running
00000075 - Number of seconds that hot water has been dispensed

I feel like on/off cycles or days since manufacture or similar should be there, but I can’t find anything like that other than 0x189 (393). I can’t seem to affect it manually. I suspect that the first and second 4-byte words are number of auto on/off cycles, since they’re basically twice the number of days between readings, but I can’t get them to change by manually turning the machine on or off.

I do note that the numbers don’t actually add up based on those definitions, but 38810 (total run time) - 38182 (pump on time) - 117 (hot water running time) = 511, which is almost exactly 510 (usage_mystery). It’s not boiler refill (no effect), so I’m stumped for now. Further, 510 (usage_mystery) - 117 (seconds_water_on) = 393 (usage_mystery2)

I’m becoming convinced that usage_mystery and usage_mystery2 heating element on-time for the coffee and steam boilers. Still trying to figure out the math, though.