Can execute automations but don't run automatically

The automation’s condition will only allow the action to be executed if device_tracker.wills_pixel_5 is not home.

Are you testing this automation while your pixel_5 is not home?

Yes, I am away from home at the moment.

When the binary_sensor changes from off to on does the automation update its LastTriggered time? It’s displayed next the automation’s name in Configuration > Automations.

As I mentioned before you can use a service call to troubleshoot this. Now I am at my PC here is more information (courtesy of Tinkerer)
To test an automation there’s three stages you can follow. Testing the action, the condition and action, and the whole automation:

  1. Use Developer tools ()-> States to find the automation, click and then push Execute. If this fails your problem is in the action: section, and details should be found in your log file
  2. Use Developer tools -> Services and call automation.trigger on the automation with skip_condition: false. If the first passes but this fails then the problem is in your condition: block
  3. Use Developer tools -> States to find the trigger entity, click the name, then change the state (at the top) to something that’ll trigger the automation before pushing Set State. If this fails then the problem is with your trigger: section.

If they all work, and your automation didn’t run, it’s probably turned off.