Create condition based on getting home

Configuration
1

I have a group of motion detectors outside. I want to create an automation that sends a notification via alexa TTS if the motion detectors are triggered between midnight to 6 am. However I want a condition that if me or my wife come home during that time, it does not activiate the notification. So inother words if the wife is home in bed asleep, and I come home at 1 am, it does not send the notification, but if I am not home or have already been at home all night, then it will send the notification. I cant figure out how to create the condition if I have only got home or been home for maybe 5 minutes.
We have been having a problem with people messing with vehicles in the middle of the night, I want an instant notification if someone is in the driveway. Cant stand thieves.

2 
conditions:
- condition: not
  conditions:
  - condition: state
    entity_id: device_tracker.your_phone
    state: 'home'
    for: '00:05:00'
  - condition: state
    entity_id: device_tracker.other_phone
    state: 'home'
    for: '00:05:00'

This will be true (allow the automation to continue) if either of you have been home for less than 5 minutes (you’ve not both been home for at least 5 minutes).

3

I came up with this, not sure if it is going to work. Created a toggle that turns on between 12 am and 6 am, waits 5 minutes, then turns itself back off, then created the automation that runs if the toggle is on.

alias: Someone came home in middle of night turn on boolean
sequence:
  - service: input_boolean.turn_on
    data: {}
    target:
      entity_id: input_boolean.someone_came_home_input_boolean
  - delay:
      hours: 0
      minutes: 5
      seconds: 0
      milliseconds: 0
  - service: input_boolean.turn_off
    data: {}
    target:
      entity_id: input_boolean.someone_came_home_input_boolean
mode: single

then the automation to turn on the script

alias: Motion in driveway Jonas came home at night
description: >-
  Turn on Boolean to prevent automation that someone is in driveway at night,
  boolean will reset itself after 5 minutes
trigger:
  - platform: state
    entity_id:
      - person.jonas
    from: not_home
    to: home
    id: jonas came home
condition:
  - condition: time
    before: "06:00:00"
    alias: Between 12 am and 6 am
    after: "00:00:00"
action:
  - service: script.someone_came_home_in_middle_of_night_turn_on_boolean
    data: {}
mode: single

and the automation if motion is detected

alias: Motion in driveway that's not supposed to be
description: ""
trigger:
  - platform: state
    entity_id:
      - binary_sensor.driveway_motion_detector_group
    from: "off"
    to: "on"
condition:
  - condition: time
    before: "06:00:00"
    weekday:
      - mon
      - tue
      - wed
      - thu
      - fri
      - sat
      - sun
    alias: 12 am to 6 am
    after: "00:00:00"
  - condition: state
    entity_id: script.someone_came_home_in_middle_of_night_turn_on_boolean
    state: "off"
action:
  - service: notify.mobile_app_jonas_phone
    data:
      message: TTS
      data:
        ttl: 0
        priority: high
        media_stream: alarm_stream
        tts_text: Someone is in the driveway and probally up to no good
  - service: notify.alexa_media
    data:
      message: Someone is in the driveway and probably up to no good
      data:
        type: tts
      target:
        - media_player.alexa_garage
        - media_player.alexa_livingroom
        - media_player.alexa_bedroom
mode: single