📦 AI Package Detector

Blueprints Exchange
, ,
1

:package::mag: AI Package Detector Blueprint

Open your Home Assistant instance and show the blueprint import dialog with a specific blueprint pre-filled.

Detect and get notified when packages appear or disappear at your doorstep! This blueprint uses camera snapshots and the Google Generative AI integration to intelligently analyze your camera feeds and alert you to changes in package status.

Version: 1.2 (2025-03-18)
- Added package status helper to track package presence as an on/off state
- Fixed issues with package detection accuracy
- Enhanced prompt response consistency
Version: 1.1 (2025-02-18)
- Improved prompt
Version: 1.0 (2024-07-24)

Overview

This automation analyzes images from your cameras, compares them to the previous state, and sends notifications when changes in the number of packages are detected. It can be triggered by a time schedule or by optional motion sensors. It’s designed to be flexible and robust, handling multiple cameras and potential errors.

Key Features

  • Intelligent Package Detection: Leverages Google’s Generative AI to accurately identify packages in camera snapshots.
  • Customizable Analysis Area: Specify details about the monitored area for improved accuracy (e.g., “Focus on the blue door and welcome mat.”).
  • Flexible Triggering: Runs on a user-defined schedule (e.g., every 30 minutes) and/or can be triggered by motion sensors.
  • Multiple Camera Support: Monitor multiple cameras simultaneously.
  • Detailed Notifications: Provides notifications with a description of the detected packages and an image of the best view.
  • Persistent State Tracking: Remembers the previous package count and description for accurate change detection.
  • Error Handling: Includes error handling for cases where the AI model returns an unexpected response.
  • Mobile App Notifications: Sends push notifications to the Home Assistant mobile app.

Requirements

  1. Google Generative AI Integration: Must be configured in Home Assistant.

    • Recommendation: For best results, configure the Google Generative AI integration to use the Gemini 2.0 Flash model.

  2. Home Assistant Mobile App: Required for receiving notifications.

  3. /config/www/ directory: This directory must exist for storing temporary camera snapshots. It is typically created automatically. If it doesn’t exist, create it manually.

Important Notes

  • Accuracy: The accuracy of package detection depends on the quality of your camera images and the capabilities of the Google Generative AI model.
  • API Usage: This blueprint utilizes the Google Generative AI API, which offers a free tier suitable for typical home use. Users can optionally choose a paid plan for higher usage limits or enhanced privacy features. Refer to Google’s Generative AI documentation for details on pricing and tiers.
  • Optimization: Adjust the Check Interval and Image Analysis Details to optimize performance and accuracy for your specific setup.
  • Package Description Helper: The Package Description Helper input_text entity must be created with a maximum length of 255 characters to prevent truncation of the AI-generated context.
  • Temporary Images: This blueprint uses the /config/www/ directory (accessible via /local/ in URLs) to store temporary images. These images are not automatically deleted. You may want to implement a separate automation to periodically clean up this directory if desired.

Blueprint Configuration

When you import the blueprint, you’ll be presented with the following configuration options:

  • Cameras: Select the cameras that monitor your delivery area. (Multiple cameras are supported.)
  • Optional Motion Sensors: (Optional) Select motion sensors that are linked to your cameras. This will trigger the package detection when motion is detected (after a short delay).
  • Image Analysis Details: Provide specific instructions for the AI to focus on particular areas in the image (e.g., “Focus on the blue door and the welcome mat.”). This helps improve accuracy.
  • Check Interval (Minutes): How often to check for packages (e.g., /30 for every 30 minutes). This sets the time_pattern trigger.
  • Notification Target: Select one or more devices running the Home Assistant mobile app to receive notifications.
  • Package Count Helper: Select an input_number helper entity to store the last known package count.
  • Package Description Helper (Internal): Select an input_text helper entity to store the AI’s context. Important: Create this helper with a maximum length of 255 characters.

How It Works (Technical Details)

  1. Triggering: The automation is triggered either by a time_pattern (scheduled check) or by a state change of your chosen motion sensors (to “on”). A 5-second delay is included after motion detection to allow the camera to capture a clear image.

  2. Snapshot Capture: For each selected camera, a snapshot is taken and saved to the /config/www/ directory. The filenames are dynamically generated based on the camera entity ID.

  3. AI Analysis: The Google Generative AI service is called, passing in the image filenames and a carefully crafted prompt. The prompt includes:

    • A list of the image filenames.
    • The previous package count and description (from the helper entities).
    • Your custom Image Analysis Details.
    • Instructions for the AI to return a specific, easily parsed format: count|human_notification|machine_context|best_image_filename.

  4. Response Parsing: The AI’s response is parsed to extract the package count, a human-readable notification message, a machine-readable context string, and the filename of the best image.

  5. Notification Logic:

    • If the parsed package count is -1 (indicating an error), a persistent notification is created in Home Assistant.
    • If the package count is greater than the previous count, a “Package Detected” notification is sent to your mobile app, including the best image (if available).
    • If the package count is less than the previous count, a “Package Removed” notification is sent.

  6. State Update: If the AI response was parsed successfully, the Package Count Helper and Package Description Helper entities are updated with the new values.

Troubleshooting

  • No Notifications:
    • Ensure the Google Generative AI integration is correctly configured and that you have API access.
    • Check the Home Assistant logs for errors related to the google_generative_ai_conversation service.
    • Verify that your cameras are working and that snapshots are being saved to the /config/www/ directory.
    • Make sure you’ve selected the correct notification target (your mobile app device).
    • Double-check that you’ve created the input_number and input_text helpers.
  • Incorrect Package Count:
    • Review the Image Analysis Details and adjust them to be more specific.
    • Ensure your cameras have a clear view of the delivery area and that lighting is adequate.
    • Experiment with different prompt wording.
  • Error Notifications:
    • The persistent notification will provide details about the error. This usually indicates a problem with the AI’s response format. Check that the response includes the 4 pipe-separated values.

I hope you find this blueprint useful! This is the first blueprint I have created and shared, so please be kind!

2

Updated to version 1.1.

I updated the prompt slightly to be way more reliable. I have been using this version for almost a week and haven’t gotten any false notifications.

3

Updated to version 1.2. I wanted the ability to have a persistent notification in my dashboard if packages were waiting to be picked up, so I added another helper variable. I was also having some inconsistent AI responses, so I slightly cleaned up the prompt.

**Version:** 1.2 (2025-03-18)
    - Added package status helper to track package presence as an on/off state
    - Fixed issues with package detection accuracy
    - Enhanced prompt response consistency