Adding entity to scene adds all entities for device

I have an ESP32 running esphome. It controls 2 windows, an overhead door, and a few other relays. I want to set some scenes for controlling the windows. The issue is, when I add one of the windows to the scene, it auto pulls in everything on the esp32. I don’t want the other things to be affected by the scene. What am I missing?