Obviously, the dimmer is working fine, and there is something different between the setup on your benchtop vs installed position. It is likely to be 1) difference in line voltage, 2) difference in switch voltage, or 3) difference in the load. It makes sense to prove and then eliminate those differences… can’t be hiding in a magic tree, so a DMM should be enough to determine the problem.
Earlier you said your bulbs aren’t dimmable above, correct? That would be difference #3. My shelly dimmers puke when non-dimmable led bulbs are installed (the flicker and behave generally weird). If it works on the bench, make sure that same bulb is used in the install location.
Difference 1, may be possible if you’re hacking it into a 3-way switch system. Unfortunately, shelly dimmers require a hot line, which makes them not suitable for a lot of different 3/4-way wiring setups. In fact, rarely do you see 3-4way with the line and load in the same j-box (and even more rare an electrician runs an extra wire for shelly compatibility).
Difference 2, again might be the 3-way wiring problem like #1. Also, shelly dimmers do require 120vac on the switch inputs, unlike other shelly devices that can use a ‘dead switch’ (those measure continuity instead of current).
All of these possibilites should be easy to determine using a good DMM. Beware of parasitic voltages when you are using cheaper DMM’s to measure volts on house wiring. This can lead to your DMM displaying 120VAC, even though it is really 0VAC (induced voltage, due to nearby current carrying wires… not only fools you into thinking the breaker isn’t shutting off, it also provides no current for shelly to read). To see through these ‘ghost voltages’, you need an impedance to load the line down (like Flukes with a Lo-Z feature). Regardless what you use to test, please be careful near mains lines!
On an unrelated side-note: the above posted image shows technically correct wiring with respect to physics, but really the line wirenut should have 2 pigtails, one for the switch and one for the dimmer. Wiring the dimmer to the switch like that isn’t as efficient, and in some places that won’t pass building inspection. Even if you aren’t getting inspected… 9/10 times this kind of cheap/lazy wiring job (especially when it is done like that at every outlet) is why that nice air compressor keeps tripping the fuse breaker on a cold start, and the lights dim when the wife is drying her hair.
[edit: I should add that at first I had the shelly app configured to “edge switch”, which allows ‘standard wall switches’ to be used as normal. If you are using one of the other modes for momentary switches, your standard wall switches will behave weird (I’m guessing you already know this). Since then, I have my dimmer switches configured as “detatched”, and configured the “url when on/off” actions to hit a nodered http endpoint. That brings the switch position in to HA, so it can acted upon by automations. This offers more flexibility and reliability with automations, because you don’t have to worry what the shelly might be doing on it’s own.]