BC04P beacon and KSensor protocol without ESP32

[au78 (Au)]

I bought a Blue Charm Beacon BC04P. It showed up via iBeacon Tracker integration right away, but it is acting very erratic. While lying on the table in the same spot, distance fluctuates from 20 ft to 150 ft (true distance is about 15 ft), goes unavailable, then reappears. Walking around with it seems have no impact on the distance readings or availability.

I am able to connect to the beacon using KBeaconPro app. I made sure it’s always on (and not motion-activated) and I set the power to maximum.

Forums seem to suggest that the problem is in iBeacon Tracker integration and the solution is using the KBeacon protocol via ESPresense/ESP32s instead. Is there any way to make this beacon work with the bluetooth dongle I have in the HA PC?

[BlueCharmBeacons (Thomas)]

Hi Au,

I am not sure what is happening on the HA side, but here’s a few tips:

To check the signal strength of the beacon, use the scan page on KBeaconPro. It’s the first page that shows when you open the app. If there are too many things showing up there, and it gets difficult to see your target beacon, use the filter method at the top of the page so that only your target beacon shows on the scan screen.

Now put the beacon down somewhere and walk with your phone away from the beacon. The RSSI strength number should get weaker, i.e. lower and lower. For example, if it was -35 at the start, it will change to -43 then -56 then -68, etc. as you walk away from the beacon.

Be sure to restart the scan on the KBeaconPro app every so often since it will automatically time out. Just tap the “Scan” button in the upper right corner when it appears again (i.e. the scan has timed out). During the scan, the Scan button will change to Stop.

Be sure not to stay connected to the beacon with the KBeaconPro app. Staying connected stops the beacon from working. Connecting is only for configuring, then when done, you need to disconnect.

Assuming the RSSI numbers shown on the KBeaconPro app screen are behaving as expected, then the beacon is fine.

Re power setting for the beacon: You should not set it higher than needed. Default is 0, and it sounds like you have turned it up to +8. If you need it that powerful for your usage case, then fine. If not, I would recommend you turn it down to save battery power. Also, lower TX power levels are more stable. That said, this is not the reason why you are seeing unstable distance readings on your HA.

Some random speculation on the HA side:

First, a guess: You may have misidentified your target iBeacon when you first started. You might be tracking some other BLE item instead of your beacon.

Distance: The distance figures jumping around oddly could be caused by something wrong with your bluetooth dongle. These distance figures are calculated by your HA using a mathematical formula with the RSSI numbers as the key input. So if you already checked the RSSI numbers from the beacon (described above), and they are fine, then the only reason for the distance numbers to be weird is something being wrong with the dongle or maybe some strange thing happening with the HA. Some HA experts here can help on this.

Re: “the solution is using the KBeacon protocol via ESPresense/ESP32s instead”: Maybe this is just a wording mistake by someone, but this makes no sense and in fact is not possible. There is no such thing as “KBeacon protocol”. All our beacons use standard iBeacon protocol. They work fine with any scanner that scans for iBeacons.

Someone may have been having trouble with the iBeacon protocol because their scanner setup was wrong, then they switched on our KSensor protocol (broadcasts xyz accelerometer data) to try to fix the problem. Then after making twelve more adjustments to their scanner/HA , everything started to work, so they concluded that using KSensor was the magical trick. But I can assure you that it is not. Using our standard iBeacon protocol will work fine as long as the scanner is setup correctly.

By the way, I wrote a comment one time about how to set up and use the iBeacon Tracker integration. Hopefully it is still roughly the same process now: Anyone else experience issues with iBeacons? - #5 by BlueCharmBeacons

[au78 (Au)]

Thank you for such a detailed and helpful answer!

Indeed one of the problems was me trying to monitor the beacon simultaneously with the app and HA. Disconnecting from the app solved the issue of HA not updating at all. I reset the power to the default 0. I did not make mistake identifying the beacon. It has a very distinct name.

Although as you pointed out the “KBeacon protocol” is not a real thing, there seems to be some truth in recommendation to use ESPresense instead of HA’s iBeacon integration. I got an ESP32 device, flashed it with ESPresense and positioned it a foot away from the HA’s host computer’s bluetooth dongle which is used for iBeacon. That set up allowed me to track the same KBeacon simultaneously with both.

The difference is stark. iBeacon integration updates once a minute or so, shows ridiculously inaccurate distances, jumps back and forth while being stationary. ESPresense updates every second, is accurate with error of a foot or two, and doesn’t do unexplained jumps. However, it only tracks the beacon up to ~20 ft and then loses it. I played with all the settings in HA and ESPresense, no luck of extending that.

I got this beacon after reading “300 meters” in the description and planned to keep it in my car so that HA can detect it from blocks away as I’m returning home. It doesn’t look like it is remotely feasible to accomplish this in HA. The beacon seems to be a quality product and probably it can be used in such a way with some platform, but I can’t see how HA/ESPresense could pull this…

[BlueCharmBeacons (Thomas)]

@au78

I got this beacon after reading “300 meters” in the description and planned to keep it in my car so that HA can detect it from blocks away as I’m returning home.

For the record, you need a Phy scanner to achieve 300 meter distance. There are no ESPs or dongles on the market that are Phy scanning capable. Only very specialized BLE scanner gateways are Phy capable at this time.

From our product description:
" This beacon can achieve ranges of up to 300 meters using its PHY-coded broadcast and a PHY scanner, 150 meters using the legacy iBeacon broadcast and a typical BLE scanner such as a smartphone."

Re the scanning reliability issue:
I have successfully used the iBeacon Tracker integration with my HA as the scanner (see my link in my earlier comment). My HA is an rPi with built-in BLE, so nothing special. It doesn’t even have an external antenna. I get pretty stable distance readings from it, but the triggers for automations are a bit slow for my taste. Still, it works fairly well. So I still would be suspicious that the problem lies with your BLE dongle.

Using my HA as the scanner and the iBeacon Tracker integration, I get “coming into range” notifications/triggers within a couple of seconds every time. So definitely the integration is not scanning only once per minute like you experienced. That makes me think again that your dongle is the root of the problem. The dongle should also have a scanning interval and scanning window setting. So you might look into how to adjust those numbers for your dongle. then it will scan more frequently and hand off results to the HA equally frequently. Set it to something like scan interval 1000ms and scan window 800ms.

On the other hand, if you got the ESP working now, then maybe stick with it!

Re the short 20 ft scanning range of the ESP:

Isn’t there a “maximum distance” thing that you need to adjust in the ESPresence settings? Something like “ignore all beacons beyond X feet”. Maybe that needs to be turned off or adjusted so you can get longer distance readings.

I get the feeling that the ESPresence was designed as a room by room scanner, so I don’t think they worried too much about super long distance scanning. Most of the people that contact me about it are concerned with the scan zones being too large not too small.

Another idea to consider is getting ESP hardware with the ability to add an external antenna. I think they usually have a “U” at the end of the product number, indicating they require an external antenna. I once played around with some ESP32WROOMU modules and attached an antenna and got improved distance results.

Something like this: https://www.amazon.com/gp/product/B09D3TGJGD/

But be careful since this is a C3 variant which is different from the typical ESP32D variant. I am not sure if ESPresence can be flashed and run on this one. check with the ESPresence gurus on this before spending money.

For antennas, I first used these below, then later switched to slightly longer ones I bought somewhere else:

https://www.amazon.com/gp/product/B07D1Z3W7T/

[au78 (Au)]

When I set up ESP32s I set the maximum distance to 100 meters for all of them. Yet the furthest they register of any beacon (and they pick up like 30 in my house - I have no idea what half of them even are) - is 10.5 meters. I tried iPhone, Blue Charm Beacon, Govee hydrometer - they are gone when beyond 10 meters or so…

[BlueCharmBeacons (Thomas)]

On the ESPresence config page, it has these settings:

So you can try to change:
Maximum distance to report: 500
Skip reporting if message age is less than this (in milliseconds): 10
Report early if beacon has moved more than this distance: 0

[au78 (Au)]

I set ESPresense to 500 but still can’t get anything register beyond 10 meters.

I replaced the HA BT dongle with a “long range” one which has external antenna and promises 100 meters. It now detects the BC04P from further (maybe 30 meters) but readings are absolutely wacky. It only updates once a minute at most and registers wild jumps. In the meantime, ESPresense gets ok readings every second from the same beacon.

Here are the readings by ESPresense sensor (purple) and iBeacon of the same BC04P (I only have one). It’s sitting ~4 ft away and not moving entire time. ESPresense and iBeacon BT are a foot apart.

Absolutely insane!

I don’t know what else to try. The choice seems to be either ok readings from ESPresense with 10m distance limit, or super slow and wildly inaccurate iBeacon readings which reach 30m. And I hoped to get readings from 100m away every second…

[BlueCharmBeacons (Thomas)]

Re ESP: Maybe ask some of the ESPresence experts in the forum for their ideas on how to adjust it.

Re the dongle: The BT dongle looks slightly hopeless from that chart. The distance reading jumping from 2m to 8m then back to 2m is basically worthless.

Checking the beacon: Just to double check that you did not accidentally change any of the beacon settings, you can use the KBeaconPro app to connect to it then click “reset configuration” to set it back to its default settings.

[otter]

I figured I would just reply here since I am working with the same BC04P beacon. I was hoping to setup the motion feature and use iBeacon integration.

I think the beacons are working fine. Just curious what the right slot config is to get home assistant to know when motion occurs? @BlueCharmBeacons do you happen to know?

Thank you !!

[otter]

Just to reply to myself. I have slot 0 setup normal ibeacon. It works. Then I setup another slot as ibeacon, point a trigger for motion at it. I see the other UUID there.

Its is definitely triggering an update when I move it (i reduced the motion sensitivity a little because it was going off constantly). Problem is I see two beacons listed now BCPro_196532 4223, but they both update every minute and when motion occurs. So that isn’t too helpful. I actually only have the UUID of the motion triggered slot setup in ibeacon… i removed the default one. It also seems to update about 14 times after motion. in the span of about 10 seconds.

So I think I am close. I just need to have one UUID that is for motion and the other default one just for distance every minute.

[otter]

Ya, it all seems to work other than the fact that I get updates on the motion “slot” every minute even thought i set the “adv interval” to 40k ms and “trigger only adv” to yes.

[BlueCharmBeacons (Thomas)]

You just need to set up slot zero with iBeacon. Set it to advertise always. That means it will normally advertise the standard UUID but after sensing motion, it will advertise a slightly different UUID. there is no need to set up another slot.

Once the motion trigger is activated by motion, it will advertise for a certain period of time according to the trigger settings.

If you take a look at the Quickstart guide for this beacon under the motion trigger section, there are detailed instructions explaining all of this.

[otter]

Not having much luck. I do get two different UUIDs, but the behavior is whacky. Both UUIDs update every minute. Sometimes motion does nothing.

[otter]

I may actually have it… im not sure. I now see a uuid ending in F5 that acts like a normal beacon and one that is F6 (which i never set) that is motion I think.

edit: nope… .still seeing updates every minute on F6. I’ll just have to tinker with this I guess unless I have a setting wrong. It seemed to work there for a minute and now its just a beacon again.

[otter]

It continues to broadcast on all sorts of UUIDs that are no longer even in the settings. I fully reset the thing and started over and I have updates on F4, F5, That 77777 uuid… it was doing the incremented UUID thing even with " Advertisement Change" off. In fact I had never turned that on when I noticed the F5, F6 etc happening. This thing doesn’t seem to update…even after power cycling.

[BlueCharmBeacons (Thomas)]

OK, your screenshots look fine. The way you have it set:

If your regular SLOT broadcast is iBeacon, after sensing motion, the iBeacon broadcast will have a slightly modified UUID. The slightly modified UUID will have one hex added to the last digit of the UUID. This modified UUID will be broadcast until the Trigger Adv Time expires (see below).

For example:

Regular UUID: 426C7565-4368-6172-6D42-6561636F6E73
UUID after motion trigger: 426C7565-4368-6172-6D42-6561636F6E74 (notice the last digit of the UUID has increased by 1).

On your history screenshots, I see UUIDs at the bottom. One ends in f4 and one ends on f5. I am not sure though where the f4 UUID came from.

On your config screenshots, I can see that the real UUID of your beacon ends in f5. So any motion triggered broadcast will change that UUID to one that ends in f6. No idea what the f4 broadcast is. maybe another beacon or some other erroneous setup?

So f5 is the regular broadcast, and f6 is the motion broadcast.

[otter]

Here is my config, slot 0 ibeacon, NO triggers setup. I left it over night. I get data on ALL these UUIDs. I need to factory reset this thing or something. I tried many configurations yesterday and almost saw it work right a few times and then it would just revert to beacon all the time. I think its cooked.

[otter]

even when i reset it, turn it back on… i still get data on those UUIDs. Not having very high confidence in these beacons. It’s about to go on the failed HA project pile.

[BlueCharmBeacons (Thomas)]

It’s extremely unlikely the beacon is changing its own uuid if no trigger is set. Even with a trigger set, it changes in a very specific way.

One thing to check first though: do you have a button trigger enabled? I do t think it is possible but please double check.

Also, did you try to reset configuration yet? It’s a menu choice on the first screen of the app after connecting. If not, do that now just to be sure.

My best guess is this is some sort of phantom data collection bug of the integration. There have been many cases in the past where the beacon integration gathered lots of erroneous data.

Try to use our app or some other beacon scanning app and do the same experiment (for a shorter time of course); leave the beacon in standard broadcast with no triggers like you did overnight. Then just visually watch the broadcasts received on the scan screen of the app. I am very confident that you will only see one single uuid. Note that only Androids will show the uuid; iPhones won’t show it due to restrictions set by Apple.

[BlueCharmBeacons (Thomas)]

To add a little more context:

The iBeacon integration has a feature to help it deal with smartphones that use randomly generated changing MAC numbers: it stores the data and then tries to consolidate multiple scans together once it concludes it is seeing a virtual beacon broadcast from a smartphone. Cool trick. But this storing of past data can lead to other problems, for example, the one you are experiencing now (according to my guess).

You can read about this feature in the docs: iBeacon Tracker - Home Assistant

Come to think of it, I’m not sure if the motion sensing feature of our beacons will work with the iBeacon integration. Each beacon has a fixed MAC and two different UUIDs when using motion sensing. I’m not sure how the integration will view that. You should check with people more familiar with that integration for more info.