Calibration & auditing

A single BlueZoo sensor can accurately count multiple thousands smartphones in a detection zone, whether the small space in front of in-store signage, the medium space in around street furniture, or an enormous space such as an entire home improvement retail store or hotel ballroom.

One challenge of counting the number of phones in a space is that spaces and phones vary. Some spaces are surrounded by materials that block Wi-Fi signals, while others allow Wi-Fi signals to pass freely. Some spaces have internal obstacles (e.g. metal filing cabinets), while others do not. Some phones probe more aggressively and with higher power than other phones… though the variation among phones is surprisingly small. Further, some people may have more than one phone, or no phone at all.

As a result, BlueZoo sensors operate best if they are calibrated, for which BlueZoo offers three independent mechanisms: Good, Better, and Best.

Good calibration, known as distance-based calibration, measures the distance from the sensor’s antenna to the most distant smartphone to be detected. Using a power-distance curve (that BlueZoo has measured for every antenna type that BlueZoo provides), a user can convert distance to minimum Wi-Fi signal strength.

Better calibration, known as range-detection-based calibration, is achieved by having the BlueZoo sensor measure the strength of smartphone signal on site. An installer, typically a BlueZoo customer or a systems integrator, carries a smartphone running the BlueZoo mobile app along the perimeter of the detection zone. The sensor records the signal strength of the smartphone as received after passing through filing cabinets, plants, and any furniture that might be present. The BlueZoo cloud software calculates received signal strength along the perimeter, and allows the installer to establish a minimum signal strength empirically.

BlueZoo’s Best calibration, known as ground-truth-based calibration, compares the true number of persons in a space with the Wi-Fi probes collected by BlueZoo sensors. Ground-truth-based calibration represents the gold standard and provides the technology for accuracy audits.

Generally, installers collect ground truth on multiple occasions, including when a space is crowded and when it is less crowded. The BlueZoo mobile app permits multiple ground truth collectors to simultaneously collect ground truth, for example at different entrances to a very large space. Our systems collate this ground truth and compare it to the values detected by our sensors to calibrate sensors for optimal accuracy. Small spaces can generally be calibrated by a single ground truth collector.

Once ground truth is collected, an administrator selects desired ground truth records and runs a calibration job to generate a series of calibration candidates. These calibration candidates combine attributes including minimum signal strength, gain, and a specific computational method. Since our founding, BlueZoo has refined a series of computational methods which reflect how people move in a half-dozen behavioral clusters. Each candidate employs one of these methods representing all similar behavioral clusters.

For each calibration candidate, BlueZoo calculates the mean square error across all sample points. These candidates take into account people with more than one phone or no phone, for example. Generally, the administrator will select the candidate that maps the sensor’s measurements most closely to the collected ground truth.

Each distinct sensor environment needs to be calibrated, and these calibrations can be applied to all similar sensors. For example, most retail stores have standard store layouts with standards for aisle width, department size, and check out areas. Once the combination of layout and BlueZoo sensor/antenna are calibrated, that combination of computation method, gain, and minimum signal strength can be applied in bulk to other sensor locations.

Calibration is usually performed once, but a succession of calibrations can also be performed. Sometimes, hardware installers will initially set a simple distance-based or ground-truth-based calibration for a sensor. This calibration can be refined with additional calibration performed during periods of high and low foot traffic.

The same ground truth collection enabled by the BlueZoo mobile app can be used for auditing. An auditor collects ground truth. But rather than use this ground truth to calibrate the sensor, the ground truth is compared to the sensor’s output to calculate the degree of accuracy. We recommend annual audits or periodic audits on a subset of sensors to record the accuracy as the physical layout of the environment evolves.

Auditing is a necessary part of any audience measurement solution. Thanks to our realtime audience measurement, auditing is possible unlike SDK-based “annual averages” that insist that advertisers simply trust the process; and thanks to our mobile apps, auditing is also easy.