Testing of laser scanning reference spheres - A comparison

In the scope of a comparison of different reference spheres, we tested how the single spheres behave in the sun and in the shadow. Position deviation and total number of scan points were subject of the investigation. Test objects were a sphere offered by FARO as well as four reference spheres manufactured by Laserscanning Europe: Basic, Flexi, UltraX and Traveler.

Test procedure

Specifications for test procedure

Number of points and position deviation were made out within three different set-ups of laser scanner (FARO Focus3D) and reference sphere:

• For the first test, the scanner was positioned in the sun and the spheres were placed in the shadow.
• Test 2 was realized reverse; that means the scanner was in the shadow und the reference spheres were positioned in the sun.
• During the third test set-up, both reference spheres and scanner were placed in the sun.

During all three tests, scanner data captured in different resolutions (1/8 to 1/1), qualities (1x to 3x) and distances (5 meters to 20 meters). Single configurations were chosen correspondingly to the common settings in practice. Number of points on the sphere and position deviation were collected as reference values. Moreover, a distinction is made between manual and automatic registration of reference spheres.

Analysis of automatic registration for all types of spheres

Analysis of manual registration for all types of spheres

Peculiarities during procedure

Automatic detection of FARO spheres turned out to be an arduous procedure. In contrast to other spheres, you often get satisfying results not before the third or fourth attempt. In some cases, we had to accept deviations of more than a centimeter. According to our observations, the deviation is primarily caused by the base which affects registration very negatively.

Result of comparison

Analysis of overall results

To begin with, we consider the overall result so that we will identify which spheres is a real all-rounder.

For this purpose, we added the numbers of scan points of the whole test for each reference sphere and represented them in a chart. In case of automatic registration, the reference sphere UltraX is in the lead. On the contrary, FARO sphere lags far behind (10% points less). The result of the average position deviation looks similar: FARO sphere shows the largest position deviation with 2.6mm whereas UltraX sphere is the most precise sphere with just 2mm deviation.

Also in case of manual detection, FARO sphere provides the lowest number of scan points in total. Providing almost 9400 points, reference spheres Basic and Flexi are outstanding. In contrast to automatic registration, average position deviation of manual registration is about 50% smaller. FARO sphere (1.14mm on average) and Basic sphere (1.19mm) make biggest deviations. The best result is achieved by UltraX (1.01mm). It is also striking that the total number of points for manual registration is significantly lower than for automatic registration.

Overall result of analysis for automatic and manual registration for all types of spheres

Correlation between quality and position deviation

In all tests, different settings for resolution were used. For each resolution setting, we performed scans with two different degrees of quality. It is striking that position deviation decreases in almost all scans and for all types of reference spheres hand in hand with increasing scan quality.

However, if we consider the impact of quality on the number of scan points, we note that there hardly any differences between higher and lower quality. In case of automatic registration, differences are so small that they are not focus of interest. In the manual registration, significant differences result from the size of the chosen circle; there, the human being has big influence on the number of scan points.

Correlation between total scan points and position deviation

One would expect that sphere detection improves hand in hand with increasing number of scan points. But the tests have shown that this assumption is true only to a limited extent. The rule is: The higher the resolution, the higher the number of scan points. As a result of all scanner/sphere set-ups, you get the best detection quality when using 1/4 or 1/5 scan resolution. If you choose a low resolution (that also means few scan points), detection accuracy is low as expected. However, if you choose 1/2 or 1/3 resolution, scan point deviation increases.

If we consider the aspect of scan speed (quality) additionally, we should differentiate between manual and automatic sphere detection. For manual registration, you generally achieve less scan point deviations with increasing scan quality. By contrast, automatic detection results in either identic or even larger deviations when quality is 3x. We could only make out in few exceptions that higher quality improves position deviation.