Trying to reach better precision and accuracy



My company is producing an app using the Structure Sensor which needs to display highly precise and accurate scans of details on a surface. I am aware of the limits of the accuracy associated with a single depth frame taken by the scanner (, but by taking many frames, aligning them with each other, getting rid of outlier points and doing some local averaging on the remaining points, I have been able to get a lot of the noise and systematic errors out of the scans to give a fairly smooth but accurate surface.

However, I am now at the stage where the remaining noise is fairly wide, and I cannot get rid of it easily without risking getting rid of real details in the surface being scanned. For example, when scanning a perfectly flat surface, I am getting fairly wide bumps and dips with heights of less than 1mm, but if I want to be able to capture a 0.5mm dent in the surface, the dents risks disappearing if I try to get rid of the wide noise just with algorithms.

That having been said, do you have any advice on getting rid of as much noise as possible from combined depth frames before I do the pruning and averaging? For example, I remember reading somewhere that hardware registered depth (which I am using) is less accurate than using the normal (not-aligned-to-color-camera) depth frames. Would it be worth me trying to use the normal depth frames if they are more accurate, and try to align them to the colour camera myself?

I’ve tried simply taking more frames and combining them, but I get harshly diminishing returns in terms of getting rid of the wider noise when I do that, so another solution would be preferred if one exists.

Some cropped screenshots to show what I mean:

All numbers are millimetre deviations from a flat plane defined by three points on the table in the scan, which will themselves have errors/noise. Number in the grid squares represent the most extreme depth within that square. As you can see, the depth on the flat surface’s cross section ranges from +0.5mm to -0.5mm, and can get more extreme in other parts of the scan.


Accurate scanning of a wall

First off, I’d like to say that this is very impressive!

Second, the only suggestion I would make would be to use only the depth image when making your scans, unless it is necessary that you also have color. Also, I think it would be very difficult for you to exceed our calibration techniques used in the Calibrator App by aligning the color image to the depth image yourself, though I’d like you to prove me wrong. :wink:

Third, you might want to play with the color and textures of the surfaces you are trying to scan on, as different levels of reflectivity will change the way the Structure Sensor performs.

Forth, you are already pushing the Structure Sensor to its limits by achieving +/- .5mm accuracy. Again, this is very impressive. You may not be able to increase the accuracy anymore before you prune and average the images.


Thanks Antony.

Unfortunately, the app users will need to be able to switch between color image and scan image, and so I do need to capture the first color image along with its depth frame (I do not use the rest of the color images associated with the next 24 depth frames, so I will double check that I am not wasting time capturing them). I agree that I cannot align the color image with the depth image better than the hardware-registered depth frames already do using the data from the Calibrator App.

As for the color and texture of the scanned surface, yes, that’s certainly going to factor into how much noise we get… Unfortunately, we’re once again limited by the app’s use, as the users will be scanning their own surfaces that are out of our control. The best we can do there is to at least find out how much noise there will be where the user scans the kind of surfaces we expect them to scan.

If there’s nothing more I can do to reduce noise of the depth frames individually directly, I guess I will have to use the frames more cleverly when joining them into a single point cloud.

Thanks for the advice!


Hello Sam. I am just getting started and found your post by searching “dent.” Is your app available? I’m basically trying to scan dents on a flat surface and get as accurate as possible with depth and a good visual representation.


Hi Phil,

Unfortunately our app is not yet available, and is currently tuned to a very specific industry, so there is a lot of extra stuff in there which is not relevant to capturing dent scans. I believe our company will be creating a more generic app with a downsized feature list so that users can just capture and download accurate scans of things like dents and damage, but I cannot say when that might become available. I’ll probably post again (either in this thread or in a new thread) when this happens.