Recently, I was trying to use depth information to reconstruct a point cloud. As far as I know, it should be pretty straightforward: you read the depth files as a matrix 240x320 of unsigned 16-bit integers.
I am modifying Scanner app to get depth information. I am using STStreamConfigDepth320x240 and storing depthFrame.shiftData.
Then in python I am doing this:
python import numpy as np import sys from cv2 import imwrite def main(): if len(sys.argv) < 3: print('Usage: python ./depth_read.py FILE OUTFILE.PNG') return srcpath, outpath = sys.argv[1:] x = np.fromfile(srcpath, dtype=np.uint16) x[x == 2047] = 0 x = x.reshape((240, 320)) imwrite(outpath, x) if __name__ == '__main__': main()
I can assure you, that I double checked the cv2’s imwrite and it saves the PNG image with bitdepth 16.
Then I used this script:
(overall I’m hoping to use PCL library for my project)
providing the camera angles for the structure sensor: vertical 45 and horizontal 58, according to this topic: Structure Sensor’s Angle of View
My problem is shown below: even for one frame there are some very weird angles of the items I scanned. In the first example u can see that a floor I pictured turns out to be heavily bent in the point cloud. Basically, there is a can of coke on the floor.
I was moving camera from right to left and I didn’t apply any transformation to point cloud. Here it is how it looks when multiple frames are combined (all of them are bent):
This is data captured by sensor(uint16 png image):
With high contrast:
The brighter, the further it is. The first depth picture is very dark because all the values are somewhere around ~700 whereas obviously 16-bit PNG file make it just about 1/100 of the brightness (700/65535).
I suspected that there may be something wrong with the angles I provide or focal lengths (I’ve also tried using focal length) etc needed to calculate the correct placements of points in the 3d space. I still suspect it should have something to do with that, since I believe that the depth itself is properly calculated. Is there a way to sensibly debug it?
Do you suspect what may be the problem? Could someone provide any guidance on what errors did I actually make and how to fix this?