How does 'Detect Inner Pixels' work?


#1

Hello,

I am using some code similar to what is described by Jim in this post, in order to have the wireframe cube render a set distance from the iPad during cube placement (I also ignore gravity so that the cube is always in the same pose with respect to the iPad during cube placement).

If I am using STCubeRenderer.renderHighlightedDepth and STCubeRenderer.renderCubeOutline to display the wireframe cube (and the sensor detection within) with my own calculated cube pose, can I still use STCameraPoseInitializer.detectInnerPixels to get the mask that described which pixels in a depth frame were within the cube, while scanning?

When I try it, I cannot get it to work (all mask values come back zero), so can someone please explain if and how I can use STCameraPoseInitializer.detectInnerPixels while scanning and getting depth frames in my case?

Thanks!


#2

This may or may not help, but upon looking up the definition of detectInnerPixels it states that “mask pixel will be 255 if the corresponding 3D point fits inside the volume, 0 if outside or if there is no depth.” Maybe you aren’t referencing a pixel that is within the set Bounding Box?


#3

Thanks for replying, Antony!

When I am using the default cube placement code, I actually get 0 and 255 values as one would expect when I loop through all the pixels in the image, so the mask appears to work in that case. It’s only when I change how the cube placement works that the mask gives zeroes for all pixel positions. Maybe when I change how the cube placement works, it messes with how some of the hidden detectInnerPixels code works?


#4

I’m also seeing the same thing here Sam. When I modify the cube placement strategy and code detectInnerPixels gives me all 0s.