SDK 0.6 Fix the Cube at a certain distance from the user


#13

hi again,
i tried something more arround and with the 0.9SDK i got it to run but just while the CubePlacementState

when i hit Scan, in enterScanningState at position:

_slamState.tracker.initialCameraPose = _slamState.cubePose;

my App crashes.

i can’t find any further error in implementing the codes that mentioned above


#14

Are you using STCaptureSession or the low level interface?


#15

I’ve found a solution which works for me i guess, the cubePose was everywhere NaN so i removed the adjustments from Step 7 of your description in Post 2 and in EnterScanningState i’ve just replaced the
_slamState.tracker.initialCameraPose = _slamState.initialDepthCameraPose;
with
_slamState.tracker.initialCameraPose = _slamState.cubePose;

without the codelines you mentioned at Step 7 in your description. Do i have to expect any issues without your codelines? Scan seemed to be OK until now

As i am a very beginner to that topics i assume that i am using the STCaptureSession.

I’ve took the Sample Scanner App (SDK 0.10) without any adjustments on the settings.

Best Regards

Thorsten


#16

The STCaptureSession based Scanner sample in a recent SDK was using an unitialized instance of the sensor controller. I’ll put together some diffs to share what I’ve done and post here today.

-jim


#17

@thorsten.steparsch try applying this patch to the Scanner sample. Note that it also has the voxel size set to 1mm.

Best regards,

Jim


#18

Hi Jim,

this works fine for me, thank you very much.

My Previous solution still works fine, do you think i will face any problems with keep using it as described in Post 15?

Best Regards

Thorsten


#19

Some changes in the latest SDK update (0.10.2) may have broken my solution, but I haven’t had a chance to look at it.


#20

I’ve applied your patch to the 0.10.2SDK in the Scanner Sample App and tried to do some changes to make it work (to be honest i don’t really know what i have done) :wink: .

    // Prepare the mapper for the new scan.
[self setupMapper];

STDepthFrame* depthFrame;
//    _slamState.tracker.initialCameraPose = _slamState.initialDepthCameraPose;
GLKMatrix4 depthCameraPoseAfterTracking = [_slamState.tracker lastFrameCameraPose];

GLKMatrix4 iOSColorFromDepthExtrinsics = [depthFrame iOSColorFromDepthExtrinsics];

GLKMatrix4 colorCameraPoseAfterTracking = GLKMatrix4Multiply(depthCameraPoseAfterTracking,
                                                             GLKMatrix4Invert(iOSColorFromDepthExtrinsics, nil));
// Make sure the pose is in color camera coordinates in case we are not using registered depth.

//GLKMatrix4 colorCameraPoseInSensorCoordinateSpace = GLKMatrix4();
//[[STSensorController sharedController] colorCameraPoseInSensorCoordinateFrame:colorCameraPoseInSensorCoordinateSpace.m];

bool invertible;
GLKMatrix4 colorCameraInversePose = GLKMatrix4Invert (colorCameraPoseAfterTracking, &invertible);
if (invertible)
{
    _slamState.cubePose = GLKMatrix4Multiply(_slamState.cubePose, colorCameraInversePose);
    _slamState.tracker.initialCameraPose = _slamState.cubePose;
}

// We will lock exposure during scanning to ensure better coloring.
_captureSession.properties = STCaptureSessionPropertiesLockAllColorCameraPropertiesToCurrent();

_slamState.scannerState = ScannerStateScanning;

in the End the Scanning starts, but after pressing the SCAN Button, the Box is hopping to the right and is not keeping the position before scanning. But currently i have no explanation for that.

Do you have an idea what the reason of this behavior could be?

Regards

Thorsten


#21

Hi @thorsten.steparsch

Does it look anything like the video in this thread (MKII Sensor Bounding Box Drift)?

If so, I think the consensus opinion is this is a bug with the new SDK/hardware, with no obvious or quick fix at the present time. We are collectively waiting on verdict and/or assistance from Occipital.

-Andy W


#22

GLKMatrix4 iOSColorFromDepthExtrinsics = [depthFrame iOSColorFromDepthExtrinsics];

Is equivalent to:

GLKMatrix4 colorCameraPoseInDepthCoordinateSpace;
[depthFrame colorCameraPoseInDepthCoordinateFrame:colorCameraPoseInDepthCoordinateSpace.m];


#23

Hi @andy.warwick

no it is just a one time hopping to the right, then the Box is fixed again.

When debugging SampleScanner App with the Changes from Jim via XCode i can see that this hopping happens when this line of code is applied in file EAGLView.mm in method (BOOL) presentFramebuffer:

success = [context presentRenderbuffer:GL_RENDERBUFFER];

In addition to that, today i recognized, that in the first case of Scanning the hopping is just small. then when i hit the restart button (not have pressed Done Button) and now hit the Scan Button again, the hopping is a lot more than before. If you do this again, the hopping is small again, than again big and so on…

Here i’ve uploaded the Video Hopping

In the end, now I’m really confused.


#24

I too get a one-time-hopping while trying to implement this fixed stuff with the new SDK.

Except mine jumps to the left and actually shrinks by half volume…I have a feeling I’m over-applying the offset values to the bounding box rendering (dividing x & y by 2 for the cubePose). I’ve been moving alot of stuff around - I too have little understanding of what I have done, taking notes - but I think that I am mis-applying/extra-applying that offset at the enterScanningState phase. I can get it to shrink again and again (more and more) by restarting the scanning process - it just keeps shrinking and moving to the left. So my theory is these one-time-hops occur at the moment of scanning and the mis-applied offset is compounding on each restarted scan. Hope this helps.


#25

@andy.warwick and just to confirm, I really appreciated seeing your documentation of the Bounding Box Drift! I am experiencing that as well! I believe these are separate issues though they could be related in some way. For me the Bounding Box Drift is more chronic/passively always occuring - it seems to be affected by which kSTCameraPoseInitializerStrategyKey I use (mine is STCameraPoseInitializerStrategyGravityAlignedAtOrigin which I can confirm is drifting the bounding box with the new sensor)


#26

@michael.p

Looks like using the new 10.3 SDK, along with using STCameraPoseInitializerStrategyTableTopCube, helps a lot with the drift. I can get scans out of the new hardware now without in happening every time.

Unsure if that key alone fixed it, if it’s the new SDK, or it’s something else I have changed.

Just hope I haven’t jinxed it. :wink:

-Andy W


#27

Had anybody a Chance to have a look on the hopping stuff? I’ll try nearly anything without having any success. :frowning:


#28

Finally had a moment to look at your video, that behavior is quite repeatable! Are you using the low level scanner sample as a starting point?

-jim


#29

I’m using the Scanner App from the Samples folder in the SDK, not the LowLevel Samples.


#30

Hello,

Has anyone found a solution to the one-time-hopping box? Its happening to me too where when enterScanningState is called, the scanning box bounces slightly to the right and then remains fixed.


#31

Unfortunately at all, i don’t know the reason for the bouncing/hopping whatever.

Could you try to adjust your enterScanningState method with those two lines directly after the setupMapper call:

_slamState.initialDepthCameraPose = _slamState.cubePose;

_slamState.tracker.initialCameraPose = _slamState.initialDepthCameraPose;

it seemed that in my case it helps that the hopping is avoided. But, as mentioned, i don’t know why.
Now i have a different problem, so that the Cube isn’t directly in the middle anymore.

Maybe this helps you, maybe not.

Thorsten


#32

In ViewController+OpenGL.m -> renderSceneForDepthFrame() when in ScannerStateCubePlacement:
_slamState.cubePose = GLKMatrix4Multiply(cameraViewpoint, iOSColorFromDepthExtrinsics);

When you enterScanCube set _slamState.tracker.initialCameraPose = _slamState.cubePose

If you scanning cube jumps, you are not initializing the tracker’s initialCameraPose correctly.