Structure SDK (for iOS) Release 0.10.3: iOS 13 Compatibility and Tracking Improvements


#1

Today we are releasing Structure SDK (for iOS) 0.10.3. This release includes iOS 13 compatibility updates, fixes a IMU related tracking regression, and resolves a number of minor bugs.

Please note – if you’re upgrading from an earlier version (0.10 or earlier) be sure to read the 0.10.1 notes on the HERE. Applications built on 0.10.1+ require special steps when submitting to the app store.

REMOVED setupIMU METHOD

Since SDK 0.9, the STCaptureSession provides direct access to IMU data. In the Room Capture and Scanner sample apps, we have been using the old setupIMU initialization for IMU events, while also utilizing the IMU information from STCaptureSession. This caused regressions in tracking, especially on iOS / iPadOS 13. We have removed the setupIMU method and all corresponding calls, which should improve performance of the sample apps.

SCAN BUTTON NOT WORKING IN SCANNER SAMPLE

Structure SDK 0.10.3 fixes an error in the Scanner sample app in which scanning would not start when the Scan button is pressed.

CALLING sensorBatteryLevel AND getBatteryChargePercentage WHEN SENSOR IS DISCONNECTED NO LONGER CRASHES APP

We have fixed a crash related to calling STCaptureSession:sensorBatteryLevel or STSensorController:getBatteryChargePercentage after a Structure Sensor Mark II is disconnected. Please contact support if you continue to experience issues pertaining to the use of these APIs.


#2

@JacobErvin what reference frame is core motion being initialized to internally?

-jim


#3

EDIT:
My wife is a teacher and she always reminds me to say something nice before giving criticism. So, the good news is it is actually scanning :slight_smile:

STCaptureSession based Scanner sample testing so far :frowning:

  • App crash on first launch after allowing camera access
  • Please connect message requiring re-plug
  • App crash on re-launch after re-plug
  • Sensor is initializing message not clearing on bringing app into foreground after allowing camera access in privacy settings
  • Spontaneous app crashes

#4

Hi @JacobErvin

Looks like this release has fixed the bug i reported about returning sensorName() incorrectly, and the disconnect notification, so thanks for that.

Still a few issues I’ve noticed, after a quick look.

Despite what the docs say, currentSDKVersion() in Structure.h does not return a string; it returns a SDKVersion struct. To get the version as an NSString you need to do the following (here setting a label):

SDKVersion SDKVersion = currentSDKVersion() ;
self.sdkVersion.text = [NSString stringWithUTF8String: SDKVersion.version] ;

Or, in Swift:

let SDKVersion = currentSDKVersion()
self.sdkVersion.text = String(cString: SDKVersion.version)

Also noticed that the Framework info.plist is still showing 0.10.2 for CFBundleVersion and CFBundleShortVersionString, so if you are getting SDK version via that route you get the wrong value.

Yet to dive in to check if other raised issues are mitigated.

-Andy W


#5

@JacobErvin any ETA on the really critical issues such as the sensor not initializing without a re-plug?

-jim


#6

STSensorController - calibrationType reports differently in 0.10.3 as compared to 0.10.2.

@JacobErvin @jeremysteward this is a really bad thing since the app doesn’t recognize the need for calibration with this broken!


#7

I think it’s time to consider EOLing the low level samples due to the lack of control over Mark II configuration. It’s interesting that STSensorController - startStreamingWIthOptions accepts a depth stream preset, but without the ability to configure e.g. IR auto exposure, that’s not going to get us all the way there.


#8

Hi Jim,

We’re working towards a major full stack update which involves driver, firmware and SDK improvements. We’re closing in on the last items now, and expect to ship by mid / late Nov. This release will resolve many long standing issues (including initialization and dropped connection bugs) and will bring Mark II to parity with Structure Sensor. We’ll be releasing a more detailed forum post this week with details on what to expect.


#9

are you having issues with the mesh rendering. Thanks


#10

No problems here, but I’m rendering the colorized mesh using SceneKit. What issues are you having?


#11

Experiencing consistent crash when putting the sample Scanner app in the background during a scan. Only on the Mark II with 0.10.3 (the old scanner recovers fine on 0.10.3).

Secondly, if the app is in a cube placement state and put in the background it sometimes crashes when running/launching the calibration app. It seems like the Scanner app may still be running something scanner-related in the background and another app that uses the scanner SDK requesting to use it can cause it to crash.

What is the safe/intended way of handling the app going into the background while scanning or placing the cube?

In the sample app there is logic for handling UIApplicationDidBecomeActiveNotification…but nothing for going into the background mode. I’ve started adding logic subscribing to UIApplicationWillResignActiveNotification and UIApplicationDidEnterBackgroundNotification but so far I can’t find how to safely shut down the scanner or even closeScanner() to stop the crashes.


#12

Capture session or low level?


#13

the mesh is rendering triangles… and looks like the mesh does not have the right anchor point…


#14

Can you post a screenshot?

-jim


#15


#16


This looks similar to your picture. Mine was from a scan 2 years ago.
It looks like its picking up the inside of the cube


#17

I just posted a workaround for the app background crash, but it’s far from robust. I suspect motion callbacks aren’t being stopped when streaming is disabled.

This might save someone a bit of time. Things have changed regarding notifications in iOS 13:

if (@available(iOS 13.0, *)) {
    [[NSNotificationCenter defaultCenter] addObserver:self
          selector:@selector(appWillResignActive)
          name:UISceneWillDeactivateNotification object:nil];

    [[NSNotificationCenter defaultCenter] addObserver:self
          selector:@selector(appDidBecomeActive)
          name:UISceneDidActivateNotification object:nil];

}
else {
    [[NSNotificationCenter defaultCenter] addObserver:self
          selector:@selector(appWillResignActive)
          name:UIApplicationWillResignActiveNotification object:nil];


    [[NSNotificationCenter defaultCenter]addObserver:self
          selector:@selector(appDidBecomeActive)
          name:UIApplicationDidBecomeActiveNotification
                                              object:nil];
}