UPDATE 1/15: Any workarounds for STCaptureSession failing to stream?

1/15 Update

Looks like this specific bug has been addressed, but I think I uncovered a new but nearly identical bug.

1/10 Update

Early development with SDK 0.11.2 suggests this issue is now resolved.

Original Post

Hi all,

I’ve been having an issue with re-enabling streaming using the STCaptureSession after disabling streaming a number of times. I’m curious to know if others have encountered this and have come up with some workaround that can serve as a temporary fix while the SDK dev team comes up with a fix for this.

To be clear, I begin a session by initializing a normal STCaptureSession to stream both depth and color frames, very similarly to how this is done in the demo Scanner app, passing config options to begin monitoring and enabling streaming. Once a model is captured, the streaming is disabled in order to review the model in another controller. If the model needs to be recaptured, we return to the previous controller and reuse the same STCaptureSession object, reenabling streaming to resume the capture process. In most cases, reenabling the stream works as you would expect, but in some cases, streaming does not work at all, causing me not to be able to receive any depth or color frame data. When I fall into this state, the screen appears to be frozen with the last color frame that was received before disabling streaming. All the STCaptureSession properties indicate that the sensor is still connected and ready and its isStreaming property does return true. I don’t receive any delegate calls from the capture session that would indicate that anything went wrong, like any changes in the sensor mode or camera mode. I believe this should a scenario that would fall in the STCaptureSessionColorCameraModeUnknown state but that is not identified through the delegate call.

Another thing that may be important to note is that I do notice that in most cases that I experience this behavior, if not all, I do notice this reduced frame rate issue mentioned in the latest SDK release post:

I’m not entirely sure if these issues are related, but this may be relevant to the SDK dev team.

At the moment, the only solution that I have in place to help in this situation involves identifying when there may be long delays between frames in a streaming session. When a delay is observed, I display a message to alert the user to reconnect the sensor, though this is not a solution I am happy with. Ideally I would prefer if there were some trick to programmatically reset the capture session without interrupting the user experience. I’m curious to know if others have encountered this and if anyone has come up with some workaround that would easily reinitialize the capture session programmatically instead of promoting to do it manually.

Any help would be appreciated.

@JacobErvin @moderators

I’m pushing the scanning view controller from a mesh viewer, so kind of the inverse of what you’re doing. I haven’t gone back and tested whether the motion updates are stopped when streaming is stopped.

1 Like

After early testing with the 0.11.2 SDK, it seems that the new changes have addressed the issue with the failed streaming. It is still a bit early to say this is completely resolved but the newest SDK release does look promising. I will continue testing in order to confirm these findings.

2 Likes

After spending even more time with 0.11.2, it’s safe to say the bug I described above has been fixed, however, I may have uncovered a similar issue. So streams failing after reenabling does not appear to be an issue anymore, but on very few occasions I have experienced streams failing right after sensor initialization. The stream fails in the same manner, the sensor completes initializing but once you would expect frames to begin coming through, it does not. You would then have to reconnect the sensor to get it going. This occurs less frequently than the initial bug I reported so the experience is not affected much but if there’s a bug somewhere, it’s gotta be squashed. Hopefully we can get this new issue addressed too.

1 Like