Custom Shader for Depth Test for Scan volume and rendering during scanning


#1

I want to use custom shaders for:

Depth Test

The depth test while placing the cube. Relevant code:

[_display.cubeRenderer renderHighlightedDepthWithCameraPose:cameraStatic alpha:alpha];

in ViewController+OpenGL.mm

Scanning live preview

What the scan looks like while scanning (as opposed to simply off-white). Relevant code:

[_slamState.scene renderMeshFromViewpoint:cameraViewpoint cameraGLProjection:cameraGLProjection // alpha:0.8 alpha:_display.meshRenderingAlpha highlightOutOfRangeDepth:true wireframe:false];

in ViewController+OpenGL.mm


How can I achieve this?


#2

I found some relevant forum posts about customer shaders that you should look into.

Here:

and here:

Basically, you should look into the CustomShader.mm file for how the shaders are implemented. From there, you could design your own shaders.


#3

Thanks for posting this Anthony - but these links do not actually help.

Neither link address passing a custom shader to cubeRenderer or renderMeshFromViewpoint

The first link: has to do with updating the shader on the review Mesh step (not what I need).

The second link: has to do with what I am trying to do. But has no solution. The OP says ‘Custom rendering does not seem to be as easy as i thought it would be.’ and then 13 days later says ‘I was able to implement custom openGL rendering in the Scanner example by following this simple Tutorial.’ The included link talks about writing a custom shader, but not passing it to cubeRendered or renderMeshFromViewpoint.


#4

The cubeRenderer bounding box is using the simple line shader in OpenGL, but, unfortunately, is not open to be edited by the public at this time.

The cubeRenderer.renderHighlightedDepthWithCameraPose is also not open to the public to be edited at this time.

You could write your own functions that do the shaders similarly to what is shown here, but this is not an easy task. You should definitely take some time to research a little more about how OpenGL shaders work before trying to implement this:

https://www.khronos.org/opengl/wiki/Shader

https://learnopengl.com/#!Getting-started/Shaders

As for the shaders used when the Mesh is being rendered, you should look into the MeshRenderer.MM and .H files for clues on how the Mesh is rendered. More specifically, the MeshRenderer.MM files uses the LightedGrayShader class found in the CustomShader.h files. If you want to change the way the MeshRenderer.mm file renders the mesh, you’ll have to change it there, or write your own shader inside the CustomerShader.h file and call it when the LightedGreyShader class is called by the MeshRenderer Object.

What is it that you are specifically trying to change with the cubeRenderer and meshRenderer shaders?


#5

Hey Anthony - could you connect me with someone on engineering about how to connect my hand-written shaders to those functions? That’s the key difficulty


#6

I can certainly help you out! If you are comfortable with it, you can post your code here, but if not, please PM me your code and an exact description of what you are trying to do with the Scanner App sample.