Remove the base floor


#1

Contrary to this post, ever since SDK 0.9 until now 0.10.2, whenever I finalize the mesh, there always be an extra mesh added as the base.
It does not appear on mesh visualization and this base is not removed even if I put extra y-offset during cube-placement initialization.

How can I remove this base? Should I do a manual post-processing or is there a configuration that I use?


#2

We have seen the same, and are using a post-processing brute force method to trim the base.

As part of our app the mesh is saved as an MDLMesh (Model I/O), which we run through the following Swift function.

You can probably do something similar with straight STMesh or .obj files.

-Andy W

clip(mesh: mesh, removingFacesGreaterThan: clipValue) { result in
    // Save result
}

func clip(mesh: MDLMesh, removingFacesGreaterThan clipValue: Float, completionHandler: (MDLMesh) -> Void) {
  guard
    // Assume our data has at least position data.
    let positionData = mesh.vertexAttributeData(forAttributeNamed: MDLVertexAttributePosition),
    // Assume our mesh contains at least one valid submesh,
    let submesh = mesh.submeshes?.firstObject as? MDLSubmesh,
    // And there is only a single submesh,
    mesh.submeshes?.count == 1,
    // And that this submesh specifies triangle faces.
    submesh.geometryType == .triangles,
    // Assume position data is stored as 3 float values.
    positionData.format == .float3
  else {
    fatalError("Unable to get clipMesh")
  }
		
  // Normal, and texture coordinate data is optional.
  let normalData = mesh.vertexAttributeData(forAttributeNamed: MDLVertexAttributeNormal)
  let textureData = mesh.vertexAttributeData(forAttributeNamed: MDLVertexAttributeTextureCoordinate)
		
  // Original face index data
  let indexCount = submesh.indexCount
  let indexData = submesh.indexBuffer(asIndexType: MDLIndexBitDepth.uint32).map().bytes.assumingMemoryBound(to: UInt32.self)

  // Data for building our new trimmed mesh.
  var indexMap = [UInt32: UInt32]()
  var positionBuffer = [Float]()
  var indexBuffer = [UInt32]()
  var latestIndex: UInt32 = 0
  // Normal, and Texture data may not be used if the original model did not contain this data.
  var normalBuffer = [Float]()
  var textureBuffer = [Float]()
		
  // Stride through the face data by 3. Each 3*UInt32 is 1 full triangle face.
  for i in stride(from: 0, to: indexCount, by: 3) {
    var validFace = true
    // Iterate through each vertex referenced by this face.
    for vertexNumber in 0..<3 {
        let oldIndex = indexData.advanced(by: i + vertexNumber).pointee
        let vertex = positionData.dataStart.advanced(by: Int(oldIndex) * positionData.stride).bindMemory(to: Float.self, capacity: 3)
        // If this face contains a vertex with a Y value greater than the clipValue, it is not a valid face.
        // No need to continue looking at the other vertices of this face.
        if vertex[1] > clipValue {
          validFace = false
          break
        }
    }

    // Only add the face and associate vertex data to our new data if it is valid, as specified above.
    if validFace {
      for vertexNumber in 0..<3 {
        let oldIndex = indexData.advanced(by: i + vertexNumber).pointee
        // If we have already come across this index and added the vertex data to our new data
        // then the value of indexMap[oldIndex] will contain the right index to re-use.
        if let index = indexMap[oldIndex] {
          indexBuffer.append(index)
        } else {
          // Else we haven't come across this index before. Add the vertex it points to, to our new data.
          let vertex = positionData.dataStart.advanced(by: Int(oldIndex) * positionData.stride).bindMemory(to: Float.self, capacity: 3)
          positionBuffer.append(vertex[0])
          positionBuffer.append(vertex[1])
          positionBuffer.append(vertex[2])
          // Add normal, and texture data if it existed in the original model.
          if let normalData = normalData {
            let normal = normalData.dataStart.advanced(by: Int(oldIndex) * normalData.stride).bindMemory(to: Float.self, capacity: 3)
            normalBuffer.append(normal[0])
            normalBuffer.append(normal[1])
            normalBuffer.append(normal[2])
         }
         if let textureData = textureData {
          let texture = textureData.dataStart.advanced(by: Int(oldIndex) * textureData.stride).bindMemory(to: Float.self, capacity: 2)
          textureBuffer.append(texture[0])
          textureBuffer.append(texture[1])
        }
        // Update the new face index buffer with the index in the new vertex data.
        indexBuffer.append(latestIndex)
        // And update the indexMap so that if we come across this index again, we know where the pointed to vertex is stored
        // in the new data.
        indexMap[oldIndex] = latestIndex
        latestIndex += 1
        }
      }
    }
  }

  // Generate a new MDLMesh from our new data. Use withUnsafeBytes in order to access the Data(_:) constructor.
  var newSubmesh: MDLSubmesh!
		
  indexBuffer.withUnsafeBytes { ptr in
  // Create a buffer to hold the actual data.
  let bufferData = MDLMeshBufferData(type: MDLMeshBufferType.index, data: Data(ptr))
  // Setup the MDLSubmesh with the data, plus information about it's organisation.
  newSubmesh = MDLSubmesh(indexBuffer: bufferData, indexCount: indexBuffer.count, indexType: MDLIndexBitDepth.uint32, geometryType: MDLGeometryType.triangles, material: nil)
}

// Generate a new MDLMesh with our submesh plus position, normal, and vertex data.
let newMesh = MDLMesh()
		
textureBuffer.withUnsafeBytes { tPtr in
  positionBuffer.withUnsafeBytes { pPtr in
    normalBuffer.withUnsafeBytes { nPtr in
      newMesh.submeshes = [newSubmesh!]
      // Always add position data.
      let positionBufferData = Data(pPtr)
      newMesh.addAttribute(withName: MDLVertexAttributePosition, format: .float3, type: MDLVertexAttributePosition, data: positionBufferData, stride: MemoryLayout<Float>.size * 3)
      // Only add normal, and texture coordinate data if the original model contained this.
      if normalData != nil {
        let normalBufferData = Data(nPtr)
        newMesh.addAttribute(withName: MDLVertexAttributeNormal, format: .float3, type: MDLVertexAttributeNormal, data: normalBufferData, stride: MemoryLayout<Float>.size * 3)
      }
      if textureData != nil{
      let textureBufferData = Data(tPtr)
      newMesh.addAttribute(withName: MDLVertexAttributeTextureCoordinate, format: .float2, type: MDLVertexAttributeTextureCoordinate, data: textureBufferData, stride: MemoryLayout<Float>.size * 2)
  }
}}}
		
// call completion handler with mesh.
completionHandler(newMesh)

}


#3

I am experiencing the same issue and have asked Occipital about it.

The solution in my original post about cropping the floor is no longer valid.


#4

So if you initialize cube placement in a gravity aligned mode and then add a slight vertical offset when you start scanning that no longer works?


#5

Yes, by adding y-offset during scanning you can see that the base is not added, but as soon as I finalize the mesh using finalizeTriangleMesh, the base is added.


#6

I see, I though it was part of the configuration update that I missed.
So the only way for now is by trimming the mesh.


#7

What happens if you use a large offset, say 1 cm?


#8

It is the same even with 5cm


#9

That seems impossible since the mesh rendering during scanning should represent the final mesh! It’s not our current use case, but I’ll try and grab some time to try it.

EDIT:
Ok, took a minute to refresh my memory on this. The offset is applied to the scanning cube. There’s not any actual detection of the ground plane as is done in the table top strategy internally by the SDK. So unless the bottom of the cube is placed precisely grazing the floor, then it’s not going to clip the floor. And if the floor isn’t perfectly level, some of it could still be picked up in the scan.

I haven’t done an A-B comparison with pre-0.9 SDKs, but wanted to add these thoughts for clarification.


#10

In our experience the STMesh.newFillHolesTask causes a ‘rounding’ on very thin floor planes, where it’s caught a few faces in the original scan, and then tries to create a load more in an attempt to fill in the reverse side, and ‘softens’ the plane. That might be what you are seeing.


#11

It seems like you’d really need to run a RANSAC on the mesh and look for the plane, then use it’s position to do the clipping.


#12

Stmapper does have an option to clip the floor plane. Docs say it defaults to yes.

It seems to be a newer option. In fact, it seems like the same update that added this option also broke its functionality.

EDIT:

Also this issue is independent of hole filling. With or without hole filling the floor plane is being scanned in.


#13

Ah, that is new. I haven’t used a support plane in a while so didn’t notice that option. Doesn’t seem like it made it to the release notes.


#14

Hi All,

I’ve logged this issue into our short term roadmap for review. Once I have any additional information about this bug I will post here.

– Jacob