Open Source Swift Starter/Data Collection Project.


#1

I figure I would share this here.

TL/DR: A simple Swift based Structure project to use as a starting point or to collect data with.

Long Version:

I have built a really simple app for my Machine Learning class I am taking this semester. I am trying to do some Deep Learning on the Structure’s Point Cloud data as a continuation of my Masters Project SEARL-RL. (A system using the Occipital Structure to allow someone who is visually impaired to navigate the world with sound).

Basically, I am trying to experiment if I can build a simple Deep Learning model to help ID obstacles that may be of concern.

I needed to collect data so I built this project to collect the raw point cloud data so I could train my ML models later.

Instead of letting the project bit rot, I figure there might be other Structure users out there trying to do the same thing or needing a more up to date Swift based starter project (the most current Structure/Swift project I could find before I started this was for Swift 2.0).

So feel free to check out the code here: https://github.com/ForeverTangent/SEAR-DC, if it may be of help.

I have tried to keep it as clean and documented as much possible (considering it is a one-off thing). I am a big stickler for documentation anyway (because I am the first to admit I am dumb and have no memory) so if people have any suggestions where I can fill in gaps, feel free to contact me and I will try to fill it in once I get a chance.

I hope this can help. The people on this board have been a big help in getting me through my Masters Project and I figure this was the least I could do to give back.

Peace

Stan


Build error with iOS 11.4, SDK 0.8
#2

Really appreciate you sharing this with us. Using Structure Sensor as a way to offer low-cost, hi-fidelity obstacle avoidance for the vision impaired is such a great concept. Please keep us in the know about SEARL-RL!


#3

Hello Phil,

Thanks for the encouragement. Work on it has been a little slower since Graduation while I look for a full-time gig, but I keep chipping at it whenever I can. Taking this Machine Learning class has helped keep it in my sights.

Peace

Stan


#4

OK finally finished my semester Machine Learning project.

If people are interested you can read the results here:

The tl;dr is pretty much what one would expect:

  • If you want to ID 3D objects with a Convolution Neural Network (CNN), you are better off using 3D data (point clouds, like from the Structure) then 2D image data (even for 2D CNNs like I used).
  • Using human reinforcement in incremental training of Neural Nets does not really improve training. It might help if you are adding new classes to ID along with the data, but that would be future work to explore.

Peace

Stan