back
16 comments
For anybody looking to understand/implement Neural Networks in C++, the books by Timothy Masters are indispensable - http://www.timothymasters.info/
There are plenty of resources on search engine results , what did you find in particular those books stays top of the others?
What's something useful to do with neural networks?
NNs are potentially very powerful arbitrary function approximators, but you have very limited control (or, arguably, insight) into the precise nature of the solutions their optimization arrives at. Because of that, they've been especially well suited to problems in vision and NLP where we have basic intuition about the phenomenology but can't practically manage a formal description of that intuition (and enumerating that description is probably not of great intellectual interest): what, in pixel space, makes a cat a cat or a dog a dog? What, in patterns of natural words, indicates sarcasm or positive/negative sentiment?

They also get tons of use in results-oriented modeling of lots of other statistics questions in structured data (home prices, resource allocation, voter turnouts, etc.) but in this luddite's opinion, these sorts of applications tend to be pretty fraught if they short-change the convenience of the model training paradigm for a deeper understanding of the data phenomenology.

I've never understood how to add "layers" when training a neural network. For example, I could train a model on historical home prices by area + date/time, but I'd also like to add for example, the unemployment rate, the trailing rate of inflation past 12 months, the current 30 year mortgage rate for the time, etc.
The things you mention after “For [the(sic)] example” are called features and they are considered as dimensions of the data. Layers are not strictly related to dimensions.
Is there a typical limit to number of features (that have almost nothing to do with each other/very little correlation) you can add to a basic "hack it together real quick from Google/copy paste/tutorials" Python neural network before the results are pretty bleh?
Here is my simplistic and dumb NN in C++ https://github.com/runvnc/mlp/blob/master/neuralnetwork.cpp based on the Hinton paper https://github.com/runvnc/nnpapers/blob/master/hinton86.pdf
Forget about the c++ code, and appreciate the beautifully put explanation of the concept. Then you can implement yours even in php if you want
After I took a C++ class, I built a small neural net in 2 weeks (no C beforehand, nets only from school a few years ago).

A few months later, I replicated the neural net in Smalltalk in 2 half days.

Fortunately, our project went with Smalltalk...

Did you have any prior Smalltalk experience? Or if not, any programming experience at all? Sounds like it might have been “I learned to program in C++ in two weeks, then ported my program to Smalltalk in 2 days”.
How fast the implementations are, in comparison?
I did my CNN implementation in C++ a couple of years back using this: http://cs231n.stanford.edu

Around 600 lines of free functions in a header file for the core functionality, including convolution, dropout, batch normalization, etc. Maybe half of it is actual code. Around 2000 lines for an executable training program including a couple of array template classes, data loaders, and convenience functions. Slow as shit but it works :P

An enjoyable exercise.

You might wanna validate your back propogation *or the final cost against a hand written very small network to see what went wrong
this is cool, but try C (modern, C99) instead: all I see in this C++ version is obsfucation and confusingly named things...

(sure, for an "industrial strength" version C++'s features are worth the cost, but for learning/education I never saw the appeal of classes and objects and smart pointers and stuff)

"shit you did in your bedroom as a kid before the internet, in a few hours, done in a weekend with enormous resources!"

you people suck.