So I decided to follow this straight forward tutorial in creating a simple neural network without any hidden layers. I created a fake sample problem to train the neural network on to test the code and parameters.
Sample Problem:
- A student is graded from four exams, scoring from 0 to 9.
- Let's take the average of these four grades and determine if a student has passed or failed.
- Passing >= 5. Failing is < 5.
- Use a neural network to determine if a student has passed.
- Training data: Let's make up about 10 students with various grades.
- Feed the network 3 more students to see if it performs well.
- Goal: to create a NN that can solve this problem with minimal loss.
Training Data:
A: 9 | 8 | 5 | 7 | 7.25 = P
B: 3 | 6 | 4 | 2 | 3.75 = F
C: 1 | 8 | 7 | 5 | 5.25 = P
D: 9 | 6 | 8 | 8 | 7.75 = P
E: 3 | 0 | 7 | 5 | 3.75 = F
F: 5 | 4 | 6 | 6 | 5.25 = P
G: 8 | 9 | 6 | 8 | 7.75 = P
H: 4 | 5 | 5 | 2 | 4.00 = F
I: 6 | 7 | 7 | 7 | 6.75 = P
J: 5 | 9 | 7 | 3 | 6.00 = P
Testing Data:
X: 3 | 4 | 4 | 3 | (3.5 = F)
Y: 7 | 6 | 4 | 1 | (4.5 = F)
Z: 8 | 7 | 6 | 8 | (7.25 = P)
Neural Net Setup:
- Four inputs
- No Hidden Layers
- One Output
What I Have to Solve?
- What sample weights to use to get the correct output?
- What bias should be added?
- How do I minimize loss?
Process:
So the setup of the NN is just a few bits of python code:
NumPy is a package that helps with computing arrays, like in this problem.
The basic equation for this neural net is:
Output = Input1 * Weight1 + Input2 * Weight2 + .... + Bias
The point of the neural net is to calculate the weights and bias on its own through the use of training data. The network determines if the results is passing or failing by how close the result is to one or zero using the sigmoid function.
After running through the sample input data multiple times, I will use the testing data to determine of the net has been properly trained with this code:
Results:
So what I first messed with is what is the proper number of training steps to take to get the most accurate results.
With only 100 training steps, the loss remained at 0.1, which is still a fairly high chance of being incorrect.
The final three numbers are the results of the testing data, which should be fail, fail, pass. But the net determines the first result as just barely passing. So clearly more steps are needed.
So at 1000 steps, the loss rate is now at 0.02, which is much better, but could potentially still be lower. The results for the test data are far more accurate now.
At 1000 steps, the loss rate is very low that I'd consider the net very reliable. The results for the training data are very sure of themselves with the second and third test being incredibly close to both 0 and 1.
I also had the net spit out what the final weights for each input and bias was.
What I learned:
The amount of time it takes and NN like this to run is almost instant. Only at 10000 steps did I actually have to wait was was probably only three seconds before I got the final results. However this is only purely using numbers. I believe much of my future time will be spent pre processing audio files for future use.
10000 or more steps will probably be needed to minimize loss in a meaningful way. The tutorial even suggested 20,000 steps so I will have to remember this for the future.
If I can figure out a way to do something similarly simple with audio data this week as well I will give it a try!
Comments
Post a Comment