Introduction
AI, one of the peaks of the knowledge, which is becoming a powerful tool for applications such as control, computer vision, robotics, econometrics, and some others. However, to reach that level, we have to start from the bottom. In this post, I'm building a neural network (NN) that works as a classifier between binary inputs. In the near future, I will show how we can set our own NN to model a currency or commodity in the stock market, or to control whatever system you have in mind.
NNs vs. Deep NNs
I'm going to avoid the explanation of how NNs works in our brain, and how they inspired the artificial NNs. Instead, there is another question that allures my attention. Which is the difference between NNs vs. Deep NNs? The true is that Deep NNs and NNs are very similar. But the key are the amount of neurons and hidden layers in the so called "Deep" approach.
NNs were forgotten for a while, most researches and scientist were unable to explain their results, and they did not fell comfortable using something that they could not explain, or fully understand. But in recent years, with the advance in processing speed, the development of parallel computing, and new techniques, neural networks are taking the place they deserve, becoming a trend in complex scenarios were they could not be use before. Different architectures and different metrics allows to understand NNs better today. The problemFor our problem, we have the following truth table referring to the operator "OR". \begin{array}{|c|c|c|c|c|c|} \hline \mathbf{k} & \mathbf{x_{0}} & \mathbf{x_{1}} & \mathbf{x_{2}} & \mathbf{d} \\\hline 1 & 1 & 1 & 1 & 1 \\\hline 2 & 1 & 1 & 0 & 1 \\\hline 3 & 1 & 0 & 1 & 1 \\\hline 4 & 1 & 0 & 0 & 0 \\\hline \end{array} The NN will have to read the values, learn them, and classify them. The architecture
For our first example, we are going to implement, the basic perceptron, which is a NN composed with only a hidden layer. Within the architecture of the NN's, it is easy to distinguish the different part that makes the NN's.
Which programming language should we use?
Theano, Torch, Caffe, TensorFlow, MXNet, and many more libraries have been created for our enjoyment. However, my intention was to fully understand, to do so, there is no better way than programming a NN from scratch. The NN was programmed using the Excel Macros, but it can be ported to your language of preference, my intention also was to use something that most machines have (if you are a Linux user, try to run it on Libre Office, I have not tried it, but I am guessing that works there too).
The program will prompt two dialog boxes, one of them will ask you to introduce an α value, this value is related with the learning rate of the NN, in our case, a value between 0.8 and 1 will be a good starting point, the other dialog box will ask you for the number of iterations, for this case, around 10 iterations are more than enough. However, you are free to try as many iterations and different α values. If you run the program correctly, you will be able to see the following result as shown in the following image and table (or similar, depending of the α and the number of iterations). \begin{array}{|c|c|c|c|c|c|} \hline \mathbf{k} & \mathbf{x_{0}} & \mathbf{x_{1}} & \mathbf{x_{2}} & \mathbf{d} & \mathbf{y}\\\hline 1 & 1 & 1 & 1 & 1 & 1 \\\hline 2 & 1 & 1 & 0 & 1 & 1 \\\hline 3 & 1 & 0 & 1 & 1 & 0 \\\hline 4 & 1 & 0 & 0 & 0 & 0 \\\hline \end{array} Conclusion
The previous post does not intent to be a full guide in NNs, it is only a initial and basic explanation to start approaching to more complicated problems. I also tried to keep it simple, more math will added soon, for a better understanding of the concept.
0 Comments
Leave a Reply. |
About meEvery venture in an unknown territory is exciting to me. I ended up working with autonomous robots using knowledge from fields such as; computer vision, Bayesian estimation, control theory, neural networks, and SLAM. I have always been fascinated by aerial and ground mobile vehicles. Thankfully I had the chance to work on algorithms that bring them to life. Archives |