Source: https://upload.wikimedia.org/wikipedia/commons/2/2a/FloorGoban.JPG[1]
Artificial Neural networks (Artificial Neural Networks) borrow their name from biological neural networkssuch as those found in the human nervous system and brain. But while our brain will probably continue to harbor many secrets in the future, the Basic principle artificial neural networks surprisingly Simple and accessible.
In this article, I would like to demonstrate the basic principles and practical use of neural networks using an easy-to-understand example. Let's imagine that we want to communicate with an electronic device via gestures and enter a number between one and five. To do this, we use one of the following hand gestures:
Our program should now be able to create the following from the photo of our hand gesture to derive the correct number. So we want to sort images into five categories from one to five. This problem is known as classification. If you would like to try out the example illustration directly, you can do so with our Demo application do.
Try out the demo application hereThe basic principle of artificial neural networks
The basic idea behind artificial neural networks is based on a principle inspired by biology: elements that are very simple in themselves can be combined to create Map arbitrarily complex constructs. In this model, our brain consists of a Large number of networked neuronswhich each assume one of two states: activated or not activatedor the neuron sends a signal, or remains silent. A neuron is activated when a sufficient number of the neurons connected to it are in turn activated. If we generalize the model, a neuron can not only assume two states, but also Any value between one and zero. Let's take a look at a simple artificial neural network:
The diagram is read from left to right. Each circle shows a neuron, which in Layers arranged (numbered here from one to four). The leftmost layer is called Input layer (input layer). If we use grayscale images as input data, for example, each pixel in a test image corresponds to a neuron that constantly outputs the corresponding gray value of this pixel. In our example image, all neurons in the input layer are now linked to each neuron of the next layer, the so-called hidden layer (hidden layers). The neurons of the first hidden layer receive as input a weighted sum of the neurons of the input layer, which is visualized in the diagram by arrows of different thicknesses. This means that the signals from the neurons in each layer are passed on to the neurons in the next layer to varying degrees.
Let us now look at any neuron of the hidden layers (layers #2 and #3). According to the arrow thickness of the incoming neurons, the target neuron a weighted sum is formed. This signal strength is then used to decide whether the neuron should be activated or not. A neuron can not only be activated or deactivated, but the Neuron activity is mapped on a scale from zero to one. For this purpose, a so-called Activation function which calculates the neuron activity from the weighted sum of the input neurons.
The Classification of an image finally takes place in the Output layer. In our hand gesture example, we would like to divide the images into five Different classes one class for each hand gesture. We therefore need five output eurons. Each output neuron indicates the probability that the image belongs to the corresponding class. The output euron with the highest probability is used for the Classification of the image used.
Before we can use a neural network to classify images, we need to determine the thickness of the arrows (the so-called weights), which determine how much one neuron influences another. For a completely new neural network, the weights are initially chosen at random.
We use existing data to determine the weights. For our gesture recognition example, we use the Sign Language Digits Datasetwhich was created by students at Ankara Ayrancı Anadolu High School in Turkey and is freely available under the Apache 2.0 license via Github. The data set includes gestures from zero to nine and has almost 200 images for each of the categories. This amount of data is quite small for an artificial neural networkbut is sufficient for our brief introduction. The following figure shows examples of five images from the data set that represent the "three" gesture:
We limit ourselves to the hand gestures from one to five, as these gestures are familiar to everyone and the effort required to take your own test photos is therefore limited. In addition, we assume that the hand gesture is photographed in front of a monochrome background, that the photo has already been cropped and that the right hand is used. None of these requirements represent a general limitation of neural networks, but are merely due to the small data set and our limited development time.
To obtain the weights of the individual neurons, we divide the data set into so-called training and test data on. For each image from a training data set, the classification of the neural network is compared with the compared with the correct classification. Correct predictions are mathematically rewarded and incorrect predictions are penalized, which is achieved mathematically via a so-called Target value function is expressed. Now a Optimization algorithm used in order to Minimize target value function and thus find a set of weights that meet the Predict training data as well as possible can.
Once a sufficiently good set of weights has been found, the test data set is used to check whether the neural network can also be used with data for which it has not been explicitly optimized. was.
Technologies for the implementation of neural networks
For the implementation of a neural network, we basically have a Variety of technologies available. From the complete own implementation in any programming language, up to and including highly specialized, prefabricated solutions with different levels of abstraction. For our introductory example, we choose the following programming language Python. Python is a popular programming language in the scientific and data analysis context, which creates a fairly broad, free available ecosystem of libraries for data processing and modeling exists. As an interpreter language, Python allows a interactive access to the data sourcewhich enables a very efficient data exploration and modeling phase. We use the libraries scikit-image (for image processing), numpy (for efficient mathematical operations), and matplotlib (for creating plots) and the project jupyter (to provide better interactive tools).
For the actual neural network, we use Tensorflow back. Tensorflow is a machine learning framework developed by Google which allows efficient, parallelized calculations and provides an interface for Python. The basis for efficient calculation with Tensorflow is the Creation of a calculation graph. Tensorflow can then perform the necessary calculations optimally and thus Highly parallelized calculations on graphics cards carry out. For this introduction, we use the High-level API of Tensorflow, which already provides parameterized, ready-made estimators (so-called estimators).
For this introduction we use the DNNClassifier-estimator, i.e. a Deep Neural Network Estimator. This allows us to specify the very simple neural network outlined above with a predefined number of hidden layers and number of neurons per hidden layer. All hidden layers initially have the same number of neurons 1.
Data preparation for neural networks
Before we can feed the neural network with data, we first have to prepare it. Our training data set is already very well prepared, so that the existing images only need to be loaded and reduced in size.
The same operations are applied to images provided by the user. To become less dependent on the background color of the image and the lighting conditions, we also convert all images to a different color space. Instead of the red-green-blue (RGB) color space, we use the HSV color space, which describes colors in color value, color saturation and brightness value.
Source: https://de.wikipedia.org/wiki/Datei:HSV_cone.png [2]
Modeling and use of data for neural networks
The processed images are used to train the tensor flow model and find optimized neuron weights. To optimize the Test the quality of the model the test data set is used. The model receives a test image as input. The recognized hand gesture of the model is then compared with the correct answer. A good model should correctly classify a large proportion of the test images, while also showing a good recognition rate with a third validation dataset.
If a user wants to classify a new image, all transformations are first applied to the image and then transferred to the neural network. As a result, the model shows us with what probability it assigns the picture to the individual hand gestures from one to five.
Artificial neural networks can generate very complex mathematical functions from very simple components. With just one Comparatively simple neural networks can sometimes produce impressive results. We demonstrate this with our Demo applicationwhich recognizes hand gestures from one to five on pictures.
We will also be discussing machine learning at our expert talk on February 21, 2019 in Munich learn more
1 However, the resulting model does not necessarily have the same number of neurons in each layer. To prevent overfitting, we use a regularization technique (dropout) that removes randomly selected neurons from the network.
Sources
[1] https://upload.wikimedia.org/wikipedia/commons/2/2a/FloorGoban.JPG
[2] Representation of colors in the HSV system. Image created by (3ucky(3all using Borland Delphi 2006, Adobe Photoshop 9.0. Not modified. CC BY-SA 3.0
