Neural Network Tutorial

The Mathematical Heart of Neural Network

neural network is just a math function, of the form

or written as

where

• The M are matrixes.
• M . v means matrix times vector.
• v is a vector, of the input. (as n by 1 matrix. or can be a matrix M0.)
• af is the activation function, typically ReLU (aka Ramp function) or sigmoid.
• There may be arbitrarily number of M. Total number of matrix is the number of layers, of the neural network.
• Each of the matrix, the number of rows is arbitrary, depending on the design on how many neurons in each layer. The number of columns must be compatible for multiplication.

(* neural network, typical form *)
Fold [ Function[af[#2 . #1 ]], {v, M1, M2, M3, M4 } ] ===
af[M4 . af[M3 . af[M2 . af[M1 . v]]]]

(* example of matrix multiplication *)
{{a,b}, {c,d}, {e,f} } . {x,y} === {a*x + b*y, c*x + d*y, e*x + f*y}

How Neural Network Work

machine learning tutorials

• Decision tree
• Random forest

• https://scikit-learn.org/
• https://keras.io/