Neural Networks

Parametric description

Download

Prerequisites

• Basics of mathematical analysis, algebra and statistics from the previous years of studies,
• Basics of optimizing methods nonlinear functions,
• Basic of numerical methods and programming in Python languages.

Course Aims

• Fundamentals of neural network operation and training. Knowledge and skills necessary for independent problem solving using neural networks.
• Fundamentals of neural networks types and applications with an indication of advantages and limitations.

Lecture:

• Introduction: basic ideas, history, applications. Description of the neuron, activation functions, basic characteristics of neural networks. Neural network structures: feedforward and recurrent.
• Machine learning types: supervised, unsupervised and reinforced. Characteristics of input data and data augmentation in neural network learning. Application of neural networks in static and dynamic modeling.
• Loss function and evaluation measures of network results used in regression and classification. Perceptron learning rule. On-line and off-line training. Gradient optimization methods in neural network learning. The Widrow-Hoff rule. Back propagation algorithm and its characteristics.
• Characteristics of overfitting and underfitting in neural networks training. Validation methods. Generalization methods of neon networks: regularization, dropout, batch normlization.
• Neural networks with radial base functions. Support vector machine. Self-organizing maps, Kohonen networks. LSTM recurrent network
• Convolutional neural networks: description of ideas, structures, learning and their characteristics. Description of basic structures of neural networks in applications of computer vision

Exercise:

• Basics of Python and Matlabie programming,
• Tensorflow and Keras libraries in neural network problem formulation,
• Practical applications of neural networks.

Recommended reading

• S. Haykin, Neural Networks and Learning Machines, Third Edition, Prentice Hall, 2009
• I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, MIT Press, 2016
• G. Dreyfus, Neural Networks: Methodology and Applications, Springer, New York 2005
• S. S. Haykin, Neural Networks and Learning Machines. Prentice Hall, 2009.

Download