Section 1 Foundation of neural network Chapter 1 Theoretical basis of biological neural network 2 1.1 Structure and function of biological neurons 2 1.2 Electrical activity of the nervous system 5 1.3 Information storage of human brain 9 1.4 Human brain and computer 11 Exercises 16 References 17 Chapter 2 Review of artificial neural network 18 2.1 Development history of artificial neural network 18 2.2 Characteristics of artificial neural network 28 2.3 Applications of artificial neural network 30 Exercises 38 References 39 Chapter 3 Mathematical basis of artificial neural network 40 3.1 Neuron model 40 3.1.1 Symbol description 40 3.1.2 Single input neuron 41 3.1.3 Transfer function 41 3.1.4 Multiple input neurons 45 3.2 Derivatives 45 3.3 Differential 47 3.4 Integrals 47 3.5 Gradient 48 3.6 Determinant 49 3.7 Matrices 50 3.7.1 Concept 50 3.7.2 Operation of matrices 51 3.7.3 Operational properties of matrices 51 3.8 Vector 52 3.9 Eigenvalues and eigenvectors 53 3.10 Random events and probabilities 53 3.11 Norm 55 Exercises 57 References 58 Section 2 Theory of artificial neural network Chapter 4 Perceptrons 60 4.1 Introduction 60 4.2 Architecture and principle of perceptron 61 4.2.1 Architecture of perceptron 61 4.2.2 Principle of perceptron 62 4.2.3 Learning strategies of perceptron 64 4.3 Single layer perceptron 65 4.3.1 Single layer perceptron model 65 4.3.2 Function of single layer perceptron 67 4.3.3 Learning algorithm of single layer perceptron 69 4.3.4 Limitations of single layer perceptron 73 4.4 Multilayer perceptron 74 4.4.1 Architecture and principle of multilayer perceptron 74 4.4.2 Functions of multilayer perceptron 75 4.4.3 Multilayer perceptron learning algorithm 78 4.5 Applications 79 4.5.1 Case �� 79 4.5.2 Case �� 81 Exercises 85 References 86 Chapter 5 Back Propagation neural network 87 5.1 Introduction 87 5.2 BP neural network architecture 89 5.3 BP algorithm 90 5.3.1 Algorithmic principles 90 5.3.2 Back propagation examples 95 5.4 Shortcomings and improvement of BP algorithm 98 5.4.1 Shortcomings of BP algorithm 98 5.4.2 BP algorithm improvement 102 5.5 Applications 105 5.5.1 Case �� 105 5.5.2 Case �� 108 5.5.3 Case �� 110 Exercises 113 References 114 Chapter 6 RBF neural network 115 6.1 Introduction 115 6.2 Architecture and principle of RBF neural network 116 6.2.1 RBF neuron model 116 6.2.2 RBF neural network architecture 117 6.2.3 Principles of RBF neural network 118 6.3 RBF neural network algorithm 119 6.4 Related problems of RBF neural network 122 6.5 Applications 123 6.5.1 Case�� 123 6.5.2 Case�� 125 Exercises 126 References 127 Chapter 7 Adaline neural network 128 7.1 Introduction 128 7.2 Architecture and principles of Adline 129 7.2.1 Single layer Adaline model 129 7.2.2 Algorithm and principles 130 7.2.3 Multilayer Adaline model 133 7.3 Applications 136 7.3.1 Case �� 136 7.3.2 Case �� 138 Exercises 141 References 142 Chapter 8 Hopfield neural network 143 8.1 Introduction 143 8.2 Discrete Hopfield neural network 144 8.2.1 Network architecture 144 8.2.2 Working principles 145 8.2.3 Network stability 146 8.2.4 Network algorithm 148 8.3 Continuous Hopfield neural network 150 8.3.1 Network architecture 151 8.3.2 Network stability 153 8.4 Applications 153 8.4.1 Case �� 153 8.4.2 Case �� 156 Exercises 161 References 162 Chapter 9 Deep convolutional neural network 163 9.1 Introduction 163 9.2 Architecture and principle of deep convolution neural network 164 9.2.1 Architecture of deep convolutional neural network 164 9.2.2 Principle of deep convolutional neural network 166 9.3 Some basic deep convolutional neural networks 168 9.3.1 AlexNet 168 9.3.2 VGGNet 168 9.3.3 ResNet 170 9.4 Applications 171 9.4.1 Several application frameworks of deep learning 171 9.4.2 Image recognition based on AlexNet 173 Exercises 177 References 177 Chapter 10 Generative adversarial networks 179 10.1 Introduction 179 10.2 Architecture of GAN 181 10.3 GAN algorithm 182 10.4 Improved GAN 185 10.4.1 DCGAN 185 10.4.2 SGAN 186 10.4.3 InfoGAN 187 10.4.4 CGAN 187 10.4.5 AC��GAN 188 10.5 Applications 189 Exercises 191 References 192 Chapter 11 Elman neural network 193 11.1 Introduction 193 11.2 Architecture and principle of Elman neural network 193 11.2.1 Elman neural network architecture 193 11.2.2 Principle of Elman neural network 194 11.3 Learning algorithm of Elman neural network 196 11.4 Stability analysis of Elman neural network 198 11.5 Applications 200 11.5.1 Case �� 200 11.5.2 Case �� 203 Exercises 205 References 206 Chapter 12 AdaBoost neural network 207 12.1 Introduction 207 12.2 Architecture and algorithm of AdaBoost network 208 12.2.1 Architecture and principles 208 12.2.2 AdaBoost algorithm 209 12.3 Influence factors in AdaBoost algorithm 211 12.3.1 Training error analysis 211 12.3.2 Loss function in AdaBoost classification 212 12.3.3 Regularization of AdaBoost algorithm 214 12.4 Applications 215 Exercises 222 References 223 Chapter 13 SOFM neural network 224 13.1 Introduction 224 13.2 Architecture of SOFM neural network 225 13.3 Principle and algorithm of SOFM neural network 226 13.3.1 Principle of SOFM neural network 226 13.3.2 SOFM neural network learning algorithm 230 13.4 Applications 230 13.4.1 Case �� 230 13.4.2 Case �� 233 Exercises 237 References 238 Section 3 Design and practical application of artificial neural network Chapter 14 Artificial neural network modeling based on Simulink 240 14.1 Introduction 240 14.2 Simulink startup and neural network module library 241 14.2.1 Startup of Simulink 241 14.2.2 Simulink neural network module library 243 14.3 Model setting and operation 247 14.3.1 Module operation 247 14.3.2 Operation of signal line 247 14.3.3 Setting of simulation parameters 248 14.3.4 Setting of common modules 250 14.4 Single neuron modeling 254 14.5 Simulink simulation model of function approximation 256 14.5.1 Model and simulation with unchanged parameters 256 14.5.2 Changing parameters of model and simulation 259 14.6 Applications 263 Exercises 268 References 269 Chapter 15 Design of artificial neural network based on GUI 270 15.1 Introduction 270 15.2 Software architecture design 271 15.3 Creating a project 272 15.3.1 FIG file editor 274 15.3.2 M file editor 276 15.4 Main page design 277 15.5 Interactive parameter setting 280 15.6 Main function design of software 284 15.6.1 Detection and recognition 284 15.6.2 Repair method 296 15.7 Accessibility functions 300 15.8 Help file design 303 Exercises 306 References 306 Chapter 16 Design of artificial neural network based on wxPython 307 16.1 Introduction 307 16.2 Design of software architecture 308 16.3 Application creation 310 16.4 Common controls 312 16.4.1 Static text 312 16.4.2 Dynamic text 313 16.4.3 Button 315 16.4.4 Dialog box 316 16.5 Event processing 319 16.6 Design of main functions of software 320 16.6.1 Face input 321 16.6.2 Face recognition 324 16.7 Help file 326 Exercises 328 References 329 Chapter 17 Deep convolutional neural network application in edge detection with feature re��extraction 330 17.1 Introduction 330 17.2 Edge detection with feature re��extraction deep convolutional network 332 17.2.1 Network architecture 332 17.2.2 Loss function 333 17.3 Experiments 334 17.3.1 Implementation 334 17.3.2 BSDS500 results 335 17.3.3 Cross��distribution generalization validation 337 17.4 Discussion 338 17.4.1 Residual leaning 338 17.4.2 Feature re��extract 340 17.4.3 Feature fusion 341 17.4.4 Loss function 341 17.5 Conclusions 342 Exercises 342 References 343 Appendix A Common properties of GUI objects 344 Appendix B Discription of special charactor formats 355 Appendix C Software codes for chapter 15 356 Appendix D Software codes for chapter 16 361