About the Editors xix
List of Contributors xx
Preface xxvi
Section I Introduction to AI-Based Regression and Classification 1
1 Introduction to Neural Networks 3
Isha Garg and Kaushik Roy
1.1 Taxonomy 3
1.1.1 Supervised Versus Unsupervised Learning 3
1.1.2 Regression Versus Classification 4
1.1.3 Training, Validation, and Test Sets 4
1.2 Linear Regression 5
1.2.1 Objective Functions 6
1.2.2 Stochastic Gradient Descent 7
1.3 Logistic Classification 9
1.4 Regularization 11
1.5 Neural Networks 13
1.6 Convolutional Neural Networks 16
1.6.1 Convolutional Layers 17
1.6.2 Pooling Layers 18
1.6.3 Highway Connections 19
1.6.4 Recurrent Layers 19
1.7 Conclusion 20
References 20
2 Overview of Recent Advancements in Deep Learning and Artificial Intelligence 23
Vijaykrishnan Narayanan, Yu Cao, Priyadarshini Panda, Nagadastagiri Reddy Challapalle, Xiaocong Du, Youngeun Kim, Gokul Krishnan, Chonghan Lee, Yuhang Li, Jingbo Sun, Yeshwanth Venkatesha, Zhenyu Wang, and Yi Zheng
2.1 Deep Learning 24
2.1.1 Supervised Learning 26
2.1.1.1 Conventional Approaches 26
2.1.1.2 Deep Learning Approaches 29
2.1.2 Unsupervised Learning 35
2.1.2.1 Algorithm 35
2.1.3 Toolbox 37
2.2 Continual Learning 38
2.2.1 Background and Motivation 38
2.2.2 Definitions 38
2.2.3 Algorithm 38
2.2.3.1 Regularization 39
2.2.3.2 Dynamic Network 40
2.2.3.3 Parameter Isolation 40
2.2.4 Performance Evaluation Metric 41
2.2.5 Toolbox 41
2.3 Knowledge Graph Reasoning 42
2.3.1 Background 42
2.3.2 Definitions 42
2.3.3 Database 43
2.3.4 Applications 43
2.3.5 Toolbox 44
2.4 Transfer Learning 44
2.4.1 Background and Motivation 44
2.4.2 Definitions 44
2.4.3 Algorithm 45
2.4.4 Toolbox 46
2.5 Physics-Inspired Machine Learning Models 46
2.5.1 Background and Motivation 46
2.5.2 Algorithm 46
2.5.3 Applications 49
2.5.4 Toolbox 50
2.6 Distributed Learning 50
2.6.1 Introduction 50
2.6.2 Definitions 51
2.6.3 Methods 51
2.6.4 Toolbox 54
2.7 Robustness 54
2.7.1 Background and Motivation 54
2.7.2 Definitions 55
2.7.3 Methods 55
2.7.3.1 Training with Noisy Data/Labels 55
2.7.3.2 Adversarial Attacks 55
2.7.3.3 Defense Mechanisms 56
2.7.4 Toolbox 56
2.8 Interpretability 56
2.8.1 Background and Motivation 56
2.8.2 Definitions 57
2.8.3 Algorithm 57
2.8.4 ToolBox 58
2.9 Transformers and Attention Mechanisms for Text and Vision Models 58
2.9.1 Background and Motivation 58
2.9.2 Algorithm 59
2.9.3 Application 60
2.9.4 Toolbox 61
2.10 Hardware for Machine Learning Applications 62
2.10.1 Cpu 62
2.10.2 Gpu 63
2.10.3 ASICs 63
2.10.4 Fpga 64
Acknowledgment 64
References 64
Section II Advancing Electromagnetic Inverse Design with Machine Learning 81
3 Breaking the Curse of Dimensionality in Electromagnetics Design Through Optimization Empowered by Machine Learning 83
N. Anselmi, G. Oliveri, L. Poli, A. Polo, P. Rocca, M. Salucci, and A. Massa
3.1 Introduction 83
3.2 The SbD Pillars and Fundamental Concepts 85
3.3 SbD at Work in EMs Design 88
3.3.1 Design of Elementary Radiators 88
3.3.2 Design of Reflectarrays 92
3.3.3 Design of Metamaterial Lenses 93
3.3.4 Other SbD Customizations 96
3.4 Final Remarks and Envisaged Trends 101
Acknowledgments 101
References 102
4 Artificial Neural Networks for Parametric Electromagnetic Modeling and Optimization 105
Feng Feng, Weicong Na, Jing Jin, and Qi-Jun Zhang
4.1 Introduction 105
4.2 ANN Structure and Training for Parametric EM Modeling 106
4.3 Deep Neural Network for Microwave Modeling 107
4.3.1 Structure of the Hybrid DNN 107
4.3.2 Training of the Hybrid DNN 108
4.3.3 Parameter-Extraction Modeling of a Filter Using the Hybrid DNN 108
4.4 Knowledge-Based Parametric Modeling for Microwave Components 111
4.4.1 Unified Knowledge-Based Parametric Model Structure 112
4.4.2 Training with l 1 Optimization of the Unified Knowledge-Based Parametric Model 115
4.4.3 Automated Knowledge-Based Model Generation 117
4.4.4 Knowledge-Based Parametric Modeling of a Two-Section Low-Pass Elliptic Microstrip Filter 117
4.5 Parametric Modeling Using Combined ANN and Transfer Function 121
4.5.1 Neuro-TF Modeling in Rational Form 121
4.5.2 Neuro-TF Modeling in Zero/Pole Form 122
4.5.3 Neuro-TF Modeling in Pole/Residue Form 123
4.5.4 Vector Fitting Technique for Parameter Extraction 123
4.5.5 Two-Phase Training for Neuro-TF Models 123
4.5.6 Neuro-TF Model Based on Sensitivity Analysis 125
4.5.7 A Diplexer Example Using Neuro-TF Model Based on Sensitivity Analysis 126
4.6 Surrogate Optimization of EM Design Based on ANN 129
4.6.1 Surrogate Optimization and Trust Region Update 129
4.6.2 Neural TF Optimization Method Based on Adjoint Sensitivity Analysis 130
4.6.3 Surrogate Model Optimization Based on Feature-Assisted of Neuro-TF 130
4.6.4 EM Optimization of a Microwave Filter Utilizing Feature-Assisted Neuro-TF 131
4.7 Conclusion 133
References 133
5 Advanced Neural Networks for Electromagnetic Modeling and Design 141
Bing-Zhong Wang, Li-Ye Xiao, and Wei Shao
5.1 Introduction 141
5.2 Semi-Supervised Neural Networks for Microwave Passive Component Modeling 141
5.2.1 Semi-Supervised Learning Based on Dynamic Adjustment Kernel Extreme Learning Machine 141
5.2.1.1 Dynamic Adjustment Kernel Extreme Learning Machine 142
5.2.1.2 Semi-Supervised Learning Based on DA-KELM 147
5.2.1.3 Numerical Examples 150
5.2.2 Semi-Supervised Radial Basis Function Neural Network 157
5.2.2.1 Semi-Supervised Radial Basis Function Neural Network 157
5.2.2.2 Sampling Strategy 161
5.2.2.3 SS-RBFNN With Sampling Strategy 162
5.3 Neural Networks for Antenna and Array Modeling 166
5.3.1 Modeling of Multiple Performance Parameters for Antennas 166
5.3.2 Inverse Artificial Neural Network for Multi-objective Antenna Design 175
5.3.2.1 Knowledge-Based Neural Network for Periodic Array Modeling 183
5.4 Autoencoder Neural Network for Wave Propagation in Uncertain Media 188
5.4.1 Two-Dimensional GPR System with the Dispersive and Lossy Soil 188
5.4.2 Surrogate Model for GPR Modeling 190
5.4.3 Modeling Results 191
References 193
Section III Deep Learning for Metasurface Design 197
6 Generative Machine Learning for Photonic Design 199
Dayu Zhu, Zhaocheng Liu, and Wenshan Cai
6.1 Brief Introduction to Generative Models 199
6.1.1 Probabilistic Generative Model 199
6.1.2 Parametrization and Optimization with Generative Models 199
6.1.2.1 Probabilistic Model for Gradient-Based Optimization 200
6.1.2.2 Sampling-Based Optimization 200
6.1.2.3 Generative Design Strategy 201
6.1.2.4 Generative Adversarial Networks in Photonic Design 202
6.1.2.5 Discussion 203
6.2 Generative Model for Inverse Design of Metasurfaces 203
6.2.1 Generative Design Strategy for Metasurfaces 203
6.2.2 Model Validation 204
6.2.3 On-demand Design Results 206
6.3 Gradient-Free Optimization with Generative Model 207
6.3.1 Gradient-Free Optimization Algorithms 207
6.3.2 Evolution Strategy with Generative Parametrization 207
6.3.2.1 Generator from VAE 207
6.3.2.2 Evolution Strategy 208
6.3.2.3 Model Validation 209
6.3.2.4 On-demand Design Results 209
6.3.3 Cooperative Coevolution and Generative Parametrization 210
6.3.3.1 Cooperative Coevolution 210
6.3.3.2 Diatomic Polarizer 211
6.3.3.3 Gradient Metasurface 211
6.4 Design Large-Scale, Weakly Coupled System 213
6.4.1 Weak Coupling Approximation 214
6.4.2 Analog Differentiator 214
6.4.3 Multiplexed Hologram 215
6.5 Auxiliary Methods for Generative Photonic Parametrization 217
6.5.1 Level Set Method 217
6.5.2 Fourier Level Set 218
6.5.3 Implicit Neural Representation 218
6.5.4 Periodic Boundary Conditions 220
6.6 Summary 221
References 221
7 Machine Learning Advances in Computational Electromagnetics 225
Robert Lupoiu and Jonathan A. Fan
7.1 Introduction 225
7.2 Conventional Electromagnetic Simulation Techniques 226
7.2.1 Finite Difference Frequency (FDFD) and Time (FDTD) Domain Solvers 226
7.2.2 The Finite Element Method (FEM) 229
7.2.2.1 Meshing 229
7.2.2.2 Basis Function Expansion 229
7.2.2.3 Residual Formulation 230
7.2.3 Method of Moments (MoM) 230
7.3 Deep Learning Methods for Augmenting Electromagnetic Solvers 231
7.3.1 Time Domain Simulators 231
7.3.1.1 Hardware Acceleration 231
7.3.1.2 Learning Finite Difference Kernels 232
7.3.1.3 Learning Absorbing Boundary Conditions 234
7.3.2 Augmenting Variational CEM Techniques Via Deep Learning 234
7.4 Deep Electromagnetic Surrogate Solvers Trained Purely with Data 235
7.5 Deep Surrogate Solvers Trained with Physical Regularization 240
7.5.1 Physics-Informed Neural Networks (PINNs) 240
7.5.2 Physics-Informed Neural Networks with Hard Constraints (hPINNs) 241
7.5.3 WaveY-Net 243
7.6 Conclusions and Perspectives 249
Acknowledgments 250
References 250
8 Design of Nanofabrication-Robust Metasurfaces Through Deep Learning-Augmented Multiobjective Optimization 253
Ronald P. Jenkins, Sawyer D. Campbell, and Douglas H. Werner
8.1 Introduction 253
8.1.1 Metasurfaces 253
8.1.2 Fabrication State-of-the-Art 253
8.1.3 Fabrication Challenges 254
8.1.3.1 Fabrication Defects 254
8.1.4 Overcoming Fabrication Limitations 255
8.2 Related Work 255
8.2.1 Robustness Topology Optimization 255
8.2.2 Deep Learning in Nanophotonics 256
8.3 DL-Augmented Multiobjective Robustness Optimization 257
8.3.1 Supercells 257
8.3.1.1 Parameterization of Freeform Meta-Atoms 257
8.3.2 Robustness Estimation Method 259
8.3.2.1 Simulating Defects 259
8.3.2.2 Existing Estimation Methods 259
8.3.2.3 Limitations of Existing Methods 259
8.3.2.4 Solver Choice 260
8.3.3 Deep Learning Augmentation 260
8.3.3.1 Challenges 261
8.3.3.2 Method 261
8.3.4 Multiobjective Global Optimization 267
8.3.4.1 Single Objective Cost Functions 267
8.3.4.2 Dominance Relationships 267
8.3.4.3 A Robustness Objective 269
8.3.4.4 Problems with Optimization and DL Models 269
8.3.4.5 Error-Tolerant Cost Functions 269
8.3.5 Robust Supercell Optimization 270
8.3.5.1 Pareto Front Results 270
8.3.5.2 Examples from the Pareto Front 271
8.3.5.3 The Value of Exhaustive Sampling 272
8.3.5.4 Speedup Analysis 273
8.4 Conclusion 275
8.4.1 Future Directions 275
Acknowledgments 276
References 276
9 Machine Learning for Metasurfaces Design and Their Applications 281
Kumar Vijay Mishra, Ahmet M. Elbir, and Amir I. Zaghloul
9.1 Introduction 281
9.1.1 ML/DL for RIS Design 283
9.1.2 ML/DL for RIS Applications 283
9.1.3 Organization 285
9.2 Inverse RIS Design 285
9.2.1 Genetic Algorithm (GA) 286
9.2.2 Particle Swarm Optimization (PSO) 286
9.2.3 Ant Colony Optimization (ACO) 289
9.3 DL-Based Inverse Design and Optimization 289
9.3.1 Artificial Neural Network (ANN) 289
9.3.1.1 Deep Neural Networks (DNN) 290
9.3.2 Convolutional Neural Networks (CNNs) 290
9.3.3 Deep Generative Models (DGMs) 291
9.3.3.1 Generative Adversarial Networks (GANs) 291
9.3.3.2 Conditional Variational Autoencoder (cVAE) 293
9.3.3.3 Global Topology Optimization Networks (GLOnets) 293
9.4 Case Studies 294
9.4.1 MTS Characterization Model 294
9.4.2 Training and Design 296
9.5 Applications 298
9.5.1 DL-Based Signal Detection in RIS 302
9.5.2 DL-Based RIS Channel Estimation 303
9.6 DL-Aided Beamforming for RIS Applications 306
9.6.1 Beamforming at the RIS 306
9.6.2 Secure-Beamforming 308
9.6.3 Energy-Efficient Beamforming 309
9.6.4 Beamforming for Indoor RIS 309
9.7 Challenges and Future Outlook 309
9.7.1 Design 310
9.7.1.1 Hybrid Physics-Based Models 310
9.7.1.2 Other Learning Techniques 310
9.7.1.3 Improved Data Representation 310
9.7.2 Applications 311
9.7.3 Channel Modeling 311
9.7.3.1 Data Collection 311
9.7.3.2 Model Training 311
9.7.3.3 Environment Adaptation and Robustness 312
9.8 Summary 312
Acknowledgments 313
References 313
Section IV Rf, Antenna, Inverse-scattering, and other Em Applications of Deep Learning 319
10 Deep Learning for Metasurfaces and Metasurfaces for Deep Learning 321
Clayton Fowler, Sensong An, Bowen Zheng, and Hualiang Zhang
10.1 Introduction 321
10.2 Forward-Predicting Networks 322
10.2.1 FCNN (Fully Connected Neural Networks) 323
10.2.2 CNN (Convolutional Neural Networks) 324
10.2.2.1 Nearly Free-Form Meta-Atoms 324
10.2.2.2 Mutual Coupling Prediction 327
10.2.3 Sequential Neural Networks and Universal Forward Prediction 330
10.2.3.1 Sequencing Input Data 331
10.2.3.2 Recurrent Neural Networks 332
10.2.3.3 1D Convolutional Neural Networks 332
10.3 Inverse-Design Networks 333
10.3.1 Tandem Network for Inverse Designs 333
10.3.2 Generative Adversarial Nets (GANs) 335
10.4 Neuromorphic Photonics 339
10.5 Summary and Outlook 340
References 341
11 Forward and Inverse Design of Artificial Electromagnetic Materials 345
Jordan M. Malof, Simiao Ren, and Willie J. Padilla
11.1 Introduction 345
11.1.1 Problem Setting 346
11.1.2 Artificial Electromagnetic Materials 347
11.1.2.1 Regime 1: Floquet-Bloch 348
11.1.2.2 Regime 2: Resonant Effective Media 349
11.1.2.3 All-Dielectric Metamaterials 350
11.2 The Design Problem Formulation 351
11.3 Forward Design 352
11.3.1 Search Efficiency 353
11.3.2 Evaluation Time 354
11.3.3 Challenges with the Forward Design of Advanced AEMs 354
11.3.4 Deep Learning the Forward Model 355
11.3.4.1 When Does Deep Learning Make Sense? 355
11.3.4.2 Common Deep Learning Architectures 356
11.3.5 The Forward Design Bottleneck 356
11.4 Inverse Design with Deep Learning 357
11.4.1 Why Inverse Problems Are Often Difficult 359
11.4.2 Deep Inverse Models 360
11.4.2.1 Does the Inverse Model Address Non-uniqueness? 360
11.4.2.2 Multi-solution Versus Single-Solution Models 360
11.4.2.3 Iterative Methods versus Direct Mappings 361
11.4.3 Which Inverse Models Perform Best? 361
11.5 Conclusions and Perspectives 362
11.5.1 Reducing the Need for Training Data 362
11.5.1.1 Transfer Learning 362
11.5.1.2 Active Learning 363
11.5.1.3 Physics-Informed Learning 363
11.5.2 Inverse Modeling for Non-existent Solutions 363
11.5.3 Benchmarking, Replication, and Sharing Resources 364
Acknowledgments 364
References 364
12 Machine Learning-Assisted Optimization and Its Application to Antenna and Array Designs 371
Qi Wu, Haiming Wang, and Wei Hong
12.1 Introduction 371
12.2 Machine Learning-Assisted Optimization Framework 372
12.3 Machine Learning-Assisted Optimization for Antenna and Array Designs 375
12.3.1 Design Space Reduction 375
12.3.2 Variable-Fidelity Evaluation 375
12.3.3 Hybrid Optimization Algorithm 378
12.3.4 Robust Design 379
12.3.5 Antenna Array Synthesis 380
12.4 Conclusion 381
References 381
13 Analysis of Uniform and Non-uniform Antenna Arrays Using Kernel Methods 385
Manel Martinez-Ramon, Jose Luis Rojo Alvarez, Arjun Gupta, and Christos Christodoulou
13.1 Introduction 385
13.2 Antenna Array Processing 386
13.2.1 Detection of Angle of Arrival 387
13.2.2 Optimum Linear Beamformers 388
13.2.3 Direction of Arrival Detection with Random Arrays 389
13.3 Support Vector Machines in the Complex Plane 390
13.3.1 The Support Vector Criterion for Robust Regression in the Complex Plane 390
13.3.2 The Mercer Theorem and the Nonlinear SVM 393
13.4 Support Vector Antenna Array Processing with Uniform Arrays 394
13.4.1 Kernel Array Processors with Temporal Reference 394
13.4.1.1 Relationship with the Wiener Filter 394
13.4.2 Kernel Array Processor with Spatial Reference 395
13.4.2.1 Eigenanalysis in a Hilbert Space 395
13.4.2.2 Formulation of the Processor 396
13.4.2.3 Relationship with Nonlinear MVDM 397
13.4.3 Examples of Temporal and Spatial Kernel Beamforming 398
13.5 DOA in Random Arrays with Complex Gaussian Processes 400
13.5.1 Snapshot Interpolation from Complex Gaussian Process 400
13.5.2 Examples 402
13.6 Conclusion 403
Acknowledgments 404
References 404
14 Knowledge-Based Globalized Optimization of High-Frequency Structures Using Inverse Surrogates 409
Anna Pietrenko-Dabrowska and Slawomir Koziel
14.1 Introduction 409
14.2 Globalized Optimization by Feature-Based Inverse Surrogates 411
14.2.1 Design Task Formulation 411
14.2.2 Evaluating Design Quality with Response Features 412
14.2.3 Globalized Search by Means of Inverse Regression Surrogates 414
14.2.4 Local Tuning Procedure 418
14.2.5 Global Optimization Algorithm 420
14.3 Results 421
14.3.1 Verification Structures 422
14.3.2 Results 423
14.3.3 Discussion 423
14.4 Conclusion 428
Acknowledgment 428
References 428
15 Deep Learning for High Contrast Inverse Scattering of Electrically Large Structures 435
Qing Liu, Li-Ye Xiao, Rong-Han Hong, and Hao-Jie Hu
15.1 Introduction 435
15.2 General Strategy and Approach 436
15.2.1 Related Works by Others and Corresponding Analyses 436
15.2.2 Motivation 437
15.3 Our Approach for High Contrast Inverse Scattering of Electrically Large Structures 438
15.3.1 The 2-D Inverse Scattering Problem with Electrically Large Structures 438
15.3.1.1 Dual-Module NMM-IEM Machine Learning Model 438
15.3.1.2 Receiver Approximation Machine Learning Method 440
15.3.2 Application for 3-D Inverse Scattering Problem with Electrically Large Structures 441
15.3.2.1 Semi-Join Extreme Learning Machine 441
15.3.2.2 Hybrid Neural Network Electromagnetic Inversion Scheme 445
15.4 Applications of Our Approach 450
15.4.1 Applications for 2-D Inverse Scattering Problem with Electrically Large Structures 450
15.4.1.1 Dual-Module NMM-IEM Machine Learning for Fast Electromagnetic Inversion of Inhomogeneous Scatterers with High Contrasts and Large Electrical Dimensions 450
15.4.1.2 Nonlinear Electromagnetic Inversion of Damaged Experimental Data by a Receiver Approximation Machine Learning Method 454
15.4.2 Applications for 3-D Inverse Scattering Problem with Electrically Large Structures 459
15.4.2.1 Super-Resolution 3-D Microwave Imaging of Objects with High Contrasts by a Semi-Join Extreme Learning Machine 459
15.4.2.2 A Hybrid Neural Network Electromagnetic Inversion Scheme (HNNEMIS) for Super-Resolution 3-Dimensional Microwave Human Brain Imaging 473
15.5 Conclusion and Future work 480
15.5.1 Summary of Our Work 480
15.5.1.1 Limitations and Potential Future Works 481
References 482
16 Radar Target Classification Using Deep Learning 487
Youngwook Kim
16.1 Introduction 487
16.2 Micro-Doppler Signature Classification 488
16.2.1 Human Motion Classification 490
16.2.2 Human Hand Gesture Classification 494
16.2.3 Drone Detection 495
16.3 SAR Image Classification 497
16.3.1 Vehicle Detection 497
16.3.2 Ship Detection 499
16.4 Target Classification in Automotive Radar 500
16.5 Advanced Deep Learning Algorithms for Radar Target Classification 503
16.5.1 Transfer Learning 504
16.5.2 Generative Adversarial Networks 506
16.5.3 Continual Learning 508
16.6 Conclusion 511
References 511
17 Koopman Autoencoders for Reduced-Order Modeling of Kinetic Plasmas 515
Indranil Nayak, Mrinal Kumar, and Fernando L. Teixeira
17.1 Introduction 515
17.2 Kinetic Plasma Models: Overview 516
17.3 EMPIC Algorithm 517
17.3.1 Overview 517
17.3.2 Field Update Stage 519
17.3.3 Field Gather Stage 521
17.3.4 Particle Pusher Stage 521
17.3.5 Current and Charge Scatter Stage 522
17.3.6 Computational Challenges 522
17.4 Koopman Autoencoders Applied to EMPIC Simulations 523
17.4.1 Overview and Motivation 523
17.4.2 Koopman Operator Theory 524
17.4.3 Koopman Autoencoder (KAE) 527
17.4.3.1 Case Study I: Oscillating Electron Beam 529
17.4.3.2 Case Study II: Virtual Cathode Formation 532
17.4.4 Computational Gain 534
17.5 Towards A Physics-Informed Approach 535
17.6 Outlook 536
Acknowledgments 537
References 537
Index 543
