Details

AI and Machine Learning for Network and Security Management

IEEE Press Series on Networks and Service Management 1. Aufl.

von: Yulei Wu, Jingguo Ge, Tong Li
96,99 €
Verlag:	Wiley
Format:	EPUB
Veröffentl.:	28.10.2022
ISBN/EAN:	9781119835899
Sprache:	englisch
Anzahl Seiten:	304

In den Warenkorb

Als Gutschein

DRM-geschütztes eBook, Sie benötigen z.B. Adobe Digital Editions und eine Adobe ID zum Lesen.

Beschreibungen

Titelbeschreibung

AI AND MACHINE LEARNING FOR NETWORK AND SECURITY MANAGEMENT Extensive Resource for Understanding Key Tasks of Network and Security Management AI and Machine Learning for Network and Security Management covers a range of key topics of network automation for network and security management, including resource allocation and scheduling, network planning and routing, encrypted traffic classification, anomaly detection, and security operations. In addition, the authors introduce their large-scale intelligent network management and operation system and elaborate on how the aforementioned areas can be integrated into this system, plus how the network service can benefit. Sample ideas covered in this thought-provoking work include: <ul><li> How cognitive means, e.g., knowledge transfer, can help with network and security management</li> <li> How different advanced AI and machine learning techniques can be useful and helpful to facilitate network automation</li> <li> How the introduced techniques can be applied to many other related network and security management tasks</li></ul> Network engineers, content service providers, and cybersecurity service providers can use AI and Machine Learning for Network and Security Management to make better and more informed decisions in their areas of specialization. Students in a variety of related study programs will also derive value from the work by gaining a base understanding of historical foundational knowledge and seeing the key recent developments that have been made in the field.

Inhaltsverzeichnis

Author Biographies xiii Preface xv Acknowledgments xvii Acronyms xix 1 Introduction 1 1.1 Introduction 1 1.2 Organization of the Book 3 1.3 Conclusion 6 References 6 2 When Network and Security Management Meets AI and Machine Learning 9 2.1 Introduction 9 2.2 Architecture of Machine Learning-Empowered Network and Security Management 10 2.3 Supervised Learning 12 2.3.1 Classification 12 2.3.2 Regression 15 2.4 Semisupervised and Unsupervised Learning 15 2.4.1 Clustering 17 2.4.2 Dimension Reduction 17 2.4.3 Semisupervised Learning 18 2.5 Reinforcement Learning 18 2.5.1 Policy-Based 21 2.5.2 Value-Based 22 2.6 Industry Products on Network and Security Management 24 2.6.1 Network Management 24 2.6.1.1 Cisco DNA Center 24 2.6.1.2 Sophie 25 2.6.1.3 Juniper EX4400 Switch 25 2.6.1.4 Juniper SRX Series Services Gateway 25 2.6.1.5 H3C SeerAnalyzer 26 2.6.2 Security Management 27 2.6.2.1 SIEM, IBM QRadar Advisor with Watson 27 2.6.2.2 FortiSandbox 27 2.6.2.3 FortiSIEM 28 2.6.2.4 FortiEDR 28 2.6.2.5 FortiClient 29 2.6.2.6 H3C SecCenter CSAP 29 2.7 Standards on Network and Security Management 29 2.7.1 Network Management 29 2.7.1.1 Cognitive Network Management 30 2.7.1.2 End-to-End 5G and Beyond 30 2.7.1.3 Software-Defined Radio Access Network 32 2.7.1.4 Architectural Framework for ML in Future Networks 32 2.7.2 Security Management 33 2.7.2.1 Securing AI 33 2.8 Projects on Network and Security Management 34 2.8.1 Poseidon 34 2.8.2 NetworkML 35 2.8.3 Credential-Digger 36 2.8.4 Adversarial Robustness Toolbox 37 2.9 Proof-of-Concepts on Network and Security Management 38 2.9.1 Classification 38 2.9.1.1 Phishing URL Classification 38 2.9.1.2 Intrusion Detection 39 2.9.2 Active Learning 39 2.9.3 Concept Drift Detection 40 2.10 Conclusion 41 References 42 3 Learning Network Intents for Autonomous Network Management 49 3.1 Introduction 49 3.2 Motivation 52 3.3 The Hierarchical Representation and Learning Framework for Intention Symbols Inference 53 3.3.1 Symbolic Semantic Learning (SSL) 53 3.3.1.1 Connectivity Intention 55 3.3.1.2 Deadlock Free Intention 56 3.3.1.3 Performance Intention 57 3.3.1.4 Discussion 57 3.3.2 Symbolic Structure Inferring (SSI) 57 3.4 Experiments 59 3.4.1 Datasets 59 3.4.2 Experiments on Symbolic Semantic Learning 60 3.4.3 Experiments on Symbolic Structure Inferring 62 3.4.4 Experiments on Symbolic Structure Transferring 64 3.5 Conclusion 66 References 66 4 Virtual Network Embedding via Hierarchical Reinforcement Learning 69 4.1 Introduction 69 4.2 Motivation 70 4.3 Preliminaries and Notations 72 4.3.1 Virtual Network Embedding 72 4.3.1.1 Substrate Network and Virtual Network 72 4.3.1.2 The VNE Problem 72 4.3.1.3 Evaluation Metrics 73 4.3.2 Reinforcement Learning 74 4.3.3 Hierarchical Reinforcement Learning 75 4.4 The Framework of VNE-HRL 75 4.4.1 Overview 75 4.4.2 The High-level Agent 77 4.4.2.1 State Encoder for HEA 77 4.4.2.2 Estimated Long-term Cumulative Reward 78 4.4.2.3 Short-term High-level Reward 78 4.4.3 The Low-level Agent 78 4.4.3.1 State Encoder for LEA 79 4.4.3.2 Estimated Long-term Cumulative Reward 79 4.4.3.3 Short-term Low-level Reward 80 4.4.4 The Training Method 80 4.5 Case Study 80 4.5.1 Experiment Setup 80 4.5.2 Comparison Methods 81 4.5.3 Evaluation Results 81 4.5.3.1 Performance Over Time 81 4.5.3.2 Performance of Various VNRs with Diverse Resource Requirements 82 4.6 Related Work 84 4.6.1 Traditional Methods 84 4.6.2 ML-based Algorithms 84 4.7 Conclusion 85 References 85 5 Concept Drift Detection for Network Traffic Classification 91 5.1 Related Concepts of Machine Learning in Data Stream Processing 91 5.1.1 Assumptions and Limitations 91 5.1.1.1 Availability of Learning Examples 91 5.1.1.2 Availability of the Model 92 5.1.1.3 Concept to be Learned 92 5.1.2 Concept Drift and Its Solution 92 5.2 Using an Active Approach to Solve Concept Drift in the Intrusion Detection Field 94 5.2.1 Application Background 94 5.2.2 System Workflow 95 5.3 Concept Drift Detector Based on CVAE 96 5.3.1 CVAE-based Drift Indicator 96 5.3.2 Drift Analyzer 97 5.3.3 The Performance of CVAE-based Concept Drift Detector 98 5.3.3.1 Comparison Drift Detectors 99 5.3.3.2 Experiment Settings 99 5.4 Deployment and Experiment in Real Networks 101 5.4.1 Data Collection and Feature Extraction 101 5.4.2 Data Analysis and Parameter Setting 103 5.4.3 Result Analysis 103 5.5 Future Research Challenges and Open Issues 105 5.5.1 Adaptive Threshold m 105 5.5.2 Computational Cost of Drift Detectors 105 5.5.3 Active Learning 105 5.6 Conclusion 105 References 106 6 Online Encrypted Traffic Classification Based on Lightweight Neural Networks 109 6.1 Introduction 109 6.2 Motivation 109 6.3 Preliminaries 110 6.3.1 Problem Definition 110 6.3.2 Packet Interaction 111 6.4 The Proposed Lightweight Model 111 6.4.1 Preprocessing 112 6.4.2 Feature Extraction 112 6.4.2.1 Embedding 112 6.4.2.2 Attention Encoder 113 6.4.2.3 Fully Connected Layer 115 6.5 Case Study 115 6.5.1 Evaluation Metrics 115 6.5.2 Baselines 116 6.5.3 Datasets 117 6.5.4 Evaluation on Datasets 118 6.5.4.1 Evaluation on Dataset A 118 6.5.4.2 Evaluation on Dataset B 120 6.6 Related Work 121 6.6.1 Encrypted Traffic Classification 122 6.6.2 Packet-Based Methods 122 6.6.3 Flow-Based Methods 122 6.6.3.1 Traditional Machine Learning-Based Methods 123 6.6.3.2 Deep Learning-Based Methods 124 6.7 Conclusion 124 References 125 7 Context-Aware Learning for Robust Anomaly Detection 129 7.1 Introduction 129 7.2 Pronouns 133 7.3 The Proposed Method – AllRobust 135 7.3.1 Problem Statement 135 7.3.2 Log Parsing 135 7.3.3 Log Vectorization 138 7.3.4 Anomaly Detection 142 7.3.4.1 Implementation of SSL 143 7.4 Experiments 145 7.4.1 Datasets 145 7.4.1.1 HDFS Dataset 145 7.4.1.2 BGL Dataset 146 7.4.1.3 Thunderbird Dataset 146 7.4.2 Model Evaluation Indicators 147 7.4.3 Supervised Deep Learning-based Log Anomaly Detection on Imbalanced Log Data 148 7.4.3.1 Data Preprocessing 148 7.4.3.2 Hyperparameters and Environmental Settings 149 7.4.3.3 Training on Multiclass Imbalanced Log Data 149 7.4.3.4 Training on Binary Imbalanced Log Data 150 7.4.4 Semisupervised Deep Learning-based Log Anomaly Detection on Imbalanced Log Data 152 7.4.4.1 The Methods of Enhancing Log Data 152 7.4.4.2 Anomaly Detection with a Single Log 153 7.4.4.3 Anomaly Detection with a Log-based Sequence 156 7.5 Discussion 157 7.6 Conclusion 158 References 159 8 Anomaly Classification with Unknown, Imbalanced and Few Labeled Log Data 165 8.1 Introduction 165 8.2 Examples 167 8.2.1 The Feature Extraction of Log Analysis 167 8.2.1.1 Statistical Feature Extraction 168 8.2.1.2 Semantic Feature Extraction 170 8.2.2 Few-Shot Problem 170 8.3 Methodology 172 8.3.1 Data Preprocessing 172 8.3.1.1 Log Parsing 172 8.3.1.2 Log Enhancement 173 8.3.1.3 Log Vectorization 174 8.3.2 The Architecture of OpenLog 174 8.3.2.1 Encoder Module 174 8.3.2.2 Prototypical Module 177 8.3.2.3 Relation Module 178 8.3.3 Training Procedure 179 8.3.4 Objective Function 180 8.4 Experimental Results and Analysis 180 8.4.1 Experimental Design 181 8.4.1.1 Baseline 181 8.4.1.2 Evaluation Metrics 181 8.4.2 Datasets 183 8.4.2.1 Data Processing 184 8.4.3 Experiments on the Unknown Class Data 185 8.4.4 Experiments on the Imbalanced Data 188 8.4.5 Experiments on the Few-shot Data 188 8.5 Discussion 190 8.6 Conclusion 191 References 192 9 Zero Trust Networks 199 9.1 Introduction to Zero-Trust Networks 199 9.1.1 Background 199 9.1.2 Zero-Trust Networks 200 9.2 Zero-Trust Network Solutions 201 9.2.1 Zero-Trust Networks Based on Access Proxy 201 9.2.2 Zero Trust Networks Based on SDP 203 9.2.3 Zero-Trust Networks Based on Micro-Segmentation 204 9.3 Machine Learning Powered Zero Trust Networks 206 9.3.1 Information Fusion 208 9.3.2 Decision Making 210 9.4 Conclusion 212 References 212 10 Intelligent Network Management and Operation Systems 215 10.1 Introduction 215 10.2 Traditional Operation and Maintenance Systems 215 10.2.1 Development of Operation and Maintenance Systems 215 10.2.1.1 Manual Operation and Maintenance 216 10.2.1.2 Tool-Based Operation and Maintenance 216 10.2.1.3 Platform Operation and Maintenance 217 10.2.1.4 DevOps 217 10.2.1.5 AIOps 218 10.2.2 Open-Source Operation and Maintenance Systems 218 10.2.2.1 Nagios 219 10.2.2.2 Zabbix 221 10.2.2.3 Prometheus 223 10.2.3 Summary 224 10.3 Security Operation and Maintenance 225 10.3.1 Introduction 225 10.3.2 Open-Source Security Tools 226 10.3.2.1 Access Control 226 10.3.2.2 Security Audit and Intrusion Detection 227 10.3.2.3 Penetration Testing 227 10.3.2.4 Vulnerability Scanning 231 10.3.2.5 CI/CD Security 234 10.3.2.6 Deception 234 10.3.2.7 Data Security 234 10.3.3 Summary 237 10.4 AIOps 238 10.4.1 Introduction 238 10.4.2 Open-Source AIOps and Algorithms 239 10.4.2.1 Research Progress of Anomaly Detection 239 10.4.2.2 Metis 242 10.4.2.3 UAVStack 244 10.4.2.4 Skyline 244 10.4.3 Summary 247 10.5 Machine Learning-Based Network Security Monitoring and Management Systems 248 10.5.1 Architecture 248 10.5.2 Physical Facility Layer 248 10.5.3 Virtual Resource Layer 249 10.5.4 Orchestrate Layer 250 10.5.5 Policy Layer 250 10.5.6 Semantic Description Layer 251 10.5.7 Application Layer 251 10.5.8 Center for Intelligent Analytics of Big Data 251 10.5.9 Programmable Measurement and Auditing 252 10.5.10 Overall Process 252 10.5.11 Summary 253 10.6 Conclusion 253 References 254 11 Conclusions, and Research Challenges and Open Issues 257 11.1 Conclusions 257 11.2 Research Challenges and Open Issues 258 11.2.1 Autonomous Networks 258 11.2.2 Reinforcement Learning Powered Solutions 259 11.2.3 Traffic Classification 259 11.2.4 Anomaly Detection 260 11.2.5 Zero-Trust Networks 261 References 262 Index 263

Autorenportrait

Yulei Wu, is a Senior Lecturer with the Department of Computer Science, Faculty of Environment, Science and Economy, University of Exeter, UK. His research focuses on networking, Internet of Things, edge intelligence, information security, and ethical AI. He serves as an Associate Editor for IEEE Transactions on Network and Service Management, and IEEE Transactions on Network Science and Engineering, as well as an Editorial Board Member of Computer Networks, Future Generation Computer Systems, and Nature Scientific Reports at Nature Portfolio. He is a Senior Member of the IEEE and the ACM, and a Fellow of the HEA (Higher Education Academy). Jingguo Ge, is currently a Professor of the Institute of Information Engineering, Chinese Academy of Sciences (CAS), and also a Professor of School of Cyber Security, University of Chinese Academy of Sciences. His research focuses on Future Network Architecture, 5G/6G, Software-defined networking (SDN), Cloud Native networking, Zero Trust Architecture. He has published more than 60 research papers and is the holder of 28 patents. He participated in the formulation of 3 ITU standards on IMT2020. Tong Li, is currently a Senior Engineer of Institute of Information Engineering at the Chinese Academy of Sciences (CAS). His research and engineering focus on Computer Networks, Cloud Computing, Software-Defined Networking (SDN), and Distributed Network and Security Management. He participated 2 ITU standards on IMT2020 and developed many large-scale software systems on SDN, network management and orchestration.

Back cover copy

Extensive Resource for Understanding Key Tasks of Network and Security Management AI and Machine Learning for Network and Security Management covers a range of key topics of network automation for network and security management, including resource allocation and scheduling, network planning and routing, encrypted traffic classification, anomaly detection, and security operations. In addition, the authors introduce their large-scale intelligent network management and operation system and elaborate on how the aforementioned areas can be integrated into this system, plus how the network service can benefit. Sample ideas covered in this thought-provoking work include: <ul><li> How cognitive means, e.g., knowledge transfer, can help with network and security management</li> <li> How different advanced AI and machine learning techniques can be useful and helpful to facilitate network automation</li> <li> How the introduced techniques can be applied to many other related network and security management tasks</li></ul> Network engineers, content service providers, and cybersecurity service providers can use AI and Machine Learning for Network and Security Management to make better and more informed decisions in their areas of specialization. Students in a variety of related study programs will also derive value from the work by gaining a base understanding of historical foundational knowledge and seeing the key recent developments that have been made in the field.