Details

Data Lakes

1. Aufl.

von: Anne Laurent, Dominique Laurent, Cédrine Madera
139,99 €
Verlag:	Wiley
Format:	PDF
Veröffentl.:	09.04.2020
ISBN/EAN:	9781119720423
Sprache:	englisch
Anzahl Seiten:	244

In den Warenkorb

Als Gutschein

DRM-geschütztes eBook, Sie benötigen z.B. Adobe Digital Editions und eine Adobe ID zum Lesen.

Beschreibungen

Titelbeschreibung

The concept of a data lake is less than 10 years old, but they are already hugely implemented within large companies. Their goal is to efficiently deal with ever-growing volumes of heterogeneous data, while also facing various sophisticated user needs. However, defining and building a data lake is still a challenge, as no consensus has been reached so far. Data Lakes presents recent outcomes and trends in the field of data repositories. The main topics discussed are the data-driven architecture of a data lake; the management of metadata – supplying key information about the stored data, master data and reference data; the roles of linked data and fog computing in a data lake ecosystem; and how gravity principles apply in the context of data lakes. A variety of case studies are also presented, thus providing the reader with practical examples of data lake management.

Inhaltsverzeichnis

Preface xi Chapter 1. Introduction to Data Lakes: Definitions and Discussions 1 Anne LAURENT, Dominique LAURENT and Cédrine MADERA 1.1. Introduction to data lakes 1 1.2. Literature review and discussion 3 1.3. The data lake challenges 7 1.4. Data lakes versus decision-making systems 10 1.5. Urbanization for data lakes 13 1.6. Data lake functionalities 17 1.7. Summary and concluding remarks 20 Chapter 2. Architecture of Data Lakes 21 Houssem CHIHOUB, Cédrine MADERA, Christoph QUIX and Rihan HAI 2.1. Introduction 21 2.2. State of the art and practice 25 2.2.1. Definition 25 2.2.2. Architecture 25 2.2.3. Metadata 26 2.2.4. Data quality 27 2.2.5. Schema-on-read 27 2.3. System architecture 28 2.3.1. Ingestion layer 29 2.3.2. Storage layer 31 2.3.3. Transformation layer 32 2.3.4. Interaction layer 33 2.4. Use case: the Constance system 33 2.4.1. System overview 33 2.4.2. Ingestion layer 35 2.4.3. Maintenance layer 35 2.4.4. Query layer 37 2.4.5. Data quality control 38 2.4.6. Extensibility and flexibility 38 2.5. Concluding remarks 39 Chapter 3. Exploiting Software Product Lines and Formal Concept Analysis for the Design of Data Lake Architectures 41 Marianne HUCHARD, Anne LAURENT, Thérèse LIBOUREL, Cédrine MADERA and André MIRALLES 3.1. Our expectations 41 3.2. Modeling data lake functionalities 43 3.3. Building the knowledge base of industrial data lakes 46 3.4. Our formalization approach 49 3.5. Applying our approach 51 3.6. Analysis of our first results 53 3.7. Concluding remarks 55 Chapter 4. Metadata in Data Lake Ecosystems 57 Asma ZGOLLI, Christine COLLET† and Cédrine MADERA 4.1. Definitions and concepts 57 4.2. Classification of metadata by NISO 58 4.2.1. Metadata schema 59 4.2.2. Knowledge base and catalog 60 4.3. Other categories of metadata 61 4.3.1. Business metadata 61 4.3.2. Navigational integration 63 4.3.3. Operational metadata 63 4.4. Sources of metadata 64 4.5. Metadata classification 65 4.6. Why metadata are needed 70 4.6.1. Selection of information (re)sources 70 4.6.2. Organization of information resources 70 4.6.3. Interoperability and integration 70 4.6.4. Unique digital identification 71 4.6.5. Data archiving and preservation 71 4.7. Business value of metadata 72 4.8. Metadata architecture 75 4.8.1. Architecture scenario 1: point-to-point metadata architecture 75 4.8.2. Architecture scenario 2: hub and spoke metadata architecture 76 4.8.3. Architecture scenario 3: tool of record metadata architecture 78 4.8.4. Architecture scenario 4: hybrid metadata architecture 79 4.8.5. Architecture scenario 5: federated metadata architecture 80 4.9. Metadata management 82 4.10. Metadata and data lakes 86 4.10.1. Application and workload layer 86 4.10.2. Data layer 88 4.10.3. System layer 90 4.10.4. Metadata types 90 4.11. Metadata management in data lakes 92 4.11.1. Metadata directory 93 4.11.2. Metadata storage 93 4.11.3. Metadata discovery 94 4.11.4. Metadata lineage 94 4.11.5. Metadata querying 95 4.11.6. Data source selection 95 4.12. Metadata and master data management 96 4.13. Conclusion 96 Chapter 5. A Use Case of Data Lake Metadata Management 97 Imen MEGDICHE, Franck RAVAT and Yan ZHAO 5.1. Context 97 5.1.1. Data lake definition 98 5.1.2. Data lake functional architecture 100 5.2. Related work 103 5.2.1. Metadata classification 104 5.2.2. Metadata management 105 5.3. Metadata model 106 5.3.1. Metadata classification 106 5.3.2. Schema of metadata conceptual model 110 5.4. Metadata implementation 111 5.4.1. Relational database 112 5.4.2. Graph database 115 5.4.3. Comparison of the solutions 119 5.5. Concluding remarks 121 Chapter 6. Master Data and Reference Data in Data Lake Ecosystems 123 Cédrine MADERA 6.1. Introduction to master data management 125 6.1.1. What is master data? 125 6.1.2. Basic definitions 125 6.2. Deciding what to manage 126 6.2.1. Behavior 126 6.2.2. Lifecycle 127 6.2.3. Cardinality 127 6.2.4. Lifetime 128 6.2.5. Complexity 128 6.2.6. Value 128 6.2.7. Volatility 129 6.2.8. Reuse 129 6.3. Why should I manage master data? 130 6.4. What is master data management? 131 6.4.1. How do I create a master list? 136 6.4.2. How do I maintain a master list? 138 6.4.3. Versioning and auditing 139 6.4.4. Hierarchy management 140 6.5. Master data and the data lake 141 6.6. Conclusion 143 Chapter 7. Linked Data Principles for Data Lakes 145 Alessandro ADAMOU and Mathieu D’AQUIN 7.1. Basic principles 145 7.2. Using Linked Data in data lakes 148 7.2.1. Distributed data storage and querying with linked data graphs 151 7.2.2. Describing and profiling data sources 153 7.2.3. Integrating internal and external data 156 7.3. Limitations and issues 159 7.4. The smart cities use case 162 7.4.1. The MK Data Hub 163 7.4.2. Linked data in the MK Data Hub 165 7.5. Take-home message 169 Chapter 8. Fog Computing 171 Arnault IOUALALEN 8.1. Introduction 171 8.2. A little bit of context 171 8.3. Every machine talks 172 8.4. The volume paradox 173 8.5. The fog, a shift in paradigm 174 8.6. Constraint environment challenges 176 8.7. Calculations and local drift 177 8.7.1. A short memo about computer arithmetic 178 8.7.2. Instability from within 179 8.7.3. Non-determinism from outside 180 8.8. Quality is everything 181 8.9. Fog computing versus cloud computing and edge computing 184 8.10. Concluding remarks: fog computing and data lake 185 Chapter 9. The Gravity Principle in Data Lakes 187 Anne LAURENT, Thérèse LIBOUREL, Cédrine MADERA and André MIRALLES 9.1. Applying the notion of gravitation to information systems 187 9.1.1. Universal gravitation 187 9.1.2. Gravitation in information systems 189 9.2. Impact of gravitation on the architecture of data lakes 193 9.2.1. The case where data are not moved 195 9.2.2. The case where processes are not moved 197 9.2.3. The case where the environment blocks the move 198 Glossary 201 References 207 List of Authors 217 Index 219

Autorenportrait

Anne Laurent is a Full Professor at the University of Montpellier, France, and teaches at the Polytech Montpellier Engineering School. She is also a member of the LIRMM laboratory at the University of Montpellier, where she works on the semantic web, data mining, data warehousing, data lakes and fuzzy logic. Dominique Laurent is Emeritus Professor at Cergy-Pontoise University, France. He is a member of the ETIS-CNRS laboratory and his main research interests include database theory, database updates, data mining and data warehousing. Cedrine Madera is an Executive Information Architect at IBM, France. She is a doctor in Data Science and, in close collaboration with the world of academics, she works on the evolution of information systems.