Cover: Emerging Extended Reality Technologies For Industry 4.0 by Jolanda G. Tromp, Dac-Nhuong Le and Chung Van Le

Scrivener Publishing
100 Cummings Center, Suite 541J
Beverly, MA 01915-6106

 

Publishers at Scrivener
Martin Scrivener (martin@scrivenerpublishing.com)
Phillip Carmical (pcarmical@scrivenerpublishing.com)

Emerging Extended Reality Technologies For Industry 4.0

Early Experiences with Conception, Design, Implementation, Evaluation and Deployment



Jolanda G. Tromp

State University of New York, Oswego, New York, USA

Dac-Nhuong Le

Haiphong University, Haiphong, Vietnam

Chung Van Le

Duy Tan University, Danang, Vietnam




No alt text required.

List of Figures

I.1 Seventeen sustainable development goals.
1.1 Task group projected workflow.
1.2 Detail from Zotero group.
1.3 Respondents reported the challenges of using MR in the classroom.
1.4 Research categories for which respondents reported use of MR.
1.5 Challenges in use of MR for researchers.
2.1 Human body simulation.
2.2 Simulated jaw and limb activity.
2.3 Simulation of the heart and circulatory system.
2.4 The process of marking and remembering anatomical points.
2.5 Users interact with the system via computer, mobile device and VR device.
2.6 Study group interacting via AR-enabled smartphones.
3.1 Ways to track children as they travel to and from school.
3.2 Systems parents want implemented in school buses.
3.3 School bus tracking sensor system use case.
3.4 Diagram of safety tracking and sensor system.
3.5 Sign up/sign in page.
3.6 Informing the bus driver and tracking the bus.
3.7 Add bus driver.
3.8 Bus driver.
3.9 School administrator.
4.1 Construction flowchart of aquantized speech image.
4.2 Flowchart of the proposed encryption technique applied to the quantized speech image for secure internet of things.
4.3 Image encrypted/decrypted.
4.4 Original speech signal.
4.5 Reconstructed speech signal after decryption and dequantization.
4.6 (a) The beginning of the original speech signal (zoomed); (b) the same region of the reconstructed speech signal after decryption and dequantization (zoomed).
4.7 (a) The middle of the original speech signal (zoomed); (b) the same region of the reconstructed speech signal after decryption and dequantization (zoomed).
4.8 (a) The end of the original speech signal (zoomed); (b) the same region of the reconstructed speech signal after decryption and dequantization (zoomed).
4.9 Correlation comparison.
4.10 (a) Histogram of the original speech quantized image; (b) Histogram of the encrypted speech quantized image.
5.1 The entrepreneurship ecosystem actors.
6.1 (left) Original navigation page with three cards on one screen; (right) redesigned navigation page with eight cards on one screen.
6.2 (left) The original 3D model page with 20 buttons; (right) new 3D model page with 8 buttons.
6.3 (left) The original buttons were small and located far from the thumb; (right) redesigned buttons are bigger and located at the bottom of the screen.
6.4 The redesigned buttons have a highlight effect.
7.1 An example of an early cumbersome HMD.
7.2 Examples of see-through AR interfaces, both of which are lightweight and hands-free augmented reality glasses that permit the user to view the world around them; Google Glass (left) and Magic Leap One (right).
7.3 Example of gesture-based interaction through the HoloLens and HoloMuse application.
7.4 An example of natural feature-based point detection for using AR on rock art. Recognition of multiple known points is required to permit accurate registrationofthe augmented image [40].
7.5 A range of white balance settings which match color temperature at set times, increasing consistency of color tone with the real scene.
7.6 An example of real-world shadow creation under varied environmental conditions; virtual teapot (left) and real teapot (right).
7.7 Virtual and augmented design space consists of layers of meaning: architectural, semantic, social, and temporal.
7.8 A user constructs a sense of presence from successful interaction with the virtual or augmented world (adapted from Tromp [49]).
7.9 Virtual and augmented design space consists of layers of possible interactive functionality and optimization of the user interactions in terms of task-flow and interaction-feedback loop mappings between real world, virtual world and augmented world/object at the architectural, semantic, social, and temporal level [50].
7.10 AR within the TombSeer project: testing functionality of gaze selection with the Meta headset (left), and in-situ testing within the replica of an Egyptian tomb (right).
8.1 A real student learning experience operating in LMS after upgrades based on the TELECI approach.
8.2 Knowledge tests and course content adaptation process diagram.
8.3 Detailed process of Preliminary Survey analysis.
8.4 TELECI Interface to VS subsystem.
8.5 Energy flow model for virtual student’s ecosystem: System Energy Depot (Depot), Virtual Student’s Energy Buffer (VS), and k Energy Storages for Learning Objects (LO1 ... LOk).
8.6 TELECI interface to AI component.
8.7 The organizational landscape environment holds TELECI system controlling learners’ data gathering, distribution, and flow among computational and storage resources.
8.8 SABI algorithm key variables in action.
8.9 Learning related events { an arrival process.
8.10 The model of behavior noise impact on data produced by Real Learner. Modality decoding can lead to detection errors.
8.11 Interaction time detection error. A small learning object requires less time to interact. Distractions produce “behavior noise.”
10.1 Components of the electronic engine control system for gas diesel engines: (1) information calculation block, (2, 10) intake manifold temperature and pressure sensors, (3, 9) cooling agent temperature sensors, (4) barometric correction sensor, (5) crankshaft position sensor, (6) camshaft position sensor, (7) crankshaft and camshaft position sensors adapter, (8) block of thermocouples.
10.2 Common rail fuel supply system for a truck diesel engine: (1) fuel feed pump, (2) filter, (3) low pressure fuel lines, (4) high pressure fuel pump, (5.1, 5.2, 5.3) high pressure fuel lines, (6.1) coupler, (6.2) common rail, (7) pressure sensor, (8) emergency valve, (9) CR injector.
10.3 HP fuel pump: (1) HP fuel pump body, (2) tappet, (3) push fitting, (4) insert section body, (5) plunger support, (6) tappet roller, (7) camshaft, (8) plunger spring, (9) plunger, (10) plunger liner, (11) delivery valve, (12) delivery valve seat, (13) delivery valve spring, (14) control valve, (15) plate fastening the HP fuel pump to the engine, (16) HP fuel pump drive flange; (17) HP fuel pump rear cover, (18) bypass valve.
10.4 CR injectors: (1) injection nozzle, (2) spacer, (3) armature, (4) armature spring, (5) electromagnet, (6) magnet spring, (7) valve seat, (8) armature body, (9) spacer, (10) ring, (11) nozzle nut, (12) injector body, (13, 14) connectors, (15) electromagnet power wires, (16) needle valve, (17) needle valve spring, (18) control chamber, (19) clamp mounting point.
10.5 Injector nozzle diagram: (1) injector nozzle body, (2) needle valve, (3, 4) spray holes of the first (lower) and second (upper) groups, (5) intake edges of the first group of spray holes in the sack volume, (6) sack volume, (7) intake edges of the second group of spray holes on the locking cone, (8) locking cone of the injector nozzle body seat, (9) needle valve locking cone.
10.6 Computational scheme of the injector nozzle.
10.7 Scheme of clearance between the locking cones of the needle valve and the seat.
10.8 Hydraulic characteristics of the injector nozzle.
10.9 Injection characteristics of the FFS of the diesel engine V8 having fm_Inline_13_2.jpg with correcting injector nozzle (nc = 1400 rpm,qinj = 58mm3).
10.10 The force of an electric magnet formed by two primary and one basic control impulses.
10.11 Results of calculation of the displacement of the needle valve hnv and control valve hv.
10.12 Calculated boot-type injection rate shape formed by two primary and one main control impulses.
10.13 Results of computer modeling of the influence f the time of the main electric impulse start on the control valve lift hv and needle valve lift hnv of the common rail injector and on the injector rate shaping.
10.14 Results of computer modeling of the influence of variation of the duration of the primary electric impulse on the control valve lift hv and needle valve lift hnv of the common rail injector and on the injection rate shaping.
10.15 Results of computer modeling of the influence of the variation of duration of the additional electric impulse on the control valve lift hv and needle valve lift hnvof the common rail injector and on the injection rate shaping.
10.16 Results of computer modeling of the control valve lift hv, needle valve lift hnv and the injection rate shaping when applying a primary impulse followed by additional electric impulses.
10.17 Load characteristic of the gas diesel engine at n = 1420 rpm.
10.18 Speed characteristic of the gas diesel engine.
11.1 Traditional data mining process.
11.2 The main components of data stream mining.
11.3 Types of concept drift.
11.4 Forms of concept drift.
12.1 Contour network with continuous state space and time.
12.2 Closed contour: (a) chain, (b) honeycomb, (c) chainmail.
12.3 Binary closed contour chain with N=8 with state (1, 1, 0, 0, 1,0).
12.4 Open chainmail of size 2m × 2n
12.5 (a) Closed chainmail with one-directional movement. (b) Closed chainmail with co-directional movement.
12.6 Diagonal on 4 × 4 chainmail
12.7 Particle movement rule.
13.1 Architecture of the software system for the visualization.
13.2 An example of complex road network created using basic road elements.
13.3 Visualization of traffic on small road network.
13.4 Basic road fragments.
13.5 Two neighboring intersections on a map of Moscow.
13.6 Simulating traffic on two neighboring intersections: T-Crossroad + X-Crossroad.
13.7 Simulating traffic on two neighboring intersections: U-Turn + X-Crossroad.
13.8 Clover leaf intersection (scheme).
13.9 Clover leaf intersection (results of the simulation).
13.10 Modeling on neighboring intersections (map (a) and scheme (b)).
13.11 Modeling of neighboring intersections (results of simulation at different time moments).

List of Tables

1.1 Software in use for classrooms and research.
1.2 Anecdotal responses about challenges.
2.1 Organ systems in the human body.
4.1 Results for hardware implementations.
4.2 Results for correlation and entropy.
5.1 Comparison of studies of entrepreneurship activities in social media.
6.1 Scenarios and tasks, as used in the experiment.
6.2 Descriptive statistics for the average TCR.
6.3 Results of independent t-test for TCR.
6.4 Descriptive statistics for the average TCR of individual tasks.
6.5 Descriptive statistics for the average TOT.
6.6 Results of independent t-test for TOT.
6.7 Descriptive statistics for the average TOT of individual tasks.
6.8 Independent t-test results of TOT of each task.
6.9 Descriptive statistics for the average ASQ score.
6.10 Results of independent t-test for average ASQ score.
6.11 Descriptive statistics for the average ASQ score of individual tasks.
6.12 Independent t-test results of the ASQ score of each task.
6.13 Descriptive plot for the average PSSUQ score.
6.14 Results of independent t-test for PSSUQ score.
11.1 Comparison of data mining and data stream mining
11.2 Data-based and task-based approaches.
11.3 Classification of algorithms for concept drift detection.
11.4 Datasets for DSM with concept drift.

Foreword

The 5th International Conference on Communication, Management and Information Technology (ICCMIT’19)1 was jointly organized in Vienna, Austria, on March 26-28, 2019,, by the Universal Society of Applied Research, Prague, Czech Republic, in collaboration with the University of Denver, Colorado, United States of America. The main objective of this conference, which has been running yearly since 2015, was to bring together researchers, societies, new technology experts, and manufacturing professionals interested or already involved in R&D with new technologies and innovative ideas at any scale and create a community spirit and learn from each other. The aim of this yearly conference is to facilitate sharing of research, ideas, and lessons learned by international researchers and explore collaborations to begin working towards achieving the highest standards of ICT. One of the major overall themes of the conference is Industry 4.0 and smart citizens, smart cities, smart factories, etc. These recent and innovative Industry 4.0 technologies are prototypes for the next generation of 21st century production systems. Advancement of information technologies and their convergence with operational technologies paves the way for an evolution of production systems. To remain competitive in the market, enterprises want to utilize these technological advancements in order to solve current challenges and serve customers in new ways which were not imagined before. In order to provide new services and products quickly, new methods and business models are needed. In order to exploit these new technologies they have to be introduced at manufacturing level.

The Fourth Industrial Revolution is emerging and evolving at an exponential rather than linear pace and disrupting almost every industry in every country around the globe. These changes are signaling the transformation of entire systems of production, management, and governance. Industry 4.0 will impact our business, and those businesses which are prepared are already implementing changes to adapt to a future where smart machines will allow them to escalate their business success. The participants of the ICCMIT’19 conference deeply discussed their diverse views on Industry 4.0 based on their expertise, and the major topics of discussion related to the digital divide, how academic institutions can support and advance the digital transformation, how to organize human/robot interactions in the digital transformation era, and how to lead the digital transformation of manufacturing companies. During the conference, researchers and practitioners exchanged their experiences with the different types of 21st century smart methods of monitoring and operating engineering, analytics and servicing activities, including the impacts of automation and smart sensing for the improvement of the quality and accuracy of the entire product or service supply chain.

Ibrahiem M. M. El Emary, PhD
Professor of Computer Science and Systems
Faculty of Arts and Humanities, King Abdulaziz University, Jeddah, Saudi Arabia
Organizer of ICCMIT 2019

Introduction

Introduction to Key Industry 4.0 Technologies

The broad adoption of seventeen sustainable development goals has strongly emphasized using new emergent technologies for creating new solutions for our 21st century problems. This also calls for new business models and the reassessment of the current modes of government and manufacturing. This will require a global collaborative effort to work out how to employ new technologies to find these solutions, leading to a “Digital Revolution.” The United Nations has identified key sustainable development goals (SDGs) to transform our world that should be part of the Digital Revolution.1 These goals are listed below and in Figure I.1.

The convergence of associated emerging technologies in the form of the Internet of Things along with Artificial Intelligence will create large-scale intelligent networks. In addition, Machine Learning (ML) will facilitate the emergence of a worldwide Internet of Smart Things. These combinations of Artificial Intelligence and the Internet of Things can be called an Artificial Intelligence supported Internet of Things (AIIoT). The networks that implement these converged technologies will be the first major events of the Digital Revolution. It marks the time when users begin to see how vendor components and smart systems implement frictionless economics across integrated Smart Cities.

The exponential growth of AIIoT is based on the numerous configurations of new, smaller, more affordable networked sensors that can communicate with each other and potentially with all other sensors and processes in the supply chain. The configurations and implementation of the networked sensors and the data analytics for business intelligence need to be tailor-made to the requirements of human users, including the entire value chain and supply chain. The estimated 26 million software developers at the end of 2019 is predicted to grow to more than 27 million by 2023. Clearly, new approaches will need to be developed to assure that system professionals are compensated at a level that assures there will be an adequate supply of skilled workers.12

fm_image_20_3.jpg

Figure I.1 Seventeen sustainable development goals.

The smooth implementation of automation across a number of industries relies on the coming together of stakeholders. Early successes will translate into rapid adoption and provide the foundation for later intelligent applications. There is a need for international standards in order to facilitate an efficient global collaboration. A number of stakeholders will be involved in collaborative efforts to make this happen. At a minimum these groups will include the following:

Voicing Concerns and Digital Twins, Blockchain, Big Data Analytics, Cognitive Computing, and 3D Printing, among others.

Communication between stakeholders is key to realizing the benefits of AIIoT. To facilitate a global conversation forum, there should be a framework for communication that permits conversations in all directions and is capable of addressing any issue. Experts with system-level experience are needed, who can draw on their experiences to avoid pitfalls and minimize risks. The framework may take the form of conferences, meetups or website forums. Moderators working closely with system experts can address issues that are raised by participants. It is important that a solid foundation is put in place that will support additional innovations that will be added at a later date.

The new digital economy is a paradigm shift, towards a data marketplace with many diverse data producers who need a distributed brokering system; a ledger, with seamless insurance and logistics, big data analytics and self-learning systems. The technologies that enable the new digital economy paradigm shift are interconnected, overlapping and converging.

These emergent AIIoT Industry 4.0 pillars currently are: Extended Reality (XR: virtual reality, augmented reality, mixed reality and other new forms still under development) development and deployment education, Sensors, Internet of Things (IoT) and Cybersecurity, Mobile Technologies and Cloud Computing, Machine-to-Machine Communication

The sections in this book are organized according to these various branches of the emergent technologies and the chapters address the evolving research that paves the way and enables solutions for smart cities and smart global solutions. Each chapter provides a time-stamp of current activities towards the paradigm shift and provides the necessary vision statements and use-case descriptions that help steer the adoption of smart city components. These vision statements will be translated into directives or regulations to be enacted by stakeholders. This involves a sequence of examinations and reviews by each participating company. The best general sequence follows the following processes or similar ones:

The early AIIoT participants will be strategically placed to exponentially grow their productivity through AI and ML analysis and optimization. The superior products and services will rapidly reduce the market demand for other products and services that are outdated and lack functionality or quality, and such operations would systematically shut down due to inefficiency and high costs. Those who are already on the underdeveloped side of the digital divide will increasingly be more rapidly pushed out of competition. The configurations and implementation of the networked sensors and the data analytics for business intelligence need to be tailor-made to the requirements of the human users, and the business and value chains. Human needs for a prosperous, healthy, happy, safe, sustainable environment, are the main drivers for change and innovation. Successful international and intercultural respectful solutions for 21st century global issues can be built, using emergent technologies in novel ways. It is therefore necessarily a human-centered innovation design and development process.

Dr. Jolanda G. Tromp
Director of Center for Visualization and Simulation
Duy Tan University, Da Nang, Vietnam


John Bottoms
CEO FirstStar Systems
Boston, Massachusetts, USA

Preface

In Industry 4.0, extended reality (XR) technologies, such as virtual reality (VR) and augmented reality (AR), are creating location-aware applications to interact with smart objects and smart processes via cloud computing strategies enabled with artificial intelligence (AI) and the Internet ofThings (IoT). Factories and processes can be automated and machines can be enabled with self-monitoring capabilities. Smart objects are given the ability to analyze and communicate with each other and their human coworkers, delivering the opportunity for much smoother processes, and freeing up workers for other tasks. Industry 4.0-enabled smart objects can be monitored, designed, tested and controlled via their digital twins, and these processes and controls are visualized in VR/AR. The Industry 4.0 technologies provide powerful, largely unexplored application areas that will revolutionize the way we work, collaborate and live our lives. It is important to understand the opportunities and impacts of the new technologies and the effects from a production, safety and societal point of view.

This book presents empirical research results from user-centered qualitative and quantitative experiments on these new applications, and facilitates a discussion forum to explore the latest trends in XR applications for Industry 4.0. Additional contributions were collected via a public call to raise the number and quality of the chapters to the highest standard.

The selected best papers in this book are from the International Conference on Communication, Management and Information (ICCMIT’19), www.icmit.net (International Conference on Communication, Management and Information, 26-28 March 2019, Vienna, Austria) plus an open call for contributions showcasing the state-of-the-art of these new technologies and applications in terms of design challenges, evaluations and long-term use implications.

As we have entered the Industrial Revolution 4.0, XR applications, in combination with AI/IoT technologies, are fundamentally changing the way we work and live, generally referred to as Industry 4.0 or IR 4.0. Developments in these fields are very important because the novel combinations of these technologies can help improve and save lives, improve the work and collaboration processes and create smart objects in smart systems and smart cities. This in turn has far-reaching effects for educational, organizational, economic and social improvements to the way we work, teach, learn and care for ourselves and each other.

This book aims to combine the early explorations and discussions of Industry 4.0 key features that need to be addressed on a global scale:

Dac-Nhuong Le, PhD
Associate Professor of Computer Science
Deputy Head, Faculty of Information Technology
Hai Phong University, Hai Phong, Vietnam

Acknowledgments

First of all, I would like to thank the authors for contributing their excellent chapters to this book. Without their contributions, this book would not have been possible. Thanks to all my colleagues and friends for sharing my happiness at the start of this project and following up with their encouragement when it seemed too difficult to complete.

I would like to acknowledge and thank the most important people in my life, my father, my mother and my partner, for their support. This book has been a long-cherished dream of mine which would not have been turned into reality without the support and love of these amazing people, who encouraged me despite my not giving them the proper time and attention. I am also grateful to my best friends for their blessings, unconditional love, patience and encouragement.

Dac-Nhuong Le, PhD
Associate Professor of Computer Science
Deputy Head, Faculty of Information Technology
Hai Phong University, Hai Phong, Vietnam