Cover
Title Page
Preface
List of Abbreviations
1 Introduction
1. 1.1 Robot Programming Methods
2. 1.2 Programming by Demonstration
3. 1.3 Historical Overview of Robot PbD
4. 1.4 PbD System Architecture
5. 1.5 Applications
6. 1.6 Research Challenges
7. 1.7 Summary
8. References
2 Task Perception
1. 2.1 Optical Tracking Systems
2. 2.2 Vision Cameras
3. 2.3 Summary
4. References
3 Task Representation
1. 3.1 Level of Abstraction
2. 3.2 Probabilistic Learning
3. 3.3 Data Scaling and Aligning
4. 3.4 Summary
5. References
4 Task Modeling
1. 4.1 Gaussian Mixture Model (GMM)
2. 4.2 Hidden Markov Model (HMM)
3. 4.3 Conditional Random Fields (CRFs)
4. 4.4 Dynamic Motion Primitives (DMPs)
5. 4.5 Summary
6. References
5 Task Planning
1. 5.1 Gaussian Mixture Regression
2. 5.2 Spline Regression
3. 5.3 Locally Weighted Regression
4. 5.4 Gaussian Process Regression
5. 5.5 Summary
6. References
6 Task Execution
1. 6.1 Background and Related Work
2. 6.2 Kinematic Robot Control
3. 6.3 Vision‐Based Trajectory Tracking Control
4. 6.4 Image‐Based Task Planning
5. 6.5 Robust Image‐Based Tracking Control
6. 6.6 Discussion
7. 6.7 Summary
8. References
Index
End User License Agreement

List of Tables

Chapter 05
1. Table 5.1 LBG algorithm.
2. Table 5.2 Mean values and standard deviations of the computation times for learning the trajectories from Experiment 1 and Experiment 2.
3. Table 5.3 Mean values and standard deviations of the computation times for learning the trajectories from Experiment 1 and Experiment 2 by applying the GMM/GMR approach.
4. Table 5.4 Means and standard deviations of the classification rates obtained by CRF for the painting task in Experiment 1.
5. Table 5.5 Means and standard deviations of the classification rates obtained by CRF for the peening task in Experiment 2.
6. Table 5.6 Means and standard deviations of the classification rates obtained by HMM and CRF for the painting task from Experiment 1.
7. Table 5.7 Means and standard deviations of the classification rates obtained by HMM and CRF for the peening task from Experiment 2.
Chapter 06
1. Table 6.1 Evaluation of the trajectory tracking with IBVS under intrinsic camera parameters errors ranging from 0 to 80%.
2. Table 6.2 Evaluation of the trajectory tracking without vision‐based control under intrinsic camera parameters errors ranging from 0 to 80%.
3. Table 6.3 Coordinates of the object position at the end of the task expressed in the robot base frame (in millimeters) under extrinsic camera parameters errors ranging from 0 to 80%.

List of Illustrations

Chapter 01
1. Figure 1.1 Classification of robot programming methods.
2. Figure 1.2 The user demonstrates the task in front of a robot learner, and is afterward actively involved in the learning process by moving the robot’s arms during the task reproduction attempts to refine the learned skills (Calinon and Billard (2007a).
3. Figure 1.3 Block diagram of the information flow in a general robot PbD system.
4. Figure 1.4 Kinesthetic teaching of feasible postures in a confined workspace. During kinesthetic teaching the human operator physically grabs the robot and executes the task.
5. Figure 1.5 The PbD setup for teaching peg‐in‐hole assembly tasks includes a teleoperated robot gripper and the objects manipulated by a human expert. Tracking is done using magnetic sensors.
6. Figure 1.6 Teleoperation scheme for PbD—master arm (on the left) and slave arm (on the right) used for human demonstrations.
7. Figure 1.7 AR training of an assembly task using adaptive visual aids (AVAs).
8. Figure 1.8 Mobile AR component including a haptic bracelet.
9. Figure 1.9 Sensory systems used for PbD task observation.
10. Figure 1.10 Learning levels in PbD.
11. Figure 1.11 Learning at a symbolic level of abstraction by representing the decomposed task into a hierarchy of motion primitives.
12. Figure 1.12 A humanoid robot is learning and reproducing trajectories for a figure‐8 movement from human demonstrations.
13. Figure 1.13 Control of a 19 DoFs humanoid robot using PbD.
14. Figure 1.14 A kitchen helping robot learns the sequence of actions for cooking from observation of human demonstrations.
15. Figure 1.15 The experimental setup used for teaching (on the left) includes an ultrasound machine, an ultrasound phantom model and a handheld ultrasound transducer with force sensing and built‐in 3D position markers for optical tracking system. The robotically controlled ultrasound scanning is also shown (on the right).
16. Figure 1.16 Robot grasp planning application.
Chapter 02
1. Figure 2.1 (a) Position camera sensors of the optical tracking system Optotrak Certus; (b) optical markers attached on a tool are tracked during a demonstration of a “figure 8” motion.
Chapter 03
1. Figure 3.1 (a) Two sequences with different number of measurements: a reference sequence of 600 measurement data points and a test sequence of 800 measurement data points; (b) the test sequence is linearly scaled to the same number of measurements as the reference sequence; and (c) the test sequence is aligned with the reference sequence using DTW.
Chapter 04
1. Figure 4.1 Graphical representation of an HMM. The shaded nodes depict the sequence of observed elements and the white nodes depict the hidden states sequence .
2. Figure 4.2 Graphical representation of a CRF with linear chain structure.
Chapter 05
1. Figure 5.1 An example of initial selection of trajectory key points with the LBG algorithm. The key points are indicated using circles. The following input features are used for clustering: (a) normalized positions coordinates ; (b) normalized velocities ; and (c) normalized positions and velocities .
2. Figure 5.2 Illustration of the weighted curve fitting. For the clusters with small variance of the key points high weights for spline fitting are assigned, whereas for the clusters with high variance of the key points low weights are assigned, which results in loose fitting.
3. Figure 5.3 Diagram representation of the presented approach for robot PbD. The solid lines depict automatic steps in the data processing. For a set of observed trajectories X₁, …, X_M, the algorithm automatically generates a generalized trajectory X_gen, which is transferred to a robot for task reproduction.
4. Figure 5.4 (a) Experimental setup for Experiment 1: panel, painting tool, and reference frame; (b) perception of the demonstrations with the optical tracking system; and (c) the set of demonstrated trajectories.
5. Figure 5.5 Distributions of (a) velocities, (b) accelerations, and (c) jerks of the demonstrated trajectories by the four subjects. The bottom and top lines of the boxes plot the 25th and 75th percentile for the distributions, the bands in the middle represent the medians, and the whiskers display the minimum and maximum of the data.
6. Figure 5.6 Initial assignment of key points for the trajectory with minimum distortion.
7. Figure 5.7 Spatiotemporally aligned key points from all demonstrations. For the parts of the demonstrations which correspond to approaching and departing of the tool with respect to the panel, the clusters of key points are more scattered, when compared to the painting part of the demonstrations.
8. Figure 5.8 (a) RMS errors for the clusters of key points, (b) weighting coefficients with threshold values of 1/2 and 2 standard deviations, and (c) weighting coefficients with threshold values of 1/6 and 6 standard deviations.
9. Figure 5.9 Generalization of the tool orientation from the spatiotemporally aligned key points. Roll angles are represented by a dashed line, pitch angles are represented by a solid line, and yaw angles are represented by a dash–dotted line. The dots in the plot represent the orientation angles of the key points.
10. Figure 5.10 Generalized trajectory for the Cartesian x–y–z position coordinates of the object.
11. Figure 5.11 Distributions of (a) velocities, (b) accelerations, and (c) jerks for the demonstrated trajectories by the subjects and the generalized trajectory.
12. Figure 5.12 (a) The part used for Experiment 2, with the surfaces to be painted bordered with solid lines and (b) the set of demonstrated trajectories.
13. Figure 5.13 (a) Generalized trajectory for Experiment 2 and (b) execution of the trajectory by the robot learner.
14. Figure 5.14 Generated trajectory for reproduction of the task of panel painting for Experiment 1, based on Calinon and Billard (2004). The most consistent trajectory (dashed line) corresponds to the observation sequence with the highest likelihood of being generated by the learned HMM.
15. Figure 5.15 RMS differences for the reproduction trajectories generated by the presented approach (X_G1), the approaches proposed in Calinon and Billard (2004) (X_G2), Asfour et al. (2006) (X_G3), and the demonstrated trajectories (X₁–X₁₂). As the color bar on the right side indicates, lighter nuances of the cells depict greater RMS differences.
16. Figure 5.16 Cumulative sums of the RMS differences for the reproduction trajectories generated by the presented approach (X_G1), the approaches proposed in Calinon and Billard (2004) (X_G2), and Asfour et al. (2006) (X_G3). (a) Demonstrated trajectories (X₁–X₁₂) from Experiment 1 and (b) demonstrated trajectories (X_1–X₅) from Experiment 2.
17. Figure 5.17 Generalized trajectory obtained by the GMM/GMR method (Calinon, 2009) for (a) Experiment 1 and (b) Experiment 2, in Section 4.4.4.
18. Figure 5.18 (a) Experimental setup for Experiment 1 showing the optical tracker, the tool with attached markers and the object for painting. (b) One of the demonstrated trajectories with the initially selected key points. The arrow indicates the direction of the tool’s motion. (c) Demonstrated trajectories and the generalized trajectory.
19. Figure 5.19 (a) Plot of a sample demonstrated trajectory for the peening task from Experiment 2 and a set of initially selected key points, and (b) generalized trajectory for the peening experiment.
Chapter 06
1. Figure 6.1 Response of classical IBVS: feature trajectories in the image plane, camera trajectory in Cartesian space, camera velocities, and feature errors in the image plane.
2. Figure 6.2 Response of classical PBVS: feature trajectories in the image plane, camera trajectory in Cartesian space, camera velocities, and feature errors in the image plane.
3. Figure 6.3 The learning cell, consisting of a robot, a camera, and an object manipulated by the robot. The assigned coordinate frames are: camera frame ℱ_c(O_c, x_c, y_c, z_c), object frame ℱ_o(O_o, x_o, y_o, z_o), robot base frame ℱ_b(O_b, x_b, y_b, z_b), and robot’s end‐point frame ℱ_e(O_e, x_e, y_e, z_e). The transformation between a frame i and a frame j is given by a position vector and a rotation matrix .
4. Figure 6.4 (a) The eigenvectors of the covariance matrix ê₁, ê₂ for three demonstrations at times k = 10, 30, and 44; (b) observed parameters for feature 1, . The vector is required to lie in the region bounded by η_min and η_max.
5. Figure 6.5 (a) Three demonstrated trajectories of the object in the Cartesian space. For one of the trajectories, the initial and ending coordinate frames of the object are shown, along with the six coplanar features; (b) projected demonstrated trajectories of the feature points onto the image plane of the camera; (c) reference image feature trajectories produced by Kalman smoothing and generalized trajectories produced by the optimization model; and (d) object velocities from the optimization model.
6. Figure 6.6 (a) Demonstrated Cartesian trajectories of the object, with the features points, and the initial and ending object frames; (b) demonstrated and reference linear velocities of the object for x‐ and y‐coordinates of the motions; (c) reference image feature trajectories from the Kalman smoothing and the corresponding generalized trajectories from the optimization; and (d) the demonstrated and retrieved generalized object trajectories in the Cartesian space. The initial state and the ending state are depicted with square and cross marks, respectively.
7. Figure 6.7 (a) Experimental setup showing the robot in the home configuration and the camera. The coordinate axes of the robot base frame and the camera frame are depicted; (b) the object with the coordinate frame axes and the features.
8. Figure 6.8 Sequence of images from the kinesthetic demonstrations.
9. Figure 6.9 (a) Feature trajectories in the image space for one‐sample demonstration; (b) demonstrated trajectories, Kalman‐smoothed (reference) trajectory, and corresponding planned trajectory for one feature point (for the feature no. 2); (c) demonstrated linear and angular velocities of the object and the reference velocities obtained by Kalman smoothing; (d) Kalman‐smoothed (reference) image feature trajectories and the generalized trajectories obtained from the optimization procedure; and (e) demonstrated and the generated Cartesian trajectories of the object in the robot base frame.
10. Figure 6.10 (a) Desired and robot‐executed feature trajectories in the image space; (b) tracking errors for the pixel coordinates (u, v) of the five image features in the image space; (c) tracking errors for x‐, y‐, and z‐coordinates of the object in the Cartesian space; (d) translational velocities of the object from the IBVS tracker.
11. Figure 6.11 Task execution without optimization of the trajectories: (a) Desired and robot‐executed feature trajectories in the image space; (b) tracking errors for the pixel coordinates (u, v) of the five image features in the image space; (c) tracking errors for x‐, y‐, and z‐coordinates of the object in the Cartesian space; (d) translational velocities of the object from the IBVS tracker.
12. Figure 6.12 (a) Demonstrated trajectories of the image feature points, superimposed with the desired and robot‐executed feature trajectories; (b) desired and executed trajectories for slowed down trajectories.
13. Figure 6.13 (a) Image feature trajectories for one of the demonstrations; (b) demonstrated trajectories, reference trajectory from the Kalman smoothing, and the corresponding generalized trajectory for one of the feature points; (c) desired and robot‐executed image feature trajectories; (d) translational velocities of the object from the IBVS tracker; (e) tracking errors for pixel coordinates (u, v) of the five image features in the image space; and (f) tracking errors for x‐, y‐, and z‐coordinates of the object in the Cartesian space.
14. Figure 6.14 Projected trajectories of the feature points in the image space with (a) errors of 5, 10, and 20% introduced for all camera intrinsic parameters; (b) errors of 5% introduced for the focal length scaling factors of the camera.

Pages

iii
iv
v
x
xi
xii
xiii
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
43
44
45
46
47
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194

This edition first published 2017
© 2017 John Wiley & Sons, Inc.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by law. Advice on how to obtain permission to reuse material from this title is available at http://www.wiley.com/go/permissions.

The right of Aleksandar Vakanski and Farrokh Janabi‐Sharifi to be identified as the author(s) of this work has been asserted in accordance with law.

Registered Offices
John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USA

Editorial Office
111 River Street, Hoboken, NJ 07030, USA

For details of our global editorial offices, customer services, and more information about Wiley products visit us at www.wiley.com.

Wiley also publishes its books in a variety of electronic formats and by print‐on‐demand. Some content that appears in standard print versions of this book may not be available in other formats.

Limit of Liability/Disclaimer of Warranty

The publisher and the authors make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties; including without limitation any implied warranties of fitness for a particular purpose. This work is sold with the understanding that the publisher is not engaged in rendering professional services. The advice and strategies contained herein may not be suitable for every situation. In view of on‐going research, equipment modifications, changes in governmental regulations, and the constant flow of information relating to the use of experimental reagents, equipment, and devices, the reader is urged to review and evaluate the information provided in the package insert or instructions for each chemical, piece of equipment, reagent, or device for, among other things, any changes in the instructions or indication of usage and for added warnings and precautions. The fact that an organization or website is referred to in this work as a citation and/or potential source of further information does not mean that the author or the publisher endorses the information the organization or website may provide or recommendations it may make. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this works was written and when it is read. No warranty may be created or extended by any promotional statements for this work. Neither the publisher nor the author shall be liable for any damages arising here from.

Library of Congress Cataloguing‐in‐Publication Data

Names: Vakanski, Aleksandar, author. | Janabi-Sharifi, Farrokh, 1959– author.
Title: Robot learning by visual observation / Aleksandar Vakanski, Farrokh Janabi‐Sharifi.
Description: Hoboken, NJ, USA : John Wiley & Sons, Inc., [2017] | Includes bibliographical references and index.
Identifiers: LCCN 2016041712| ISBN 9781119091806 (cloth) | ISBN 9781119091783 | ISBN 9781119091998 (epub) | ISBN 9781119091783 (epdf)
Subjects: LCSH: Robot vision. | Machine learning. | Robots--Control systems. | Automatic programming (Computer science)
Classification: LCC TJ211.3 .V35 2017 | DDC 629.8/92631-dc23
LC record available at https://lccn.loc.gov/2016041712

Cover image: fandijki/Gettyimages

Cover design by Wiley

Preface

The ability to transfer knowledge has played a quintessential role in the advancement of our species. Several evolutionary innovations have significantly leveraged the knowledge transfer. One example is rewiring of the neuronal networks in primates’ brains to form the so‐called mirror neuron systems, so that when we observe tasks performed by others, a section of the brain that is responsible for observation and a section that is responsible for motor control are concurrently active. Through this, when observing actions, the brain is attempting at the same time to learn how to reproduce these actions. The mirror neuron system represents an especially important learning mechanism among toddlers and young kids, stimulating them to acquire skills by imitating the actions of adults around them. However, the evolutionary processes and modifications are very slow and prodigal, and as we further developed, we tended to rely on employing our creativity in innovating novel means for transferring knowledge. By inventing writing and alphabets as language complements, we were able to record, share, and communicate knowledge at an accelerated rate. Other innovations that followed, such as the printing press, typing machine, television, personal computers, and World Wide Web, each have revolutionized our ability to share knowledge and redefined the foundations for our current level of technological advancement.

As our tools and machines have grown more advanced and sophisticated, society recognized a need to transfer knowledge to the tools in order to improve efficiency and productivity, or to reduce efforts or costs. For instance, in the manufacturing industry, robotic technology has emerged as a principal means in addressing the increased demand for accuracy, speed, and repeatability. However, despite the continuous growth of the number of robotic applications across various domains, the lack of interfaces for quick transfer of knowledge in combination with the lack of intelligence and reasoning abilities has practically limited operations of robots to preprogrammed repetitive tasks performed in structured environments. Robot programming by demonstration (PbD) is a promising form for transferring new skills to robots from observation of skill examples performed by a demonstrator. Borrowed from the observational imitation learning mechanisms among humans, PbD has a potential to reduce the costs for the development of robotic applications in the industry. The intuitive programming style of PbD can allow robot programming by end‐users who are experts in performing an industrial task but may not necessarily have programming or technical skills. From a broader perspective, another important motivation for the development of robot PbD systems is the old dream of humankind about robotic assistance in performing everyday domestic tasks. Future advancements in PbD would allow the general population to program domestic and service robots in a natural way by demonstrating the required task in front of a robot learner.

Arguably, robot PbD is currently facing various challenges, and its progress is dependent on the advancements in several other research disciplines. On the other hand, the strong demand for new robotic applications across a wide range of domains, combined with the reduced cost of actuators, sensors, and processing memory, is amounting for unprecedented progress in the field of robotics. Consequently, a major motivation for writing this book is our hope that the next advancements in PbD can further increase the number of robotic applications in the industry and can speed up the advent of robots into our homes and offices for assistance in performing daily tasks.

The book attempts to summarize the recent progress in the robot PbD field. The emphasis is on the approaches for probabilistic learning of tasks at a trajectory level of abstraction. The probabilistic representation of human motions provides a basis for encapsulating relevant information from multiple demonstrated examples of a task. The book presents examples of learning industrial tasks of painting and shot peening by employing hidden Markov models (HMMs) and conditional random fields (CRFs) to probabilistically encode the tasks. Another aspect of robot PbD covered in depth is the integration of vision‐based control in PbD systems. The presented methodology for visual learning performs all the steps of a PbD process in the image space of a vision camera. The advantage of such learning approach is the enhanced robustness to modeling and measurement errors.

The book is written at a level that requires a background in robotics and artificial intelligence. Targeted audience consists of researchers and educators in the field, graduate students, undergraduate students with technical knowledge, companies that develop robotic applications, and enthusiasts interested in expanding their knowledge on the topic of robot learning. The reader can benefit from the book by grasping the fundamentals of vision‐based learning for robot programming and use the ideas in research and development or educational activities related to robotic technology.

We would like to acknowledge the help of several collaborators and researchers who made the publication of the book possible. We would like to thank Dr. Iraj Mantegh from National Research Council (NRC)—Aerospace Manufacturing Technology Centre (AMTC) in Montréal, Canada, for his valuable contributions toward the presented approaches for robotic learning of industrial tasks using HMMs and CRFs. We are also thankful to Andrew Irish for his collaboration on the aforementioned projects conducted at NRC‐Canada. We acknowledge the support from Ryerson University for access to pertinent resources and facilities, and Natural Sciences and Engineering Research Council of Canada (NSERC) for supporting the research presented in the book. We also thank the members of the Robotics, Mechatronics and Automation Laboratory at Ryerson University for their help and support. Particular thanks go to both Dr. Abdul Afram and Dr. Shahir Hasanzadeh who provided useful comments for improving the readability of the book. Last, we would like to express our gratitude to our families for their love, motivation, and encouragement in preparing the book.