“Doloc's book is a masterfully written and essential handbook for anyone involved in utilizing data to gain insights into their respective industries. With intellectual honesty, Doloc separates hype from reality, skillfully and intricately weaving a framework to harness the advances and recent developments in quantitative and computational finance. He challenges readers to adopt the best approaches for their applications, knowing the potential but also the limitations, and wisely problem solve. The author may have expertly designed this book for the trading community, but the takeaways are industry agnostic. A must-read for any academic or practitioner in data science, machine learning, and AI fields.”
—Rob Friesen, president & COO, Bright Trading, LLC; CEO & Director of Education, StockOdds, Inc.
“Cris Doloc's book is a great introduction to a fascinating field of Computational Intelligence and its applications to quantitative finance. Through examples and case studies covering a wide range of problems arising in quantitative finance from market making to derivative valuation and portfolio management the author demonstrates how to apply complex theoretical frameworks to solving practical problems. Using a sequence of case studies, Doloc shows quantitative researchers and practitioners the power of emerging Computational Intelligence and machine learning technologies to build intelligent solutions for quantitative finance.”
—Yuri Burlakov, Ph.D., head of Proprietary Research, Volant Trading
“Cris Doloc has created a valuable guide to Computational Intelligence and the application of these technologies to real-world problems. This book establishes a firm foundation to update the Financial Mathematics program curriculum and practitioners in this domain by presenting a systematic, contemporary development of data-intensive computation applied to financial market trading and investing. Using a sequence of case studies, Doloc shows quantitative researchers and practitioners the power of emerging Computational Intelligence and machine learning technologies to build intelligent solutions for quantitative finance.”
—Jeff Blaschak, Ph.D., data scientist and co-founder, Social Media Analytics, Inc.
“Cris Doloc has written a book that is more than just a solid introduction to the current state of the art in AI for quants; it is a solid introduction in how to think about AI for quants. In a field that is changing daily, the focus on application of techniques and critical thinking about the strengths and weaknesses of different approaches rather than on details of the latest tools makes time spent with this book a good investment in the future. The case studies in particular help ground the material in the real world of quantitative finance and provide powerful examples of the informed application of AI to finance.”
—John Ashley, Ph.D., director of Global Professional Services, Nvidia
“Doloc's book masterfully distills the complex world of quantitative trading into a clear guide that's an ideal starting point for new, would-be quants. It provides so many fresh insights into the space that even more seasoned practitioners can learn from it.”
—James L. Koutoulas, Esq., CEO, Typhon Capital Management
“Through a series of case studies, Doloc illustrates a number of examples of real-world problems designed to prepare the reader to work in the contemporary world of quantitative finance. I recommend this book to students of financial engineering and quantitative finance, and to all quantitatively oriented participants in all areas of finance.”
—Ilya Talman, president, Roy Talman & Associates, Inc.
Copyright © 2020 by Cris Doloc. All rights reserved.
Published by John Wiley & Sons, Inc., Hoboken, New Jersey.
Published simultaneously in Canada.
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600, or on the Web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permissions.
Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.
For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993, or fax (317) 572-4002.
Wiley publishes in a variety of print and electronic formats and by print-on-demand. Some material included with standard print versions of this book may not be included in e-books or in print-on-demand. If this book refers to media such as a CD or DVD that is not included in the version you purchased, you may download this material at http://booksupport.wiley.com. For more information about Wiley products, visit www.wiley.com/go/permissions.
Library of Congress Cataloging-in-Publication Data is Available:
ISBN 978-1-119-55050-1 (hardback)
ISBN 978-1-119-55052-5 (ePDF)
ISBN 978-1-119-55051-8 (ePub)
Cover Design: Wiley
Cover Images: © metamorworks/iStock.com, © BiksuTong/Shutterstock, © AVIcon/Shutterstock, © Classica2/Shutterstock
Dedicated to the memory of my father, Emil
Cris Doloc holds a PhD in Computational Physics and worked for more than two decades at the intersection of Quantitative and Computational Finance. He is an accomplished technology leader, who designed and led the implementation of several firm-wide trading, valuation, and risk systems. Cris's expertise extends from enterprise software architecture to High Performance Computing and Quantitative Trading.
Cris is currently teaching at the University of Chicago in the Financial Mathematics program, and is the founder of FintelligeX, a technology platform designed to promote quantitatively data-driven education. He is very passionate about the opportunities that recent developments in Cognitive Computing and Computational Intelligence could bring to the field of Quantitative and Computational Finance.
The metamorphosis of my ideas into the format of a book would not have been possible without the participation and help of many people. Unfortunately, I will not be able to name all of them, but I would like to start by thanking Bill Falloon, my editor at Wiley, who was at the origin of this project. Bill believed in this project from the beginning and helped me tremendously to navigate through the very complex and time-consuming process of writing a book. I would also like to thank Michael Henton, Beula Jaculin and Elisha Benjamin from Wiley for all the editorial help they have provided me with throughout this process.
This project could not have been completed without the constant understanding and support of my beloved wife, Lida, and of my precious daughter, Marie-Louise.
I am extremely grateful to my reviewers for their time and invaluable feedback. I would like to thank Professor Dan Nicolae, the Chair of the Statistics Department at the University of Chicago, Professor Roger Lee, the director of the Financial-Mathematics program at the University of Chicago, and Linda Kreitzman, the Executive Director of the MFE program at UC Berkeley, for their guidance and suggestions throughout the review process. I feel privileged to have had among the reviewers of this book some very influential names from the practitioner's realm like:
I am profoundly grateful to a large group of people that helped me to grow in my career, both as a physicist, and as a quant-technologist. I am deeply grateful to my high school physics teacher, Constantin Vasile, and to my PhD thesis adviser, Dr. Gilles Martin, for instilling in me the love for physics and problem solving. I am also very thankful to many amazing entrepreneurs and business leaders who entrusted me with important projects throughout my career. I would like to acknowledge many of my colleagues and mentors who helped me to shape my current views on how to apply the latest technology to solve the most important problems at hand.
Finally, I would like to thank my students for helping me to understand the importance of promoting problem-solving skills over content acquisition or tools management. The complexity of modern financial markets demands a continuous assimilation of the newest technology available, and a new breed of quant workforce will have to emerge. The quant of the twenty-first century will have to combine classical quant skills with deep knowledge of computer science and hands-on knowledge of modern HPC technologies. My message to them is this: There is no magic tool other than our own intelligence! Neither AI, nor any other “intelligence”-containing idiom could be a substitute for human intelligence! Our duty as educators is to kindle your interest in innovating, to nurture your problem-solving skills, and to guide your professional development. My earnest hope is that this book will be a useful device for reaching this purpose!
Cris Doloc, PhD
August 2019, Chicago
Additional materials for the book can be found at: https://www.fintelligex.com/book
The website includes the following materials: book's cover and description, table of contents and weblinks to coding resources.
“Life on earth is filled with many mysteries, but perhaps the most challenging of these is the nature of Intelligence.”
–Terrence J. Sejnowski, computational neurobiologist
Two decades of participation in the digital transformation of the trading industry as a system architect, quant, and trader, coupled with the experience of teaching in the Financial‐Mathematics program at the University of Chicago, provided me with a unique perspective that I will convey to the reader throughout this book. As both a practitioner and an educator, I wrote this book to assert the fact that the trading industry was, and continues to be, a very fertile ground for the adoption of cutting‐edge technologies.
The central message of this book is that the development of problem‐solving skills is much more important for the career advancement of a quantitative practitioner than the accretion and mastering of an ever‐increasing set of new tools that are flooding both the technical literature and the higher education curricula. While the majority of these tools become obsolete soon after their release into the public domain, acquiring an adequate level of problem‐solving expertise will endow the learner with a long‐lasting know‐how that will transcend ephemeral paradigms and cultural trends.
If the use of an exhaustive tool set is providing the solution architect with horizontal scalability, mastering the expertise of what tools should be used for any given problem will grant the user with the vertical scalability that is absolutely necessary for implementing intelligent solutions. While the majority of books about the application of machine intelligence to practical problem domains are focused on how to use tools and techniques, this book is built around six different types of problems that are relevant for the quantitative trading practitioner. The tools and techniques used to solve these problem types are described here in the context of the case studies presented, and not the other way around.
The impetus to write this book was triggered by the desire to introduce to my students the most recent scientific and technological developments related to the use of computationally intelligent techniques in quantitative finance. Given the strong interest of my students in topics related to the use of Machine Learning in finance, I decided to write a companion textbook for the course that I teach in the Financial‐Mathematics program, titled Case Studies in Computing for Finance.
Soon after I started working on the book, I realized that this project could also benefit a much larger category of readers, the quantitative trading practitioners. An important motivation for writing this book was to create awareness about the promises as well as the formidable challenges that the era of data‐driven decision‐making and Machine Learning (ML) are bringing forth, and about how these new developments may influence the future of the financial industry. The subject of Financial Machine Learning has attracted a lot of interest recently, specifically because it represents one of the most challenging problem spaces for the applicability of Machine Learning.
I want to reiterate that the central objective of this book is to promote the primacy of developing problem‐solving skills and to recommend solutions for evading the traps of keeping up with the relentless wave of new tools that are flooding the markets. Consequently the main purpose of this book is pedagogical in nature, and it is specifically aimed at defining an adequate level of engineering and scientific clarity when it comes to the usage of the term artificial intelligence, especially as it relates to the financial industry.
The term AI has become the mantra of our time, as this label is used more and more frequently as an intellectual wildcard by academicians and technologists alike. The AI label is particularly abused by media pundits, domain analysts, and venture capitalists. The excessive use of terms like AI disruption or AI revolution is the manifestation of a systemic failure to understand the technical complexity of this topic. The hype surrounding the so‐called artificial intelligence revolution is nothing but the most noticeable representation of a data point on Gartner's hype curve of inflated expectations.
This hype could be explained eventually by a mercantile impulse of using any opportunity to promote products and services that could benefit from the use of the AI label. It is rather common that a certain level of misunderstanding surrounds novel technology concepts when they are leaving the research labs and are crossing into the public domain. The idea that we are living in an era where the emergence of in silico intelligence could compete with human intelligence could very well qualify as “intellectual dishonesty”, as Professor Michael Jordan from Berkeley said on several occasions. Consequently, one of the main goals of this book is to clarify the terminology and to adjust the expectations of the reader in regard to the use of the term AI in quantitative finance.
Another very important driver behind this book is my own opinion about the necessity of updating the Financial‐Mathematics curriculum on two contemporary topics: data‐driven decision‐making (trading and investing) and Computational Intelligence. As a result, the first half of this book is dedicated to the introduction of two modern topics:
The general confusion created by the proliferation of the term AI is at the same time enthralling and frightening. While mass fascination comes from the failure to grasp the complexity of applying machine intelligence techniques to practical problems, the fear of an AI‐world taking over humanity is misleading, distracting, and therefore counterproductive. Whether or not Science will be able any time soon to understand and properly model the concept of Intelligence, enrolling both computers and humans into the fight to enhance human life is a major challenge ahead.
While solving the challenge of understanding general intelligence will be quintessential to the development of Artificial Intelligence it may also represent the foundation of a new branch of engineering. I will venture to call this new discipline Quantitative and Computational Engineering (Q&CE). Like many other classic engineering disciplines that have emerged in the past (e.g. Civil, Electrical, or Chemical), this new engineering discipline is going to be built on already mature concepts (i.e. information, data, algorithm, computing, and optimization). Many people call this new discipline Data Science. No matter the label employed, this new field will be focused on leveraging large amounts of data to enhance human life, so its development will require perspectives from a variety of other disciplines: from quantitative sciences like Mathematics and Statistics to Computational, Business, and Social sciences. One of the main goals of writing this book is to acknowledge the advent and to promote the development of this new engineering discipline that I label Quantitative and Computational Engineering.
The intended purpose of this book is to be a practical guide for both graduate students and quantitative practitioners alike. If the majority of books and papers published on the topic of Financial Machine Learning are structured around the different types and families of tools, I decided to center this book on practical problems, or Case Studies. I took on the big challenge to bridge the perceived gap between the academic literature on quantitative finance, which is sometimes seen as divorced from the practical reality, and the world of practitioners that is sometimes labeled as being short on scientific rigor. As a result I dedicated the second half of the book to the presentation of a set of Case Studies that are contemporarily relevant to the needs of the financial industry and at the same time representative of the problems that practitioners have to deal with. For this purpose I will consider categories of problems such as trade execution optimization, price dynamics forecast, portfolio management, market making, derivatives valuation, risk, and compliance. By reviewing dozens of recently peer‐reviewed publications, I selected what I believed to be the most practical, yet scientifically sound studies that could illustrate the current state‐of‐the‐art in Financial Machine Learning. I earnestly hope that this review of recently published information will be useful and engaging for both Financial‐Mathematics students as well as practitioners in quantitative finance who have high hopes for the applicability of Machine Learning, or more generally Computational Intelligence techniques in their fields of endeavor.
Last but not least I hope that other industries and sectors of the digital economy could use the financial industry's adoption model to further their business goals in two main directions: automation and innovation. Therefore, another important motivation in writing this book was to share with decision‐makers from other industries (e.g. Healthcare and Education) valuable lessons learned by the financial industry during its digital revolution.
The message that I want to convey in this book is one of confidence in the possibilities offered by this new era of data‐intensive computation. This message is not grounded on the current hype surrounding the latest technologies, but on a deep analysis of their effectiveness and also on my two decades of professional experience as a technologist, quant, and academic. Throughout my career I was driven by the passion to adopt cutting‐edge technologies for as long as they could be useful in solving real‐world problems. I wanted to convey this philosophy to my students as well as to the readers of this book. This book is an attempt to introduce the reader to the great potential offered by the new paradigm of Data‐Intensive Computing, or to what is called the fourth paradigm of scientific discovery to a variety of industries. Throughout this book I am going to promote the concept of Computational Intelligence as an umbrella of new technologies aimed at augmenting human performance (through automation) and engendering intelligence (via innovation and discovery) with examples from the emerging field of data‐driven trading. The use of computer systems to analyze and interpret data, coupled with the profound desire to learn from them and to reason without constant human involvement, is what Computational Intelligence is all about. As a means to convey the message I chose to introduce the reader to the realm of Computational Intelligence by presenting a series of Case Studies that are actionable and relevant in today's markets, as well as modern in their data‐driven approach.
This book is primarily intended for students and graduate students who contemplate becoming practitioners in the field of Financial Machine Learning and Computational Intelligence as well as for more‐seasoned trading practitioners who are interested in the new paradigm of data‐driven trading by using machine intelligence methodologies.
Another possible target audience is represented by technologists and decision‐makers from other sectors of the economy that currently undergo structural digital transformations and could have a major societal impact, like Education and Healthcare. This very large potential audience could learn extremely useful lessons from the digital revolution that shaped the financial industry in the last 10 to 15 years and could apply similar approaches for the successful early adoption of the newest technology available.
As mentioned before, the main goal of this book is to promote and advocate for the use of Computational Intelligence framework in the field of data‐driven trading. Since this is a quite novel and technically advanced topic, I choose to embed this message into a more readable narrative, one that will not exclude readers who may not be very fluent in the language of quantitative and computational sciences. By embedding the main message into a more readable narrative, I hope it will make it more appealing to nontechnical people.
The first part of the book is dedicated to introducing the two main topics of the book: Data‐Driven Decision‐Making and Computational Intelligence. As such:
The second part of the book introduces the reader to a series of Case Studies that are representative of the needs of today's financial industry. All the Case Studies presented are structured as follows: an introduction to the problem, a brief presentation on the state‐of‐the‐art in that specific area, a description of the implementation methodology employed, and a presentation of empirical results and conclusions.
The book concludes with Chapter 12, a summary of the three main goals of this book, namely to: