Table of Contents
Wiley & SAS Business Series
Title Page
Copyright Page
Can Futurists Be Trusted?
What Exactly Is a Hinge of History?
Chapter 1 - The Art, Act, and Science of Knowing
Why are Humans at the Top of the Food Chain?
Recent History of Knowing
Chapter 2 - A Transformed World
Analytics Makes Change Understandable
Finance Industry: One of the First to Be Transformed by Analytics
Healthcare Will Be Transformed by Analytics
Retail Is on the Cusp of Being Transformed
Grocery Stores Lead Transformation
Advertising Is Being Transformed
What We Must Know about Customers Is Transforming
Business Models Are Transforming
Logistics Has Been Transformed
The Military Has Been Transformed
Science Is Transforming
Agriculture and Food Are Being Transformed
Work Itself Is Transforming
Transportation Is Being Transformed
Politics Is Transforming
Technology Companies Slow to Embrace Analytics
The Media Industry Has Been Slow to Transform
Chapter 3 - The Analyst
What Do Analysts Do?
Understanding Customers
Evolution of Analytics
Analyst Brand
Education of Analysts
Career Path
Managing Analysts
Analytics Everywhere
Chapter 4 - Where Analysts Live in the Organization
The History that Got Us Where We Are Today
State of Organizational Structure Today
A “Hot Mess”
Thoughts on the New Normal
Power of Connected Analytics
Power of Conversation
The Path Ahead
But When You Get It Right . . .
A Different Model to Build On
Chapter 5 - Relationships Betwixt and Between
Why Are Relationships Important?
A More Nuanced Understanding of Relationships
Paying Attention to What People Pay Attention To
Managing Relationships Requires Understanding Info-War
Whom Do You Want to Have a Relationship With?
Relationships with Peers
Analysts Need to Get in the Game
Social Network Analysis: The Forgotten Piece of Business Analytics
Evolution of Relationships
Intense Focus on “Same Paging”
Social Capital Is Teamwork Scaled Up
Analysts are Weak at Relationship Math
Chapter 6 - The Evolution of the Technologies of Analytics
Technology Evolves
Where Business Analytics Sits Today
It is Time to Wake Up and Analyze the Data
What People Agree On
Chapter 7 - Value of Analytics
Business Analytics as Source of Strategic Understanding
Marketing to Customers
Customer Knowledge
Real-Time Product Pricing
Every Five Years: A “Do Over”
Applying Analytics to Innovation
Choice Architecture
Innovation Matters
Role of Business Analytics in Innovation
Innovation Requires Process
Chapter 8 - Where Is All This Going?
A New Kind of World
A New Kind of Decision Making
We Now Know Better
A New Kind of Skill Set
A New Customer Knowledge
Experiments, Mistakes and Questions
A New Kind of Leadership through Analytics
Analytical Infrastructure
The New Know

Additional Praise for The New Know
Thornton May has his finger on a key mantra of the future: Mastery of information will differentiate world class enterprises from those that will flounder in the uncharted waters ahead. And the future masters of this universe will be those that excel at analytics—knowing not just that things are related, but why, and how. Seems simple, doesn’t it? As anyone knows who has lived through the past year, it’s just not that simple.This book shows you why.
J. Pari Sabety
Director, Office of Budget and Management, State of Ohio
This book is nothing more than the usual from Thornton May. Brilliantly reasoned and insightful thinking, presented in a rational and eccentrically entertaining way as only Thornton can do. This book could not have come along at a better time. Analyze this!
—Jerry Gregoire
Former CIO of Pepsi Cola and Dell Computer Company,
Chairman, Redbird Flight Simulations, Inc.
As the tub was to Archimedes and the apple to Newton, so is Thornton May to the CIO community. Thornton’s unique style and insights create eureka moments on a regular basis. Known as much for his scathing wit and delivery as he is for his intellect, what truly sets Thornton apart is his intimate understanding of the CIO habitat and his ability to anticipate the future in a way few others can. In his new book Thornton takes a lifetime of observation and research and creates his magnum opus, a true Eureka for all of us.
—Tom Murphy
Senior Vice President and CIO, Amerisourceberge
It’s not what you know, nor who you know it’s HOW you know! The New Know unlocks the secrets of how to think for success in the Information Age 2.0 and best of all it’s vintage Thornton ... professionally provocative and endlessly entertaining.
—Maria A. Cirino
Entrepreneur and Venture Capitalist, Managing Director,
406 Ventures
Thornton May is one of those rare individuals whose line of sight always extends far beyond the horizon. In The New Know Thornton shares a thoughtful vision of the impact that unlocking the treasure trove of digital information being amassed will have on every aspect of our lives in the years to come. Fasten your seatbelt and discover how knowledge analytics is becoming the new alchemy transforming bits into insight.
—Dennis Devlin
CISO, Brandeis University
Everyone is on a transformational journey, rapid, glacial or somewhere between.This body of knowledge will help you speed your journey up tremendously. Whether your world is retail, services, hospitality, finance IT or …….. you will realize quickly upon picking up this work that you will gain significant advantage by reading and absorbing.
—Shannon Stowell
President, Adventure Travel Trade Association

Wiley & SAS Business Series
The Wiley & SAS Business Series presents books that help senior-level managers with their critical management decisions.
Titles in the Wiley and SAS Business Series include:
Business Intelligence Competency Centers: A Team Approach to Maximizing Competitive Advantage by Gloria J. Miller, Dagmar Brautigam, and Stefanie Gerlach
Case Studies in Performance Management: A Guide from the Experts by Tony C. Adkins
CIO Best Practices: Enabling Strategic Value with Information Technology by Joe Stenzel
Credit Risk Scorecards: Developing and Implementing Intelligent Credit Scoring by Naeem Siddiqi
Customer Data Integration: Reaching a Single Version of the Truth by Jill Dyche and Evan Levy
Enterprise Risk Management: A Methodology for Achieving Strategic Objectives by Gregory Monahan
Fair Lending Compliance: Intelligence and Implications for Credit Risk Management by Clark R. Abrahams and Mingyuan Zhang
Information Revolution: Using the Information Evolution Model to GrowYour Business by Jim Davis, Gloria J. Miller, and Allan Russell
Marketing Automation: Practical Steps to More Effective Direct Marketing by Jeff LeSueur
Performance Management: Finding the Missing Pieces (to Close the Intelligence Gap) by Gary Cokins
Performance Management: Integrating Strategy Execution, Methodologies, Risk, and Analytics by Gary Cokins
Credit Risk Assessment: The New Lending System for Borrowers, Lenders, and Investors by Clark Abrahams and Mingyuan Zhang
Business Intelligence Success Factors: Tools for Aligning Your Business in the Global Economy by Olivia Parr Rud
The Data Asset: How Smart Companies Govern Their Data for Business Success by Tony Fisher
For more information on any of the above titles, please visit www.wiley.com.


The Blending of Insight and Technology
By Jean E. Engle
Thornton, in this book, has provided one of the most comprehensive perspectives on a truly unsung hero, the analyst. These are the go-to guys who seamlessly cut through the organizational bureaucracy and technical mumbo jumbo to succinctly turn data into a point of view that makes sense for decision makers.
One of the aspects that I enjoyed most is this book is not all about information technology or all about people, but that natural blend of human intellect with enabling technologies to produce insight out of chaos. It is refreshing to see the value of human intelligence highlighted in this enlightening discussion; it is far too easy to rely on technology to provide the “right” answers.
The views expressed are not necessarily those of the National Aeronautics and Space Administration or the United States Government.
I have had the fortune of working in an environment where analytics is critical to the success of human exploration. Every day at the Johnson Space Center, thousands of men and women work in one of the most complex organizations in the world making the almost impossible look easy. These dedicated personnel spend years devoted to the discipline of human spaceflight—a complete system of systems that requires in-depth analysis expertise in the definition, design, development, verification and validation, systems integration, science integration, testing, astronaut training, and operations tied to a strong safety culture and effective risk management to ensure mission success at every turn.
Where are the analysts located within the space center? They can be found at every level of the organization—are depended on at every level of the organization. The tools and technologies have certainly evolved over the decades; however, each successive mission has been built on the lessons and expertise gained from those preceding: the knowledge and wisdom passed from generation to generation. As we celebrate the fortieth anniversary of man’s first steps on the moon, it is interesting to note that the computing capability in most handheld devices is far more powerful than what was used in the Mission Control Center during that era.
The future of analytics is one certainly open for further debate, and Thornton has provided an excellent framework to shape the discussion. However, one cannot beat Yogi Berra for stating the obvious: “The future ain’t what it used to be.”
Jean E. Engle is the chief knowledge office for the Johnson Space Center. In this capacity, Ms. Engle is responsible for the development of an integrated knowledge management plan across the entire center as well as collaborating with other NASA Centers and industry to identify and utilize best practices. At Johnson Space Center, Ms. Engle has also held the positions of chief information officer, where she was named to the senior executive service of the federal government, and director of information resources.
Ms. Engle earned a BA in Mathematics and an MBA from the University of Houston—Clear Lake. She has been the recipient of numerous awards including the astronaut’s personal award, the Silver Snoopy; two NASA Exceptional Service Medals; the JSC Certificate of Commendation; and the Goddard Space Flight Center Group Achievement Award.

Are We at the Advent of Something Big?
By John E. Chickering, CRM
Like it or not, the way we manage information today is not much different from how we, as a “superior” species, have done for millennia. Let’s start with a very early form of recorded information: prehistoric cave drawings. The information is there, it’s in an analog graphic format, and the rock on which it’s written has proven to be more stable than anything sold by any technology vendor ever. Depending on your religious views, you may even regard the manufacturer as still “in business.” Somebody decided to create this record, which means that we now have something more enduring than oral tradition.
So storing the data has been something we’ve been able to do for a long time. Where we’ve had trouble has been in access and meaning. The cave drawings required the user to be physically colocated (and with a light source) in order to support retrieval. That’s an access issue. Once you get there with your torch, you then must ponder the two-dimensional characters that appear to illustrate people, animals, and weapons. That’s when meaning comes in. Is the picture a menu? Is it perhaps a census record or an instruction manual? Or is it just art? Oh, and whose meaning is most important? Does it matter why the picture was created in the first place? Say it’s art. If we can look at it today and infer something about the society, the picture in and of itself conveys information the artist may never have meant to record in the first place.
Over the next several centuries, we can see a thread of information management evolution that really doesn’t go very far. We change technologies along the way, but information management doesn’t go all that far. We became pretty good at creating paper, which allows for transportability far beyond what granite could provide, meaning we could bring the medium bearing information to the beholder. That was a step forward, but it did come at a cost, because we became so efficient at paper that we created another problem: too much to search through when seeking to know. We became so efficient that our ability to discern the fleeting imperative of what needs to be known at any particular instant requires the same basic set of skills: finding and understanding in a world of incredible volume growth.
Computers have only begun to make their mark. During the early automation years, we started a decades-long movement from simply keeping the computer running to getting it to do something useful in much less space. Individual applications came along, forming the infamous islands of automation and spawning a natural desire to link them together. As networks evolved, the connectivity helped exchange data, but meaning was something altogether different. Simply linking the computers together has proven much easier than connecting the information.
At the same time, two other trends were unfolding:
1. Storage media were getting cheaper and faster.Viewing the art had gone from “you had to be there,” to “we’ll bring you a copy,” to a point where you could see it without anything being physically delivered. The “friction” of physicality that had both hampered and helped (think security issues here) us was melting away.
2. The ability to create a record was getting easier. What must have been a very arduous task in a cave was turned into mass production with the printing press and became so easy that we create huge volumes today without consciously trying to create anything at all. No wonder some industry experts believe that data storage doubles in fewer than three years.
Based on my own survey of senior information technology (IT) professionals, we’re pretty good at managing the data, but we have a way to go in managing the information. We know how to build networks, operate data centers, and set up multisite high-availability computing complexes resilient in the event of even significant disasters. We don’t always do so, typically when the economics don’t justify the investment, but we know how and the technology keeps getting cheaper. But the age-old twin challenges of finding and understanding remain today.
Once we make the technology cheap and reliable, the next frontier is for us to make it useful. Actually, we need to make it more useful for both ourselves and our customers than our competitors do. We make these investments so that we can deliver to our doorstep the information that will let us have the understanding we really need. We need understanding to take action and get rewards. And those rewards can range from finding a terrorist, to being first to market with the latest hot-selling electronic gadget, to prescribing the right medical treatment the first time consistently.
I believe that conquering the next frontier will require more than simple technology savvy. It will require something more like a cultural shift. While a manufacturer doesn’t have to know celestial navigation to ship parts and product overseas, understanding both the art and the science of supply chain logistics can create a competitive advantage. Knowledge as a competitive advantage comes when we have the ability to consistently work the interface between business (art) and technology (science) to know what we need to know, when we need to know it, where we didn’t know what we needed to know when we first started.To raise the bar even higher, the competitive factor of copycat flattery means we can’t expect our innovations to be unique competitive advantages for very long. We can’t expect a one-trick pony to command the stage for more than a few moments.
Recognize that once we started making marks on rocks or paper or optical disks, information made a great leap from a transient asset (e.g., someone’s memory) to a retained asset. Evolution in technology has helped with the ability to record and retrieve information, but the ability to apply understanding, to filter through the irrelevant to the critical nuggets of knowledge, has had trouble keeping pace. With volumes growing at rates unimaginable just a few years ago, we have data, data everywhere, but nary a drop to think. The data is all out there, stored several times over—but what does it all mean?
We’re at a point where we need to move beyond simply getting the computers to run. We need to get the appropriate meanings to be as usable and sharable as the bits and bytes that we can push across networks today. Economic Darwinism tells us that the best among us—those who find the innovative ways to exploit technology and do it consistently better than our competitors and to the delight of our customers—will survive. This is about moving beyond having the data to knowing what we need to know when we need to know it. This is way more than a new project. It’s a new business ecosystem to create a New Know.
John E. Chickering is a director at AIIM International (Association for Information and Image Management) and a vice president at a leading financial services provider. Mr. Chickering is an experienced consultant, software vendor, end user, and lecturer. He has also served as chief information officer at two human resource services companies and was a pioneer in the enterprise content management industry.
Mr. Chickering holds an MBA in operations research from the University of Maryland and a BS in marine engineering from the United States Merchant Marine Academy. He is the author of articles in the trade press and in peer-reviewed journals as well as an occasional speaker at both industry conferences and continuing education seminars hosted in academia.

Standing at a Hinge of History
Timelines and chronologies are a big part of knowing. We know from physicists that 13 billion years ago, our universe started expanding.We know from cosmologists that 4.6 billion years ago, our planet, the rotating rock we call home, was formed. We know from paleo-anthropologists that it was approximately 250,000 years ago that our hominid ancestors first developed the capability of language, which enabled the all-important and species-preserving trait: collective learning. I knew from friends, colleagues, and thought leaders1 that the time had come to write a book2 about a phenomenon and set of practices I came to call The New Know. The New Know encompasses and celebrates the tools, processes, people, and practices of business analytics (statistics, forecasting, operations research, and data mining).
Most readers are semi-aware (i.e., they do not know) that every day—every day—there is more to know, more ways to know, and heightened expectations on the parts of customers, citizens, investors, and regulators that you will do something efficacious with what you know. The intersection of more to know, more tools to help us know, and expectations that we do something with what we know provides the environmental backdrop for The New Know. The New Know is a book, a time period, and a societal reality.
As we move forward from this point in time, unaugmented human cognition (the old know) is not sufficient. In every vertical market and just about every field of human endeavor, the magical carbon-chemo-electro processes known as thinking and the heretofore unaudited mysterious arts of decision making have to be linked with the tools, processes, and practices of business analytics if we are to expeditiously understand and then efficaciously act on the physical and virtual worlds we simultaneously inhabit.The mantra of the New Know age is to expeditiously understand and then efficaciously act.
The days of showing up, of attendance-based compensation, are over. The backlash associated with the bonuses paid to financial service executives involved in destabilizing the global financial infrastructure have created an atmosphere of measurement vigilantism. The desire to know will define the next quarter century. In the not-so-distant future, guessing/making things up, not having the right data, or employing the wrong algorithms to data will come to be viewed as termination offenses and egregious social taboos.3 Your success—personal and professional—will increasingly come to depend on your ability to collect, organize, analyze, and act on the exponentially increasing mass of information that defines the first moments of the twenty-first century.
The global media, business school professors, and your children agree that most organizations are but passively engaged in managing the information swirling around and through them. A phrase increasingly heard in common parlance is “clueless.” Many organizations don’t even know what they know. Historically, plausible deniability (i.e., statements to the effect of “I didn’t know”) has been viewed as an acceptable excuse for poor performance. No more.
The big contemporary headline-grabbing news today is the after-the-fact shock and postmeltdown anguish regarding what senior executives did not know—about their employees, about their risk, about their cash flow, about their carbon footprint, and about their customers. The next big story, the headlines you and your team will be writing after reading The New Know, focuses on what can be known, what must be known, and most important what actions you will take because you know.This is the power of business analytics.
The new law of the new jungle stated simply is: Know and prosper; not know: Whither and die.You are standing at a hinge of history. The choice is yours.
Chapter 1 documents what can be known, what must be known, and how you come to know what (the various people, processes, and technologies whereby an enterprise migrates data to information, information to knowledge, knowledge to insight, and insight to action) has changed.
Chapter 2 documents how the rapid rise in data, the rapid expansion of tools, and maturation of information management processes are changing various vertical markets. I examine the advertising, agriculture, finance, government, grocery, healthcare, logistics, media, military, politics, retailing, science, technology, and transportation industries.
The New Know is changing how we do science, how we design and sell products, how we diagnose and medicate illnesses, how we run businesses, how we govern ourselves, how we fight, and how we relate to one another.What is “not knowable” and the permissible time frame for not knowing are undergoing a fundamental transformation.
Chapter 3 shines a much-needed spotlight on the professionals at the beating heart of business analytics. Analysts, you will soon discover, are very smart, interesting, and engaging people, doing very important things. These are people you should know and know about. I hope this book will serve as a wake-up call, similar to Rachel Carson’s Silent Spring or Upton Sinclair’s The Jungle.
The driving reason behind why I decided to write The New Know is because I really like analysts: the professionals who collect and organize the data; create the models and conduct the analysis that creates the insight that stimulates the action that leads to value. Over the course of the last year, I have immersed myself in the world of the analyst. This is a world little covered by the business or trade press and virtually unknown to many of the key decision makers at the top of the world’s organizations—despite the fact that “statistics” and “analysis” surround us constantly.
Americans today are accustomed to a seemingly endless stream of questions from survey researchers, political pollsters, marketers, and census takers. They are equally familiar with the battery of results flowing from scientific investigation, of knowing that the majority of the nation supports the death penalty or that half of all marriages end in divorce. Public life is awash in statistics documenting phenomena as diverse as consumer confidence and religious faith.4
What is a bit surprising is that the people who crunch the numbers, create the models, and do the analysis have never really turned a social science or analytical lens on themselves. Socrates spoke of the power of the “examined life.” The lives of analysts, for the most part, have never been systematically examined. The New Know is a humble attempt to at least partially fill that void. The New Know provides a map to the universe of analytics. No business book has really examined or celebrated these folks. This book will be the first vernacular ethnographic and anthropological study of the analytic community (patterned on anthropologist Ruth Benedict’s Chrysanthemum and the Sword).
Being studied, and being privy to the results, is increasingly an understood and unexceptional feature of modern life. This book analyzes the analyzers. The New Know hopes to remove ignorance of the substantive and courageous work analysts do to make our world a better place.When possible, I attempted to use the tools of new journalism to bring the realities of The New Know front of mind.5
Most of the very fine books written to date about analytics or business intelligence tend to focus on ideas, methodologies, or technologies. This is a book about the people who conduct the analysis that makes life in these complex times possible. Business analytics is a critically important discipline that is tragically underfunded in most enterprises today.
Chapter 4 examines the habitat of the analyst. My research probed where analysts have been, where they are now, and where they are going. How do analysts live their lives? How do their minds work? What are some broadly held misconceptions about this very important subset of the working population? The data on analyst ethnographics is based on fieldwork—semistructured and conversational interviewing, individual and group observation, and the collection of documentation.
Every age has its heroes. During the mid-Industrial Age, Harvard scholar Joseph Schumpeter favored the word “entrepreneur”—the person who is willing and able to convert a new idea or invention into a successful innovation. Peter Drucker celebrated “knowledge workers.” During the Internet boom, Bob Reich, Clinton’s first-term secretary of labor, thought “symbol manipulators” would come to dominate society. I have no doubt that the new hero of the next age will be the analyst.This forecast is subject to one major contingency. Will the analytic community be able to master relationship management? Chapter 5 examines the day-today work of the analyst in context of the portfolio of relationships they manage and/or are managed by.What is working and what is not working, relationship-wise is highlighted.
Technology choice is an important success driver. Business history is full of stories of organizations that, upon making the right technology choice, moved on to greatness. The commercial fossil record also showcases the skeletal remains of entities guilty of “technological oversight”—of strategic missteps resulting from failure to notice or act on a technological opportunity. I believe that not stepping up to the value opportunities inherent to business analytics is a error similar in severity to choosing not to use fire. Chapter 6 utilizes three important frameworks to assess where business analytics sits in its life cycle and in its ability to add value to your enterprise.
Decision making has changed. Information systems used to be designed to move information to the top of the house. Senior managers were the ones who made decisions. In that world, technology tools for assisting decision making were designed for senior executives and managers. This is no longer the case. If you are not a decision maker, you probably won’t have your job for long. The business analytics industry will continue to make tools for rocket scientist brainiacs. However, the market has expanded to include the rest of us mortals.
Chapter 7 examines how business analytics creates measurable value. I think evangelical enthusiasm is warranted in support of analytics. This surprisingly affordable, accessible, and powerful set of tools and practices is available to all and yet—inexplicably—many are still living in the analytical dark ages. Research surfaced that some organizations, although fully aware just how powerful analytics is and what a source of sustainable competitive advantage it can be, actively seek to keep it a secret. An executive at a prominent financial services firm in charge of customer analytics for one of the business groups told me:
What we did was so critical I was not allowed to go out. I was asked many times to go out and talk about how we built this customer information management platform that joined U.S. consumer behavior and [our] business marketplace behavior. They found that so valuable that I was not allowed to talk about it outside because they didn’t want competitors knowing what we had. It transformed [our] ability to look at share of wallet.
Organizations getting full value from their investments in business analytics ask themselves: Do we know stuff that nobody else knows? Do we see how the pieces fit together? Can we make connections nobody else can make?
This chapter also examines the facts of how innovation happens in complex organizations today. Most of the stories you read about innovation are false. James G. March, emeritus professor at Stanford University who is known for his groundbreaking research on real-world organizational decision making, explains:
Changes in technologies, practices, and products are characteristically described in terms of the triumph of the new over the old. Initially a new idea, institution or practice is introduced into a small part of the system. Ultimately it becomes pervasive. The stories use the power of retrospection to identify individual and organizational genius in this triumph of good over evil.6
We humans like stories which feature humans as heroes.
Innovation is a crap shoot. As such, it is subject to probabilities and chance. As such, innovation performance can be materially improved by the enlightened use of business analytics. Oil companies know this. Pharmaceutical companies know this. Consumer packaged goods behemoths know this. Globe-spanning quick-service restaurants know this. Shouldn’t you? To innovate without the assistance of business analytics is madness.
Chapter 8 puts a stake in the ground regarding what we will know and how we will come to know it in the future. The “future” is not out there waiting to be discovered. What China’s gross domestic product will be in five years has not already been determined.Whether the North American auto industry will continue to decline is not preordained. Right now a particular probability for each of a broad range of possible futures is calculable. The question becomes: Is what we can know (probabilistically) efficaciously actionable?
Setting up a process to examine the future rigorously and seriously is not easy.
The future is uncertain. Attempts to predict it have a checkered history—from declarations that humans would never fly, to the doom-and-gloom economic and environmental forecasts of the 1970s, to claims that the “New Economy” would do away with economic ups and downs. Not surprisingly, those who make decisions tend to stay focused on the next fiscal quarter, the next year, the next election. Feeling unsure of their compass, they hug the familiar shore.7
Peter Drucker, the patron saint of management, claimed that “forecasting is not a respectable human activity, and not worthwhile beyond the shortest periods.”8 Furthermore, in the spirit of full disclosure, it is worth noting that forecasters’ track records historically have been pretty poor. For example, in 1984, European journalists asked four finance ministers, four chairmen of multinational companies, four Oxford economics students, and four London garbage collectors to generate 10-year forecasts on a number of key economic variables.9 In 1994, they assessed the results and found that company chairmen had managed to tie with the garbage collectors, with the finance ministers finishing last.
You will have to read further for me to prove that business analytics has materially improved. One thing remains constant: We want to know the probabilities of what is likely to happen so that we can do something about it.
We know that people get to the future at different times (i.e., there are people who are ahead of the curve, in the middle of the curve, behind the curve, and those who don’t even know there is a curve).
We know that organizations and individuals who embrace business analytics will operate at a competitive advantage over those who do not. Bruce Sterling, the science fiction writer and social commentator, speaking at the 2006 PopTech Conference told a powerful story detailing the folly of trying to compete without the tools at hand:
An engineering professor at Harvey Mudd split his students into two groups. The John Henry group had to hit the library, the encyclopedias and were forbidden to use the net.The Baby Hueys were forbidden to access ink on paper, and had to use Wikipedia and “bizarre blogger blither.” He had to end the experiment because “the Baby Hueys were wiping the floor with the John Henrys.”
We know that business analytics can expand our capacity for foresight. What cannot be known with any confidence, however, is how people actually will use this technology. A reporter once asked Theodore Roosevelt if he knew what the American people thought—that is, what their immediate desires were. “I don’t know what the American people think,” Roosevelt groused. “I only know what they should think.” The New Know puts a stake in the ground and states unequivocally that executives should think much more analytically in the future.The true magic is to figure out how to make that happen.

Can Futurists Be Trusted?

I am a futurist. As a futurist, I am required to expose myself to a whole lot of thinking from a whole lot of people in a whole lot of places. For that reason, I am very well traveled. I am on the road about 250 days a year. I am a promiscuous networker. I go through about 1,000 business cards every five to six weeks. The purpose of all this movement is not to serve as a lab experiment for the commercial airline industry. The purpose is to place myself inside a flow of human experience with an eye toward understanding what people are thinking about, worrying about, and spending money and time working on.
Futurists love and live for hinges of history. An analysis of the past quarter millennium reveals that historical hinges have tended to occur once every 50 years or so in the early Industrial Age, once every 20 years in the later Industrial Age, and once every five years in the Meso-Information Age.10 We know that change happens and that it is accelerating.
We futurists are pretty good at the rough timing of change. Sadly, we are not terribly accurate about what shape the change will manifest. From the “Overture” of George Friedman’s very provocative The Next 100 Years: A Forecast for the 21st Century11 we learn just how wrong futurists have been:
If you were a European living in London in 1900 contemplating the future you would be convinced that war was impossible and European dominance of the globe assured. In the Spring of 1914, economically, politically, culturally, educationally, everything was getting better in every way in most places. And then World War I happened....
If you were living in London in 1920, there would be many questions on your mind but one thing was certain—the peace treaty that had been imposed on Germany guaranteed that it would not soon emerge. And then World War II happened....
In 1940, Germany had not only reemerged but conquered France and dominated Europe. From the point of view of most reasonable people, the war was over. Futurists might question the thousand-year Reich, but conventional wisdom thought Europe’s fate had been decided for a century. Germany would dominate Europe....
In 1960, the United States had the Soviet Union surrounded and, with an overwhelming arsenal of nuclear weapons, could annihilate it in hours. The United States had emerged as the global superpower. It dominated all the world’s oceans, and with its nuclear force could dictate terms to anyone in the world. Stalemate was the best the Soviets could hope for—unless the Soviets invaded Germany and conquered Europe. That was the war everyone was preparing for.
In 1980, the United States was seen, and saw itself, as being in retreat. Expelled from Vietnam, it was then expelled from Iran as well. To contain the Soviet Union, the United States had formed an alliance with Maoist China.
In 2000, the Soviet Union had completely collapsed. China was still communist in name but capitalist in practice. NATO had advanced into Eastern Europe and even into the former Soviet Union. The world was prosperous and peaceful. Everyone knew that geopolitical considerations had become secondary to economic considerations, and the only problems were regional ones in basket cases like Haiti or Kosovo. And then September 11th happened.
Every 20 years, a new bit of conventional wisdom. Every 20 years, a new surprise to render the conventional wisdom not only wrong but dysfunctional. This being the case, it is understandable that readers would look with justifiable skepticism at anyone claiming that a new age was upon us. Authors who pen business books with the adjective “new” in the title frequently overstate the novelty of the age, method, technology, and/or phenomenon they are examining. They mistake or misinterpret a contingent thing as a new eternal. Recognizing this danger on the front end, I undertook to make sure that The New Know was in fact something new, not just a fad or short-lived blip on the cognitive radar.
In addition to an extensive literature search, I conducted numerous in-the-field interviews with practitioners, C-suite executives, industry analysts, and journalists. I buttonholed a select group of people who, in my opinion, over the years had demonstrated a unique and uncanny ability to know when something big was going on. Most of my magi agreed wholeheartedly with my hinge of history hypothesis.12 All this nosing about and amateur ethnography indicates unambiguously that we stand at a hinge of history that requires commentary, sensemaking, and, most important, executive action.

What Exactly Is a Hinge of History?

A door hinge separates two distinct zones of physical space. For example, the hinge swings between outside and inside of a room and the hallway. In a similar fashion, a historical hinge is something that separates two distinct temporal spaces. Prior to 1492, the North American continent was viewed one way. After 1492 and the visit of Christopher Columbus, the New World was viewed quite differently.
Students of basic history, perhaps for mnemonic purposes, portray the historical landscape as featuring abrupt ruptures between one chunk of history, era, or epoch and the next. These historical Rubicons place significant importance and emphasis on hinge dates—the dividing line between one era and another. Such hinges of history are a natural part of how humans make sense of the world. “Chunking” information in this way is not a learned strategy but is instead a fundamental aspect of the human mind. Indeed, if you don’t think in hinges, you are a little unhinged.
The more literary among you will recall that Shakespeare had his tragic period (the years 1601 through 1608, when he wrote Macbeth, King Lear, Hamlet, and Othello). Art lovers among us know that Picasso had his “blue period” (1901 to 1904). Less sympathetic observers of the IT scene would have you believe that we are only now emerging from the “We suck less” era of IT leadership.13 Mark Twain, a close chronicler of how Americans thought, viewed the American Civil War as a hinge of history, observing that, in the South:
The war is what A.D. is elsewhere: they date from it. All day long you hear things “placed” as having happened since the waw; or du’in the waw; or befo’ the waw; or right aftah the waw; or ‘bout two yeahs or five yeahs or ten yeahs befo’ the waw or aftah the waw.14
I believe we are currently standing at a hinge of history. It may not be as significant as the hinge that swung between living in trees and walking erect, having fire and not having fire, and hunting/ gathering and agriculture, but I am convinced and the rest of the book will attempt to convince you that in these last years of the first decade of the twenty-first century, the changes we are living through are as momentous and life changing as the dropping of the atomic bomb over Hiroshima on August 6, 1945, and the fall of the Berlin Wall on November 9, 1989.
Hinges divide the world into “before” and “after.” Big hinges require major rethinking. Alan Webber, close personal friend, mentor, former editorial director at the Harvard Business Review, cofounder of Fast Company Magazine, author of Rules of Thumb: 52 Trends for Winning at Business Without Losing Yourself, and the craftsman of the magnificent afterword that concludes this humble tome, agrees with my hinge of history hypothesis, “The time has come to rethink, reimagine, and recalibrate what is possible, what is desirable, and what is sustainable,” he says.
Bernard Brodie, a brilliant new addition to the Yale University political science faculty, was an important player on the national intellectual scene. The Naval War College had assigned his book Guide to Naval Strategy as a standard text for officers. Brodie had established a solid reputation as one of the nation’s foremost naval strategists. On August 7, 1945, he went to buy the New York Times at a drugstore in the neighboring village of Woodbridge. The banner headline captured his attention: “First Atomic Bomb Dropped on Japan: Missile Is Equal to 20,000 Tons of TNT; Truman Warns Foe of a Rain of Ruin.” Brodie read just two paragraphs of the story that followed, looked up for a few seconds, turned to his wife and said, “Everything that I have written is obsolete.”15 The atomic bomb had changed the world.
The New Know has changed our world too. The combination of massive computing power, massive expansion in data management tools and practices, and exponentialized increases in customer expectations have created a world so complex and a customer so demanding that unaugmented human cognition—by this I mean making decisions without the assistance of a robust business analytic tool set—is no longer good enough. What you need to know, whom you need to know, how you come to know and the very abbreviated time window available for making efficacious use of knowledge are transforming.
Lawrence Summers, director of the White House’s National Economic Council and Charles W. Eliot Professor at Harvard University’s Kennedy School of Government, senses that we stand at a hinge of history. “I suspect that when the history is written 200 years from now, it will emerge that something very important happened in human thinking during the time when we were alive, and that is that we are becoming rational, analytical and data-driven in a far wider range of activity than we ever have been before.”16 Business, government, and society are moving from a faith-based mode of operation to a fact-based way of working. Is your organization ready for this fundamental transformation?
Hinges of history are important moments. They are filled with opportunity and risk. One of the biggest risks is not being aware that one is living in such a moment. Margaret Lavinia Anderson, who teaches “History 5: The Making of Modern Europe, 1453 to the Present, at Berkeley” likes to share with her students a cartoon from the New Yorker that portrays a town crier progressing through a European city center declaring “The Middle Ages are over, the Renaissance has begun. The Middle Ages are over, the Renaissance has begun.” The joke of course is that typically—at least historically—people in a given age or epoch are not savvy to the labels later generations will use to describe the world they live in. People in the Dark Ages probably didn’t know they were living in the Dark Ages.
The term “Renaissance” was first prominently used by the French historian Jules Michelet in 1858, and it was set in bronze two years later by Jacob Burckhardt when he published his great book, The Civilization of the Renaissance in Italy.17 The usage stuck because it turned out to be a convenient way of describing the period of transition between the medieval epoch, when Europe was “Christendom,” and the beginning of the modern age.18
One of the reasons I wrote this book is to make sure that readers know we are standing at a hinge of history, on the cusp of entering a new age—the age of the New Know, an age when just showing up is not enough.
An age is made more memorable and easy to grasp if it can be labeled by a word that epitomizes its spirit. I thought I would throw “New Know” into the ring to give people a cognitive buoy to float on until a better phrase comes along.
When we were astride another major change point—the fall of the Berlin Wall—the New York Times held a contest for readers to name the new age.
[T]the majority of respondents were male and extremely pessimistic. Popular labels included the “Age of Anxiety,” “Age of Uncertainty,” “Age of Fragmentation,” “Age of [Great and Failed] Expectations,” and “Age of Disillusion [and Dissolution].” One reader from NY suggested the “Cold War Lite Era” . . . From Greenwich High School in Connecticut came the “Era of the New Meanies.” And one wag from Santa Monica offered the “Age That Even Historians from Harvard Can’t Name.”19
The historian David Halberstam called the epoch “a time of trivial pursuits.”When it came to foreign policy, there was one phrase that the Clinton administration could not seem to escape: “post-cold war era.” The president hated the phrase. “What kind of message does that send? It defines where we are in terms of where we’ve been rather than where we’re going. I’m tomorrow’s boy, and I don’t like being seen as doing yesterday’s business.”20 During the Clinton administration, Strobe Talbott was ambassador at large and special advisor to the secretary of state on the New Independent States. When he visited Stanford University in September 1997, Condoleezza Rice, provost of the university, slipped in one dig: The namelessness of the 1990s, she said, spoke volumes about the aimlessness of the administration. Talbott reported back to Clinton, “She’s wrong about us not having a sense of direction, but she’s right about our doing a lousy job of explaining it.”21 The New Know is my attempt to make sure that very important patch of time we are currently traversing does not go nameless.
Foreign policy mavens will recall that the age prior to the fall of the Berlin Wall was called the cold war. The person responsible for that name and the strategy for it was George Kennan. Kennan continually counseled national leaders that they should stop trying to come up with a clever name for the era and concentrate rather on what should be done about the emerging challenges of the age. He advised members of President Clinton’s foreign policy team to embrace the world’s complexities and compose “a thoughtful paragraph or more, rather than trying to come up with a bumper sticker.”22
Executives need to be very mindful of what is happening in the world—the whole world. Leaders need to recognize that we are living in a moment when decisions really matter; a time when the buttons they push, the systems they deploy, the employees they hire, and the relationships they create will materially impact their personal and institutional success trajectories. One of the most important things leaders can do is understand and master the realities of the New Know.
The primary purpose of The New Know is not just to make you aware of the information explosion that exceeds the human brain’s capacity to process and the litigational implications of ignorance in the world just around the corner.This is an awareness and action book.This book celebrates human agency—the capacity of humans to make and impose choices; the ability of an actor to organize future situations and distribute resources.