If a philosopher were writing this book, we would begin by defining the terms discovery and invention (or perhaps dismissing them as useless concepts because they are so hard to define). Instead of beginning with definitions, I will offer examples illustrating discovery and invention. I will use the same examples to introduce most of the other concepts we will use throughout the book. Then in Chapter 2 we will take a more rigorous look at these terms, using the cognitive science literature. But the primary goal of this chapter is to give us a sense of the diverse range of activities to which labels like discovery and invention are applied.
The roads by which men arrive at their insights into celestial matters seem to me almost as worthy of wonder as the matters themselves (Kepler, quoted in Gentner et al., 1997, p. 403)
Kepler propped himself against the wall and watched the goatish dancers circling in a puddle of light from the tavern window, and all at once out of nowhere, out of everywhere, out of the fiddle music and the flickering light and the pounding of heels, the circling light and the Italian's drunken eye, there came to him the ragged fragment of a thought. False. What false? That principle. One of the whores was pawing him. Yes, he had it. The principle of uniform velocity is false. He found it very funny, and smiling turned aside and vomited absent-mindedly into a drain. (Banville, 1981, p. 72)
The above quotation comes from the novel Kepler by John Banville. It is a classic romantic account of discovery--the flash of insight that leads to Kepler's Second Law comes in the midst of a drunken revel. The false principle of uniform velocity refers to the widely-held assumptions that each planet orbited the Earth, or the Sun in the Copernican system, at a constant speed in a circular orbit. Kepler's Second Law states that if you draw an imaginary line between a planet and the sun, then look at the area swept out by that line at two equal intervals anywhere on the planet's orbit, the areas will be equal.
Banville's novel is based on Arthur Koestler's The Sleepwalkers (Koestler, 1963), in itself an inspiring, romantic account of Kepler's tortured path to his discoveries. Kepler developed an early model of the solar system in which the five Pythagorean perfect solids could be put in the five intervals between planetary orbits: in between the spheres of Saturn and Jupiter he placed a cube, between Jupiter and Mars a dodecahedron, and so on. This model gives us an insight into the view of the solar system Kepler had to transcend. We think of planets as having orbits through space, but before Kepler's time, they were thought to ride on giant spheres; therefore, the orbits had to be perfectly circular. Kepler's perfect solids were therefore inserted between these spheres.
This geometric relationship served Kepler as a mental model of the solar system. In the next chapter, I will discuss the psychological literature on mental models at length. For now, it will suffice to say that a mental model is a three-dimensional representation of a system that a scientist or inventor can manipulate in his or her imagination. Frequently, this sort of model is also sketched or prototyped. Kepler convinced Frederick, Duke of Wittemberg, to have his silversmiths create a drinking cup based on Kepler's nested spheres and geometric shapes; when the Duke agreed, Kepler created a paper model. Plans for the drinking cup expanded into a clockwork planetarium. In the end, it was never built (Koestler, 1963). But this hands-on experience is an important aspect of mental models, which are tactual. One of the themes of this book will be the way in which the hands and devices built with them serve as extensions of the mind.
Mental models differ from formal theoretical models in their incompleteness, their fuzziness; they represent the kind of system a discoverer hopes to find, or an inventor hopes to create. Even though the geometric model failed to fit the data and Kepler abandoned its literal form, it still survived as a mental model of what he was trying to achieve. Kepler later added the possibility of a correspondence between musical harmonies and planetary orbits; he explored quite a number of relationships and thought to the end that this approach held promise (Stephenson, 1994). Kepler was convinced there was an underlying harmony that governed the planetary orbits, and that it depended on the sun, which was the center of the solar system in more ways than one: it served "as the mathematical center in the description of celestial motions; as the central physical agency for assuring continued motion; and above all as the metaphysical center, the temple of the Deity" (Holton, 1973, p. 81).
Beginning in 1600, Kepler worked at the observatory of Tycho Brahe, first as Tycho's collaborator and then, when Tycho died after eighteen months, as Imperial Mathematicus. Tycho had the best observational data on the positions of the planets, data Kepler had to cajole out of him in little pieces until Brahe died. The most perverse of the planets was Mars. Kepler struggled for five years to fit four of Tycho's observations of Mars into a circular orbit. Then, when he finally succeeded, he managed to wrestle two more observations from Tycho, and these did not agree. The amount of error was only eight minutes of arc, but for Kepler it was enough to disconfirm his current orbital model: "he refused to accept an error of 8 minutes of arc because he believed Tycho's observations were a gift from God and hence deserved to be given utmost importance" (Job Kozhamthadam, 1994, p. 88).
Here again the Kepler case allows us to preview an issue that we will take up at greater length later. The philosopher Karl Popper argued that science advances not by embracing new truths, but by a ruthless willingness to discard old ideas which no longer fit new data (Popper, 1959). Much of the work of discarding is done as a group process: one scientist or group of scientists adheres strongly to an idea, another seeks to disprove it, and so the idea either holds up or is discarded. But successful discoverers and inventors have to do a certain amount of their own testing--have to be willing to abandon cherished hypotheses. In hindsight, this appears easy--why wouldn't Kepler simply throw out circular orbits. One has to realize that circularity was not just a hyopothesis in Kepler's time. Planetary orbits were governed a complex series of interconnected spheres (Margolis, 1993). Kepler's observations literally threatened to shatter the spheres.
Kepler had already thrown out uniform motion. Circular motion followed, but not before he arrived at his Second Law:
Since I was aware that there exists an infinite number of points on the orbit and accordingly an infinite number of distances [from the sun] the idea occurred to me that the sum of these distances is contained in the area of the orbit. For I remembered that in the same manner Archimedes too divided the area of a circle into an infinite number of triangles (Kepler quoted in Koestler, 1963, p. 329).
According to Koestler, Kepler knew that his assumptions about area and circular orbits were not quite right, but he claimed they canceled each other out, allowing him to propose the Second Law.
Now he returned to Mars' orbit; if it were circular, three observations should suffice to determine its path. But the path suggested by one set of three observations did not agree with the path suggested by another, indicating that the orbit was not a circle. He rejected the possibility that the observations were in error. An oval fit the orbit better, though even that was not perfect (Job Kozhamthadam, 1994).
For Kepler, observational reasons were never enough--there had to be an underlying model that explained the planetary orbits. Here he came up with another mental model based on an analogy with a ferry (Gentner, et al., 1997). The current supplies the main force, in a way analogous to the sun, but there were forces originating from the boat itself--the ferryman's rudder, the cord and pulley connection to the shore. Just as the clever ferryman could guide his in a circular path by using the rudder to take advantage of the currents, so the planets could move in circles while being swept by the power emanating from the sun.
But planets have no ferryman and no rudder. So Kepler evolved yet another analogy. He imagined the planetary orbit as a kind of 'magnetic river', with the poles of the planet alternately attracted to and repelled from the sun (Gentner et al., 1997; Job Kozhamthadam, 1994).
The point is, Kepler's evolving mental model of planetary motion had come to preclude the possibility of a circular orbit. If a planet is alternately attracted and repelled by the sun, then its orbit cannot be a circle. He settled on an oval, which did not perfectly fit the data, but gave results superior to a circle. He knew this was not entirely satisfactory, however, and spent another eighteen months wrestling before he realized that the orbit could be described by the equations governing an ellipse. The result was his First Law, discovered after his second.
Kepler published his two laws in 1609 in a work boldly entitled "A New Astronomy Based on Causation or a Physics of the Sky." The title reveals Kepler's other heresy. The laws or principles governing what we would now call the physics of the heavens were supposed to be totally different from those that operated on Earth. Kepler thought differently; as we have seen, his evolving mental model presumed not only a geometric regularity to the planetary orbits, but a physical connection between the planets and the sun: "so intense was Kepler's vision that the abstract and concrete merged." Here we find the key to the enigma of Kepler, the explanation for the apparent complexity and disorder in his writing and commitments. In one brilliant image, Kepler saw the three basic themes or cosmological models superimposed: "the universe as physical machine, the universe as mathematical harmony, and the universe as central theological order." (Holton, 1973, p. 86). Of particular note here is the combination of the abstract and the concrete into a powerful mental model that guided his effort.
The Third Law emerged from this mental model. "He had been searching for this Third Law, that is to say, for a correlation between a planet's period and its distance, since his youth. Without such a correlation, the universe would make no sense to him; it would be an arbitrary structure. If the sun had the power to govern the planet's motions, then that motion must somehow depend on their distance from the sun; but how? Kepler was the first who saw the problem--quite apart from the fact that he found the answer to it, after twenty-two years of labor. The reason why nobody before him had asked the question is that nobody had thought of cosmological problems in terms of actual physical forces." (Koestler, 1963, p. 395) Briefly put, the resultant law states that the cube of a planet's distance from the sun will be proportional to the square of its orbital period around the sun.
According to Job Kozhamthadam (Job Kozhamthadam, 1994, p. 8), Kepler "announced this law without providing any clear clue as to how he arrived at it." According to Koestler, Kepler discovered it by 'patient slogging'.
Recently, a group of researchers in Artificial Intelligence have created a computer program called BACON that, among other things, discovered Kepler's Third Law via the same sort of patient slogging--or what the program's authors call 'data-driven discovery' (Langley, 1987). Essentially, the program takes two columns of data, one containing the distance of a planet from the sun (D) and another its period of revolution (P), and applies a set of heuristics to this data.
A heuristic is a rule of thumb, a shortcut that one can use to reduce the size of the problem space when seeking a solution. Obviously, there are hundreds of possible relationships that could be used to link two columns of numbers; one needs either a mental model, or a set of heuristics, or both to reduce the possibilities to a limited solution space. In the next chapter, we will say more about the cognitive literature on heuristics.
The earliest version of BACON used three heuristics, which are shown in Figure 1.
Figure 1: BACON.1's Heuristics. Adapted from Michael E. Gorman, Simulating Science (1992) with the permission of the Indiana University Press.
Each of these heuristics was applied whenever its conditions for execution were met. As indicated in Figure 1, the second heuristic was applied first, resulting in the ratio D/P, then the third heuristic was applied twice to produce D2/P and D3/P. This last ratio is a constant. Voila! BACON had discovered Kepler's Third Law in a matter of seconds, leading Herbert Simon, one of the architects of BACON, to wonder why it took Kepler so long. Furthermore, four out of a sample of fourteen students, given the same two columns of numbers but no information about heuristics, made the same discovery (Qin, 1990). The solvers included a graduate student and an undergraduate in physics, and a graduate student and an undergraduate in chemical engineering. All the students were allowed to use calculators, and their processes were described at length. None of them knew the data had anything to do with Kepler's laws. The authors concluded that data driven discovery "was found to proceed in the same manner as many other problem-solving processes that have been studied and described. We believe that this result can be generalized to cover most, perhaps all, of the processes of scientific discovery" (Qin, 1990, p. 308) .
Here we have the first hint of how students can be turned into discoverers like Kepler--make them into good problem-solvers. Of course, making students into good problem-solvers is not trivial; we will talk more about this issue later in the book. But no special genius or magic is required for discovery.
The portrait of Kepler that emerges from historians and novelists resembles one of the mythical heroes described so eloquently by Joseph Campbell , involving several stages: a call to adventure, that takes the hero (or heroine) away from the ordinary world; a journey into the unknown, in which the hero is required to perform superhuman feats; and a return bearing a boon that benefits her tribe or, in some cases, the entire planet.
Like Arthur and his knights, Kepler begins with a vision of a Holy Grail, a model of the solar system that could be nested in a silver cup. But this is a fleeting vision; like the knight-errant, he has go on a long journey, with Brahe as both helper and tormentor and Mars as a kind of monster he has to battle. After twenty-two years of struggle, he returns with his three laws, which Newton uses to create a new model of the universe.
If we adopt this perspective, students will need to become more than good problem-solvers to become discoverers; they will have to go on the kind of hero (or heroine's) journey discussed by Campbell.
The portrait that emerges from machine discovery is different. If a computer program can discover, then it diminishes the heroic, mythical stature of the human discoverer. It took poor old Kepler years to do what a machine does in seconds, and a science student with a calculator does in an hour or so--the data was right there, in a form anyone could recognize.
But what about finding the problem? Can we not give Kepler credit for realizing that there had to be a regular pattern of the form suggested by his third law? After all, he was not given two columns of numbers and asked to find the relationship. Qin & Simon counter that,
It is sometimes argued that the real problem of scientific discovery is not to find laws in the data, but to define the problem and to discover the relevant data. But it has just been seen that defining the problem and discovering the data were not Kepler's primary contribution. He inherited the problem of describing the heavens parsimoniously from a long line of predecessors, and the data, as explained above, were mainly inherited from Brahe and Copernicus. His merit was that he converted the data into a form that revealed the geometry of the heavens and laid the foundation for Newton's inertial and gravitational explanation. From a scientific standpoint, his attempts to provide "physical" explanations for his empirically derived laws are now only historical curiosities (Qin, 1990, p. 306).
One of the questions raised by this book will be whether beliefs like Kepler's faith in a geometrical relationship between the sun and the planetary orbits are central to discovery, or whether they are epiphenomenal. To put it in other terms, was Kepler's mental model simply irrelevant, perhaps even a distraction? Was his discovery really just about number-crunching and curve-fitting?
Qin & Simon pose Kepler's problem as 'describing the heavens parsimoniously'. That is, of course, too broad a statement to narrow the problem space significantly. Instead, Kepler created a new problem: how to discover a relationship based on geometric and/or musical harmonies between the sun, planetary distances, and periods of revolution.
No one else had put the problem in this form. Copernicus did locate the sun more centrally than the Earth but the actual center of the solar system was a point near the sun. Copernicus also believed in perfectly circular orbits with planets moving at uniform speeds. Brahe still believed in an Earth-centered solar system, with several planets circling the sun, which in turn circled the Earth. Kepler, in contrast, put the sun at the actual center of the solar system. This made his problem of 'describing the heavens parsimoniously' different from Copernicus', or Brahe's, or anyone else's at his time. As Simon himself said, "solving a problem simply means representing it so as to make the solution transparent" (Simon, 1981, p. 153).
BACON represents the problem as finding a relationship between two columns of data, using heuristics that dictate the relationship will resemble Kepler's third law. It could be programmed to search for a different kind of relationship. Collins (Collins, 1990) argued that discoveries like BACON's only work if one narrows the choices to Third Law or no law and if one has perfectly accurate data. Kepler's original data regarding Mercury did not fit his Third Law with complete precision (Stephenson, 1994) but he was convinced of the relationship on theoretical as well as empirical grounds and therefore did not abandon his new hypothesis. If one introduces the possibility of small errors into BACON's data, then it increases the probability that a program armed with more heuristics could discover other numerical relationships.
In Kepler's day, there was serious argument about whether scientific theories were primarily heuristic devices for describing the results of calculations or whether they corresponded to underlying realities. Cardinal Bellarmine, who eventually condemned Galileo, put it this way:
To say that the supposition that the earth moves and the sun stands still all the appearances are saved better than on the assumption of eccentrics and epicycles, is to say very well--there is no danger in that, and it is sufficient for the mathematician: but to wish to affirm that in reality the sun stands still in the center of the world, and that the earth is located in the third heaven and revolves with great velocity about the sun, is a thing in which there is much danger...(Job Kozhamthadam, 1994, p. 114)
This argument presaged the later debate between Max Planck and Ernst Mach, the former holding that theories did correspond to an underlying reality and the latter that they were merely useful human constructs, valuable for mathematical calculations and as a way of summarizing results (Matthews, 1994).
To put this issue in simpler terms, BACON does not know what it has discovered. It is BACON's creators who comprehend the significance of the discovery. So who is the real discoverer, human or machine? The answer is both are part of a system, or a network, to use the term preferred by sociologists (Law, 1987). Indeed, more sophisticated versions of their BACON system could serve as expert assistants to scientists searching for relationships among data. This is exactly the role played by sophisticated statistical packages like SPSS and SAS: they allow social scientists to explore data. But to make certain that relationships discovered are not chance alignments of numbers, some theory or model is necessary--even an imperfect one.
1. Discovery depends on finding a problem significant enough to be labeled an important achievement.
According to Keith Noll, a planetary scientist at the Space Teles cope Science Institute, "One of the hardest things about being a scientist is selecting a problem that's small enough for you to actually attack and pursue as a project yet big enough to add something significant to what's already known" (Sobel, 1996, p. 87). In Noll's case, he settled on the study of the four moons of Jupiter that were discovered in 1610 by Galileo. Since then, spacecraft and the Hubble have found many more moons. Noll brought a new instrument to their study--the Faint Object Spectrograph on the Hubble telescope; he among other things, he has found evidence of an oxygen atmosphere around Ganymede, one of the four.
Similarly, the astronomer Margarte Geller spent a year at Cambridge thinking about the problem of the universe's structure. Galaxies were thought to be distributed at random. The first survey, encompassing only a few hundred galaxies, had found a great empty area in the constellation Bootes, but even Geller thought that was probably an error. Geller decided there needed to be a survey that would reach deeper into the universe, searching for large patterns like the apparent void in Bootes. It was she who discovered the structure often referred to as 'the stick man': "The distribution of galaxies looked like a child's drawing of a somewhat bowlegged person. It's a whimsical name for a grand figure: the stickman extended 500 million light years across the universe. Its torso was composed of hundreds of galaxies, a massive congregation known to astronomers as the Como cluster. Its arms were two more sheets of galaxies streaming across the night sky" (Taubes, 1997, p. 54).
Like Kepler, Geller now had data that suggested one of the fundamental assumptions or dogmas about the structure of the universe was wrong. Unlike Kepler, she had generated the data herself. The solution to her riddle is still a work in progress; she is one of the astronomers leading projects that will map even more of the universe, in an effort to find a Keplerian pattern in the structure of galaxies.
In all three cases, Kepler's, Geller's and Noll's, improved instrumentation produced new data that led to discoveries. All three found a problem they could attack, given their background and equipment, and that would also make a real contribution to science. In Kepler's case the contribution was revolutionary. Thomas Kuhn has proposed that science evolves by going through periods of revolution, or crisis, in which the reigning paradigm or world-view goes through a dramatic shift (For more on Kuhn's view, see 2.1 below and Kuhn, 1962). Consider Kepler's discoveries. The reigning paradigm, or world view, was that planetary orbits were circular--even the Copernican solar system included them. Tycho's data constituted what Kuhn has called an anomaly--a result that does not coincide with the current paradigm. According to Kuhn, anomalies cause the period of crisis that leads to a revolution. Kepler was the only one who saw this anomaly; it certainly precipitated a crisis in his thinking! His three laws became part of a new paradigm; Newton used Kepler's laws in the creation of his theory of gravity (Kuhn, 1957).
2. Discovery depends on transforming that problem into a form that suggests a promising path to solution.
A key aspect of this is formulating or having a powerful mental model, one that sets up a creative tension between the ideal--what the discoverer expects or hopes will happen--and the real. Kepler began with his geometric solids; the failure of that model created a tension, a need to replace it with some other geometric order.
It would be premature to argue at this point that mental modeling is an essential part of this process of transformation. In physics, for example, some scientists seem to rely on it , while others have almost a horror of visualization, preferring purely mathematical transformations (Miller, 1989).
3. Discovery depends on finding good data.
Again, a powerful mental model can play an important role here, suggesting what data is relevant. But one also needs resources and connections to get at the data. Kepler had to maneuver to get his appointment to Brahe's observatory. Modern astronomers still have to compete for access to observatories, though the internet is making access to good astronomical data much less problematic.
4. Discovery depends on a combination of flexibility and stubbornness
In other words, a willingness to discard a hypothesis on the basis of negative evidence, while keeping what the philosopher Imre Lakatos (Lakatos, 1978) called the 'hard core' of a research program. No amount of negative evidence would have persuaded Kepler to shift the sun from the center of the solar system.
These generalizations, vague as they are, do suggest that discovery is not totally mysterious. Nor is it easily reduced to a set of algorithms, BACON notwithstanding. Unfortunately, at this point, we can offer only the most general advice on how to discover. Furthermore, our advice would be too dependent on a single scientist working on a particular kind of problem. To improve these generalizations and broaden the sample on which they are based, we will apply them to:
(1) Further examples of discovery: The rest of this chapter will include two cases of discovery that differ from Kepler's in important respects. The great Devonian controversy will provide us with an example of a discovery that emerged out of the interactions among competing research teams of geologists. Michael Faraday had to build the apparatus that generated his data; he could not collect his numbers from another source. In order to discover, Faraday had to invent.
(2) Invention: We will also explore whether generalizations about discovery can be applied to invention. Once again, we will begin with a case--the invention of the telephone. Alexander Graham Bell had to discover in order to invent. Since I have spent much of the last five years studying Bell and his competitors, this section will set up one of the cases we will come back to again and again.
(3) Recent research on the cognitive psychology of science and technology: Even the three cases cited above are an inadequate base upon which to make generalizations, particularly as all three are historical. Fortunately, the literature on cognitive psychology of science includes modern practitioners and students. It will also allow us to refine our tentative generalizations in the light of the latest research. Chapter 2 will focus on psychology of science, and Chapter 3 on invention .
All this talk about how discovery works does not, of course, answer the question of why. This book will only touch on what motivates discoverers and inventors. Certainly, fame and fortune are an important part of the puzzle. But Kepler gives us a hint of another motive:
Sagan in Silesia, in my own printing press, November 6, 1629: When the storm rages, and the state is threatened by shipwreck, we can do nothing more noble than to sink the anchor of our peaceful studies into the ground of eternity" (Kepler quoted in Koestler, 1963, p. 422)
Kepler lived during the time of the thirty years war. In 1611, his imperial patron Rudolph had to abdicate the throne of Bohemia; his favorite son died, followed shortly by his wife:
Numbed by the horrors committed by the soldiers, and the bloody fighting in the town; consumed by despair of the future and by an unquenchable longing for her lost darling...in melancholy despondency, the saddest of all states of mind, she gave up the ghost (Kepler, 1963, p. 381)
Kepler had just finished his Dioptrice, his major work on optics, so at this point he was experiencing the contrast between the eternal order he was discovering and the chaos of human affairs. Kepler was not a man who withdrew from the world; he did try to reconcile Calvinists with Catholics, but usually ended-up offending both.
So, one of the motives behind Kepler's discoveries was a desire to glimpse the eternal. Like the Arthurian knights, Kepler was on a quest for a Grail. Glimpsing the Grail did not guarantee temporal success--it was a spiritual goal, to be achieved by one who had risen above all ordinary desires. Einstein asked why some have chosen to enter the Temple of Science. The answer is not easy to give, and can certainly not apply uniformly. To begin with, I believe with Schopenhauer that one of the strangest motives that lead persons to art and science is flight from the everyday life, with its painful harshness and wretched dreariness, and from the fetters of one's own shifting desires. One who is more finely tempered is driven to escape from personal existence and to the world of objective observing and understanding. This motive can be compared with the longing that irresistibly pulls the town dweller away from his noisy, cramped quarters and toward the silent high mountains, where the eye ranges freely through the still, pure air and traces the calm contours that seem to be made for eternity.
With this negative image, there goes a positive one. Man seeks to form for himself, in whatever manner is suitable for him, a simplified and lucid image of the world, and so to overcome the world of experience by striving to replace it to some extent by this image. This is what the painter does, and the poet, the speculative philosopher, the natural scientist, each in his own way. Into this image and its formation, he places the center of gravity of his emotional life, in order to attain the peace and serenity that he cannot find within the narrow confines of swirling, personal experience (Holton, 1978, pp. 231-2).
For Einstein , the Grail is 'a simplified and lucid image of the world'. Scientists often portray themselves as disinterested pursuers of truth for its own sake. Kepler was drawn to Einstein's 'silent mountains'--but he was also motivated by a desire to surpass Brahe and Copernicus. Recognition is the coin of science (Barnes, 1985) and part of this glory is attained by passing one's peers. Scientists know discovery is the road to recognition, and inventors hope their new technologies will lead to fortune--witness the bitter battles over priority and patent rights.