Communications of the ACM
When Predicting, Start With Humility
Credit: iStockPhoto.com
In his viewpoint "Don't Feel Bad If You Can't Predict the Future," Peter J. Denning (Sept. 2012) wrote: "Make sure your models are validated and that their assumed recurrences fit the world you are forecasting. Ground your speculations in observable data..." Hmm... Who validates the models? Physics-based models can be validated, in light of, say, their ability to predict/replicate the results of observable phenomena (such as gravity and inertia); experts in the discipline agree that the assumptions, calculations and/or algorithms, and predicted results match what is seen in the real world. On the other hand, social models rely on assumptions about human behavior, both individual and en masse, that cannot be measured or demonstrated and on predictions that can never be more than "face-validated." That is, "I can't tell you we got the right answer for the right reason; the best I can say is the predicted behavior corresponds to what is observed in real life x% of the time."
This inability to validate the quantification of variables is seen in efforts to model military interactions, as well as social, economic, and political phenomena; for example, no version of either the Lanchester model reflecting the relative strengths of a predator/prey pair or of the many "expanded" Lanchester variants is capable of predicting the outcome of the Battle of Rorke's Drift depicted in the 1964 movie Zulu between British troops and Zulu warriors in South Africa in 1879. Tank on tank, we can predict the odds; add human crews, and things get dicey; witness the dramatically uneven results of combat in Operation Desert Storm when a U.S.-led coalition reversed Iraq's 1991 invasion and nominal annexation of Kuwait. Similarly, one has only to open the newspaper to understand the degree to which we have so far failed to model the American economy sufficiently to suggest effective measures to relieve the ongoing recession. As Denning pointed out, predicting the future is difficult and fraught with danger. Be humble...
Joseph M. Saur, Atlanta, GA
I very much agree with Peter J. Denning (Sept. 2012) that one should be humble when predicting anything, especially if the prediction depends on some future human action or decision. Unlike atoms and molecules, humans have free will. More than 60 years ago, the economist and philosopher Ludwig von Mises explored this idea in his monumental book Human Action. More recently, Walter Isaacson's biography of Steve Jobs and Malcolm Gladwell's book Outliers: The Story of Success only reinforced the impossibility of predicting human behavior. Historian J. Rufus Fears wrote: "Nations and empires rise and fall not because of anonymous social and economic forces but because of decisions made by individuals" in the description of his course Wisdom of History. As for Jobs, predicting even the next five minutes would have been futile. Any given human action or even random event might have yielded a totally different technological (or economic or political) world from the one we have today.
Per Kjeldaas, Monroe, LA
"I can't tell you we got the right answer for the right reason; the best I can say is the predicted behavior corresponds to what is observed in real life x% of the time."
Depends Who Pays
Moshe Y. Vardi's Editor's Letter "Why ACM?" (Sept. 2012) explored the publishing dichotomy of "reader pays" vs. "writer pays." When the author of an article pays for publication, the reader gets thinly veiled advertising. Consider all the free publications we get (as "qualified professionals") but never read because we realize the content is really nothing more than marketing.
Robert Wilkens, Levittown, NY
Pearl's Grace and Science
For those unaware of the life, interests, and tragic death of the brilliant Wall Street Journal reporter Daniel Pearl, a suitable place to begin would be his Wikipedia page (http://en.wikipedia.org/wiki/Daniel_Pearl). Most readers of Communications know of his murder in Pakistan in 2002 by anti-U.S. militants, so read especially the "Aftermath" and "Legacy" sections, marveling how the empathetic world reacted, with the most creditable the formation of the Daniel Pearl Foundation (http://www.danielpearl.org/), seeking to heal the bitterness and distrust that has led to so much violence since 9/11.
Among those helped by the Foundation are a number of talented young Pakistani journalists. Another aspect of the Foundation's work from which Pakistan can continue to benefit are the transcripts and videos of the Annual Daniel Pearl Lecture Series at the University of California, Los Angeles and at Stanford University, with links through the Foundation's ++Web site; subjects covered are open-ended and reflect Pearl's diverse personal interests.
Almost certainly unknown to Pakistanis and others outside the computer science community are the outstanding scientific and scholarly achievements of Judea Pearl, Daniel's father and the Foundation's prime mover, whose smiling face looks out from the cover of Communications (June 2012), which included an interview "A Sure Thing" (http://www.tinyurl.com/94qfqps) covering his work in artificial intelligence and his winning the ACM A.M. Turing Award, the equivalent of a Nobel prize in computer science.
The interviewer's final question and Pearl's response should be of interest to all: "Does your research inform your work at the Daniel Pearl Foundation, especially in conducting interfaith dialogues?" to which he replied, "I have an advantage over my dialogue partners in that I'm an atheist, and I understand religious myths are just metaphors, or poetry, for genuine ideas we find difficult to express otherwise. So, yes, you could say I use computer science in my religious dialogues, because I view religion as a communication language. True, it seems futile for people to argue if a person goes to heaven from the East Gate or the West Gate. But, as a computer scientist, you forgive the futility of such debates, because you appreciate the computational role of the gate metaphor."
It is indeed gratifying that the venerated Pakistani public figure Abdul Sattar Edhi is on the Foundation's board. It is through him I would urge the government of Pakistan to invite Pearl as a state guest to speak to our media and on our campuses.
Q. Isa Daudpota, Islamabad, Pakistan
On Proving Continuity of Programs
Swarat Chaudhuri et al.'s article "Continuity and Robustness of Programs" (Aug. 2012) said: "The most basic reason why software systems can violate continuity is conditional branching" but ignored a more fundamental cause, namely that program variables have a limited number of states. Computer representation of real numbers is inexact, and only a finite subset of the integers can be represented exactly. Consequently, in computer arithmetic, equations (such as (x+y) + (z-y) = x+z) need not hold and can introduce discontinuity. The article ignored the problem by both declaring, "...our reals are infinite-precision" and not specifying upper and lower bounds for integers. These assumptions are common in mathematics but not valid for computer programs. Some programs can be shown to be continuous by Chaudhuri's method but will exhibit discontinuous behavior when executed.
Models often have properties real mechanisms do not have, and it is possible to verify the correctness of a model of a program even if the actual program will fail.
The article also ignored real problems by proposing metrics that are based on data types alone, saying: "The metric over arrays of reals or integers of the same length is the L∞-norm: d(A1, A2) = maxi{|A1[i] A2[i]|}. ... We define d(A1, A2) = ∞ if A1 and A2 have different sizes." As illustrated in the following example, to get a relevant definition of continuity, the nature of the application must be considered: The inappropriateness of the article's metric for some applications can be seen by considering a "data mining" application that identifies a family through the children's Social Security numbers. If the article's metric is applied to three records
the distance between A and B would be infinite and A and C would be very close. However, record B, an extension of record A, describes the family described by A after the birth of the third child. Record C describes a different family. An appropriate metric would consider A close to B and far from C.
Moreover, the article described its examples as "everyday programs," but these programs were typical textbook algorithms and not typical of the software we use every day. For example, the article proved that programs that compute the length of the shortest path between two nodes are continuous. However, widely used route-finding software outputs a path, not just length. A small change in one arc's length could change the output drastically by suggesting a completely different route. One reason software continues to replace analog devices is that users often require discontinuous behavior from those devices. Software with continuous behavior will always be rare.
Proving programs correct has been a goal of computer scientists for half a century; the article reflected how far we still are from achieving that goal. Rather than verify properties of an actual program, it examined models of programs. Models often have properties real mechanisms do not have, and it is possible to verify the correctness of a model of a program even if the actual program will fail.
The article's approach is useful because attempting to prove a model of a program correct can reveal subtle errors. However, when a correctness proof is obtained, it must be taken with a grain of salt.
David Lorge Parnas, Ottawa, Canada
Authors' Response
From a purely mathematical perspective, any function between discrete spaces is continuous, so all computer programs are continuous. But this fact does not carry any useful information. In practice, some programs behave robustly and some do not, and infinite-precision models of programs offer a good way to predict whether a program is robust.
Also, our framework extends to programs that operate on values ranging over finite sets. Continuity is not a good robustness property for such programs, but, say, Lipschitz continuity is. The example programs in our article remain robust under these definitions; we also have evidence that our robustness analysis can be adapted to this context.
Swarat Chaudhuri, Houston, TX, and Sumit Gulwani, Redmond, WA
Footnotes
Communications welcomes your opinion. To submit a Letter to the Editor, please limit yourself to 500 words or less, and send to [email protected].
©2012 ACM 0001-0782/11/01 $15.00
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and full citation on the first page. Copyright for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or fee. Request permission to publish from [email protected] or fax (212) 869-0481.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2012 ACM, Inc.