Forecasting the future is a difficult task, as there are many more ways to be wrong than right! In addition, a forecast is not just a statement about what might happen, it is also embedded in a context that may be as relevant as the forecast itself when it comes to assessing its credibility and uptake. The focus is on natural disasters such as earthquakes; it is shown that next to extreme geophysical factors, equally real legal risks may also be lurking  in the background for the forecasters. For a note on the broader issue of forecasting the future of our society, see News from the future: how good are our forecasts? (First published: 20091016 / Last Updated: 20131116)

Mieux vaut prévoir sans certitude que de ne pas prévoir du tout (It is far better to foresee without certainty than not to foresee at all) – Henri Poincaré, La Science et l’hypothèse, IV Partie, La Nature, Chapitre IX, 1902. The full text of La science et l’Hypothèse available here).

Il vaut mieux employer notre esprit à supporter les infortunes qui nous arrivent qu’à prévoir celles qui nous peuvent arriver ( It is a better use for our mind to endure misfortunes that happen to us than to foresee those that may happen) – François de La Rochefoucauld, Maxime 174, Maximes (5 editions from 1665 to 1678). Full text from Project Gutenberg.

With apologies to Tennyson, ’tis better to have explored and lost than never to have explored at all. Scott E. Page and John H. Miller. 2007. Complex Adaptive Systems: An Introduction to Computational Models of Social Life (Princeton Studies in Complexity). Kindle location 585.

Source: Wikipedia

We all remember the recent (20090406) earthquake in l’Aquila in central Italy, which was so ably recycled by Silvio Berlusconi for national and international political purposes and image polishing!

According to the Wikipedia article, this earthquake was caused by movement on a NW-SE trending normal fault according to moment tensor solutions. Although Italy lies in a tectonically complex region, the central part of the Appenines has been characterised by extensional tectonics since the Pliocene epoch (i.e. about the last 5 million years), with most of the active faults being normal in type and NW-SE trending. (links removed)

Now this is not the cause, it is the mechanisms with some history. A mechanism, even with some history, is not sufficient an explanation why the quake happened. Listing mechanisms instead of causes is a very common in the geosciences, for instance in meteorology, where we are told that the heavy rain occurred “because high pressure fields moved north or south and a low over central Europe did this or this.” Again, this does not tell why it happened but how it came about.

If we exclude mystical explanations of various kinds (1),  the real causes of the event being forecast should ideally be known. Of course, it is understandable that the final cause is difficult to trace, even with a lot of history; the “real” cause is linked to the event by a chain of mechanisms of the type we are presented with as the “cause”. Nevertheless, a honest attempt should be made to trace back the chain of causalities up to the point where causes still play a part… There is a famous statement by Cicero (see foot note 2) regarding  our ignorance about the causes “which only gods can know”…This is why we humans  use whatever “signs” we have at our disposal. But those signs are no causes, they are indicators. Maybe the “rationale” in the Roman mind is that “everything is connected with everything?”. Or maybe the gods do send us messages , as when the Romans interpreted the flight of birds? I find it a bit intriguing to understand early “warning signals” as “signs”. Scheffer et al. (2009) argue that complex systems may “emit signals” that indicate that a tipping point is about to be reached.  The signals in this case, are unusaul values of variables, or unusual combinations of values.

Let’s come back to the earthquake in l’Aquila: there is an interesting story about a forecast issued in the days and hours before the event  by Giampaolo Giuliani, a research technician (“tecnico di ricerca”)  at the National Physical Laboratory of Gran Sasso. As stated by the Slashdot website, “Giuliani, it turns out, was partially right”. The problem is that a forecast that is only “partially right” is best called “wrong”.  Providing forecasts with lots of estimates about uncertainty of the magnitude and probabilities of occurrence is rarely useful! It also seems that the science behind Giuliani’s forecast was not foolproof, so that, interestingly, we can add it to the many examples of a “correct” forecasts that are the product of chance.  This should not come as a surprise. The discussion on earthquake prediction has been going on for more than 100 years — “both within the scientific community and in its relation with the public — [it] has always been tumultuous.” (Scholtz, 2010)

Sometimes I lie awake at night, and I ask, “Where have I gone wrong?” Then a voice says to me, “This is going to take more than one night.” (Charlie Brown, Charles Schulz’ Peanuts)

Forecasts are difficult because there are many ways in which they can be wrong, but only one way to be correct, i.e. on target regarding it’s occurrence (yes/no), location (coordinates), timing and intensity (all with a range of uncertainty, i.e. the “target” can be large, or small!). Then we have the fact that actual forecasts are a mix of art, science and psychology, plus minor doses of a couple of other ingredients, such as common sense, prudence, guts. For instance, had Giuliani been the director rather than a technician at the National Physical Laboratory of Gran Sasso, his forecasts would certainly have been “less wrong”, even if not necessarily “more correct.” It remains that forecasts are rarely “neutral” when they entail immediate consequences for other people, from slight inconvenience to outright suffering. A well known example is the alarm raised in 1976 by French scientist Claude Allegre, who later became minister, about the imminent eruption of  the Soufriere volcano on the island of Guadeloupe. Haroun Tazieff argued against the evacuation of  75,000 people and  turned out to be right – or luckier – La soufrière controversy illustrates the complex nature of some forecasts: it was not a clash of solid scientic assessments, but a clash of two strong personalities!

Back to l’Aquila: the association of  the victims of the earthquake filed a court case against six members of the Committee for Great Risks and a member of the government who failed to issue a warning (Cartlidge, 2011; Nosengo, 2011) . In fact, the seven defendants are not accused of having issued a wrong forecast. One of them (in the presence of the others, who did not deny the point) stated in a press conference shortly before the events that the risk was actually decreasing because some recent precursor tremors (one of which at the rather high magnitude of 4) had actually released some of the stored energy. This may have encouraged very scared people to stay in their houses instead of taking shelter elsewhere. The specific legal risk incurred by some forecasters is not life threatening but nevertheless very real! The first hearings took place in September 2011 (Panorama and The Economist). The case was   watched closely by many forecasters world wide, as witnessed by the wide international press coverage. At the end of October 2012, six scientists and a former government official were sentenced to six years in prison, after having been found guilty of multiple manslaughter (see BBC, IHT, La Stampa, La Repubblica.) Apart from populism, it’s difficult to understand what is  the deep motivation of the judges. What about the people who did not respect building norms? They get away with it! Let’s wait for the appeal, but be prepared to wait long: for the Vajont disaster of 1963, the final court settlement took place in 1997, 34 years later !

As sometimes happens, the Vatican has a very balanced and measured position, taken from  the address Benedict XVI delivered on 20061106 to the members of the Pontifical Academy of Sciences on the occasion of their plenary assembly held in Rome: scientific predictability also raises the question of the scientist’s ethical responsibilities. His conclusions must be guided by respect for truth and an honest acknowledgment of both the accuracy and the inevitable limitations of the scientific method. Certainly this means avoiding needlessly alarming predictions when these are not supported by sufficient data or exceed science’s actual ability to predict. But it also means avoiding the opposite, namely a silence, born of fear, in the face of genuine problems.

Back to the safer “technical” point of view:  there are many different approaches to forecasting. As I have written elsewhere, forecasting methods can be subdivided into various categories according to the relative share of judgement, statistics, models and data used in the process. But the most common and most obvious method is based on the fact that “things don’t change overnight”, i.e. all systems we deal with have a marked inertia and resist change, so that the safest way of forecasting is to assume that conditions will remain the same, preferably average, and that whatever trends have been underway will continue into the future, at least for “some” time. Easy, but not very exciting or, as the saying goes: no guts, no glory!

The interesting problems arise with drastic changes, fast or violent variations, and surprises, such as extreme events. The word “extreme” itself needs clarifying. It means “rare”, i.e. extreme from a statistical point of view. It does not normally or primarily mean “violent” or “extremely intense”, although the two meanings of “extreme” may overlap. Surprises, by definition, are difficult to forecast because they have a very weak statistical basis, and the usual toolbox of the statistician (levels, trends, seasonal patterns, correlations and autocorrelations) cannot be resorted to.

Nature published an article by a group of scientists who claim even wars can be predicted (See Gilbert, 2009, which is a one-page introduction to Bohorquez et al, 2009). We could probably also quote the recent economic crisis as an example of a major phenomenon that was not forecast with any degree of accuracy. In a recent article in the Italian daily Il Corriere della Sera,  Andrea Ichino, wonders why hurricanes can be forecast, but economic crises cannot.

Wall Street indices predicted nine out of the last five recessions ! (Paul A. Samuelson in Newsweek, Science and Stocks, 19 Sep. 1966. From: Famous forecasting quotes)

He wrongly assumes that one of the reasons is that meteorologists have better data, but he rightly thinks that the second reason is human interference in the form of management and policies. They constitute what Raskin et al. (2002) have categorised as the third type of indeterminacy: volition, next to ignorance and surprise, although I would tend to categorize surprise as a form of ignorance that cannot be modelled with “scientific methods” (Raskin et al., 2002); also look up the internet for “Black Swan Theory” and N.N. Taleb).

Volition also explains why many forecasts that were originally correct become incorrect over time. For instance, if I forecast that 50,000 cows will drown because of floods in the coming weeks, people may take my forecast seriously and move their cattle out of the valley, thereby falsifying the forecast. This is often referred to as “second-order errors” and constitutes the most difficult type of error to deal with in forecasting.

My conclusion is that, in the same way as some animals are more equal than others – and possibly for the same reasons- some forecasts are more reliable than others. The reasons why this is so are complex, and span to whole spectrum from science to  art to politics. In the end, the only way to be reasonably sure of the future is to make it happen!

Notes

(1) According to this AP report (dated 20100419),  A senior Iranian cleric says women who wear revealing clothing and behave promiscuously are to blame for earthquakes. Iran is one of the world’s most earthquake-prone countries, and the cleric’s unusual explanation for why the earth shakes follows a prediction by President Mahmoud Ahmadinejad that a quake is certain to hit Tehran and that many of its 12 million inhabitants should relocate. “Many women who do not dress modestly … lead young men astray, corrupt their chastity and spread adultery in society, which (consequently) increases earthquakes,” Hojatoleslam Kazem Sedighi was quoted as saying by Iranian media.

(2) Marci Tulli Ciceronis de divinatione liber prior, LVI 127:  Qui enim teneat causas rerum futurarum, idem necesse est omnia teneat quae futura sint. Quod cum nemo facere nisi deus possit, relinquendum est homini, ut signis quibusdam consequentia declarantibus futura praesentiat. This translates to Qui tiendrait, en effet, les causes des événements futurs saurait nécessairement quels ils seront. Mais nul autre qu’un dieu n’étant capable de pareille connaissance, il reste que l’homme s’applique à prévoir l’avenir d’après les signes qui le présagent. Source: click here.

References

Bohorquez, J.C, S. Gourley, A.R. Dixon, M. Spagat & N.F. Johnson. 2009. Common ecology quantifies human insurgency. Nature, 462:911-914
Cartlidge, E. 2011. Quake experts to be tried for manslaughter. Science, 332:1135-116
Gilbert, N. 2009. Modellers claim wars are predictable. Nature, 462:836.
Nosengo,N.  2011. Scientists on trial over L’Aquila deaths Seismologists charged for giving apparent reassurances on Italian earthquake. Nature, 404:15.
Raskin, P., T.Banuri,  G.Gallopín, P.Gutman, Al Hammond, R.Kates & R. Swart. 2002. Great transition, the promise and lure of the times ahead. Stockholm Environmental Institute. 99 pp.
Scheffer,M., J.Bascompte, W.A. Brock, V.Brovkin, S.R. Carpenter, V.Dakos, H.Held, E. H.van Nes, M. Rietkerk & G.Sugihara. 2009. Early-warning signals for critical transitions. Nature 461: 53-59.
Scholz,C.H.,  2010. The Prediction Puzzle. Science, 327:1082. A review about “Predicting the unpredictable, the tumultuous science of earthquake prediction”, by Susan Hough.