Source: Adobe stock. Source of featured image: The MIndcircle.

This post was originally published on 21 November 2011. It was reformatted on 20260614 and a short paragraph, a new conclusion and some references were added. 2026 is listed as the new publication date.

Forecasting is very difficult, especially when it involves the future. Source: Yogi Berra in AZQuotes.

After “The risky business of forecasting“, which focused on  natural disasters, here is a note that looks more closely at the broader picture of our societies. It reviews different articles published about 15 years ago that have shown “some skill” in forecasting the future. The first comprises of simple power law equations the coefficients of which have been empirically determined over several orders of magnitude. The underlying models are at the border of diverse disciplines, from ecology to sociology. Next comes  the systematic monitoring of the news of which we now have many digital sources available, an approach known as culturomics. Finally, we have a revival of the old methods of critical thresholds which was popular among the old polemologists à la Bouthoul¹ (1962). The exceedence of the thresholds triggers qualitative changes, for instance in the case of riots associated with food prices.

Can the future be predicted based on an analysis that combines history, sociology, and mathematical statistics to make general predictions about the future behaviour of very large groups of people². It seems it can, if we trust the approaches below.

1. Empirical equations

Let us start with a couple of articles published about 2010 by Bohorquez et al. (2009) and by Johnson et al. (2011). The authors of the first paper are engineers, physicists and an economist.  At the time of the publication, Bohorquez was working at the Department of Industrial Engineering and CEIBA Complex Systems Research Center à l’Universidad de Los Andes at Bogota, Colombia. The scientists that co-sign the paper by Johnson include a larger number of disciplines, from biology to sociology, computer sciences and physics. Johnson himself is a physiscist at the university of Miami. Also note that the two groups collaborate.

What’s in the articles? To start with, there is a very simple power law that describes the interval between terrorist attacks (or war actions). This interval tends to keep shrinking while the terrorists learn their “jobs”. If the law is known, the date of the next attack can be inferred (with some uncertainty, of course). There is also a simple relation between the magnitude of the attacks and their frequency: the frequency decreases in inverse proportion to the magnitude at the power of 2.5  (Gilbert, 2009).

The merit of those studies is that they quantitavely relate some violent (as well as non-violent) human behaviours (i.e. they apply beyond terrorism), ecology and some economic models (don’t forget “eco” is a common denominator between eco-logy and eco-nomics!). They are reminiscent of other studies that use power laws to link the size of cities (expressed by their number of inhabitants) and a varied collection of indicators, from average salary to the number of inventors and from the power consumption of households to the density of petrol stations (Bettencourt et al., 2007; Bettencourt and West, 2011). These studies can be used to “predict” how some variables will vary in, say, 2050. This is because many indicators are directly related to population (independent variable). It happens that future population numbers are rather predictable since the majority of people who will be living in 2050 have already been born. Compared with other projections, populations tend to be amazingly accurate at the global scale (see, for instance Chi, 2009).

2. Culturomics

Recently, some authors (esp. Leetaru, 2011), have adopted a radically different approach to forecasting, based on the fact that we can now avail ourselves of huge digital databases of the written press, news agencies, not to mention the websites of national and international newspapers and magazines.  The databases cover at least thirty years. Using data mining techniques, it is possible to retreive certain terms, their frequencies, asociation with other terms, as well as their tone and geolocation. Tone and geolocation are the two innovations introduced by Leetaru. The tone (mood would be a better term!) is given by “positive” and “negative”  terms such as “terrible”, “improvement” or “happy”. Geolocation simply associates geographic coordinates with the above-mentioned terms. The approach, which Leetaru calls “culturomics” has enabled him to issue short-term forecasts about the revolutions in  Egypt, Tunisia and  Lybia, to see the conflict build up in Serbia and to forecast the stability of Saudi Arabia until at least 2012. When applied to the whereabouts of Osama bin Laden, the method identifies an area that includes Abbotabad where the US raid eventually caught up with him.

3. Exceedence of critical thresholds

I’ll conclude by quoting a study by  Lagi et al. (2011) that attracted considerable attention, a very readable account of which is given by  E.M. Johnson, 2011 The authors have abserved a historical association between riots and the cost of food. The threshold above which problems start is about  220 $/ton in current US$ and about 190 US$/ton in constant 2004 prices. The threshold was exceeded  in 2008, and again during the Arab spring. If current food price trend persist, the next revolutions are expected between July 2012 and August  2013.

4. The new future (June 2026)

Today, in 2026, the intellectual, political and economic landscape bears very little resemblance with the situation in 2011, essentially because we now have AI which, we keep being told (promised? threatened?) will change our societies and our lives lives completely. There is also, of course, Trump II and the erosion of US democracy which few people have forecast a couple of years ago. Very much influenced by my former Bulgarian⁴ boss at the UN, I used to believe that individuals have little influence on the overall direction taken by the world. Instead, the drivers are economics and economics, not to forget economics.

In many lectures I have given over the years on “The future”, a favourite subject of mine, I frequently quoted from Raskin’s Great Transition report (Raskin et al., 2002): Global futures cannot be predicted because of three types of indeterminacy: ignorance, surprise, volition. There have been many global and integrative simulations of the future, including The Limits to growth in 1972 and it’s 2002 update, the US produced Mapping the global future⁵ (2005). I have always had some doubts about volition but I must admit that Trump and his fellow autocrats seem to be changing that!

Be that as it may, I have not seen any global study of the future for a couple of years now, which I find extremely surprising, given that Artificial Intelligence is presumably good at (for) everything, and rarely shy to express it’s “opinions”. All I have seen recently⁶ is a quote of a study of the same ilk as those mentioned in the preceding paragraphs known as The 3.5% rule. In fact, “popular” IA has been around only for just a couple of years (Large Language Models and ChatGPT appeared in 2022). This means that, for about 10 years no new Global society-wide projections were issued. Instead, we have scenarios, risk thresholds and the like, some of which have achieved immense popularity, e.g. the Planetary boundaries study by Will Steffen et al that was issued in 2015 after some preparatory papers starting around 2010. The IMAGE 2.1 model was published in 2000 (Knight, 2000) comes probably closer to the spirit of modeling approach of the pioneers of the Club of Rome. Arguably, IMAGE is one of the most comprehensive and complex models available; it is best described as a sectoral impact study. In reality, one of the reasons why we have seen no new global societal projections for 15 years is that, probably, we have come to realise that the global system “Earth” is too complex⁷ to model meaningfully.

5. Conclusion

My first conclusion to this post (the 2011 version) found that the methods listed above (in sections 1. 2 and 3) were rather promising; the attention triggered by the papers by  Leetari, Lagi et those that originate in the “school” of Geoffrey West testified to the interest of forecasts for the  scientific community and the press at large.

It seems, however, that the performance of the methods over earlier approaches had more to do with the volume of data available than with the  methods themselves.

There have been some recent publications on the generic issue of global forecasting à la Seldon (Kossowska et al. 2023, The Forecasting Collaborative 2023, Rocha 2024). Here is a quote from the conclusion of Isabella Rocha’s paper: In summary, this paper highlights the potential of integrating TDA⁸ and AI techniques with social media data to advance our understanding of societal trends. By building on the legacy of Psychohistory, we aim to inspire further research that holistically incorporates historical, psychological, mathematical, and computational studies, paving the way for significant advancements in the field of Computational Social Sciences.

Access to data has never been easier. I am really looking forward to the sweeping AI-generated narratives, graphs and tables telling us everything about our future?⁹

References

Bettencourt, L.M.A., J.Lobo, D.Helbing, C.Kühnert & G.B. West. 2007. Growth, innovation, scaling, and the pace of life in cities. PNAS, 104(17):7301–7306.

Bettencourt, L.M.A & G.B. West. 2011. Bigger Cities do more with less: new science reveals why cities become more productive and efficient as they grow. 305(3):51-53.

Bohorquez, J.C., S.Gourley, A.R.Dixon, M.Spagat & N.F.Johnson. 2009. Common ecology quantifies human insurgency. Nature 462:911-914.

Bouthoul, G. 1962. Le Phénomène-Guerre. Petite bibliothèque Payot, Paris. 283 pp.

Chi, G. 2009. Can knowledge improve population forecasts at subcounty levels? Demography,46:405–427. Available on the net. See also http://www.esri.com/library/whitepapers/pdfs/evaluating-population.pdf and http://www.ageing.ox.ac.uk/files/workingpaper_507.pdf

Gilbert, N. 2009. Modellers claim wars are predictable.Insurgent attacks follow a universal pattern of timing and casualties. Nature 462:836. This article is a presentation of the one by Bohorquez et al., 2009.

Johnson, E.M. 2011. Freedom to Riot: On the Evolution of Collective Violence.

Johnson, N.F., S.Carran, J.Botner, K.Fontaine, N.Laxague, P.Nuetzel, J.Turnley & B.Tivnan. 2011. Patterns of Escalations in Insurgent and Terrorist Activity. Science 333(81):81-84. Also consult  NPR staff, 2011. Math Can Predict Insurgent Attacks.

Knight, C.G. Global Change Scenarios of the 21st Century, Results from the IMAGE 2.1 Model. J. Alcamo, R. Leemans, E. Kreileman (eds). GeoJournal 51, 272–273. Link.

Kossowska, M., P. Kłodkowski, A. Siewierska-Chmaj,A. Guinote, U. Kessels, M. Moyano, J. Strömbäck. 2023 . Internet-based micro-identities as a driver of societal disintegration. Humanit Soc Sci Commun 10, 955 (2023). Link.

Lagi, M., K.Z.Bertrand & Y.Bar-Yam. 2011. The Food Crises and Political Instability in North Africa and the Middle East. http://arxiv.org/abs/1108.2455v1. The article is downloadable.

K.H.Leetaru. 2011. Culturomics: forecasting large-scale human behaviour using glocal news mwdia tone in time and space. First Monday, 16(9). This is an internet publication. See this site. Also see the animations in http://www.kurzweilai.net/culturomics-2-0-forecasting-large-scale-human-behavior-using-global-news-media-tone-in-time-and-space.

Raskin P, T Bbanuri, G. Gallopín, P. Gutman, A. hammond, R. Kates, R. Swart. 2002. Great Transition, The Promise and Lure of the Times Ahead. Stockholm Environmental Institute (SEI) and Tellus Institute. 111 pp. Downloadable.

Rocha I. 2024. Towards Asimov’s Psychohistory: Harnessing Topological Data Analysis, Artificial Intelligence and Social Media data to Forecast Societal Trends. 21 pp. ArXiv link and this link.

Steffen, W., K. Richardson, J. Rockström, S. E. Cornell, I. Fetzer, E.M. Bennett, R. Biggs, S. R. Carpenter, Wim de Vries, C. A. de Wit, C. Folke, D. Gerten, J. Heinke, G.M. Mace, L.M. Persson, Veerabhadran Ramanathan, B. Reyers, S. Sörlin. 2015.Planetary boundaries: Guiding human development on a changing planet. Science 347. Research article and summary available from Science.

The Forecasting Collaborative. 2023. Insights into the accuracy of social scientists’ forecasts of societal change. Nat Hum Behav 7, Link.

Notes

1. Gaston Bouthoul was a French “polemologist”. Refer to Persée for some of his work. He is the co-founder, with Louise Weiss, of the Institut de Polémologie. Gaston did not make it to the English language Wikipedia, but Louise is available in 24 languages! ↩︎
2. This note is a wink at all science-fiction lovers! The wording is borrowed, with minor changes, from  Wikipedia: Psychohistory is a fictional science in Isaac Asimov‘s Foundation universe which combines history, sociology, and mathematical statistics to make general predictions about the future behavior of very large groups of people, such as the Galactic Empire. It was first introduced in the five short stories (1942–1944) which would later be collected as the 1951 novel Foundation. According to the French edition of Wikipedia, the term was was coined by Nat Schachner and later adopted by Asimov. ↩︎
3. The top part of the figure (a) indicates that  100% of war actions result in 1 casualty, while 1/1000 are resposnible for 100 casualties. Lower part (b):  8 events per day alnmmost never happen, while 30% of the days are characterised by two events. ↩︎
4. Bulgarian working for an international organisation before 1989, which is to say loyal Marxist and extremely right-wing politically. ↩︎
5. I had difficulties locating the original document on the Net. I assume it has fallen victim to some typically Trumpesque limitation, i.e. not sufficiently racist, mention of gender issues… Just in case: ask me for a copy if you feel like reading it. ↩︎
6. In a Guardian article on Trump as a mafia boss. ↩︎
7. Refer to my 2023 post Un jumeau numérique pour le Système alimentaire mondiall? which looks with some more detail at IMAGE and also concludes that the global simulation and near-real-time monitoring system of global food production would incompass many other sectors of the economy (e.g. the energy sector) ? In other words, global projections of the planet may well reach costs that are comparable with current global AI boom, with similar negative side effects and some doubts about the overall usefulness of the undertaking. ↩︎
8. Topological Data Analysis. ↩︎
9. Some people do indeed believe that AI Will Save the World. Others believe that there is a high probability that we will eventually succumb to AI. Others still prefer weighting the pros and the cons (Scientific American, Wikipedia) which will, in all likelihood, take some time before a conclusion is reached. ↩︎