“Think about what you do”: Why Jürg Fröhlich wants scientists to study psychology and philosophy
A bright idea, a clever innovation, and finally a chaotic and destructive mess that the creator cannot control… That’s the plot of Goethe’s The Sorcerer's Apprentice. It’s also, as Professor Emeritus Jürg Fröhlich points out, a very apt warning for science in our times.
Fröhlich, a theoretical physicist who founded the ETH Center for Theoretical Studies, introduced the metaphor of the sorcerer's apprentice in this year’s John von Neumann lecture under the topic “Science and the (Ir)responsibility of Scientists”. In the lecture, he considered von Neumann himself as a brilliant mathematician and computer scientist, but one whose work contributed to setting destructive forces loose in the world. We had the pleasure of speaking with Fröhlich to explore the theme further.
“It is very important to confront students with the great problems and dangers the modern world is facing, and to make them aware of the responsibility we all have to secure the future of our species,” says Fröhlich. “Richard Feynman, one of the members of the Manhattan Project, admitted that the scientists in Los Alamos, developing the first nuclear weapons, made the big mistake of not pausing the project after Germany was defeated. He went on to say that he learned from that experience that one should perpetually reconsider one’s motivation for doing something.”
Fröhlich goes on: “You know, this can be taught. People can get used to the idea that they have to think about what they are doing, and decide whether it's compatible with their moral compass.”
Does this sound melodramatic? It’s possible that the development of atomic weapons already seems like ancient history. Today’s young scientists are more likely to be turning their attention to innovations more closely woven into our daily lives, without realising that these too can be very dangerous.
Fröhlich cites the social media and AI industries. In both cases, personalities who were engaged in driving the development of these new technologies have spoken out about their regrets. Facebook’s founding president Sean Parker has turned against social media, describing himself as a “conscientious objector” as he reported that the conscious thought process behind building the social media giant was: “Let’s exploit a vulnerability in human psychology” (to get users hooked). And of course Geoffrey Hinton, one of the “godfathers of AI”, has expressed concern about the possible unintended consequences. How is it that these highly intelligent people could knowingly devote themselves to building such dangerous tools?
The blinkered view of science education
For students in STEM subjects, Fröhlich argues, “things like psychology, the history of ideas, the history of good and bad developments growing out of scientific progress, are not really taught. And most people have no idea of the forces that drive them, introspection is not encouraged. I think that's a really bad mistake. There’s no attempt to form the personalities and characters of young people.”
Fröhlich himself was very interested in psychology as a young man, and even considered pursuing a career in medicine and psychiatry. Having chosen physics instead, he regularly attended lectures in subjects beyond the hard sciences. “In the second half of the 1960s, this was quite common. There were outstanding professors of history, philosophy and psychology teaching at ETH. Our interest in such subjects was paired with political activism. I think that’s missing now.”
In former times, he argues, “there was more concern about developing personalities and acquiring some kind of moral compass. Nowadays we focus too much on the purely rational understanding and manipulation of the world, and grossly neglect trying to understand the fundamental irrational forces driving human behaviour.”
That may be changing. It is striking that at many prestigious, tech-focused universities (Stanford, Cornell and MIT come to mind), ethics and social responsibility are starting to form an important part of the curriculum. Perhaps this is a natural result of the proliferating crises that can be seen all around us.
As Fröhlich says: “I find it very hard to imagine that young people are unaware of the multitude of very grave unsolved problems we are facing. To be confronted with such a chaotic, unpredictable world should compel one to look sideways, beyond one’s own subject, and to absorb what other people have to say about the present state of the world.”
Those grave problems extend across multiple spheres of life. “War has not been made obsolete,” Fröhlich points out. “Peace, freedom, economic and social justice and prosperity will not be established merely with the help of game theory, computer algorithms and AI. It is important to not only be aware of all the great problems and threats, but also to consider and encourage those developments that inspire hope.”
Poised on the brink
In January this year, the Science and Security Board of the Bulletin of the Atomic Scientists set the Doomsday Clock at 85 seconds to midnight, the closest it has ever been, citing an array of threats that encompass not only the possibility of nuclear war, but also risks posed by climate change, biotechnology, AI and more. Fröhlich too feels that humanity is “poised on the brink”: he concluded his John von Neumann lecture with a slide showing the Baroque painting The Tree of Life, which hangs in the Cathedral of Segovia. In this allegory, a lively banquet is taking place in the branches of a tree – which is poised to topple, as a skeletal Death figure has hacked almost all the way through the trunk.
“The presence of nuclear weapons in many countries, including North Korea and possibly Iran – it is utterly frightening. The threat of nuclear war never went away, it has only been growing. And now there are very concrete threats from escalating wars, such as the ongoing wars in Ukraine and the Middle East.”
But uncontrolled technological development also poses enormous dangers, he warns.
“I think of autonomous, non-human agents applied to modern warfare (fighting robots, guiding systems for drones and missiles, military hardware in space, etc.), surveillance tools (cameras with face recognition), hidden data sets trained on millions of people, tools serving to manipulate opinions and decisions, tools to do mathematics and science with results nobody sees through, applications of AI in medicine and pharmacology that nobody understands – quite generally, the replacement of human understanding, decision making and action by AI tools that no human really controls.”
So then: what exactly is the responsibility of scientists? Fröhlich cites a Latin proverb: “Quidquid agas, prudenter agas et respice finem.” (Whatever you do, do it wisely and consider the consequences.) With this in mind, he suggests, it is crucial not to give up the hope that while engaging in scientific work, decision making, management and politics, we all have a chance to contribute to improving the state of the world.
And the responsibility of the academy: train students to take this seriously.
The annual John von Neumann lecture is organised by the NCCR Automation and the University of Zurich’s Zurich Center for Market Design. Since 2020 we have invited leading researchers in game theory, computer science and other fields in which Von Neumann was a leader. You can find previous interviews here:
