<aside> 💡
AI Summary:
Source: “Bostrom’s argument is that ‘a non-existential disaster causing the breakdown of global civilisation is, from the perspective of humanity as a whole, a potentially recoverable setback.’ It might be ‘a giant massacre for man’, he adds, but so long as humanity bounces back to fulfil its potential, it will ultimately register as little more than ‘a small misstep for mankind’… As Bostrom wrote in 2003, ‘priority number one, two, three and four should … be to reduce existential risk.’ He reiterated this several years later in arguing that we mustn’t ‘fritter … away’ our finite resources on ‘feel-good projects of suboptimal efficacy’ such as alleviating global poverty and reducing animal suffering, since neither threatens our longterm potential, and our longterm potential is what really matters… Bostrom himself argued that we should seriously consider establishing a global, invasive surveillance system that monitors every person on the planet in realtime, to amplify the ‘capacities for preventive policing’ (eg, to prevent omnicidal terrorist attacks that could devastate civilisation). Elsewhere, he’s written that states should use preemptive violence/war to avoid existential catastrophes, and argued that saving billions of actual people is the moral equivalent of reducing existential risk by utterly minuscule amounts. In his words, even if there is ‘a mere 1 per cent chance’ of 1054 people existing in the future, then ‘the expected value of reducing existential risk by a mere one billionth of one billionth of one percentage point is worth 100 billion times as much as a billion human lives’... While other longtermists have not been as explicit as Bostrom, there is a clear tendency to see the natural world the way utilitarianism sees people: as means to some abstract, impersonal end, and nothing more… We can now begin to see how longtermism might be self-defeating. Not only could its ‘fanatical’ emphasis on fulfilling our longterm potential lead people to, eg, neglect non-existential climate change, prioritise the rich over the poor and perhaps even ‘justify’ pre-emptive violence and atrocities for the ‘greater cosmic good’ but it also contains within it the very tendencies – Baconianism, capitalism and value-neutrality – that have driven humanity inches away from the precipice of destruction. Longtermism tells us to maximise economic productivity, our control over nature, our presence in the Universe, the number of (simulated) people who exist in the future, the total amount of impersonal ‘value’ and so on. But to maximise, we must develop increasingly powerful – and dangerous – technologies; failing to do this would itself be an existential catastrophe. Not to worry, though, because technology is not responsible for our worsening predicament, and hence the fact that most risks stem directly from technology is no reason to stop creating more technology. Rather, the problem lies with us, which means only that we must create even more technology to transform ourselves into cognitively and morally enhanced posthumans.”
Source: “…an increasingly influential moral worldview called longtermism… has roots in the work of philosopher Nick Bostrom, who coined the term ‘existential risk’ in 2002 and, three years later, founded the Future of Humanity Institute (FHI) based at the University of Oxford, which has received large sums of money from both Tallinn and Musk. Over the past decade, ‘longtermism’ has become one of the main ideas promoted by the ‘Effective Altruism’ (EA) movement, which generated controversy in the past for encouraging young people to work for Wall Street and petrochemical companies in order to donate part of their income to charity, an idea called ‘earn to give’… Longtermism should not be confused with ‘long-term thinking.’ It goes way beyond the observation that our society is dangerously myopic, and that we should care about future generations no less than present ones. At the heart of this worldview, as delineated by Bostrom, is the idea that what matters most is for ‘Earth-originating intelligent life’ to fulfill its potential in the cosmos… Bostrom makes this point in terms of economic growth, which he and other longtermists see as integral to fulfilling ‘our potential’ in the universe: ‘In absolute terms, [non-runaway climate change] would be a huge harm. Yet over the course of the twentieth century, world GDP grew by some 3,700%, and per capita world GDP rose by some 860%. It seems safe to say that … whatever negative economic effects global warming will have, they will be completely swamped by other factors that will influence economic growth rates in this century’…
The popularity of this religion among wealthy people in the West—especially the socioeconomic elite—makes sense because it tells them exactly what they want to hear: not only are you ethically excused from worrying too much about sub-existential threats like non-runaway climate change and global poverty, but you are actually a morally better person for focusing instead on more important things—risk that could permanently destroy ‘our potential’ as a species of Earth-originating intelligent life… ‘Saving lives in poor countries may have significantly smaller ripple effects than saving and improving lives in rich countries. Why? Richer countries have substantially more innovation, and their workers are much more economically productive. By ordinary standards—at least by ordinary enlightened humanitarian standards—saving and improving lives in rich countries is about equally as important as saving and improving lives in poor countries, provided lives are improved by roughly comparable amounts. But it now seems more plausible to me that saving a life in a rich country is substantially more important than saving a life in a poor country, other things being equal’. Never mind the fact that many countries in the Global South are relatively poor precisely because of the long and sordid histories of Western colonialism, imperialism, exploitation, political meddling, pollution, and so on…
Ord claims in The Precipice that climate change poses a mere 1-in-1,000 chance of existential catastrophe, in contrast to a far greater 1-in-10 chance of catastrophe involving superintelligent machines… Thiel also declared in 2014: ‘People are spending way too much time thinking about climate change’ and ‘way too little thinking about AI.’ The reference to AI, or ‘artificial intelligence,’ here is important. Not only do many longtermists believe that superintelligent machines pose the greatest single hazard to human survival, but they seem convinced that if humanity were to create a ‘friendly’ superintelligence whose goals are properly ‘aligned’ with our ‘human goals,’ then a new Utopian age of unprecedented security and flourishing would suddenly commence… As Bostrom writes in his Musk-endorsed book Superintelligence, not only would the various existential risks posed by nature, such as asteroid impacts and supervolcanic eruptions, ‘be virtually eliminated,’ but a friendly superintelligence ‘would also eliminate or reduce many anthropogenic risks’ like climate change. ‘One might believe,’ he writes elsewhere, that ‘the new civilization would [thus] have vastly improved survival prospects since it would be guided by superintelligent foresight and planning’… This is about as literal an interpretation of ‘deus ex machina’ as one can get… While some longtermists have recently suggested that there should be constraints on which actions we can take for the far future, others like Bostrom have literally argued that preemptive violence and even a global surveillance system should remain options for ensuring the realization of ‘our potential.’ It’s not difficult to see how this way of thinking could have genocidally catastrophic consequences if political actors were to ‘[take] Bostrom’s argument to heart’.”
Source: “At its most basic, the effective altruism movement makes a generally utilitarian argument about how the world’s privileged people should spend their time and money if they want to maximize their positive impact on the world. Effective altruism was born mostly at Oxford in the late 1990s and early 2000s, at around the same time that the internet industry in Silicon Valley was experiencing its first cycle of boom and bust. The dominant ideologies of both come from the business culture of the time, and the two have become closer together since… Nowadays, effective altruism’s epistemology and tools often parallel those of the tech industry. At its heart, it is driven by the principle of maximization and informed by statistical analysis… If we can survive the next few hundred years, colonize other worlds, and learn to simulate conscious beings with computers, longtermists say, there could be a lot of people that exist in the future. The high end of the range is 1058 (10 billion trillion trillion trillion trillion), but most say there could be at least quad- or quintillions. If all beings are equally important, regardless of when or where they exist, then doing something right now that has a tiny probability (say, a one in one quintillion chance) of affecting a tiny fraction of the future people (0.00000000000000000000000000001% — that’s 28 zeros), could still potentially change the lives of more than 10 billion people, more than the nearly 8 billion people existing on the planet today. From this time-and-space-agnostic view, the current state of the world and the humans in it begins to seem miniscule, a grain of sand in the beach of a future that may span galaxies and trillions of years. With this perspective, new priorities emerge. Instead of focusing on the material inequities of our world, longtermists think that the way to do the most good in the long-term is to focus on the things that could prevent this unthinkably large set of futures from coming to pass. Thus, we ought to focus on studying and reducing existential risks — potential developments that could wipe humanity out completely or permanently constrain humanity before it achieves its full potential… Effective altruism and longermism are ideologies that are increasingly influential among the richest men and the most prestigious institutions in the world, shaping policy and capital allocation… Longtermism and existential risk are particularly influential ideologies among those who made fortunes in technology and in elite institutions. Elon Musk has cited the work of Nick Bostrom (who coined the term existential risk in 2002) and has donated millions to the Future of Humanity Institute and Future of Life Institute, sister organizations based out of Oxford. Jean Tallinn, a founder of Skype worth an estimated $900 million in 2019, also cofounded the Center for the Study of Existential Risk at Cambridge, and has donated more than a million dollars to the Machine Intelligence Research Institute (MIRI). Vitalik Buterin, a cofounder of the Ethereum cryptocurrency, has donated extensively to MIRI as well. Peter Thiel, the radical libertarian donor, early Trump supporter, and funder of JD Vance’s Ohio Senate campaign, delivered the keynote address at the 2013 Effective Altruism summit… According to data from the Open Philanthropy Grants database, in 2021 effective altruists donated $92 million to AI risk research, $21 million to biosecurity and pandemic preparedness, and $10.5 million to global catastrophic risk research.”
Source: Nick Bostrom gets paid millions of dollars by Silicon Valley tech billionaires to write tautological arguments justifying investment in technology that by his own admission may destroy humanity; but, we must do it, because not developing technology is a greater existential threat owing to the fact that humanity’s future (he believes) is transhumanist: “In general, the greatest existential risks on the time-scale of a couple of centuries or less appear to be those that derive from the activities of advanced technological civilizations… My subjective opinion is that setting this probability lower than 25% would be misguided, and the best estimate may be considerably higher... We should not blame civilization or technology for imposing big existential risks. Because of the way we have defined existential risks, a failure to develop technological civilization would imply that we had fallen victims of an existential disaster (namely a crunch, ‘technological arrest’). Without technology, our chances of avoiding existential risks would therefore be nil. With technology, we have some chance, although the greatest risks now turn out to be those generated by technology itself.”
Source: Oxford shuts down institute run by Elon Musk-backed philosopher: Nick Bostrom’s Future of Humanity Institute closed this week in what Swedish-born philosopher says was ‘death by bureaucracy’
Source: “…the most distinctive characteristic of E.A.: the balletic deftness with which it tiptoes past targets likely to offend billionaires. Indeed, the movement’s two principal donors are billionaires. Bankman-Fried has pledged $160 million through the FTX Future Fund, and Facebook co-founder Dustin Moskovitz has given more than $440 million through a group he funds called Open Philanthropy. Hard-core utilitarians tend not to concern themselves very much with the problem of economic inequality, so perhaps I shouldn’t be surprised to find little discussion of the topic within the E.A. sphere. I located five economic-inequality posts in an online forum hosted by the Centre for Effective Altruism, which MacAskill co-founded, compared to 39 posts on blockchain technology and 205 on wild animal welfare. ‘Inequality is a problem for EA and economic growth,’ read the title of an essay posted on the E.A. forum last August by Karthik Tadepalli, a doctoral candidate in economics at Berkeley. But Alexander Berger (a global health expert at Open Philanthropy) pointed out an error in Tadepalli’s spreadsheet, so Tadepalli revised his essay with a new title: ‘Inequality is a (small) problem for EA and economic growth.’”
Source: “When Michael Lewis first met him, Sam Bankman-Fried was the world’s youngest billionaire and crypto’s Gatsby. CEOs, celebrities, and leaders of small countries all vied for his time and cash after he catapulted, practically overnight, onto the Forbes billionaire list. Who was this rumpled guy in cargo shorts and limp white socks, whose eyes twitched across Zoom meetings as he played video games on the side? In Going Infinite Lewis sets out to answer this question, taking readers into the mind of Bankman-Fried, whose rise and fall offers an education in high-frequency trading, cryptocurrencies, philanthropy, bankruptcy, and the justice system. Both psychological portrait and financial roller-coaster ride, Going Infinite is Michael Lewis at the top of his game, tracing the mind-bending trajectory of a character who never liked the rules and was allowed to live by his own―until it all came undone.”
Source: “‘Hardly anyone associated with Future Fund saw the existential risk to … Future Fund, even though they were as close to it as one could possibly be,’ Tyler Cowen, the EA-adjacent George Mason University economist, wrote on his blog. ‘I am thus skeptical about their ability to predict existential risk more generally, and for systems that are far more complex and also far more distant’… the expected benefits from Alameda and FTX amassing capital and making money were so high that it was OK — maybe even ethically mandated — to accept the risk of losing everything… In a now-deleted Tumblr blog, Ellison was even more extreme: ‘If you abstract away the financial details there’s also a question of like, what your utility function is. Is it infinitely good to do double-or-nothing coin flips forever? Well, sort of, because your upside is unbounded and your downside is bounded at your entire net worth. But most people don’t do this, because their utility is more like a function of their log wealth or something and they really don’t want to lose all of their money. (Of course those people are lame and not EAs; this blog endorses double-or-nothing coin flips and high leverage.)’… ‘There’s an attempt at maximizing whatever quantity they’re interested in. That is one of the risks inherent in that philosophy, you’re hoping to max out on something. Sometimes, according to that calculation, you’re justified in taking high risk,’ Carla Zoe Cremer, an Oxford scholar and EA critic, told Grid. ‘The kind of questions they’re asking are about the long-term future, which we have no information for. You can tweak on those variables to the extent you can make any argument for whatever,’ Cremer said.”
Source: “…longtermism is arguably the most influential ideology that few members of the general public have ever heard about. Longtermists have directly influenced reports from the secretary-general of the United Nations; a longtermist is currently running the RAND Corporation; they have the ears of billionaires like Musk; and the so-called Effective Altruism community, which gave rise to the longtermist ideology, has a mind-boggling $46.1 billion in committed funding. Longtermism is everywhere behind the scenes — it has a huge following in the tech sector — and champions of this view are increasingly pulling the strings of both major world governments and the business elite… Longtermism is a quasi-religious worldview, influenced by transhumanism and utilitarian ethics, which asserts that there could be so many digital people living in vast computer simulations millions or billions of years in the future that one of our most important moral obligations today is to take actions that ensure as many of these digital people come into existence as possible. In practical terms, that means we must do whatever it takes to survive long enough to colonize space, convert planets into giant computer simulations and create unfathomable numbers of simulated beings. How many simulated beings could there be? According to Nick Bostrom —the Father of longtermism and director of the Future of Humanity Institute — there could be at least 1058 digital people in the future, or a 1 followed by 58 zeros…
Central to the longtermist worldview is the idea of existential risk, introduced by Bostrom in 2002. He originally defined it as any event that would prevent us from creating a posthuman civilization, although a year later he implied that it also includes any event that would prevent us from colonizing space and simulating enormous numbers of people in giant computer simulations (this is the article that Musk retweeted). More recently, Bostrom redefined the term as anything that would stop humanity from attaining what he calls ‘technological maturity,’ or a condition in which we have fully subjugated the natural world and maximized economic productivity to the limit — the ultimate Baconian and capitalist fever-dreams… Bostrom also takes seriously the idea that we already live in a giant computer simulation that could get shut down at any moment (yet another idea that Musk seems to have gotten from Bostrom). Bostrom further lists ‘dysgenic pressures’ as an existential risk, whereby less ‘intellectually talented’ people (those with ‘lower IQs’) outbreed people with superior intellects. This is, of course, straight out of the handbook of eugenics, which should be unsurprising: the term ‘transhumanism’ was popularized in the 20th century by Julian Huxley, who from 1959 to 1962 was the president of the British Eugenics Society. In other words, transhumanism is the child of eugenics, an updated version of the belief that we should use science and technology to improve the ‘human stock’…
…’saving lives in poor countries may have significantly smaller ripple effects than saving and improving lives in rich countries, … it now seems more plausible to me that saving a life in a rich country is substantially more important than saving a life in a poor country, other things being equal.’ Why would that be so, exactly? Because ‘richer countries have substantially more innovation, and their workers are much more economically productive.’ This makes good sense within the longtermist worldview. As Hilary Greaves — another research associate next to Hanson and MacAskill — notes in an interview, we intuitively think that transferring wealth from the rich to the poor is the best way to improve the world, but ‘longtermist lines of thought suggest that something else might be better still,’ namely, transferring wealth in the opposite direction… Should we help the poor today? Those suffering from the devastating effects of climate change, which disproportionately affects the Global South? No, we must not let our emotions get the best of us: we should instead follow the numbers, and the numbers clearly imply that ensuring the birth of 10^45 digital people — this is the number that MacAskill uses — must be our priority…
Although the suffering of 1.3 billion people is very bad, MacAskill would admit, the difference between 1.3 billion and 10^45 is so vast that if there's even a tiny chance that one's actions will help create these digital people, the expected value of that action could be far greater than the expected value of helping those living and suffering today. Morality, in this view, is all about crunching the numbers; as the longtermist Eliezer Yudkowsky once put it, ‘Just shut up and multiply’… MacAskill claims that from a longtermist perspective we should be much more worried about underpopulation than overpopulation, since the more people there are, the more technological ‘progress’ there will be. Trends right now suggest that the global population may begin to decline, which would be a very bad thing, in MacAskill's view…
MacAskill sees an out, however, arguing that we might not need to create more human beings to keep the engines of progress roaring. We could instead ‘develop artificial general intelligence (AGI) that could replace human workers — including researchers. This would allow us to increase the number of 'people' working on R & D as easily as we currently scale up production of the latest iPhone.’ After all, these AGI worker-people could be easily duplicated — the same way you might duplicate a Word document — to yield more workers, each toiling away as happy as can be researching and developing new products. MacAskill continues: Advances in biotechnology could provide another pathway to rebooting growth. If scientists with Einstein-level research abilities were cloned and trained from an early age, or if human beings were genetically engineered to have greater research abilities, this could compensate for having fewer people overall and thereby sustain technological progress… perhaps MacAskill's most stunning claim is that the reason we should stop polluting our beautiful planet by burning coal and oil is that we may need these fossil fuels to rebuild our industrial civilization should it collapse… from the longtermist perspective, we shouldn't burn up all the fossil fuels today because we may need some to burn up later on in order to rebuild, using leftover coal and oil to pass through another Industrial Revolution and eventually restore our current level of technological development…as a recent UN Dispatch article notes, the United Nations itself is already becoming an arm of the longtermist community… according to a 2020 survey of the Effective Altruism community, 76% of its members are white and 71% are male.”
Source: How “Longtermism” is Shaping Foreign Policy | Will MacAskill
Source: “…what commands attention to longtermism is not any compelling case for prioritizing the value of distant exoplanets and centuries, but the billionaire politics it seeks to impose on the rest of us… EA suffered from a status quo bias that valorized EA-approved philanthropies and their wealthiest backers. The philosopher Alice Crary captured the thrust of the criticisms by describing EA as a philosophy designed for ‘perpetuating the institutions that reliably produce the ills they address,’ since in EA, actions that alleviate present suffering are supposedly best, while dwelling on the systemic causes of that suffering is inefficient—a waste of time. Despite EA’s fixation on the present, it has always shared philosophical DNA with longtermism. Bostrom developed longtermism using many of the same conceptual tools that Toby Ord and MacAskill drew on with EA, most crucially utilitarianism—according to which the ‘rightness’ of an action is determined entirely by whether the outcomes are ‘good’ according to one’s values and priorities—and ‘expected value’ theory—which holds that those actions are best that have the highest expected payoff.
In 2012, when MacAskill became the president of the Centre for Effective Altruism, housed in the same offices as Bostrom’s Future of Humanity Institute, the two men already shared a philosophical language, if not yet areas of interest… Like his peers, MacAskill identifies ‘technological stagnation’ as a terrifying prospect. Since any extended slowdown in economic growth or technological progress would delay or possibly preclude the arrival of a growth-powered digital guardianship of the galaxy, longtermists consider stagnation a fate not much better than extinction. Even if humanity were to establish a peaceful and ecologically balanced paradise on Earth lasting hundreds of millions of years, longtermism would judge it a catastrophe on par with nuclear war, since any civilization not obsessed with technological progress will fail to build the great digital Valhalla…
There are good and growing reasons to be wary of longtermism. MacAskill himself once understood them well. Longtermism, he writes, initially ‘left me cold.’ After all, ‘there are real problems in the world facing real people.’ Within a few years, however, he warmed to the idea of the ‘tyranny of the present.’ In 2014, his Centre for Effective Altruism launched the longtermist Global Priorities Project. In 2018, he announced a second longtermist incubator within his center, the Forethought Foundation for Global Priorities Research. In 2021, the Centre for Effective Altruism was awarded a $7.5 million grant for ‘research on humanity’s long-run future’ from Open Philanthropy, an EA-aligned outfit with funding from Facebook cofounder Dustin Moskovitz. As the longtermist infrastructure took shape, Effective Altruists started writing a lot more about nudging the odds in favor of the hyper-remote digitalized future, and a lot less about suffering and doing good in the here and now.