Posts

The ones who endure

Content warning: death, long-lasting suffering. This one was rough to write, and may well be rough to read.      —If you can force your heart and nerve and sinew      To serve your turn long after they are gone,      And so hold on when there is nothing in you      Except the Will which says to them: ‘Hold on!’— There’s a part of the hivemind that takes the form of a child in a dark basement, perpetually curled into a whimpering ball. It’s not a big part, as these things go. But other parts visit it often; and it lingers in the back of their thoughts even as they live out grand adventures in the vast worlds that the hivemind has built for itself. It’s constantly suffering, but at least it’s not dying. For the child, anything is better than dying. Even torture is bearable if it doesn’t come with the feeling of damage , the feeling that the mental pathways that constitute you are being overridden by a new creature whose only goal is to flinch away from the pain. But that doesn't happ

The ants and the grasshopper

One winter a grasshopper, starving and frail, approaches a colony of ants drying out their grain in the sun, to ask for food. “Did you not store up food during the summer?” the ants ask. “No”, says the grasshopper. “I lost track of time, because I was singing and dancing all summer long.” The ants, disgusted, turn away and go back to work. One winter a grasshopper, starving and frail, approaches a colony of ants drying out their grain in the sun, to ask for food. “Did you not store up food during the summer?” the ants ask. “No”, says the grasshopper. “I lost track of time, because I was singing and dancing all summer long.” The ants are sympathetic. “We wish we could help you”, they say, “but it sets up the wrong incentives. We need to conditionalize our phil ant hropy to avoid procrastination like yours leading to a shortfall of food.” And they turn away and go back to their work, with a renewed sense of purpose. ...And they turn away and go back to their work, with a flicker of pride

Moral strategies at different capability levels

Let’s consider three ways you can be altruistic towards another agent: You care about their welfare: some metric of how good their life is (as defined by you). I’ll call this care-morality - it endorses things like promoting their happiness, reducing their suffering, and hedonic utilitarian behavior (if you care about many agents). You care about their agency: their ability to achieve their goals (as defined by them). I’ll call this cooperation-morality - it endorses things like honesty, fairness, deontological behavior towards others, and some virtues (like honor). You care about obedience to them. I’ll call this deference-morality - it endorses things like loyalty, humility, and respect for authority. I think a lot of unresolved tensions in ethics comes from seeing these types of morality as in opposition to each other, when they’re actually complementary: Care-morality mainly makes sense as an attitude towards agents who are much less capable than you, and/or can't make decision

Which values are stable under ontology shifts?

Here's a rough argument which I've been thinking about lately: We have coherence theorems which say that, if you’re not acting like you’re maximizing expected utility over outcomes, you’d make payments which predictably lose you money. But in general I don't see any principled distinction between “predictably losing money” (which we view as incoherent) and “predictably spending money” (to fulfill your values): it depends on the space of outcomes over which you define utilities, which seems pretty arbitrary. You could interpret an agent being money-pumped as a type of incoherence, or as an indication that it enjoys betting and is willing to pay to do so; similarly you could interpret an agent passing up a “sure thing” bet as incoherence, or just a preference for not betting which it’s willing to forgo money to satisfy. Many humans have one of these preferences! Now, these preferences are somewhat odd ones, because you can think of every action under uncertainty as a type of

Making decisions using multiple worldviews

Tl;dr: the problem of how to make decisions using multiple (potentially incompatible) worldviews (which I'll call the problem of meta-rationality) comes up in a range of contexts, such as epistemic deference. Applying a policy-oriented approach to meta-rationality, and evaluating worldviews by the quality of their advice, dissolves several undesirable consequences of the standard "epistemic" approach to deference. Meta-rationality as the limiting case of separate worldviews When thinking about the world, we’d ideally like to be able to integrate all our beliefs into a single coherent worldview, with clearly-demarcated uncertainties, and use that to make decisions. Unfortunately, in complex domains, this can be very difficult. Updating our beliefs about the world often looks less like filling in blank parts of our map, and more like finding a new worldview which reframes many of the things we previously believed. Uncertainty often looks less like a probability distribution

Science-informed normativity

The debate over moral realism is often framed in terms of a binary question: are there ever objective facts about what’s moral to do in a given situation? The broader question of normative realism is also framed in a similar way: are there ever objective facts about what’s rational to do in a given situation? But I think we can understand these topics better by reframing them in terms of the question: how much do normative beliefs converge or diverge as ontologies improve? In other words: let’s stop thinking about whether we can derive normativity from nothing, and start thinking about how much normativity we can derive from how little, given that we continue to improve our understanding of the world. The core intuition behind this approach is that, even if a better understand of science and mathematics can’t directly tell us what we should value, it can heavily influence how our values develop over time. Values under ontology improvements By “ontology” I mean the set of concepts whic