Recently, I've been mulling over the question of whether it was a good idea or not to join a frontier AI company's safety team for the purposes of reducing extinction risk. One of my big cons was something like: Jay, you think the incentives are less likely to affect you compared to most people. But most AI safety people who join frontier labs probably think this. You will be affected as well. So I decided on a partial mitigation strategy, entirely as a precautionary principle and not at all because I thought I needed to. I committed to myself and to several people I'm close to that if I were to join a frontier lab safety team, I would donate 100% of the surplus that I would gain as a result of taking that job instead of a less lucrative job somewhere else. At this time I was applying for a few jobs, one of which was at a frontier company. Approximately immediately, my System 1 became way less interested in that job. And I didn't even have an offer in hand for a specific amount of money. I don't have good reasons to care a lot about getting more money for myself. I have enough already, and I voluntarily live well below my means. This did not stop the effect from existing, and I didn't notice the effect before. I still don't notice the effect on my thinking in a vacuum. I only notice it by doing a mental side-by-side comparison. I now think anyone who is considering joining a frontier company in order to reduce extinction risk should make this same commitment as a basic defensive measure against perverted incentives. I am sure there exist people who are entirely indifferent to money in this way - this is at least partially a skill issue on my part. But it does seem that "Thinking you are indifferent to the money" is not a reliable signal that your thinking is unaltered by it. This is also an opportunity to say that, if I ever do join a frontier safety team, I officially give you permission to ask me if I'm meeting this commitment of mine in conversation, even if

Hey all, I'm $500 away from our minimum goal on Manifund (4 days left). Would appreciate anyone who helps out! The project is a subscription app where our members fund cool nature & animal welfare projects through voting. The Manifund is to help seed our funding pool so our members can start voting on projects sooner than later. A lot more info at the link below --ben https://manifund.org/projects/nohotdoglove-fund-cool-nature--animal-projects-with-others

wrote a review of Tomas Bjartur's fiction, a sci-fi writer with many ideas that's interesting and relevant to this forum (especially AI risk) https://linch.substack.com/p/tomas-bjartur 

Nerd Blog Power Rankings. Does exactly what it says on the tin. And in the process explains how our best and brightest consultants naming F1 fantasy teams after their employer proves we need regulation of AI right now.

I've found the following abstract frame/set of heuristics useful for thinking about how we can try to affect (or predict) the long-term future:  “How do we want to spend our precision/reach points? And can we spend them more wisely?” [Meta: This is a rough, abstract, and pretty rambly note with assorted links; I’m just trying to pull some stuff out and synthesize it in a way I can more easily reference later (hoping to train habits along these lines). I don't think the ideas here are novel, and honestly I'm not sure who'd find this useful/interesting. (I might also keep editing it as I go.] ---------------------------------------- An underlying POV here is that (a) scope and (b) precision are in tension. (Alts: (a) "ambition / breadth / reach / ...” — vs —  (b) “predictability / fidelity / robustness / ...”). You can aim at something specific and nearby [high precision, limited reach] or at something larger and farther away, fuzzier [low precision, broad reach]. And if you care about the kind of effect you’re having (you want to make X happen, not just looking for influence ~for influence’s sake), this matters a bunch. Importantly, I think there are “architectural” features of the world/reality[1] that can ease this tension somewhat if they're used properly; if you channel your effort through them, you can transmit an intervention without it dissipating (or getting warped) as much as it otherwise would. Any channels like this will still be leaky (and they’re limited), but this sort of “structure” seems like the main thing to look for if you’re hoping to think about or improve the long-term future.  (See a related sketch diagram here. I also often picture something like: “what levers could reach across a paradigm shift?” (or: what features are invariant in relevant ways?)) ---------------------------------------- Some examples / thinking this through a bit: 1. Trying to organize or steer a social movement (/big group of people) might extend your reach, but