Feds appoint “AI doomer” to run AI safety at US institute

The US AI Safety Institute has appointed Paul Christiano as its head of AI safety, despite criticism from some quarters that his appointment may be risking non-scientific thinking. Christiano is a former OpenAI researcher who has pioneered a foundational AI safety technique called reinforcement learning from human feedback (RLHF), but is also known for predicting a 50% chance of AI development leading to "doom." Some fear that by appointing a so-called "AI doomer," NIST may be risking encouraging non-scientific thinking. There have been rumors that NIST staffers oppose the hiring, and some scientists have warned that focusing on hypothetical killer AI systems or existential AI risks may distract from more pressing concerns about how people use technology. However, the US Secretary of Commerce directly appointed Christiano, citing the need for top talent in the field of AI safety. The leadership team of the US AI Safety Institute will also include Mara Quintero Campbell, Adam Russell, Rob Reich, and Mark Latonero.