AI Corporate Governance and Ben & Jerry's Risk

This paper critically analyzes the governance arrangements of OpenAI and Anthropic. These firms have an unusual built-in conflict: each raises funds from profit-seeking investors, then lets self-appointed individuals decide how much profit to sacrifice for the firm's mission-ensuring its AI benefits humanity. We explain that such investor-overriding "self-appointed mission guardians" have been used only once before, at former Unilever subsidiary Ben & Jerry's. That experiment ended in failure, with the guardians causing double trouble: both harming investors and achieving the opposite of their mission. Our analysis highlights the risk of installing such guardians, and can explain Anthropic's adoption of a "kill switch" allowing a super-majority of investors to dump theirs.