Hey there, I'm Austin, currently running https://manifund.org. Always happy to meet people; reach out at [email protected]!
Thanks again for writing this, Chris. I'm revisiting this essay right now as part of a reading group my colleague @Jen baik is hosting, so wanted to take the chance to write some reflections:
"GiveWell for AI safety" is a subject I've discussed a lot for Manifund and have tried to do very very very briefly (see https://manifund.substack.com/p/how-cost-effective-are-ai-safety)
I would love to do more of this ourselves (or, fund someone to do this), but as you mention the talent required for it is scarce. I wish, to start, that CG would just publish their own internal analyses about their $10m+ grants, because I expect they spend a fair amount of their own grantmaker time creating those botecs, and having them published would be so much better than nothing.
Thanks! Cool to see Kaya's data; I've always been proud of pushing Manifold to make its stats page public as well
Can you say more about regulatory hazards in global health, and/or give an example? I'm imagining something like "you publish what you did in a developing country, then the regime changes, and then they notice you did a bunch of things they don't like, and put you in jail"
Thank you for the work you do at AIM; it's long been one of my favorite charities. At our wedding, my wife and I each picked a charity for attendees to donate to in lieu of a wedding gift; my wife picked The Humane League, and I picked the then-Charity Entrepreneurship, now AIM. AIM has also been one of the inspirations for the incubator I now organize, Surplus. Best wishes going forward.
Grace, thank you for the work that you do at EA Australia, and at GWWC previously. I still have fond memories of the EAG talk you gave about the importance of marketing for EA, a subject that I think many people still underrate. And while I haven't been to Australia myself, again and again I'm impressed by the quality of the impactful work and people, from the land down under; keep it up.
You already mentioned, but I think it's worth repeating -- in my view, the best startups do not take the YC deal anymore. I think partly it's because YC is a victim of its own success (the startup ecosystem is now much more developed, and other incubators and VCs bid at higher valuations); partly, the brand has fallen somewhat. Also, my guess is that a lot of the best people who would have used to start a startup, now join/start a lab, or sometimes, a nonprofit.
Some startups (and I presume, their incubators) do achieve extraordinary growth due to AI, though. One example is Cursor -- a VC recently asked "on a ROI basis, which would it have been better to seed invest into, Anthropic or Cursor?", and the answer was actually Cursor due to its rapidity of growth + fast acquisition. (And, the incubators such as https://aigrant.com/ are a inspiration for me when thinking about https://surplus.dev/)
Thank you, Zach. I appreciate you speaking up in public on behalf of CEA; CEA and your leadership is particularly important at times of high stress and uncertainty. I think the actions CEA are taking are entirely reasonable.
I also want to note for the general public, that Manifund has been but a minor player within the larger EA ecosystem, on many dimensions. (Quick estimates: Manifund moved ~$10m in a year when EA moved $1-10B; has 2 FTE in a year when EA has 1000-10,000 FTE. I think we were more impressive as a share of seed-stage AI safety projects funded, but, still not even the biggest funder in that space.) We did not expect this hiring decision to represent all of EA, and I appreciate anyone who pushes back on that narrative.
I hope that someday, I, Caroline, and/or Manifund will be able to regain CEA's trust through our actions. Until then, I wish you the best with your work, and apologize for making your lives harder.
I think I can share that the $500k was a grant (not investment) from the Future Fund, to fund a program where successful Manifold traders could choose which charities would receive donations: https://forum.effectivealtruism.org/posts/mbBfFHKEDBTwTSSJX/predicting-for-good-charity-prediction-markets
Manifund ran this program for ~2 years and distributed most of this funding to charities & charitable projects, with the top 3 being Givewell (~$61k), LTFF (~$38k), Rethink Priorities (~$27k). We offered to return remaining funds to the estate; we are still in litigation. This case has been going on for years, and may continue for months at least; litigation takes a while. I do hope to write a retro on this overall program someday.
For reference, Lightcone received ~$4.2m from FTX Future Fund and eventually returned ~$1.7m (and then paid some more in legal fees, maybe ~$0.6m): https://www.lesswrong.com/posts/5n2ZQcbc7r4R8mvqc/the-lightcone-is-nothing-without-its-people
Ah, to be clear, I don't wish to take particularly much credit for his outlook change (though, I do believe he eg somewhat reconsidered his outlook on trans people after meeting a trans person at Manifest. I also believe he did meet some EAs which he collaborated more with afterwards); mostly I cite it as an example of prescience
Thank you very much for writing this! We've integrated Pangram into Manifund, as a flag and visibility filter on projects; on net it's been very helpful due to the volume of low-quality AI submissions we get through the site. However, I am sympathetic eg to your example of a non-native speaker using AI to get their points across. Recently, I had a back and forth with a Chinese-speaking student on Manifund who started with an AI writing proposal which put me off, but I eventually decided to give him a small grant after more dialogue.
I appreciate your headline statement that Pangram's 99.9% accuracy is overstated as well, that was a useful update.
(Disclaimer, I've also recently made a minor ($10k) personal investment into Pangram.)