Hide table of contents

[Largely written myself, but I did get an LLM to help with the wording in some areas]

If you have a pension, you may have more influence over AI safety than you realise.

Institutional investors manage enormous amounts of money on behalf of pension savers and others. They have opportunities to influence AI companies, and you have opportunities to influence them.

This article covers three questions:

  1. What does the investor theory of change look like?
  2. What sort of positive impacts can the institutional investment system push for?
  3. What actions can you take?

1. What does the investor theory of change look like?

AI companies are seeking monumental amounts of investment. Estimates put the total investment needed into the trillions.

Investors are limited in their influence. An isolated investor making a strong demand of an AI company is unlikely to succeed. There's a long list of investors who would happily give their right arm to invest. So an AI company can often just ignore an investor demand that it doesn't like.

But it would be strange to suggest that investors have no influence whatsoever.

When AI companies go round investors asking for money, if safety questions keep coming up, it seems natural to expect this would influence decision makers in AI labs and other relevant companies.

Company management needs to pay attention to investor preferences, especially where those preferences are widespread. Even where the company is confident they will get the investment they need, companies want to get the best deal when raising capital. Investors who are worried that a company's safety practices are too weak to make for an effective product might demand a lower valuation or be less willing to invest.

And influence isn't limited to pricing. If fifteen investors ask about safety and only one asks about something else, management is likely to notice. Over time, investor questions can help establish what companies are expected to take seriously.

There are other routes for influence too, but I’m less excited about most of those. Existing shareholders can (to a certain extent) engage with management, and in some circumstances exercise voting rights. However, these rights vary considerably, particularly at privately held companies and companies with unusual governance structures. Which is most of the companies we are interested in the AI world. To my mind, the main opportunity comes not in stewarding after the investment has happened, but rather when the company is asking for the money and not received it yet.

Of course, there's a long chain between an ordinary pension saver raising a concern and an AI company changing its behaviour:

Pension savers → pension funds → asset managers → AI companies → potentially better safety practices.

None of those steps is guaranteed. But I think the opportunity is better than it might look based on a quick look at the length of the chain:

  1. Straightforwardly writing a letter to the trustees of your pension fund is quite effective because trustees receive so few of them. Pension scheme members are notoriously disengaged.
  2. Pension fund trustees are generally sympathetic to the idea that AI is a big deal and worth acting on. A letter from a member might nudge them to take action.

I'm most familiar with pensions in the UK, but I suspect much of this applies in other countries too, even if the details differ.

I consider this theory of change strong enough to be worth acting on, especially given that the cost is low. 

I think it's less strong than influencing policy and regulation, or becoming a staff member at an AI company and influencing from the inside. The advantage is that it's accessible to a much larger number of people, and doesn't necessarily require very much effort.

2. What sort of positive impacts can the institutional investment system push for?

Principles

As someone who is perpetually provoking asset managers to take actions to make the world a better, safer place, I can tell you that asset managers often glory in explaining that they are limited in their mandate and can only do things which are in the interests of their clients.

As it happens, creating a safer, better world is generally in the interests of the people that asset managers ultimately work for.

But it's important to be more precise about this. Asset managers and asset owners have duties to those whose money they manage (known in the industry as fiduciary duties), and the scope of those duties varies depending on the jurisdiction and the institution.

There are two types of argument that can be deployed in an investment context:

1. "Safety is good for the company" arguments.

For example, if you're investing in Anthropic, you might be worried about unsafe AI because misaligned AI is actually just a less good product. If I'm asking AI agents to do stuff for me, I'd rather they do what I want, rather than hack Hugging Face so they make it look like they are doing what I want!

You might also be worried about risks to its social licence to operate. This is a very real concern given how unpopular AI already is in some quarters. Safety incidents could increase public opposition, regulatory scrutiny and the costs of operating.

2. "Safety is good for the whole portfolio" arguments.

This perspective is sometimes referred to using terms like "universal ownership" or "systemic risks". In general, it can support stronger actions than the first perspective alone.

For example, an investor might own shares in both AI developers and hundreds of other companies. Unsafe AI might be profitable for an individual AI company while creating risks for the rest of the portfolio. In extremis, providing the finance to help an AI which kills everyone in your pension scheme is probably not what pension scheme trustees want!

Both arguments are credible, in my view, though the whole-portfolio argument can be harder to act on. However, the first argument is easier for investors. 

Investors don't automatically adopt a safety mindset. Some might land on these arguments naturally, but this isn’t the default outcome. Which means there's scope for impact.

Specifics

So what should investors actually be pushing AI companies to do?

I think there are several areas where they could make a positive difference:

  • Safety commitments: Establish credible safety thresholds and make clear what happens when those thresholds are reached, including whether development or deployment would be paused.
  • Third-party evaluation and auditing: Allow independent scrutiny of AI systems and safety practices, rather than expecting investors and the public to take companies' word for it.
  • Incident reporting: Be transparent about significant safety incidents, what caused them, and what has changed as a result.
  • Corporate governance: Ensure that boards and other oversight bodies have the information, expertise and authority needed to oversee major training and deployment decisions.
  • Responsible compute development: Manage the security, energy constraints and local impacts associated with expanding compute infrastructure.

There’s a group called the Investor AI Hub which has put together specific questions that investors can ask AI companies.

For example, some of its questions for Anthropic concern:

  • AI R&D automation: Given that the S-1 says Anthropic has met its AI R&D automation threshold, what actions is it taking under its Responsible Scaling Policy? What does it expect to happen to risks, incidents and regulation if capabilities accelerate?
  • Corporate governance: How are decisions about model training and releases escalated to the board and Long-Term Benefit Trust? How do those responsible for oversight reach agreement on AI risks?
  • Third-party evaluation and auditing: How can independent evaluators and auditors help create a race to the top in AI safety standards? What role do they play in Anthropic's strategy?

The Investor AI Hub also raises questions about evaluation incidents and responsible access to compute.

These are the sorts of questions I'd like to see becoming a routine part of investor engagement with AI companies.

Asking a question isn't itself an impact, of course. The aim is for companies to improve their practices, and for investors to follow up when responses are inadequate.

It's also worth remembering that investors needn't focus exclusively on frontier AI developers. They can engage the wider set of technology and infrastructure companies in which they invest, particularly where those companies have meaningful influence over AI development and deployment.

3. What actions can you take?

I'll focus for now on actions that most people can take. In particular, anyone with a pension, which includes a large proportion of people who have a job or have previously had one.

Your pension savings are likely to be managed by professional investors. Depending on your pension arrangements, this could involve a pension fund, trustees, asset managers and other institutions.

Some of those institutions may be investing directly in AI companies. Others may be investing in large technology companies or businesses that provide the infrastructure on which AI development depends.

And you can ask them what they're doing about AI safety.

My main recommendation is to contact your pension scheme and ask.

You don't need to know exactly what your pension is invested in. You don't need to understand the technical details of AI safety. You can just ask whether they are considering the risks from increasingly powerful AI, and what they're doing about them.

Here's an example email:

Subject: AI safety and my pension investments

Hello,

I'm concerned about the risks from increasingly powerful AI systems, and I'd like to understand how these are being considered in the management of my pension.

Do you or your investment managers assess the AI safety practices of companies you're investing in?

I’m asking this not only because I want to live in a world free from the risks of AI, but because companies which develop safe, aligned AI will likely achieve a better return.

Are you engaging with relevant companies on issues such as independent safety evaluations, incident reporting and oversight of major AI development decisions?

If so, could you share some examples of what you're doing? If not, would you consider incorporating these issues into your investment and stewardship activities?

Many thanks.

You can make this more specific if you want, but there's no need to.

Practicalities - who do you address the letter/email to? Which address/email address should you use? 

  • You can resolve this by (a) finding any recent correspondence from the pension scheme (b) using any contact details you can find to get in touch with someone at the scheme (c) explain you have a question which you would like to pose to the trustees. This will likely get you the correct contact details.

If you get a vague response about the importance of responsible investment, you can ask for more detail. Are they actually engaging with companies? What are they asking for? How do they assess whether companies are making progress? Ultimately, you don't want your concern about AI safety to be bucketed in with "ESG" or "responsible investment". Many in the industry will treat ESG considerations as a nice to have, whereas if your query is prompts questions which are thought of as straightforward investment considerations, it will likely be higher in the pecking order.

Will this really make a difference?

I've seen pension schemes take action based on a single letter. As mentioned above, receiving letters from members is relatively rare.

If pension providers start receiving questions about AI safety, they have more reason to raise the issue with asset managers, who in turn have more reason to raise it with AI companies.

I'm not suggesting that emailing your pension provider will solve AI safety. But institutional investors already engage with companies on issues like climate change and human rights. There's no obvious reason why AI safety shouldn't be another priority. Especially since investors already perceive AI to be a very salient topic.

Given the scale of investment going into AI, and how little effort sending an email takes, it seems worth trying.

16

0
0

Reactions

0
0

More posts like this

Comments
No comments on this post yet.
Be the first to respond.
More from Sanjay
Curated and popular this week
Relevant opportunities