🤖AI Transparency Statement: This will be flagged as partially or mostly AI written. The role profile section was initially drafted by me, went through editing loops with Claude Opus where I ended up replacing a good bit of its generated text. I used Claude Opus to create the initial sample job description section using my inputs, notes and about 20 industry job descriptions I pulled. I edited the descriptions lightly then expanded with my own success section - I feel like this is a good use of AI writing since job descriptions are usually very mechanical. Job description analysis was completely authored by me. Once I had the complete paper, I did a couple rounds of adversarial reviews and grammatical polish with Claude.
Experienced Builders Role Profile
Overview
This experienced builder role profile expands on the mid-career and generalist operator profiles already shared in 2026. It describes a family of senior roles spanning product, program, engineering, data and operations leadership that owns what gets built, for whom, in what order, and they are accountable to outcomes. They have 10+ years of experience in industry where they have built functions, products or organizations from zero, and carry the tacit knowledge that only comes from navigating difficult paths to success and experiencing setbacks along the way.
Gaps Observed in AI Safety
While these role profiles are standardized in industry and at the frontier AI labs, independent AI safety orgs have yet to adopt them.
Inferred from their job postings, Anthropic and OpenAI‘s operating models reflect the rest of tech where they staff matrixed positions that serve as the connective tissue to the research teams to promote working on the highest impact items, while also performing the strategy and planning work of how to integrate the research into their infrastructure, products and outputs like model cards.
The gap is a mapping problem: AI safety organizations are aware of the symptoms they’re facing, but aren’t creating org structures, roles and job descriptions to address them that would be familiar to someone entering from outside of AI safety. The job never gets posted, candidates don’t see the opportunity, and both sides walk away concluding the other one was not there.
For orgs to scale their impact, they not only need more headcount, including operators, but experienced builders that can help build matrixed orgs, prioritization and accountability frameworks, and infrastructure and tooling to allow them to work efficiently, diffuse that work, and have it adopted across AI safety.
Experienced Builders Role Profile
Given a vague problem, no process and no precedent, an experienced builder runs discovery, generates a plan, aligns it, sees it through to implementation and adoption, and measures if it worked. Ownership covers what gets built, for whom, in what order, and then are accountable to outcomes (not just the outputs), which separates them from someone executing fixed requirements. They support building the machine that produces the output, but don’t write the code or perform the research themselves. With enough technical fluency to earn credibility with researchers, they hold their own on strategy, and coordinate across multiple disciplines and functions to create an aligned plan. Ruthless prioritization is the norm, where they use standard prioritization and organizational frameworks to direct efforts including stopping work already underway, which is rarer and harder than knowing what to start.
An experienced builder’s background shows a track record demonstrating building from zero, designing processes, creating tooling and measurements, and managing teams. On their resume you’ll find them driving high-stakes initiatives in large complex organizations, making tough decisions, and defending them. When you talk to them you’ll learn how they navigated fragmented tooling, lack of shared standards, work repeated across silos, and no meta-analysis being performed. This allows them to pattern match on those past experiences to deftly navigate challenging scenarios and avoid common pitfalls.
The profile spans responsibility levels without changing shape, from owning one core function, to several, to overseeing the people who own multiple functions.
Difference from Generalist Operations Roles
The generalist operations roles that AI safety has already started building out are very valuable, but they are a different job. An operator works inside a system that already exists, driving goals someone else has set and producing previously agreed upon outputs. They are the oil that keeps the organizational machine running smoothly, and support executing how the machine is built.
In contrast, the experienced builder defines what machine needs to be built, and why. They define the strategy, and oversee tough prioritization decisions, and are ultimately held accountable to the outcomes, not just that the output was created. The work starts a step earlier than a generalist operational brief usually does, at the point where the requirements do not exist yet, and the scoping is the work.
The Value Add
An organization that hires this experienced builder role profile shifts tooling design, procedural loads and adoption efforts to those with background and expertise in driving them, leaving researchers to spend more of their week performing research. Output rises in volume and in consistency because the standards live in tooling that’s been democratized and adopted, resulting in faster and more defensible decisions and research. With each person working within their core competencies, burnout is reduced. The organization then gains the ability to absorb additional money and headcount without breaking, which matters in a field where capital is arriving faster than the capacity to coordinate it.
Sample Experienced Builder Job Description
Background on this section: I thought it would be helpful to write a sample job description containing the keywords that would be familiar to me. I’ve chosen an evals ‘product’ that represents something I’ve thought about, but a builder could post under many other role titles with differing responsibilities, and different applications.
What’s different about this role: Today, AI safety orgs thinking about this type of work may see it purely as tech-led infrastructure or tooling engineering and administration, missing the tight loops of gathering user requirements and validating against them, or driving adoption of tooling and process.
Product Manager, Evals Tooling Infrastructure
About the Role
Several of our teams run evaluations of frontier models, with others researching advanced evaluation methods. Most of them start from the same open-source tools, and over time each team has adapted those tools in its own way.
That works for any single project. Across the organization it creates friction. Results from two teams are hard to compare, and past work is hard to find or query, so meta-analysis across evaluations rarely happens. When one team develops a promising technique, or a new method shows up in the field, getting a second team or organizational adoption is high effort.
This role owns that problem. You’ll work with the researchers who design evaluations, the engineers who build and run them, and the downstream users that consume the results to figure out what would help most. The answer might be shared infrastructure, common standards, or better use of the tools teams already have, and it will likely change as the field does. Researchers set the research direction. You own making the work around it faster, more consistent and easier to build on, and making sure what gets built is actually used.
Key Responsibilities
- Run discovery across teams: interview researchers, engineers, research managers, and operational leads to observe how evaluations are set up, run, analyzed and reported today, and turn what you learn into clear problem statements and prioritized requirements.
- Own the strategy and roadmap for evaluation tooling and infrastructure, including the calls on what to build, what to adopt from the wider ecosystem, what to standardize, and what to stop maintaining. Comfortable presenting to and collaborating with leadership.
- Partner with engineering to refine requirements, scope work down, ship quickly, and validate from usage and feedback.
- Drive adoption of shared tooling and standardized processes across teams through onboarding, documentation, and migration paths off bespoke setups.
- Develop shared standards with research leads, such as common data structures, metric definitions and reporting formats, so results can be compared across teams and queried later for meta-analysis.
- Build the path for new techniques to spread: when internal or external research produces a promising technique, establish a pipeline into the tooling and workflow to validate it and decide if it should become fully integrated and adopted.
- Define how success is measured (time to set up and analyze an evaluation, share of work on shared tooling, reuse of past results) and report on it honestly, including when something isn’t working.
- Perform external analysis and work closely with others in the ecosystem, such as other evaluation orgs and open-source tool maintainers, to avoid duplicated effort and contribute improvements back for everyone to use.
Qualifications
- 10+ years in industry, 5+ years in product management, working closely with engineering teams to build and ship technical products, ideally internal platforms, developer tools, experimentation or data infrastructure.
- A track record of taking an ambiguous problem with no established process to a shipped solution that people adopted, and measuring whether it worked.
- Strong discovery skills with highly technical users. You can interview researchers and engineers, reconcile conflicting needs, and say no when it matters.
- Enough technical fluency to discuss tradeoffs credibly with researchers and engineers, and a working understanding of how AI evaluations are built and run, or the ability to get there quickly.
- Experience driving adoption and behavior change across teams you don’t manage, including standardizing work that people previously did their own way.
- Clear written communication and a real commitment to reducing catastrophic risks from advanced AI.
What Success Looks Like
- In 3 months: Leaders and stakeholders are excited about the newly defined user-centric strategy and roadmap. Their requests are captured consistently with transparent prioritization against the backlog, which is already shipping the first quick wins.
- In 6 months: Early measures of success show reduced time spent on evals as the first larger initiatives complete and early adoption pilots prove the work.
- In 12 months: Volume of evals has increased while the total time spent creating, running, analyzing and reporting on them has decreased. This is because the evals infrastructure growth and socialization of its success has contributed to broad adoption across the org. There is UX and other work to support standard practices, making it easier to produce trustworthy results, and allowing researchers to delegate more tasks. The roadmap for the next 12 months shows how maturing the platform by closing key pain points will continue this momentum and be worth the continued investment.
Current AI Safety Job Description Analysis
Disclaimers: This is a very small sample from the 80,000 Hours job board. You can find deeper analysis on postings in posts like this. The purpose of this section is not to diminish the importance of these roles, or to imply that hiring orgs have done anything incorrectly. I’d also acknowledge that in small orgs roles tend to be very fluid, so it’s very reasonable that a builder may still support covering off some of these tasks.
Analysis Summary
TL;DR – This sample included operationalists that keep orgs running smoothly, and technical counterparts that implement the systems, but lack the builder to coordinate and prioritize the most impactful work.
I analyzed 4 recent job descriptions: 2x operations, 1x chief of staff and 1x tech lead which I found to be representative of the public postings I frequently review.
I observed that the operational roles don’t directly own what gets built to solve the broader organizational goals, prioritization of those efforts and defining the measures of success. They focus on supporting the organization to be its best, and take administrative tasks off leaders and other team members. They include tasks like creating new employee onboarding, defining policies, ensuring adoption of tools or processes and applying a project management style of planning and reporting. A builder works alongside this role to understand where the existing infrastructure and tooling wasn’t meeting the organization’s needs, continuously backlogging and re-prioritizing a product feature list.
The tech lead manager owns implementation details, organizing their team, and ensuring the health of their infrastructure and tooling. While unreliable tech would be a drag on the organization, it doesn’t define what to build next to meet the org’s next biggest needs. To build a unified roadmap, the builder brings user and org needs, the tech lead owns the technical side, and they work together to resolve tradeoffs.
While working with both of these roles, the builder is also constantly talking to users, other stakeholders, leaders and analyzing trends in industry to build a complete view of all the challenges, trends and potential solution options.
In this matrixed approach, each role gets to leverage their strengths, rather than being asked to cover a broader scope. They work together very closely with the builder as the connective tissue. The outcome is arriving at the right solutions, faster, with more incremental progress along the way by taking time to more deeply understand the full picture before creating the next thing.
If you want to see the more detailed analysis I did on this section, you can view it on the linkpost. I couldn't get EA Forums editor to collapse this section and wanted to limit the default length.
Counterpoint
How can we trust these senior builders will assimilate into AI safety’s unique culture?
As Sam Smith put it in an earlier post, “scrappiness is a means, not an end”. AI safety has done well to scale to this point, but also is now cognizant of the growing pains of scaling.
I think:
- Organizationally:
- We have to meet somewhere in the middle here. Something has to give. Change is inevitable when growing and evolving.
- The frontier orgs AI safety is trying to keep pace with already utilize this operational model, which mirrors tech and industry at large.
- Many senior builders are generalists, an industry term meaning they’ve traversed many companies, orgs and teams in their career’s journey. To be effective, it’s non-optional to first assimilate to an org before trying to enact change. They’re skilled in quickly gaining context to speak the language, understand the challenges, and begin building a new path forward. I wrote another entire article to address this common fear I’ve heard.
Conclusion
I believe that maturing role types and org structures is a critical component to scaling the impact of AI safety’s work. I hope this can become a starting point for discussion with fieldbuilding and AI safety organizations. The goal is that this role becomes recognized, shows up in job descriptions, and hired for allowing experienced builders to fully leverage their abilities and for AI safety organizations to scale their headcounts and impact.
Asks: Tell me whether you recognize the need inside your own program or organization, what makes you hesitant or what you see as incorrect in this assessment. Even better, let’s schedule time to discuss the opportunity, needs and how this could be built out.
I’d like to thank Sam Smith for reviewing this work with me, and supporting fieldbuilding for mid-senior folks like myself, including through other posts like Good Senior People Exist and We’re Wasting Them.
Thanks for putting this out Paul, excited for hiring managers to think more about folks with product management backgrounds and how they can have the most impact!