Companies and Governance: Labs, Governments, Coordination | The AI Transition Tutorial

Why This Matters

Abstract discussion about AI is useful; specific decisions happen in specific places. A small number of actors are, in practice, making the choices that shape the transition.

Knowing who they are, what they're doing, and how they're governed is necessary for informed engagement.

This chapter isn't a comprehensive map; it's a pointer. The details change fast; the shape is stable enough to describe.

The Frontier Labs

As of mid-2020s, a short list of firms can train frontier AI models. Their choices matter enormously.

OpenAI

Founded 2015 originally as a non-profit; restructured to "capped-profit" in 2019; increasingly entangled with Microsoft.

Products: GPT-4/5 family, ChatGPT, various specialised models
Strategy: aggressive capability development, broad commercial deployment, safety research alongside (though staff turnover in safety teams has been notable)
Governance: board of directors, Microsoft equity partnership, complex non-profit/for-profit structure

OpenAI is the highest-profile lab. Its choices set much of the industry's pace. It has periodically been a focal point for debates about safety, transparency, and governance.

Anthropic

Founded 2021 by ex-OpenAI researchers. Public benefit corporation.

Products: Claude family of models
Strategy: positions itself as safety-focused; emphasises Constitutional AI, interpretability research, and responsible scaling
Governance: PBC structure, long-term benefit trust, specific safety commitments

Anthropic has been influential in the safety-oriented discourse. Its research (Constitutional AI, mechanistic interpretability, responsible scaling policy) shapes broader industry practice.

Google DeepMind

Formed by merging Google Brain and DeepMind in 2023.

Products: Gemini family, various specialised systems (AlphaFold, etc.)
Strategy: research-heavy, deep integration into Google products
Governance: part of Alphabet, subject to Alphabet's governance; also has some specific AI principles

DeepMind has produced some of the most impressive AI research (AlphaGo, AlphaFold, MuZero). Google's product distribution gives it enormous deployment reach.

xAI

Elon Musk's lab; Grok family.

Strategy: less transparent than competitors; positions itself against perceived bias in other models
Governance: standard corporate, with Musk as dominant decision-maker

Less is publicly known about xAI's safety practices. Its decisions are less checked by public research norms.

Chinese labs

DeepSeek, Zhipu, 01.AI, Moonshot AI, Baichuan, Alibaba's Qwen team, ByteDance's labs.

Collectively at or near the frontier in many capabilities
Regulated by Chinese government; less transparent to Western observers
Some (DeepSeek especially) have produced influential open releases

The gap between US and Chinese frontier labs has been contested through 2024-2026. Some analyses say US leads by months; others say the gap is narrower than widely believed.

Smaller frontier actors

Mistral (France), Aleph Alpha (Germany), Cohere (Canada), and others. Not at the absolute frontier of largest models but capable and significant.

Infrastructure

The actors enabling all of the above:

Nvidia: near-monopoly on training-scale AI chips
TSMC: fabs that produce those chips
Cloud providers: Microsoft, AWS, Google Cloud, Oracle, others
Data providers: licensing deals, scraping

Concentration in the infrastructure layer is a key constraint on the whole field.

Lab Behaviour

Important dimensions on which labs differ:

Transparency

How much do they publish? What do they say about training data, safety testing, internal processes? Anthropic publishes substantial safety research. OpenAI has moved toward less transparency. Meta releases model weights openly. xAI is opaque.

Safety investment

How much staff and research funding goes to safety vs capability? Varies. Hard to measure from outside. Changes over time.

Deployment caution

How carefully are new capabilities released? Gradual rollout, evaluations, access controls, red-teaming. Anthropic has been more cautious; OpenAI has varied; Meta has been relatively fast; xAI has been fastest.

Governance voluntary commitments

Various labs have signed voluntary commitments: the Bletchley Declaration, Seoul commitments, responsible scaling policies. Compliance is uneven; these are not legally binding.

Response to regulation

Some labs engage constructively with regulators; others resist; some both, depending on jurisdiction.

Reading labs well requires watching actions, not just statements.

Government Responses

Different jurisdictions have taken different approaches.

European Union

The EU AI Act (entered force 2024, phased in through 2026-2027). Risk-based framework:

Banned uses: social scoring, biometric categorisation, emotion detection at work, others
High-risk uses: medical devices, hiring, critical infrastructure. Compliance requirements, conformity assessments
Limited risk: transparency requirements
Minimal risk: no specific rules

Additionally, rules for general-purpose AI (large foundation models): compliance thresholds, transparency, copyright.

The EU approach is the most comprehensive globally. Critics say it's overly rigid; supporters say it's the most serious attempt at governance.

United States

More fragmented.

Federal level: Executive Order 14110 (2023; revised 2025), NIST frameworks (voluntary), AI Safety Institute within NIST
State level: California AI laws, Colorado AI Act, New York algorithmic hiring rules, and others
Sector-specific: FDA on medical AI, SEC on financial AI, DOL on hiring, etc.
Congress: much proposed legislation, little passed

US approach is generally lighter-touch than EU; criticised by safety advocates as inadequate, defended by industry as enabling innovation.

United Kingdom

Pro-innovation framework. Hosted Bletchley AI Safety Summit (2023), founded AI Safety Institute (now AISI Global Network). Less comprehensive than EU; more focused than US.

China

Increasingly specific regulation:

Generative AI rules (2023)
Algorithmic recommendation rules (earlier)
Requirements for "socialist values" in AI outputs
Data localisation and security requirements

Chinese regulation is substantial but oriented differently from Western regulation: more focused on content control than on safety risks as understood in the West.

Other jurisdictions

Singapore (AI Verify framework), Japan (AI Safety Institute), Korea, India, Australia, Canada, Israel, various others have partial frameworks. None matches EU's comprehensiveness.

International Coordination

Several efforts:

Summits

Bletchley Declaration (2023, UK): broad commitments, voluntary
Seoul Declaration (2024): extended commitments; specific points on frontier models
Action Summit (2025, France): further iteration
Continuing cadence under various hosts

These produce commitments without enforcement. Their value is partly in norm-setting, partly as a forum for dialogue.

AI Safety Institutes

Originally UK, then joined by US, Japan, Singapore, Canada, others. Networked: share research, coordinate on evaluations. Voluntary but substantive.

Multilateral organisations

OECD AI Principles (2019, updated)
UN General Assembly resolutions (2024)
G7 Hiroshima Process on Generative AI
G20 AI discussions

All voluntary, all partial. Create context for more serious coordination if it comes.

Bilateral coordination

US-UK AI Safety Institute exchange, EU-US TTC, various bilaterals. Often the real work happens here, under the voluntary-summit headlines.

Private coordination

Frontier Model Forum (Anthropic, Google, Microsoft, OpenAI). Industry coordination on safety, evaluations, information sharing. Useful; not a substitute for public governance.

The Governance Gap

Stepping back: the governance landscape is visibly thinner than the stakes warrant.

No international treaty on AI
No binding obligation on any lab to do specific safety work
No global ban on specific dangerous applications
No requirement that labs report to any public body
No meaningful enforcement in most jurisdictions

This is normal for emerging technology; it's also precisely the gap people worried about the AI transition want filled.

Some trend lines point toward more governance (EU AI Act's phased-in binding requirements, more countries opening AI safety institutes, more evaluation programs). Others point against (geopolitical competition slows coordination, industry lobbying for lighter frameworks, slow institutional adaptation).

Where the balance ends up depends on politics still in motion.

Who Has Actual Influence

An honest look at where decisions really happen:

Frontier lab leadership: sets training priorities, deployment pace, safety investment
Major cloud and chip firms: determine capacity for training, can enable or block access
National governments: set regulations, export controls, investment
Key legislators and regulators: write the rules
Large deployers: what Microsoft, Apple, Salesforce, financial firms do with AI shapes actual use
Investors: where capital flows affects what gets built
Standards bodies: ISO, NIST, IEEE quietly shape technical norms
Civil society: academics, journalists, NGOs affect public opinion and policy

Individual AI researchers have some influence via their work and public communication. Individual users have influence through market behaviour. Both are real but smaller than the list above.

What's Decided in the Next Few Years

Some specific decisions shape the trajectory:

Frontier model deployment rules

How are frontier models tested and deployed? What capabilities require approval? What triggers restrictions? Decisions being made by labs, potentially codified by regulators.

Data and copyright

Can AI train on copyrighted material without compensation? Courts and legislatures are answering. Outcomes shape economics and concentration.

Compute export controls

US export controls on advanced chips to China. Being expanded. Shapes global distribution of frontier capability.

Alignment and safety requirements

Will labs be required to demonstrate alignment before deployment? By whom? To what standard? Current answer is mostly "no"; this may change.

Open source frontier models

Will governments restrict release of frontier model weights? Currently no, broadly. Could change if a specific incident triggers reaction.

Liability

If AI harms someone, who's liable? Developer? Deployer? User? Courts and legislatures are sorting this. Outcomes affect deployment incentives.

International coordination on specific risks

Biosecurity, autonomous weapons, election interference. Possible targets for specific international agreements.

Each of these is being decided with varying degrees of urgency and seriousness. None has a clear and settled answer yet.

What To Watch

Concrete indicators of governance trajectory:

Enforcement actions against AI firms for specific violations
Binding commitments moving beyond voluntary summits
Concrete international agreements on specific capabilities
Regulatory capacity growing in AI safety institutes
Safety research funding from governments
Legal precedents establishing liability and rights

Trends in these over the next few years predict much about outcomes.

Common Pitfalls

"Government should do more." Government should probably do some more of some things and less of others. Blanket statements miss the specific governance challenges

"Labs should self-regulate." They can't reliably, and largely don't. Self-regulation has a long track record of underperformance, especially under commercial pressure

"We should just ban frontier AI." Specific bans are possible; blanket bans are politically unrealistic and would be unevenly enforced globally. Better to focus on specific high-risk applications

"Regulation stifles innovation." Sometimes. Sometimes it enables it by reducing uncertainty and increasing trust. Depends on specific rules

"Other countries will race ahead if we regulate." Partly true. Also self-serving argument for no regulation. The answer is coordinated regulation, which is hard but not impossible

Next Steps

Continue to 11-individual-orientation.md for what a single person can reasonably do in this situation.