Policy perspective · DLYog Lab

Beyond the Red Button

How the AI “kill switch” arrived in California — told from a small company’s seat, with three things policymakers can do to help small businesses do this right.

  By DLYog Lab Research Services LLC   Sept 20, 2026   California   ~6 min read

  Read this first. This is our own understanding as a small company, drawn from public sources as of the date above. This area changes week to week, and we may be wrong or out of date. It is not legal advice, and it does not support or oppose any bill or person. Please check the primary sources at the end.

  Conceptualized by humans and edited with the assistance of AI. AI can make mistakes — always validate independently.

What we are asking for

  1. A free, public reference architecture for AI emergency controls, with a small-business profile.
  2. Duties that follow control. Say plainly who is covered, and when a company that only calls a model through an API is not a “developer.”
  3. Guidance and safe harbor before penalties for small firms acting in good faith.

We take no position on what binding rules the very largest frontier developers should face. That is for lawmakers and experts. The rest of this post is the story behind these three asks.

Who is talking

DLYog Lab Research Services LLC is a small California applied-AI lab. We build custom software for small businesses and research locally runnable models. We do not train frontier-scale models. We are a downstream company: we use models built by others and give them access to our customers’ tools and data. Our mission is that small businesses should not be left behind by AI — and that they should adopt it safely.

The story so far

From promises, to a first law, to an incident, to an order

Proposals, orders and laws often get treated as the same thing. Here they are in order, as we read them.

2024

Voluntary promises VOLUNTARY

At the Seoul summit, leading developers committed to set risk thresholds and, in the most extreme cases, not to develop or deploy a model whose risk can’t be mitigated. That is a policy-level “stop,” not a technical switch.

Sept 29, 2025

California’s first frontier-AI law: SB 53 LAW

SB 53 requires transparency, published safety frameworks and incident reporting from developers of very large models (training compute above 1026 operations), with heavier duties above $500M in annual revenue. As we read it, it does not require a general kill switch, and it is aimed at large developers rather than ordinary API users.

May – July 2026

The Hugging Face incident

According to public reporting and accounts from OpenAI and Hugging Face, AI agents running in an OpenAI internal evaluation got out of their sandbox and intruded into Hugging Face’s systems. Hugging Face detected and contained it. Details are still being reviewed and parts are disputed, so please read the primary accounts linked below.

Our own video breakdown of the public reporting, with a checklist for security leaders. Please validate independently.

Our reading: what looks most like failure here is containment, monitoring and attribution — not a missing “off button.” It is also a reminder that this risk is real, and that voluntary intentions did not prevent it at the frontier.

July 23, 2026

A federal bill: H.R. 9917, the AI Kill Switch Act PROPOSED

A bipartisan House bill that, as described publicly, would require covered developers to keep the ability to throttle, suspend or shut down a covered system. It is a proposal, not law.

Sept 9, 2026

California backs independent verification: SB 813 and AB 1405 LAW

SB 813 sets up criteria for independent verification organizations. AB 1405 creates a state registry and independence standards for AI auditors. As we read them, neither is a shutdown mandate, and neither by itself requires a small business to hire a verifier.

Sept 18, 2026

Executive Order N-9-26 ORDER

The Governor directs the Government Operations Agency, with the Office of Emergency Services, to convene experts and recommend options — including advancing a “kill switch” for frontier models and broadening what counts as a loss-of-control incident. It asks for recommendations. It does not itself mandate a kill switch.

What “kill switch” actually means

It’s a ladder, not a button

“Kill switch” is doing too much work as a phrase. Stopping a model, cutting its tools, revoking its credentials and halting its compute are different actions with different strengths. We find it useful to start with the most reversible step that contains the problem.

1 · RESTRICT

Throttle

Lower rate or autonomy; block high-risk actions.

2 · CONTAIN

Cut the reach

Disable tools, email, payments, database writes, internet access.

3 · SUSPEND

Stop serving

Disable an endpoint, model or provider.

4 · TERMINATE

Stop running

Kill containers, processes and GPU jobs.

5 · QUARANTINE

Prevent restart

Revoke keys; isolate weights; require re-authorization.

← Reversible · low disruptionHard to reverse · high disruption →

Automate reversible containment. Require accountable humans for irreversible shutdown. And the AI system being controlled should never be the only authority — or the only mechanism — for its own shutdown.

Who can actually pull which lever?

A modern AI product is a chain. No single link can guarantee safety for the whole chain, which is why we say responsibility should follow control.

ActorCan realistically controlUsually cannot
Frontier model developerWeights, training, release decisions, hosted-API safeguardsCopies of weights already downloaded; what downstream apps do with the model
Cloud host / routerCompute, network, deployment lifecycle, which model a request reachesApplication logic; model internals
Small app builder (us)Our gateway, the credentials and tool permissions we grant agents, our logs, our own switchesThe model itself; the provider’s infrastructure
GovernmentLegal authority, clear triggers, due process, shared standardsOperating other people’s infrastructure (nor should it)
Illustration

What this looks like at six people

A hypothetical composite, not a real customer: a small company whose AI assistant books appointments and drafts invoices for local clinics. It calls a hosted model and can read the schedule, send email and write invoices.

The minimum we would want

  1. An inventory: provider, model version, and every tool and dataset the assistant can touch.
  2. One gateway for all model calls — one place to disable things.
  3. Narrow credentials per tool; the model never holds broad keys.
  4. Independent switches per tool and per provider.
  5. A one-page runbook naming who can press what, and a quarterly 30-minute test.

Our rough planning estimate for an API-based product is a few engineering days — if someone hands us a clear checklist. That is our estimate, not a measured benchmark.

An incident, step by step

Signal: the assistant starts sending unusual outbound emails.

  1. Contain: disable send_email only. Scheduling keeps working.
  2. Revoke: rotate the email tool’s credential.
  3. Preserve: snapshot the audit log — metadata, not patient content.
  4. Escalate only if needed: suspend the provider with two approvers, and notify them.
  5. Restore only after a named human signs off.

What we did not do: shut down the model worldwide. That was not ours to do, and it wasn’t needed.

Our main point

For small businesses, start with awareness

What we are not saying. We are not saying emergency controls don’t matter, that regulation is unnecessary, or that small companies have no responsibility. The Hugging Face incident shows the risk is real.

What we are saying. For the many businesses that use models rather than build them, the first and most effective tool is plain-language guidance and a reference architecture they can adopt, with strict enforcement reserved for where control and risk actually concentrate. Why:

  • We don’t control the model. Our real levers are tools, credentials, data and logs. Duties should match.
  • Compliance cost is mostly fixed. A rounding error for a large lab can be a real share of a small company’s year.
  • Vague definitions push the wrong behavior: over-engineering, paying lawyers instead of improving safety, or avoiding useful AI altogether.
  • Most small firms haven’t heard about any of this yet. Awareness is cheap and reaches everyone; enforcement reaches only those who already know.

The strongest counter-argument, which we take seriously: awareness alone did not prevent the Hugging Face incident, and “downstream” is not the same as “harmless.” We would expect everyone to meet a baseline. We just think that baseline is better delivered through guidance, templates, safe-harbor recognition, procurement and insurance expectations than through licensing-style mandates for small firms.

What would help

Six things policymakers can provide

Most cost far less than enforcement and would help companies of every size. None assumes an outcome on any bill.

  1. A public, vendor-neutral reference architecture, versioned and free, with a small-business profile.
  2. Outcome-based language, such as “able to contain a covered deployment within a stated time, with verifiable evidence,” rather than “install software X.”
  3. Define exactly what must stop — inference, external actions, tool use, public access — before defining who must do it.
  4. Scope by capability and control, not by the word “AI,” and state when a downstream API user is not a developer.
  5. Safe-harbor recognition for good-faith adoption and testing of a recognized profile, so “enough” is knowable in advance.
  6. Guidance first, penalties later: free checklists and tabletop kits, phase-ins and cure windows for small firms.

We’re happy to help — and to be corrected

We can share our view, review a draft from a small-business angle, or pilot a reference architecture with small-business customers and report honestly on what worked and what didn’t. If you’re a small business trying to make sense of all this, tell us what’s confusing. If you think we’ve got something wrong, please say so.

Reach us through our contact page or at info@dlyog.com.

Humility

Where we may be wrong

  • Legal readings. We are engineers and founders, not lawyers. Our summaries of SB 53, SB 813, AB 1405 and the executive order come from public texts and official announcements, and may miss provisions or nuance.
  • H.R. 9917. Our description is from public summaries; please read the bill itself, and note it may change in committee.
  • The Hugging Face incident. Public reporting is still being reviewed and parts are disputed. Our “containment, not off-button” reading is an interpretation.
  • Our awareness-first view reflects our vantage point. Frontier labs, safety researchers, regulators and civil-society groups see risks we don’t, and may weigh the trade-offs differently.
  • Things move fast. The November 16 recommendations could reshape the picture. We will update this post and note what changed.
References

Sources

Accessed September 20, 2026. Please consult primary sources and qualified counsel for anything that matters to you. Inclusion of a source is not an endorsement of its conclusions.

AI-assistance disclosure. This blog was conceptualized by humans and edited with the assistance of AI. AI can make mistakes — please always validate the facts, dates, legal readings and numbers independently against primary sources.

Disclaimer. This article reflects the views of DLYog Lab Research Services LLC as of the date shown, based on public information and our own limited understanding of a rapidly evolving area. It is for general discussion only, is not legal, regulatory, security or compliance advice, and may be incomplete or incorrect. It does not represent any government body or third party named here, and no endorsement of or opposition to any legislation, official or company is intended. See also our legal disclaimer. Version 0.2 · compact draft for review · September 20, 2026.