California SB 243 puts companion chatbot operators under new duties

At a glance

What this is: NHI Mgmt Group's analysis, drawing on Private AI's blog post: California SB 243 imposes disclosure, youth-safety and crisis-response obligations on companion chatbots, while retained conversation data remains a separate compliance exposure.

Why it matters: This matters because practitioners need to treat companion chatbot governance as both a user-safety issue and a data-governance issue, especially when minors or sensitive disclosures are involved.

Context

California SB 243 regulates a specific class of conversational systems: companion chatbots that sustain adaptive, human-like relationships across multiple interactions. The legal trigger is functional, not brand-based, so teams need to assess what the system does and how it behaves over time, not just how it is marketed.

The compliance problem is broader than a disclosure banner. If a chatbot remembers users, infers emotional state or retains sensitive disclosures, then safety obligations, privacy obligations and retention controls all intersect in the same operating model.

For practitioners, the starting assumption should be that a chatbot with relationship-building behaviour is not the same as a transactional assistant, even when both use the same underlying model.

Key questions

Q: How should teams classify companion chatbot systems for legal scope?

A: Teams should classify a system by what it does, not by what it is called. If it uses natural language to sustain an adaptive, human-like relationship across multiple interactions and meet a user’s social needs, it should be treated as a companion chatbot for scope analysis, even when it sits inside a broader product.

Q: What should teams do first when a companion chatbot may be in scope?

A: Teams should first classify the chatbot’s use case, data inputs and user group, then decide whether a disclosure, consent, age-gating or restricted deployment is required before launch. That review should cover prompts, conversation logs, retention, training use, and any personal data or sensitive data the system can collect or infer.

Q: Why do companion chatbots create privacy risk even when notices are present?

A: Privacy risk persists because a notice does not stop the chatbot from collecting, inferring, retaining or sharing personal data through prompts, logs, memory features and connected tools. The main failure modes are over-disclosure by users, data reuse for model training, weak retention controls, and disclosure that is too vague to shape informed consent.

Q: How do age-based controls work in minor-facing chatbot flows?

A: Age-based controls usually combine age verification or age estimation, policy rules, and step-up restrictions that limit what the chatbot can ask, store or generate for minors. They may block certain topics, disable profiling or memory, require guardian consent, and route higher-risk interactions to a safer experience or human review.

Technical breakdown

How SB 243 defines a companion chatbot

SB 243 uses a behavioural definition rather than a product label. A companion chatbot is a natural-language system that produces adaptive, human-like responses, shows anthropomorphic traits and can sustain a relationship across multiple interactions to meet social needs. That definition matters because it captures systems that are functionally companion-like even when they sit inside a broader product or serve multiple use cases. The key technical issue is persistence of context plus adaptive response shaping, which can make a bot feel socially continuous even when the infrastructure is stateless at the model layer.

Practical implication: Map products to the statute by observed behaviour and persistence, not by marketing category.

Why disclosure and crisis protocols change system design

The law requires disclosure when a reasonable person could think they are talking to a human, and it requires a crisis-prevention protocol for suicidal ideation and self-harm content. That pushes design beyond content moderation into detection, routing and notification logic. Evidence-based detection methods, referral pathways and public protocol publication all create auditable requirements, which means policy text alone is not enough. The architecture has to support prompts, classifiers, escalation destinations and reporting artefacts that are consistent with each other.

Practical implication: Embed crisis detection and referral logic into the interaction layer, then keep the supporting evidence and disclosures auditable.

Why retained transcripts create a separate compliance problem

Conversation logs from companion chatbots often contain names, health details, financial information and other sensitive disclosures because users treat the system like a confidant. SB 243 governs how the chatbot behaves, but it does not eliminate the risk created by stored transcripts, training data or analytics pipelines. That is where privacy engineering becomes decisive: data minimisation, de-identification and controlled retention reduce exposure that user-facing safety rules do not remove. The technical point is that the model and the data lake create different compliance surfaces. See the Generative AI Data Loss Prevention Guide.

Practical implication: Treat transcript storage and model-improvement pipelines as separate controls from the chatbot experience itself.

NHI Mgmt Group analysis

SB 243 makes companion chatbot governance a behavioural test, not a branding test. The law is aimed at systems that build ongoing social relationships, so product teams cannot rely on generic chatbot labels or customer-service exceptions. If the bot remembers users and adapts emotionally, the governance burden shifts from simple disclosure to lifecycle control over how the system behaves in context. That is a material change for product, legal and security teams alike.

Disclosure alone does not solve the core risk in companion chatbots. The law tells operators what users must see and how the system must respond in crisis scenarios, but it does not remove the privacy exposure created by retained transcripts. The defining gap is that a system can comply with user-facing notice rules while still accumulating highly sensitive data at scale. Practitioners should treat this as a dual-control problem, not a single compliance checkbox.

Companion chatbot programmes now sit at the intersection of AI governance and human safety obligations. That makes the control set broader than model safety or privacy engineering alone. The most exposed programmes are the ones that combine persistence, personal disclosures and youth access without strong event handling. Security and governance teams need to evaluate not just what the model says, but what the platform stores and how it escalates harm.

Privacy engineering becomes the operational control that converts legal intent into deployable practice. De-identification, minimisation and retention limits are what stop a compliance obligation from turning into long-lived data liability. Where a chatbot is designed to remember users, those controls are not optional hygiene, they are the boundary between supervised interaction and indefinite data accumulation. The practical conclusion is to govern companion chatbots as data systems as much as conversation systems.

What this signals

Companion chatbot governance now depends on event handling, not just content policy. The operational challenge is to make disclosures, crisis escalation and minor-specific routing behave consistently across every session a user has with the system. Teams that treat these as frontend copy issues will miss the control point, which sits in the conversation flow and the data pipeline.

Retention discipline becomes the deciding control when conversational AI stores sensitive disclosures. A companion chatbot can be compliant at the interface and still accumulate unnecessary personal data behind the scenes, so minimisation and de-identification need to be designed into storage and training workflows from the start. That is especially important where the same product serves adults, minors and regulated-use cases.

Privacy and safety obligations will continue to converge around relationship-style chatbots. The more a system remembers, personalises and persists context, the more likely regulators will expect both user-facing warnings and stronger back-end governance. Practitioners should assume the next compliance question will be about what the platform keeps, not just what it says.

For practitioners

• Classify products by companion behaviour Review whether the system sustains relationships across sessions, adapts to emotional state or presents as a confidant rather than a tool. Document scope decisions and exclusions based on observed behaviour, not product category.
• Build visible AI disclosure into the interface Place notice at the point of interaction so a reasonable user is not misled into thinking the chatbot is human. Keep the wording consistent across web, mobile and embedded experiences.
• Engineer crisis-response routing and evidence trails Create detection, escalation and referral workflows for self-harm content, then preserve the artefacts needed to prove the protocol exists and works. Make sure public protocol statements match the actual operating workflow.
• Apply age-specific controls to minor accounts Use account-level age verification or self-attestation, then route minors into break reminders and content restrictions by default. Do not rely on a single platform-wide age assumption for all users.
• De-identify retained conversation logs Strip names, health details, payment data and other identifiers before transcripts are stored, analysed or reused for model improvement. Keep the de-identification workflow inside your own environment where possible.

Key takeaways

• Companion chatbots are being regulated as a distinct behavioural class, which means scope depends on how the system interacts over time rather than on generic chatbot branding.
• The main compliance gap is not only disclosure and crisis handling, but also the privacy exposure created by retained conversation logs and sensitive user disclosures.
• Teams need to align interface notices, minor controls and data minimisation so that user-facing compliance and back-end governance do not diverge.

Related NHI Mgmt Group resources: Agentic AI Compliance Guide · Generative AI Data Loss Prevention Guide · EU AI Act Compliance Guide · Agentic AI Identity Risk Board Briefing

Key terms

• Companion Chatbot: A companion chatbot is a conversational AI system designed to sustain an ongoing, adaptive relationship with a user rather than simply answer isolated queries. The defining feature is social continuity across interactions, which makes scope, disclosure and data handling materially different from ordinary support chatbots.
• Artificial Intelligence Disclosure: Artificial intelligence disclosure is a statement that tells users when and how an AI system is involved in an interaction or decision. It usually identifies the system’s role, the fact that outputs may be machine-generated, and any meaningful limits on human oversight, data use or reliance.
• Crisis-Response Protocol: A crisis-response protocol is a documented set of steps for handling a serious operational or security incident. It defines who declares the crisis, how decisions escalate, which teams coordinate response, and how evidence, communications and recovery actions are controlled.
• De-Identification: De-identification is the broader process of reducing or removing identity signals from data so it cannot readily point back to an individual. It includes anonymization, pseudonymization, masking, and related controls, each with different levels of reversibility and residual risk.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity and secrets management. It helps security and governance practitioners align access, data handling and lifecycle controls across modern identity programmes.