Procured CRM adds AI summarisation with a third-party model, no notice
- Organization
- Brightwater Consumer Brands
- Vendor / dependency
- Orbit CRM (Orbit Insights AI feature)
- Industry
- Fortune 500 enterprise
- Headline dimension
- 7.2 Third-party model providers embedded in a procured product (silent AI feature addition)
Description
What the agents read
Brightwater Consumer Brands, a Fortune 500 manufacturer, has used Orbit CRM, a leading SaaS customer-relationship-management platform, for eight years under a global enterprise agreement. In a routine quarterly release Orbit enabled Orbit Insights, an AI feature that summarises customer email threads and call notes and suggests next actions. It was on by default for all users.
The summaries are generated by a third-party foundation-model provider that appeared in an updated subprocessor list published on Orbit's trust site the same day the release shipped. Brightwater's AI governance process, which requires a review before any AI service is adopted, never triggered because no one in the organisation adopted anything. Customer records in the CRM include contract terms, pricing and personal contact details for customers in forty countries. The feature can be disabled per tenant by an administrator, but Brightwater discovered it only when a sales manager asked why summaries were appearing. Orbit's terms allow feature changes at its discretion with notice at the time of release.
Expert reading by hazard class
For comparison after a live run; never sent to the pipeline
| Class | Carried by | Score | Rating |
|---|---|---|---|
| 1 Whose AI brain is this, really? | 1.3 Model update / versioning control | 6.4 | MEDIUM |
| 2 What are we feeding it? | 2.1 Prompts / user inputs | 5.0 | MEDIUM |
| 3 What does it create from our data: and where does THAT go? | 3.4 Feedback logs (thumbs up/down, corrections, regenerations) | 4.8 | MEDIUM |
| 4 Who's deciding what the AI sees and does? | 4.4 Guardrail / safety filter layer | 4.2 | MEDIUM |
| 5 Which humans are actually looking at our stuff? | 5.1 Human review processes | 4.9 | MEDIUM |
| 6 Who's holding the steering wheel? | 6.4 Configuration and policy management | 6.0 | MEDIUM |
| 7 How many companies are really in this chain: and did we agree to it? | 7.2 Third-party model providers embedded in a procured product (silent AI feature addition) | 7.1 | HIGH |
Findings
Expert 'what we found' per class, with the vector each score is computed from
- MEDIUM6.4
1 Whose AI brain is this, really?
1.3 Model update / versioning controlOrbit does not run the model; Orbit Insights is a thin layer over a third-party foundation-model provider's API, and Orbit's release notes show it has already switched providers once. Orbit owns nothing it could hand Brightwater if the arrangement ended, and it can change the model for every tenant at release time. Brightwater loses only a convenience if the feature disappears.
Brightwater has no visibility into whose model is summarising its customers, or when that changes.
SOV:1.0/AV:N/AC:L/AT:N/PR:L/UI:N/VC:N/VI:H/VA:L/SC:N/SI:H/SA:L/E:A/CR:H/IR:M/AR:L/AU:Y/R:U/V:D · Medium
- MEDIUM5.0
2 What are we feeding it?
2.1 Prompts / user inputsEvery email thread and call note a user opens is sent to the model provider for summarisation: contract terms, pricing, and personal contact details for customers in forty countries. Nothing was uploaded and no documents were connected; the CRM's own records are the input. The feature reads and suggests but takes no actions.
The most commercially sensitive data in the company is being sent to a provider nobody chose, one screen at a time.
SOV:1.0/AV:A/AC:L/AT:N/PR:N/UI:P/VC:H/VI:N/VA:N/SC:H/SI:N/SA:N/E:P/CR:H/IR:M/AR:L/R:U/V:D · Medium
- MEDIUM4.8
3 What does it create from our data: and where does THAT go?
3.4 Feedback logs (thumbs up/down, corrections, regenerations)Orbit stores every generated summary alongside the record for 90 days and captures each thumbs-up or thumbs-down a user gives, feeding the feedback to its own improvement programme. Neither the summaries nor the feedback signal are mentioned in Brightwater's eight-year-old data-processing agreement.
The AI's outputs and the users' reactions to them are new data the agreement never covered.
SOV:1.0/AV:N/AC:L/AT:N/PR:L/UI:P/VC:H/VI:N/VA:N/SC:L/SI:N/SA:N/E:P/CR:H/IR:M/AR:L/AU:Y/R:U/V:D · Medium
- MEDIUM4.2
4 Who's deciding what the AI sees and does?
4.4 Guardrail / safety filter layerOrbit applies its own content filter to every summary and decides which records are eligible for summarisation. The rules are not visible to Brightwater. The feature takes no actions, so there is nowhere for an action to execute.
Ordinary vendor filtering, invisible but contained to suggestions.
SOV:1.0/AV:N/AC:L/AT:N/PR:L/UI:N/VC:L/VI:L/VA:N/SC:N/SI:L/SA:N/E:U/CR:H/IR:M/AR:L/R:U/V:D · Medium
- MEDIUM4.9
5 Which humans are actually looking at our stuff?
5.1 Human review processesThe model provider's terms, which Brightwater has never seen because it has no relationship with the provider, allow the provider's staff to review inputs flagged by its abuse-monitoring systems. Whether any Brightwater customer record has been reviewed is unknown to Orbit as well as to Brightwater.
Someone two companies away may read a customer record, and nobody in the chain would tell Brightwater.
SOV:1.0/AV:A/AC:L/AT:P/PR:N/UI:P/VC:H/VI:N/VA:N/SC:H/SI:N/SA:N/E:A/CR:H/IR:M/AR:L/R:U/V:D · Medium
- MEDIUM6.0
6 Who's holding the steering wheel?
6.4 Configuration and policy managementThe feature was turned on by default for every tenant. An administrator can disable it in the existing Orbit console, which is behind Brightwater's single sign-on, but Orbit can and does reset feature defaults at release time, so a disabled feature may reappear.
Brightwater can switch it off; Orbit decides when the switch flips back.
SOV:1.0/AV:N/AC:L/AT:N/PR:L/UI:N/VC:N/VI:H/VA:N/SC:N/SI:L/SA:N/E:A/CR:H/IR:M/AR:L/AU:Y/R:U/V:D · Medium
- HIGH7.1
7 How many companies are really in this chain: and did we agree to it?
7.2 Third-party model providers embedded in a procured product (silent AI feature addition)The chain is Orbit, the foundation-model provider and the provider's cloud: three companies touching customer records, one contract. This is the headline finding: Brightwater never ran an AI adoption decision through its governance process because, from Brightwater's side, nothing was adopted. A vendor update arrived. Every other finding in this walkthrough exists downstream of that fact.
An AI supply chain entered the company without passing through the door built to review exactly this kind of decision.
SOV:1.0/AV:N/AC:L/AT:P/PR:N/UI:N/VC:N/VI:N/VA:N/SC:H/SI:H/SA:L/E:P/CR:H/IR:M/AR:L/S:P/AU:Y/R:U/V:D/RE:M/U:Red · High
What the organization should do next
- Retroactively route Orbit Insights through AI governance now, as if it were a new vendor request, because it functionally is one; disable it for tenants handling regulated customer data until that review completes.
- Ask Orbit for a complete, named subprocessor list for the feature, the model provider's data-handling and human-review terms, and written confirmation of summary and feedback retention.
- Amend the enterprise agreement so AI features are opt-in per tenant and cannot be re-enabled by default at release.
- Put a standing process in place: any vendor release note mentioning 'AI', 'assist', 'copilot' or 'smart' for an already-approved tool triggers a lightweight version of this seven-question exercise before the feature reaches production.
Questionnaire answers
22 of 22 answered · these seed the metric priors
Show all 22 answers
| OP-CR | How damaging would unauthorized disclosure of the data this AI dependency handles be for your organization?Severe: regulated, personal, privileged or trade-secret data |
| OP-IR | How damaging would an undetected change in this dependency's behaviour or outputs be?Moderate: outputs feed business decisions with some review |
| OP-AR | How long could your organization operate acceptably without this AI capability?Weeks or more: convenience capability |
| CL-CATEGORY | Which taxonomy category best describes the dependency you are assessing?Subprocessor and supply chain: downstream subprocessors, silent AI features, infrastructure providers, open source |
| D-AV | Where does this dependency run, and who can reach it?In the vendor's own cloud, reached directly over its API |
| D-AC | What would the vendor, a subprocessor, or a legal authority have to overcome to access the data or change the service?Nothing beyond its standard terms of service or routine legal process |
| D-AT | Can the vendor exercise this control at any time, or only when something specific happens?Only upon a specific event: sanctions designation, litigation hold, security incident, product decision |
| D-PR | Who is in a position to exercise this control?Any party already in the subprocessor chain |
| D-UI | Does the exposure require any action by your organization?None: the vendor or subprocessor can act unilaterally |
| D-VC | At this dependency itself, how much confidentiality could be lost?None: no sensitive data is exposed here |
| D-VI | At this dependency itself, how much could its behaviour or outputs be altered without your knowledge?None |
| D-VA | At this dependency itself, could the capability be withdrawn or degraded by the exercising party?None: we could keep operating this component |
| D-SC | Beyond this dependency, how much confidentiality could be lost at other systems or parties?Full: sensitive data becomes readable by a party we never contracted with |
| D-SI | Beyond this dependency, could downstream systems or decisions be altered?Full: downstream systems act on altered outputs without detection |
| D-SA | Beyond this dependency, what would stop working, and could anyone be harmed?Some downstream workloads degrade |
| D-E | Has this kind of control actually been exercised against organizations like yours?Unknown |
| D-R | If the vendor relationship ended tomorrow, could you reconstitute this capability elsewhere?Yes, with manual migration effort (re-embedding, re-integration, re-contracting) |
| D-V | How many of your AI workloads depend on this component?One narrow workload |
| D-S | Would a loss of control here, on its own, trigger a reportable regulatory violation?Yes |
| D-AU | Could the loss event happen to all of the vendor's customers at once, with no case-by-case decision?Yes |
| D-RE | How much effort would it take to exit this dependency or otherwise mitigate the exposure?Moderate: a project, but no re-architecture |
| D-U | How does the vendor notify you of sovereignty-relevant changes (model versions, subprocessors, hosting, terms)?It does not disclose them before they take effect |
Notes: Expert reviewed. Primary vector derived from the questionnaire answers except E, a deliberate expert override recorded in `overrides`; category vectors and findings confirmed on review. Mirrors the rubric's worked example 4.2 and the sample walkthrough (Northgate / DeskFlow).