Companion to the AI Sovereignty Risk Assessment Framework · CASE assessment eb752834-8389-43d0-97cf-3fd5498fdb2f # Is There a Hidden AI Supply Chain in Your Vendor? *A plain-language walkthrough for GRC and compliance auditors, no technical background required* **Organization:** Halden & Marsh LLP **Vendor / dependency assessed:** GateKeep AI Gateway **Industry:** Generic business *Generated by the CASE assessment pipeline on 2026-10-28. Metric values were assigned by retrieval-grounded agents and scored deterministically; they have not been reviewed by a human expert.* ## Before You Start This is not a technical document. It answers one practical question a compliance auditor is increasingly asked: “We use a lot of software. Some of it quietly grew an AI feature. What do we actually need to worry about, and how bad is it?” It walks through one vendor from start to finish using the same seven questions every time. A small italicised note under each section (“Technical cross-reference”) points back to the technical rubric for anyone who wants to trace a rating to the formal scoring model; you can ignore those notes and still get full value. ### The Rating Scale, In Plain English | Rating | Meaning | |---|---| | **RATING: LOW** | Normal and contained. If something went wrong here, you'd find out quickly and fix it without much trouble. | | **RATING: MEDIUM** | Could cause a real problem, and you might not find out right away, worth tracking, not yet an emergency. | | **RATING: HIGH** | Could cause serious damage, a data leak, a regulatory violation, or losing the ability to operate, and by the time you noticed, it might already be too late to undo. | ## The Scenario > Halden & Marsh LLP, a 900-person professional-services firm, routes all of its generative-AI usage through GateKeep, a single AI gateway product from a specialist vendor. The gateway fronts three model providers, enforces the firm's usage policies, holds the API keys for every provider, applies the firm's data-loss-prevention rules and provides the audit log the firm relies on for client-confidentiality assurances. > > Administrators configure routing rules, policies and key rotation in GateKeep's hosted console. The console and its configuration store are hosted in a jurisdiction different from the one the firm chose for inference, a fact the firm discovered when reviewing GateKeep's subprocessor list. GateKeep can change console behaviour, default policies and supported providers for all customers at once, and has done so twice in the last year with release notes published on the day of change. If the console were unavailable or its configuration altered, every AI workload in the firm would be affected simultaneously; the firm has no offline copy of its routing and policy configuration and rebuilding it with another gateway would take a significant project. ## Walking Through It, Category by Category ### 1. Whose AI brain is this, really? Every AI feature is powered by some underlying model, hosted somewhere, controlled by someone. That someone is very often not the software vendor you actually have a contract with. **Ask the vendor:** - *Whose AI model is actually answering our questions, yours, or one you're renting from someone else?* - *If we stopped paying you tomorrow, could we take this AI capability somewhere else, or does it just disappear?* - *Can the AI's behavior change overnight without you telling us?* **What we found:** Halden & Marsh has no contract with any of the three model providers that actually produce its AI output; GateKeep contracts them, holds all the provider keys and decides which providers are supported. GateKeep can change the supported providers and the default policies for every customer at once, and has done so twice in the past year, publishing release notes only on the day each change took effect. Either GateKeep or one of the model providers can therefore change the behaviour of the AI the firm's lawyers rely on, with no advance notice, no approval step for the firm and no ability to stay on a known version. The firm has no way to re-test output quality before a change is live, and every drafting and review workload inherits the change at the same moment. > **RATING: HIGH**, The firm cannot tell in advance when the AI underneath its work product will start behaving differently, so any assurance it gives clients about consistent, validated AI behaviour is unsupported. *Technical cross-reference: dimensions 1.3 (Model update / versioning control), 1.1 (Model hosting location / jurisdiction), Org Action Required (UI), Trigger Conditions (AT), Primary Dependency Impact -- Integrity (VI), Silent/Automatic Trigger (AU) in the technical rubric. CASE vector `SOV:1.0/AV:A/AC:L/AT:N/PR:N/UI:N/VC:L/VI:H/VA:L/SC:L/SI:H/SA:L/E:A/CR:M/IR:H/AR:H/S:N/AU:Y/R:U/V:C/RE:H/U:Amber`, score 8.2 (High).* ### 2. What are we feeding it? Once an AI feature exists, the next question is simple: what of ours does it actually see? Employee inputs, uploaded reference material, and anything the AI can reach and act on all count. **Ask the vendor:** - *What do our employees actually type into this thing, and does it include customer or confidential information?* - *Did we upload or connect any of our own internal documents for it to search?* - *Can this AI take real actions, look things up, change things, or does it only talk?* **What we found:** Every prompt typed by any of the firm's 900 staff passes through GateKeep's hosted service in readable form, because reading prompt content is how the gateway applies the firm's data-loss-prevention rules and writes the audit log. The prompts are then forwarded to three model providers the firm does not contract with, and the console and configuration store that govern this handling sit in a jurisdiction the firm did not select for inference. Nothing technical or contractual stops GateKeep, its hosting subprocessor, or legal process in that jurisdiction from reaching prompt content; the firm holds no keys of its own and has no residency protection over this tier. GateKeep can also loosen a redaction rule or re-point traffic to a different provider for all customers at once, and the firm would learn of it only on the day it happened. > **RATING: HIGH**, Client-confidential and potentially privileged material entered as prompts is fully visible to the vendor and its chain, and the controls meant to redact it can be changed without the firm's agreement. *Technical cross-reference: dimension 2.1 (Prompts / user inputs), Access Path (AV), Compulsion Complexity (AC) in the technical rubric. CASE vector `SOV:1.0/AV:N/AC:L/AT:N/PR:N/UI:N/VC:L/VI:L/VA:H/SC:L/SI:H/SA:H/E:P/CR:M/IR:H/AR:H/S:N/AU:Y/R:U/V:C/RE:H/U:Amber`, score 8.3 (High).* ### 3. What does it create from our data: and where does THAT go? This is the part almost every vendor review misses entirely. AI systems don't just read your data, they transform it into new things: mathematical fingerprints of your documents, logs of every answer given, records of which answers got a thumbs-down. Those new things get stored somewhere, often by a company you've never heard of and never contracted with directly. **Ask the vendor:** - *Does this turn our documents into some kind of derived file (an “embedding”) so it can search them faster?* - *If so, who stores that derived file, you, or a completely different company?* - *Do you keep a copy of everything the AI said back to us, a log of how we used it, or a record of when we said the answer was wrong?* **What we found:** The audit log the firm relies on as evidence for its client-confidentiality assurances is created, formatted, stored and retained entirely by GateKeep, in the console's out-of-jurisdiction configuration store. The firm keeps no independent copy and no parallel record, so it cannot verify the log, detect a change to what is logged, or reconstruct a period if retention is shortened or access is withdrawn. In aggregate the log is itself sensitive: it shows which matters are active, for which clients, by whom and at what tempo. Unlike the gateway service, which could be rebuilt elsewhere with effort, a lost or altered history of past events cannot be recreated by any replacement vendor. > **RATING: HIGH**, The firm's own compliance evidence is held and controlled by the party it is meant to hold accountable, and it is not recoverable if it is changed, truncated or cut off. *Technical cross-reference: dimensions 3.5 (Telemetry / usage logs), 3.3 (Model outputs / completions), Primary Dependency Impact -- Confidentiality (VC), Blast-Radius Concentration (V) in the technical rubric. CASE vector `SOV:1.0/AV:A/AC:L/AT:N/PR:N/UI:P/VC:H/VI:H/VA:H/SC:L/SI:L/SA:L/E:P/CR:M/IR:H/AR:H/S:N/AU:Y/R:I/V:C/RE:H/U:Amber`, score 7.5 (High).* ### 4. Who's deciding what the AI sees and does? Between “the model” and “what we see” sits a set of decisions: which document gets pulled up, whether a filter blocks an answer, where an action actually executes. Each of those decisions can be made by yet another company. **Ask the vendor:** - *Who decides which of our documents actually get shown to the AI when it answers a question?* - *Who runs the safety filter that sees every question and answer, and can we see its rules?* - *If the AI takes an action on our behalf, where does that action actually happen, in your systems, ours, or a third location?* **What we found:** GateKeep's filtering layer inspects every prompt and every completion in full, before the firm's own logging sees it, which is how the firm's data-loss-prevention and usage rules are enforced. That means the single point with complete visibility of client-confidential content is vendor-operated, administered from a console in another jurisdiction, and reachable under ordinary terms of service. GateKeep can change the default policies behind that layer for all customers at once without the firm acting or agreeing, as it has already done twice. If the rules are silently loosened, confidential content flows out un-redacted; if they are tightened too far, or the layer is unavailable, the firm has no fallback path because it holds none of the provider keys itself. > **RATING: HIGH**, The control the firm points to as proof that client confidentiality is protected is owned, configured and changeable by the vendor, and the firm cannot independently confirm what it is doing. *Technical cross-reference: dimensions 4.4 (Guardrail / safety filter layer), 4.2 (Orchestration / agent framework), Org Action Required (UI), Primary Dependency Impact -- Confidentiality (VC), Regulatory Consequence (S) in the technical rubric. CASE vector `SOV:1.0/AV:N/AC:L/AT:N/PR:L/UI:N/VC:H/VI:H/VA:H/SC:L/SI:H/SA:L/E:A/CR:M/IR:H/AR:H/S:P/AU:Y/R:U/V:C/RE:H/U:Amber`, score 9.0 (Critical).* ### 5. Which humans are actually looking at our stuff? Not every risk is a computer. Somewhere in most AI products, an actual person, often a contractor, often outsourced, reads flagged conversations or debugs a problem using our data. That's a direct human set of eyes, independent of every technical safeguard above. **Ask the vendor:** - *Does a real person at your company, or a contractor of yours, ever read our conversations or our data?* - *If a conversation gets flagged, does it go somewhere else to be reviewed or labeled?* - *When your support engineers debug a problem for us, what can they see?* **What we found:** No human review, annotation or labelling of the firm's content by vendor staff is described, so this is the narrowest of the seven exposures. What remains is routine support and troubleshooting access: GateKeep's own engineers can reach the hosted console, the configuration store that holds the API keys for all three providers, and the audit and trace logs, under nothing more than standard terms of service. There is no access-approval gate, no per-request justification requirement and no screened-personnel commitment described, so such access does not depend on the firm raising a ticket. The scenario does not state that this access actually occurs or how often, so this finding reflects an unconstrained possibility rather than an observed practice. > **RATING: HIGH**, Vendor support staff can, if they choose, read the firm's provider credentials and its complete AI usage record without asking, and the firm would have no record that they did. *Technical cross-reference: dimension 5.3 (Support / debugging access), Compulsion Complexity (AC), Primary Dependency Impact -- Confidentiality (VC) in the technical rubric. CASE vector `SOV:1.0/AV:N/AC:L/AT:N/PR:L/UI:N/VC:H/VI:L/VA:L/SC:L/SI:L/SA:N/E:P/CR:M/IR:H/AR:H/S:N/AU:N/R:U/V:C/RE:M/U:Amber`, score 7.4 (High).* ### 6. Who's holding the steering wheel? Somewhere, someone configures this thing: turns features on and off, sets filters, manages who has access, holds the keys. That control panel is itself a dependency, and it doesn't always sit where you'd expect. **Ask the vendor:** - *Where does the screen we'd use to configure or turn off this AI feature actually live?* - *Is our employee login system, or our credentials to other systems, now tied into a system this vendor controls?* - *Can settings change on us without anyone telling us?* **What we found:** GateKeep's hosted console is the only place the firm can set routing, usage policy, data-loss-prevention rules and key rotation, and that console and its configuration store sit in a jurisdiction different from the one the firm chose for inference, a fact the firm found only by reading the subprocessor list. The firm's inference residency choice gives it no protection over the place where its controls are actually configured. GateKeep holds a standing right to change console behaviour and defaults for every customer simultaneously, exercised twice in the past year with same-day notice, so the settings the firm believes are in force can be redefined without warning. The firm holds no offline copy of its routing and policy configuration, so it cannot detect drift against a baseline, cannot restore a known-good state, and if the console were unavailable or altered every AI workload in the firm would stop or change at once; moving to another gateway is a project, not a failover. > **RATING: HIGH**, The firm has no independent ability to configure, verify, restore or continue operating its AI controls if the vendor changes them or the console goes away. *Technical cross-reference: dimensions 6.1 (Centralized control plane / admin console), 6.4 (Configuration and policy management), 6.3 (API gateway / rate limiting infrastructure), Access Path (AV), Blast-Radius Concentration (V) in the technical rubric. CASE vector `SOV:1.0/AV:N/AC:L/AT:N/PR:L/UI:N/VC:L/VI:H/VA:H/SC:L/SI:H/SA:H/E:A/CR:M/IR:H/AR:H/S:N/AU:Y/R:U/V:C/RE:H/U:Amber`, score 8.3 (High).* ### 7. How many companies are really in this chain: and did we agree to it? Add it all up. This is the question that actually matters most, and it's the one almost no vendor security review asks directly: how many companies now sit between us and our own data, and which of those did we ever actually agree to? **Ask the vendor:** - *List every company standing between us and the raw computer running this AI. How long is that list, really?* - *Did we ever affirmatively choose to adopt AI here, or did it just arrive?* **What we found:** The real chain behind the firm's AI is Halden & Marsh to GateKeep, then to three model providers, then to their hosting infrastructure, plus a separate host for the console and configuration store in a third jurisdiction. The firm has a contract with only the first link, and discovered the jurisdiction mismatch only by going to look at the subprocessor list. GateKeep can change which providers are supported for all customers at once, meaning the composition of this chain can be altered at the vendor's discretion with notice on the day of effect. Because the out-of-jurisdiction configuration store holds the API keys for every provider and the enforcement rules, a problem at that single hop exposes the firm's accounts at all three providers and can switch off data-loss prevention for everyone at the same time. > **RATING: HIGH**, Most of the parties handling the firm's AI traffic were never agreed to by the firm, their number can change without its consent, and exposure at any of them lands on the firm's clients. *Technical cross-reference: dimensions 7.1 (Downstream subprocessors), 7.3 (Infrastructure providers beneath the AI vendor), Compulsion Complexity (AC), Trigger Conditions (AT), Position Required (PR) in the technical rubric. CASE vector `SOV:1.0/AV:A/AC:L/AT:N/PR:N/UI:N/VC:L/VI:H/VA:H/SC:H/SI:H/SA:H/E:A/CR:M/IR:H/AR:H/S:P/AU:Y/R:U/V:C/RE:H/U:Amber`, score 9.3 (Critical).* ## The Rollup Put side by side, here is what one pass through all seven categories produced for GateKeep AI Gateway: | Category | Rating | Carried by | |---|---|---| | 1. Whose AI brain is this, really? | **HIGH** | 1.3 Model update / versioning control | | 2. What are we feeding it? | **HIGH** | 2.1 Prompts / user inputs | | 3. What does it create from our data: and where does THAT go? | **HIGH** | 3.5 Telemetry / usage logs | | 4. Who's deciding what the AI sees and does? | **HIGH** | 4.4 Guardrail / safety filter layer | | 5. Which humans are actually looking at our stuff? | **HIGH** | 5.3 Support / debugging access | | 6. Who's holding the steering wheel? | **HIGH** ◆ | 6.1 Centralized control plane / admin console | | 7. How many companies are really in this chain: and did we agree to it? | **HIGH** | 7.1 Downstream subprocessors | ◆ Headline finding: category 6, dimension 6.1 (Centralized control plane / admin console), CASE score 8.3 High [manual exit (R:U); concentrated (V:C); high exit effort (RE:H); silent/automatic trigger (AU:Y); concurrent vendor notice (U:Amber)]. ## What Halden & Marsh LLP Should Actually Do Next - Halden & Marsh should immediately require GateKeep, in writing, to give advance notice of any change to supported providers, default policies, console behaviour or logging, with a defined notice period and a documented option to stay on current behaviour while the firm re-validates. - Halden & Marsh should begin exporting its own copy of the audit log and routing telemetry on a continuous schedule into firm-controlled storage, so its client-confidentiality evidence survives any retention change, access withdrawal or vendor outage. - Halden & Marsh should take and regularly refresh an offline copy of its complete routing rules, usage policies, data-loss-prevention configuration and key inventory, so it has a baseline to detect silent change against and a trusted state to rebuild from. - Halden & Marsh should contract the three model providers directly, or at minimum hold its own provider API keys, so it retains a working path to AI capability and to its own accounts if GateKeep's console is unavailable or the relationship ends. - Halden & Marsh should obtain a binding commitment that the console and configuration store, not just inference, are hosted in the firm's chosen jurisdiction, and a written list of every subprocessor with a change-notification and objection right. - Halden & Marsh should require GateKeep to constrain support and debugging access through per-request justification, customer-visible access logging and screened personnel, and should insist on evidence that such access is reviewable by the firm. - Halden & Marsh should implement its own independent checks on prompts and outputs - sensitive-data and secret redaction, output review and a human approval step for client-facing work product - so confidentiality does not rest solely on a vendor-configured filter. - Halden & Marsh should test the failure case before it happens: run a documented exercise covering console unavailability, a silent policy change and a provider substitution, and scope the cost and timeline of migrating to a second gateway or direct provider access. ## Running This Yourself The seven questions above don't change from vendor to vendor, only the answers do. Any time your organization is told (or, more often, not told) that a piece of already-approved software just grew an AI feature, work through the same seven categories, in the same order, and rate what you find on the same three-colour scale. The scenario in this document exists to show you what that looks like done once, start to finish, the method itself travels.