Step 8 Secure: Secrets, Dependencies, Access and Personal Data

A tool can pass every test of Step 7 and still be unsafe, because the tests ask whether it does what the specification says and not whether it does anything else. This chapter treats Step 8 Secure, the step in which the builder makes the tool safe at the level its tier requires. It corresponds to the second half of the developer function D4 Prove (secure-code and supply-chain checks) and rests on D3 Build, because most of the problems arise when the code is written, and on D5 Steward, because somebody must hold the keys and answer for the data. Step 7 Prove ended in a verification plan; Step 8 Secure ends in a security checklist and a provenance record, and Step 9 Ship uses both.

The chapter is defensive. It describes the classes of problem that generated applications show, by mechanism, so that a builder can recognise them and put a control in their way. It contains no exploit code, no payloads and no injection strings, and none is needed to do the work. Three ideas run through it. The tool should hold as little as it can: no secrets in the code, no more data than the purpose needs. It should trust as little as it can: nothing it reads from outside, whether a file, a web page or a package name, is taken at face value. And whatever the builder checked, the builder records, with a date.

The security pass runs in a fixed order and ends in a checklist and a provenance record

Figure 12.1: The security pass of Step 8 Secure. The tier decides how much of it a named developer must review.

What a developer does at this step

A developer works from three habits, which the standards of the field state in more formal words. The first is secure by default: the safe setting is the one that applies unless somebody changes it on purpose. The second is least privilege: each person, program and key can do what its job needs and nothing more. The third is minimising data: the software holds the data it needs for its purpose and no more, for no longer than necessary. The NIST Secure Software Development Framework collects such practices for software producers in four groups (prepare the organisation, protect the software, produce well-secured software, respond to vulnerabilities) (137, Souppaya et al., 2022), and a community profile extends it to generative AI (138, Booth et al., 2024). The framework addresses producers, not builders, so what follows is this book's translation of it for a builder.

In practice the developer's habits become a short list of questions asked of every build. Where is each secret kept? Who can reach the data, and how does the tool know who is asking? What does the tool take in from outside, and what does it do with it? Which packages did it add, and does each one exist, belong to the project it claims, and have a maintainer? What personal data does it touch, and where does that data end up, in the database, the logs or the output? The OWASP Top 10 for web applications, the best known list of the most serious classes of application security risk, puts broken access control first and security misconfiguration second in its 2025 edition, and has a new category for software supply chain failures (146, OWASP, 2025). The list is a list of classes, not of tools, and the classes below are drawn from it and from the studies cited.

What vibe coding does instead

Vibe coding reasons from the fact that the tool works. It runs, it shows the right output, and so it is taken to be safe. The reasoning is wrong because security failures are failures to refuse: the tool does what was asked, and also does what a stranger asks. Nothing in a demonstration shows that.

The evidence on generated code points the same way, with the usual cautions about studies. Researchers who prompted GitHub Copilot in 89 scenarios built around common weaknesses and analysed 1,689 generated programs found approximately 40 per cent of them vulnerable (133, Pearce et al., 2021). The study concerns one early product and tasks chosen by the researchers. A vendor report on code from more than 100 large language models, in Java, JavaScript, Python and C#, stated that AI-generated code introduced risky security flaws in 45 per cent of its tests (134, Veracode, 2025); it is vendor research, and the page does not give the per-language figures this chapter would need. On the user side, one study found that programmers with an assistant wrote significantly less secure code and were more confident in it (23, Perry et al., 2023), while another, of low-level C programming, found at most about 10 per cent more critical bugs and called the effect small (139, Sandoval et al., 2023).

The closest evidence for a builder is a 2026 preprint on applications built by conversation with Claude Code and Lovable. The authors collected 9,041 open-source applications and audited 200 publicly deployed ones, finding 1,186 vulnerabilities, with 91.0 per cent of the audited applications carrying at least one and 65.77 per cent of the vulnerabilities rated critical or high, clustered in broken access control, injection and authentication failures (109, Deng et al., 2026). It has not been peer reviewed, and the 200 applications are a sample of those that are public, which are not a sample of all builds. A record in the National Vulnerability Database describes an insufficient row-level security policy in a generator of web applications that, through a given date in April 2025, allowed unauthenticated remote attackers to read or write arbitrary tables of the generated sites; the record is tagged as disputed, with the supplier stating that each customer is responsible for protecting its application's data (144, NIST, 2025). The pattern is older than AI: a 2021 report by a security vendor found 38 million records exposed across portals built with a low-code product, because table permissions were off by default, a behaviour the producer first called by design and later changed for new portals (143, UpGuard, 2021).

These sources do not show that generated code is always insecure, and none of them measures builders in European public authorities. They show that the classes of failure are well known, recur in tools built by people who are not developers, and are not visible when the tool runs.

How AI helps and where it misleads

AI helps at this step in four ways. It explains a class of problem in plain language when asked. It reviews a file against a list ("look for secrets, missing access checks and unvalidated input") and finds some of what is there. It writes the boring defensive code (reading a secret from the environment, validating a field, writing a log line without personal data), which a builder would not write unprompted. And it drafts the security checklist and the provenance record.

It misleads in three ways. First, an assistant may tell the builder that the code is secure. That sentence is a text generated in response to a prompt; it is not a check, and the independence rule of Chapter 11 applies: whatever verifies the security of the code must not be the thing that generated it. Second, an assistant tends to solve the problem in front of it. If a secret is missing and the tool does not run, the quickest fix is to paste the secret into the code, and an assistant that is given the secret in the conversation will do so. Third, an assistant that is allowed to install packages installs what the code needs, including packages that do not exist. Each of these is addressed below by a control that does not depend on the assistant's judgement.

Secrets and access

A secret is any value that proves identity or grants access: a password, a token, an API key, a certificate, a connection string with credentials. The rule is that a secret is never in the repository, never in the conversation with the assistant, and never in a log. The repository's history keeps every version of every file, so a secret committed once and deleted still lives in the history and in every copy made since. The scale of the problem in public code is documented by a vendor that sells secret detection: its 2025 report counted 23,770,171 new hardcoded secrets added to public GitHub repositories in 2024, 25 per cent more than the year before, and found that 70 per cent of the secrets leaked in 2022 were still valid when analysed (141, GitGuardian, 2025). The same page says that public repositories using Copilot had a 6.4 per cent secret leakage rate; the page states no selection method for that subset and no sample size, so the figure is reported here and not relied on.

Where does the secret live, then? In the platform's secret store, which hands the value to the program at run time as an environment variable, a named setting that exists only while the program runs. Both organisations in this book provide one: the Region's builder workspace and Hansa Mutual's pipeline. The code reads the variable and fails with a clear message if it is absent:

import logging
import os

log = logging.getLogger("spcheck")


def wiki_token() -> str:
    """The token comes from the platform's secret store, as an environment variable."""
    token = os.environ.get("WIKI_TOKEN")
    if not token:
        raise SystemExit("WIKI_TOKEN is not set; the platform injects it at run time")
    return token


def log_claim(claim_id: str, status: str) -> None:
    """Log the claim id and the outcome only: no names, no free text."""
    log.info("claim=%s status=%s", claim_id, status)

Two controls make this a habit rather than a hope. The first is a secret scan that runs before every commit and in the pipeline. The configuration below uses the pre-commit pattern, in which a small program is run on the files about to be committed and refuses the commit if it fails; the script it calls is shown after it. A real organisation will have a better scanner in its pipeline, and the builder uses that. The point is to have the check where the assistant cannot turn it off.

repos:
  - repo: local
    hooks:
      - id: scan-secrets
        name: Block secret-like assignments
        entry: python scripts/scan_secrets.py
        language: system
        types: [text]

"""Fail the commit if a file assigns a long literal to a secret-like name."""
import re
import sys
from pathlib import Path

PATTERN = re.compile(
    r"""(?i)\b(\w*(token|secret|password|api_?key)\w*)\s*[:=]\s*['"][^'"\s]{12,}['"]"""
)


def scan(path: Path) -> list[str]:
    hits = []
    for number, line in enumerate(path.read_text(errors="ignore").splitlines(), 1):
        if PATTERN.search(line):
            hits.append(f"{path}:{number}: assignment to a secret-like name")
    return hits


if __name__ == "__main__":
    problems = [h for name in sys.argv[1:] for h in scan(Path(name))]
    print("\n".join(problems))
    sys.exit(1 if problems else 0)

The script was run on a file containing an assignment of that kind and exits with status 1, naming the file and line; it exits with 0 on the tool's own source. It catches the commonest shape and no other: a value that is not assigned to a name that looks like a secret passes. That is the limit of a static check, stated in Chapter 11, and the reason the second control exists. The second is the rule that the assistant's mandate excludes secrets: it may not read them, and the builder never pastes one into the conversation. The mandates of both builds say so.

The leaked-key procedure. If a secret has reached the repository, a log, a chat or a page, treat it as public from that moment. In order: revoke or rotate the secret at the system that issued it (deleting the file does not help); tell the owner of that system, and the tier's named developer, within the hour; look at that system's access log for use of the secret between the leak and the revocation; and, if the secret protected personal data, tell the data protection officer at once. Under Article 33(1) of the GDPR a controller must notify the competent supervisory authority of a personal data breach without undue delay and, where feasible, not later than 72 hours after having become aware of it, unless the breach is unlikely to result in a risk to the rights and freedoms of natural persons (4, European Union, 2016). The clock belongs to the organisation as controller and runs from its awareness, so the builder's first duty is to tell the people who can start it. The builder writes the event in the build record, with the dates.

Access. The second half of this section concerns who may use the tool and what the tool may reach. Broken access control is the first item of the OWASP list (146, OWASP, 2025): the tool shows or changes something to a person who should not be able to. For a builder the working rules are four. Start from "nobody", and add people by name or by group. Check access on the side that holds the data, because a screen that hides a button does not protect the data behind it. Give the tool's own identity the narrowest right that works: a read-only identity for a tool that only reports. And never turn on a "public" or "anyone with the link" setting to get a demonstration working. A data store that is open to the world is the commonest and most expensive of these failures, and the cases in the previous section show how it arises: a default that nobody looked at.

Dependencies and provenance

A dependency is a package of code, written by others, that the tool uses; it is installed by name from a public registry. The listing of dependencies is the file requirements.txt in Python projects. Two classes of problem arise. Some packages are unmaintained, outdated or compromised. Others do not exist, and that is a feature of how assistants write code. Researchers who generated 576,000 code samples in two languages with 16 models found that at least 5.2 per cent of the package names suggested by commercial models and 21.7 per cent of those suggested by open-source models on average did not exist, and collected 205,474 unique hallucinated names (113, Spracklen et al., 2025). The risk follows directly: someone can register the missing name and wait. In one experiment a researcher registered an empty package under a name that models had hallucinated and counted more than 30,000 genuine downloads over three months; his account reports no malicious use (145, Lanyado, 2024). The OWASP list of risks for applications that use language models includes supply chain as its own item (135, OWASP, 2025), and the 2025 web list expanded its older "vulnerable and outdated components" into software supply chain failures (146, OWASP, 2025).

The builder's control is a dependency policy: a file that lists the packages the tool is allowed to use, with a reason and a date, and a check that fails when the requirements name anything else.

policy_version: 1
rule: every package in requirements.txt must be listed here with a reason
allowed:
  - name: pyyaml
    purpose: read rules.yaml and the oracle cases
    checked: 2031-10-27
    checked_by: Tomasz Wrona
  - name: pytest
    purpose: run the tests
    checked: 2031-10-27
    checked_by: Tomasz Wrona

"""Fail if requirements.txt names a package the policy file does not allow."""
import re
import sys
from pathlib import Path

import yaml


def names(requirements: str) -> set[str]:
    found = set()
    for line in requirements.splitlines():
        line = line.split("#")[0].strip()
        if line:
            found.add(re.split(r"[=<>~!\[; ]", line, maxsplit=1)[0].lower())
    return found


def main(requirements="requirements.txt", policy="dependency-policy.yaml") -> int:
    allowed = {p["name"].lower() for p in yaml.safe_load(Path(policy).read_text())["allowed"]}
    unknown = sorted(names(Path(requirements).read_text()) - allowed)
    for name in unknown:
        print(f"not in dependency policy: {name}")
    return 1 if unknown else 0


if __name__ == "__main__":
    sys.exit(main(*sys.argv[1:]))

With a requirements.txt that names pyyaml and pytest, the check passes. When a line naming an invented package is added (the name used in the test was made up for it), it prints not in dependency policy: followed by the name and exits with status 1. The check does not know whether a package is good. It forces a person to look at each new name, which is the control. For each new package the person confirms that the name is spelled exactly, that the package appears in the registry with the project it claims to be, that it is used by others and has a maintainer, and that it is pinned to a version. A package the builder cannot confirm goes to a developer. These are the questions of the SLSA specification, which describes levels of supply chain assurance for software artefacts (136, OpenSSF, n.d.), in plain form; a Tier C tool does not aim at a level, and a developer can say which one the pipeline gives.

The provenance record (PRV-*) documents where the pieces came from. It says which assistant and mandate produced the code, which dependencies were added and who checked them, what scan was run and when, and what personal data the tool touches. A machine-readable bill of materials is the heavier cousin of this record, and one open standard for it is CycloneDX (142, Ecma, 2025); a builder's PRV-* is a one-page version and a developer can generate the formal one from it.

Input and output

Input is anything the tool receives that it did not write: a file, a spreadsheet, a web form, a message, the contents of a web page, the reply of another system. The principle is that input is data, never instructions. A tool that builds a database query by pasting a user's text into it, or a file path by joining a name from a form, or a page by pasting a text into it, lets the text act on the system. The class is called injection; the control is to keep the data and the command separate (the database library's parameter mechanism, a fixed list of permitted file names, an output step that escapes what it prints) and to check that each field has the shape the specification says it has. The assistant can write those checks and usually does when asked. A builder asks for them by name: "validate every field against the specification and reject anything else, and show me the tests for rejection". The tests then go into the suite of Step 7, and the oracle owner writes at least one case of the kind that must be refused.

Output carries the same rule in reverse. Whatever the tool writes into a page, a message or a draft is read by someone else, so it is escaped or constrained to the format the reader expects. Claims-doc-check, for instance, produces a draft request text that a handler reviews and sends from the claims core; the tool never sends anything itself, which is a never-do in its specification and removes a whole class of problems by design.

Prompt injection applies when the tool itself calls a language model: to summarise a document, to classify a message, to draft a reply. The model receives instructions and data in the same stream of text, so text in the data that reads like an instruction may be followed. When the text arrives from a source the tool's user did not write, such as a web page or an email, the class is called indirect prompt injection, and the 2023 paper that described it demonstrated it against real applications that integrate language models (140, Greshake et al., 2023). OWASP lists prompt injection first among the risks for such applications and, as a separate item, excessive agency, meaning a model-based component that has more tools and rights than its job needs (135, OWASP, 2025). The controls are by class, and no string is needed to apply them. Give the model-calling part no rights beyond reading what it needs. Never let its output trigger an action, such as sending, deleting or paying, without a person's approval. Treat its output as untrusted input to the next step. Keep secrets, and data the user is not entitled to see, out of what the model receives. Log what was sent and returned, without personal data. Neither build in this book calls a model at run time: the assistant helped to write the code, and the tool itself runs as ordinary code. A tool that does call a model is a tier higher than its other properties suggest, and the builder asks a developer to review it.

Personal data

Personal data is any information about an identified or identifiable person: a name, an address, a claim number that can be linked to a claimant. The GDPR states the principles of its handling in Article 5: data are to be limited to what is necessary for the purposes (data minimisation, Article 5(1)(c)), kept in identifiable form no longer than necessary (storage limitation, Article 5(1)(e)) and processed with appropriate security (integrity and confidentiality, Article 5(1)(f)). Article 25 requires the controller to implement appropriate technical and organisational measures designed to put such principles into effect, such as data minimisation, and to ensure that by default only the personal data necessary for each specific purpose are processed, which concerns the amount collected, the extent of processing, the period of storage and the accessibility. Article 32 requires measures suited to the risk and names pseudonymisation and encryption among the examples (4, European Union, 2016). The European Data Protection Board has issued guidelines on Article 25, whose final version is dated 20 October 2020 (17, EDPB, 2020). The organisation, not the builder, is the controller. The architectural consequence for a builder is that the organisation's duties become the builder's habits, and the statements here are not legal advice.

The working rules are five. Minimise: ask for the columns the tool needs, not the whole export, and say in the specification which fields it reads. Keep it out of development: the repository, the tests and the conversation with the assistant hold synthetic data only, so a mistake does not copy real people into places where they should not be. Mask: where a tool must show an identifier, show the identifier and not the name. Log without personal data: logs are kept for a long time and read by many people, so they carry identifiers, counts and outcomes. A claim identifier can be linked to a person through the claims core, so it remains personal data in the sense of the GDPR, held in a smaller circle with narrower rights than a name would be; the builder still treats the log as restricted. Retain: say how long output files live and who deletes them.

Finally, screen with the data protection officer before a tool touches real personal data. The officer decides whether the processing is within existing records, whether a data protection impact assessment is required (Article 35 requires one where processing is likely to result in a high risk to the rights and freedoms of natural persons), and what conditions apply. The screening is attached to the build record and its date and result go in the provenance record.

Proportion: what each tier needs

The checklist is cumulative: a higher tier does what the lower tier does and adds to it.

Control Tier A personal Tier B team Tier C business Tier D project
No secrets in repository, conversation or log Required Required, secret scan before commit Required, scan in the pipeline Not a citizen build
Secrets from the platform's store Where a secret is needed Required Required, rotation date recorded
Access Only you Named people or group; read-only identity for the tool As Tier B plus named owner of access list; reviewed on a date
Data Your own, or synthetic Synthetic in development; real data only after screening Screening by the data protection officer attached; logs without personal data
Dependencies Listed in requirements.txt Dependency policy; each new name looked up Policy checked by the pipeline; a developer confirms each addition
Input and model calls Validate what you can Validate against the specification Validated and tested; any model call reviewed by a developer
Provenance record A note in the build record PRV-*, one page PRV-* reviewed by a named developer, with date
Evidence valid for Until the next change 90 days or the next change 90 days or the next change of code, dependency or assistant

Tier D covers the cases in which the tool ranks or decides about a named person, is or feeds a high-risk AI system, sits in a security-critical path or acts irreversibly on production data. It is not a citizen build, and none of the controls in the table makes it one.

Practical guide: The security pass

Purpose. To run the security checklist for the tier, close or escalate what it finds, and write the provenance record, so that Step 9 Ship has something to ship against.

Roles. The builder runs the pass. The deputy repeats the secret scan on a clean copy. At Tier C a named developer (the receiving developer of Step 12) reviews the checklist and the provenance record. The data protection officer screens any tool that touches real personal data. The platform engineer provides the secret store, the access list and the pipeline. The assistant drafts and explains.

Inputs. The repository at the release candidate; the verification plan of Step 7; the specification with its list of fields read; the tier decision; the mandate; the platform's secret store and pipeline.

Steps.

  1. Search for secrets. Run the secret scan over the working tree and the history. If it finds anything, follow the leaked-key procedure before going on.
  2. Move every secret to the store. Replace each with a read from the environment, ask the platform engineer to add it to the store, and run the tool once to confirm it fails with a clear message when the variable is absent.
  3. Set the access. Write the list of who may run the tool and who may see its output. Confirm that the tool's identity is read-only where the specification allows.
  4. Check the dependencies. Write or update dependency-policy.yaml, run the check, and for every name added since the last pass confirm the spelling, the registry page, the maintainer and the version. Hand anything unclear to a developer.
  5. Check the input and the output. List every place the tool reads from outside, with the rule that validates it. Ask the assistant to draft the checks and the tests for refusal; the oracle owner adds one case that must be refused.
  6. Check the data. List the personal data the tool touches, the logs it writes and the files it leaves. Confirm that logs hold identifiers only. Attach the data protection officer's screening where real data are used.
  7. Write the provenance record and run the checklist for the tier. Record what was not checked.
  8. Review. The deputy repeats the secret scan; at Tier C the named developer reads the checklist and the record before any AI comment.

Exit gate. The gate is met when the builder can show: a secret scan of the tree and the history with its date and a clean result; the dependency check passing against a policy that lists each package with a reason; the access list; the data protection officer's screening where real personal data are used; the filled-in checklist for the tier; and the provenance record, with an expiry date, reviewed by the person the tier requires. A statement that the tool is secure is not a gate.

Artefact. PRV-* and the security checklist, referenced from BLD-*.

provenance_record:
  id: PRV-HM-2031-0001
  build: BLD-HM-2031-0001
  version: v0.1.0
  generated_by:
    assistant: approved assistant of the builder lane
    mandate: MND-HM-BLD-2031-01
    run_records: [RUN-HM-2031-0007, RUN-HM-2031-0011]
  dependencies:
    - {name: pyyaml, pinned: "6.0.2", purpose: read rules and cases, listed_in_policy: true, exists_in_registry: true, checked_by: Tomasz Wrona, checked_on: 2031-11-13}
    - {name: pytest, pinned: "8.3.4", purpose: run the tests, listed_in_policy: true, exists_in_registry: true, checked_by: Tomasz Wrona, checked_on: 2031-11-13}
  secrets_in_repository: none
  secret_scan: {tool: scripts/scan_secrets.py, run_on: 2031-11-13, result: clean}
  personal_data:
    in_repository: none               # synthetic data only
    in_logs: claim ids and counts only
    screening: {by: Dr. Sabine Vogel, on: 2031-10-20, decision: AIDR-1251, dpia_required: false}
  not_checked: [behaviour under a registry outage]
  valid_until: 2032-02-11             # 90 days from 2031-11-13, or the next change

The two builds at this step

The service-point-check build at Nordvik Region

Hanna Wikström's tool reads public data: the portal export, the phone-service export and the notices spreadsheet, none of which holds personal data. It never writes to any of them. It needs one secret, WIKI_TOKEN, to write its weekly report to the e-services team's wiki page. At version 0.1.0, which Hanna runs on demand in the builder workspace, the platform's secret store injects the token as an environment variable; from version 0.2.0 the scheduled-job template does the same on Mondays at 06:00. The first draft by the assistant had the token as a default value in a configuration file. The secret scan stopped the commit on 2031-11-06, the value was removed, the platform engineer Joakim Wester issued a new token (the pasted one was treated as leaked, because it had been in the conversation) and the old one was revoked the same day. The event is in the build record. The dependencies are pyyaml and pytest, and the platform engineer Joakim Wester looked up each. The provenance record PRV-NR-2031-0001 is dated 2031-11-06 and carries the same fields as the Hansa record above, with personal_data: none. Hanna's tool uses a read-only path to the three exports, so a mistake in the code cannot damage the sources. The security pass was complete on 2031-11-06 and the gate is recorded in BLD-NR-2031-0001.

The claims-doc-check build at Hansa Mutual

Katarzyna Nowak's tool processes claimant data, and it is the build in which the personal-data rules matter. The group data protection officer, Dr. Sabine Vogel, screened the tool on 2031-10-20, narrowed the scope to property-damage claims, found that no data protection impact assessment was required for the narrowed scope, and recorded the decision as AIDR-1251. Development and testing use only the synthetic export data/synthetic/claims_synthetic.csv (identifiers from SYN-0001); the production export is read only by the pipeline's identity, never by the builder or the assistant. The tool reads no free-text field and logs nothing but claim identifiers and counts, following the pattern of the log function shown earlier. Tomasz Wrona checked the two dependencies and the secret scan on 2031-11-13. The tool holds no secret at all at version 0.1.0, because the pipeline identity reads the export; the scan was run to show that. The provenance record above is the one filed with the build record, and the gate was met on 2031-11-13, four days before the release to the pilot team on 2031-11-17.

Across the degrees of delegation

The table sets what the assistant may do at Step 8 Secure. Both builds use Mode 2 in the editor and Mode 3 for delegated tasks inside a sandbox, and Level 4 Autonomous is not used for any action class.

Action class Level The assistant may The assistant may not
Explain a class of problem; review a file against a list Level 0 Advisory Explain and list findings, which a person reads State that the code is secure
Draft input checks, log lines, the provenance record Level 1 Assisted Draft; the builder reads and the deputy checks Fill in a field it has not checked, such as the date or the result of a scan
Add or change a dependency Level 1 Assisted Propose a name; the builder looks it up and the policy file is updated by a person Install an unlisted package on its own
Read a secret, production data or production configuration Not delegated Nothing Any access; the mandate excludes them
Edit the secret scan, the pipeline, the dependency policy or the mandate Not delegated Nothing Anything; Level 4 is never used for a change to the assistant's own mandate, permissions, credentials or pipeline gates
Code in a security-critical path Not a citizen build Nothing Level 4 is never used for code in a security-critical path without review by a named developer

Nordvik Region and Hansa Mutual

At Nordvik Region the platform team's workspace carries most of the controls: the secret store, the read-only exports, the Region's Git service with the scan, and the scheduled-job template that injects what the tool needs. Hanna works at Tier B, and the controls she must supply herself are the dependency list and the provenance record. At Hansa Mutual the same split is drawn differently: Group Application Development provides the lane, the pipeline identity and the receiving developer, and the data protection officer sits in the process because the tool touches claimant data. In both cases the organisation, not the builder, is the controller of any personal data and holds the secrets, so each builder's checklist is a form of handing evidence to the people who answer for them.

When to call a developer

Call a developer, or stop and escalate, when any of the following is true.

  • A secret has been in the repository, a log, a chat or a page, and the builder is not certain that it has been revoked.
  • The tool would need to read real personal data, special-category data or a free-text field, and no screening by the data protection officer exists.
  • The tool must be reachable from outside the organisation's network, or from people who are not named in the access list.
  • The tool calls a language model that reads text the tool's user did not write, or whose output can trigger an action.
  • A dependency cannot be confirmed to exist, to be what its name says, or to have a maintainer, and the assistant says it is fine.
  • The assistant proposes to switch off a scan, widen a permission, make something public or lower a threshold in order to make the tool run.
  • The tool touches a payment, a decision about a named person or a security-critical path. These are Tier D matters.

Observable, expected, speculation

What is observable: studies of vulnerabilities in generated code, from 2021 to 2026, report substantial shares of vulnerable output, within the limits of each study's design (133, Pearce et al., 2021); (134, Veracode, 2025); (109, Deng et al., 2026). Package hallucination is measured and has been shown to attract real downloads (113, Spracklen et al., 2025); (145, Lanyado, 2024). What is expected: platforms for builders will close more of these classes by default, with secret stores, closed data stores and package allow-lists built in, in line with the secure-by-default practices of the NIST framework (137, Souppaya et al., 2022); only in part, because a platform cannot know what a tool is meant to expose. What is speculation: that assistants will become reliable enough at security to remove the need for a named developer's review of Tier C tools. No study cited here shows it. The statement that the checks in this chapter are the minimum for each tier is this book's own conclusion.

Summary

Step 8 Secure, the part of D4 Prove that concerns secure code and the supply chain, makes the tool safe at the level its tier requires. The builder keeps secrets out of the code, the conversation and the logs, and gets them from the platform's store; sets access from nobody upwards; looks up every package and records it in a dependency policy that a check enforces; treats input as data and, where the tool calls a model, treats the model's output as untrusted; and holds as little personal data as the purpose needs, with a screening by the data protection officer where real data are used. The gate is a dated secret scan, a passing dependency check, the access list, the screening, the filled-in checklist and a provenance record with an expiry date.

In the two builds the controls did their work in a modest way. At Nordvik Region the secret scan stopped a token the assistant had put in a configuration file, and the token was rotated at once because it had been in the conversation. At Hansa Mutual the tool holds no secret, reads no free text and logs claim identifiers only, under the data protection officer's decision of 2031-10-20 (AIDR-1251). Both records state what was not checked. The classes of failure are drawn from the studies and standards cited; the minimum checklist per tier is this book's own conclusion.

D3 Build supplies the code and the defensive code the assistant drafts; D5 Steward keeps the records and the person who answers for the keys and the data. Chapter 13 turns to Step 9 Ship, the step in which the tool reaches its users through a path that can be undone.

Sources

  1. European Parliament and Council of the European Union (2016). Regulation (EU) 2016/679 (General Data Protection Regulation). Official Journal of the European Union. (4, European Union, 2016)
  2. OWASP Foundation (2025). OWASP Top 10:2025. (146, OWASP, 2025)
  3. European Data Protection Board (2020). Guidelines 4/2019 on Article 25 Data Protection by Design and by Default, version 2.0. (17, EDPB, 2020)
  4. Souppaya, M., Scarfone, K. & Dodson, D. (2022). Secure Software Development Framework (SSDF) Version 1.1. NIST SP 800-218. (137, Souppaya et al., 2022)
  5. Booth, H. et al. (2024). Secure Software Development Practices for Generative AI and Dual-Use Foundation Models. NIST SP 800-218A. (138, Booth et al., 2024)
  6. Pearce, H. et al. (2021). Asleep at the Keyboard? arXiv:2108.09293. (133, Pearce et al., 2021)
  7. Veracode (2025). October 2025 Update: GenAI Code Security Report. (134, Veracode, 2025)
  8. Perry, N. et al. (2023). Do Users Write More Insecure Code with AI Assistants? ACM CCS 2023. (23, Perry et al., 2023)
  9. Sandoval, G. et al. (2023). Lost at C. arXiv:2208.09727. (139, Sandoval et al., 2023)
  10. Deng, J., Fan, Z. & Meng, R. (2026). Understanding the (In)Security of Vibe-Coded Applications. arXiv:2606.23130. (109, Deng et al., 2026)
  11. National Institute of Standards and Technology (2025). CVE-2025-48757 Detail. National Vulnerability Database. (144, NIST, 2025)
  12. UpGuard (2021). By Design: How Default Permissions on Microsoft Power Apps Exposed Millions. (143, UpGuard, 2021)
  13. GitGuardian (2025). The State of Secrets Sprawl 2025. (141, GitGuardian, 2025)
  14. Spracklen, J. et al. (2025). We Have a Package for You! arXiv:2406.10279. (113, Spracklen et al., 2025)
  15. Lanyado, B. (2024). Lasso Research: AI Package Hallucinations. (145, Lanyado, 2024)
  16. OWASP Gen AI Security Project (2025). OWASP Top 10 for LLM Applications 2025. (135, OWASP, 2025)
  17. Greshake, K. et al. (2023). Not what you've signed up for. arXiv:2302.12173. (140, Greshake et al., 2023)
  18. Open Source Security Foundation and Linux Foundation. SLSA specification. (136, OpenSSF, n.d.)
  19. Ecma International (2025). ECMA-424: CycloneDX Bill of materials specification. (142, Ecma, 2025)