Skip to content

What about AI Security?

💡 TL;DR - AI Security Concerns

Three questions come up in almost every security review:

  1. Does AI make mistakes that corrupt our data? AI's job is translation — natural language into declarative rules. Once built, rules are enforced deterministically by a rules engine at the database commit point, on every transaction, on every access path (API, UI, agent, message queue) — guaranteed by construction, not by AI behaving correctly at runtime. Mistakes are visible: rules are reviewed before activation, and every run produces a Logic Report tracing requirement → rule → execution.
  2. Does AI expose our data to the internet? Rarely, and never by default. Most business logic runs as local Python/SQL with zero network calls once built. The one runtime exception — AI selecting between candidates (e.g., "best supplier") — is opt-in per project, sends only the specific fields named in that rule, and logs a full audit trail of exactly what was sent.
  3. Does AI expose our trade secrets? The translated logic lives as plain, readable rules in your own source-controlled codebase, executing locally. AI's involvement is a one-time (or occasional, on change) translation of what you wrote in your own prompt — not autonomous extraction of your algorithms.

Short version: AI writes the logic; it doesn't run your business. A deterministic rules engine does that, locally, under your control.

1. Will AI Make Mistakes That Corrupt Our Data?

This is the right question to ask, because unconstrained AI-generated code absolutely does make this kind of mistake — silently, and expensively. The relevant design decision is what AI is asked to produce, and what enforces it afterward.

AI's job here is narrow: translate intent into rules, not write procedural code. Five rules replace 200+ lines of hand-written or AI-generated code for the same logic — see Executable Requirements and Top CIO Concerns for the full architecture. That matters for correctness because procedural code is path-based — a developer has to remember to call the credit check on every code path that touches an order (API, admin UI, an agent, a Kafka subscriber), and miss one and the data is silently wrong. Declarative rules are data-based — they fire once, at the ORM commit point, on every transaction regardless of how it arrived. There is no path that bypasses them. A new agent or endpoint added next year inherits the same rules automatically, with no additional governance work.

AI-generated rules are reviewed before they run. Generated logic is inserted as commented guidance in the project's logic file; the convention is to keep it inactive until reviewed. Nothing generated by AI is live by default.

AI proactively flags its own judgment calls, instead of waiting to be asked. Building a system from a requirement inevitably requires filling in details the spec didn't spell out — which field means "prohibited," how to map an XML tag with no obvious target. Rather than silently guessing, AI writes an ad-libs.md report itemizing every such decision: 🔴 for ones that need your confirmation, 🟡 for standard patterns that needed no judgment call. A real example, from a customs-eligibility rule where the spec never named a source field: AI recorded that it derived is_prohibited from hazardous_material_cd being non-blank, and flagged it 🔴 for confirmation rather than assuming it was correct. This is proactive human-in-the-loop — you review a short, itemized list of the decisions that actually needed judgment, not the entire diff, and zero 🔴 items means the spec was unambiguous. See Executable Reqmts.

The system proves its own correctness, rather than asking you to trust it. Every project generates its own test suite from the rules (not hand-written), runs it, and produces a Logic Report: requirement → rule → execution trace, readable by developers, business users, and auditors alike. If AI mistranslated a requirement, that's visible in the report — the same way a bug in hand-written code would be — rather than hidden inside opaque generated code.

Where AI does make a live, runtime decision — the narrower "AI Rules" pattern (e.g., "use AI to pick the best supplier based on cost, lead time, and world conditions") — its output is still constrained by deterministic rules on either side of it, every decision is written to an audit-trail table (the exact request sent, the reason given, whether a fallback was used), and a deterministic fallback (e.g., minimum cost) applies automatically if the AI call fails or isn't configured. See Logic Using AI.

2. Will AI Expose Our Data to the Internet?

Not by default, and only in one well-defined, opt-in case.

Once built, most of a GenAI-Logic system runs with no AI involvement at all. Deterministic rules — the vast majority of business logic in any system — execute as local Python against your database. There's no network call, no external service, nothing to expose, whether the transaction arrives via API, UI, agent, or message broker.

AI is involved at design time, translating a description you write — a schema, a natural-language requirement, a Gherkin scenario — into rules or a data model. You control what goes into that description; it is not an automatic export of your production database. See Internals - LLM for exactly what's sent and how.

The one runtime exception is opt-in, scoped, and audited. The "AI Rules" pattern above is the only place a running system calls an AI model live — and it sends only the specific candidate fields named in that rule (e.g., a handful of supplier attributes), not arbitrary data, with the exact request and response captured in the audit trail. A project that doesn't use this pattern has no reason to call any AI vendor at runtime, and can be built and deployed without an AI SDK present at all.

Whatever calls are made are governed by the model vendor's own data-handling terms, which are usually stronger than assumed. As one concrete example: under Anthropic's commercial API terms, customer content is never used to train Anthropic's models by default, and organizations can request Zero Data Retention — Anthropic doesn't store prompts or responses at all after the response returns. Check the specific terms of whichever model provider your organization has approved; most major providers offer comparable commercial guarantees, and it's reasonable to require them.

3. Will AI Expose Our Trade Secrets (e.g., Proprietary Algorithms)?

The short answer: your algorithm doesn't live with an AI vendor — it lives in your own repository, as rules you can read.

Translation, not extraction. AI doesn't discover your business logic by inspecting your systems; it translates a description you wrote — in Gherkin, plain language, or an interview — into declarative rules. Whatever's proprietary about "how we calculate optimal allocation" or "how we classify risk" is only exposed to the extent you chose to describe it in that prompt, the same way it would be if you handed the same description to a contractor.

Once translated, the rules run locally, forever, without AI. They're loaded into the rules engine at startup and enforced at the commit point by your own infrastructure. There's no ongoing dependency on an AI vendor for the algorithm to keep working — unlike, say, a hosted model you'd query every time you needed the answer.

Rules are easier to protect than the code they replace. Five rules expressing your actual business logic, in your own source control, under your existing access controls, are a smaller and more legible surface to govern than 200+ lines of procedural code doing the same thing scattered across a codebase.

See Also