Map the complete journey

Identify the source documents, retrieval service, inference endpoint, logs, and backups. A model can run inside a customer environment while logs still leave it. The whole data path matters.

Separate model choice from privacy

An open-weight model does not automatically make a deployment private. A managed service is not automatically unsuitable. Architecture, provider terms, access controls, and operating requirements determine the boundary.

Preserve source permissions

Retrieval must apply the current user’s permissions before information is included in a model request. Test cross-team access, revoked permissions, and stale indexes.

Make the tradeoffs explicit

Compare performance, cost, residency requirements, hardware, and the team needed to operate the system. Document the agreed boundary before implementation.

Further reading

NIST AI Risk Management Framework covers evaluation and operational risk. OWASP’s prompt injection guidance explains untrusted inputs and permission boundaries.

An AIS Lab editorial perspective. Architecture and operating choices depend on the requirements of each application.