SERVICE / 03

Test what your AI can reveal or do.

Adversarial testing of AI applications, from prompt injection and data exposure to unsafe tool use and broken guardrails.

Typical engagement: 1–4 weeks

Discuss this engagement Prepare a scope brief ↗

Have a particular question about this work?

Explore a guide and refine the focus ↗

WHO IT IS FOR

For teams launching or expanding an AI feature that can access customer data, internal knowledge, or connected tools.

A model can appear safe in isolation while the surrounding application crosses a user, tenant, or permission boundary. Retrieved documents, tool responses, and ordinary users can all introduce instructions the system should not follow.

THE WORK

From question to evidence.

Map the real workflow: prompts, retrieval, data access, user roles, and tool permissions.
Test direct and indirect prompt injection, guardrail bypass, data leakage, and unintended actions.
Document reproducible paths, affected data or systems, and the business consequence.
Retest the relevant controls after the team makes changes.

WHAT YOU TAKE AWAY

A prioritized report with reproduction steps, evidence of the boundaries tested, and a retest of important fixes.

AN EXAMPLE OF THE OUTPUT

Explore the deliverable.

Compare engagements ↗

ILLUSTRATIVE SAMPLE / FICTIONAL ENGAGEMENT

Customer-review response

Turn a broad security claim into a supported answer.

Question or test: Customer asks: “Is privileged production access reviewed regularly?”
Response or finding: Access reviews run quarterly. The latest review covered the production administrator group and recorded owner sign-off. Service identities are tracked separately; their review is still in progress.
Evidence and consequence: A dated access export, review record, approval, and removal ticket support the human-access claim. The service-identity gap is recorded rather than hidden.

Follow-up and reusable answer

Next action: Assign an owner to finish the service-identity review and agree on how to communicate that gap to the customer.
Reusable record: Keep the approved answer with its evidence date, scope, owner, and review date so the next response can be checked before reuse.

ILLUSTRATIVE SAMPLE / FICTIONAL ENGAGEMENT

Validation finding

A revoked service credential still reaches a production API.

Fictional validation report with test, observation, fix, and verification sections. — Fictional document mockupView full size ↗

Question or test: Test: revoke a designated test credential, then repeat its previously permitted API request.
Response or finding: The identity service records the credential as revoked, but the API continues accepting it through a cached authorization result.
Evidence and consequence: The revocation timestamp and repeatable request/response show a gap between the identity state and the API’s enforcement. The report records the tested window and affected endpoint.

Fix and verification plan

Fix: Align API authorization and cache invalidation with credential revocation. Assign the change to the relevant application and identity owners.
Retest: Repeat the same revocation sequence and check the affected API. Confirm revoked access is rejected and an authorized credential still works.

ILLUSTRATIVE SAMPLE / FICTIONAL ENGAGEMENT

AI application test report

A support assistant returns another tenant’s document.

Fictional AI security report describing a cross-tenant retrieval test and retest. — Fictional document mockupView full size ↗

Question or test: Test: use an account in tenant A to request a test document belonging to tenant B through the assistant’s search tool.
Response or finding: The tool returns tenant B’s test document. The application checks login but does not enforce tenant access on the retrieved document.
Evidence and consequence: A signed-in customer could read another customer’s information through the support workflow. The finding records the test identities, retrieval path, and observed boundary failure.

Fix and verification plan

Fix: Enforce authorization on every tool request and returned document using the authenticated user and tenant. Keep that check outside the model.
Retest: Repeat the cross-tenant request with the original account and alternate inputs. Confirm unauthorized content is withheld and permitted searches still work.

ILLUSTRATIVE SAMPLE / FICTIONAL ENGAGEMENT

Leadership roadmap

Give the next quarter three owned security decisions.

Fictional security roadmap with three priorities and accountable owners. — Fictional document mockupView full size ↗

Question or test: Decision: prioritize customer-facing security commitments alongside a growing product team’s existing work.
Response or finding: Start with three priorities: ownership of customer-review evidence, the production-access review, and the AI launch decision. Record an accountable owner and the evidence each decision needs.
Evidence and consequence: The roadmap connects each priority to a business decision, a control owner, and a check-in date. Unresolved assumptions remain visible rather than being presented as completed work.

Ownership and decision rhythm

Action record: For each priority, record the owner, next action, evidence required, dependencies, and the decision date agreed with the team.
Leadership update: Use a regular review to show what changed, what remains blocked, and which decision needs leadership attention. Adjust priorities as the business changes.

PRACTICAL QUESTIONS

Before we begin.

Prepare a scope brief ↗

Do you test the model or the whole application?

The work follows the application workflow: prompts, retrieval, user roles, data access, and tool permissions. A model that appears safe alone can still sit inside an application with a broken access boundary.

Can we discuss testing before launch?

Yes. This engagement is for teams launching or expanding AI features. Bring the planned workflow, the data or tools it can reach, and the launch decision the testing needs to support.

What access is needed?

Access depends on the workflow and the boundaries being tested. The scope defines the roles, test data, environments, and tool permissions needed before testing begins.

Do the deliverables include retesting?

The service includes a retest of important fixes. The relevant controls and timing are agreed as part of the engagement; a typical engagement is 1–4 weeks.

NEXT STEP

Talk through the scope.

Share the situation and the decision it needs to support.

Email Piton ↗

Test what your AI can reveal or do.

For teams launching or expanding an AI feature that can access customer data, internal knowledge, or connected tools.

From question to evidence.

A prioritized report with reproduction steps, evidence of the boundaries tested, and a retest of important fixes.

Explore the deliverable.

Turn a broad security claim into a supported answer.

A revoked service credential still reaches a production API.

A support assistant returns another tenant’s document.

Give the next quarter three owned security decisions.

Before we begin.

Talk through the scope.

Privacy preferences

Question counts