The First 90 Days of an AI Automation Project

Questions behind the search

What the reader is trying to decide

• What should happen in the first 90 days of an AI automation project?
• What needs to be ready before day one?
• How should an AI automation project be scoped?
• Who should own the business outcome and human approval boundaries?
• What baseline should be measured before implementation?
• How should representative test cases be selected?
• What should a manual fallback and recovery plan include?
• When should a system move from offline testing to shadow mode?
• What evidence is required before a limited live pilot?
• Which measures should be monitored during the pilot?
• What should trigger a pause or early stop?
• How should the day-90 go/no-go decision be made?

The first 90 days of an AI automation project should move through four controlled decisions: define the job, establish the baseline and boundaries, test with real cases, then make a documented go, change, or stop decision. Do not spend three months building a broad system and reveal it at the end. Start with one workflow, keep consequential authority with named people, preserve a usable manual path, and expand only when operating evidence supports it.

A practical 90-day AI implementation plan uses days 1-15 for scope and ownership, days 16-30 for baseline and design, days 31-60 for testing and shadow operation, and days 61-90 for a limited pilot and decision. The schedule is a planning frame, not a promise that every project should launch by day 90. A good result may be a smaller scope, a repaired process, or a responsible stop.

This guide begins after a candidate workflow has been chosen. If that decision is still open, start with how to choose the first AI workflow. If the process itself is unstable, review why automating a broken process makes it worse before starting the clock.

What should exist before day one?

Before the kickoff, write a one-paragraph project charter. Name the business outcome, workflow trigger, end point, intended users, accountable owner, operator, affected people, systems involved, and the action being considered for automation. Also name what is out of scope. “Improve customer service with AI” is not a workable charter. “Classify incoming service requests, prepare a routing recommendation, and hold uncertain or urgent cases for dispatcher review” is bounded enough to examine.

NIST's Map guidance calls for documenting the intended purpose, deployment context, business value, risk tolerance, specific task, limits, and human oversight. It also says application scope should follow the system's capability and context.^[6] These are not paperwork after the build. They are inputs to the design.

Confirm five prerequisites:

• Outcome: Two informed people can tell whether a case ended acceptably.
• Owner: One person is accountable for the business result and can pause the project.
• Evidence: The team can obtain representative normal cases, exceptions, corrections, and failures lawfully and safely.
• Authority: The business can enforce what the system may read, draft, recommend, write, or execute.
• Continuity: Work can continue manually or through a known alternate path.

If any prerequisite is missing, day one should begin with repair, not implementation. The calendar does not make an unowned or unmeasurable workflow ready.

The first 90 days of an AI automation project at a glance

A phased plan with evidence and decision gates

Each gate is a real decision. NIST's Manage function says organizations should determine whether a system achieves its intended purpose and whether development or deployment should proceed. It also calls for mechanisms to supersede, disengage, or deactivate systems whose performance is inconsistent with intended use.^[2]

Days 1-15: How do you define scope and human authority?

During the first phase of the first 90 days of an AI automation project, follow actual work from trigger to completion. Interview the people who perform it and the people who receive its output. Capture normal cases, delays, exceptions, workarounds, competing records, approvals, and moments when judgment changes the path. A procedure document is useful, but observed work is the stronger starting point.

Turn the observation into four artifacts:

1. Current-state map. Show the trigger, inputs, decisions, systems, handoffs, outcome, and exception routes.
2. Scope statement. Specify included cases, excluded cases, users, data, action, and operating setting.
3. Authority matrix. For each action, mark whether the system may observe, draft, recommend, act within a tested limit, or must request human approval.
4. Risk register. Record a plausible failure, who could be affected, likelihood, consequence, detection method, control, owner, and residual risk.

Human authority needs more than a sentence saying “human in the loop.” Name who can approve, override, correct, pause, recover, and retire the system. NIST assigns executive leadership responsibility for deployment risk and calls for clear roles for human-AI configurations and oversight.^[1] GAO likewise organizes accountability around governance, data, performance, and monitoring, with clear goals and stakeholder involvement.^[4]

Keep external commitments, money movement, access changes, deletion, publication, sensitive disclosure, safety actions, and high-stakes decisions under human approval unless qualified review and evidence justify a narrower exception. For a deeper authority map, see which AI actions should require human approval.

Day-15 gate: proceed only if the workflow has a clear outcome, an accountable owner, enforceable authority boundaries, an exception destination, and a plausible benefit worth testing. Change scope or stop if the team cannot agree on the intended result, the source of truth, or who accepts residual risk.

Days 16-30: What baseline, tests, and fallback do you need?

The second phase of the first 90 days of an AI automation project turns the charter into a testable operating contract. Measure the current process before claiming improvement. A baseline might include accepted completion rate, correction rate, unresolved exception count, time waiting for review, duplicate actions, customer-visible errors, or staff handling time. Choose measures that reflect the intended outcome and mapped risks, not merely what a vendor dashboard exposes.

NIST recommends selecting metrics from the most significant mapped risks, documenting risks that cannot be measured, recording test sets and methods, and evaluating performance under conditions similar to deployment.^[7] A small, clean demonstration set is not representative evidence. Include ordinary cases, legitimate variation, incomplete inputs, ambiguous cases, known edge cases, and degraded dependencies.

Write acceptance rules before testing:

• What counts as a correct and complete result?
• Which errors are tolerable, and which trigger an immediate stop?
• How quickly must uncertainty reach a person?
• What sample size and operating period are meaningful for this workflow?
• Who reviews disagreements between system output and the accepted outcome?
• Which risks remain qualitative because a reliable metric is unavailable?

Then design fallback. The fallback should state how operators recognize a failure, stop new automated actions, identify work already in progress, prevent duplicates, restore authoritative records where possible, route affected cases, communicate with affected people, and resume safely. NIST's Manage Playbook calls for response, recovery, appeal, override, decommissioning, incident response, and change-management mechanisms.^[2]

Day-30 gate: do not build toward live authority unless the team has a representative test set, documented baseline, acceptance thresholds, stop conditions, monitoring owner, and a fallback that has been rehearsed at least as a tabletop exercise.

Days 31-60: How should you test before live action?

The third phase of the first 90 days of an AI automation project is for the narrow build, offline evaluation, and shadow operation. Build only the minimum path needed to test the stated job. Prefer restricted permissions, approved data sources, explicit validation, idempotent actions where possible, and complete logs of inputs, outputs, approvals, actions, errors, and model or configuration versions.

Start offline. Run the test set without touching production records or contacting customers. Review average performance as well as consequential failures, patterns by case type, unknown responses, and whether the system stays within scope. Test missing data, stale data, conflicting records, unavailable services, timeouts, repeated events, malicious or irrelevant instructions, and attempts to exceed permission.

Next, use shadow mode where appropriate. The system observes permitted events and records what it would have recommended or done, while the existing process remains authoritative. Compare its output with the accepted human outcome and record why they differ. Shadow mode still handles data and therefore still needs access, retention, privacy, and security controls.

NIST says system behavior should be monitored in production, limitations beyond development conditions should be documented, and a system should fail safely, especially beyond its knowledge limits.^[7] CISA's 2026 guidance emphasizes careful design, deployment, and operation of agentic services, aligning AI risk management with cybersecurity practices and maintaining oversight as autonomy grows.^[3]

Keep a test log with the case type, expected result, actual result, reviewer, severity, root cause, corrective action, retest result, and decision. Do not quietly remove hard cases to improve the score. If a case is legitimately out of scope, document the exclusion and route it to a person.

Day-60 gate: authorize a limited live pilot only when the system performs acceptably on representative cases, severe failures have effective controls, operators can understand and use the review path, logging is complete enough to investigate, and fallback works. Otherwise narrow, redesign, collect better evidence, or stop.

Days 61-90: How do you run a controlled pilot?

The final phase of the first 90 days of an AI automation project introduces limited live authority. Limit the pilot by case type, location, team, volume, time window, system permissions, or action. Do not increase all of them at once. Tell operators what changed, what did not, where uncertainty goes, and who is on call for operational and business decisions.

Use a short operating checklist:

• Confirm the named business owner, operator, technical owner, reviewer, and escalation contact.
• Verify permissions and data sources before each material change.
• Watch outcome, correction, exception, latency, override, fallback, and incident measures.
• Review samples of accepted output alongside reported errors.
• Record user and affected-party feedback, including problems the metrics miss.
• Pause when a stop condition occurs; do not wait for the next status meeting.
• Re-test after model, prompt, integration, policy, or source-data changes.

Monitoring is not proof by dashboard. NIST calls for tracking existing, unanticipated, and emerging risks over time and for feedback and appeal processes to inform evaluation.^[7] Its Manage guidance also calls for monitoring third-party resources and pre-trained models, not just the components built internally.^[2]

Hold weekly pilot reviews and a formal day-90 decision. Compare pilot evidence with the baseline and thresholds. Separate observed results from explanations and forecasts. A quiet week is not evidence that every failure mode is controlled, and a time saving does not automatically outweigh an increase in consequential errors.

What should the day-90 go/no-go decision include?

The end of the schedule is a decision point, not an automatic launch. Use four outcomes:

Document the scope tested, evidence reviewed, unmet thresholds, incidents, overrides, user feedback, known limitations, unmeasured risks, residual risk, decision owner, and next review trigger. NIST explicitly includes viable non-AI alternatives in risk management.^[2] A form change, clearer policy, simpler deterministic rule, or better handoff may be the better outcome.

What usually derails the 90-day plan?

• Starting with a platform instead of a job. The team configures features before agreeing on the outcome.
• Calling a demo a baseline. Clean examples hide exception volume and correction work.
• Treating review as free. Human oversight needs time, context, authority, and a queue owner.
• Letting scope expand silently. New data, users, actions, and systems change the risk picture.
• Leaving fallback until launch. Recovery becomes improvisation during an incident.
• Measuring activity instead of value. Messages generated or cases touched do not prove acceptable outcomes.
• Assuming no complaint means success. Some errors remain unnoticed unless output is sampled and affected people can report or appeal.

The remedy is simple but disciplined: return to the charter, evidence, authority boundary, and current gate. The schedule serves the decision. The decision does not serve the schedule.

Frequently asked questions

Do we need a working system by day 30?

No. By day 30, you need a bounded job, baseline, tests, controls, and fallback design. A rushed integration is less valuable than a testable operating contract.

Should the pilot run for exactly 90 days?

No. Ninety days is a useful planning horizon, not a universal evidence threshold. Low-volume or seasonal workflows may require more time. A dangerous failure may justify stopping much sooner.

Can AI make live decisions during the test phase?

Offline and shadow tests should not create live effects. During the controlled pilot, only explicitly authorized, reversible, low-consequence actions should run without review. Consequential actions remain with people.

What if the baseline data is poor?

Improve data capture or narrow the claim. Do not turn missing evidence into an ROI estimate. If the business cannot measure an outcome directly, document the limitation and use responsible qualitative review where appropriate.

Who owns the project after day 90?

The business outcome owner remains accountable. Technical support may maintain the integration, but it should not silently inherit authority over policy, affected people, or residual risk.

When should a project stop early?

Stop or hold when the intended outcome is disputed, representative data cannot be used responsibly, serious harm cannot be detected or reversed, required controls fail, ownership disappears, or expected value no longer justifies the operating burden.

What should you do next?

For the first 90 days of an AI automation project, choose one operating result and make every phase produce evidence for the next decision. By day 15, know the job and authority. By day 30, know the baseline, tests, and fallback. By day 60, know how the system behaves on representative and difficult cases. By day 90, decide from recorded evidence whether to expand, change, hold, or stop.

If you need help turning a selected workflow into a bounded implementation plan, contact Ordisyn. Bring the current process, intended outcome, known exceptions, source systems, and actions that must remain human. That is enough to begin a responsible implementation conversation without pretending the answer is already known.

Sources

1. NIST AI RMF Core
2. NIST AI RMF Playbook: Manage
3. Careful Adoption of Agentic AI Services
4. GAO AI Accountability Framework
5. NIST AI RMF Playbook: Map
6. NIST AI RMF Playbook: Measure

A practical next step

Start with the work, the authority, and the failure path.

Ordisyn begins with the operating problem and defines the smallest responsible implementation before access expands.

Start the conversationExplore solutions