For the CIO or change agent, that means maintaining concentration on the criteria that will trigger a company to implement the AI-based process at any scale, versus having the rush to an AI-based process.
Select one business process that has one or more of the following pain points: limited capacity, long cycle times, manual verification.
Identify the metrics to be measured for this process: – cases processed – average processing time – error rates – manual review rates – business impact of these limiting factors.
Enterprise implementation roadmap 10.
Your enterprise implementation roadmap should be scoped out with a particular, narrow use-case scenario and owner in mind.
That owner will set the criteria for success and failure.
If your team cannot clearly identify success and failure, then they are not prepared to start building this capability.
Implementing the roadmap in a gated manner will prevent falling into the "limited release wins all the awards and going to production is like running production" mindset.
For each of the gates in this enterprise implementation plan, there should be an entry criterion, evidence, a gatekeeper to advise approval or rejection, and options.
The process owner and product lead will establish how the AI-augmented process integrates within existing process flows.
They will identify tasks that can be delegated to the AI and will specify the rest of the limits and requirements, as well as specifications for human review steps.
Team members will list their present training data, the financial feasibility of accepting that training data, and the accessibility of that training data.
Project manager or release manager will establish baseline metrics to measure progress.
This stage on the enterprise implementation roadmap should be documented in a problem statement that contains a testable hypothesis.
You will want to do a risk analysis to better understand what might go wrong, with whom, and at what cost.
Any proposal should have a designated executive sponsor willing to approve or deny it.
Goals need to be specific with quantifiable results, such as a reduction in review time not just an increase in productivity.
It must execute test scenarios simulated by the implementation; the small set of ideal inputs is not designed into the implementation.
The functionality we measure will include the accuracy, precision, and reliability, and security.
The functionality we measure will also include the number of failures, response time, latency, or failures that result from missing information.
If the AI is to produce acceptable output or recommendations, then there must be standards as to when that output is checked and how.
The technical lead will be responsible for the design and test results of this implementation; the data owner ensures that correct information is being utilized; and the security and privacy experts consider associated information controls.
The enterprise implementation roadmap will identify these owners during the next gate design, rather than seeking approval after this step. Production-ready criteria should be drafted at the beginning of this step.
A limited release will take this implementation and start to roll it out either in a limited way or with a small pilot.
Always allow for an escape valve where you can go back to doing the previous process when things aren't working or fail to gain acceptance.
Having a good collection of measurements and user feedback, including monitoring overrides or incidents will give the evidence needed to move on to the next stage in the enterprise implementation roadmap.
This example limited release should be used for testing the entire procedure, not just the AI component.
Measuring the quality of the AI suggestions/outputs will not be sufficient on its own, instead the organisation must know whether or not the users are actually conducting the reviews needed, and whether limited AI support affects the entire system.
This should be put into the enterprise implementation roadmap as what would need to be done to take this into general release.
If this Pilot execution isn't safe per the criteria, the Release criteria will be adjusted accordingly, rather than de-risking the bar to deliver a deadline.
Upon final approval there will need to be monitoring and support procedures.
Support coverage, support response time, access review, incident escalation policy and change control guidelines should all be determined prior to this step.
The enterprise implementation roadmap should specify the model and prompt versions, constraints and testing outcomes.
This phase of the enterprise implementation plan should end in an approval from the business owner and risk or technology approvers, if any, that the evidence warrants.
A matrix assignment may indicate overlap or omission.
The business process owner has overall results and risk, the product owner will accommodate evolving requirements as users adapt to the new process, and Engineering will care for day-to-day ops support and maintenance of the application.
The data owner has final say on granting access to data and provides sample usages for training, and technical specialists provide relevant privacy, security or control requirements.
This responsibility matrix should establish who has approval and operational authority at each step without the need for consensus.
In a smaller organization, a single individual may hold numerous roles; still, there should always be someone responsible for each particular aspect of the enterprise implementation roadmap.
Equally as critical, a responsible to assignment matrix should show who has the authority to stop implementation as well as what risks can be tolerable without further scrutiny.
Metrics for business performance and process quality need to be established in a postrelease assessment.
The former will also measure corrections, complaints, escalation incidents, etc.
The enterprise implementation roadmap is intended to allow periodic reassessment of these performance metrics, as well as updates whenever the procedure or model, prompt, data, etc. undergo significant changes.
Similarly, there should be a formal way of providing feedback about other non-critical production problems.
There should be someone responsible for receiving this, and checking to see if it is symptomatic of a larger problem, and taking appropriate action if it is.
The enterprise implementation plan and process will be continuously reviewed, not a static set of instructions.
These updates should happen as soon as the initial assumptions seem to be wrong rather than carrying on with pilot configuration just because it is there.
A gated implementation offers defined acceptance standards for each stage and includes designated approvers to halt the work.
Instead of looking at one specific model change, by focusing on process changes and continuity of achievement, the enterprise implementation roadmap will lead to every incremental AI business process having realized benefits that outweigh continued support and resources.
