Key takeaways
Many middle market organizations are adopting artificial intelligence faster than they are establishing guardrails and cost controls. Successful implementation starts with a clear business problem and treats AI as a business transformation challenge, not simply a technology implementation. Organizations that capture value without taking on unnecessary risk align AI adoption with business priorities, data readiness and appropriate controls.
As AI adoption accelerates, leaders should assess whether their data is ready, decide whether to buy versus build and establish how to measure return on investment (ROI).
Starting with a focused use case, establishing a defensible economic rationale and maintaining accountability for outcomes can help move AI initiatives beyond the pilot stage and generate business value. As AI gains more authority, controls should evolve to address what it can access, influence and execute.
A disciplined approach can help leaders move from isolated successes to repeatable business value while managing the risks that come with broader AI adoption.
Leading AI adoption with confidence
Watch RSM’s latest webinar, Navigating AI risk: A strategic compass for technology executives, to hear RSM US leaders discuss how organizations are applying AI, lessons from successful and unsuccessful implementations and the principles that help leaders scale AI responsibly.
Buy versus build: Choosing an AI approach
AI encompasses a broad and rapidly evolving technology landscape, from familiar tools such as ChatGPT, chatbots and Microsoft Copilot to more specialized and engineering-focused applications.
Buying an AI solution involves adopting existing technology, which can accelerate implementation and reduce the effort required to get started. However, AI tools that an organization builds typically require greater engineering and implementation effort but can provide more tailored capabilities based on its specific data, processes and needs.
The right approach depends on the organization, its objectives and the problem it needs to solve. What remains consistent is that the AI landscape continues to expand, creating opportunities to apply these technologies across operations.
Middle market organizations are typically investing in AI for these key reasons:
Competitive advantage
Improving products and services, differentiating from competitors and adopting new technology to remain attractive to customers
Growth
Expanding features and capabilities to drive adoption and revenue growth
Operational efficiency
Achieving more with existing labor and resources while improving productivity and margins
Measurable ROI
Demonstrating a clear connection between AI investment and business outcomes
Whether an organization buys or builds, AI delivers the most value when it starts with a defined use case and appropriate governance in place from the outset. Therefore, establishing clear ownership and risk controls can help organizations scale responsibly as adoption grows.
Common AI use cases
AI is being applied across a growing range of business functions, including through copilots and AI agents, to drive sustainable growth and gain a competitive advantage.
Key applications include:
“While productivity assistance is the common starting point, software development is changing more fundamentally, as the way teams build software today may differ significantly from how it is built going forward.”
Dave Williams, Director, RSM US
Traits of successful AI implementation
AI initiatives across multiple business functions address different problems, yet the same foundations appear in every case. Starting with a narrower scope can be more effective than attempting to do everything at once.
Successful efforts tend to share these key characteristics:
- Grounded in trusted data: Base AI outputs on approved internal data rather than relying on the open internet.
- Embedded in existing workflows: Integrate AI into the systems and processes people already use to reduce the need to learn new tools or move data between systems.
- Tied to measurable business value: Connect AI initiatives to business metrics, such as fraud losses avoided, hours or dollars saved, rather than activity measures such as prompts or tokens.
- Narrowly scoped: Start with focused use cases that automate routine activities and route complex or sensitive matters to people.
- Designed for human verification: Apply AI to repeatable tasks, such as code review, with outputs that people can review and validate.
Use case: Copilot agent for project knowledge search
Challenge: For this engagement, the team conducted more than 30 interviews with the client as part of the assessment. The team used the interview transcripts, along with supporting client documentation and project artifacts, as the knowledge base for the Copilot agent. The goal was to help team members efficiently search across this large volume of unstructured information, identify relevant insights and retrieve specific details without manually reviewing individual transcripts and documents.
Solution: To work more efficiently, the team built a Copilot agent on top of the existing content rather than building a custom model or fine-tuning anything. The team placed the content in SharePoint, pointed the agent to it with appropriate prompts and allowed team members to ask questions in plain English.
Result: The value came down to two things: scale and speed. Rather than reviewing one interview at a time, the team could ask questions across interviews and supporting documents and receive answers based on the full set. Answers that would have taken days or weeks of manual effort arrived more quickly. The agent also made the information more accessible to team members who had not participated in the original interviews.
“The Copilot agent was grounded in the organization’s content rather than the open internet. It sat inside the tools the team was already using, and its scope was narrow enough to verify its answers, and it saved a significant amount of time.”
Hina Akbar, Senior Associate, RSM US
Risks that undermine AI implementation
AI can deliver significant value, but problems can emerge when organizations deploy it without the right controls.
Key issues often include:
- Unclear policies and guardrails: AI adoption can outpace organizational policies. Define acceptable uses and data handling requirements before employees use AI for sensitive work.
- Changing conditions: AI models can produce poor decisions when underlying assumptions or market conditions change. Monitor performance and reassess models as conditions evolve.
- Inaccurate customer-facing outputs: AI can create risk when it generates inaccurate or unsupported information. Customers may assume information on an organization’s website is accurate, so validate responses before publication.
- Excessive AI access and permissions: An AI agent with more access than it needs can cause harm through unintended actions. Grant access only to the systems and actions required for its tasks and apply least privilege principles.
Use case: AI-assisted vulnerability remediation
Challenge: A client had strong visibility into vulnerabilities across a large cloud environment through Wiz, a cloud security posture management tool that served as the client’s source of truth. Engineers still had to analyze findings, research options, write scripts, document changes and determine how and where to deploy them through manual, time-consuming processes.
Solution: The team developed an AI-assisted workflow using Wiz findings to analyze vulnerabilities, recommend remediation approaches, generate scripts, create documentation and explain proposed changes. The team also explored an agentic AI workflow to prepare proposed remediations for the continuous integration and continuous delivery pipeline as reviewable pull requests.
The goal was not autonomous remediation. Instead, the workflow automated analysis and preparation while keeping a human involved in the final review and approval.
Result: Engineers now review and validate proposed remediations rather than start from scratch, which can shorten the time between detection and action and can create a more repeatable process.
“Focus on the business problem first. Give AI a clearly defined role, keep humans accountable for the outcome and make sure that the controls grow alongside the capabilities.”
Jake Kaminoff, Manager, RSM US
Best practices and pitfalls for successful AI implementation
Knowing what to put in place before AI adoption begins and what to watch for can help business leaders identify and address problems later, particularly as AI becomes embedded in core business processes and operating models. Organizations should consider these best practices and pitfalls at any stage of their AI journey to turn AI’s potential into lasting value.
Security and risk
- Establish guardrails early by defining policies and controls before rollout to mitigate identity, authorization and other risks.
- Avoid controlling approved AI tools while leaving unmanaged consumer AI tools open to uploads of sensitive data, intellectual property or client information.
Finance
- Manage usage and licensing costs early, since tying incentives to token usage can exhaust an annual budget quickly.
- Avoid running proofs of concept without establishing the economics required to scale.
Operations
- Focus on repetitive, manual work where AI can help rather than adopting technology without a clear need.
- Avoid treating AI as a substitute for human understanding and judgment.
Innovation
- Start small, target opportunities with real potential and build a supportable business case for the investment.
- Avoid scaling a promising prototype before security, financial and operational considerations are addressed.
Across all four areas, the common thread is a purposeful, targeted approach. Organizations that address these considerations from the start may be better positioned to scale effective AI applications as the technology evolves.
Frequently asked questions
AI adoption is accelerating, but a clear business problem should guide the approach, from identifying the right use case to determining how to scale it. Leaders should assess whether their data is ready, determine where AI can provide meaningful value and establish appropriate controls before scaling.
For middle market organizations, a sound first step is to start with a focused use case, measure the results, and learn and apply lessons from the implementation. Buying common capabilities can help limit unnecessary development, while custom solutions can be reserved for areas where processes, data or intellectual property provide a substantial advantage.
Ready to get started? RSM’s AI advisory teams can help leaders identify AI opportunities, assess readiness and establish an approach to responsible adoption and scaling.