1. Theory vs practice
Deploying AI models and apps has been the main concern of companies for the good part of the last three years. A repetitive pattern is balancing revenue and responsible use.
When a decision-maker is faced with a worse-than-marketed model, the question becomes: should we simply advise users, should we work on a fast fix silently, or do we simply halt the model?
Responsible AI says you stop, pause and fix. Practice is not always as straightforward. The examples of this are too many to count.
Simply put: the exam's cases exist, but the answer is sometimes theoretical and lacks the high-stakes decision making that risks the survival of projects. Practice, however, is dealing with stakeholders, slowness, mistakes, etc.
2. Learn as we blunder
Usually, a learn-as-we-build approach can be good for R&D and to provide a competitive edge. Nonetheless, given our hasty approach to adopting AI, design mistakes turn into blunders: critical issues leading to debt and lost revenue. Even though the AWS Cloud Adoption Framework for AI tries to phase the steps, real-life constraints of competitiveness force rapid deployments and initiatives.
The clearest example of this is the buy vs build decision. In the exam, the same thought process we apply to cloud is adopted (low expertise or urgent needs mean buy; competitive advantage and governance concerns mean build). However, history tells us we learned that through sheer trial and error, not by inferring it from the cloud. Companies spent billions building their own chatbots, buying data and training them on large resources, to minimal success. Today, we simply advise substituting training LLMs with the available APIs from Claude or ChatGPT. It's a good practice, but one that cost billions of dollars.
3. Man vs tool
To their credit, AWS go above and beyond to show us that tooling itself comes after many steps to build readiness, and that building readiness matters to companies as much as tooling. However, there seems to be a general trend in AI strategy to underestimate how much knowledge the average worker lacks about AI, and how much it's costing organizations. Because most people use or have used AI at least a couple of times, one can assume that people readiness is sufficient or can be built naturally. It cannot, and there's sufficient research to show how overconfident we are about our AI use. The rule of thumb is simple:
Knowing how to use a tool doesn't equate to knowing how to work with it efficiently.
We cannot create well-defined architectures for every use case rapidly, so people will use assistants or whatever solution they're handed. The amount of unseen incidents, debt created and general loss of productivity and satisfaction is disproportionate at the time being.