If you just vibe AI, it will lull you into thinking you're doing great.
- Define what successful AI usage looks like and include metrics and measurements — include AI usage in your story/ticketing system.
- Define parameters for usage. Will it write documentation? Write tests? Write functions? Classes? Whole features? Will it review code? How are you making sure everyone understands the new code (i.e. you don't want devs committing code they don't understand or reviewers passing it)
- How will you reign in costs? I had 3 devs spend over $1500 (each) in thirty days and am in the process of a explaining this to my COO. Purchasing plans instead of using API keys gives you some natural boundaries for reigning in costs.
- Create standard processes. Don't vibe. Have a standard for how your AI-facing documentation is structured. Have standards/templates for how prompts are structured. Have standard prompts with limited scope for specific tasks. For example have a prompt just for review that specifies to review from the standpoint of security and best practices.