Vetting checklist
Use this to separate demoware vendors from production-grade consultants.
- Show me a Langfuse / LangSmith trace from a live production agent.
- Walk me through your golden test set and CI eval gates.
- Which MCP servers have you written or contributed to?
- How do you handle ACL propagation in retrieval?
- What is your rollback plan for a regressed prompt or model?
- Show one workflow KPI before and after deployment.
Red flags
Avoid consultants who pitch a single framework regardless of fit, demo agents in chat-only environments, can't show eval discipline, or quote multi-million dollar 'AI transformation' programs without a single shipped workflow.