Reflections on creating ExCrap without looking at the code

The author created an Elixir library for calculating CRAP (Change Risk Anti-Pattern) scores while deliberately avoiding reading the generated code to test whether AI-written software can be verified through external validation rather than code inspection. The experiment used multiple LLMs (GPT, Gemini, Claude) for initial research, then GPT 5.5 and OpenCode for implementation, developing verification strategies focused on external behavior and product outcomes rather than internal code quality. This approach attempts to answer whether software factories can produce trustworthy code through black-box testing and external metrics alone, rather than traditional code review.

Visit Original Article →