The engineering problems that break AI

A model will size a beam, balance a reaction, or compute a load case competently. Problems with one method and one answer are largely handled, and writing more of them is not a good use of a chartered engineer's time.

The interesting territory starts where the problem is under-specified, over-constrained, or has several defensible answers that differ in cost, buildability or risk.

The most common failure in standards-bound work is not ignorance. It is confident misapplication: the correct clause cited, then applied to a case it does not cover.

Because the citation looks right, this survives any review by someone who is not familiar with the standard. It is caught immediately by someone who is. That asymmetry is the whole argument for expert evaluation.

The other recurring pattern is an answer that is numerically correct and practically impossible — a detail that cannot be fabricated, an access requirement nobody can meet, a sequence that assumes two trades occupy the same space.

This knowledge comes from having been on site, and it does not appear in textbooks. It is the clearest example of something a model cannot acquire without practitioners.

Real constraints, a defensible answer you can justify, and at least one plausible wrong path that a competent-sounding response would take.

The last part is what makes a problem discriminating. If every reasonable attempt gets it right, the problem is not measuring anything.