highlight · part 0 · 5

Garbage in, garbage out.

Engineers moved on. Scientists can't.


When ChatGPT first came out, engineers obsessed about prompt engineering. Then models got smart enough to guess what they meant. The discipline faded.

Scientists can't take that shortcut. We can't ask AI to guess what we meant by "the intervention." We can't trust it to fill in our intent. Measurement is the work of being precise about what we measure.

The codebook is the specification.

Bad specification, bad data. No model is smart enough to fix that.

And no model should be. The specification is the science.

@karlrohe