The research that should precede a model choice

Model selection is a research problem with a business constraint. Public leaderboards rarely describe the task in front of you.

Teams often pick a model the way they pick a default: the one already in the account, the one a peer mentioned, the one at the top of a general benchmark. That is a reasonable start for a prototype. It is a weak way to commit an enterprise workflow.

The research we want to see is narrower. Define the task in the language of the operation: inputs, allowed sources, tools the system may call, what a wrong answer costs, and what a slow answer costs. Then build a small set of cases that include the boring majority and the ugly exceptions. Only then is a comparison meaningful. A model that wins on open-domain trivia can lose on your clause extraction, your ticket triage, or your policy lookup.

Vendor evaluations belong in the same pass. Context windows, data retention, regional availability, tool use, and price at the expected volume change the architecture. So does the question of build, buy, or integrate. A bought copilot is the right answer when the workflow is generic and the vendor will let you evaluate it. It is the wrong answer when the advantage lives in private process and the controls have to be yours.

None of this requires a research lab detached from delivery. It requires the discipline to write down the task before falling in love with a demo. The output is not a thought piece. It is a shortlist, a reason for the shortlist, and a harness you can run again when the shortlist goes stale.