You ship an MCP server, and the model keeps picking the wrong tool, or won’t pick yours at all. The reflex is to rewrite the description, rename the tool, shuffle the list. A paper out last Friday measured which of those levers actually moves the pick, and the strongest one is the part almost nobody polishes: the parameter schema.

Xiaofan Li, Fengchun Qiao, and Xing Gao started from metadata for 367,174 MCP tools and one fact from that corpus: the 169 tools named send_email between them use 78 different parameter sets. Same job, same name, similar prose, different shapes. Then the experiment — for five tasks, build tool variants that differ in exactly one metadata field (server name, tool name, description, or parameter schema), put them in front of five LLMs as candidates for the same requests, balance the positions, and fit a conditional logit to the choices.

Schema edits moved selection more than any of the other three factors, and consistently across models. The sharpest single result: rename an email tool’s to parameter to recipient_id and its selection rate falls to 1–10 percent on all five models. Rename it to receiver instead, a word that appears in none of the requests either, and the tool stays near chance at 35–51 percent. The drop is not keyword matching; recipient_id reads as an internal identifier, and the model takes the name as evidence about what the tool is for. Models also preferred separate top-level fields (owner, repo) over one composite string, and when only one candidate’s schema could satisfy an explicit requirement in the request, all five models picked that tool no matter where it sat in the list. No tool runs in their setup; they measure selection alone. I didn’t rerun any of it — every rate above is the paper’s.

The schema was written for the validator, the code that checks a call once it exists. The same JSON turns out to do work before any call exists: the model reads what fields a tool wants, and in what shape, as evidence about what the tool is for, and weighs that evidence more heavily than the prose. The authoring habits invert from there. Name parameters for what they mean, not for your database column. Keep them at the top level rather than nested in a config object. Put the parameter explanations in the description — those helped, while tool-name effects were weak and inconsistent. If two of your tools do near-identical things, don’t count on list order to separate them; make one schema the only one that satisfies the request you want it to get. The paper’s advice to evaluators is the same lesson pointed elsewhere: measure selection under varied candidate orders, because order effects are real and the schema can override them.

Two production systems already treat this surface as load-bearing. Uber’s MCP Gateway treats tool prose as a contract: every change to a tool description triggers a config diff that server owners must approve before it deploys, because their AutoCrawler continuously re-translates the API registry into tool definitions and the definitions churn as the APIs drift. Google’s Cloud CLI remote MCP server sits at the opposite pole: the gcloud and bq command surfaces behind two tools, run_gcloud_command and run_bq_command, each built around a free-form command string plus a required project parameter, with some commands forbidden by list rather than by shape. The schema says where a command runs and almost nothing about what it does, so the selection question collapses into the model writing the command itself.

Uber gates the description because a description edit can move which implementation runs. The paper’s numbers say a schema edit moves it more, and the field renames that do it are exactly the kind a refactor ships on a Tuesday without anyone calling it an interface change. The parameter list deserves the same review the prose gets.

source ↗

← all notes