Daily Digest
Ask multi-step questions on their own
Bundling questions saves calls, but the ones that need thinking break.
The problem
It's tempting to save calls by sending a model several questions at once: five support tickets, a page of math, a list of checks. If the model gets each one right on its own, it seems safe to assume it will get them right together. It often doesn't. Each question gets less attention than it would alone, and the ones that need the most working out are the first to break.
The technique
Give every question that needs real working its own prompt, and bundle only the quick ones. A study posted Oct 19 (Li and others, TU Munich and MIT, arXiv:2610.27190, code released) took problems that models had solved alone three times out of three and simply put several of them in one prompt. Accuracy fell across eight models, including Claude Opus 5. As more problems shared a prompt, the models spent less reasoning on each, with no limit set by anyone, and problems that took long working alone failed most. Bundling six science questions at random cut one 7B model from 49% to 24%. Grouping by need (quick questions together, long ones alone or in small groups) kept accuracy level and still cut the text the models generated by 43% on average. Those are their numbers.
- Ask anything with several steps on its own.
- Bundle short lookups or yes/no checks if you want to save calls.
- Test any bundle before you trust it.
What we saw
Our own check was much smaller: Qwen2.5 3B, on 40 grade-school math word problems, each asked alone and again inside a prompt of four (same problems, paired). Alone it got 31 right (78%). In random groups of four it got 20 right (50%), and it never got a problem right in a group that it had missed alone (p = 0.001). When grouped, it wrote about 40% less working per problem.
The reason surprised us. In 6 of the 11 lost problems, the working was right, but the final answer list was wrong: answers shifted by one place, or one number repeated for every problem. Scoring each problem's own working, the drop was 8 points and could be chance.
When we bundled only the 20 problems with the shortest working and kept the rest alone, it got 30 of 40 right (75%), close to asking everything alone.
Limits
This was one small model, compressed to 4-bit on a CPU, one pass each, on 40 problems in ten groups, so the ranges are wide. We also changed the paper's prompt slightly, adding a line asking for step-by-step working and starting each answer for the model. Unlike the paper, problems with longer working weren't hit harder in our test; short and long ones fell about the same. Our quick-first grouping cut generated text by only 6%, not 43%, because short problems use few words anyway. We used the problems' measured working length to sort them, which you won't know in advance. The paper tested independent problems of one kind, not real mixed requests or agents.
Try it
Take 40 questions you already send in bundles. Ask each one alone and in its usual bundle, and compare. Check each answer against its own working, so you can tell a mixed-up answer list from a real mistake.