← ALL FIELD NOTES Field notes / Methods

How we intend
to report results.

Baselines, costs, failure cases and configurations: the checklist every DeltaTune Lab release should pass before it goes public.

A result is only as useful as the context around it. Before we publish any experiment, model, or method, we want it to pass a short checklist. We are writing it down now so that readers, and we ourselves, can hold our future work to it.

The checklist

  1. A clear question. State what the experiment was meant to find out before showing what it found.
  2. Honest baselines. Compare against reasonable, well-tuned alternatives, and say how they were configured.
  3. Cost alongside capability. Report memory, training time, and hardware used, not only the final score.
  4. Full configuration. Share hyperparameters, data sources, seeds, and evaluation settings so the work can be reproduced.
  5. Uncertainty. Where possible, report variance across runs rather than a single best number.
  6. Limits and failures. Describe where the method did not help, and experiments that did not work.

Why write this down?

It is easy to drift towards reporting only the most flattering result. A public checklist makes that harder. It also makes our releases more useful to other small teams, who need to know whether a method will work under their own constraints.

This is a living standard. As we learn what is missing, we will update it and say what changed.

Questions or suggestions for this checklist? Email deltatunelabs@gmail.com.