What to change
Require comparable conditions before accepting the claimed improvement. Ask for the exact route, tool configuration and repeated observations used on both sides. If the original conditions cannot be reproduced, describe the new result as a new baseline rather than a proven change. Preserve representative test evidence with the release record so the next supplier can repeat it without relying on an unexplained screenshot of a favourable score.
A worked example
Illustrative example
Illustrative test: The supplier reruns both versions using one agreed testing setup. A supplier may present a slow mobile before-test beside a fast desktop after-test. Once the conditions are made explicit, both versions are compared under one agreed setup with repeated observations. The practice can then judge a supported change and its variation. If the earlier setup cannot be reproduced, the current measurement becomes a documented baseline rather than being used to claim an improvement that was never fairly tested.
What this means for your practice
A slow-phone test before a change cannot fairly be compared with a fast-desktop test afterwards. Ask your supplier to show the page, device, connection and tool settings used on both sides. Repeat comparable observations rather than choosing the worst earlier run and best later run. If the original conditions cannot be reproduced, treat the current result as a new baseline instead of a proven improvement. Keep the evidence with the release record so another supplier can repeat it. Your practice is not asking for a perfect number; it is asking whether the claimed change survives a reasonable, consistent comparison that relates to the audience you serve.
How to check the result
The improvement claim is supported by a reproducible comparison.
A mistake to avoid
Choosing the best after-run and worst before-run exaggerates a noisy result.
Turn reading into a next step
Your action checklist
Work through these checks with your team or supplier. Tick the ones you have resolved and leave unknowns open.
Record what you know, what is still missing and the answer you need from your team or supplier.
Use project decisions only, without personal, patient or confidential details. Entries stay in this page and are not submitted to Kay & Co. Copy or download before leaving; this page does not save your notes.
Further reading
These sources provide background for the topic. The practice examples and checklist are illustrative planning suggestions from Kay & Co.
Related guides
Need help with this?
Tell us what is getting in the way. Kay & Co. can help you understand the options and turn the next step into something that works for your practice.
Explore support & operations services or discuss your project.
Try the free Healthcare Digital Planner to find your starting priority.