Save the experiment version and expected assignment
Record experiment identity, variant versions, assignment rule and exclusions. Use an isolated test mode with fictional events. Editorial preview routes should be clearly distinguished from a real participant's route.
A fictional example compares A and B explanations of a working process. B receives new copy after the initial checks. Old and new B must not be mixed as identical content without an explicit version rule.
Compare actual content with its event
Reach each variant through the agreed test mechanism and inspect content and recording together. Define when exposure can count. A script starting or an identifier being assigned can precede the relevant content appearing.
Try reloads, direct links and back navigation where the implementation uses them. Check caches reuse the right response and page sections do not mix variants. Keep evidence of the displayed version rather than just an event name.
| Test | Risk | Expected evidence |
|---|---|---|
| B assigned with cache active | A delivered but B recorded | Content and exposure signal agree |
| Return visit | Unintended variant switching | Approved continuity rule holds |
| Preview | Internal review enters the experiment | Test route identified and handled |
| Version change | Different B versions combined | Versions separable under analysis decision |
| Stopped experiment | Old assignment continues | Agreed final state without test triggering |
Exercise missing storage and internal routes
Test a fresh state and one where agreed storage is unavailable. Expectations follow the approved data requirements and experiment model. Do not promise recognition of the same person across devices or blocked clients.
Open the editorial review link and check whether its events are separated under the plan. A parameter named test does not help if another tag still collects the same outcome. Inspect real receiving routes and avoid actual leads in test cases.
Document drift before interpreting results
When recorded and displayed variants differ, identify affected version, route and period. Correct one established cause and rerun positive controls and exceptions. Do not relabel the report just to make a result look cleaner.
Record variant, actual display, exposure condition, outcome and failure boundary. Decide separately how affected observations are treated. A corrected implementation does not establish whether earlier data is sufficient or whether the change has a reliable business effect.