Practical research guide · desk research and BespokeData methodology · 18 Nov 2026
AI readiness & data quality assessment
Decide whether a limited AI pilot is worth running, and identify the evidence and controls it needs first.
Intended reader: Business owners, operations leads and data owners considering AI for a specific recurring decision.
Define one decision, verify input quality and permission, and compare a bounded pilot with a credible baseline before scaling. Pause if ownership, access or evaluation is missing.
Executive answer: start with a decision, not a maturity score
An organisation is not simply ready or unready for AI. A particular use may be appropriate while another is unsafe, uneconomic or impossible to evaluate. Begin with one decision, the people affected by it and the records needed to support it. Establish what a useful result would look like before buying a tool. If permission, a credible baseline or an accountable reviewer is missing, pause the pilot and fix that gap rather than converting uncertainty into a reassuring score.
This guide is an original BespokeData assessment method informed by public desk research. NIST presents AI risk management as voluntary and organises its core around Govern, Map, Measure and Manage [N1, N2]. We use that structure as a reminder to connect responsibility, context, testing and action. This is not a NIST assessment, accreditation or certification. The worksheet and thresholds below are local planning devices, not universal standards.
1. Define the decision and the acceptable failure
Write a one-sentence use statement: who will use the output, when, to decide what? For example, an operations coordinator might use a daily list of orders at risk of delay to decide which orders need manual investigation. That is narrower and more testable than asking AI to improve fulfilment. Specify whether the output is advice, a draft or an action. Keep authority with a named person during the initial pilot; do not quietly turn a recommendation into an automated customer commitment.
Describe both ordinary errors and consequential ones. A false alert consumes staff time; a missed alert may leave a customer uninformed. Set an escalation route and explain how someone can challenge an output. Decide who can stop the test, who reviews incidents and which activities are excluded. A bounded internal investigation aid needs different controls from a system that makes consequential decisions about people. Where specialist legal or security advice is needed, record it as a dependency, not as a completed check.
2. Inventory inputs, ownership and permission
List every required input before counting rows. Record the system, extraction date, record grain, field definition, responsible owner and permitted purpose. Note whether the data covers all relevant locations, customers and time periods. Trace joins explicitly: an order identifier in one export may refer to a line item in another. A technically successful join can still duplicate revenue or misstate the number of orders. Keep the original extraction unchanged and document any transformation in a separate working copy.
Ask the owner to confirm authority to use each input in the proposed environment. Access to a file does not by itself establish permission to send it to an external service or use it for model training. Identify personal and confidential fields, remove unnecessary ones and agree retention and deletion arrangements. If supplier terms or internal policy are unclear, stop at a category-level inventory. Do not upload sensitive records merely to discover whether a tool might help.
3. Measure quality against the intended use
The UK Government framework distinguishes accuracy, completeness, uniqueness, consistency, timeliness and validity, and treats quality as fitness for purpose [Q1]. For this assessment, turn each relevant dimension into a check with a denominator. Completeness might count orders with a promised date; uniqueness might count distinct order IDs; validity might check dates against permitted formats. Consistency can compare definitions across exports. Timeliness should measure the age of information when the decision occurs, not just when a file was downloaded.
Accuracy needs corroboration. A non-empty date can be perfectly formatted and still be wrong. Compare a documented sample with an authorised source such as the original order confirmation, preserving the sampling rule and disagreement count. Investigate whether gaps cluster by location, channel or system change. Do not invent missing values without a documented rule and a visible flag. Prioritise defects that can alter the decision; an unused optional description may matter less than an incorrect delivery commitment.
4. Worked hypothetical example: 1,000 orders
Suppose an authorised extract contains 1,000 distinct orders and 80 have no promised date. There are 920 populated dates, so completeness is (1,000 − 80) ÷ 1,000 × 100 = 92%. The worksheet records the denominator, missing count and extraction period. This calculation establishes only whether the field is populated. It does not establish whether the dates are accurate, whether the orders are representative or whether the AI can identify future delays.
Now suppose 46 of the 920 orders with dates were late under the agreed definition. That subgroup has a late rate of 46 ÷ 920 = 5%. The full population's late rate remains unknown because the other 80 cannot be classified using that field. Excluding them could bias comparisons if missing dates are concentrated in the hardest cases. Report the subgroup result and the 8% unclassifiable share separately; do not present 5% as a proven company-wide rate.
A defensible first action is to investigate missing-date entry with the order-system owner and verify a sample of populated dates. Only then decide whether the candidate pilot has sufficient coverage. Neither 92% nor any other single threshold grants permission to proceed. The relevant question is whether the remaining uncertainty and consequences are acceptable for this specific, limited use.
5. Keep an evidence register and establish a baseline
For every assessment claim, record a source ID, owner, date, permitted use, calculation and limitation. Separate observed values from assumptions and proposed actions. Keep a short decision log explaining why a dataset or test was excluded. An evidence register should let another authorised reviewer reproduce the denominator without guessing which file was used. It also exposes where a recommendation depends on an interview rather than an independently checked record.
Compare the pilot with a practical alternative: the current manual process or a simple rule. Freeze a representative evaluation set and define success before examining results. For a delay-alert task, measure missed delays, unnecessary alerts and review time together, rather than celebrating a single accuracy figure. Keep training or tuning examples separate from evaluation cases. Include difficult and infrequent cases, and inspect performance across relevant groups without exposing personal identities in public reporting.
6. Run a limited pilot with explicit go/no-go gates
Agree a bounded period, authorised users and a rollback plan. Use historical replay or shadow operation first where appropriate: suggestions are evaluated but do not change live commitments. Name the operational reviewer and the person responsible for resolving data issues. Log overrides and failures with enough context to investigate them. A human reviewer must have time, information and real authority to disagree; a rubber-stamp approval step is not meaningful accountability.
Write gates as concrete questions. Is permission confirmed? Can the evaluation be reproduced? Does the candidate improve on the agreed baseline without unacceptable errors? Can reviewers explain when not to use it? Can the process continue safely if the tool is unavailable? Choose proceed, revise or stop, with an owner and review date. If a gate fails, record the reason and the next evidence needed. Do not average away a failed permission or safety gate with strong scores elsewhere.
7. Reuse the assessment worksheet
The downloadable CSV provides dataset, use, owner, metric, baseline, gap and action columns. Its initial rows are explicitly hypothetical, followed by a blank working row. Replace them with authorised category-level descriptions and add evidence IDs and dates in the metric or action notes. Keep distinct rows for distinct checks so completeness and accuracy do not collapse into one judgement. Store sensitive working versions only in an appropriate private environment.
Review the worksheet with both the process owner and the data owner. Agree which gaps must be resolved before a pilot and which can be managed during it. Revisit the assessment after a change in data collection, intended use or supplier arrangements. Readiness expires when its assumptions change; a useful register is a living record of decisions, not a badge to display indefinitely.
| dataset | use | owner | metric | baseline | gap | action |
|---|---|---|---|---|---|---|
| Hypothetical order extract | Investigate delay risk | Order-system owner (example) | Promised-date completeness; 1000 distinct orders | 920/1000 = 92% | 80 missing; accuracy not tested | Investigate missingness; verify dates against authorised source |
| Hypothetical dated-order subset | Describe recorded lateness | Operations lead (example) | Late rate among 920 classifiable orders | 46/920 = 5% | 80 excluded; full-population rate unknown | Report denominator and missing share separately |
| — | — | — | — | — | — | — |
Limitations and next steps
This is desk research and a practical assessment method, not a primary study, independent expert review or evaluation of a named AI product. The example is hypothetical. It cannot establish your data quality, legal compliance, model safety or expected return. Begin by completing one decision statement and one dataset row. If the evidence is scattered or the decision spans several systems, consider a scoped Data & Analytics Opportunity Assessment with authorised inputs and agreed outputs.
Dated method & source register
Read and reviewed 18 Nov 2026. Desk research and original BespokeData recommendations and hypothetical examples; no primary study, independent expert review or Atora generation.
- [N1] NIST · AI Risk Management Framework
Public overview; voluntary framework, not certification.
- [N2] NIST · AI RMF 1.0 (January 2023)
Core Govern, Map, Measure and Manage functions; contextual risk guidance.
- [Q1] UK Government · Data Quality Framework
Fit-for-purpose quality dimensions; public-sector context.
Apply this to your business
Agree a scope around your decision and authorised evidence.
Discuss this goal →Read the fictional sample report →