1. Inventory the relevant sources
Begin with the proposed task, then list the sources it needs. These may include SharePoint libraries, customer relationship management exports, SQL databases, spreadsheets and scanned PDFs. Record the owner, access method, format and update process for each. Avoid collecting an entire organisation’s files simply because storage is available.
Distinguish an authoritative record from a convenient copy. A spreadsheet emailed between teams may not reflect the system of record. An exported file can also lose permissions or omit later corrections. Write down which source takes precedence when records disagree and identify a person who can resolve ambiguity without relying on a model to guess.
2. Inspect meaning as well as format
A file opening successfully does not mean its contents are suitable. In a CSV export, check the character encoding, column definitions and treatment of empty values. A blank field, zero and “unknown” may represent different conditions. Preserve leading zeros in identifiers and avoid interpreting reference numbers as quantities.
For documents, inspect headings, tables, page order and footnotes. OCR, or optical character recognition, converts image text into machine-readable text but can confuse similar characters or join unrelated lines. Select examples from different document types for inspection. Record which extraction issues would change an answer rather than treating every formatting difference as equally important.
3. Resolve duplicates and conflicting records
Exact duplicates can be identified through file checksums, which are calculated representations of file contents. Near-duplicates require more context: a revised policy may differ by only one paragraph, yet that paragraph can matter. Keep version and approval information where available. Do not delete records automatically just because their text is similar.
For structured records, agree matching rules before merging. Two customers can share a name, and one customer can appear under several spellings. Use suitable identifiers where available and send ambiguous matches for review. Preserve a record of transformations so that someone can explain why the prepared dataset differs from the source.
4. Establish permission and purpose
Access to a file does not automatically authorise every later use. Identify the purpose of processing, the relevant lawful basis and any contractual restrictions. Under UK GDPR, personal data processing must meet applicable requirements, including data minimisation and purpose limitation. Special category data requires additional conditions. Refer uncertain cases to a data protection professional.
Remove personal information that the task does not need. Pseudonymisation replaces direct identifiers but does not necessarily make data anonymous. A combination of remaining fields may still identify someone. Apply access controls to prepared copies, indexes and evaluation sets as well as source files. Review Information Commissioner’s Office guidance when establishing your approach.
5. Build a repeatable preparation process
A one-off clean file can support a demonstration while hiding the work needed for operation. Define how new records arrive, how changes are detected and how deletions are propagated. A schema is a description of a data structure; validating incoming records against it can reveal unexpected columns, missing fields or incompatible types before downstream processing.
Keep failed imports visible. Quarantine, meaning separating suspect inputs for investigation, is often safer than silently discarding them. Log transformations without copying unnecessary sensitive content into logs. Maintain links back to source records so that corrections can be traced. Agree whether updates replace a dataset or are applied incrementally.
6. State what is ready and what is not
A data readiness engagement can be scoped to produce a source inventory, documented quality issues, preparation rules and a recommendation about the proposed use. Some problems are technical; others require a business decision. A tool cannot determine which conflicting policy the organisation intends to follow without an authoritative instruction.
For a first discussion, describe the source systems, document formats and proposed task. Do not email a raw customer export. Secure transfer arrangements and processing responsibilities should be agreed before access is granted. Once the inputs are suitable, move to knowledge assistants for document-based answers or workflow automation for controlled processing and hand-offs.
