Data archiving is the process of moving data that is no longer actively used out of the main operational system into a separate storage layer, where it can still be retrieved if needed but no longer affects day-to-day performance. It is distinct from backup, which exists to recover from failure, and from deletion, which removes data entirely. Understanding that distinction early prevents costly design errors later.
In a business system such as a CRM, customer portal or internal workflow tool, the active database accumulates records over months and years. Completed orders, closed support tickets, expired user sessions and historical document versions all sit alongside current work. As the dataset grows, queries slow down, storage costs rise and reporting becomes muddied by stale information. Archiving addresses this by separating the data your team needs every day from the data you must keep but rarely access.
Archiving decisions sit between several concerns that are often handled by different people. Operations managers care about system speed and storage bills. Compliance and legal teams care about retention periods and the ability to produce evidence. Finance needs to understand total cost of ownership. Developers need to know where archived data lives and how it will be queried. A workable archiving plan has to translate between those perspectives rather than favouring one at the expense of the others.
The legal framework around data retention in the UK is not something to infer from memory. The UK GDPR and the Data Protection Act 2018 require a lawful basis for keeping personal data, and that basis typically includes a defined retention period. Sector-specific regulations—such as those governing financial services, healthcare or legal practice—add further requirements. The practical point for planning is not to memorise these rules but to ensure that your archiving strategy is designed so that a compliance specialist can verify it against current regulation, and that the system can actually enforce the retention periods they set.
| Data decision | Operational consequence | Control or evidence |
|---|---|---|
| Where archived data lives | There are several architectural options, each with different trade-offs. | A common approach in custom business applications is to add an archived flag to the relevant database tables. |
| Identifying what to archive | The starting point is a clear inventory of the data types in the system and the business rules that govern their lifecycle. | For each data type, you need to establish: when does a record transition from active to archived? |
| Planning retrieval | Retrieval is where many archiving plans fail. | If archived data cannot be found and presented to the right person within a reasonable timeframe, the archive is not fit for purpose—regardless of how cleanly it was built. |
| Referential integrity | Business data is connected. An order references a customer, a product and a payment. | A support ticket references a user and an account. |
Where archived data lives
There are several architectural options, each with different trade-offs. A common approach in custom business applications is to add an archived flag to the relevant database tables. Records marked as archived are excluded from default queries but remain in the same database. This is simple to implement and makes retrieval straightforward, but it does not reduce the storage footprint of the primary database and the data still needs to be included in backups.
A second approach moves archived records into a separate database, often on cheaper storage infrastructure. The main application connects to the archive database only when a user explicitly requests historical data. This improves the performance of the active system and can reduce costs, but it introduces complexity: the application must know which database to query, and any joins across active and archived data become more involved.
A third approach exports archived data into files—CSV, JSON or a structured format—stored in object storage or offline media. This minimises ongoing infrastructure costs but makes ad-hoc retrieval significantly harder. It tends to suit scenarios where archived data is needed only for regulatory compliance or occasional legal disclosure, not for routine business queries.
The right choice depends on how often archived data will actually be accessed, who needs to access it, and what queries they will run. That is why planning archiving means first understanding the retrieval requirements, not just the storage requirements.
Identifying what to archive
The starting point is a clear inventory of the data types in the system and the business rules that govern their lifecycle. For each data type, you need to establish: when does a record transition from active to archived? What triggers that transition—a date, a status change, a period of inactivity? Is the rule the same for all records of that type, or does it vary by customer, region or product line?
In a CRM, for example, a lead record might be considered active while the sales process is underway, become archived after the opportunity has been won or lost and a defined period has passed, and eventually become eligible for deletion once the retention period expires. In a customer portal, completed orders might move to archive after thirty days, while account settings remain active indefinitely. In a document workflow system, draft and in-review documents are active, approved and closed documents are archived after a set period, and documents subject to legal hold must not be archived or deleted at all regardless of age.
The key practical step is to write these rules down in a form that both business stakeholders and developers can agree on. Vague instructions such as "archive old data" lead to disputes during acceptance testing. Specific rules such as "move support tickets to archive sixty days after status changes to Closed, unless the ticket is tagged Legal Hold" can be implemented and verified.
Planning retrieval
Retrieval is where many archiving plans fail. If archived data cannot be found and presented to the right person within a reasonable timeframe, the archive is not fit for purpose—regardless of how cleanly it was built.
For each data type being archived, document who will need to access it, why, and how quickly. A finance team pulling a three-year-old invoice for an audit needs structured search by date, amount and customer. A compliance officer responding to a subject access request needs to find all personal data relating to a named individual, regardless of whether it sits in the active system or the archive. A customer service agent handling a complaint about a past order needs to see the order record alongside any associated correspondence.
These different access patterns influence the archive design. If the primary use case is structured reporting, a separate archive database with indexed fields may be sufficient. If the primary use case is ad-hoc investigation across multiple record types, the archive may need its own search interface or integration with the main application's search function.
Referential integrity
Business data is connected. An order references a customer, a product and a payment. A support ticket references a user and an account. When you archive one side of these relationships, you must decide what happens to the links.
Options include: keeping a reference identifier in the active record that points to the archived record; duplicating key summary fields into the active record so that basic information remains visible without querying the archive; or leaving the relationship in place if both records are archived together. Each approach has implications for query complexity, data consistency and storage.
The practical check is to walk through the most common user workflows and ask what information needs to be visible on the active record once related data has been archived. If a customer service agent opens an active account and sees a blank order history because those orders are archived, the archive design is failing the business, not the other way around.
Archiving in legacy systems
Older business systems often have no archiving mechanism at all. Data accumulates until performance degrades or storage limits are reached. Introducing archiving to a legacy system requires careful sequencing: first audit what data exists and how large each table is, then identify which data types are candidates for archiving, then assess whether the existing schema and application code can support an archive flag or whether a separate archive store is needed.
If the legacy system is being modernised or replaced, archiving should be considered during the migration planning rather than deferred. Migrating years of inactive data into a new system inflates the project scope and the new system's running costs from day one. A cleaner approach is often to migrate only active data into the new system, archive the historical data from the old system in parallel, and ensure the new system can reference the archive when needed.
Confusing archiving with backup
This is the single most common misunderstanding. A backup is a point-in-time copy of the entire system, designed for disaster recovery. An archive is a curated subset of data, organised for long-term retention and specific retrieval. Backing up data is not a substitute for archiving it, because a backup restores everything—including data that should have been deleted—and provides no structure for finding individual records. Equally, archiving is not a substitute for backup, because the archive itself can be lost or corrupted.
Archiving without a retrieval test
Many organisations implement archiving, assume it works, and only discover problems months or years later when someone actually needs the data. Common failures include: archived data that cannot be queried because no search interface was built; archived data that has been stored in a format the current application cannot read; and archived data that is missing context—such as the user who created it or the business process it belonged to—because those fields were not included in the archive extract.
The practical safeguard is to define, before going live, at least three specific retrieval scenarios and test them against the archive after each archiving run. Document what was searched for, how it was found, how long it took and whether the result was complete and accurate.
Breaking compliance by archiving incorrectly
Archiving can inadvertently create compliance problems. If personal data is moved to an archive that lacks the same access controls as the main system, you may have a data protection breach. If archived data cannot be found when a regulator or data subject requests it, you may be unable to meet your legal obligations. If data that should have been deleted under a retention policy is instead archived indefinitely, you are holding data beyond its lawful basis.
The check here is straightforward: for each data type being archived, confirm with whoever manages compliance in your organisation that the archive arrangement satisfies current retention and access requirements, and that deletion will still occur when the retention period expires.
Vendor and format Lock-In
If archived data is stored in a proprietary format or a system controlled by a single vendor, retrieving that data without that vendor's involvement may be difficult or expensive. This matters both for ongoing access and for exit planning. When specifying archive requirements, include the format in which data will be stored and the conditions under which it can be exported. Standard formats such as CSV, JSON or Parquet are preferable to proprietary database dumps unless there is a compelling reason to use them.
Key questions for a supplier
- How does the system determine which records are eligible for archiving, and can those rules be changed without a code change?
- Where exactly will archived data be stored, and who controls access to that storage?
- How will we retrieve a specific archived record, and what is the expected retrieval time?
- What happens to relationships between archived and active records?
- How is archived data included in, or excluded from, backup and recovery processes?
- What format is the archived data in, and can it be exported independently of the application?
- How does the system handle records that must not be archived, such as those under legal hold?
- What evidence does the system produce to show that archiving ran correctly and completely?
When to seek specialist review
Archiving intersects with data protection law, information security and, in regulated sectors, industry-specific rules. This article covers planning and design considerations, but it does not constitute legal or compliance advice. If your system handles personal data, financial records, health information or other regulated data categories, have your archiving plan reviewed by a qualified specialist before implementation. The cost of a review is modest compared to the cost of discovering, during an audit or legal proceeding, that archived data is inaccessible, insecure or retained beyond its lawful basis.