Click here for our Financial Systems Division

The role of technology in managing records retention and disposal by Jaume Vilar of BEP Systems (part 3)

This paper is an extract from ‘Managing Records Retention and Disposal’ by Alison North and published by Ark July 2009

Auto-classification

Much has been made of auto-classification and this is a rapidly evolving area. Because the capabilities and applicability of this technique cannot be generalised, some points are made which should be considered when deciding if the technique is relevant to a particular document type or process.

  • Auto-classification is appropriate to structured documents and some process-driven activities;
  • Current technology is very sensitive to fine tuning (garbage in, garbage out);
  • Auto-classification can cater to very high volumes and is potentially powerful if used as a first-line classification method (‘better than nothing!’);
  • It is showing increasing capability by combining metadata surrounding a document with content to give an assessment of document classification; and
  • Nobody has yet properly answered the general problem of how to address the output if the classification error rate is important. Unless appropriately filtered, it requires a highly-skilled individual to validate the auto-classification, thereby negating the benefits. (However, this is not an issue if this auto-classification is ‘better than nothing’ and errors can be accepted.)

Collections of records/documents

This paper focuses on individual documents and records, each of which has its own properties or metadata. However, this is not the complete picture since a document may form part of a collection of documents which logically exist together – such as a series of recordings at a single studio session. Clearly, there is also the potential for nesting of collections which adds to the complexity. This section explains why it is reasonable to focus on individual documents as a simplification.

Two types of collections of documents exist: breakable and unbreakable. An unbreakable collection will all be retained and all be destroyed together. Breakable collections can have items purged from within them. Systems (and in particular the records retention management system, EDM systems and collaboration tools) must cater for both.

  • Collections will usually be pre-agreed with policy makers since the concept of breakability and unbreakability must be agreed and approved for a collection;
  • An unbreakable collection has a ‘document type’ property of its own which will dictate the retention period. Essentially this collection acts as a single document;
  • A breakable collection has no document type and merely acts as a cross-reference to link different documents or sub-collections; and
  • The concept of a collection will usually be duplicated in both the storage system (because the collection will be a ‘business’ concept) and the records retention management system (because it has to understand the relationship of the items and trigger their disposal).

Duplication of records

This is primarily an issue for electronic documents since duplicate physical objects/ documents cannot be seen by IT systems (they only know about references to them). Duplication should be treated as wider issue, not just relating to records retention. It is a cultural, process and storage problem that must be reviewed holistically.

EDM systems and collaboration tools are specifically designed to eliminate duplication, but culture and process are key. Organisations which address duplication as an issue at source will normally see operational efficiency improvements, as well as avoiding costly IT spend to manage the issue once it has occurred.
E-mail ‘reply’ trails create a particular duplication problem – an organisation must decide its view on copies of e-mails effectively existing in later e-mails. De-duplication tools can address the existence of uncontrolled documents across the corporate space, e-mail and other storage systems, but this is a costly approach. Again, it comes back to managing the issue at source.

Using IT systems in the different records retention approaches

This paper is an extract from ‘Managing Records Retention and Disposal’ by Alison North, chapter one of which describes three approaches to records retention; the reference tool approach, the records management approach and the process and workflow approach.

The IT systems deployed to support retention policy management depend on the approach taken, the nature of the organisation and the need to mitigate risk. The IT systems discussed below in relation to the three approaches are not three discrete points, but merely points on a continuum of increasing automation (note that the approaches themselves are very distinct!). At the most basic end are systems providing information to support manual processes, ranging through manual archiving with semi/fully-automated disposal to automated archiving and disposal.

Reference tool approach

This is the most straightforward from an IT point of view. It involves a system which provides the end user and records manager with the retention policy and related collateral. Normally this will involve a database of the organisation retention periods agreed for the different document types, along with the background legislation, regulation and company policy. It may also include a ‘classification’ collecting together the document types by business process or functional areas.

How IT supports the approach
There will be limited IT support for the approach – most likely only a reference tool. The IT variations will centre upon how this tool will be seen by the users and how the data in it will be maintained.
As a minimum requirement, the reference tool will be tailored to provide users with retention policy information (timescales, trigger points and supporting documentation) to allow appropriate retention decisions to be made at the relevant point in the business process.

The reference tool will supply up-to-date information direct to the users, ensuring that if policy changes, new rules are immediately rolled out. The tool may also be able to supply a top-down breakdown from high-level policy into process and finally into document type, allowing the user to see all the levels comprising the policy, including the ‘hows and whys’. This ensures that the policy is a ‘living document’. This approach also enables swift rollout of changes and, with appropriate configuration, ensures only the people who need to be updated are updated (preventing information overload).

There is a requirement for an internal or external agent to co-ordinate both the policy detail and research into regulatory/ legislative changes (see Chapter 1). Ensuring an appropriate IT deployment to make this process as efficient as possible will keep costs down.
Points to note are:

  • This tool is used by the organisation as a ‘reference’ when executing manual archiving processes, therefore there are no IT controls to ensure the policy is rigorously implemented. . No systemised records are kept of the link between the document archived and the policy information which dictated the Managing Records Retention and Disposal retention period. Therefore the question ‘which documents of this type do we hold?’ cannot easily be answered. This also carries the risk that if the regulations or legislation change the policy, it cannot be implemented easily to existing archived documents, which would be the case when deploying a records management tool.
  • In order to keep the database up-to­date, regular research and policy reviews are required.
  • A reference tool of this type underpins all the later approaches as they all require the base retention policy data to support increased levels of automation.

/more…