Estonia
From Archives Ingest Cooperation Wiki
You are currently at Main Page -> Country Profiles -> Estonia
Country Profile for Estonia
Legislative and organisational background
The National Archives of Estonia (NAE) is a system of state owned public archives, including 13 archives in different locations of Estonia, and acts under the administration of director general, who resides in Tartu.
The NAE is a subordinate agency to the State Chancellery of the Republic of Estonia. The main legislative basis for the NAE is the Archives Act [1], which covers most of the organisational structure and general responsibilities.
Based on the Archives Act, there are several other legal acts, which regulate the archival system in Estonia. The Archival Rules [2] (government regulations) are intended to elaborate on the routines of the archival procedures: scheduling, appraisal, disposition, archival processing, preservation, protection and use of archival records, as well as acquisition procedures of public archives. Other archival practices are exercised according to internationally acclaimed directives that harmonize with the Estonian legitimate procedures.
Main partners of the NAE are the National Library of Estonia who is responsible for preserving governmental webpages and publications, the State Chancellery who is issuing regulations for the active phase of electronic records management and the Ministry of Economic Affairs and Communication whose responsibility is to coordinate the organisation and management of national datasets.
Management of databases
The management of databases in public agencies of Estonia is most influenced by the Ministry of Economic Affairs and Communication and its subagency, the Estonian State Information Agency. The main strategical documents issued by them are:
- basic IT interoperability framework [3]
- Instructions for the Semantic Description of Databases and Operations Performed by Databases[4]
National datasets in Estonia have to be connected to a common technical interoperability layer X-Road (in Estonian: X-Tee) which allows to design public services using data from different datasets. Also an Administration system of the state information system (RIHA) is available since 01.01.2008 to allow central registration and management of data about national datasets and services. All the datasets and already available services connected to the X-Road have to be registered in RIHA by July 2008 and semantically described using the semantic description guidelines by 2009.
The National Archives of Estonia has been strongly collaborating in the projects and is going to:
- use the semantic descriptions in RIHA as the main technical metadata while ingesting closed datasets;
- use RIHA as the central point to start to analyse the archival value of data in different datasets and to coordinate pre-ingest and ingest actions.
Until now there is no central approach for ingesting common support databases (financial, personelle etc.) in Estonia, mostly because the data has no archival value and the paperbased records are not being preserved in the National Archives. However, some work allowing central preservation of data from these datasets will probably start in 2009.
Electronic Records Management Systems
In Estonia the legal authority to issue guidelines on records management is the State Chancellery of Estonia. Current standards and guidelines involve:
- Functional requirements for Electronic Records Management Systems [5]
- A model record lifecycle description[6]
- Records Management Metadata Set[7]
The RM Metadata Set has a modular approach to records management and does describe in depth the following description blocks:
- classification scheme
- record
- computer file
- mandate
- function
- actor
- RM activity
Some work is still left to be done in further standardising the use of the metadata set, especially classification of mandates, functions and RM activities has been left out of the initial metadata set and needs to be dealt with.
Work is going on in standardising the usage of XML in RM. In 2008 best practice guidelines for creating and using XML Schemas to generate documents in ERM systems will be issued and a central schema repository will be established. The State Chancellery has made plans to localise MoReq2 within 2008 to replace the current Functional Requirements.
Using the X-Road interoperability layer a Document Exchange Centre (DEC) has been established in 2005. The DEC is built as a central server which has Service Level Agreements with different ERM systems in agencies. Its function is to receive centrally records and their metadata from ERM systems and automatically forward to the recipients ERM system. If an agency wants to use the DEC for exchanging records with other agencies it has to implement a DEC-client in the ERM system. Currently DEC-clients exist for most of the ERM systems used in Estonia. Also an universal client is freely available to simplify the development of DEC-clients for different ERM systems.
By the beginning of 2008 most of the agencies in central administration are connected to the DEC.
The main problem in Estonia is the actual state of ERM systems and their lacking functionality. Most of the ERMS installations don’t fulfill all the required functional requirements. To solve that both the National Archives and the State Chancellery are building or have already established collaboration networks among records managers and software providers to further disseminate knowledge and to support wider discussion regarding ERM issues. Also cooperation with universities is starting to enable better formal education for both records managers and IT specialists.
Ingest and pre-ingest in general
In Estonia most of the ingest and pre-ingest actions have to be done in the agencies. The procedure begins usually with the agency asking for permission to transfer archival records to the National Archives. After that a timeframe has to be agreed on and the agency can start to prepare the transfer. The agency has to provide most of the information for archival descriptions and the records transferred have to be provided according to the archival guidelines (archival materials have to be used, metadata has to be provided according to certain rules etc.). The prepared archival records are then transferred to the National Archives using optical media (mostly CD-R). In the National Archives the archival descriptions are put into the Archival Information System (AIS), the metadata is validated and the data is sent to off-line preservation.
Until now there have been only a few (and mostly manual or ad hoc) transfers of digital information and the Ingest module in the National Archives is going to be first tested in summer 2008. The future plans regarding the workflow are concentrating on automating most of the pre-ingest and ingest actions and also enabling online transfers from databases and ERM systems.
Pre-ingest actions for datasets
The archival requirements for datasets mostly concentrate on the metadata and archival formats. The agencies have to deliver technical, administrative and descriptive metadata in XML format, the database tables have to be in CSV format, the relations between tables have to be described using SQL3 and the services implemented have to be described using WSDL. Also some examples about the look and feel of the database are preserved as screenshots of different user interfaces. Usually in cooperation with the National Archives the database structure is simplified in some extent (elimination of some codification tables and if possible, tables with no archival value).
The approach described was used in a pilot project which resulted in transferring the National Buildings Register to the National Archives. Specific guidelines and tools helping the agencies in pre-ingest activities are not yet available but are expected late 2008 and will then also take into account the possibility to ingest most of the metadata using the Administration system of the state information system (RIHA), which is going to hold all the descriptions about active national datasets in Estonia.
Most probably there will rise the need for some more standardised databases (e.g. statistical, financial, personnel databases). However, this issue hasn't been dealt with by the National Archives of Estonia but plans have been made to start some research in 2009.
Pre-ingest actions for ERM systems
According to the Estonian Archival Law[1] and Archival Rules[2] next rules have to be followed while preparing records for transfer
- archival file formats have to be used
- metadata has to follow the national Records Management Metadata Set[7]
- records have to be of archival value
Archival formats (for text and pictures) in Estonia are as follows:
- PDF, preferred PDF/A
- plaintext using ISO 8859-1 or ISO 8859-15 character codes
- TIFF
- PNG
Currently an Universal Archiving Module (UAM) to be extending the ERM systems is being developed in the National Archives. The UAM will be installed in agencies and it will semi-automatically generate archival metadata out of the RM metadata, allow restructuring of the records and validation of metadata and file formats. It will support coordination and approval procedures between data producers and the National Archives as well. As a last step it allows data encapsulation for transfer.
The UAM will be based upon the following available documents:
- Guidelines for preparing digital records for transfer to the National archives[8]
- XML Schema of the archival description[9]
- XML Schema of the encapsulated record[10]
According to current plans the UAM will be piloted in the first half of 2008 and will be available to agencies late 2008.
Transfer to archives
Currently the Archival Rules[2] state, that the only possible transfer method is by using off-line transfer using CD-R media. A guideline[11] is available addressing following issues:
- creation of CD-R media ("burning" data)
- handling of CD-R media
- preservation of CD-R media
- recommended list of archival quality media (updated annually).
Future plans involve using the X-Road system and specifically the Document Exchange Centre solution for transferring SIPs to the National Archives. Technically it means, that online transfer will rely on XML-RPC and SOAP transfer protocols and uses the Estonian digital signature architecture to ensure the authenticity of transferred data.
For both the records out of ERM systems and datasets procedures for test transfers and criteria for accepting transfers are being developed. For ERM systems these procedures will be included into the development of the Universal Archiving Module (UAM) and be usable late 2008, for the transfer of datasets these procedures will be available in 2009.
Ingest actions for datasets
Until now no formal procedures are set in place for the SIP to AIP conversion. That means the National Archives of Estonia is accepting the datasets in archival format (see: Pre-ingest for databases) on CD-R as two identical copies and preserves them as is.
Some manual control is done in the archives, checking the metadata and integrity of transferred data but this process still needs more automation mostly in the areas of automatic validation and file normalisation.
Future plans include developing these ingest and validation tools for datasets during 2009.
Ingest actions for electronic records
Until now the ingest actions are done manually. An automated workflow is prepared which consists of
- receiving and automatically validating SIPs (incl. virus checks and quarantine);
- manual validation of metadata;
- half-automated creation of archival descriptions into the Archival Information System;
- normalisation of files to internal file format rules (not yet in place);
- creation of AIPs.
The workflow will be in testing possibly in the middle of 2009.
References
- ↑ 1.0 1.1 Estonian Archives Act (in English)
- ↑ 2.0 2.1 2.2 Estonian Archival Rules (government regulations, in Estonian)
- ↑ IT interoperability framework (in English)
- ↑ Instructions for the Semantic Description of Databases and Operations Performed by Databases (in English)
- ↑ Functional requirements for Electronic Records Management Systems (in Estonian)
- ↑ Model record lifecycle description (in Estonian)
- ↑ 7.0 7.1 Estonian Records Management Metadata Set (in Estonian)
- ↑ Guidelines for preparing digital records for transfer to the National archives (in Estonian)
- ↑ XML Schema of the archival description
- ↑ XML Schema of the encapsulated record
- ↑ Guideline: Preserving digital data on CD-R media (pdf, in Estonian)
Back to Country Profiles
