Managing digital information

Dramatic growth in digital information is forcing innovation in digital asset management and enabling new forms of services and collaborations to emerge globally, as well as at national and local level in the UK. Global services for online content searching such as Google and Yahoo are expanding their interests beyond search to content hosting and digitisation. At the same time there is a growing “democratisation” in content creation and publishing, enabling participation by individuals and small organisations. Rising costs and funding restraints are driving interest in new business models and greater inter-institutional collaboration. There is consequently a greater need for interoperability and linking between different types and sources of information.

“Democratisation” of Content
Rapid developments in processing and storage technologies are placing remarkable levels of computing power within reach of individuals and small organisations that can now access tools and reach audiences previously only possible through large institutions and businesses. Although liberating, this “democratisation” also brings new challenges. With digital media being easily manipulated and copied, traditional mechanisms of filtering for quality and authenticating content are often bypassed and the quality, originality, provenance of resources are put in question.

Open Access
Democratisation of content is impacting heavily on media industries and traditional distribution mechanisms with the possibility of still greater challenges to existing revenue streams and business models in the future. In the academic sector, university libraries have increasing difficulty in keeping up with the growing volume and cost of scholarly resources. In the UK, journal prices rose 158 per cent between 1991 and 2001 compared to a 28 per cent increase in inflation.
Concerns over access to publicly funded research are leading to greater interest by research funders and institutions in opening up access to research outputs (data and papers) that they have funded. Institutions and funders worldwide are now making policy decisions to support open access, a trend reflected in the UK through the activities of the Wellcome Trust, the Joint Information Systems Committee (JISC), and the Research Councils.

Since the mid-1990s it is estimated that some £130 million has been invested in digitisation of cultural and research materials in the UK. Even so, only a small proportion of analogue heritage content in the public sector has been digitised so far. The announcements by Google to work on digitisation with major research libraries such as Oxford University, and Microsoft to work with the British Library may marked a tipping point in the creation of mass digital content. The participation of major commercial service providers should be welcomed, as the public sector could never digitise the world’s knowledge alone.

Digital Policy Management
Digital Rights Management (DRM) technology is commonly portrayed as a mechanism for restricting access to and use of digital content but it can also be deployed with Digital Policy Management for enabling use. “Access and use policies” are traditional elements in the management of collections. Policies may derive from internally determined practice, from the legal framework within which the organisation operates or from licence agreements with controllers of copyrights.
In the physical world, the management of these policies is relatively easily managed by physical constraints and occasional human intervention. However, in the digital world relying on physical constraints and human intervention often creates insurmountable barriers to efficient management. For the orderly management of policy, rights and permissions need to be possible to express in machine interpretable formats.

Shared Content Services
There is increasing interest in a range of institutional and subject repositories that will support the sharing and dissemination of content. Institutional repositories are important for universities and colleges in helping to manage and capture intellectual assets as a part of their information strategy. A digital repository can hold a wide range of materials for a variety of purposes and users, supporting research, learning, and administrative processes.

Inter-operability focuses on the technical policies and standards needed to exchange and use digital information across computer networks. Inter-operability is essential for e-government. It is particularly important in distributed systems to ensure software and hardware from multiple vendors can communicate. In the UK the e-GIF initiative has defined the technical policies and specifications governing information flows across government and the public sector. They cover interconnectivity, data integration, e-services access and content management.

Search and Resource Discovery
Since the invention and explosive early growth of the Web, a range of technologies and approaches have been employed to render the web navigable, usually focusing on informed human selection. While early search engines were often crude and imprecise in their results, their power and usability have improved dramatically in recent years, with searching going much further, much deeper, than any previous index could, and unearthing material which was previously largely undiscoverable.
These services meet a clear need, and impact on an increasing number of institutions in the public sector. The challenge all these institutions now face is one of understanding the potential and pitfalls of such major new forces in search and resource discovery, and of re-positioning themselves in this new landscape.

Text and Data Mining
The amount of information now being published is far greater than any individual could hope to read, even in a narrow specialised field. Text mining uses the vast store of available digital resources to discover new, previously unknown information. Such technologies are applicable to all subject areas but biomedicine is one of the notable areas where major successes have occurred so far.

The Semantic Web
Web pages are currently designed to be read by humans, not machines. The Semantic Web aims to make web pages understandable by computers, so that they can search and analyse a vast range of web pages and perform actions in a standardised way. The potential benefits of new technologies such as the semantic web and intelligent agents (software agent exhibiting some form of artificial intelligence) are that computers can analyse and harness the enormous network of information and services online, which are beyond unaided human capabilities to exploit.

Digital Preservation
Digital preservation is needed for our literature, art, archives, and research data held in digital formats. It will affect our research and capacity to monitor issues such as global climate change. Major organisations in the UK public sector including the British Library, BBC, JISC, The National Archives, the Research Councils, and bodies such as the UK Digital Preservation Coalition have become internationally recognised for their work in addressing these issues.
In contrast to printed materials, digital information will not survive and remain accessible by accident: it requires ongoing active management. The extent to which this active management differs radically from preservation activities in the paper environment is something that is only now beginning to be understood.
Digital preservation issues, however, are increasingly affecting other organisations in the public sector. Increasing regulation, compliance and accountability often mean organisations must retain and keep digital information accessible for 5-75 years, well beyond the life of any one records management or content management system. Digital preservation challenges can begin to manifest themselves in as little as a decade. Hence similar issues are beginning to impact on other public sector organisations with a requirement for long-term management of digital information.

Digital Storage
Information growth and the increasing shift to digital forms of storage and communication have made digital asset management central to effective delivery of modern services. We are in a period of intense and rapid change. As a result of these changes, business and organisational models are in flux and new options are being tested across a range of global content industries and public institutions.

Please register to comment on this article