This essay was first published in Informaa Quarterly, February 1999.
It is republished here with the kind permission the Editor.
Catching Up with the Last Technology Train at the Next Station
by Richard E. Barry
Rick Barry, Principal of Barry Associates, is an internationally recognised authority in the field of information management, records management and electronic records. He may be remembered from the RMAA 1996 Convention in Canberra and subsequent workshops. More recently, he has had engagements with the United Nations Headquarters, the International Council for Archives, the US Department of Justice, and provided consultancy services to the Electronic Records Work Group set up by the US National Archivist.
Tracking developments in other disciplines and emerging technologies provides one means for recordkeeping professionals to avoid being overtaken by technological change, and concurrently permits proactive innovation to adapt to such change, using the technology in the interests of organisational business. The technologies discussed include multimedia, the implications of the Internet and internets, the World Wide Web, intranets and extranets, Web-enabled recordkeeping systems, Enterprise Resource Planning (ERP) systems, and Thin-Client technology, suggesting some recordkeeping implications for them.
In the ‘good old days’ major changes came in periodic cycles. There was time in between to acknowledge, digest, accept, adapt and master these changes to the best of our abilities. At least in the knowledge and service sectors, most of the change was in ways of managing organisations and not very often because of threatening new office technologies. Now that has changed too. As far ahead as we can see, change will be an inherent feature of modern organisations. Organisations will have to sense, learn and adapt to movements in the social and economic ecosystems in which they operate in ways that earlier we attributed only to humans and other biological forms. It is not difficult to understand why some archives and records management (ARM) professionals throw up their hands and hope that it will all somehow work itself out before anything really bad happens or before they move on. This essay will stress the importance of tracking developments in other disciplines and emerging technologies as a way for ARM professionals to avoid being blindsided and to catch 'technology trains' rather than be overrun by them. It will briefly discuss some of the more recent and upcoming of these trains and suggest some recordkeeping implications for them.
The value of weird disciplines and technology
Coming from an interdisciplinary background in both the arts and sciences, and professional experience in information management and technology as well as records management, one frequently finds that important changes in one discipline or technology pose risks, or offer opportunities to another that are not always readily apparent across those disciplinary or even technological lines.
One of the things I have tried to do over the years has been to alert the archives and records management community of emerging technologies, mainly in the information management and information technology fields, and the potential effect of these technologies on recordkeeping practices. Examples of this have included alerts concerning natural language processing and computational linguistics research in the 1980s that resulted in the development of new recordmaking systems that were not necessarily trustworthy recordkeeping systems, and therefore present new records management risks to the organisations using them. Such systems are already in place in many organisations, and are likely to be more prevalent in the future. They include speech generation, a fairly simple text-to-voice conversion, and more recently desktop speech recognition, a very complex voice-to-text conversion. These applications provide the tools for easy conversion between text and voice, including email and vmail. Now we have quite inexpensive continuous speech recognition systems such as Dragon Systems, Inc.’s Naturally Speaking™ and Point and Speak™, Dictaphone’s Boomerang™ system; and integrated voice/email messaging systems such as Voice E-mail 3.0 for AOL™ and ITServ’s FirstGate. There are also highly sophisticated intranet -ready integrated telephony and messaging systems such as the IBM/CallWare™ system.
On the other side, the same basic research is beginning to yield automatic language translation and automatic document abstracting tools: for example, by Xerox and others. A more recent example of this technology involves artificial intelligence research being carried out by the U.S. Army Research Lab which offers great potential as means of addressing one of the most intractable problems facing records managers: how to consistently and reliably ensure that email and other electronic records are identified and captured into trustworthy recordkeeping systems. The current method of leaving it up to document creators is not an effective way of identifying and capturing such records. However, up until now, no other alternatives have been available. EDMS vendors, anxious to provide a technical solution to this problem, would do well to embrace this technology.
In 1996 when the original version of this article was written, it was becoming increasingly apparent that there was a new entry into the Internet/World Wide Web (WWW) arena – the ‘intranet’ – that appeared likely to bring about considerable changes in the way we organise and conduct business in the public and private sectors. It seemed clear that the then emerging intranet technology was something that archivists and records managers would want to learn more about.
A couple of years ago, an archivist told me that the best strategy for dealing with electronic records was to simply print them to paper and treat them in the old fashioned way as paper records. It was something I had heard and recommended in the short term myself before. We understand paper. We know what to do with it. It's a more stable medium, et cetera. I agreed that this was a common and wise stopgap approach that many archivists and records managers are finding necessary to employ until such time as they can deal more directly with electronic records in their native forms. I now believe that it is a wise approach to take in any case at least until after we discover what the real fallout will be following Friday, 31 December 1999, Y2K.
I would even recommend that practice for vital electronic records where at all economically feasible. Until we see a clear path beyond Y2K, we should seriously consider running dual electronic and paper records, at least in the final months of 1999, and at least for systems that are either not Y2K compliant or that rely heavily on external systems over which the organisation has little control. Does this mean that we should put a moratorium on the development and implementation of trustworthy electronic recordkeeping systems? Not at all. However, it does suggest a short-term strategy of 'prudent avoidance' of Y2K problems where it is economically feasible to do so. Pay a small amount up front as we would for fire insurance, and don’t consider it a waste if there isn’t a fire. Consider it a reasonable cost of risk reduction. Even taking Y2K into account, however, managers should continue to work toward electronic-based recordkeeping systems and not defer substantial reductions in paper and paperwork that are, at best, a holding strategy to buy some time to get effective, ARM-proof, electronic recordkeeping systems in place.
In my opinion, innovations in information management and technology have not changed the nature of the record, but they are significantly changing the manner in which records will have to be managed. The increasing use of the Internet and the World Wide Web (WWW) have given people a glance at how effective multimedia documents (including records) can be in conveying business messages and attracting clients. Virtually all new desktop computers are multimedia equipped for sound, CD-ROM, extensive memory and storage facilities, and in many cases, Web browser and video capture software. This, coupled with the relative ease and low costs associated with access to the WWW, and the growing up of a generation of children who were raised on multimedia games and systems, will hasten the day when multimedia will become standard fare in business communication in both the private and public sector. It does not need to be emphasised that multimedia records cannot be printed on paper.
The WWW Glossary of the World Wide Web Federal Consortium offers the below definitions:
Internet and internets
The internet concept has been with us for many years. As defined above, any interoperable set of local area networks constitutes an internet. When capitalised, as in Internet, the term refers to the Internet of internets, the largest worldwide collection of internets. The Internet has also been with us for decades. As a young naval aviator in the Pentagon in the 1960s, the author recalls the early form of the Internet, then known as the Advanced Research Projects Agency Network or ARPANET. However, it was limited in membership largely to organisations carrying our federally funded research. Only in the 1990s did it blossom into the worldwide, fully open Internet.
There are few recordkeeping implications of the Internet that do not apply also to internal email systems. While the recordkeeping aspects of internal email systems are not trivial by any means, the recordkeeping aspects of Internet email can normally be addressed at that same time as the internal system is addressed, and should be. An added complication of Internet mail is that, while internal email is usually controlled in terms of linking employees' names, functions and locations, external mail may not include sufficient person and organisational identification information to fulfil requirements for recordkeeping metadata for the record’s provenance. Thus some system needs to be employed both to require, and to enable the recipient of the incoming Internet mail to ensure that it is so identified and marked.
The WWW had its beginnings about 1993 with the invention of the first web browser by Tim Berners-Lee, while at CERN (a European centre for physics research) and the development of a markup language that would make it possible to view documents across disparate technology platforms. Subsequently Marc Andreessen and other undergraduate students at the National Center for Supercomputing Applications (NCSA) in the United States, developed Mosaic which was the first web browser to become widely used. At that time, most people who had Internet access did not have access to the WWW. In recent years, however, the two most common browsers, Netscape Navigator™ and Microsoft Internet Explorer™ became free offerings and were included with ISP accounts. This has resulted in a blurring of the distinctions between the Internet and the WWW even though they serve different functions in bringing web information to the user.
At first, when organisations began to use the WWW to establish customised WWW home pages or web sites, there were few recordkeeping implications because there was little information on most of these pages beyond what might be found in a newspaper advertisement. Also, the information didn’t change much so was it probably well reflected in the organisation’s recordkeeping system. Organisations using web technology soon discovered, however, that people would visit a web site once or twice, but not more if the content was not changed. Soon organisations began providing interesting and regularly updated information on products, schedules, etc., mainly to present its desired image to the outside world. More recently, especially with the advent of secure systems for making credit card charges, web sites have become widely used for public access to purchase goods and services in what is now called electronic commerce or e-commerce applications. As the use of web sites for business purposes increases (whether private sector or public sector), so do the recordkeeping implications.
A recent study funded by the National Historical Publications and Records Commission (the research grant arm of the U.S. National Archives and Records Administration), and carried out by Charles McClure and Timothy Sprehe studied web site usage patterns of various federal and state government organisations. The study found that, in numerous cases, there were more up-to-date records on the web sites than there were in the paper record systems of the parent organisations. And, it noted that no provisions had been made to capture web site records into a recordkeeping system or otherwise to make the web sites functionally recordworthy. The study recommended guidelines for the management of web site information, recommended that federal and state organisations examine their web sites from a recordkeeping perspective, and suggested the following separation of responsibilities in the management of web sites:
Intranets and Extranets
Intranets are the logical extension of WWW technologies and applications in systems that are designed for use within an organisation, rather than between an organisation and the public at large. Think of the typical web site that is now common on the WWW, only now restricted to internal use within an organisation.
Extranets are external intranets, the next wave in TCP/IP internetworking. Extranets allow an organisation to permit selected customers or suppliers to securely connect via the Web to carry out e-commerce transactions, or to access typically product-oriented information stored behind the organisation's firewall on its intranet while disallowing access to others not so authorised. They operate much like private networks but enable external access, virtual project team operations, and secure electronic commerce applications.
Intranets are similar to the Internet and World Wide Web in that they use like standards and technologies. It isn't important to the thrust of this article to explain them here. For the purposes of this discussion, it is enough to note that they include the open system standard Hypertext Markup Language, or HTML, Web browsers, servers, and related software and Internet protocols such as Transmission Control Protocol/Internet Protocol (TCP/IP). Intranets may use architectures that are logically or physically centralised or decentralised depending on the need. HTML is a daughter of the long-established and internationally recognised Standard Generalised Markup Language (SGML or ISO Standard 8879). Structure is an important characteristic of records that is apparent in traditional paper documents. One of the important features needed in electronic records is that they have, or can be given, that needed structure. SGML, and now HTML help do this, and thus are standards of considerable interest to the ARM community.
Intranets also differ from WWW web sites in some very important ways. First, most WWW sites on the Internet are open to anyone in the world who has access to the Internet, and has a Web browser. The public is the customer. By contrast, intranets exist behind organisational, password-controlled, telecommunications/computer security 'firewalls'. Its customers are the internal operating units and staff.
The definition of 'internal' is up to the implementing organisation. Access can be selectively extended outside of the immediate organisation to global field offices and designated suppliers or customers. This selective extension ensures the security of the system, protecting it from the general public, while providing access to the 'extended family' directly involved in the business of the intranet organisation. When so extended, the intranet becomes an 'extranet'. To illustrate the recentness of the term 'extranet', the term 'extended family' was used in the original version of this essay in September 1996 because the term 'extranet' had not yet been coined. Similarly, as may be seen above, the WWW Federal Consortium Glossary, that was last revised in November 1996, did not include a definition for 'extranet' or any alternative term for the concept. This technology is ideal for supporting emerging workplace patterns involving strategic partnering, supplier and other e-commerce transactions, and outsourced services.
Secondly, the content and applications of intranet systems are normally vastly different from those of the typical WWW site or homepage. The content of web sites is normally information about the owner organisation or individual -- usually concerning public or private sector services or products (including information) that the homepage owner wants the public to use. They were originally of primary interest to public relations or sales managers. By contrast, the intranet is typically used as a vehicle for conducting the core internal business and support processes of an organisation, for example:
However, as public-access web sites are increasingly used for business transactions, the demarcation lines between them and extranets are becoming blurred. Thus web sites, extranets and intranets are of increasing interest to senior executives and operational managers.
WWW applications and intranet systems are in no way mutually exclusive. On the contrary, most organisations implementing intranets will probably also have WWW sites or homepages where they can have a public image and presence. They will serve very different but complementary purposes. Similarly, intranets and groupware (eg. LOTUS NOTES) are not presently mutually exclusive. Today, most intranets are more in the nature of networking utilities than groupware. However, as groupware developers race to give their products Web functionality, and intranet technology matures to include groupware functionality, the differences between the two will become less distinct. However, intranet technology, in contrast to most groupware products, uses open systems architectures rather than proprietary ones, and typically costs much less than most groupware products. Intranets will become quintessential groupware.
About two and a half years ago, a client, who is an EDMS developer, asked me what, beyond recordkeeping functionality, would be wise to consider for the next generation EDMS. My response was to make the EDMS interface browser-like and intranet/extranet/WWW enabled. As more and more record creation takes place in an intranet, extranet, and Web environment, it will become critical for EDMS applications to be where that action is.
Web-Enabled Recordkeeping Systems
What is most interesting about this emerging technology from an ARM practitioner's perspective is its potential for the delivery of textual and multimedia records and recordkeeping services to the clients of the ARM function. Moreover, if transactions and other organisational business are taking place over them, then intranets will be record-making systems whether they are so regarded or not. As ARM professionals have learned the hard way, the fact that a system produces records does not guarantee that it is a trustworthy recordkeeping system. The fact that a system produces documents that are not considered part of the recordkeeping system, does not technically lessen the evidentiary value of those documents, or the likelihood that they will become the subject of court discovery findings.
The ARM function should, of course, deal with the recordkeeping aspects of the underlying individual systems independently of the intranet. But this will often be very costly to undertake; for example, in legacy mainframe applications. The intranet offers an excellent platform for the delivery of certain records management services to the client within the organisation and its extended family or, in the case of web sites, to the public. Similarly, web sites offer the same opportunity for records that are available for public access. A technical barrier to this approach has been the fact that HTML is not of sufficient depth and strength to serve recordkeeping functions. The emergence of XML and other advanced markup languages for special purpose applications, will reduce that barrier. However, it will continue to be necessary for ARM practitioners to deal with the difficult issue of preserving the underlying video, image, sound and text materials which exist in their separate formats.
Is web-enabled recordkeeping services a panacea or ready made solution? No! As noted above, while it can significantly automate services and improve responsiveness to client needs and productivity in the conduct of ARM business activities, it is still necessary to worry about long-term preservation of electronic records in different and complex formats. Working through some of the recordkeeping functionality of intranet/WWW applications would make an excellent candidate for anyone interested in electronic records management R&D. With or without the benefit of such applied research, ARM professionals are going to have to step onto the platform again (or hire their own IT specialists) to ensure the appropriate incorporation of recordkeeping functionality in web-based systems. Archivists and records managers should approach their chief information officer or equivalent, and find out if plans for implementation of an intranet are already under way. If not, ARM managers may be in the unusual position of advancing a state-of-the-art technology to their IT colleagues as a means of improving their own ARM services.
Enterprise Resource Planning (ERP) Systems
A technology that is becoming increasingly of interest to organisations is Enterprise Resource Planning or ERP systems, using software such as the SAP/R3 and PeopleSoft systems. These are large, transaction-based systems for handling such applications as those required by high-volume financial and human resources transactions. They are very attractive systems for a number of reasons. They offer the possibility of totally replacing a large number of outdated, non-integrated systems that require a great deal of duplication of effort for the same transaction. ERP systems may be used to handle all aspects of a single transaction for financial, human resources and other functions. As some large companies found themselves behind the power curve in reaching Y2K compliance for their myriad systems, they concluded that they might just as well replace all of those systems with a newer, much more efficient, Y2K compliant ERP system. Others considered what it was going to cost to make their legacy systems Y2K compliant, and found that they would wind up spending millions of dollars to become compliant and still be left with outdated, unintegrated systems. Again, system replacement for as much or maybe even less seemed like the obvious solution.
As these are typically high volume systems; for example those implemented at General Electric and Kraft Foods, it is not ordinarily desirable to allow organisational users to access the ERP directly. Thus, some such organisations have created data warehouses that are periodically updated by the ERP. In some cases, duplicate warehouses are created under a conscious policy of data replication for security reasons. Finally, as this is still often quite raw data, a 'data mart' may be created that draws from one of the data warehouses and organises the information in ways that make it easy to access and use. External information may also be introduced into the data mart, for example, foreign exchange rates, pricing schedules, etc., as well as a menu of common inquiries for user convenience.
There are some important implications of this technology for recordkeeping purposes. First and most important, if organisations are replacing old systems on a wholesale basis, it offers an extraordinary opportunity for the ARM function to make the case for electronic recordkeeping functionality to be built into the system. It also affords the opportunity for typically lower priority recordkeeping requirements to be bundled in with very high priority Y2K requirements, if the former are not too resource intensive. Further, ARM professionals would do well to encourage such systems, including a data replication architecture. One of the continuing issues in electronic records remains the ability to demonstrate that a digital record has not been tampered with. Making the case that no alterations have taken place in a record will be much stronger if it can be demonstrated by comparison of identical electronic originals in replicated systems. This is particularly so if the replicated systems are partitioned from one another in a manner so as to make it highly unlikely that the same person could tamper with both sites.
'Thin client' is another emerging technology that is becoming very attractive for both cost and operational reasons. Thin client refers to the use of inexpensive network-centric PCs (network computers) that have little or no hard drive, or limited user access to the hard drive, and no floppy disk drive. These PCs carry out most of their operations on a network server rather than in the PC. The extent to which this is the case varies with system and application. In some cases, it is tantamount to a return to the old days of mainframes and dumb terminals. However, the jury is still out as far as how many thin PCs will actually make it to the market, in what quantities they will be used; and how well they will be supported by NT operating software.
Use of these products is driven by two main forces. One is the 'TCO' or total cost of ownership concept wherein many IT managers have found that hardware and software installation, upgrades and maintenance for a large number of PCs distributed across an organisation constitute a major organisational cost. By substantially reducing the complexity of PCs and forcing them to use network resources (including software, cycles, hardware and data), the TCO for the systems combined can be cut by as much as one half. Another factor is to shift control of the PC from the user to the IT function for data security, or other reasons.
If thin client technology does take off, it could be a God-send for records management. One of the most intractable problems facing electronic records systems is the total lack of control over large numbers of electronic records scattered over difficult to control organisational PC hard drives, on diskettes in people’s desks and notebook computers, and in their homes. Attempting to properly execute a records management program including comprehensive records disposition management under these circumstances is a daunting challenge. The possibility of having all organisational records created, used, preserved and otherwise managed on network servers would greatly reduce the size and scope of that challenge. This approach, especially if combined with an ERP architecture, would also require us to re-think our ideas about the virtues of centralised versus distributed recordkeeping systems.
ARM professionals can better position themselves and their organisations for the next waves of technological and organisational change, whatever they are, if we look up from our desks frequently enough to hear the sounds of the distant locomotives driving other disciplines and emerging technologies. We can opt and lobby for technologies that better meet our organisations’ needs for trustworthy recordkeeping. If we do those things, chances are we will be ready for the next train as it pulls into our station or even to run ahead to catch it at the next stop.