root/other-projects/nightly-tasks/diffcol/trunk/model-collect/Enhanced-PDF/archives/HASH019c.dir/doc.xml @ 28933

Revision 28933, 42.6 KB (checked in by ak19, 6 years ago)

AUTOCOMMIT by script. Message: Clean rebuild of model collections 1/2. Clearing out deprecated archives and index.

1<?xml version="1.0" encoding="utf-8" standalone="no"?>
2<!DOCTYPE Archive SYSTEM "">
5  <Description>
6    <Metadata name="gsdldoctype">indexed_doc</Metadata>
7    <Metadata name="Language">en</Metadata>
8    <Metadata name="Encoding">utf8</Metadata>
9    <Metadata name="URL">http://research/ak19/gs2-svn-22Aug2013/collect/Enhanced-PDF/tmp/1395215070_1/pdf03.html</Metadata>
10    <Metadata name="UTF8URL">http://research/ak19/gs2-svn-22Aug2013/collect/Enhanced-PDF/tmp/1395215070_1/pdf03.html</Metadata>
11    <Metadata name="Title">Applications for Bibliometric Research in the Emerging Digital Libraries Sally Jo Cunningham...</Metadata>
12    <Metadata name="gsdlsourcefilename">import/pdf03.pdf</Metadata>
13    <Metadata name="gsdlconvertedfilename">tmp/1395215070_1/pdf03.html</Metadata>
14    <Metadata name="OrigSource">pdf03.html</Metadata>
15    <Metadata name="Source">pdf03.pdf</Metadata>
16    <Metadata name="SourceFile">pdf03.pdf</Metadata>
17    <Metadata name="Plugin">PDFPlugin</Metadata>
18    <Metadata name="FileSize">35935</Metadata>
19    <Metadata name="FilenameRoot">pdf03</Metadata>
20    <Metadata name="FileFormat">PDF</Metadata>
21    <Metadata name="srcicon">_iconpdf_</Metadata>
22    <Metadata name="srclink_file">doc.pdf</Metadata>
23    <Metadata name="srclinkFile">doc.pdf</Metadata>
24    <Metadata name="NumPages">17</Metadata>
25    <Metadata name="gsdlthistype">Paged</Metadata>
26    <Metadata name="ex.ExifTool.ExifToolVersion">8.57</Metadata>
27    <Metadata name="ex.File.Directory">/research/ak19/gs2-svn-22Aug2013/collect/Enhanced-PDF/import</Metadata>
28    <Metadata name="ex.File.FileModifyDate">2014:03:19 20:42:28+13:00</Metadata>
29    <Metadata name="ex.File.FileName">pdf03.pdf</Metadata>
30    <Metadata name="ex.File.FilePermissions">644</Metadata>
31    <Metadata name="ex.File.FileSize">35935</Metadata>
32    <Metadata name="ex.File.FileType">PDF</Metadata>
33    <Metadata name="ex.File.MIMEType">application/pdf</Metadata>
34    <Metadata name="ex.PDF.Author">Bronwyn</Metadata>
35    <Metadata name="ex.PDF.CreateDate">1999:09:27 16:05:06</Metadata>
36    <Metadata name="ex.PDF.Creator">Microsoft Word</Metadata>
37    <Metadata name="ex.PDF.Linearized">false</Metadata>
38    <Metadata name="ex.PDF.PDFVersion">1.1</Metadata>
39    <Metadata name="ex.PDF.PageCount">17</Metadata>
40    <Metadata name="ex.PDF.Producer">Acrobat PDFWriter 2.0 for Macintosh</Metadata>
41    <Metadata name="ex.PDF.Title"></Metadata>
42    <Metadata name="Identifier">HASH019c5dca7f5bb781460a6b9c</Metadata>
43    <Metadata name="lastmodified">1395214948</Metadata>
44    <Metadata name="lastmodifieddate">20140319</Metadata>
45    <Metadata name="oailastmodified">1395215070</Metadata>
46    <Metadata name="oailastmodifieddate">20140319</Metadata>
47    <Metadata name="assocfilepath">HASH019c.dir</Metadata>
48    <Metadata name="gsdlassocfile">doc.pdf:application/pdf:</Metadata>
49  </Description>
50  <Content>
88  <Description>
89    <Metadata name="Title">1</Metadata>
90  </Description>
91  <Content>&lt;br /&gt;
92&lt;b&gt;Applications for Bibliometric Research&lt;/b&gt;&lt;br&gt;
93&lt;b&gt;in the Emerging Digital Libraries&lt;/b&gt;&lt;br&gt;
94Sally Jo Cunningham&lt;br&gt;
95Department of Computer Science&lt;br&gt;
96University of Waikato&lt;br&gt;
97Hamilton, New Zealand&lt;br&gt;
99&lt;b&gt;Abstract:&lt;/b&gt;  Large numbers of research documents have recently become available on&lt;br&gt;
100the Internet through  “digital libraries”, and these collections are seeing high levels of&lt;br&gt;
101use  by  their  related  research  communities.  A  secondary    use  for  these  document&lt;br&gt;
102repositories and indexes is as a platform for bibliometric research.  We examine the&lt;br&gt;
103extent to which the new digital libraries support conventional bibliometric analysis, and&lt;br&gt;
104discuss  shortcomings  in  their  current  forms.    Interestingly,  these  electronic  text&lt;br&gt;
105archives also provide opportunities for new types of studies:  generally the full text of&lt;br&gt;
106documents are available for analysis, giving a finer grain of insight than abstract-only&lt;br&gt;
107online databases;  these repositories often contain technical reports or pre-prints, the&lt;br&gt;
108“grey  literature”  that  has  been  previously  unavailable  for  analysis;  and  document&lt;br&gt;
109“usage”  can  be  measured  directly  by  recording  user  accesses,  rather  than  studied&lt;br&gt;
110indirectly through document references.&lt;br&gt;
111&lt;b&gt;1.  Introduction&lt;/b&gt;&lt;br&gt;
112In recent years a number of &amp;quot;digital libraries&amp;quot; have become available through the&lt;br&gt;
113Internet.  While the technology promises in the future to support large, heterogenous&lt;br&gt;
114collections, at present the most widely used of the academically-focussed digital&lt;br&gt;
115libraries are generally repositories of one or two types of document (typically technical&lt;br&gt;
116reports, journal articles, pre-prints, or conference proceedings), grouped by discipline.&lt;br&gt;
121  <Description>
122    <Metadata name="Title">2</Metadata>
123  </Description>
124  <Content>&lt;br /&gt;
125A distinguishing characteristic of these digital libraries is that the full text of documents&lt;br&gt;
126are often available for retrieval, as well as bibliographic records.The sciences are&lt;br&gt;
127represented much more heavily in the present crop of digital libraries than the social&lt;br&gt;
128sciences, arts, or humanities. They are maintained by professional societies,&lt;br&gt;
129universities, research laboratories, and even private individuals.  Access is generally&lt;br&gt;
130free, both to search and to download documents.&lt;br&gt;
131The emergence of these subject-specific digital libraries is particularly important&lt;br&gt;
132given the pattern of access to materials presently employed by research scientists.&lt;br&gt;
133Informal exchanges of preprints, reprints, and photocopies of papers passed on by&lt;br&gt;
134colleagues currently are major venues for the transmission of scientific information&lt;br&gt;
135between researchers in the sciences.  In one study, the dependence on these sources&lt;br&gt;
136ranges from 12% (for chemistry)  to 39% (for mathematics) of all papers cited in&lt;br&gt;
137researchers' own publications [11]. A qualitative study of study of how computer&lt;br&gt;
138scientists locate and retrieve documents (computing is one of the domains considered&lt;br&gt;
139later in this paper) indicates that for that field, technical reports and research documents&lt;br&gt;
140found in various locations on the Internet are a preferred source of information [6].&lt;br&gt;
141Many of the digital library systems discussed in this paper are repositories for just this&lt;br&gt;
142type of literature.  The documents tend to be of high quality:  primarily  technical&lt;br&gt;
143reports or working papers from research institutions (both academic and commercial),&lt;br&gt;
144as well as advance copies of work accepted for publication in conventional paper&lt;br&gt;
145journals. Moreover, these digital libraries are also coming to include refereed work&lt;br&gt;
146published digitally (in electronic journals).  Anecdotal evidence suggests that in their&lt;br&gt;
147fields, these digital libraries are coming to be the resource of choice for locating cutting&lt;br&gt;
148edge work.&lt;br&gt;
149For specialized subjects such as high energy physics, this dependence on&lt;br&gt;
150informal or extra-library dissemination can be much higher. Ginsparg ([9], [10])&lt;br&gt;
151reports that fields in physics have traditionally relied heavily on preprint exchanges, and&lt;br&gt;
152the digital repositories of physics preprints begun in 1991 (the PHYSICS E-PRINT&lt;br&gt;
153ARCHIVES) have to a large extent supplanted conventional publishing and physical&lt;br&gt;
158  <Description>
159    <Metadata name="Title">3</Metadata>
160  </Description>
161  <Content>&lt;br /&gt;
162paper mailing of technical reports.  By providing ready access to information sources&lt;br&gt;
163that are already preferentially utilized by scientists, the digital libraries show potential to&lt;br&gt;
164increase access to information that until recently was expensive or difficult to acquire in&lt;br&gt;
165paper form.  Indeed, in some fields (most notably physics) this process has already&lt;br&gt;
166begun, as researchers in less developed countries report access to ongoing research&lt;br&gt;
167through the Internet repositories that their local libraries could not afford to acquire&lt;br&gt;
168through conventional journal subscriptions ([9], [10]).&lt;br&gt;
169The primary use for new bibliographic resources is, of course, for the contents&lt;br&gt;
170of the documents involved.  A secondary use for emerging resources is as a basis for&lt;br&gt;
171bibliometric analysis of the subject field.  With the conventionally published scientific&lt;br&gt;
172literature, the sheer difficulty of accumulating statistics discouraged bibliometric&lt;br&gt;
173research until the advent of large bibliographic databases in the 1960's. Computerized&lt;br&gt;
174bibliographic databases sparked a significant increase in the number of large-scale&lt;br&gt;
175bibliographic studies, as significant portions of the collection and analysis of data could&lt;br&gt;
176be automated ([12], [13]).  The availability of CD-ROM versions of bibliographic&lt;br&gt;
177databases has been of particular importance, since they provide a cheaper alternative to&lt;br&gt;
178the online commercial databases [3].&lt;br&gt;
179These computerized bibliographic resources have drawbacks, however.  The&lt;br&gt;
180greatest is that the full text of documents are rarely available, and even abstracts are not&lt;br&gt;
181always present.  This obviously limits the types of bibliometric research that can be&lt;br&gt;
182conducted &lt;i&gt;solely&lt;/i&gt; through these databases.  In addition, these databases are generally&lt;br&gt;
183limited to formally published documents (those appearing in selected books, journals,&lt;br&gt;
184and conference proceedings).  The &amp;quot;grey literature&amp;quot; of technical reports, pre-prints, and&lt;br&gt;
185other works not formally published are largely ignored, and it is this absence of easy&lt;br&gt;
186access to these documents that has hampered the analysis of these important forms of&lt;br&gt;
187scientific communication.&lt;br&gt;
188The digital libraries currently in existence complement the online and CD-ROM&lt;br&gt;
189bibliographic databases.  They are best suited for examinations of the &amp;quot;physical&amp;quot;&lt;br&gt;
190characteristics of documents (for example, document length), analysis based on&lt;br&gt;
195  <Description>
196    <Metadata name="Title">4</Metadata>
197  </Description>
198  <Content>&lt;br /&gt;
199bibliographic information that can be automatically extracted from the document text or&lt;br&gt;
200the sometimes unevenly formatted bibliographic records (such as obsolescence&lt;br&gt;
201studies),  and usage studies (geographic or institutional origin of users, date/time of&lt;br&gt;
202access, individual patterns of document retrieval, etc.).  Because  references are present&lt;br&gt;
203in the document file but not identified by field, co-citation and bibliographic coupling&lt;br&gt;
204research is not well-supported, and conducting these studies requires considerable&lt;br&gt;
205effort on the part of the researcher.&lt;br&gt;
206The variety of bibliographic repositories in the available digital libraries in itself&lt;br&gt;
207has great potential in conducting bibliometric research.  Sigogneau et al [15] present a&lt;br&gt;
208case study illustrating the ways in which the strengths of different databases can be&lt;br&gt;
209played off each other; they conduct a fine-grained analysis of the emergence of research&lt;br&gt;
210fronts in molecular and cellular biology, and demonstrate that the observations gleaned&lt;br&gt;
211from two complementary bibliographic databases provide greater insight into their&lt;br&gt;
212problem. Similarly, it appears that  the types of bibliographic data that can be gleaned&lt;br&gt;
213from the relatively unstructured digital libraries can be profitably combined with data&lt;br&gt;
214from online databases, CD-ROMS, and other more conventional bibliographic&lt;br&gt;
216This paper is organized as follows:  Section 2 discusses the types of indexing&lt;br&gt;
217and searching available with current digital libraries; Section 3 gives examples of&lt;br&gt;
218conventional bibliometric techniques applied to Internet-accessible archives; Section 4&lt;br&gt;
219discusses opportunities to directly measure usage of documents and to detect&lt;br&gt;
220information-seeking patterns in researchers; and Section 5 presents our conclusions.&lt;br&gt;
221&lt;b&gt;2.  Indexing and searching in current digital libraries&lt;/b&gt;&lt;br&gt;
222At present, the types of indexing fields for most academically-oriented digital&lt;br&gt;
223library systems are limited.  Many schemes index on user-supplied document&lt;br&gt;
224descriptions, abstracts, or similar document surrogates (for example, the PHYSICS E-&lt;br&gt;
225PRINT ARCHIVE [10], a collection of physics pre-prints and technical reports). As will&lt;br&gt;
230  <Description>
231    <Metadata name="Title">5</Metadata>
232  </Description>
233  <Content>&lt;br /&gt;
234be discussed below, the quality of this user-provided data can be highly variable, and&lt;br&gt;
235may unfavorably impact the usefulness of the index for searching. Alternatively, a&lt;br&gt;
236designated site librarian may maintain a catalog (eg, the WATERS [14] system, now&lt;br&gt;
237subsumed by NCSTRL (, both primarily collections of&lt;br&gt;
238computer science technical reports);  in this case the quality of the bibliographic&lt;br&gt;
239information may be expedited to be higher, but fewer sites will be likely to support&lt;br&gt;
240such a librarian and therefore fewer documents are likely to be included in the digital&lt;br&gt;
241library. In a “harvesting” system such as the computer science technical report&lt;br&gt;
242collections supported by HARVEST  [2] or the NEW ZEALAND DIGITAL LIBRARY&lt;br&gt;
243computer science technical report collection ([16], [17]), documents are indexed from&lt;br&gt;
244passive repositories (that may not even be aware that their documents are being&lt;br&gt;
245included in the digital library). Harvesting systems therefore cannot rely on the&lt;br&gt;
246presence of bibliographic data of any sort.&lt;br&gt;
247Because of the relative paucity of high-quality bibliographic data available to&lt;br&gt;
248many of the current academically- or research-focussed digital library collections, their&lt;br&gt;
249search interfaces tend to be more primitive than those ordinarily found in online&lt;br&gt;
250bibliographic databases or library catalogs.  Systems such as NCSTRL can support&lt;br&gt;
251author, title, and subject searching, but  this more sophisticated search functionality&lt;br&gt;
252comes at the expense of requiring participating repositories to use specific software.  As&lt;br&gt;
253a consequence, these latter systems may provide access to a small number of sites than&lt;br&gt;
254harvesting systems. Harvesters may access a broader range of providers, but at the&lt;br&gt;
255penalty of being limited to unfielded, keyword searches over the raw text of the&lt;br&gt;
256document or document surrogate.&lt;br&gt;
257Specifically, the indexing in existing digital libraries has a variety of shortcomings for&lt;br&gt;
258bibliometric applications:&lt;br&gt;
260&lt;i&gt;lack of fielded indexing:&lt;/i&gt;   As noted above, some large and widely used digital&lt;br&gt;
261libraries (such as the computer science technical report collection of the  NEW&lt;br&gt;
262ZEALAND DIGITAL LIBRARY) may lack formal cataloging entirely, and rely on&lt;br&gt;
267  <Description>
268    <Metadata name="Title">6</Metadata>
269  </Description>
270  <Content>&lt;br /&gt;
271keyword searching over the raw document text. Obviously this  makes  field-&lt;br&gt;
272dependent analysis more difficult (for example, locating documents produced by&lt;br&gt;
273specific authors), and in the worst case my require a manual examination of all&lt;br&gt;
274files in the collection in order to reliably identify a desired document subset.&lt;br&gt;
275However, keyword search techniques that approximate fielded searching results&lt;br&gt;
276may suffice:  for example in the  NEW ZEALAND DIGITAL LIBRARY computer&lt;br&gt;
277science technical report collection,  limiting the keyword search for “Johnson”&lt;br&gt;
278to a search of first pages only is likely to retrieve documents written by Johnson&lt;br&gt;
279(since for the  majority  of  computer  science  technical  reports,  the  first  page&lt;br&gt;
280contains little more than author, title, date, and institution details).&lt;br&gt;
281A more principled  approach to extracting bibliographic information is embodied&lt;br&gt;
282in  the  CiteSeer  tool  [1].  This  software  parses  raw,  unfielded  academic&lt;br&gt;
283documents and attempts to identify such indexing information as author, title,&lt;br&gt;
284reference list, etc. Obviously such a tool cannot attain 100% accuracy over a&lt;br&gt;
285heterogenous document collection, but in practice it appears useful in that it can&lt;br&gt;
286make a good first pass in processing a set of documents, providing an initial set&lt;br&gt;
287of parsed documents for analysis. The remaining (presumably much smaller) set&lt;br&gt;
288of unparsable documents can then be dealt with manually.&lt;br&gt;
290&lt;i&gt;lack of consistency in field formatting:&lt;/i&gt;  Current digital libraries usually acquire&lt;br&gt;
291bibliographic  information  from  either  the  authors  of  submitted  articles  or&lt;br&gt;
292automatic extraction routines (retrieving bibliographic details from catalog files&lt;br&gt;
293that may or may not be in a given document site, and that may or may not be in&lt;br&gt;
294an  easily  parsable  form).  Neither  of  these  methods  produce  records  with&lt;br&gt;
295standard  formatting,  which  causes  problems  with  automated  bibliometric&lt;br&gt;
296analysis.   Consider the following examples selected from entries in the hep-th&lt;br&gt;
297(high energy physics) collection of the PHYSICS E-PRINT ARCHIVES:&lt;br&gt;
302  <Description>
303    <Metadata name="Title">7</Metadata>
304  </Description>
305  <Content>&lt;br /&gt;
307Authors: A. Yu. Alekseev, V. Schomerus&lt;br&gt;
309Authors: Adel Bilal and Ian. I. Kogan&lt;br&gt;
311Authors: Paul S. Aspinwall and David R. Morrison (with an appendix &lt;br&gt;
312by Mark Gross)&lt;br&gt;
314Authors: A. H. Chamseddine and Herbi Dreiner (ETH-Zurich)&lt;br&gt;
315In this case, typical for existing digital libraries, there is no standardized format&lt;br&gt;
316for authors' names (here, appearing with full names, initials plus last name, and&lt;br&gt;
317a mixture of  the  two);  no  standard  convention  for  separating  author  names&lt;br&gt;
318(here, either a comma or &amp;quot;and&amp;quot; are  used);  and  parenthetical  information  can&lt;br&gt;
319include a variety of information such as the name of an associate author or the&lt;br&gt;
320institutional affiliations of an author.  Manual processing or specially  crafted&lt;br&gt;
321software would be required to reformat these fields for analysis.&lt;br&gt;
323&lt;i&gt;duplicate entries: &lt;/i&gt; Digital libraries that draw documents from a variety of sources&lt;br&gt;
324may  inadvertently  contain  duplicate  items.    Unfortunately,  the  irregular&lt;br&gt;
325formatting of the bibliographic information makes it difficult to  automatically&lt;br&gt;
326detect these duplicates.&lt;br&gt;
328&lt;i&gt;implicit field tagging:&lt;/i&gt;  In some repositories, items are not explicitly tagged with&lt;br&gt;
329certain  types  of  information  –  most  commonly  the  document's  date  of&lt;br&gt;
330publication or production.  Instead, the date is implicit in the document's title&lt;br&gt;
331(eg, its numeration in a technical report series) or in the location of the document&lt;br&gt;
332in the file structure of the repository  (eg,  separate  directories  exist  for  each&lt;br&gt;
333year).  A second common piece of  implicit  data  is  the  authors’  institutional&lt;br&gt;
334affiliations.  This may be contained in the document itself (typically on a cover&lt;br&gt;
335page),  or  may  be  implicit  in  the  document’s  location  (for  example,  a&lt;br&gt;
336corporation’s technical reports are stored in its ftp repository).  Again, in these&lt;br&gt;
341  <Description>
342    <Metadata name="Title">8</Metadata>
343  </Description>
344  <Content>&lt;br /&gt;
345cases  special  processing  is  required  to  append  this  field  information  to  a&lt;br&gt;
346document record for bibliometric analysis.  &lt;br&gt;
348&lt;i&gt;extraction of document text:&lt;/i&gt;  Few of  the  documents  stored  in  the  research-&lt;br&gt;
349oriented digital libraries discussed in this paper are straight ascii text; instead,&lt;br&gt;
350documents may appear in a variety of file formats, such as LaTeX, PostScript,&lt;br&gt;
351PDF, etc.  If the contents of the documents are to be automatically processed&lt;br&gt;
352(for  example,  to  count  the  words  in  a  document,  or  to  extract  reference&lt;br&gt;
353publication dates for an obsolescence study), then the text must be extracted.&lt;br&gt;
354Utilities are available to convert most common document formats to ascii.&lt;br&gt;
355It is likely that many of these problems will be addressed as the Internet-based&lt;br&gt;
356document indexing systems mature.  Even minor changes can greatly increase the&lt;br&gt;
357useability of a bibliographic database for bibliometric research.  For example, the&lt;br&gt;
358addition of an explicit date tag to many online databases in 1975 sparked new&lt;br&gt;
359applications in time series research [3].&lt;br&gt;
360&lt;b&gt;3.  Opportunities for applications of bibliometric techniques&lt;/b&gt;&lt;br&gt;
361One type of bibliometric research concentrates on quantifying fundamental,&lt;br&gt;
362structural details about a subject literature:  how many items are published, how many&lt;br&gt;
363authors are publishing, over what time period documents are likely to be used, etc.&lt;br&gt;
364More complex studies analyze the relationships between documents, such as how&lt;br&gt;
365documents cluster into subjects.  The following examples give a flavour of the&lt;br&gt;
366bibliometric research that is possible using the emerging digital libraries:&lt;br&gt;
367&lt;i&gt;examining the “physical” characteristics of archived documents&lt;/i&gt;&lt;br&gt;
368One relatively straightforward type of bibliometric study characterizes the&lt;br&gt;
369formats of different literatures.   For example, Figure 1 presents a the range of the size&lt;br&gt;
374  <Description>
375    <Metadata name="Title">9</Metadata>
376  </Description>
377  <Content>&lt;br /&gt;
378of computer science technical reports as measured by their length in pages.   Of the&lt;br&gt;
37945,720 documents in the CSTR collection as of April 1998, nearly 1600 did not contain&lt;br&gt;
380page divisions in their files (and hence are excluded from analysis). Note that the&lt;br&gt;
381number of pages in the shorter documents (&amp;lt;50 pages) falls into an approximately&lt;br&gt;
382normal distribution (slightly skewed to the left), while presumably the longer&lt;br&gt;
383documents represent Masters’ and Doctoral theses. A surprising number of documents&lt;br&gt;
384are very short (between one and 5 pages); these may represent the type of condensed&lt;br&gt;
385results frequently found in the “technical notes”, “short papers”, and “poster sessions”&lt;br&gt;
386of computing conferences and journals. The average number of pages per document,&lt;br&gt;
38727.5, appears to be slightly longer than the common upper bound for a computing&lt;br&gt;
388journal article, although this observation must be confirmed by a similar study of the&lt;br&gt;
389lengths of formally published computing articles.&lt;br&gt;
390This type of analysis is of particular interest for technical reports, since they&lt;br&gt;
391have not been studied in the same detail as formally published papers.  A comparison of&lt;br&gt;
392the physical characteristics of the formal and informal literature could provide&lt;br&gt;
393supporting evidence for common beliefs about the relationship between the two types&lt;br&gt;
394of documents. For example,  do publishing constraints force journal and proceedings&lt;br&gt;
395articles to be shorter than technical reports, and therefore presumably omit technical&lt;br&gt;
396details of findings?  Do technical reports contain more/less extensive reference sections?&lt;br&gt;
397If reference sections of technical reports are longer than those of published articles, then&lt;br&gt;
398citation links are being ommitted in published works; if technical reports contain fewer&lt;br&gt;
399references, then this may confirm earlier indications that computer scientists tend to&lt;br&gt;
400“research first” and do literature surveys later [6].&lt;br&gt;
401Figure 1.  Range of sizes of CS technical reports, measured by number of pages&lt;br&gt;
402&lt;i&gt;obsolescence studies.&lt;/i&gt;&lt;br&gt;
403A document is considered obsolete when it is no longer referenced by the&lt;br&gt;
404current literature. Typically, documents receive their greatest number and frequency of&lt;br&gt;
409  <Description>
410    <Metadata name="Title">10</Metadata>
411  </Description>
412  <Content>&lt;br /&gt;
413citations immediately after publication, and the frequency of citation falls rapidly as time&lt;br&gt;
414passes. One technique for estimating the obsolescence rate of a body of  literature– the&lt;br&gt;
415&lt;i&gt;synchronous&lt;/i&gt; method –  is to find the median date in the references of the documents.&lt;br&gt;
416This median date is subtracted from the year of publication for the documents, yielding&lt;br&gt;
417the &lt;i&gt;median citation age&lt;/i&gt;.  As would be expected, this median varies between the&lt;br&gt;
418disciplines.  Typically the social sciences and arts have a higher median citation age&lt;br&gt;
419than the “hard” sciences and engineering, indicating that documents obsolesce more&lt;br&gt;
420quickly for the latter fields.&lt;br&gt;
421As noted in Section 2, references are not generally explicitly tagged in existing&lt;br&gt;
422digital repositories.  However, reference dates can usually be extracted from the&lt;br&gt;
423document text by first locating the reference section (usually delimited by a &amp;quot;references&amp;quot;&lt;br&gt;
424or &amp;quot;bibliography&amp;quot; section heading), and then extracting all numbers in the appropriate&lt;br&gt;
425ranges for dates  for the field under study.&lt;br&gt;
426To illustrate this process, 188 technical reports were sampled from Internet-&lt;br&gt;
427accessible repositories1 and used as source documents for a synchronous obsolescence&lt;br&gt;
428study.  Conveniently, the repositories chosen organize technical reports into sub-&lt;br&gt;
429directories by their date of publication.  The reference dates for each technical report&lt;br&gt;
430were automatically extracted by software that scanned the document’s file for numbers&lt;br&gt;
431of the form 19XX, since previous studies indicate that few if any computing reports&lt;br&gt;
432reference documents published in previous centuries [5].  Table 1 presents the median&lt;br&gt;
433citation age calculated for these documents, broken down by repository and the year of&lt;br&gt;
434publication for the source documents from which the reference dates were extracted:&lt;br&gt;
435Table 1.  Median citation ages for technical report repositories&lt;br&gt;
436The median citation age ranges between 2 and 4 years, which is consistent with&lt;br&gt;
437previous examinations of computing and information systems literature ([5], [4]).&lt;br&gt;
438When graphed, the distribution of reference dates show the exponential curve typically&lt;br&gt;
439found in obsolescence studies, including the final droop due to an “immediacy effect”&lt;br&gt;
444  <Description>
445    <Metadata name="Title">11</Metadata>
446  </Description>
447  <Content>&lt;br /&gt;
448as fewer very new documents are available for citation [7].  These types of results&lt;br&gt;
449provide confirmation that references used in computer science technical reports (the pre-&lt;br&gt;
450eminent “grey literature” of  the computing field) conforms to the same patterns as&lt;br&gt;
451references found in the formally published literature.&lt;br&gt;
452&lt;i&gt;co-citation and bibliographic coupling studies&lt;/i&gt;&lt;br&gt;
453The rate at which documents cite each other (co-citation) or cite the same&lt;br&gt;
454documents (bibliographic coupling) can be used to produce &amp;quot;maps&amp;quot; of a subject&lt;br&gt;
455literature.  These techniques rely on analysis of the references of documents, and these&lt;br&gt;
456references must be in a common format.  While digital libraries contain full text of&lt;br&gt;
457documents, their references are not standardized, and indeed are not  even tagged as&lt;br&gt;
458such.  To perform these studies the references must be manually extracted and&lt;br&gt;
459processed–a tedious process that is only worthwhile for documents (such as technical&lt;br&gt;
460reports) that are not included in existing citation databases such as the Science Citation&lt;br&gt;
461Index and Social Science Citation Index.&lt;br&gt;
462&lt;i&gt;detecting cycles or regularities in the rate of production of research&lt;/i&gt;&lt;br&gt;
463Analysis of trends in the production of technical reports can give indications&lt;br&gt;
464about working conditions that affect research; for example, is more research produced&lt;br&gt;
465over the summer, when the teaching load is lighter?  or is research steadily produced&lt;br&gt;
466throughout the year?&lt;br&gt;
467Figure 2.  Distribution of the number of documents submitted to hep-th, 1992-1994&lt;br&gt;
468Figures 2 and 3 present statistics on document accumulation in the hep-th (high&lt;br&gt;
469energy physics) e-print server, a part of the PHYSICS E-PRINT ARCHIVE.  This system&lt;br&gt;
470is one of the oldest formal pre-print archives, and has become the primary means for&lt;br&gt;
471information dissemination in its field.  Examination of these figures reveals several&lt;br&gt;
472trends.  Clearly the absolute number of documents deposited in the repository has&lt;br&gt;
477  <Description>
478    <Metadata name="Title">12</Metadata>
479  </Description>
480  <Content>&lt;br /&gt;
481tended to increase over the time period.  For all three years, research production has its&lt;br&gt;
482lowest point in January and February, increases through May and June, then decreases&lt;br&gt;
483until August and September.  At that point the rate of production steps up, reaching a&lt;br&gt;
484yearly peak in November and December.  This pattern is less clear for 1992, which&lt;br&gt;
485might be expected as the archive was established in mid-1991.&lt;br&gt;
486Figure 3.  Distribution of the percentage of documents submitted to hep-th, 1992-1994&lt;br&gt;
487&lt;b&gt;4.  Analysis of usage data&lt;/b&gt;&lt;br&gt;
488The emerging Internet-based digital libraries will permit research on scientific&lt;br&gt;
489information collection and use at a much finer grain than is possible with current paper&lt;br&gt;
490libraries or online bibliographic databases.  Current bibliometric or scientometric&lt;br&gt;
491research of this type must measure information use indirectly – for example,  through&lt;br&gt;
492examination of the list of references appended to published articles.  However, it is well&lt;br&gt;
493known that authors do not necessarily include in the reference list all documents that&lt;br&gt;
494could have been cited, and conversely that not all references listed may have been&lt;br&gt;
495actually “used” in performing the research; citation behavior can be affected by a&lt;br&gt;
496number of motivating factors (Garfield lists &lt;i&gt;15&lt;/i&gt; possible reasons in [8]).&lt;br&gt;
497Digital library transaction logs provide a powerful tool for direct analysis of&lt;br&gt;
498document “usage”: since  digital libraries contain the actual document (rather than only a&lt;br&gt;
499document surrogate), the relative amount of “use” that a digital library’s clients make of&lt;br&gt;
500a given document sees can be estimated from the number of times the document file is&lt;br&gt;
501downloaded (and, presumably, the document is read). Note that file downloading is a&lt;br&gt;
502much stronger statement on the part of the user than, for example, having a&lt;br&gt;
503bibliographic record appear in the query result set for a conventional bibliographic&lt;br&gt;
504system; the user downloads only &lt;i&gt;after&lt;/i&gt; the document has been found potentially relevant&lt;br&gt;
505through examination of its document surrogate. Additionally,  downloading is&lt;br&gt;
506frequently time-consuming  and sometimes costly (depending on local pricing for&lt;br&gt;
511  <Description>
512    <Metadata name="Title">13</Metadata>
513  </Description>
514  <Content>&lt;br /&gt;
515Internet access). Downloaded documents are therefore highly likely at least to be&lt;br&gt;
516scanned, if not read closely.  The transaction logs for a digital library can provide a&lt;br&gt;
517global picture of the use of documents in the collection, since all user interactions with&lt;br&gt;
518the library can be automatically logged  for analysis. By contrast, it is of course&lt;br&gt;
519impossible to track usage of print bibliographies, and very difficult to monitor usage of&lt;br&gt;
520bibliographic data available on CD-ROM across more than one or two sites.&lt;br&gt;
521Furthermore, analysis of search requests by geographic location, institution,&lt;br&gt;
522and sometimes even individual user are also possible.  As an example, Table 2 presents&lt;br&gt;
523a portion of the summary of usage statistics (broken down by domain code) for queries&lt;br&gt;
524to the computer science technical collection of the NEW ZEALAND DIGITAL LIBRARY.&lt;br&gt;
525Examination of the data indicates that the heaviest use of the collection comes from&lt;br&gt;
526North America, Europe (particularly Germany and Finland), as well as the local New&lt;br&gt;
527Zealand community and nearby Australia.  As expected for such a collection, a large&lt;br&gt;
528proportion of users are from educational (.edu) institutions; surprisingly, however, a&lt;br&gt;
529similar number of queries come from commercial (.com) organizations, indicating&lt;br&gt;
530perhaps that the documents are seeing use in commercial research and development&lt;br&gt;
532Table 2. Accesses to the NEW ZEALAND DIGITAL LIBRARY CS collection  by Domain&lt;br&gt;Code&lt;br&gt;
533Of course, usage levels can also be further broken down by IP number&lt;br&gt;
534(indicating  institutions), and systems requiring users to register may also be able to&lt;br&gt;
535analyze usage on an individual basis. Since the query strings themselves are also&lt;br&gt;
536recorded in the transaction logs, this domain/institution/individual activity could also be&lt;br&gt;
537linked to specific subjects through the query terms.  Summaries of this type could be&lt;br&gt;
538invaluable for studies of geographic diffusion and distribution of research topics.&lt;br&gt;
539Transaction log analysis can also indicate time-related  patterns in the&lt;br&gt;
540information seeking behavior of digital library users.   As a sample of this type of&lt;br&gt;
541analysis,  Paul Ginsparg notes a seven day periodicity in the number of search requests&lt;br&gt;
546  <Description>
547    <Metadata name="Title">14</Metadata>
548  </Description>
549  <Content>&lt;br /&gt;
550made to the PHYSICS E-PRINT archives (Figure 4, reproduced from [9]).  From this he&lt;br&gt;
551adduces that many physicists do not yet have weekend access to the Internet (an&lt;br&gt;
552alternative, slightly more cynical hypothesis is that even high energy theoretical&lt;br&gt;
553physicists take the weekend off).&lt;br&gt;
554Figure 4.  Summary of search requests to the physics pre-print archives&lt;br&gt;
555&lt;b&gt;5.  Conclusion&lt;/b&gt;&lt;br&gt;
556This study suggests opportunities for conducting bibliometric research on the&lt;br&gt;
557evolving digital libraries.  These repositories are suitable platforms for conventional&lt;br&gt;
558bibliometric techniques (such as obsolescence studies, quantification of physical&lt;br&gt;
559characteristics of documents comprising a subject literature, time analysis, etc.).  The&lt;br&gt;
560ability to directly monitor access to documents in digital libraries also enables&lt;br&gt;
561researchers to explicitly quantify document usage, as well as to implicitly measure&lt;br&gt;
562usage through citations.  Additional facilities could aid in the performance of&lt;br&gt;
563bibliographic experiments, such as: improved tagging of document fields; provision of&lt;br&gt;
564utilities to strip out titles, authors, etc. from common document formats; and the ability&lt;br&gt;
565to easily eliminate duplicate entries from downloaded library subsets.  Unfortunately,&lt;br&gt;
566the most useful of these additional facilities – those associated with a higher degree of&lt;br&gt;
567cataloging – run counter to the underlying philosophy of many digital libraries:  to&lt;br&gt;
568avoid, if possible,  manual processing and formal cataloging of documents.   While&lt;br&gt;
569adherence to this principle can limit the accuracy of fielded searching (or indeed,&lt;br&gt;
570preclude it altogether), it can also avoid the cataloging bottleneck and permit digital&lt;br&gt;
571libraries to provide access to larger numbers of documents.&lt;br&gt;
572The digital libraries complement the information currently available through&lt;br&gt;
573paper, online, and CD-ROM bibliographic resources.  While these latter databases&lt;br&gt;
574generally have the advantage of standardized formatting of bibliographic fields, the&lt;br&gt;
575digital libraries are freely accessible, often contain &amp;quot;grey literature&amp;quot; that is otherwise&lt;br&gt;
580  <Description>
581    <Metadata name="Title">15</Metadata>
582  </Description>
583  <Content>&lt;br /&gt;
584unavailable for analysis, and generally make the full text of documents available.  The&lt;br&gt;
585insights gained from analysis of digital libraries will add to the store of &amp;quot;information&lt;br&gt;
586about information&amp;quot; that we have gained from older types of bibliographic repositories.&lt;br&gt;
588[1] Bollacker, K.D., S. Lawrence, and C.L.Giles, CiteSeer: An Autonomous Web&lt;br&gt;
589Agent for Automatic  Retrieval  and  Identification  of  Interesting  Publications,&lt;br&gt;
590&lt;i&gt;Proceedings of the Second International Conference  on  Autonomous  Agents&lt;/i&gt;&lt;br&gt;
591(Minneapolis/St. Paul, May 9-13), 1998.&lt;br&gt;
592[2] Bowman, C.M., P.B. Danzig, U. Manber,  and M.F. Schwartz,  Scalable Internet&lt;br&gt;
593resource discovery:  Research problems and approaches,  &lt;i&gt;Communications of&lt;/i&gt;&lt;br&gt;
594&lt;i&gt;the ACM 37(8)&lt;/i&gt;  (1994)  98-107.&lt;br&gt;
595[3] Burton, Hilary D. , Use of a virtual information system for bibliometric analysis,&lt;br&gt;
596&lt;i&gt;Informaton Processing &amp;amp; Management 24(1)&lt;/i&gt;  (1988) 39-44.&lt;br&gt;
597[4] Cunningham, S.J., An empirical investigation of the obsolescence rate for&lt;br&gt;
598information systems literature, &lt;i&gt;Library and Information Science&lt;/i&gt;&lt;br&gt;
599&lt;i&gt;Research&lt;/i&gt;., 1996,;br&gt;
600 [5]  Cunningham,  S.J.,  and  D.  Bocock,    Obsolescence  of  computing  literature.&lt;br&gt;
601&lt;i&gt;Scientometrics&lt;/i&gt;  &lt;i&gt;34(2) &lt;/i&gt; (1995), pp. 255-262.&lt;br&gt;
602 [6] Cunningham, S.J. and Lynn Silipigni Connaway, Information searching&lt;br&gt;
603preferences and practices of computer science researchers, &lt;i&gt;Proceedings of&lt;/i&gt;&lt;br&gt;
604&lt;i&gt;OZCHI '96&lt;/i&gt; (1996)  294-299.&lt;br&gt;
605[7] de Solla Price, D.J.,  Citation measures of hard science, soft science, technology,&lt;br&gt;
606and nonscience.  In: C.E. Nelson  and  D.K.  Pollock  (eds),  &lt;i&gt;Communication&lt;/i&gt;&lt;br&gt;
607&lt;i&gt;among scientists and engineers&lt;/i&gt;  (Heath Lexington, 1970).&lt;br&gt;
608[8]  Garfield, E., &lt;i&gt;Citation Indexing:  Its theory and application in Science, Technology&lt;/i&gt;&lt;br&gt;
609&lt;i&gt;and Humanities (&lt;/i&gt;Wiley, 1979).&lt;br&gt;
614  <Description>
615    <Metadata name="Title">16</Metadata>
616  </Description>
617  <Content>&lt;br /&gt;
618[9]  Ginsparg, P.  After dinner remarks:  14 Oct ‘94 APS meeting at LANL, 1994&lt;br&gt;
619(&amp;lt;URL:;gt; ).&lt;br&gt;
620[10] Ginsparg, P., First steps towards electronic research communication, &lt;i&gt;Computers&lt;/i&gt;&lt;br&gt;
621&lt;i&gt;in Physics 8(4)&lt;/i&gt; (1994)  390-401.  &lt;br&gt;
622[11] Hallmark,  J., Scientists' access and retrieval of references cited in their recent&lt;br&gt;
623journal articles, &lt;i&gt; College and Research Libraries 55(3)&lt;/i&gt;  (1994) 199-210.&lt;br&gt;
624[12] Hawkins, D.T. , Unconventional uses of on-line information retrieval systems:&lt;br&gt;
625on-line bibliometric studies,  &lt;i&gt;Journal of the American Society for Information&lt;/i&gt;&lt;br&gt;
626&lt;i&gt;Science 28&lt;/i&gt;  (1977)  13-18.&lt;br&gt;
627[13] McGhee, P.E. , P.R. Skinner, K. Roberto,  N.J. Ridenour,  and S.M. Larson,&lt;br&gt;
628Using online databases to study current research trends:  an online bibliometric&lt;br&gt;
629study, &lt;i&gt;Library and Information Science Research 9&lt;/i&gt;  (1987)   285-291.&lt;br&gt;
630[14] Maly, K., E.A. Fox,  J.C. French,  and A.L. Selman,  Wide area technical report&lt;br&gt;
631server    (&lt;i&gt;Technical    Report  ,  &lt;/i&gt;  Dept.  of  Computer  Science,  Old  Dominion&lt;br&gt;
632University, &lt;br&gt;
6331994. &lt;br&gt;
634Also &lt;br&gt;
635available &lt;br&gt;
636at &lt;br&gt;
637 &lt;br&gt;
638 &lt;br&gt;
640;gt; ).&lt;br&gt;
641[15] Sigogneau, M.J. , S. Bain, J.P. Courtial, and H. Feillet,  Scientific innovation in&lt;br&gt;
642bibliographical databases:  a comparative study of the Science Citation Index&lt;br&gt;
643and the Pascal database,  &lt;i&gt;Scientometrics 22(1)&lt;/i&gt;   (1991)  65-82.&lt;br&gt;
644[16] Witten, I.H., S.J. Cunningham, M. Vallabh,  and T.C. Bell,  A New Zealand&lt;br&gt;
645digital library for computer science research,  &lt;i&gt;Proceedings of Digital Libraries&lt;/i&gt;&lt;br&gt;
646&lt;i&gt;'95&lt;/i&gt; (1995) 25-30.&lt;br&gt;
647[17]  Witten, I.H., C. Nevill-Manning, and S.J. Cunningham, A public library based&lt;br&gt;
648on full-text retrieval, &lt;i&gt;Communications of the ACM&lt;/i&gt; 41(4), 1998, p. 71&lt;br&gt;
653  <Description>
654    <Metadata name="Title">17</Metadata>
655  </Description>
656  <Content>&lt;br /&gt;
657                                    &lt;br&gt;
6581Documents were randomly sampled from the DEC&lt;br&gt;
659(, Sony&lt;br&gt;
660(, and Ohio (ftp://archive.cis.ohio-&lt;br&gt; technical report repositories&lt;br&gt;
Note: See TracBrowser for help on using the browser.