source: trunk/gsdl3/docs/manual/manual.tex@ 3711

\documentclass[a4paper,11pt]{article}
\usepackage{times,epsfig}
\hyphenation{Message-Router Text-Query}

\begin{document}

\title{A modular digital library:\\
       Architecture and implementation of Greenstone3}

% if you work on this manual, add your name here
\author{Katherine Don and Ian H. Witten \\[1ex]
        Department of Computer Science \\
         University of Waikato \\ Hamilton, New Zealand \\
        \{kjdon, ihw\}@cs.waikato.ac.nz}

\date{}

\maketitle

\newenvironment{bulletedlist}%
{\begin{list}{$\bullet$}{\setlength{\itemsep}{0pt}\setlength{\parsep}{0pt}}}%
{\end{list}}


\noindent
Greenstone Digital Library Version 3 is a complete redesign and
reimplementation of the Greenstone digital library software.  The current
version (Greenstone2) enjoys considerable success and is being widely used.
Greenstone3 will capitalize on this success, and in addition it will
\begin{bulletedlist}
\item improve flexibility, modularity, and extensibility
\item lower the bar for ``getting into'' the Greenstone code with a view to
   understanding and extending it
\item use XML where possible internally to improve the amount of
   self-documentation
\item make full use of existing XML-related standards and software
\item provide improved internationalization, particularly in terms of sort order,
   information browsing, etc.
\item include new features that facilitate additional ``content management''
   operations
\item operate on a scale ranging from personal desktop to corporate library
\item easily permit the incorporation of text mining operations
\item use Java, to encourage multilinguality, X-compatibility, and to permit
   easier inclusion of existing Java code (such as for text mining).
\end{bulletedlist}
Parts of Greenstone will remain in other languages (e.g. MG, MGPP); JNI (Java
Native Interface) will be used to communicate with these.


\section{Architecture}

This section is covered by the paper: An agent based architecture for dynamic digital library construction and configuration. Either cut and paste it in here, or link to the text?? or have two separate docs. dont want to have to maintain two separate versions of the same thing.

\section{Greenstone Implementation}
\label{sec:impl}

\subsection{Configuring Greenstone}
\label{subsec:config}

Greenstone3 involves several different kinds of configuration files, all
expressed in XML. Each site has a configuration file that binds parameters for
the site, {\em siteConfig.xml}.  Each collection has two configuration files, {\em collectionConfig.xml} and {\em buildConfig.xml\/}, that give metadata for the
collection.\footnote{These replace {\em collect.cfg} and {\em build.cfg} in
Greenstone2.}  The first includes user-defined metadata for the collection,
such as its name and the {\em About this collection} text; and also gives
instructions on how the collection is to be built.  The second is produced by
the build-time process and includes any metadata that can be determined
automatically.\footnote{Currently only the buildConfig.xml file is used - collections are built using gs2 style building and therefore use the old collect.cfg.}

\subsubsection{Site configuration file}

The file {\em siteConfig.xml} specifies the URI for the site ({\em
localSiteName\/}), any services or service clusters provided by the site that are not connected
with a particular collection (for example, translation services, or collection building), and a list of
known external sites to connect to.  Collections are not specified in the site
configuration file, instead they are determined by the contents of the site's
collections directory.

Here is a configuration file for a rudimentary site with no site-wide services,
which does not connect to any external sites.\footnote{should the code be tolerant of missing elements? or do we require empty elements?}
\begin{quote}\begin{footnotesize}\begin{verbatim}
<config>
  <localSiteName value="org.greenstone.localsite"/>
  <serviceClusterList/>
  <serviceRackList/>
  <siteList/>
</config>
\end{verbatim}\end{footnotesize}\end{quote}
The following configuration file is for a site with one site-wide service cluster - a collection building cluster.  It also connects to the previous site using SOAP.
\begin{quote}\begin{footnotesize}\begin{verbatim}
<config>
  <localSiteName value="org.greenstone.gsdl1"/>
  <serviceRackList/>
    <servicesImpl name="TranslationServices"/>
  </servicesImplList>
  <serviceClusterList>  
    <serviceCluster name="build">
      <metadataList>
        <metadata name="Title">Collection builder</metadata>
        <metadata name="Description">Builds collections in a gsdl2-style manner</metadata>
      </metadataList>
      <serviceRackList>
        <serviceRack name="GS2Construct"/>
      </serviceRackList>
    </serviceCluster>
  </serviceClusterList>
  <siteList>
    <site name="org.greenstone.localsite"
      address="http://localhost:8080/soap/servlet/rpcrouter"
      type="soap"/>
  </siteList>
</config>
\end{verbatim}\end{footnotesize}\end{quote}

These two sites are running on the same machine. For site1 to talk to localsite, a SOAP server must be run for localsite. The address of the SOAP server, in this case, is "http://localhost:8080/soap/servlet/rpcrouter"

\subsubsection{Building configuration file}

The file {\em buildConfig.xml} contains all metadata and other information about the collection that can
be determined automatically when building the collection, such as the number of
documents it contains.  It also includes a list of serviceRack classes that are
required at runtime to provide the services that have been built into the
collection.  The serviceRack names are Java classes that are loaded
dynamically at runtime. Any information inside the serviceRack element is
specific to that service---there is no set format.  Here is an example:

\begin{quote}\begin{footnotesize}\begin{verbatim}

<buildConfig>
  <metadataList>
    <metadata name="numDocs">11</metadata>
    <metadata name="colIcon">mgppdemo.gif</metadata>
    <metadata name="colName">Greenstone demo collection</metadata>
    <metadata name="colDescription">This is a demonstration collection for the Greenstone digital library software. It contains a small subset  of the Humanitarian and Development Libraries.</metadata>
  </metadataList>
  <serviceRackList>
    <serviceRack name="GS2MGPPRetrieve">
      <defaultLevel name="Section"/>
    <!-- something list this should be used to advertise what metadata the collection has available to be retrieved - however, it is not used yet -->
      <metadataList> 
        <element name="Title"/><element name="Subject"/><element name="Organization"/><element name="URL"/>
      </metadataList>
    </serviceRack>
    <serviceRack name="GS2MGPPSearch">
      <defaultIndex name="tt"/>
      <defaultLevel name="Section"/>
      <levelList>
        <level name="Document"/>
        <level name="Section"/>
      </levelList>
      <indexList>
        <index name="tt"/>
        <index name="t0"/>
      </indexList>
      <fieldList>
        <field name="TX"/><field name="SU"/><field name="TI"/>
      </fieldList>
    </serviceRack>
    <serviceRack name="PhindPhraseBrowse"/>
    <serviceRack name="GS2Browse">
      <classifierList>
        <classifier name="CL1"><metadataList><metadata name="Title">Subject</metadata></metadataList></classifier>
        <classifier name="CL2" ><metadataList><metadata name="Title">Title</metadata></metadataList></classifier>
    <classifier name="CL4"><metadataList><metadata name="Title">Organization</metadata></metadataList></classifier>
    <classifier name="CL5" ><metadataList><metadata name="Title">Keyword</metadata></metadataList></classifier>
      </classifierList>
    </serviceRack>
  </serviceRackList>
</buildConfig>   
\end{verbatim}\end{footnotesize}\end{quote}
Note: because {\em collectionConfig.xml} is not used yet, the {\em colIcon}, {\em colDescription}
and {\em colName} metadata elements have been specified here.

\subsubsection{Collection configuration file}

The format of {\em collectionConfig.xml} has not yet been defined.

\subsubsection{Starting up}

We use the Tomcat web server, which operates either stand-alone in a test mode
or in conjunction with the Apache web server.  The Greenstone LibraryServlet
class is loaded by Tomcat  and the servlet's {\em init()} method is called.  Each time a
{\em get\/}/{\em put\/}/{\em post} (etc.) is used, a new thread is started and
{\em doGet()\/}/{\em doPut()\/}/{\em doPost()} (etc.) is called.

The {\em init()} method creates a new Receptionist and a new instance of the
MessageRouter. The appropriate system variables are set in each (interface
name, site name, etc.) and then {\em configure()} is called. A MessageRouter
reference is given to the Receptionist. The servlet then communicates only with
the Receptionist, not with the MessageRouter.

The Receptionist loads up all the different Action classes. A
static list is used initially, and other Actions may be loaded on the fly as needed.

The MessageRouter reads in its site configuration file {\em siteConfig.xml}. This
lists the ServiceRack classes that need to be loaded, and lists any sites that need
to be connected to.  It looks inside the {\em collect} directory which contains
all the site's collections and loads up a Collection object for each valid
collection found.

The Collection object reads its {\em buildConfig.xml} and {\em collectionConfig.xml}
files, determines the metadata, and loads ServiceRack classes based on the 
names specified in {\em buildConfig.xml\/}. The {\footnotesize \verb#<ServiceRack>#} XML element is passed to the object to be used in  configuration.

\section{System messages}

Once the system is up and running (the configuration
process described in Section~\ref{subsec:config} has been carried out), it is passing messages back and forth. All modules communicate via message passing.
 First, we examine the basic message
formats, then how the system creates and responds to the messages.

All messages are enclosed in
\begin{quote}\begin{footnotesize}\begin{verbatim}
<message>
\end{verbatim}\end{footnotesize}\end{quote}
Messages contain either {\em <request>\/} or {\em <response>\/} elements--- a single message may contain multiple requests. Each {\em <request>\/} (and {\em <response>\/}?) has a language attribute, of the form ``lang='xx'''.
The language attribute is used by the XSLT to determine the language currently
being used by the user interface.  Virtually all messages contain text strings,
and services use this attribute to return strings in the appropriate language.

There are two different styles of messaging, explained in the two subsections
below.  The first is the communication between the servlet (or other external agent) and the Greenstone system (via the Receptionist). The request contains a simple representation of the arguments in a Greenstone URL, and has the same format as any request in the system.  The response is a page of data, typically in HTML.  The second style of messaging is the internal Greenstone communication. Requests and responses follow a basic format, and both are in XML.\footnote{We format names in lower case with the first letter of internal words capitalized, like 'matchDocs'.} They typically request one service or one action, and the response contains either the data requested, or a status message.

This section describes the two message formats. The following section looks at how the front-end (Receptionist plus Actions) responds to the URL-type messages, and creates internal xxx-type\footnote{are there good names to distinguish the two types of messages?} messages to pass into the system.

\subsubsection{Servlet to Receptionist messages}\label{subsec:url-type}

Servlet to Receptionist messages are requests for a 'page' of data---for example, the home page for a site; the query page for a collection; the text of a document. They contain, in XML, a representation of the arguments in a
Greenstone URL.  The two main arguments are {\em a} (action) and {\em sa}
(subaction).\footnote{The {\em sa} replaces Greenstone's old {\em p} arg for
the page action, and is new for other actions.  For example, a text query could
be encoded as {\em a=q \& sa=text\/}.}  All other arguments are treated as
parameters.

Here is the XML representation of the arguments:

\begin{quote}\begin{footnotesize}\begin{verbatim}
<request type='cgi' action='a-arg-value' subaction='sa-arg-value'
         lang='en' output='html'>
  <paramList>
    <param name='xx' value=''yyy'/>
    <param name=...
  </paramList>
</request>
\end{verbatim}\end{footnotesize}\end{quote}
The receptionist routes the message to the appropriate action.  The output
field is used to indicate what type of output to return. The actions do not
return responses in the normal format; instead they return a page of
information, expressed by default in HTML. Alternative formats could be XML or WML.

The LibraryServlet class communicates with the Receptionist, which is the entry
point into the system.  Future GUIs could communicate either with the
Receptionist or directly with the MessageRouter. If they communicate with the Receptionist they must use the cgi-args type of request, asking for predefined pages of information.  If they communicate with the MessageRouter directly, they must use the internal message format described in the next section---this is more powerful, but involves more work by the client. Individual services are requested---the results need to be put together by the client.

The cgi arguments used currently are shown in Table~\ref{tab:args}.
Other arguments can be specified by  particular actions.. For example, when the query action recieves a list of parameters from the TextQuery service, it creates short names for them and adds them to the global list of cgi-args. 

\begin{table}
\center{\footnotesize
\begin{tabular}{llll}
\hline
\bf Argument & \bf Meaning &\bf Typical values \\
\hline
a & action & a (applet), q (query), b (browse), p (page), pr (process) \\
sa & subaction & home, about (page action)\\
c & collection or service cluster & demo, build \\
s & service name & TextQuery, ImportCollection \\
rt & request type & d (display), r (request), s (status) \\
ro & request only & 0 or 1 - if set to one, the request is carried out but no processing of the results is done \\
o & output type & xml, html, wml \\
l & language & en, fr, zh \\
d & document id & HASHxxx \\
r & resource id & ???\\
id & process handle & an integer identifying a particular process request \\
\hline
\end{tabular}}
\label{tab:args}
\caption{Generic rguments that can appear in a Greenstone URL}
\end{table}

Here is an example message that retrieves the home page in French:
\begin{quote}\begin{footnotesize}\begin{verbatim}
<message>
  <request lang='fr' type='cgi' action='p' subaction='home' output='html'/>
</message>
\end{verbatim}\end{footnotesize}\end{quote}

This message represents a text query:
\begin{quote}\begin{footnotesize}\begin{verbatim}
<message>
  <request  lang='en' type='cgi' action='q'  output='html'>
  <paramList>
    <param name='s' value='TextQuery'/>
    <param name='c' value='demo'/>
    <param name='rt' value='r'/>
    <!-- the rest are the service specific params -->
    <param name='ca' value='0'/> <!-- casefold -->
    <param name='st' value='1'/> <!-- stem -->
    <param name='m' value='10'/> <!-- maxdocs -->
    <param name='q' value='snail'/> <!-- query string -->
  </paramList>
</message>
\end{verbatim}\end{footnotesize}\end{quote}

**** UP TO HERE **************
\subsubsection{Module to module messages}

In Greenstone3's modular architecture messages are used extensively to pass
information from one module to another, for example from an Action to the
MessageRouter module, and from that module to a service module.  Requests have
a {\em to} attribute and responses have {\em from\/}.  These are addresses used
by routing modules.  For example {\em to='site1/site2/demo/TextQuery'} routes a
message to a MessageRouter ({\em site1\/}), from there to another MessageRouter
({\em site2\/}), from there to a collection ({\em demo\/}), and from there to a
particular service ({\em TextQuery\/}).

Each request asks for a description of a single module, or requests a particular service. Unlike the first type of message which requests pre-defined types of pages, these internal requests can ask for any functionality available in the system.

The most basic message is ``describe-yourself'', which can be sent to any module in the system. The module responds with a predefined piece of XML, making these requests very efficient.
\begin{quote}\begin{footnotesize}\begin{verbatim}
<message>
  <request lang='en' type='describe' to=''/>
</message>
\end{verbatim}\end{footnotesize}\end{quote}
If the {\em to} field is empty, the request is answered by the first module that it is passed to.
An example response from a MessageRouter might look like this:
\begin{quote}\begin{footnotesize}\begin{verbatim}
<message>
  <response lang='en' type='describe'>
    <serviceList>
      <service name='CrossCollectionSearch' type='query' />
    </serviceList>
    <siteList>
      <site name='org.greenstone.gsdl1'
            address='http://localhost:8080/soap/servlet/rpcrouter'
            type='soap' />
    </siteList>
    <collectionList>
      <collection name='org.greenstone.gsdl1/
                  org.greenstone.gsdl2/fao' />
      <collection name='org.greenstone.gsdl1/demo' />
      <collection name='org.greenstone.gsdl1/fao' />
      <collection name='myfiles' />
    </collectionList>
  </response>
</message>
\end{verbatim}\end{footnotesize}\end{quote}
This MessageRouter has one site-wide service, a cross-collection searching service. It
communicates with one site, {\em org.greenstone.gsdl1\/}.  It is aware of four
collections.  One of these, {\em myfiles\/}, belongs to it; the other three are
available through the external site.  One of those collections is actually from
a further external site.

It is possible to ask just for a specific part of the information provided by a
describe request, rather than the whole message.  For example, these two
messages get the {\em collectionList} and the {\em siteList} respectively:
\begin{quote}\begin{footnotesize}\begin{verbatim}
<message lang='en'>
  <request type='describe' to='' info='collectionList'/>
</message>

<message lang='en'>
  <request type='describe' to='' info='siteList'/>
</message>
\end{verbatim}\end{footnotesize}\end{quote}
When a collection is asked to describe itself, what is returned is all of the 
collection specific metadata and a list of services.  For example, here is such
a message, along with a sample response.

\begin{quote}\begin{footnotesize}\begin{verbatim}
<message lang='en'>
  <request type='describe' to='demo'/>
</message>

<message>
  <response lang='en' type='describe' from='demo' >
    <collection name='demo'>
      <serviceList>
        <service name='TextQuery' type='query' />
        <service name='DocRetrieve' type='query' />
        <service name='MetadataRetrieve' type='query' />
      </serviceList>
      <metadataList>
        <metadata name='numDocs'>321</metadata>
        <metadata name='numSections'>5532</metadata>
        <metadata name='title'>The demo collection</metadata>
        <metadata name='aboutText'>This is a demo collection.</metadata>
      </metadataList>
    </collection>
  </response>
</message>
\end{verbatim}\end{footnotesize}\end{quote}
A {\em describe} request sent to a service returns a list of parameters that
the service accepts, and describes the content type for the request and
response.

Parameters have the following format:
\begin{quote}\begin{footnotesize}\begin{verbatim}
<param name='xxx' type='integer|boolean|string' default='yyy'/>
<param name='xxx' type='enum_single|enum_multi' default='aa'/>
  <option name='aa'/><option name='bb'/>...
</param>
<param name='xxx' type='multi' occurs='4'>
 <param .../>
 <param .../>
</param>
\end{verbatim}\end{footnotesize}\end{quote}
If no default is specified, the parameter is assumed to be mandatory.
Here are some examples of parameters:
\begin{quote}\begin{footnotesize}\begin{verbatim}
<param name='Case' type='boolean' default='0'/>

<param name='MaxDocs' type='integer' default='50'/>

<param name='Index' type='enum' default='dtx'>
  <option name='dtx'/>
  <option name='stt'/>
  <option name='stx'/>
<param>

<!-- this one is for the text box and field list for the simple field query-->
<param name='simple' type='multi' occurs='4'>
  <param name='fqv' type='string'/>
  <param name='fqf' type='enum_single'>
    <option name='TI'/><option name='AU'/><option name='OR'/>
  </param>
</param>

\end{verbatim}\end{footnotesize}\end{quote}
Here is a message, along with a sample response.
\begin{quote}\begin{footnotesize}\begin{verbatim}
<message>
  <request lang='en'  type='describe' to='demo/TextQuery'/>
</message>

<message>
  <response lang='en' type='describe' from='demo/TextQuery' >
    <service name='TextQuery' type='query'>
    <paramList>
      <param name='matchDocs' type='integer' default='50/>
      <param name='case' type='boolean' default='1'/>
      <param name='index' type='enum' default='tt'>
        <option name='tt'/>
    <option name='t0'/>
      </param>
    </paramList>
  </response>
</message>
\end{verbatim}\end{footnotesize}\end{quote}

So far, we have only looked at ``describe'' requests. These can be asked of any module. Other requests are ``configure'' requests, and requests for services.

``Configure'' requests are used to tell the MessageRouter to update its cached information and activate or deactivate other modules. For example, the MessageRouter has a set of Collection modules that it can talk to. It also holds some XML information about those collections---this is returned when a request for a collection list comes in. If a collection is deleted or modified, or a new one created, this information may need to change, and the list of available modules may also change.

So far, we have {\em activate} and {\em deactivate} configure requests.
Some examples are as follows.
\begin{quote}\begin{footnotesize}\begin{verbatim}
<message><request type='configure' to=''>
<configure action='deactivate' type='collection' name='demo'/>
</request></message>

<message><request type='configure' to=''>
<configure action='activate' type='collection' name='demo'/>
</request></message>

<message><request type='configure' to=''>
<configure action='activate' type='serviceRack' 
           name='TranslationServices'/>
</request></message>
\end{verbatim}\end{footnotesize}\end{quote}

The first request is used to remove a collection from the running system once it has been physically deleted. The Collection module is removed from the module list, and information about the collection is removed from the collection list XML. The second request is used when the demo collection has either been modified, or has been newly created. The MessageRouter first checks whether a Collection module of that name already exists, and if so deactivates it, as described above.  Then a new Collection module is created and configured, and information added into the XML tree. The final request (re)activates the services provided by the serviceRack class TranslationServices. The site config file is re-read, and the appropriate element used for configuration of the new serviceRack object. As for collections, if one already exists, it is deactivated first.

The response to a configure request is a status or an error message. No data is sent back, just success or error. An example is:
\begin{quote}\begin{footnotesize}\begin{verbatim}
<message><response from='' type='configure'>
  <status>demo collection activated</status>
</response></message>
\end{verbatim}\end{footnotesize}\end{quote}
\footnote{this format not properly defined yet}

Configure requests are only answered by the MessageRouter at this stage. It is possible that other modules may need to respond to these requests also. 

The main type of requests in the system are for services. There are different types of services: query, browse, retrieve, process, applet. Query services do some kind of search and return a list of documents. Retrieve services can return those documents, metadata about the documents, or other resources. Browse is for browsing lists or hierarchies of documents. process type services are those where the request is for a command to be run. A status code will be returned immediately, and then if the command has not finished, an update of the status can be requested. Applet services are those that run an applet.

  Other possibilities include  transform, enrich, extract, accrete. These types of service generally enhance the functionality of the first set. They may be used during collection formation: 'accrete' documents by adding them to a collection, 'transform' the documents into a different format, 'extract' information or acronyms from the documents, 'enrich' those documents with the information extracted or by adding new information. They may also be used during querying: 'transform' a query before using it to query a collection, or 'transform' the documents you get back into an appropriate form.

The basic structure of a service request is as follows:
\begin{quote}\begin{footnotesize}\begin{verbatim}
<message>
  <request lang='en'  type='query' to='demo/TextQuery'>
    <paramList/>
    <content/>
  </request>
</message>
\end{verbatim}\end{footnotesize}\end{quote}

The parameters are name value pairs corresponding to parameters that were specified in the service description sent in response to a describe request. 

\begin{quote}\begin{footnotesize}\begin{verbatim}
<param name='case' value='1'/>
<param name='maxDocs' value='34'/>
<param name='index' value='dtx'/>
\end{verbatim}\end{footnotesize}\end{quote}

Some requests have a content---for document retrieval, the content is the list of documents to retrieve. For metadata retrieval, teh content is the list of documents, and a list of metadata to retrieve for each document.

Responses vary depending on the type of request. 
Responses to query requests contain a content, which is the actual result, along with some metadata about the query\footnote{is this called metadata or something else?}. For instance, a text query on 'snail farming', with the parameter 'maxDocs=10' might return the first 10 documents, and one of the query metadata items would be the total number of documents that matched the query.\footnote{no metadata about the query result is returned yet.}

The following shows some example query requests and their responses.

Find at most 10 Sections containing the word snail (stemmed), returning the results in unsorted order:
\begin{quote}\begin{footnotesize}\begin{verbatim}
<message>
  <request lang='en'  to="mgppdemo/TextQuery" type="query">
    <paramList>
      <param name="maxDocs" value="10"/>
      <param name="queryLevel" value="Section"/>
      <param name="stem" value="1"/>
      <param name="matchMode" value="some"/>
      <param name="sortBy" value="natural"/>
      <param name="index" value="t0"/>
      <param name="case" value="0"/>
    </paramList>
    <content>snail</content>
  </request>
</message>
\end{verbatim}\end{footnotesize}\end{quote}

\begin{quote}\begin{footnotesize}\begin{verbatim}
<message>
  <response lang='en' from="mgppdemo/TextQuery" type="query">
    <content>
      <documentList>
        <document name="HASH010f073f22033181e206d3b7"/>
        <document name="HASH010f073f22033181e206d3b7.2"/>
        <document name="HASHac0a04dd14571c60d7fbfd"/>
      </documentList>
    </content>
  </response>
</message>
\end{verbatim}\end{footnotesize}\end{quote}

Give me the Title metadata for these documents:
\begin{quote}\begin{footnotesize}\begin{verbatim}
<message>
  <request lang='en'  to="mgppdemo/MetadataRetrieve" type="retrieve">
    <content>
      <documentList>
        <document name="HASH010f073f22033181e206d3b7"/>
        <document name="HASH010f073f22033181e206d3b7.2"/>
        <document name="HASHac0a04dd14571c60d7fbfd"/>
      </documentList>
      <metadataList>
        <metadata name="Title"/>
      </metadataList>
    </content>
  </request>
</message>
\end{verbatim}\end{footnotesize}\end{quote}

\begin{quote}\begin{footnotesize}\begin{verbatim}
<message>
  <response lang='en' from="mgppdemo/MetadataRetrieve" type="retrieve">
    <content>
      <documentList>
        <document name="HASH010f073f22033181e206d3b7">
          <metadataList>
            <metadata name="Title">Farming snails 1: 
Learning about snails; Building a pen; Food and shelter plants
            </metadata>
          </metadataList>
        </document>
        <document name="HASH010f073f22033181e206d3b7.2">
          <metadataList>
            <metadata name="Title">Learning about snails</metadata>
          </metadataList>
        </document>
        <document name="HASHac0a04dd14571c60d7fbfd">
          <metadataList>
            <metadata name="Title">Farming snails 2: 
Choosing snails; Care and harvesting; Further improvement
            </metadata>
          </metadataList>
        </document>
      </documentList>
    </content>
  </response>
</message>
\end{verbatim}\end{footnotesize}\end{quote}

Give me the text for this document:
\begin{quote}\begin{footnotesize}\begin{verbatim}
<message>
  <request lang='en'   to="mgppdemo/DocumentRetrieve" type="retrieve">
    <content>
      <documentList>
        <document name="HASH010f073f22033181e206d3b7.2"/>
      </documentList>
    </content>
  </request>
</message>
\end{verbatim}\end{footnotesize}\end{quote}

\begin{quote}\begin{footnotesize}\begin{verbatim}
<message>
  <response lang='en' from="mgppdemo/DocumentRetrieve" type="retrieve">
    <content>
      <document name="HASH010f073f22033181e206d3b7.2">
        <content> 
&lt;/B&gt;&lt;P ALIGN=&quot;JUSTIFY&quot;&gt;&lt;/P&gt;
&lt;P ALIGN=&quot;JUSTIFY&quot;&gt;11. To farm snails is not hard; however, 
it is quite different from keeping chickens or ducks or from growing crops 
such as maize, rice, cassava or groundnuts.&lt;/P&gt;
&lt;P ALIGN=&quot;JUSTIFY&quot;&gt;&lt;/P&gt;
&lt;P ALIGN=&quot;JUSTIFY&quot;&gt;12. Since farming snails is so different 
from other kinds of farming, you will have to learn a lot of new things.
&lt;/P&gt;....
        </content>
      </document>
    </content>
  </response>
</message>
\end{verbatim}\end{footnotesize}\end{quote}

Build requests are not a request for data---they are a request for some action to be carried out, for example, create or import or build or activate a collection. The response is a status or an error message. The import and build commands may take a long time to complete, so a message is sent back after a successful start of the command. The status may be polled by the requester to see how the process is going. 

Build requests generally do not need a content, they just have a parameter list.\footnote{or is the collection the content?} Like any service, the parameters used by the service can be obtained by a describe request to that service.

Some example requests (note that the build services are grouped into a service cluster called 'build', hence the addresses all begin with 'build/'):

\begin{quote}\begin{footnotesize}\begin{verbatim}
<message>
  <request lang='en'  type='process' to='build/NewCollection'>
    <paramList>
      <param name='creator' value='[email protected]'/>
      <param name='collName' value='the demo collection'/>
      <param name='collShortName' value='demo'/>
    </paramlist>
  </request>
</message>

<message>
  <request lang='en'  type='process' to='build/ImportCollection'>
    <paramList>
      <param name='collection' value='demo'/>
    </paramlist>
  </request>
</message>
\end{verbatim}\end{footnotesize}\end{quote}


\subsection{Generating the pages}

URL-style requests are received by the Receptionist. Based on the arguments, a page of data must be returned to the servlet. As described in Section~\ref{subsec:url-type}, the requests are XML representations of Greenstone URLs. One of the arguments is action (a). This tells the Receptionist which Action module to pass the request to. Action modules decode the rest of the cgi-arguments to determine what requests need to be made to the system. 
System requests are received by the MessageRouter, which answers them one by one, either itself or by passing them on to the appropriate module. 

Once the data needed from the system has been accumulated, it is put into a 'page' of XML. The page is transformed to its output form, currently HTML, via XSLT transformations, and returned to the user.

The basic  page format  is:
\begin{quote}\begin{footnotesize}\begin{verbatim}
<page>
 <config/>
 <display/>
 <request/>
 <response/>
</page>
\end{verbatim}\end{footnotesize}\end{quote}

There are four main elements in the page: config, translate, request, response. The request is the original request that came into the Receptionist---this is included so that any parameters  can be preset to their previous values, for example, the query options on the query form.\footnote{this should be saved instead in some sort of state saving - if you leave a page and go back you want your parameters to be the same as well}. The response contains all the data that has been gathered from the system by the action. The other two elements contain extra information needed by XSLT. Config contains run-time variables such as the location of the gsdl home directory, the current site name, the name of the executable that is running (eg library)---these are needed to allow the XSLT to generate correct HTML URLs. Display contains some of the text strings needed in the interface---these are separate from the XSLT to allow for internationalization.

The following subsections outline, for each action, what data is needed and what requests are generated to send to the system. Following that, Section~\ref{subsec:xslt} describes the config and display information, and the xslt files.

\subsubsection{Page action}

Depending on the subaction argument, different pages can be generated. For the 'home' page, a 'describe' request is sent to the MessageRouter---this returns a list of all the collections, services, serviceClusters and sites known about. For each collection, its metadata is retrieved via a 'describe' request. This metadata is added into the previous result, which is then added into the page.  The page is
transformed using {\em home.xsl\/}.  For the 'about' page, a {\em
describe} request is sent to the module that the about page is about: this may be a collection or a service cluster.  This returns a list of metadata
and a list of services, and the result is transformed using {\em about.xsl\/}.

\subsubsection{Query action}

There are three query services which have been implemented: TextQuery, SimpleFieldQuery, and AdvancedFieldQuery. These are all handled in the same way by query action.
For each page, the service description is requested from the  service  of the current collection (via a describe request).  This is done every time the query page is
displayed.\footnote{This information should be cached.} The description includes a list of the parameters available for the query, such as case/stem, max num docs to return, etc. If the request type (rt) parameter is set to d for display, the action only needs to display the form, and this is the only request to the service. Otherwise, the submit button has been pressed, and a query request to the TextQuery service is sent. This has  all the parameters from the URL put into the parameter list. A list of document identifiers
is returned. A followup query is sent to the MetadataRetrieve service of the collection: the content includes the list of
documents, with a request for their {\em Title} metadata.  The service description and query result are combined into a page of xml, which is
transformed using {\em basicquery.xsl\/} to produce the html page.

\subsubsection{Applet action}

There are two types of request to the applet action: {\em a=a \& sa=d\/} and
{\em a=a \& sa=r\/}.  The value {\em sa=d\/} means ``display the applet.'' A
{\em describe} request is sent to the service, which returns the {\footnotesize \verb#<applet>#} HTML element.  The transformation file {\em applet.xsl} embeds this
into the page, and the servlet returns the HTML.

The value {\em sa=r} signals a request from the applet.  The result is returned
directly to the applet code, in XML.  The other parameters are sent to the
service untransformed, and the result is passed directly back to the applet.
Applet action can therefore work with any applet whose service understands the
messages.

Here are two examples of requests generated by the Applet action, along with their corresponding responses.

The first request corresponds to the URL arguments {\em a=a \&
sa=d \& sn=Phind \& c=mgppdemo\/}, which translate to ``display the Phind
applet for the mgppdemo collection''.

\begin{quote}\begin{footnotesize}\begin{verbatim}
<message>
  <request type='describe' to='mgppdemo/PhindApplet'/>
</message>

<message>
  <response type='describe'>
    <service name='PhindApplet' type='query'>
      <applet ARCHIVE='phind.jar, xercesImpl.jar, gsdl3.jar, 
            jaxp.jar, xml-apis.jar'
              CODE='org.greenstone.applet.phind.Phind.class'
              CODEBASE='lib/java'
              HEIGHT='400' WIDTH='500'>
        <PARAM NAME='library' VALUE=''/>
        <PARAM NAME='phindcgi' VALUE='?a=a&amp;sa=r&amp;sn=Phind'/>
        <PARAM NAME='collection' VALUE='mgppdemo' />
        <PARAM NAME='classifier' VALUE='1' />
        <PARAM NAME='orientation' VALUE='vertical' />
        <PARAM NAME='depth' VALUE='2' />
        <PARAM NAME='resultorder' VALUE='L,l,E,e,D,d' />
        <PARAM NAME='backdrop' VALUE='interfaces/default/
                      images/phindbg1.jpg'/>
        <PARAM NAME='fontsize' VALUE='10' />
        <PARAM NAME='blocksize' VALUE='10' />
        The Phind java applet.
      </applet>
    </service>
  </response>
</message>
\end{verbatim}\end{footnotesize}\end{quote}

The second request corresponds to the  arguments {\em a=a \& sa=r \& sn=Phind \& c=mgppdemo \& pc=1 \& pptext=health \& pfe=0 \& ple=10 \& pfd=0 \& pld=10 \& pfl=0 \& pll=10}---this 
indicates a request to the service itself. The extra arguments (not a, sa, sn, c)  are simply copied into the
request as parameters. The response is in a form suitable for the applet, placed inside
{\footnotesize \verb#<appletData>#} in a standard Greenstone message.  AppletAction returns the
contents of appletData to the browser, i.e. to the applet itself.

\begin{quote}\begin{footnotesize}\begin{verbatim}
<message>
  <request type='query' to='mgppdemo/PhindApplet'>
    <paramList>
      <param name='pc' value='1'/>
      <param name='pptext' value='health'/>
      <param name='pfe' value='0'/>
      <param name='ple' value='10'/>
      <param name='pfd' value='0'/>
      <param name='pld' value='10'/>
      <param name='pfl' value='0'/>
      <param name='pll' value='10'/>
    </paramList>
  </request>
</message>

<message>
  <response type='query' from='mgppdemo/PhindApplet'>
    <appletData>
      <phindData df='9' ef='46' id='933' lf='15' tf='296'>
        <expansionList end='10' length='46' start='0'>
          <expansion df='4' id='8880' num='0' tf='59'>
            <suffix> CARE</suffix>
          </expansion>
          ...
        </expansionList>
        <documentList end='10' length='9' start='0'>
          <document freq='78' hash='HASH4632a8a51d33c47a75c559' num='0'>
            <title>The Courier - N??159 - Sept- Oct 1996 Dossier Investing 
                   in People Country Reports: Mali ; Western Samoa
            </title>
          </document>
          ...
        </documentList>
        <thesaurusList end='10' length='15' start='0'>
          <thesaurus df='7' id='12387' tf='15' type='RT'>
            <phrase>PUBLIC HEALTH</phrase>
          </thesaurus>...
        </thesaurusList>
      </phindData>
    </appletData>
  </response>
</message>
\end{verbatim}\end{footnotesize}\end{quote}

Note that the applet HTML may need to know the name of the {\em library}
program.  However, that name is chosen by the person who installed the software
and will not necessarily be ``library''.  To get around this, the applet can
put a parameter called ``library'' into the applet data with a null value:
\begin{quote}\begin{footnotesize}\begin{verbatim}
<PARAM NAME='library' VALUE=''/>\/}
\end{verbatim}\end{footnotesize}\end{quote}
When the Applet action encounters this parameter it inserts the name of the
current library servlet as its value.

\subsubsection{Document action}

DocumentAction sends a query to the DocumentRetrieve service of the collection requesting the text of the specified document.  At this stage no additional information is obtained, but in future stuff like Title and
table of contents would be needed to make the display nicer.

\subsubsection{Formatting the page using XSLT}\label{subsec:xslt}

Once the xml page has been put together, the page to return to the user is created by transforming the XML using XSLT. The output is HTML at this stage, but it will be possible to generate alternative outputs, such as XML, WML etc. A set of XSLT files defines an 'interface'. Different users can change the look of their web pages by creating new XSLT files for a new 'interface'. Just as we have a sites directory where different sites 'live' (ie where their configuration file and collections are located), we have an interfaces directory where the different interfaces 'live' (ie their transforms and images are located there). The default XSLT files are
located in interfaces/default/transforms. Collections, sites and other interfaces
can override these files by having their own copy of the appropriate
files. New interfaces have their own directory inside interfaces/. Sites and collections can have a transform directory containing XSLT files. The order in which the XSLT files are looked for is collection, site, current
interface, default interface.\footnote{this currently breaks down for remote sites - need to rethink it a bit.}

\subsection{Internationalization}

Internationalization is a big part of Greenstone3. Language specific text strings are separated out from the rest of the system to allow for easy incorporation of new languages.

Language specific text strings are specified in resource bundle property files. These live in resources/java.

There is a properties file per class, and one per interface. At the moment, we have

GS2MGPPSearch.properties
GS2MGPPRetrieve.properties etc - the service classes

interface_default.properties. - for the default interface

To add other languages, create eg GS2MGPPSearch_fr.properties.

The interface ones are treated differently from the other ones. The action doesn't know which text strings are needed by a particular transform, so it gets them all out of the properties file, and puts them into an xml $<$display$>$ element - the xslt can get the ones it needs from there.
xslt could perhaps get the stuff from the properties bundle on the fly using java extension elements - would this be better?

All other class specific text strings are just retrieved one by one as they are needed and added into the xml - for example, the names for query params are retrieved when the service description is created.

\subsection{Collection formation}

Greenstone 2 compatible building has been implemented in gsdl3. so far only mgpp collections will work.

Collection construction can be done through the web, using the build servicecluster in localsite. Just sequence through the steps needed. So far, addDocument does not work, so documents need to be manually added to teh import directory.

You need to carry out the following services:
NewCollection
- add docs to import directory
ImportCollection
BuildCollection
ActivateCollection

If you want anything other than the default for the config file, you need to add it by hand - there is currently no ConfigureCollection service which would enable you to do this.

Collection building can also be done on the command line:

ConstructCollection -site <site-path> -mode new|import|build|activate [options] <coll-name>

eg

ConstructCollection -site /research/kjdon/home/gsdl3/sites/localsite -mode new -creator [email protected] testcol

the options get passed to the underlying script, - there is no good help message yet.

import and build use gs2 import.pl and buildcol.pl so you can specify any of their options if you like.

Building stuff is in src/java/org/greenstone/gsdl3/build.

CollectionConstructor is the base class for building control. GS2PerlConstructor is the implementation that uses greenstone 2 perl scripts. The building process sends events (ConstructionEvent) to any listeners (ConstructionListener) as important stages happen. You can add one or more listeners to the constructor which will get notified of events.

\section{Details}

This section describes the directory structure of the Greenstone source, and provides an installation guide to installing Greenstone from CVS.

\subsection{Directory structure}

The first part of Table~\ref{tab:dirs} shows the common stuff which can be shared between
Greenstone users---the src, libraries etc. These will eventually be installed into appropriate system directories. The second part shows
stuff used by one person/group---their sites and interface setup
etc. There can be several sites/interfaces per installation.

\begin{table}
\center{\footnotesize
\begin{tabular}{l p{7cm}}
\hline
gsdl3 
  & The main installation directory---gsdl3home can be changed to something more standard\\
gsdl3/src 
  & Source code lives here \\
gsdl3/src/java/org/greenstone/gsdl3   
  & Contains the top level classes that either have main programs, or are server/servlet classes\\
gsdl3/src/java/org/greenstone/gsdl3/core 
  & ModuleInterface, MessageRouter, Receptionist---the central classes that the others hang off\\
gsdl3/src/java/org/greenstone/gsdl3/service 
  & The various service modules---these things do the work\\
gsdl3/src/java/org/greenstone/gsdl3/util
  & Utility classes \\
gsdl3/src/java/org/greenstone/gsdl3/collection 
  & ServiceCluster and Collection classes\\
gsdl3/src/java/org/greenstone/gsdl3/comms 
  & Communicator classes, eg SOAP\\
gsdl3/src/java/org/greenstone/gsdl3/build 
  & stuff for collection building \\
gsdl3/src/java/org/greenstone/gsdl3/action 
  & Action classes used by the Receptionist---do the work of displaying the pages\\
gsdl3/src/java/org/greenstone/gsdl3/classes 
  & On compilation, the Java classes get put here---they can then be combined into a single jar file, and copied to the java lib directory \\
gsdl3/src/java/org/greenstone/gdbm 
  & Java wrapper for gdbm---uses j-gdbm, a jni gdbm wrapper\\
gsdl3/src/java/org/greenstone/testing
  & Junit scaffolding for unit testing.\\
gsdl3/src/cpp/
  & Place for any cpp source code---none yet \\
gsdl3/packages 
  & Imported packages from other systems eg mg, mgpp \\
gsdl3/lib
  & Shared library files\\
gsdl3/lib/java
  & Java jar files\\
gsdl3/resources 
 & any resources that may be needed\\
gsdl3/resources/java
 & properties files for java resource bundles - used to handle all the language specific text\\
gsdl3/bin
  & executable stuff lives here\\
gsdl3/bin/script
  & some perl building scripts\\
gsdl3/bin/linux
  & linux executables for eg mgpp\\
gsdl3/comms
  & Put some stuff here for want of a better place---things to do with servers and communication. eg soap stuff, and tomcat servlet container\\
gsdl3/docs
  & Documentation :-)\\
gsdl3/web
  & The place to put any web stuff that the servlet needs. html files go here\\
gsdl3/web/WEB-INF
  & The web.xml file lives here (configuration information for tomcat)\\
gsdl3/web/WEB-INF/classes
  & Servlet classes go in here\\
\hline
gsdl3/sites
  & Contains directories for different sites---a site is a set of collections and services served by a single MessageRouter (MR). The MR may have connections (eg soap) to other sites\\
gsdl3/sites/localsite 
  & One site\\
gsdl3/sites/localsite/collect 
  & The collections directory \\
gsdl3/sites/localsite/images 
  & Site specific images \\
gsdl3/sites/localsite/transforms 
  & Site specific transforms \\
gsdl3/interfaces
  & Contains all interface specific stuff (eg images and XSLT transforms\\
gsdl3/interfaces/default
  & The default interface\\
gsdl3/interfaces/default/images
  & The images\\
gsdl3/interfaces/default/transforms
  & The XSLT files\\
\hline
\end{tabular}}
\label{tab:dirs}
\caption{The Greenstone directory structure}
\end{table}

\subsection{Installation guide}

\newcommand{\gsdlhome}{\begin{footnotesize}{\em \$GSDL3HOME}\end{footnotesize}}

Cuurently, greenstone3 is only available through CVS. The installation procedure has  been automated. 

\subsubsection{Get the source}

\noindent If you have a greenstone\_cvs account, you can use the following:

\begin{footnotesize}\begin{tt}
\noindent export CVSROOT=:ext:{\em your-username}@cvs.scms.waikato.ac.nz:\\
\indent /usr/local/global-cvs/gsdl-src\\
export CVS\_RSH=ssh\\
cvs co gsdl3\\
\end{tt}\end{footnotesize}

\noindent Otherwise, you can get it through anonymous access:

\begin{footnotesize}\begin{tt}
\noindent export CVSROOT=:pserver:cvs\[email protected]:2402\\
\indent /usr/local/global-cvs/gsdl-src\\
export CVS\_RSH=ssh\\
cvs co gsdl3\\
\end{tt}\end{footnotesize}

\noindent If you need it, the password for anonymous CVS access is {\footnotesize \verb#anonymous#}.

\subsubsection{Compile and install greenstone}\label{subsec:compile}

An install.sh script has been constructed (thanks, Stuart) to compile and install greenstone 3. What you nee to do is:

\begin{footnotesize}\begin{tt}
cd gsdl3
source setup.bash
install.bash
source setup.bash
\end{tt}\end{footnotesize}

If you want to do greenstone2 compatible building (currently the only type) you need to have greenstone 2 installed, 'source setup.bash' in the top level greenstone 2 directory, then re-'source setup.bash' for greenstone 3. This is to set GSDLHOME for tomcat.

\noindent Note: 'source setup.bash' needs to be done once in any xterm window before doing a make or running tomcat. setup.bash sets the environment variables {\footnotesize \verb#CLASSPATH#, \verb#PATH#, \verb#JAVA_HOME#} etc.

If you want to use SOAP to talk to remote sites, you also need to do the following:

\begin{footnotesize}\begin{tt}
install-soap.bash  
\end{tt}\end{footnotesize}

Thats it. 

You dont want to run install.bash twice - it adds stuff into files

To update your installation, you can run update.bash - this remakes all the java stuff.


\subsubsection{The sample sites}

\noindent There are two greenstone ``sites'' that come with the checkout: localsite, and site1. localsite has several collections, only two of which have any actual data. The third is a dummy collection. site1 has one dummy collection. Each site has a configuration file which specifies the site name,  site-wide services if any, and a list of remote sites to connect to.
localsite does not connect to any other sites. site1 specifies a SOAP connection to localsite.

\noindent The collections which do not have data can be looked at but you cant do any queries on them.


\subsubsection{Tomcat}

\noindent Tomcat is a servlet container. It is used to serve a greenstone site using a servlet.
\\
\\
\noindent The file \begin{footnotesize}{\tt \gsdlhome/web/WEB-INF/web.xml}\end{footnotesize} contains the setup information for tomcat---tells it what servlets to load, what initial paramaters to pass them, and what web names map to the servlets.
There are three servlets specified in web.xml: one is a test servlet that just prints ``hello greenstone'' to a web page. This is useful if you are having trouble getting tomcat set up. The other two are greenstone library servlets, ``library'', which serves localsite, and ``library1'' which serves site1.
\\
\\
\noindent One initialisation parameter for the library servlets is {\footnotesize \verb#gsdl3home#}.
\begin{footnotesize}\begin{verbatim}
<init-param>
  <param-name>gsdl3home</param-name>
  <param-value>/research/kjdon/home/gsdl3</param-value>
</init-param>
\end{verbatim}\end{footnotesize}

The file \gsdlhome/comms/tomcat/jakarta/conf/server.xml is the tomcat configuration file. setup.bash adds a context for gsdl servlets - this tells tomcat where to find the web.xml file, and what url (eg /gsdl3) to give it.

\noindent Note: tomcat runs on port 8080 - you can change that if you wish in this file

\subsubsection{Serving your site using tomcat}\label{subsec:runtomcat}

\noindent To run tomcat, you need to have sourced {\footnotesize \verb#setup.bash#} in \gsdlhome\  to set up {\footnotesize \$CLASSPATH} (see \ref{subsec:compile}). Then, 

\begin{footnotesize}\begin{tt}
\noindent cd \gsdlhome/comms/tomcat/jakarta-tomcat-4.0.1/bin\\
./startup.sh
\end{tt}\end{footnotesize}

\noindent ({\footnotesize \verb#./shutdown.sh#} shuts down tomcat)
\\
\\
\noindent The tomcat server can be accessed on the web at {\footnotesize \verb#http://localhost:8080#}---this gets you to a welcome page.
The greenstone stuff is at {\footnotesize \verb#http://localhost:8080/gsdl3#}---this displays {\footnotesize \gsdlhome/web/index.html}. You should be able to run the test servlet and both library servlets from this page.

\noindent Note: tomcat must be shutdown and restarted any time you make changes in the following for those changes to take effect:\\
\begin{bulletedlist}
\begin{footnotesize}\begin{tt}
\item \gsdlhome/web/WEB-INF/web.xml
\item \gsdlhome/comms/tomcat/jakarta-tomcat-4.0.1/conf/server.xml
\end{tt}\end{footnotesize}
\item any classes or jar files used by the servlets
\end{bulletedlist}
\noindent Note: stdin and stdout for the servlets both go to\\
\begin{footnotesize}{\tt \gsdlhome/comms/tomcat/jakarta-tomcat-4.0.1/logs/catalina.out}\end{footnotesize}

\subsubsection{Using SOAP to talk to a remote site}

\noindent The previous installation stuff is fine if you only want to talk to local sites. However, if you want to connect using SOAP to a remote site, some more stuff needs to be done. site1 specifies a SOAP connection to localsite. If you run site1 without connecting to localsite, you can only see the local  collections, eg the dummy collection myfiles. However, if you connect to localsite, you can see all of {\em its} collections as well.
\\
\\
\noindent The SOAP server we use is actually run as a servlet in tomcat. You need to set up SOAP, set up the SOAP server class which will be your service, and then deploy that service.

this is done by install-soap.bash.
You can also deploy a service through the website.  If tomcat is not running, start it up (see \ref{subsec:runtomcat}).

\noindent The SOAP servlet can be accessed at \begin{footnotesize}{\tt http://localhost:8080/soap}\end{footnotesize}. You should see a welcome page. Click on ``Run the admin client''. This enables you to list, deploy and undeploy SOAP services.

\noindent To deploy the SOAPServer for localsite:

\noindent Click on ``deploy'' and edit the following fields in the deploy form:

\begin{tabular}{ll}
ID: & org.greenstone.localsite\\
Scope: (any will do) & Request---new instantiation for each request\\
        &    Session---same instantiation across a session\\
        &    Application---only uses one instantiation\\
Methods: &process\\
Java Provider / Provider Class: & org.greenstone.gsdl3.SOAPServer\\
\end{tabular}

\noindent Now click the ``deploy'' button at the bottom of the page. If the service has been deployed, it should appear when you click on the lefthand ``List'' button.

\noindent Information about deployed services is maintained between tomcat sessions---you only need to deploy it once. To get the library1 servlet talking to the SOAP server, you need to shutdown and restart tomcat (see \ref{subsec:runtomcat}). You should see more collections when you run the library1 servlet.

\subsubsection{Debugging SOAP}

\noindent If you need to debug the SOAP stuff for some reason, or just want to look at the SOAP messages that are being passed back and forth, there is a program called TcpTunnelGui. This intercepts messages coming in to one port, displays them, and passes them to another port.

\noindent To run it:

\noindent {\footnotesize \verb#java org.apache.soap.util.net.TcpTunnelGui 8070 localhost 8080#}

\noindent tomcat uses port 8080 - you need to modify greenstone to talk to port 8070 instead of 8080. - this is specified in the {\footnotesize \verb#site#} element of the site configuration file. 
\\
\\
\noindent eg, in \begin{footnotesize}{\tt \gsdlhome/sites/site1/siteConfig.xml}\end{footnotesize}:
\begin{footnotesize}\begin{verbatim}
<site name="org.greenstone.localsite" 
      address="http://localhost:8080/soap/servlet/rpcrouter" 
      type="soap"/>
\end{verbatim}\end{footnotesize}

\noindent You can replace the 8080 with 8070 if you want to run TcpTunnelGui.

\noindent Note that \begin{footnotesize}{\tt http://localhost:8080/soap/servlet/rpcrouter}\end{footnotesize} is the
address for talking to the tomcat SOAP servlet services.


%\clearpage
%\addcontentsline{toc}{chapter}{Bibliography}
%\bibliography{main}

\end{document}