source: main/tags/3.02/gsdl3/TODO.txt@ 21113

my specific todo list. in no particular order. smallish  jobs.

* nzdl.org mirror presentation - from ian
 replace macros
 horizontal list -> first item should be selected
 expand text, detach page buttons
 remove unneccesary blank lines
 how to in hdl - the entries are double spaced and also vertical lists
 remove \n in config files
 the image under the about/search etc titles is funny under macs
 change font if language english/european??
 rollover background for buttons and nav bar

* get computer ggraphic design people to give input?? dave N matt J, polly cantlon ??

* self documenting collections?? eg provides links to the source of the  class files , the xslt , the config file etc everything needed to generate teh coll/page 
 
* UPDATE DOCUMENTATION!!!!

* text on nzdl query page to resource bundle

* collection getMetadata service

* rethink how we determine docType - current method doesn't work for phind cos it only gets doc ids - it should come from the document if not specified.

* niupepa display - abstract and commentary links. new action? or just new xslt?

* is the 'to' att compulsory? do we have to have to=""?

* query info - eg num docs found from a search, query term info - 
is this called metadata for the query and returned in a metadataList? or 
should it have another name?

<response>
<metadataList>
<metadata name="numDocs">55</metadata>
<content>
<document name="HASHxxx"/>
...
</content>
</response>

????

* format statements - need to add If and Or. do we test only on metadata? <gsf:if test="Title"> and <gsf:if test="Title='snail farming'> or more sophisticated?
also need to fix the bit where <a href='[URL]'> gets turned into <a href='<gsf:metadata name='URL'/> - needs to be added as an attribute.

* format statements for doc display

* including named templates - need to replace the ones they overwrite - not allowed two templates with teh same name. eg docHeading, docContent. or should they be match=document mode=heading/content ??

* coll specific xslt/actions?? how get these for remote colls?

* process messages - do we still have pid and handle atts? make them the same, and both atts instead of params??

* equivTermList contains terms

* page link style paging eg 1, 2, 3, .... can have links to all, links to same eg 5 either side of current page. 1,2,3...24,25,26,27,28...231

* multi query - finish.
 1. check if coll specified.
 2. check if ccs coll specified.
 3. get background info (description and format) for current coll - or all colls?? will we format stuff differently for different colls? have templates based on collection??
 4. do the query over all colls.
 5. get metadata for each result from appropriate coll.
 6. merge the results - append initially. sort by rank? this means we need to return the rank.

* metadata retrieve determination - use an action helper? separate class? should look at xslt as well as format statements? 

* what if have two services with teh same name eg two phind applets

* phind applet not working at the moment. why??

* tei building stuff into gsdl3 build code.

* subcollection/language handling for gs2 collections

* datelists formatted nicely with years at top and months down the side.

* on/off format stuff in config file eg query term highlighting, display cover image and toc, offer enriching services to user, buttons, 

* different templates for different positions in the hierarchy??
X
 ---
   ~~~~
   ~~~~
 ---
   ~~~~
   ~~~~
X
 ---

etc can we test for that? gs2 tests for [parent:Title]

* global resource bundles, not class named ones. getGlobalText? getTExt(key, bundle name)?

* fix errors in javadoc make

* send back error messages to user? rather than getting null pointer exceptions. eg if ask for non-existant metadata. also invalid syntax info?

* tidy up phind error messages

* check text strings in code and xslt - put all into resource bundles

* finalise the nzdl interface, and write script to do teh translations from the macro files.

* query results: 7 out of 10 matches

* bibliographic display??

* look at what assumptions OID makes about ids - need it to be general not just working for hash ids.

* specific tool tips for images. alt/title. eg this file is 380Mb.

* nzdl prefs page - where do the defaults come from? do we need a full cgiargs class? with default value? nzdl, we already know what query options there are so dont need to get them from the service, so dont have displayinfo or default.

* proper query term highlighting. phrases. fast.

* system requests shouls also return status id not just a message. like the build stuff. 

* should preferences be carried over between collections? currently they are.

* xslt assumes that something is saving the query args. is this ok?

* nzdl query mode - some/all, boolean/ranked, ranked/natural. result in different query parsing. new queryaction??

* should actions have access to config? currently only applet actionactually uses it. - used for filling in library  - should receptionist do this instead?

* should actions tell recepitonist if they need metadata? should receptionist pass in a list of metadata that the xslt requires?

* polling for collections? delete and start again? check dates and only reload new ones?

* caching. LInkedHashMap. overwrite the removeEldestEntry bits.

* macros in coll description and titles. eg this collection was built _date_ days ago and contains _numdocs_ documents. 
niupepa titles "_vol_1, _num_ 2" _commentary_

*dynamic query content as well as dynamic query params?

*communicators: 2 modes. 1 relays messages eg to a remote service
2. site to site - needs to modify addresses and names.

* make sure mgpp colls work with new config file stuff

* presentation prefs - how do we decide on the list of langs? from interface config file? how do we use encoding?

* mg/mgpp errors should not crash the servlet. 

* some stuff for actions can go in config files. metadata determination, query term highlighting, enrich services. result pre or post processing. is there a generic way?

* do we want a 'link to external page' action that displays the page but with teh greenstone header? a=p&sa=external?? also what about displaying whole cached html pages as frames??  gsdl uses a nav frame (a=p&p=nav) and then a second frame which is the html page.

* doc ranking

* generic multi query. a=q&c=demo&ccs=fao,... use a base coll??

* what to do when config file says : {If}{[parent:Title],[parent:Title],[Title]}. maybe we need to do root_or_self type thing?? or have some way of testing.

* stuff from talking to ian - I am sure that I have typed this in already - where is it????

 macros in gdbm db - turn them into real text.
horizontal list - first item should be selected
expand text, detach page etc buttons.
remove extraneous blank lines eg under nav bar
how to in hdl is double spaced - shouldn't be
and vertical lists appear to have more space
remove \n from config files
on the mac, about image looks weird - the green bar is on teh wrong side
change font if language is english or european?? san serif font
rollover background or colour on nav bar and buttons

get meeting with computer design person, dave N, matt J? Polly cantlon?? colours, any quick hints?

self documenting collections: eg links to config file, service code, xslt files,automatically generate but also can provide manually if wish.

cstr browse format statement.
turn off cover images

Release: present existing gs2 colls
 skeleton build for eg text and html.
documentation
scrap the collector?? ask the group
release README: how to build a collection - gs2+convert? basic gs3?
EASY installation
consolidate and document
adhoc testing
translation of resource bundles

help page: type in gsf:format statemenets and it shows you xslt? or html?

test protocol - sample thing that could generate every message?? 



General todo list. in no particular order. large  jobs.

* gs3 release - basic stuff, translations, installation, documentation. oct 31? gs2 building only, both default and nzdl interfaces? gatherer?

* generic resource retrieval - associated documents, src docs eg pdf/word, niupepa image files etc. if http accessible, put a link otherwise use a general resource retrieve?

* collector collection building - do we want to finish this? needs addDocument and configureCollection services.

The addDocument service (in GS2Construct) has not been implemented. It should take a file name and add the document to the import directory of teh collection.
there are problems with just transmitting a file name - the service may live remotely and therefore the document is not there. You should probably send it attached to the html - therefore need to work out:
how to get the document attached to the form (theres something to do with post and encoding = multipart ?? otherwise the browser just sends the filename),
and then where to get it in the servlet - is it a parameter? or something else?

and then it needs to be added into the xml request to be passed to the service.

If the program is running locally its much simpler just to send a filename - can we somehow check for this?

* configure Collection for building

also to do with building, a little harder, is the ConfigureCollection service. There is no stub for it yet, but easy enough to add one. - need to add this service to the service description xml stuff, and write a processConfigureColleciton() method.  It would be easy enough to display the config file in a big text box, and have the user edit it like the collector does.

the hard bit is that when you click ConfigureCollection, you dont know what collection you are going to be dealing with - all the building services, you select the collection on the service page. with the configure stuff, you need to select the collection, and then the config file needs to be retrieved. so its really a two step process to configure the coll - first select the coll, submit that, then edit the config file, and submit that.

All the services currently are one step - need to think about how this type of service fits into the model.

maybe it needs a hidden arg? - to tell teh service if you're at stage 1 or 2?
when teh action does the request, it then asks for the service description again to redisplay it for the user. maybe if the service knows that it has done the first half, it sends the second type of description?

also do we use the collect.cfg file or the collectionConfig.xml file to show the user?

* sequence of services

some service clusters  have services that you are supposed to carry out in sequence such as building, but there may be others.

can we do a generic action or xslt or something that sends teh user to teh next service once they've completed the first one? 

Maybe the service cluster/serviceRack class specifies teh sequence of services, and they are all handled individually like present except that some xslt puts a next button on each page with a link to teh next service in the list.

* proper install package

* currently nothing is cached - service decriptions, text strings etc could all be cached. (cgi args are cached by tomcat)

* document structure - new greenstone archive format? TEI, XHTML, OO, GML compatible? parallel document structures?
  - associated resources/documents - xlinks

* greenstone 3 building - is building going to change for gs3, or will we continue to use the perl scripts? 
  - import, build, activate
  - info extraction, augmentation
  - incremental update?
  - modular? xml pipeline?

* additional services - music search
              - keyphrase stuff
              - search history

* all the gs2 admin/security stuff - user management, authentication

* server side threading issues - is it thread safe?

* usage logs

* better error handling

* document version control

* combining requests to MR - results from first one becoming content for second one??

* language translation stuff - when translating, we want to click on a text piece or macro and go to a sample page with that text string in it. can we do this?

can we tell in gs3 where bits will be in pages?
I think we could have a new type of request, that provides a page containing all teh text strings used by a service/agent. - would it be class specific or service specific?
eg it would contain the service form, some sample results, and if its a process type of thing, all teh possible error and success messages.

what about the interface bits?

* are message formats fixed? can we create a DTD? or will they evolve too much over time?

* sitewide service agents - may want to specify that only some of the services that could potentially be provided by an agent are actually provided.

* should actions respond to describe requests? and what would they say if they did? are they agents and part of teh system, or somehow outside the system? 

*  what about soap? is there a new version of that?

* combined query services - eg one page with two service forms on there eg text query and music query - need to combine the results. eg teh action could carry out both queries then use xslt to combine the results.

or do the same thing across title and creator indexes in mg to approximate fielded searching?

true cross collection search would probably need a new service??

* more generally, combining services in general, action and service level

* action helpers: query term highlighting, page transformation and metadata determination (remove from receptionist?)
* if leave an applet page you lose all teh previous info eg for the status display for importing. need a function to retrieve again all the previous messages to redisplay them.

* can the xslt dynamically retrieve the metadata it needs for say query results??

* CSS instead of tables? - use html lists than use css to format them.

* cross coll search - if colls are built the same, just present the form and search both. need to think about ranking.
if colls are not built the same, and have indexes, subcolls, langs - do we merge the lists? and only search those that have the right ones? what if two indexes have different meanings and display text but the same id? are they treated as same index? which name do you display?
what about colls with different levels eg oneonly has document, the other doc and section. what do you display? field lists? just combine?

* search history

* use objects instead of applets? java provides a conversion tool. NPX_PLUGIN_PATH=

* danas combined search and browse stuff

* can xslt be used to generate requests as well as transform teh output?

* kea.