__group__ ticket summary component severity priority type owner status created modified _description reporter Greenstone 2 wishlist 17 Search results (terminology issue?) Greenstone2&3 minor moderate task nobody new 2007-06-18T09:13:31+12:00 2009-08-03T15:44:40+12:00 "Search results: ""document"" is not always appropriate (e.g., it is not appropriate to refer to an image search result as a document?)" oranfry Greenstone 2 wishlist 20 Win Server 2003 and IIS 6.0 Greenstone2 Runtime minor moderate defect nobody new 2007-06-18T09:16:57+12:00 2009-08-03T15:40:00+12:00 Windows Server 2003 running IIS 6.0. When library.exe called, it creates an empty gsdlconf.cfg file in %GSDLHOME%, then tries to use this and can't run because it's empty. Need to copy real gsdlconf.cfg file from bin dir oranfry Greenstone 2 wishlist 21 Punctuation in search queries Greenstone2 Runtime major moderate defect nobody new 2007-06-18T09:17:39+12:00 2009-08-03T15:58:57+12:00 Punctuation in a query craps out the server? MG all, not some. oranfry Greenstone 2 wishlist 22 exportcol.pl multi install bug Collection Exporting minor moderate defect nobody new 2007-06-18T09:19:04+12:00 2010-04-13T12:39:54+12:00 Installing exported collection multiple times causes error? oranfry Greenstone 2 wishlist 27 mg accumulator problems Greenstone2 Runtime minor moderate defect nobody new 2007-06-18T09:30:01+12:00 2009-08-03T15:38:48+12:00 mg accumulator problems (documented at around line 335 of mgsearch.cpp) - can we fix it? oranfry Greenstone 2 wishlist 36 Detect absent XML::Parser Greenstone2 Runtime minor moderate enhancement nobody new 2007-06-18T09:45:13+12:00 2009-08-03T15:40:16+12:00 Add facility to detect the absence of XML::Parser and the incompatibility of XML::Parser and expat. oranfry Greenstone 2 wishlist 40 Improve the RelatedDocuments run-time code Greenstone2&3 enhancement moderate enhancement nobody new 2007-06-18T11:34:24+12:00 2009-08-03T15:44:54+12:00 Currently hard-wired to work with dc.Relation and Title (no namespace) and Subject (same) metadata only. Needs a format statement to control it, or something. oranfry Greenstone 2 wishlist 42 Separate code and data Greenstone2&3 enhancement moderate enhancement nobody new 2007-06-18T11:35:59+12:00 2009-08-03T15:39:37+12:00 "Look at separating code and data directories, ie. allow ""collect"" directory to be put elsewhere (eg. a ""My Greenstone Collections"" folder in ""My Documents"" on Windows). Building code should already support this, and GLI definitely does, but receptionist will need modifications. See Stefan's e-mail 27 Aug 2003." oranfry Collection building wishlist 45 BibTex plugin - honour {} Collection Building: Plugins minor moderate enhancement nobody new 2007-06-18T11:39:40+12:00 2009-08-03T15:39:07+12:00 Can we make BibTex plugin respect {} in author field? e.g. {{Sugar Company}} should be kept as is, rather than converted to Company, Sugar. But, by the time we are getting to specific processing for the various fields, all the {} have been stripped away :-( oranfry Greenstone 2 wishlist 46 Mac OS X Intel binaries?? Greenstone2&3 enhancement moderate enhancement nobody new 2007-06-18T11:40:18+12:00 2009-08-03T15:46:03+12:00 Mac OS X Intel binaries?? oranfry Collection building wishlist 48 Convert to plug output Collection Building: Plugins enhancement low documentation nobody new 2007-06-18T11:42:16+12:00 2010-04-13T12:32:29+12:00 Has messages from eg HTML Plug, but you may not have HTMLPlug in the list - confusing. Could all messages appear to come from the top level plugin?? oranfry Greenstone 2 wishlist 51 Local library network address Greenstone2 Runtime enhancement moderate enhancement nobody new 2007-06-18T11:46:25+12:00 2009-08-03T15:46:18+12:00 "Local library network address - add user defined?? " oranfry Greenstone 2 wishlist 56 Installer to detect image magick Installation enhancement moderate enhancement oranfry new 2007-06-18T11:52:19+12:00 2009-08-03T15:49:19+12:00 Installer: doesn't detect existing image magick. oranfry Collection building wishlist 57 Coll building and empty classifiers Collection Building enhancement low documentation nobody new 2007-06-18T11:53:29+12:00 2010-04-13T12:32:06+12:00 Coll building: Need warnings for empty classifiers, no documents in the collection. oranfry Greenstone 3 wishlist 61 Greenstone interoperablity with Koha Greenstone2&3 enhancement moderate enhancement nobody new 2007-06-18T11:56:19+12:00 2009-08-03T15:48:11+12:00 Greenstone interoperablity with Koha oranfry Greenstone 2 wishlist 63 Bundle for GLI on Mac Installation enhancement moderate enhancement oranfry new 2007-06-18T11:57:27+12:00 2009-08-03T15:48:56+12:00 "Installer: create ""bundle"" for the GLI on Mac OS X?" oranfry Greenstone 2 wishlist 69 Unicode at start of macro files Greenstone2 Runtime minor high enhancement nobody new 2007-06-18T12:02:36+12:00 2008-08-07T11:22:23+12:00 Greenstone should be able to handle Unicode byte-order characters at the start of macrofiles oranfry Greenstone 2 wishlist 73 Translate Local Library server interface Greenstone2&3 enhancement moderate enhancement nobody new 2007-06-18T12:41:12+12:00 2009-08-03T15:37:59+12:00 Translate Local Library server interface oranfry Greenstone 3 wishlist 78 Searching multiple collections at once (eg. Oyster) Greenstone2&3 enhancement moderate enhancement nobody new 2007-06-18T13:01:18+12:00 2009-08-03T15:38:22+12:00 Searching multiple collections at once (eg. Oyster) oranfry Greenstone 2 wishlist 80 Highlighting and linking search terms Greenstone2&3 enhancement moderate enhancement nobody new 2007-06-18T13:02:44+12:00 2009-08-03T15:37:45+12:00 Link highlighted query terms together, and link from the start of the document to the first oranfry Collection building wishlist 97 Tidy up unbuilding scripts ExtraPackages enhancement very low task nobody new 2007-06-18T13:15:52+12:00 2010-04-13T12:29:23+12:00 Tidy up unbuilding scripts oranfry Collection building wishlist 102 Fix long import times Collection Building major moderate enhancement nobody new 2007-06-18T13:25:48+12:00 2009-08-03T15:49:52+12:00 Diego (several emails to greenstone users in March 2007) claims that importing a collection of paged images takes 2-3 times longer in 2.72 vs 2.60. Why is this? Does it happen for all types of collection or just paged img ones? Can we do anything about it? oranfry Collection building wishlist 104 OpenDocument parsing of XMl files Collection Building: Plugins minor moderate defect nobody new 2007-06-18T13:28:52+12:00 2009-08-03T15:31:10+12:00 OpenDocument plugin - sometimes parsing of XMl files causes a segmentation fault. This kills the building - its not trapped by the eval. I think its to do with byte order markers - if I strip this off before parsing the file, then the parsing works. When I tried to do it in the code, then XML parser couldn't find any elements in the XML. Does this affect other XML parsing? Haven't fixed it. oranfry Greenstone 2 wishlist 106 Groupsize problem Collection Building minor low defect nobody new 2007-06-18T13:30:13+12:00 2009-02-26T14:21:41+13:00 Groupsize doesn't work properly with associated files?? johnT reported this, did he fix it?? oranfry Collection building wishlist 109 Word files with image links Collection Building: Plugins minor moderate defect nobody new 2007-06-18T13:32:35+12:00 2009-08-03T15:29:50+12:00 Word files containing links to images (not embedded images). The image is not copied to tmp and therefore can't be found by HTMLPlug oranfry Collection building wishlist 110 MGPP Paragraph searching Indexers minor moderate defect nobody new 2007-06-18T13:33:23+12:00 2009-08-03T15:29:33+12:00 MGPP paragraph searching isn't all good: paragraph numbers are converted to section numbers but the duplicates aren't filtered out. oranfry Greenstone 2 wishlist 111 Segmentation and the space character Collection Building minor low defect nobody new 2007-06-18T13:34:29+12:00 2009-02-26T14:21:25+13:00 Using the wrong character for (non-displaying) space - change to a utf-8 character if it doesn't affect gsdl2. oranfry Collection building wishlist 112 List direction request Collection Building: Classifiers enhancement high enhancement nobody new 2007-06-18T13:35:46+12:00 2010-11-09T13:45:41+13:00 "GenericPlug request - some one had documents using two scripts - arabic and english. The classifier ended up ....Wil-X Y-[arabic char] [arabic char-arabic char] This displayed funny cos the arabic stuff goes in the opposite direction. He wanted to have the list split into two, with the english ending in Y-Z, and on a new line all the arabic categories. Is this possible?? " oranfry Collection building wishlist 115 Include_unclassified_documents option Collection Building: Classifiers enhancement moderate enhancement nobody new 2007-06-18T13:37:43+12:00 2009-08-03T15:51:23+12:00 "Classifiers - add -include_unclassified_documents option - can add in an _unknown_ bucket at the end of the list. " oranfry Collection building wishlist 117 Incremental updates in MGPP Collection Building enhancement moderate enhancement nobody new 2007-06-18T13:38:53+12:00 2009-08-03T15:29:11+12:00 CVS version of MGPP used by Perseus can do incremental updates. Investigate? oranfry Greenstone 2 wishlist 119 Can't resurrect crashed library GLI minor moderate defect nobody new 2007-06-18T13:42:28+12:00 2009-08-03T15:32:50+12:00 GLI doesn't seem to be very successful at resurrecting the local library if it has crashed oranfry Greenstone 2 wishlist 124 Export to CD twice causes problems Collection Exporting trivial moderate defect nobody new 2007-06-18T13:46:05+12:00 2009-02-26T14:10:17+13:00 Exporting to cdrom - if do it twice with the same name (eg exported_collections) then you are getting some of the old stuff in the new one. Need to delete the old exported folder before doing the new one. oranfry Collection building wishlist 127 Fixed Mode GLI enhancement moderate enhancement nobody new 2007-06-18T13:48:27+12:00 2009-08-03T15:19:26+12:00 What about modes? should the mode be fixed, or able to be fixed - library assistants can currently change modes and bugger up the collection design. oranfry Greenstone 2 wishlist 128 Build empty collection on windows problem Collection Building minor moderate defect nobody new 2007-06-18T13:49:24+12:00 2009-02-26T14:09:47+13:00 Bad result when building collections with no documents on Windows oranfry Collection building wishlist 130 Mirroring on Windows GLI enhancement moderate defect nobody new 2007-06-18T13:50:36+12:00 2009-08-03T15:19:03+12:00 Proxy user and password are arguments to the command because environment variables didn't seem to work oranfry Greenstone 2 wishlist 131 TranslateView, can't delete all translations GLI minor low enhancement nobody new 2007-06-18T13:51:19+12:00 2009-02-26T14:21:07+13:00 Translate view - can't delete all translations? oranfry Collection building wishlist 146 ScriptOptions cache GLI enhancement low enhancement nobody new 2007-06-18T14:04:44+12:00 2009-08-03T15:18:07+12:00 The collection/ScriptOptions class should probably cache (in memory, NOT on disk) the options for a particular combination of script and language, so they don't have to be re-gathered whenever the loaded collection changes. oranfry Collection building wishlist 147 Metadata access and editing times GLI enhancement moderate enhancement nobody new 2007-06-18T14:05:06+12:00 2009-08-03T15:17:37+12:00 Metadata access and editing times could be improved slightly further (at the expense of memory) by creating a mapping in MetadataXMLFile.java between filename regular expressions and FileSet elements. Currently the code looks through the metadata.xml files from the start to find the right FileSet elements to read/edit. oranfry Collection building wishlist 151 Shortcut to desktop GLI enhancement moderate enhancement nobody new 2007-06-18T14:07:55+12:00 2009-08-03T15:17:15+12:00 Shortcut to desktop in workspace tree? local file system open by default on windows? oranfry Collection building wishlist 153 Exportcol output GLI enhancement moderate enhancement nobody new 2007-06-18T14:09:16+12:00 2009-08-03T15:16:18+12:00 tidy up output from exportcol - repeats info, provide a file browse option? oranfry Collection building wishlist 156 Plugin file extensions GLI enhancement moderate enhancement nobody new 2007-06-18T14:11:24+12:00 2009-08-03T15:15:59+12:00 Can we display, for each plugin, a list of the file extensions that they process. Librarian mode don't see process expression, so its hard to tell what each one processes. oranfry Greenstone 2 wishlist 164 Feedback stuff GLI enhancement low enhancement dmn new 2007-06-18T14:16:30+12:00 2009-02-26T14:16:19+13:00 Turn Veronica's feedback stuff on from within the GLI? oranfry Collection building wishlist 167 Format statement editing GLI enhancement moderate enhancement nobody new 2007-06-18T14:18:29+12:00 2009-08-03T15:15:26+12:00 Dave's wacky research idea of the day -- nice format statement editing oranfry Collection building wishlist 185 Language and font GEMS enhancement moderate enhancement nobody new 2007-06-18T14:52:46+12:00 2009-08-03T15:13:14+12:00 File->Preferences for language and font oranfry Collection building wishlist 186 Help menu GEMS enhancement moderate enhancement nobody new 2007-06-18T14:53:11+12:00 2009-08-03T15:11:43+12:00 Help menu oranfry Collection building wishlist 190 New value in the attribute tables GEMS minor moderate defect nobody new 2007-06-18T14:54:38+12:00 2009-08-03T15:11:00+12:00 A new value in the attribute tables doesn't get saved unless Enter is hit, or the table selection moves to another row. oranfry Collection building wishlist 191 Controlled vocabulary GEMS enhancement moderate enhancement nobody new 2007-06-18T14:55:09+12:00 2009-08-03T15:10:30+12:00 Make it able to add in a controlled vocabulary for metadata values. - see DTD. Also allow import from and export to an hfile, so values can be ordered?? oranfry Collection building wishlist 193 Element attributes GEMS minor moderate defect nobody new 2007-06-18T14:56:11+12:00 2009-08-03T15:09:00+12:00 Element attributes currently all marked as required? This means that you can't set them to be empty, even though they are initially empty oranfry Greenstone 2 wishlist 201 Zettair Collection Building enhancement moderate task nobody new 2007-06-18T15:05:21+12:00 2009-08-03T15:06:58+12:00 A new search engine from http://www.seg.rmit.edu.au/zettair/index.php. It can handle larger amounts of text than MG which has an inherent 2GB size limit for the index file. Zettair breaks the index file into smaller pieces. It apparently also does phrase searching, document summarisation, incremental building. Look at putting it into Greenstone?? 2 or 3?? oranfry Greenstone 3 wishlist 202 Unit testing in Greenstone 3 Greenstone3 Runtime enhancement moderate task nobody new 2007-06-18T15:05:57+12:00 2009-08-03T15:06:26+12:00 We have a framework for Java Unit Testing using JUnit. The test suite can be run from Ant (ant test). We need to set up some decent tests for the runtime system and services etc. Perhaps we can use something like Jeffery's collection test script to compare changes. Eg download the html page and compare with a saved copy. oranfry Greenstone 2 wishlist 204 Valid XHTML Greenstone2 Runtime minor moderate defect nobody new 2007-06-22T09:48:33+12:00 2009-08-03T16:00:39+12:00 "The DOCTYPE claims Greenstones pages to be XHTML Strict, yet when displaying html documents, Greenstone still lets old tags like into the document. Also, Greenstone does not close off tags properly. I haven't investigated where in the code this is supposed to happen or how it works. But, one thing I can tell is not happening is closing off tags '''per-section''', which is necessary for valid XHTML when viewing a section on its own. Need to: * Ensure valid xhtml in imported html documents, both globally and per-section. * Strip html documents of old tags like and old attributes like 'cellpadding' and 'border'" oranfry Greenstone 2 wishlist 206 Spaces in Install paths GLI major moderate defect oranfry assigned 2007-08-01T13:51:25+12:00 2009-04-22T16:55:01+12:00 "Ensure that all installers can handle installing to a path with spaces, and running gli, local library server etc. See this sub-ticket: #547" oranfry Greenstone 2 wishlist 207 wget processes started by GLI keep running on GLI crash, and more wget tasks GLI major moderate defect ak19 new 2007-08-14T10:58:15+12:00 2009-02-26T14:03:18+13:00 If you start a web download, and then GLI crashes (which it is apt to do if something goes wrong with the download), the wget process which is handling the download stays alive and runs in the background. This bug is especially troubling when this orphaned wget process has been set to follow links a to a ridiculous depth, like 10, which pretty much covers the whole internet. In this case, the wget process silently attempts to download the whole internet over a connection which might be pay-per-byte. This bug could attract 'GLI cost me hundreds with my ISP!' complaints so should be looked at promptly. oranfry Greenstone 3 wishlist 212 End-user debugging Greenstone3 Runtime enhancement moderate defect nobody new 2007-10-09T10:47:22+13:00 2009-08-03T15:05:43+12:00 " Provide easy ways for users to debug collections. e.g. an extra url parameter (&debug=1) that embeds debugging info in the HTML source" anonymous Greenstone 3 wishlist 213 static pages Greenstone3 Runtime enhancement moderate defect nobody new 2007-10-09T10:48:01+13:00 2009-08-03T15:05:23+12:00 Ensure we have a tidy solution for static pages in collections anonymous Collection building wishlist 214 test java.nio as as alternative file copy mechanism GLI enhancement moderate defect nobody new 2007-10-09T10:52:04+13:00 2009-08-03T15:05:03+12:00 "public void copyFile(File source, File destination, boolean overwrite) in FileQueue.java uses a buffer to copy files. We should test whether this solution is quicker (it is probably more likely to be correct as well): http://www.javalobby.org/java/forums/t17036.html {{{ public static void copyFile(File sourceFile, File destFile) throws IOException { if(!destFile.exists()) { destFile.createNewFile(); } FileChannel source = null; FileChannel destination = null; try { source = new FileInputStream(sourceFile).getChannel(); destination = new FileOutputStream(destFile).getChannel(); destination.transferFrom(source, 0, source.size()); } finally { if(source != null) { source.close(); } if(destination != null) { destination.close(); } } }}} " anonymous Greenstone 3 wishlist 215 integrate JHOVE as a plugin Collection Building: Plugins blocker moderate enhancement nobody new 2007-10-12T15:22:43+13:00 2008-11-10T14:19:55+13:00 " For both metadata extraction and object validation. http://hul.harvard.edu/jhove/" anonymous Collection building wishlist 216 integrate Xena as a plugin Collection Building: Plugins enhancement moderate enhancement nobody new 2007-10-12T15:24:22+13:00 2009-08-03T15:04:30+12:00 " http://xena.sourceforge.net/ xena does several useful conversions of documents to open formats" anonymous Collection building wishlist 227 Provide friendly text at end of Message Log after building Collection Building enhancement moderate enhancement nobody new 2007-11-07T14:33:55+13:00 2009-08-03T15:01:59+12:00 " {{{ The end of the Message Log after a successful build is very unfriendly: buildcol.pl> *** outputting information for classifier: CL1 buildcol.pl> *** outputting information for classifier: CL2 buildcol.pl> *** outputting information for classifier: CL3 buildcol.pl> *** outputting information for classifier: oai buildcol.pl> *** creating auxiliary files buildcol.pl> Command complete. The end of the Import phase is nicer: import.pl> ********************************************* import.pl> Import complete import.pl> ********************************************* import.pl> * 500 documents were considered for processing import.pl> * 500 were processed and included in the collection import.pl> Command complete. import.pl> Extracting new metadata from archive files. import.pl> Archived metadata extraction complete. but still unpleasant. It would give users more confidence if we said something like this at the end of the Message Log: Build successful. 500 documents were considered for processing and all were included. 2 search indexes were constructed using the Lucene indexer: text: 5000 words, 23K ex.Title: 300 words, 2K 4 browse classifiers were constructed: CL1: dc.Title using an AZList CL2: dc.Subject using an AZCompactList CL3: dc.Creator using an AZList CL4: dc.Source using an AZCompactList 5 plugins were used: GAPlug, ArcPlug, RecPlug, OAIPlug, XMLPlug 2 plugins were not used: TextPlug, PDFPlug }}} " anonymous Collection building wishlist 228 compile warnings for GLI GLI minor moderate defect nobody new 2007-11-08T16:41:49+13:00 2009-08-03T15:01:29+12:00 "we should aim to remove these warnings: ===================================== Compiling the Greenstone Librarian Interface... src/org/greenstone/gatherer/gui/HelpFrame.java:156: warning: [deprecation] toURL() in java.io.File has been deprecated return (new File(help_file_path)).toURL(); ^ src/org/greenstone/gatherer/feedback/FeedbackInterface.java:1382: warning: com.sun.image.codec.jpeg.JPEGImageEncoder is Sun proprietary API and may be removed in a future release JPEGImageEncoder encoder; ^ src/org/greenstone/gatherer/feedback/FeedbackInterface.java:1383: warning: com.sun.image.codec.jpeg.JPEGCodec is Sun proprietary API and may be removed in a future release encoder = JPEGCodec.createJPEGEncoder(out); ^ src/org/greenstone/gatherer/feedback/FeedbackInterface.java:1384: warning: com.sun.image.codec.jpeg.JPEGEncodeParam is Sun proprietary API and may be removed in a future release JPEGEncodeParam param; ^ Note: Some input files use unchecked or unsafe operations. Note: Recompile with -Xlint:unchecked for details. 4 warnings Note: src/org/greenstone/gatherer/gems/GEMS.java uses unchecked or unsafe operations. Note: Recompile with -Xlint:unchecked for details. Note: src/org/greenstone/gatherer/remote/ZipFiles.java uses unchecked or unsafe operations. " anonymous Collection building wishlist 238 generic XML plugin Collection Building: Plugins enhancement moderate task nobody new 2008-01-09T10:57:08+13:00 2009-08-03T11:37:36+12:00 " a plugin which allows a user to specify certain XML elements to be treated as metadata with namespace options This would allow us to create browsing structures over XML workflow files such as these: http://workflows.mygrid.org.uk/repository/myGrid/PaulFisher/" dmn Greenstone 3 wishlist 243 a no-web mode for GS3 Greenstone3 Runtime enhancement moderate enhancement nobody new 2008-02-14T11:07:01+13:00 2009-08-03T11:37:16+12:00 " if a collection/site is only going to be used via XML/SOAP messages and not have a web presence then can we have a mode to indicate this that will allow us to turn off some resources associated with web requests. " dmn Greenstone 3 wishlist 246 investigate using the Apache Tomcat Native library Greenstone3 Runtime enhancement low enhancement nobody new 2008-02-20T10:06:44+13:00 2009-08-03T11:36:57+12:00 " In the Tomcat catalina logs you can see: INFO: The Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: I think this refers to this: http://tomcat.apache.org/tomcat-5.5-doc/apr.html for better performance. At the least we could make it easy for people wanting to add native library support. " dmn Greenstone 3 wishlist 247 integrate the Mat metadata analysis tool into the main GS3 distribution Greenstone3 Runtime blocker moderate enhancement dmn assigned 2008-02-20T15:14:13+13:00 2008-09-02T14:14:49+12:00 " Get Eric's metadata tool into GS3 svn. Try to get some consistency into how we add extra functionality/content into GS3 Investigate whether we need the Java tool as well as the HTML output. Make standalone HTML output an option. Test on a Mac. Port it to Windows?" dmn Greenstone 3 wishlist 251 consider upgrading to latest Tomcat (i.e. 5.5.26) Collection Building minor moderate task oranfry assigned 2008-02-29T14:37:19+13:00 2008-08-07T12:19:00+12:00 " we currently have Tomcat 5.5.25, there has been an update to 5.5.26, with these bug fixes: http://tomcat.apache.org/tomcat-5.5-doc/changelog.html note that we need to require Java 5: ""Tomcat 5.5 requires JRE 5.0 by default."" http://apache.tradebit.com/pub/tomcat/tomcat-5/v5.5.26/README.html" dmn Greenstone 3 wishlist 273 collection grouping Greenstone3 Runtime enhancement moderate feature kjdon assigned 2008-03-17T15:24:58+13:00 2011-04-06T17:03:12+12:00 "GS3 should have the ability to group collections into arbitrary sets for display purposes. Rather than listing all collections on a home page like nzdl.org we could just display a few group icons/titles, which lead to 'group home pages'. Thoughts: - how to specify titles and icons? - how to specify group home pages, how to customise? - how to search and browse across groups?" dmn Greenstone 3 wishlist 276 tidy up classes directory / Greenstone extensions Greenstone3 Runtime minor moderate defect nobody new 2008-03-20T15:03:42+13:00 2008-03-20T15:03:42+13:00 "the directory {{{ /greenstone3/web/WEB-INF/classes }}} is full of .properties files, especially language ones. Can these be tidied into a sub-directory? How might this link in to dealing with extensions and other applications? How do we stop this important directory being a mess? " dmn Greenstone 2 wishlist 283 GLI log files first character GLI minor high annoyingness nobody new 2008-03-31T11:43:22+13:00 2009-02-26T13:45:58+13:00 "In each log file produced by GLI we start it with a single character that is used to indicate whether the process was successful, unsuccessful, cancelled. A dot is used while the process is going on. It shows up in the log display. Some people at Ians workshops are very concerned about this character and Ian wants it gone. Can we hide the first line? Or put the status into the filename?" kjdon Greenstone 3 wishlist 286 CD/DVD Collections for GS3 Greenstone3 Runtime major moderate defect nobody new 2008-03-31T14:44:15+13:00 2008-03-31T14:56:27+13:00 " Can we write collections to Read-only media such as CDs and DVDs (and USB drives) in GS3 in the same way as we do in GS2? These might help: http://www.javalobby.org/articles/tomcat2go/ http://www.javalobby.org/java/forums/t18625.html http://www.velocityreviews.com/forums/t136551-setting-up-tomcat-to-run-jspservlets-from-usb-keychain.html http://forums.alfresco.com/viewtopic.php?p=2175& http://www.mail-archive.com/tomcat-user@jakarta.apache.org/msg101700.html http://www.velocityreviews.com/forums/t143213-running-tomcat-from-a-cd.html http://www.stunnix.com/prod/aws/tomcat-cdrom.shtml http://www.mail-archive.com/tomcat-user@jakarta.apache.org/msg148166.html http://www.mail-archive.com/search?l=tomcat-user%40jakarta.apache.org&q=CD http://max.limpag.com/2006/11/19/running-a-web-server-wordpress-on-my-phone/ " dmn Collection building wishlist 287 PDFPlugin: pdftohtml requires '-c' option to properly process multi-column pdf files Collection Building: Plugins minor moderate defect nobody new 2008-04-02T12:09:26+13:00 2009-08-03T16:00:01+12:00 "When using the PDFPlugin pdftohtml to generate HTML for the HTMLPlugin to extract text, there is a flaw with multi-column pdfs. In the simple version (currently used by Greenstone), text is extracted left to right, ignoring columns. This results in correct HTML where text is aligned properly for viewing, but the text is not in a logical order. When the text is extracted, the content of the columns are interleaved line by line. This can be fixed by adding the '-c' option on pdftohtml when called. This has the added benefit of providing additional structure that can be extracted (sections, etc.)" mcennis Greenstone 3 wishlist 290 optional server-side JavaScript processing of metadata for formatting Greenstone3 Runtime major moderate enhancement nobody new 2008-04-03T09:33:48+13:00 2008-04-03T09:33:48+13:00 " along with XLST it should be possible to write custom JavaScript (we should allow custom Java somewhere as well) functions to modify metadata values for display in any sequence JS -> XSLT -> JS JS -> XLST XLST -> JS Java 1.6 has the necessary scripting code built in." dmn Greenstone 3 wishlist 291 Tomcat performance tuning Greenstone3 Runtime minor moderate enhancement nobody new 2008-04-03T09:56:55+13:00 2008-04-04T15:28:25+13:00 " There are several configuration options we can tune for performance: http://wiki.apache.org/tomcat/FAQ/Performance_and_Monitoring http://marc.info/?l=tomcat-user&m=103598932529887&w=2 But we should do some profiling first. " dmn Greenstone 2 wishlist 295 Remote GSDL must get client-side imported files when these have been edited in external app (replace_srcdoc_with_html) Collection Building enhancement moderate task ak19 new 2008-04-03T11:48:09+13:00 2009-02-26T13:57:18+13:00 "Since GLI allows files to be launched in external applications, some of these apps may be editors as well. That means that imported files can be edited by users on the client side before building, while the remote GSDL server would be building the copy of the unedited files in its import folder. In a more specific case, the replace_srcdoc_with_html.pl feature's whole point is to allow GLI users to import files, convert them to HTML, edit those HTML files and have then built with those edits intact. This works easily enough in the local GSDL case, as the files being edited and those being built are of the same import folder. But when GSDL is remote, there's at present no way that the server end knows of edits made to documents using external applications launched from within GLI. Therefore, building still works with the unchanged server side copy of the import folder." ak19 Greenstone 2 wishlist 297 Incremental import using OIDtype incremental Collection Building enhancement low enhancement nobody new 2008-04-08T11:38:24+12:00 2009-02-26T14:16:06+13:00 "Can we make incremental import work using OIDtype incremental? Each import could create a top level folder, F0, F1... and then use that name in the import, get oids like F0D1, F0D2 etc. Next import (if -keepold/incremental is switched on) you look to see what folder you are up to, and start from there. I guess you could also do this without folders, just looking at the D0, D1 folders to see what number we are up to. But it would be faster with top folders. And also its easier to see what happened at each separate import." kjdon Greenstone 2 wishlist 298 No-OID-conflict flag for import Collection Building enhancement low feature nobody new 2008-04-08T11:48:08+12:00 2009-02-26T14:15:42+13:00 "Diego Spano has reported this problem: A collection of images, on Windows. Each day a set of images is added to the import directory with the same names as the previous set (img001, img002 etc). Importing uses the -keepold flag. For some reason, the images aren't being hashed, so they are ending up with the same identifiers as the previous set. This means they overwrite the previous ones in archives.inf. (in my testing, the images were hashed, so they got different ids because they were different images.) Can we have a flag where you are not allowed to duplicate OIDs? Need to check the list of assigned ids before assigning the next one. This assumes that the user knows what they are doing and has no duplicate documents. See also ticket #297" kjdon Greenstone 2 wishlist 301 GLI translate pane GLI enhancement low enhancement nobody new 2008-04-08T13:25:07+12:00 2009-02-26T14:12:27+13:00 "Since we are focusing on multilingual stuff, should we look at overhalling this pane? Does anyone ever use it? see also #131" kjdon Greenstone 2 wishlist 302 true multilingual classifiers Collection Building: Classifiers enhancement moderate feature nobody new 2008-04-08T13:27:45+12:00 2009-02-26T13:56:41+13:00 "Can we base all classifiers on genericlist so that they are all truly multilingual? There are some things generic list can't do yet - hierarchical metadata, removing prefixes, other?? Investigate what functionality is missing and whether other classifiers are just wrappers to genericlist." kjdon Greenstone 2 wishlist 307 Hardcoded Prev/Next page buttons for paged document Collection Building enhancement low defect nobody new 2008-04-10T15:53:54+12:00 2009-02-26T14:11:55+13:00 "Prev/Next page buttons and the page number above the ""go to form"" are hard coded in pagedbrowserclass.cpp, which shouldn't be the case. Need to move the display to macro instead." anna Greenstone 3 wishlist 312 web inspection of available metadata fields Greenstone3 Runtime minor moderate enhancement dmn assigned 2008-04-22T09:50:34+12:00 2008-10-30T10:35:04+13:00 " We should have a page that summarises the metadata available for formatting: server/collection/metadata which produces a HTML page listing: - server/site level metadata (if any) - collection level metadata - union of metadata fields in a collection site: gsi.server_name collection (demo): collection_name number_of_docs dc: dc.title dc.subject etc " dmn Greenstone 3 wishlist 313 metadata cleaning phase Collection Building minor moderate feature dmn assigned 2008-04-28T09:06:43+12:00 2009-05-21T03:14:40+12:00 " Before or after metadata enrichment might it be useful to have a metadata cleaning phase? Possible tasks include: - integration with metadata control - modification of metadata values - creation of new metadata values from other ones this would enable building a classifier on cleanTitle whilst still keeping originalTitle around for display Where should this go? In BasPlug so it can be part of every plugin or as a separate phase of the workflow? " dmn Possible 2.88 Release 315 exported lucene collections don't work on windows 10. Collection Exporting major high defect nobody new 2008-05-08T10:23:48+12:00 2016-09-13T10:00:17+12:00 "Here is some stuff from an email that I wrote. The library server (server.exe) needs to have Perl in its PATH variable to be able to search lucene collections. On my computer, where Perl is not automatically in the PATH, for normal local library server: if I start server by double clicking in the gsdl folder, it can't find perl and searhcing lucene doesn't work. if I open a terminal, add perl to the path and run server.exe from there, it works. If I install greenstone from a binary release, and run it from the start menu, it works. If I export a lucene collection, then copy it into an existing greenstone, it works (dependent on the above conditions). But, If I try to run the server.exe from the exported collection folder (by double clicking OR from a terminal with perl in the path), it doesn't work. I have copied in the same server.exe from my other greenstone but it still doesn't work. This is strange. I haven't tried this with a cd that is installed first - this gets server into the start menu. Would it work then??" kjdon Greenstone 3 wishlist 318 MG phrase search hasn't been implemented Greenstone3 Runtime major moderate task nobody new 2008-05-08T14:56:23+12:00 2009-02-16T14:25:47+13:00 shaoqun Greenstone 2 wishlist 319 multilingual metadata Collection Building enhancement high enhancement nobody new 2008-05-12T09:36:03+12:00 2009-02-26T13:43:24+13:00 "We need to be able to have different language versions of document metadata. For example, a dc.Title in English, and a dc.Title in French. The user will see the appropriate version of the metadata depending on the language he is viewing the page in. Will need a default language then. This will affect: * GLI: allow entering metadata in different languages. Maybe an extra column in Enrich panel? * metadata.xml file needs language attribute in Metadata element. *doc.xml object and archive file needs language attributes. *indexing metadata - do we just put it all in, or will we have separate indexes for each language? *classifiers - have an option to only use metadata for a particular language? * gdbm db - need to store language info for each metadata - can do this like we do language specific collection metadata * runtime - make sure we get the right language metadata to display. Is that all?" kjdon Greenstone 2 wishlist 329 Support for dynamic classifiers in GLI GLI enhancement high enhancement nobody new 2008-07-03T10:18:44+12:00 2009-02-26T13:39:28+13:00 "Michael has implemented dynamic classifiers using sqlite. Need to upgrade GLI to offer the infodbtype option for the collect.cfg file. And also add support for the new dynamicclassifier directive. Ask Michael for more information :-)" kjdon Greenstone 3 wishlist 330 investigate removing jsession ids Greenstone3 Runtime major moderate defect nobody new 2008-07-08T16:37:56+12:00 2008-07-08T16:37:56+12:00 " URLs in GS3 have become even more messy than they were in GS2: http://www.greenstone.org/greenstone3/nzdl;jsessionid=E8BFF88D792F4075601045456EC801AD?a=p&sa=about&c=hdl&ct=0 this is meaningless to users, horrible for SEO, a security risk etc: http://randomcoder.com/articles/jsessionid-considered-harmful or at least allowing site/collection maintainers to turn them off. Most of the time the preferences are never used (I expect) - do we have any evidence they are? The default values should just be that, default and not clutter up the URL. We could use cookies like everyone else, or provide a session independent URL when necessary. And/or implement the filter as described in the URL above. Suppose someone wants to remove all the preference - would we still get jsessionids in the URLs? " dmn Possible 2.88 Release 336 GLI auto plugin detection GLI enhancement moderate annoyingness kjdon new 2008-07-21T11:54:45+12:00 2012-06-07T13:38:22+12:00 "When copying files into a collection, if there are no plugins that can process the file, it gives you a popup: None of the plugins in this collection are expected to process the file ""p23.png"". The following plugins can be used to process this file. You can add one to your collection now by selecting it and pressing the Add button. (Note that you can change the list of plugins at any time by going to ""Document Plugins"" on the Design Pane.) We need a button ""Don't ask me again"" to stop it doing this if you have hundreds of files that can't currently be processed." kjdon Greenstone 3 wishlist 339 sections in URLs: can we use doc IDs that contain a dot? Greenstone2&3 major high defect nobody new 2008-07-23T10:47:19+12:00 2008-07-23T10:47:19+12:00 " if we use a . to indicate a section, e.g. HASHxxxxxx.2 in a URL. Then we use something else to replace a HASH-style ID, so what happens if we want to use a DOI such as: 10.1002/asi.20825 from http://www3.interscience.wiley.com/journal/118640514/abstract The most likely replacement for HASH is usually full of dots! We need to find a new way of representing sections in URLs. " dmn Greenstone 2 wishlist 344 global block tidy up. Collection Building major high enhancement kjdon assigned 2008-08-04T10:31:40+12:00 2009-02-26T13:45:09+13:00 "now that we have a global block pass, some plugins that currently use block expressions can be changed to work out blocking based on their files. Check order of 'smart' blocking vs block exp vs no blocking." kjdon Greenstone 2 wishlist 351 HBPlugin.pm method near-duplicated in HTMLPlugin.pm Collection Building: Plugins minor very low defect nobody new 2008-08-07T13:17:21+12:00 2009-02-26T14:21:54+13:00 There's mentions of HB related tasks in HTMLPlugin. In particular, there's a method called HBgetText in both HBPlugin and HTMLPlugin. Katherine has inspected both and found them to be subtly different In future, may need to investigate this further to find out whether they can be merged. ak19 Greenstone 2 wishlist 354 Lucene doesn't work with IIS Greenstone2 Runtime major moderate defect nobody new 2008-08-11T11:58:26+12:00 2015-10-06T17:08:57+13:00 "You can't search lucene collections in IIS. It can't run Perl. Is this a general problem with running external programs from a cgi program? Is it a user permission problem? What user is IIS running as? Try calling the Java directly instead of through perl - does this help? The computer on Katherine's desk (glen) is running IIS." kjdon Greenstone 2 wishlist 358 new option type for pluginfo GLI enhancement high feature nobody new 2008-08-25T09:03:06+12:00 2009-03-30T11:35:36+13:00 "Add a new option type for plugin/classifier/script options: file. Specifies that the option is for a file. In GLI, this can then be handled using a popup file browser window so the user can navigate to the file rather than typing in the path. " kjdon Greenstone 2 wishlist 359 CJK character segmentation Greenstone2 Runtime enhancement low enhancement nobody new 2008-08-25T10:01:55+12:00 2009-02-26T14:11:24+13:00 "Need to implement handling for high number ranges. code values are in the code, but commented out. perllib/cnsseg.pm runtime-src/src/recpt/querytools.cpp text_t can't handle numbers > 0xffff (unsigned short). " kjdon Greenstone 2 wishlist 362 HTML file blocking Collection Building: Plugins minor moderate defect nobody new 2008-08-28T15:06:38+12:00 2009-02-26T14:09:04+13:00 "In the tudor collection, we had a file import/englishhistory.net/tudor/contents.html There was an image import/englishhistory.net/tudorrosemain.jpg The link in contents.html was link The right thing happens when replacing image links in the file, end up with _httpdocimg_/tudorrosemain.jpg. However, in the block list its ending up as ...import/englishhistory.net/tudor/http:/englishhistory.net/tudorrosemain.jpg and then not getting blocked. Fix this. I have fixed the collection by changing the link to a relative link, but the plugin should be changed." kjdon Collection building wishlist 363 automatic metadata extaction from filenames Collection Building enhancement moderate enhancement nobody new 2008-09-01T10:31:03+12:00 2009-08-03T15:51:05+12:00 "From an email 28/03/08 from Pablo San Juan I need your help. I´m testing Greenstone 2.80 release and I´m trying to use MetadataXMLPlug plugin in order to include automatic metadata to my collection files. Here is an extract from metadata.xml file: \w{3}d\{8}\.pdf $1 It´s a newspaper´s collection. The filenames of my collection are like ""ABC20080327.PDF""; the three first characters indicates newspaper abreviature, and next 8 characters indicate new date(year,month and day). In tag I´m using a perl expression and I want to use its match to insert this value as new metadata value in metadata tag. How can I use $1 variable (the match result in perl expression) in Metadata tag? " kjdon Possible 2.88 Release 364 sorting cross collection search results Greenstone2 Runtime enhancement moderate enhancement kjdon assigned 2008-09-01T13:37:02+12:00 2013-09-24T10:22:22+12:00 "Currently you only get sorting by rank. Is there anyway we can get sorting in natural order? Requested by users, here are the emails:: Well here is another sorting question - we have several collections that we want users to cross search. My question is that within the individual collection I was able to ""sort"" the results alphabetically. But once I select the ""cross-collection search"" feature, when you do a search in the ""all"" search mode - the results are not in alpha order. So here lies my problem, is it possible to sort it in alpha order in AND/OR have it group by collection. i.e. Smith, John - Doe, Jane Vital Statistics Collection Smith, John World War I Service Records Smith, Lance - Doe, Jane Vital Statistics Collection Smith, Mark Passenger Manifest Collection Smith, Nancy Passenger Manifest Collection This is basically what we would like users to see in their search for ""Smith"". ---------------------------- I have two collections built on Greenstone, both of them have sortmeta configured, so that the results come out in a reverse order, according to the date. Now I want to be able to cross-search from one to the other, but as soon as I check the cross-collection option and relaunch the library, the results don't come out sorted. ¿Can the two options co-exist?" kjdon Greenstone 2 wishlist 365 plugout tidy up Collection Exporting enhancement moderate enhancement kjdon new 2008-09-04T10:02:22+12:00 2009-02-26T13:53:16+13:00 "Tidy up plugouts in same way we have done plugins. PrintInfo would be almost exactly the same as plugin one - use the same class?? export.pl has lots of options that are specific to each plugout. remove them from the script, and make plugouts specified in collect.cfg so options are passed in in the same way that plugin options are. import saveas thing should also use plugout options from config file see also #332." kjdon Greenstone 2 wishlist 366 Improve gs2build and gs2runtime syncronisation with gsdl Greenstone2&3 major moderate enhancement nobody new 2008-09-05T12:14:54+12:00 2010-04-13T12:38:39+12:00 "At the moment there are several files (~15) in the root of gs2build and gs2runtime, which are copies (or copies with slight modifications) of files in gsdl/trunk. If the files in gsdl change, the same change needs to be applied to the files in gs2build and gs2runtime. We could save time manually syncing these files, by setting up an automatic system to keep the files in sync. Ideas: * A cron job to check for changes to the gsdl files and apply them to the gs2build/gs2runtime files * Identify the files in gs2build and gs2runtime that are exact copies of the files in gsdl/trunk and delete them from gs2build and gs2runtime, and invent another system to help get those files directly from gsdl (making the problem _smaller_) " oranfry Greenstone 2 wishlist 368 Ghostscript on windows Collection Building enhancement low task nobody new 2008-09-11T09:38:10+12:00 2009-02-26T14:10:58+13:00 "For PS to txt (in gsConvert.pl) we assume that gs doesn't work on windows, and just use regular expressions to strip out the text. However, we are now installing ghostscript to be used with imagemagick (on a cd release) on windows, so can we use it for pstotext?" kjdon Greenstone 3 wishlist 370 Investigate djatoka as an image server Greenstone3 Runtime major moderate enhancement nobody new 2008-09-17T10:21:09+12:00 2008-09-17T10:21:09+12:00 " http://dlib.org/dlib/september08/chute/09chute.html http://african.lanl.gov/aDORe/projects/djatoka/ http://sourceforge.net/projects/djatoka " dmn Greenstone 3 wishlist 372 investigate Media Info as a multimedia metadata extractor Collection Building: Plugins major moderate defect nobody new 2008-09-24T14:32:03+12:00 2009-02-01T05:33:20+13:00 "http://mediainfo.sourceforge.net/en http://sourceforge.net/projects/mediainfo/ " dmn Greenstone 2 wishlist 373 Windows Remote GS server can't delete GLI major moderate defect nobody new 2008-10-01T18:06:28+13:00 2008-10-23T13:32:08+13:00 Remote GS server isn't allowed to delete on Windows. This is a problem, because even rebuilding an open collection on Windows using a remote GS fails because it is unable to delete lock files between rebuilds. ak19 Greenstone 2 wishlist 374 Linux remote GS server can't delete when GLI user folder is on network GLI major moderate defect nobody new 2008-10-01T18:07:20+13:00 2009-02-26T13:59:01+13:00 Remote GS server and deleting on Linux: When the .gli folder is on the network (home), GLI has problems deleting a collection. ak19 Greenstone 3 wishlist 376 implement SWORD protocol for deposit Collection Building enhancement moderate defect nobody new 2008-10-10T13:08:14+13:00 2008-10-10T13:08:14+13:00 " http://www.ukoln.ac.uk/repositories/digirep/index/SWORD " dmn DEC Release 2.86 379 tidy up demo collections-1 Sample Collections major high task kjdon assigned 2008-10-23T10:38:52+13:00 2011-01-11T15:14:32+13:00 "Check the documented example collections to make sure the descriptions are still valid with upcoming 2.84 release, and that they work properly. " kjdon DEC Release 2.86 381 tidy up demo collections-3 Greenstone2 Runtime minor low feature kjdon new 2008-10-23T10:44:55+13:00 2009-03-30T11:59:31+13:00 "We want to get rid of the collection images - the scripts to generate them currently don't work, and they are therefore hard to edit. We can use the same idea as general collection icons by having background images specified in css. Revisit our idea about collection types. Currently have public/private. Add a new type 'dec' or better word. change public true|false to collection_type public|private|dec Library, when it goes through and makes the home page should put dec collections into a separate area, and tag the area/collection with dec so that a different background image can be used. Incorporate David's group collections into this too so that the same mechanism is used. " kjdon Greenstone 2 wishlist 383 collection specific cgi-args Greenstone2 Runtime enhancement moderate feature kjdon assigned 2008-10-23T11:36:37+13:00 2009-02-12T09:36:31+13:00 "Can we make it possible to have cgiarg directives (like we can put into main.cfg) in the collect.cfg file? " kjdon Greenstone 3 wishlist 384 user specified cgi arg defaults Greenstone3 Runtime enhancement moderate feature nobody new 2008-10-23T11:38:42+13:00 2008-10-23T11:38:42+13:00 "Greenstone 2 allows cgiarg directives in main.cfg to specify what the default should be, eg for language, some/all searching, whether casefolding is on by default etc. Can we implement a similar mechanism for greenstone 3? Need site wide and collection wide defaults. Does it make sense for gs3?" kjdon Greenstone 2 wishlist 386 check old plugin code Collection Building: Plugins minor low task kjdon assigned 2008-10-23T12:00:17+13:00 2010-04-13T13:34:58+12:00 "During the plugin overhaul, I have left a couple of methods in, renamed to ***_old. Check the code in these and make sure its not still needed, and then delete." kjdon Greenstone 2 wishlist 387 Modifying plugin arguments Collection Building enhancement low feature nobody new 2008-10-23T12:04:03+13:00 2009-02-26T14:10:35+13:00 "It would be nice to add in to PrintInfo (or other place?) some methods to modify plugin arguments. Useful things: Change the default value for a particular argument. This way plugins can override the defaults for super class arguments. This should get rid of the need to have process_exp arg in each plugin. Make an argument disabled, with a reason, which GLI could then display. For example, ImageCOnverter could disable all its arguments if ImageMagick is no where to be found. Similarly for GISExtractor, if the gis stuff is not installed." kjdon Greenstone 2 wishlist 388 Disabled plugin arguments GLI enhancement moderate feature nobody new 2008-10-23T12:05:40+13:00 2009-02-26T13:52:45+13:00 "Once we have added a disabled element to the argument info stuff used by plugins, scripts etc (see #387), make gli look for that, and display the option greyed out and disabled, with the reason added to the tooltip. " kjdon Greenstone 2 wishlist 389 depositor link back to collection Depositor enhancement moderate enhancement nobody new 2008-10-23T12:56:22+13:00 2008-10-23T12:56:22+13:00 "When using the depositor, once you have added a document, there is not an easy way to add another document. You have to click on the depositor link and start from scratch, choosing a collection. And on the final page if you have depositonly, you don't even get the depositor link. Make a link back to the first page of adding a document, for both sample workflows, so that you don't have to select the collection each time." kjdon Possible 2.88 Release 390 pdf conversion to text Collection Building: Plugins minor moderate defect nobody new 2008-10-23T13:01:33+13:00 2011-06-23T20:29:34+12:00 "If you select convert_to text for PDFPlugin, it tries to run pdftotext. But we don't supply this, and the conversion fails. Should we supply it? Should we try a different format? " kjdon Greenstone 2 wishlist 394 Download panel: Logs GLI major moderate defect ak19 new 2008-10-24T18:26:15+13:00 2009-02-26T13:58:37+13:00 "Need to have a separate log file for each download instance. Check that this also reduces the memory consumed by the Java program on each subsequent download that has been launched and completed." ak19 Greenstone 2 wishlist 397 Plugin tidy up Collection Building: Plugins minor low task kjdon new 2008-11-06T14:01:04+13:00 2009-02-26T14:19:31+13:00 "Several of the convert type plugins have a method called convert_post_process. This used to ensure utf8. Someof these have been renamed convert_post_process_old, some are still there. Need to go through and see if we need this or not. PostScript, PDF, Word, Excel plugins." kjdon Greenstone 2 wishlist 398 MGPP document output Greenstone2 Runtime minor moderate defect nobody new 2008-11-10T10:35:42+13:00 2009-02-26T14:07:23+13:00 "The text output for a document from mgpp includes tags (or tags if viewing doc level??). These are invalid tags for HTML and so causes an error in the resulting page. Can we get rid of them??" kjdon Greenstone 2 wishlist 399 GLI config file GLI minor moderate enhancement nobody new 2008-11-10T12:57:13+13:00 2009-02-26T14:06:57+13:00 "Currently GLI has changed to use one config file for all greenstone 2 and/or 3 installations. If you are regularly switching between multiple greenstone 2 or 3 or both installations, then you have to keep entering your library_url. SHould we have one file per installation? eg it could have the version number in the file name. And the greenstone uninstaller should delete the config file (or the whole .gli directory?? - not so good ifyou have other installations using it)" kjdon Greenstone 3 wishlist 400 image processing tools Collection Building enhancement moderate defect nobody new 2008-11-11T10:26:32+13:00 2008-11-11T10:26:32+13:00 " investigate whether we could utilise any of these tools: http://www.linux.com/feature/152694 " dmn Greenstone 2 wishlist 404 large file upload remote GLI GLI minor low defect nobody new 2008-11-17T13:11:38+13:00 2009-02-26T14:19:15+13:00 "Renate reported problems with remote gli when uploading large image files: If we upload folders with images that are quite big in files size (3-5 MB) then the images are not reaching the server and it just hangs. We then uploaded folders with small sized images, and then there is no problem. There was a Java error message about 'Java heap stack out of memory' while doing this when we tested it from the client on VWWare with Windows. On the Mac OS the error message did not show. Can we, a: fix it, or b: at least show the error on the Mac. " kjdon Greenstone 2 wishlist 408 search result sorting-lucene Collection Building enhancement moderate enhancement nobody new 2008-11-24T11:05:08+13:00 2009-02-26T13:51:10+13:00 "Two queries from Diego: Lucene has the option to sort search results by choosing criteria in a list box. Can we also specify ascending or descending order? Is it possible that Lucene only sorts search results when doing queries at document level?. When querying by section level, no order is possible!!!. -this may be because the metadata is not added to the section level index. Does it work if you specify sections index document metadata?" kjdon Greenstone 2 wishlist 409 reverse sort in classifiers Collection Building: Classifiers enhancement moderate enhancement nobody new 2008-11-24T12:24:28+13:00 2009-02-26T13:49:59+13:00 Add an option to the classifiers to sort in reverse order. kjdon Greenstone 2 wishlist 410 Provide a simple way to install GS2 on a hosting company Installation enhancement moderate feature nobody new 2008-11-24T13:19:27+13:00 2009-02-26T13:49:19+13:00 "We could provide a small php installation script that people could upload to their hosting company (ISP) in order to simplify the installation process. Then this php script will download a tar.gz binary distribution of GS2 directly from greenstone.org servers in just a few seconds (from a server to another server). Then the user will be able to choose where to install Greenstone2, the script will first check for permissions (and ask the user to chmod using ftp if needed) then will extract everything in the right place. The script will then edit the lines in gsdlsite.cfg according to the current configuration. So in just a few minutes anyone should be able to install GS2 on their cheap hosting server as long as FTP, PHP, PERL, are available. It should be as simple are installing a modern Blog or CMS script. We should also take into consideration the CPU usage of GS2 since most people will use inexpensive (or even free) shared hosting servers. So in order to not have their account suspended we must make sure that GS2 will not use too much memory and CPU. This is explained in another ticket ""GLI update for remote greenstone server""." max Greenstone 3 wishlist 411 GS3: Deleting a collection on Windows when server is running Greenstone3 Runtime minor moderate defect nobody new 2008-11-24T15:09:21+13:00 2009-01-06T15:31:34+13:00 When the GS3 server is running on a Windows machine, a collection cannot be completely deleted either from GLI or Windows explorer. The database file in the collection's index folder is (also) kept open so that it can't be deleted. ak19 Greenstone 3 wishlist 412 GS3: txtgz file to gdbm file requires that server side runs setup.bat Greenstone3 Runtime major moderate defect nobody new 2008-11-24T15:19:13+13:00 2008-11-24T15:19:13+13:00 "The txtgz-to-gdbm.pl converts the txtgz database file into the ldb (bdb?) file. It is called from util/GDBMWrapper.java This conversion works successfully when gs2build/setup.bat is first run in a DOS prompt followed by starting the tomcat GS3 server from the same terminal. If setup.bat is not run, the greenstone.log file indicates that txtgz-to-gdbm.pl could not be found by the server side code in util/GDBMWrapper.java While the temporary solution of running setup.bat before starting the GS3 tomcat server works for now, the goal is that the web directory of GS3 should be all that is required for the server to get going. This means that the bin/script folder containing txtgz-to-gdbm.pl would not be accessible in those cases where the web directory is located elsewhere (like inside an external tomcat folder). And similarly, txtgz-to-gdbm.pl is dependent on txt2db which is also not part of the web folder. Therefore some other solution needs to be found to solve this in future." ak19 Greenstone 2 wishlist 413 GLI update for remote greenstone server Collection Building enhancement moderate feature nobody new 2008-11-24T15:28:15+13:00 2009-02-26T13:48:58+13:00 "We may think of an option when using GLI with a remote GS2 server (hosted on a shared server for instance) that will allow to do the building locally and then zip the built collection and send it to the server. This will help the server to not over use the CPU or their cheap shared servers when building collections so people won't have their account suspended. The server will only be used to serve collections." max Greenstone 2 wishlist 418 ifl arg changes display Greenstone2 Runtime minor low defect nobody new 2008-11-27T13:27:41+13:00 2009-02-26T14:18:56+13:00 "If you view search results (eg 'mail samoa' in demo collection) with ifl=2 (do a search then manually add this to the url), then the encoding is wrong for the 'No' in the title. Its correct if ifl=2 is not part of the url. Why????" kjdon Greenstone 3 wishlist 419 lucene uses deprecated classes Collection Building minor moderate defect nobody new 2008-11-27T20:47:53+13:00 2008-11-27T20:47:53+13:00 "When compiling common-src/indexers/lucene-gs get the message that org.apache.lucene.index.IndexModifier in org.apache.lucene.index has been deprecated. Fix." kjdon Greenstone 2 wishlist 420 depositor in local library Greenstone2 Runtime minor moderate defect nobody new 2008-11-27T21:20:18+13:00 2008-11-27T21:20:18+13:00 "You can't use the depositor with local library. It doesn't really make sense. However, if you try, you get some invalid cgi param error. Can we make it give a better error message (eg Depositor doesn't work with local library)?" kjdon Greenstone 2 wishlist 421 delete collection in GLI GLI minor low defect nobody new 2008-12-01T13:33:53+13:00 2009-02-26T14:18:37+13:00 "The delete prompt has been changed to not allow you to delete the collection that is open. That is fine except that it still appears in the list of collections, but nothing happens when you click it. It would be good if the collection details pane changed to display a message such as ""this collection is currently open in GLI and so can't be deleted. Please close the collection then delete it""" kjdon Greenstone 2 wishlist 423 missing usage.txt Greenstone2 Runtime minor low defect nobody new 2008-12-01T14:22:29+13:00 2009-02-26T14:18:16+13:00 "If usage.txt has been deleted (or is not there) then you don't get any pages logged. usage.txt has been removed from svn so is not there is an svn repository, but is there in a release. The code should create it if its not already present." kjdon Greenstone 2 wishlist 425 build success test Depositor minor moderate defect nobody new 2008-12-04T12:45:07+13:00 2009-02-26T14:05:07+13:00 "Currently the build script (build, not buildcol.pl) used by depositor/collector tests for existence of ldb/bdb file to see if colleciton built successfully. I just added db file as an option - this is created for sqlite dbs. And we may make gdbm file gdb. Is build.cfg file a sufficient test, then will always have the same name. Check and change code if this is enough." kjdon Greenstone 3 wishlist 429 mixed path separators in CATALINA_TMPDIR Greenstone3 Runtime major moderate defect nobody new 2008-12-05T10:05:50+13:00 2008-12-05T10:05:50+13:00 " in the GS3 server log this directory has mixed up path separators: INFO : [2008-12-05 08:52:06] RunAnt.run() [exec] Using CATALINA_TMPDIR: C:\Documents and Settings\dmn\Greenstone3/packages/tomcat\temp" dmn Collection building wishlist 431 OAI static repository Collection Exporting enhancement moderate feature nobody new 2008-12-08T12:02:45+13:00 2009-08-03T15:50:44+12:00 "Instead of running a OAI repository, you can generate [http://www.openarchives.org/OAI/2.0/guidelines-static-repository.htm OAI static repository files] Do we want to provide this as an option to export?" kjdon Possible 2.88 Release 432 RemoteGreenstoneServer and GLIApplet tasks GLI major moderate defect ak19 new 2008-12-08T13:56:06+13:00 2011-06-07T16:38:36+12:00 "1. BUG: When the GLIapplet is used to build and preview a collection, and if the browser is then pointed elsewhere before the user returns to open the GLI applet and rebuilds the collection, then on clicking Preview the browser crashes. Possible solution: Consider turning authentication task into an Action as well, by adding a ping command on the gliserver.pl end that is sent the username and password and which it can verify. 2. Applet: If either the gliserver.pl or authentication weren't provided, then pressing the Launch GLI button should once again ask for whichever piece of data was missing before GLI itself is launched. 3. When working with the GLI client against a remote GS server, files with French filenames are zipped in such a way that the original encoding is ruined. This means that the files the remote greenstone server receives have the wrong filenames and things go wrong from there (interlinking, file display names, missing images in htmls). The solution maybe to check whether 7-zip can handle such filenames and if so, to consider moving to a Java-based solution that uses a 7-zip API." ak19 Greenstone 3 wishlist 434 6Gb catalina.out logfile created by GS3 on Puka nzdl.org major high defect nobody new 2008-12-09T13:13:03+13:00 2009-08-03T11:36:25+12:00 This is hogging up a lot of space and is being constantly appended to. Need to find a way of managing this data. Perhaps by truncating the file by a certain amount each time or hunting down what part of the Tomcat webserver is outputting all those statements and change the settings of it (maybe so it outputs statements to the log only on warning or error?) ak19 Greenstone 3 wishlist 436 .Propeartie file Utf-8 defect !!!! Greenstone3 Runtime major moderate defect nobody new 2008-12-13T22:29:26+13:00 2008-12-13T22:29:26+13:00 "when AbstractBrowse.properties file is saved as a utf-8 file the Browse text in the gs3 online interface is not recegnized any more !!! probably a defect" aminhjz Greenstone 2 wishlist 440 GLI fails to stop the local library server GLI minor moderate defect nobody new 2008-12-18T15:10:36+13:00 2008-12-18T15:10:36+13:00 "Conditions to trip bug: On a windows machine, start the local library server and then try to start GLI (from start menu or with the batch scripts). Expected behavior: GLI will start up promptly and either work with the currently running server or stop the existing server and start a new one. (Of these choices, the former would clearly be the best.) Actual behavior: GLI launches (i.e. the JVM is created and GLI code starts running) but it takes 60 seconds for any GUI components to appear. The first and only component to appear is a message reporting that GLI failed to stop the server and asking if you wish to wait another minute. Closing the server and then clicking 'yes' does not make GLI appear straight away as I had hoped, but started what looked like another 60 second wait (during which time I gave up and closed everything, and just started GLI on its own)." oranfry Greenstone 3 wishlist 442 Background image for greenstone collections Collection Building major moderate defect nobody new 2008-12-18T15:26:24+13:00 2009-08-03T15:53:35+12:00 "The method for getting the left-aligned strip of waves as a background image in greenstone collections is severely old-school, and wider monitors are now revealing its flaws. Instead of a 1500px-wide background image which is mostly whitespace, we need an image which is only as wide as the actual strip of waves, and to use css to align it and make sure it's not repeated. Something like: background: transparent url('waves.gif') scroll repeat-y left top;" oranfry Collection building wishlist 444 Search result context display with Lucene Greenstone2 Runtime enhancement low feature nobody new 2008-12-20T21:25:56+13:00 2015-11-13T15:43:43+13:00 "Can we do this? Showing the context of the search result like google does. Jeff Crump says that he has seen this with other Lucene implementations." kjdon Greenstone 2 wishlist 445 image collections affecting pagedimg collections Collection Building minor low defect nobody new 2008-12-21T13:21:36+13:00 2009-02-26T14:17:54+13:00 "Gavin Spomer has reported that building an image collection affects the thumbnail files in a paged image collection. This was in 2.72. Now we use different file names so I don't think this will happen any more. But if someone has time it would be useful to check to make sure." kjdon Greenstone 3 catch-up 450 CJK GS3 Greenstone3 Runtime minor high enhancement nobody new 2009-01-06T15:44:21+13:00 2009-08-03T13:05:36+12:00 "I don't think that Greenstone 3 does anything special with a query when CJK segmentation has been used during collection building. The query needs to be segmented using the same rules that were used during building. " kjdon Greenstone 3 catch-up 451 sqlite support for greenstone3 Greenstone3 Runtime enhancement moderate feature nobody new 2009-01-06T16:06:21+13:00 2009-08-03T13:02:25+12:00 "Collection building can now use sqlite as the database. Need service classes for Greenstone3 to serve these databases. " kjdon Greenstone 3 catch-up 452 dynamic classifiers in gs3 Greenstone3 Runtime enhancement low feature nobody new 2009-01-06T16:08:24+13:00 2009-08-03T12:57:50+12:00 "Implement service classes to support dynamic classifiers for GS3. #451 needs to be done first (adding general support for sqlite databases). " kjdon 3.11 Release 454 pageHead template declaration in style.xsl expected for GS3 admin authentication Greenstone3 Interface major moderate defect max new 2009-01-07T15:35:58+13:00 2021-03-22T09:46:26+13:00 "The change made in revision 17018 (see http://trac.greenstone.org/log/greenstone3/trunk/web/interfaces/default/transform/style.xsl?action=stop_on_copy&rev=18317&stop_rev=&mode=stop_on_copy&verbose=on) is presenting me with an error about the template pageHead being unknown when I click on the Administration page link off Greenstone 3's default interface. The error message was encountered when I was following the Remote_Greenstone3 instructions on the wiki to test that the Remote Greenstone3 was still working (http://wiki.greenstone.org/wiki/index.php/Remote_Greenstone3). The change made in revision 17018 is that the pageHead template declaration and another template element were removed from style.xsl in web/interfaces/default/transform/ because this ""HTML skeleton has been copied to other XSL files."" Perhaps that change causes an error only when trying to display the authentication after clicking the Admin page link. I've temporarily reintroduced the xsl code that was removed, but is there a way that the original change can be preserved and still get it to work with the authentication of the Admin page?" ak19 Greenstone 3 wishlist 457 java version detection Installation minor moderate task oranfry assigned 2009-01-08T13:34:04+13:00 2009-03-20T13:45:30+13:00 "I saw this discussion in an email from David B, and thought I'd add a ticket to make sure that it has been thought about and implemented if its a good idea. --- As a result of your initial bug report, Oran, Max and I had started to discuss the idea of checking more carefully the version and vendor of Java being used as a general approach (regardless of OS you're running on) as a way to give better feedback to the user. My thinking is we should print out a warning if the Java property ""java.vendor"" turns out to be GNU's, but for now let any of the other ones through without a warning. I don't know for certain, but I suspect the JRE you get on Windows will be Microsoft branded, and we don't want people using a standard Windows solution to be continually pestered with a warning when it's been running on the OS untroubled for a long time. Oran -- we should put this check in both the installer and GLI/GEMs. Also, can you update GLI/GEMs to print out the java.vendor property as part of the start up sequence. That way we can monitor it, longterm, and begin to get a better feeling for what sort of variations we might see in this field. " kjdon Collection building wishlist 465 GLI handling deprecated script options GLI enhancement moderate enhancement nobody new 2009-01-15T15:26:24+13:00 2009-08-03T15:50:27+12:00 "It would be nice is for plugin/classifier/script options there was a new field, deprecated. Then if its set to true, GLI will remove the option from the collection config file if it has been set, and somehow highlight to the user that the option is deprecated. eg maybe there is a deprecated message that GLI displays which tells the user what to do instead. " kjdon Greenstone 2 wishlist 467 Installer issues when installing GS2.81 in R-block Installation major moderate defect oranfry assigned 2009-01-19T11:25:33+13:00 2009-06-24T15:21:15+12:00 "When installing 2.81 in the R-block lab on Harry Johnson's server machine, there were two strange things: 1. Some file called ""orange volt something"".dll was left behind. 2. Search4J seemed to have worked out the correct path for java.exe (where the path referred to the included jre), but then the JAVA_EXECUTABLE was still set to nothing and GLI and gems would not run. The temporary solution adopted was to manually fix JAVA_EXECUTABLE to be the correct path. " ak19 Greenstone 3 wishlist 470 GLI in GS3 metadata.xml file error GLI minor moderate defect nobody new 2009-01-27T13:38:25+13:00 2009-01-27T13:38:25+13:00 When using GLI in gs3, if you try to drag in a file from another greenstone collection, it tries to read metadata.xml from the site directory. I guess its going up directories to try and find metadata.xml files so it can bring the metadata with it, but it needs to stop somewhere maybe. kjdon Greenstone 2 wishlist 473 language selector on each page Greenstone2&3 enhancement moderate feature nobody new 2009-02-12T09:41:45+13:00 2009-02-26T13:47:02+13:00 "In the areu collection, they have a language selection drop down in the navigation bar. I think this is really neat, and its easy to do. It would be good if we can add in macros/xslt for this, then the users can easily add it in if they want." kjdon Greenstone 3 wishlist 477 GS3 help pages Greenstone3 Interface enhancement moderate feature nobody new 2009-02-16T10:10:09+13:00 2009-02-18T12:26:28+13:00 Need to add these like the gs2 ones. kjdon Greenstone 3 wishlist 478 search prefs Greenstone3 Interface minor low defect nobody new 2009-02-16T10:13:54+13:00 2009-02-16T10:13:54+13:00 In search preferences, the type of search is bolded. If you change the type and click set prefs, then the new one becomes bold. However, just changing the type doesn't change the bolding, which it should. kjdon Greenstone 2 wishlist 480 get rid of LOCAL_LIBRARY=1 from top level win32.mak? Greenstone2 Runtime minor low annoyingness kjdon assigned 2009-02-16T10:31:10+13:00 2009-02-26T14:17:40+13:00 When you compile greenstone 2 currently, you need to run nmake twice, once with LOCAL_LIBRARY=1 and once without. It would be nice if the top level makefile handled this for you, ie it ran make twice itself with the relevant flags. kjdon Greenstone 3 wishlist 481 GS3 with no coll building Greenstone3 Ant Code enhancement low enhancement kjdon new 2009-02-16T10:33:01+13:00 2009-02-16T10:33:01+13:00 "If we want to install GS3 with no collection building ability, which bits of gs2 do we need? all of common-src? Just indexers? gdbm/sqlite? Work this out and add code into ant file to make it work." kjdon Greenstone 2 wishlist 482 sqlite tar file Installation minor moderate task nobody new 2009-02-16T10:35:35+13:00 2009-02-16T10:35:35+13:00 "sqlite is in svn as a tar.gz file. Do we provide tools to extract this on windows? All the other windows zip files are now unzipped by the make file, so the user doesn't have to remember it. Can we do the same for this? If we change to a zip then its a pain on linux. file permissions are not maintained in a zip file." kjdon Greenstone 3 wishlist 485 do boolean operators work? Greenstone2&3 minor moderate defect nobody new 2009-02-16T10:46:06+13:00 2009-02-18T12:44:50+13:00 For searching in gs2 and 3, do boolean operators work? I had a note to check this. Perhaps someone complained that they didn't?? kjdon Greenstone 3 wishlist 488 test partition collection GS3 Greenstone3 Runtime minor moderate defect kjdon new 2009-02-16T11:05:04+13:00 2009-02-16T11:05:04+13:00 "I tried this once in GS3 and it didn't work. I think the runtime didn't get the right index name or something. Test again, and make it work if not working." kjdon Greenstone 3 wishlist 489 Derby DB Greenstone3 Ant Code minor moderate defect nobody new 2009-02-16T11:15:01+13:00 2009-02-16T11:15:01+13:00 "This is used by authentication stuff in gs3. In web/sites/localsite/etc there is a lot of derby stuff. I don't think this should be in svn. Here's what Quan had to say: You are right about the log files. Sorry, I didn't pay much attention to the details of this folder that time, because those log files are created automatically along with the db file. I think creating a user database file when greenstone starts is a good idea. Actually, usersDB will be checked or created (if a usersDb couldn't be found) when a user visiting Authentication from a browser. But if the user directly starts a remote gli and no usersDB available for the site, it is going to be problem. So I think a usersDB should be created while creating a new site directory. The java code for creating usersDb and adding a admin user to the table is Greenstone3\src\java\org\greenstone\gsdl3\util\DerbyWrapper.java. So, maybe we can get rid of all the stuff from svn, but then get ant to create a new db during installation?? " kjdon Greenstone 2 wishlist 491 textual interface looks yuck Greenstone2 Runtime minor moderate defect nobody new 2009-02-18T09:11:26+13:00 2009-06-24T15:21:28+12:00 "The textual interface (choose form preferences page) which is for visually impaired people looks horrible - mainly the top area with coll icon, home help pref and nav bar. This is talked about in the book, so needs tidying up. Should just be a matter of a few macro/css changes." kjdon Greenstone 2 wishlist 492 handle missing GreenstoneXMLPlugin GLI minor moderate defect nobody new 2009-02-23T12:27:00+13:00 2009-02-23T12:27:00+13:00 "I managed to get a collection with no GreenstoneXMLPlugin (outside of GLI probably). I didn't realise and couldn't work out why my collection wasn't working. In Librarian mode there was no indication that anything was wrong. When I changed to Expert mode, I saw in the log that nothing could process the doc.xml files. We need to warn the user. Either tell them that they have no archive processing plugin, or bring through the "" no plugincould process..."" messages from buildcol" kjdon 3.11 Release 494 new coll change back to MG GLI major moderate defect nobody new 2009-02-23T15:04:32+13:00 2021-03-22T09:46:26+13:00 "Check that changing back to MG from a new collection (which is mgpp by default) still works ok. I have removed the mg lines from the config file. " kjdon Greenstone 2 wishlist 496 authenticate administration page Collection Building enhancement moderate enhancement kjdon new 2009-02-25T09:56:15+13:00 2009-02-25T09:57:26+13:00 "Some people would like to have the entire administration pages needing authentication, rather than just the add user bits. This is easy to do in the code: in check_cgiargs in statusaction.cpp, always set {{{ args[""uan""] = ""1""; args[""ug""] = ""administrator""; }}} We need to make an option in main.cfg for this so users don't need to edit code to achieve this." kjdon Greenstone 2 wishlist 502 update manuals Documentation enhancement moderate task nobody new 2009-02-26T13:34:27+13:00 2009-09-02T11:16:02+12:00 "The manuals will go on the UNESCO cd-rom. There could be several stages of updating: 1. update GLI section 2. Update installers guide to work with new installer 3. Update other bits - a big job. What about translations?" kjdon Greenstone 2 wishlist 506 more security for admin pages Collection Building critical high defect nobody new 2009-02-28T10:09:32+13:00 2009-02-28T10:09:32+13:00 Currently, username and password are sent in the clear in the URL. Can we use https or something to make it more secure? kjdon Greenstone 2 wishlist 512 dodgy text_t functions Greenstone2 Runtime major moderate defect kjdon new 2009-03-15T16:15:54+13:00 2009-03-18T13:32:38+13:00 "From an email from Michael: We've recently found that these functions (findchar, findlastchar, findword, and maybe others in the text_t class) are a bit dodgy. It's not really clear what should be done about these, and unfortunately I don't have any time to work on them. Hopefully someone there can look into this. The worst function is findlastchar(). The problem is that it dereferences the ""last"" iterator -- but in most cases this will be a text_t end() value, in which case Greenstone will crash if the OS is strict about checking memory access. It seems that the best way to fix this is to just change findlastchar() so it decrements the last iterator before starting the loop. However, you can't do this because the last iterator might not be a text_t end() value -- in which case you're not searching the full range that was specified. The other find functions (e.g. findchar) are a bit different -- the problem with these is the return value. If you're not passing in a text_t end() value as the ""last"" parameter, there is no way to tell if the character you're searching for is at the end of the range specified -- because you get the same value back in this case or if the character doesn't appear at all. Things would be much cleaner if these functions just took a text_t as the parameter, instead of two iterators. In this case you could write implementations of these find functions that always worked properly and returned sensible values. I think this would cover about 80% of the usage of these functions too -- in most cases text_t.begin() and text_t.end() are used as the arguments. It's the other cases that are the problem, and I don't know what to do about those. Maybe all we can do is mark the two iterator versions of the find functions as being deprecated, and add comments explaining their shortcomings. I guess the other approach is that we clearly mark these functions as taking ""first"" and ""last_plus_one"" iterators, since this is what is being provided in most cases. If it is clear that the last character isn't being considered then the return value problem goes away, and we can fix findlastchar() as well. In this case I think we just need to change the ""last"" variable names in the functions (so very inaccurate), add comments to explain what is being expected for this argument, and fix findlastchar(). " kjdon Greenstone 3 wishlist 515 Go over tutorials with Greenstone 3 Collection Building major moderate defect ak19 new 2009-03-16T16:27:19+13:00 2009-03-30T11:43:03+13:00 "The book states that the tutorials work with Greenstone 3 as well. Lets make Greenstone 3 versions of tutorials that work with the 3.04 release." ak19 Collection building wishlist 520 Change installer to allow client-gli installations Installation major moderate task oranfry assigned 2009-03-20T16:13:04+13:00 2009-08-03T15:53:04+12:00 "Plan B: Scrap the idea of a client-gli distribution, and add options to the main installer to allow the user to install just client-gli if they want to. To achieve this, create a new page in the installer interview with two check-boxes: {{{ Do you want to: [ ] Build collections locally? [ ] Build collections remotely? }}} (Please correct the following as necessary!) If neither option is checked, install only runtime system. If only ""locally"" is checked, install everything but don't link client-gli script on the start menu. If only ""remotely"" is checked, this is the client-gli installation. Install gli and gs2build, and runtime system for local previewing. Link only remote-gli on the start menu. If both are checked, install everything and link gli and remote-gli on the start menu." oranfry Greenstone 2 wishlist 521 Installer instructions for web library Installation major moderate task oranfry assigned 2009-03-20T16:14:47+13:00 2009-08-03T15:52:29+12:00 Put into the main installers a message explaining how to set up a web library, towards the end of the install process. oranfry Greenstone 3 wishlist 524 make install Installation major high defect kjdon new 2009-03-25T13:48:25+13:00 2009-03-30T11:46:22+13:00 "make and make install seem to be doing some things twice (indexers, just jni??) Compilation is happening on make install. FIX!!" kjdon Greenstone 2 wishlist 531 Change Greenstone2 build code to Ant Greenstone2 Build Code minor moderate task oranfry assigned 2009-04-08T14:48:40+12:00 2009-06-25T10:32:44+12:00 "There are some problems with the existing Greenstone2 build system. Currently gsdl is compiled with the usual 'configure, make, make install' in the top level gsdl folder. This will first perform configure on all components, then make all components, then make install all components. This is problematic as the configure part of some components requires the compiled binaries of other components to be present. A better build system would do 'configure, make, make install' on individual components one by one in an appropriate order. Propose to replace the top level gsdl build files (configure.*, Makefile.*, etc), and those for build-src, runtime-src (and common-src?) and indexers with ant build.xml files, and move the logic for building each component (including any necessary compilation options) into these files. Also, any java-only components which are controlled by us (ie, gli and maybe some others) could use a build.xml file to replace the existing build code. We should put a minimal ant distribution into gsdl in the repository, to remove the prerequisite of having ant to compile Greenstone. Could also provide, as an alternative to the build.xml files, the cascadeMake.sh files which davidb has started work on. All this would go in a top level folder inside gsdl. " oranfry Greenstone 2 wishlist 543 tidy up windows compiling Installation major moderate defect nobody new 2009-04-19T16:04:38+12:00 2009-04-19T16:04:38+12:00 "Currently, indexers seems to be compiled twice when compiling on windows. tidy this up. Also, look at tidying up the jni compilation so it only gets done if needed (on linux and windows) " kjdon Greenstone 2 wishlist 544 Replacing GSDLHOME var in gsdlsite.cfg on Windows Collection Building trivial low defect ak19 new 2009-04-20T15:07:11+12:00 2009-04-20T15:07:11+12:00 "This turned out to be a bigger problem than anticipated even after finding a GPL version of sed. 1) Get gsed from http://student.northpark.edu/pemente/sed/ (gsed407x.zip) and put it in bin windows. 2) Modified makegs2.bat with the following prepended at the start of the existing ENDOK target (Oran managed to successfully get backlashes in the gsdlpath replaced with escaped/double backslashes): :ENDOK if not exist llssite.cfg copy llssite.cfg.in llssite.cfg if not exist glisite.cfg copy glisite.cfg.in glisite.cfg pushd ""%CD%"" set gsdlpath=%~dp0 CD /D ""%gsdlpath%"" echo ""GSDLPATH: %gsdlpath%"" :: the following line escapes backslashes in gsdlpath for the sed regex for /F %%T in ('""echo %gsdlpath% | bin\windows\sed.exe s@\\@\\\\@g""') do set safepath=%%T echo %safepath% if not exist cgi-bin\gsdlsite.cfg bin\windows\sed.exe ""s@\*\*GSDLHOME\*\*@\""%safepath%\""@g"" cgi-bin/gsdlsite.cfg.in > cgi-bin/gsdlsite.cfg set gsdlpath= popd 3) While the above works when gsdlpath contains no spaces, it fails at other times." ak19 Greenstone 2 wishlist 546 check branding Installation major high defect nobody new 2009-04-22T10:16:53+12:00 2009-04-22T10:16:53+12:00 check that the release kit successfully brands the java server program with the version string - looks like its not working (I get 'version 2-trunk'). oranfry Collection building wishlist 547 Space-proofing gli Collection Building major moderate defect oranfry assigned 2009-04-22T10:33:38+12:00 2009-08-03T15:52:10+12:00 "Discovered under ubuntu linux. The gli java code has trouble with spaces in install paths. I installed Greenstone to /home/oranfry/Desktop/Green Stone Two and ran gli: {{{ oranfry@bedrock:~/Desktop/Green Stone Two/gli$ ./gli.sh Greenstone Librarian Interface (GLI) Copyright (C) 2006, New Zealand Digital Library Project, University Of Waikato GLI comes with ABSOLUTELY NO WARRANTY; for details see LICENSE.txt This is free software, and you are welcome to redistribute it Greenstone version found: 2 Checking GSDL2: /home/oranfry/Desktop/Green Stone Two Your environment has successfully been set up to run Greenstone Adding in executable path for apache-httpd GSDLHOME is: /home/oranfry/Desktop/Green Stone Two Perl: /usr/bin/perl Java: /home/oranfry/Desktop/Green Stone Two/packages/jre/bin/java ImageMagick bundled with Greenstone will be used, if you wish to use the version installed on your system (if any) please go to /home/oranfry/Desktop/Green Stone Two/bin/linux and rename the folder called imagemagick to something else. Running the Greenstone Librarian Interface... Version: 2s2009.04.22 java.io.IOException: Cannot run program ""/home/oranfry/Desktop/Green"": java.io.IOException: error=2, No such file or directory at java.lang.ProcessBuilder.start(Unknown Source) at java.lang.Runtime.exec(Unknown Source) at java.lang.Runtime.exec(Unknown Source) at java.lang.Runtime.exec(Unknown Source) at org.greenstone.gatherer.Gatherer$ExternalApplication.run(Gatherer.java:1126) Caused by: java.io.IOException: java.io.IOException: error=2, No such file or directory at java.lang.UNIXProcess.(Unknown Source) at java.lang.ProcessImpl.start(Unknown Source) ... 5 more }}} A minute passes and a dialog box appears, plus more output to the console: {{{ User cache dir: /home/oranfry/.gli/cache/ Done! }}} The dialog box said: ""The greenstone local library server has not responded to the automatic close command for sixty seconds. Do you wish to wait for another minute?"" Clicking ""no"" brings up gli as normal. It looks like gli tried to execute a command to stop the server if it was running, the command failed because of spaces in the install path, and as a result caused a 60 second delay. Fix would be to make sure the command is properly executed by gli even if there are spaces in the install path. " oranfry Greenstone 2 wishlist 548 GLI new collection in collectgroup GLI enhancement moderate enhancement nobody new 2009-04-22T14:52:20+12:00 2009-04-22T14:52:20+12:00 "Currently you can only create collections in the collect folder. It would be nice to be able to create a new collect group, and create a collection in a collect group. Need to make sure it works for remote gli too if we do this. " kjdon Greenstone 2 wishlist 549 GLI code tidy GLI major moderate defect kjdon new 2009-04-23T14:33:09+12:00 2009-04-23T14:33:09+12:00 "with collection groups, we have added methods to getQualifiedNames instead of getName. (search for GroupQualified). Would we ever want to get the base name of a group collection?? Can we just change getName to return the qualified name and get rid of the other one?? need to change import etc calls as currently they are setting collectdir to the group." kjdon Greenstone 2 wishlist 552 GLI existing collections in file tree GLI major moderate defect kjdon new 2009-04-23T15:04:01+12:00 2009-04-23T15:04:01+12:00 "There is a test to see if the coll is the current collection - only checking the final directory name. will probably give false positives for colls with the same anmes in collect groups. Can we do something better? file/WorkspaceTreeNode line 171" kjdon Greenstone 2 wishlist 555 refresh depositor metadata GLI major moderate defect nobody new 2009-04-27T13:15:29+12:00 2009-04-27T13:15:29+12:00 "The depositor metadata list doesn't refresh if a new metadata set is added to the collection. but it should. " kjdon Greenstone 2 wishlist 556 DSpace plugin: process_exp option should be regexp type? Collection Building: Plugins minor low defect nobody new 2009-05-05T02:24:14+12:00 2009-05-05T02:24:14+12:00 "I have had some difficulty with the process_exp option while using the DSpace plugin (which might be the result of my own misunderstanding of its purpose). In any event, I noticed that the mp3 plugin used to have a problem there [http://trac.greenstone.org/changeset/14687], and I see that the DSpace plugin might also need the same fix. " rspeer Greenstone 2 wishlist 559 GLI default format statements GLI minor moderate defect nobody new 2009-05-11T14:15:06+12:00 2009-05-11T14:15:06+12:00 "GLI is adding in searchtypes format statement by default. don't need this if there are no indexes. can we stop GLI adding it in in this case?? Also, if AllowExtendedOptions is set to true, then some of the others are not used eg DocumentButtons (I think). Can we prevent GLI adding them in?? " kjdon Greenstone 2 wishlist 560 metadata ordering in GLI GLI major moderate defect nobody new 2009-05-13T15:27:05+12:00 2009-05-13T15:27:05+12:00 "The metadata sets have an order, and this is kept in Enrich panel. However, in the drop down lists in index/classifier/plugins etc, the order is not maintained. They are in alphabetical order. Can we make these places respect the element ordering too? will need to use MetadataSet.compareElements or something I think." kjdon Greenstone 2 wishlist 561 accessing subcollection metadata Greenstone2 Runtime major moderate defect kjdon new 2009-05-18T15:29:35+12:00 2009-05-18T15:29:35+12:00 "Is there any way we can access _numdocs_ for each subcollection from one of the collection's about pages? _collection:numdocs_ only works for the current collection. greenstone only loads up the current collection I think. We'd need to get colinfo for each subcoll. but maybe could do _demo:numdocs_ where demo was the subcoll name..." kjdon Greenstone 2 wishlist 563 ex metadata set GLI enhancement moderate defect kjdon new 2009-05-20T15:55:14+12:00 2009-05-20T15:55:14+12:00 "Add in all the descriptions for the new elements, and then send off for translation. " kjdon Collection building wishlist 564 metadata type script options don't work GLI major high defect kjdon new 2009-05-27T15:58:14+12:00 2010-04-13T13:55:07+12:00 "If you have metadata type script options (import.pl etc), then in GLI they show up nicely with the language dependent metadata names, and there is a drop down list. However, the display name is being written to the .col file and in the script args (eg dc.Resource Identifier instead of dc.Identifier). Which is wrong. I have changed the type back to string for now. Make this work and change them back." kjdon Greenstone 2 wishlist 565 Group size thing Collection Building: Plugins major moderate defect kjdon new 2009-06-10T13:37:55+12:00 2009-06-10T13:37:55+12:00 "If you use groupsize (eg like in documneted-examples/gsarch-e) then you don't get archiveinf-doc.gdb and therefore archives inf plugin doesn't control the build. Instead, all files in archives get built. I don't know if there is a problem with this or not (don't think so) apart from the fact that you get error messages about archiveinf-src.gdb couldn't be processed. Should they be blocked? and does blocking work anyway in a build??" kjdon Greenstone 2 wishlist 567 GLI build log not helpful for incremental rebuild GLI minor moderate defect nobody new 2009-06-10T16:34:46+12:00 2009-06-23T10:56:27+12:00 "Using Lucene, doing full rebuild and minimal rebuild in librarian mode, get the same output for buildcol. There is nothing there to say that no new text was added or anything. Should try and modify the output to be more helpful." kjdon Greenstone 3 wishlist 569 investigate Aperture as a metadata/text extractor Collection Building major moderate defect ak19 new 2009-06-14T03:54:21+12:00 2009-06-14T03:54:21+12:00 " http://aperture.sourceforge.net/ Aperture is a Java framework for extracting and querying full-text content and metadata from various information systems (e.g. file systems, web sites, mail boxes) and the file formats (e.g. documents, images) occurring in these systems. " dmn Possible 2.88 Release 570 ISIS coping with errors in records Collection Building: Plugins minor moderate defect nobody new 2009-06-16T09:12:42+12:00 2011-06-07T16:40:46+12:00 "ISISPlugin currently fails if there are errors in the records. I am not sure exactly what happens, but maybe it just stops processing when an error occurs. Deleted records in the database that are not cleaned out cause an error. see messages from Ramon Sampang. He has sent a sample database for us to try." kjdon Collection building wishlist 572 Indexing the scanned image pdf document with meta data from winisis Collection Building major moderate defect nobody new 2009-06-16T20:54:07+12:00 2009-08-03T15:51:47+12:00 Sir, We are already having the winisis meta database of about 5000 records(reports) Now we are scanning the original document (each document as a single pdf file). How I can link above meta database with the scanned pdf file, while doing collection building in Greenstone. (I am using ver 281). (Each meta data of the record in the winisis is a document which is a pdf file). Can any body can give suggestion for easy method to combine these two. Ohterwise I have to build the PDF file in Greenstone, then type the meta data of each record. Kindly give your suggestion for easy method. Yours C. Sugumaran, LPSC, ISRO. Tvm. csugumaran Greenstone 3 catch-up 580 scheduling for gs3 Collection Building enhancement moderate enhancement nobody new 2009-08-03T13:18:06+12:00 2009-08-03T13:32:47+12:00 "schedule.pl is gs2 specific. Make it work for gs3. I am going to disable the schedule option bit in GLI for gs3 until this is fixed. " kjdon Greenstone 3 catch-up 582 cdrom for gs3 GLI major moderate defect nobody new 2009-08-03T14:03:34+12:00 2009-08-03T14:03:34+12:00 "how are we going to ""write CD/DVD image"" for gs3??" kjdon Collection building wishlist 605 Use the FITS tool for metadata extraction Collection Building major moderate defect nobody new 2009-08-10T11:49:16+12:00 2009-08-10T11:49:16+12:00 " http://code.google.com/p/fits/ The File Information Tool Set (FITS) identifies, validates, and extracts technical metadata for various file formats. It wraps several third-party open source tools, normalizes and consolidates their output, and reports any errors. " dmn 3.11 Release 612 gs2 interface help messages Greenstone3 Interface minor moderate defect nobody new 2009-08-17T11:04:18+12:00 2021-03-22T09:46:26+13:00 "the way help messages are done is a bit dumb. get the content from teh classifier element, and stick help and short/long on either side. Predefined ones for Title, Subject etc. This doesn't work very well when you have eg dc.Title;Title. I have added this particular example into the properties file as this is what is used by modelcol. But can we think of a better way of doing it? eg if you have dc.Title;Title, really you just want it to use Title in the name." kjdon Collection building wishlist 613 hierarchy classifier for sections Collection Building enhancement moderate feature nobody new 2009-08-17T11:15:02+12:00 2009-08-17T11:15:02+12:00 can we make the Hierarchy classifier work on sections? kjdon Greenstone 2 wishlist 614 regex in conditional format statement Greenstone2 Runtime enhancement moderate feature nobody new 2009-08-17T14:26:30+12:00 2009-08-17T14:26:30+12:00 "Can we implement regex matching in format statements? See http://wiki.greenstone.org/wiki/index.php/How_to_format_the_output_of_your_collection#Conditional_expressions_in_formatstrings Needs a matches operator. A bit tricky as we are doing to operations on text_t so would need to implement regex for text_t. " kjdon Collection building wishlist 615 build option for no copy docs to index Collection Building enhancement moderate enhancement nobody new 2009-08-17T14:50:42+12:00 2009-08-17T14:50:42+12:00 "From an email from Diego: it is well known that every document in import folder generates a folder in archives after the import process. Then, building process copies all source files to index/assoc, so we have duplicated disk space needed to host all files. I have a collection with almost 700.000 tiff files, all imported with Pagedimgplug. This collection is not a static one, every couple of days we add new documents, so we have two options: 1- Use Lucene and incremental building: this sounds interesting but we have many problems with parsing doc.xml files, accents and many other things. 2- Use MGPP: it works great, we have all the features we need but incremental indexing is not possible. So every few days we have to reindex all again, and again... This approach consumes a reasonable time to generate indexes, but it spends a lot of time copying 700.000 files from archives to building/assoc, and deleting the old index folder with the other 700.000 files. The questions are: a- Is it possible to link to source files directly from archives folder?. This will result in saving a lot of time because copying files form archives to assoc is no more necessary. I remember that someone asked for something like this, but I can´t find the mail in the email archives collection. I think that builcol.pl must be modified to work this way. Is there anybody out there that can do it?. b- Is it possible to add an option to future releases where the user can choose weather buildcol with source docs in place (in archives folders) or not?." kjdon Collection building wishlist 620 metadata display names in format statements GLI major moderate enhancement nobody new 2009-08-31T15:51:03+12:00 2009-08-31T15:51:03+12:00 "Can we use metadata display names in format statements in GLI? Would need to convert to identifiers before writing out. " kjdon Greenstone 2 wishlist 622 group cross coll searching Greenstone2 Runtime major moderate feature nobody new 2009-09-02T11:00:26+12:00 2009-09-02T11:00:26+12:00 Would be nice to have automatic cross coll searching over all collections in a group, that appeared on the group home page. perhaps switched on/off in group.cfg. kjdon Greenstone 3 wishlist 626 Integrate GS3 interface into GTI system Greenstone3 Interface minor low enhancement anna assigned 2009-09-07T15:34:24+12:00 2009-09-07T15:36:13+12:00 When start a new language interface in Greenstone 3, first check if there exist reusable translations in Greenstone 2 and reuse them as much as possible. anna Greenstone 2 wishlist 627 Translation consistency check Documentation minor moderate enhancement anna assigned 2009-09-07T15:41:15+12:00 2009-09-07T15:41:31+12:00 Sometimes HTML tags and macros are missed out in translations. Manual checking of each translation is tedious and error-prone. Need to add an additional consistency checking function for these two types of variables when translations is submitted. anna Greenstone 2 wishlist 635 mg query weirdness Indexers minor moderate defect nobody new 2009-09-28T11:19:20+13:00 2010-04-14T11:53:45+12:00 "John McP reported this in 2001 on sourceforge. I have tried it again and still the same result. In greenstone documentation collection on nzdl.org, search for: search for: mg++ results: Word count: mg: 763 41 documents matched the query. Search for: ""mg++"" (ie has quotes around it) results: Word count: mg: 763 post-processed to find ""mg++"" More than 50 documents matched the query. Great! a more specific search gets more documents! Is this a mg problem, or how we deal with mg results? Actually, if we do an ""all"" search (above were done as ""some"" search) for mg++ (no quotes) we get the same result as the quoted search: Word count: mg: 763 More than 50 documents matched the query. is this just to do with ranked search being more efficient and not searching all docs? " kjdon Greenstone 2 wishlist 636 legacy options in URLs Greenstone2 Runtime major low defect nobody new 2009-09-28T11:25:50+13:00 2009-09-28T11:25:50+13:00 "Reported by stuart on sourceforge in 2001. I guess this will still be a problem... Citeseer (http://citeseer.nj.nec.com/) links to the CSTR collection on www.nzdl.org, but all the links are broken. they are broken because of what appear to be legacy options encoded into the URL. Can we not do a better job of checking these?" kjdon Possible 2.88 Release 637 mgpp search results order Greenstone2 Runtime major moderate enhancement kjdon new 2009-09-29T14:33:25+13:00 2011-06-07T16:40:31+12:00 "Currently, results order is tied in with some/all - some is ranked, all is build order. This was to keep it the same as mg. But some people want the results always to be date order for example. We need to separate the some/all and ranked/natural order options so you can do natural some and ranked all searches. And as this will involve modifying the query interfaces, maybe we can make an easy way to remove some options from query form." kjdon Collection building wishlist 642 new options for List classifier Collection Building: Classifiers enhancement moderate feature nobody new 2009-10-12T10:47:28+13:00 2011-08-02T14:49:14+12:00 "1. AZCompactList had -firstvalueonly and -allvalues options. So, from a list of metadata names, eg dc.Subject,ex.Subject you could choose to use * just the first available value * all values of the first metadata element that had values * all values of all metadata For List, we don't have these options, and it always uses all values. Add these (or alternative) options to expand the functionality?? 2. Add an option ""never_bookshelf_last_level"" to prevent bookshelfing items with the same metadata value, so that it behaves more like the AZList. 3. Check whether List handles section metadata well - can it use multiple metadata for one field " kjdon Collection building wishlist 644 synonyms for searching Collection Building enhancement low enhancement nobody new 2009-10-14T15:38:54+13:00 2009-10-14T15:42:01+13:00 "Is there any easy way to use a synonym list to aid searching? Would you use it at build time, creating a ""stem"" index, but which is actually a synonym index? Or at runtime? 2 pass search? search an index for the synonyms, then search the documents for all the words? Or could you do it with javascript, and modify the query before passing to greenstone?" kjdon Greenstone 2 wishlist 645 restricting access to pdf documents Greenstone2 Runtime enhancement moderate enhancement nobody new 2009-10-17T21:30:19+13:00 2009-10-17T21:30:19+13:00 "Currently, can specify which documents need password to access them. But this won't work for srclinks, as these are direct links. Any way to restrict these too?" kjdon DEC Release 2.86 646 stray files in beatles collection Sample Collections minor moderate task nobody new 2009-10-19T11:18:49+13:00 2011-01-11T15:20:38+13:00 "Ian said: sample_files/beatles/advbeat_small/discography... seems to contain some stray images: title.gif and some images in a folder called imagesBIG. I think these have been there for years, but they shouldn't be. Please remove them. " kjdon Collection building wishlist 649 new version of pdftohtml Collection Building major moderate defect nobody new 2009-11-04T15:38:16+13:00 2020-02-18T21:37:24+13:00 There is an experimental 0.40 version - try this, or 0.39 in greenstone and upgrade if better. Does it support pdf 1.6? kjdon Collection building wishlist 656 animated gifs Collection Building major moderate defect nobody new 2009-12-18T11:44:14+13:00 2009-12-18T11:44:14+13:00 "When image magick creates thumbnails (or just when converting to jpg?) for animated gifs, you get lots of images. named xxx-0, xxx-1 etc. Greenstone is only looking for the xxx.jpg filename so doesn't find them. Should at least choose the xxx-0 one to add to the document." kjdon Collection building wishlist 658 weird datelist Collection Building: Classifiers minor moderate defect nobody new 2010-01-27T11:21:23+13:00 2010-01-27T11:21:23+13:00 "Julian has reported in the past about datelist disappearing. it seems that its fine, then he adds some more documents, and it disappears. If he takes out those docs, then its fine. But dates appear to be fine for those docs." kjdon Possible 2.88 Release 661 extraneous files in client gli download? GLI major moderate task ak19 new 2010-02-03T14:43:21+13:00 2011-06-07T16:40:23+12:00 "The client gli download has gli.sh/bat in it. But user should be using client-gli.sh/bat. Are gli.sh/bat useful? Or should they be removed from the download? Any other unnecessary files?" kjdon Possible 2.88 Release 664 Apache Tika for document Conversion Collection Building: Plugins major moderate feature nobody new 2010-02-15T12:06:59+13:00 2011-06-07T16:40:17+12:00 "We have talked about using Open Office to do document conversion. Doug Carter suggested Apache Tika as an alternative: *** I had a lot of trouble messing with the Open Office for conversion, but found a better solution using Apache Tika: http://lucene.apache.org/tika/ I put a shell wrapper around a java command line and created a ooxmltohtml.pl script that handles nearly all of the Office 2007 document formats. I hacked gsConvert.pl to include the new doc types, and a single new plugin OOXMLPlug to handle the importing. It works great, but performance is bit of an issue. If the performance problem was addressed, you could probably dump nearly all of the proprietary document converters. *** see also #426" kjdon Greenstone 2 wishlist 668 customising document _pagetitle_ Greenstone2 Runtime major moderate enhancement nobody new 2010-02-17T14:56:42+13:00 2010-02-17T14:56:42+13:00 "_pagetitle_ macro is defined in source code (in documentaction.cpp). It uses Title metadata. It would be nice to enable the user to specify a different metadata field to get the title from, eg dc.Title. You'd probably want a list, eg dc.Title,Title in case there was no dc.Title. How to implement this? format statement type thing? would it be collection specific? in that case there could be a gs.Metadata element that stored the list of metadata to look up." kjdon 3.11 Release 672 disable OAI for gs3 in GLI GLI enhancement moderate feature nobody new 2010-03-15T12:59:37+13:00 2021-03-22T09:46:26+13:00 OAI serviceRack automatically added into collectionCOnfig.xml. To disable OAI serving for a collection, currently have to comment out the service Rack. Would be good if GLI offered a way to dis/enable it, and even to configure it. kjdon Greenstone 2 wishlist 679 li tags in DocumentVList Greenstone2 Runtime enhancement moderate enhancement nobody new 2010-04-19T15:55:54+12:00 2010-04-19T15:55:54+12:00 "It would be nice to be able to use li tags instead of td tags in the DocumentVList, and then greenstone needs to add ul tags instead of tr tags. Need to modify output_section_group in vlist_browser_class" kjdon Collection building wishlist 681 mgpp word separator Collection Building major moderate defect nobody new 2010-05-05T14:13:53+12:00 2010-05-05T14:13:53+12:00 "As reported in mailing list, 12-4-2010 Where can I define my own ""word separator character"" or bypass some characters in word separator functions in the Greenstone? It seems like, my collection in the Greenstone is considering some Unicode special control characters as a space. For example, according to the Unicode standard, Mongolian text contents have four special control characters to change shapes (glyphs). Those are 1.Free Variation Selector One(FSV1) (U+180B), 2. Free Variation Selector Two (FSV2)(U+180C), 3. Free Variation Selector Three (FSV3)(U+180D) and 4. Mongolian vowel separator (MSV)(U+180E). Those control characters must be considered as a part of the word whether are in the middle, beginning and end of the word. For example, abc'MSV'defg is the single word, not two words 'abc' and 'defg'. I`ve failed to retrieve such words in the Greenstone. The Greenstone retrieves Mongolian words with control characters as two or more separate words (several control characters could used in a single word). " kjdon Possible 2.88 Release 687 conflicting shortnames for index and subcollection Collection Building major moderate defect kjdon new 2010-06-14T14:51:12+12:00 2011-06-07T16:40:07+12:00 "It is possible to get the same shortname in the indexmap and in the subcollectionmap. Shortnames are used like _TI_ as macros. When generating shortnames, need to check both lists for clashes." kjdon Possible 2.88 Release 691 imagemagick on mac 64bit Installation major moderate defect nobody new 2010-06-15T14:18:54+12:00 2011-06-07T16:38:29+12:00 "Our mac is 64bit, and the image magick we release will not work on 32bit macs. apparently. What to do about this???" kjdon Greenstone 2 wishlist 694 browsing through a classifier Greenstone2 Runtime major moderate feature nobody new 2010-06-29T10:59:47+12:00 2010-06-29T10:59:47+12:00 "A request on the mailing list asked for macros to give next and prev documents of a classifier, like next and prev search result buttons. Would need to be implemented in source code, perhaps just setting next and prev doc id, then the link can be implemented in macros." kjdon Possible 2.88 Release 699 handling sections in PDF Collection Building: Plugins major moderate enhancement sjm84 new 2010-06-29T15:38:26+12:00 2011-06-14T21:34:56+12:00 "Users want the ability to extract section info from PDF like we can from HTML or Word. Does -complex work for this? Will new converters handle this better?" kjdon 3.11 Release 700 gs3 config file parsing in GLI GLI major moderate defect nobody new 2010-07-13T12:02:43+12:00 2021-03-22T09:46:26+13:00 "Currently all the elements we look for are hardcoded. util/collectionConfigXMLReadWrite. We need to do a parse through the file and process elements we know about, and just remember those we don't so they can be written out again. Otherwise any new elements added by hand will be deleted." kjdon Collection building wishlist 701 language specific classifiers Collection Building: Classifiers major moderate feature nobody new 2010-07-26T10:25:41+12:00 2010-07-26T10:25:41+12:00 "User request. John Rose and Zineb Naji Documents have metadata assigned in different languages. eg an English Title and a French Title. Want to have a title list that just has English titles in it, and one that just has french titles in it. Two ways to achieve this metadata: 1. separate metadata elements for each language, eg dc.Title_en, dc.Title_fr. Can be used right now by modifying a metadata set. 2. GLI needs modifying to have a language option for each metadata element. So can add an english and a french version of dc.Title. metadata.xml file would need a language attribute for each piece of metadata. Would be fairly easy to modify the metadata set dtd, but reasonably complicated to modify GLI. To get the classifiers, in case 1, just build separate classifiers on each metadata element. In case 2, would need to modify the classifiers (probably just List as if you are working in different languages then List is the best) to have another language argument, and only use metadata in that language. The test would need to be on the language of the metadata, rather than the language of the document? Or maybe this would be another option? To have language attributes for metadata would involve quite a lot of changes to perl code and runtime code. Then, once there are separate classifiers for the languages, then in the navigation bar, we want to display the appropriate one for whatever language the user is in. ie in English, the titles list will be the english titles list. in French, the titles list will be the french list. Can this be done semi dynamically, without the user having to write the navigation bar macro? They would need to indicate which classifiers belong together, which belongs to which language and which is the default. or something like this." kjdon Possible 2.88 Release 702 script output piped to less/more Greenstone2 Build Code major low defect nobody new 2010-08-02T14:23:43+12:00 2011-06-07T16:40:57+12:00 "import.pl, pluginfo.pl etc pipe their output to less or more on linux. (for text output not for xml output) This is commented out on windows. While activestate perl claims to handle open(STDIN, ""-|""), it doesn't seem to work. can we do it a different way for windows? IPC::cmd??" kjdon Possible 2.88 Release 703 e arg in cookie Greenstone2 Runtime major moderate enhancement nobody new 2010-08-02T14:27:30+12:00 2011-06-07T16:39:55+12:00 "Greenstone URL's are hideous, mainly due to e arg. If we store this in a cookie, then url's will be much nicer. current URL's will no longer be transferable, as lots of state is stored in e, incl. collection name. Would need to make the explicit args contain all essential args, a, c, d etc. So when you copy a url and give to someone it still works. but with some preferences lost." kjdon Possible 2.88 Release 704 perl testing for programs Collection Building major moderate enhancement nobody new 2010-08-02T14:29:26+12:00 2011-06-07T16:39:49+12:00 "can we use IPC::cmd to test for imagemagick, wget, openoffice etc? Will that give us nicer code when trying to work out whether something is installed or not?" kjdon Possible 2.88 Release 705 WordPlugin to extract number of pages? Collection Building: Plugins enhancement moderate enhancement nobody new 2010-08-09T14:04:17+12:00 2011-06-07T16:39:04+12:00 "Can we make word plugin extract NumPages metadata? I guess if it's converted to HTML using wv then it might not know. But what about using open office or windows scripting? Is this metadata available? " kjdon Possible 2.88 Release 706 plugin/classifier arg deprecation Collection Building major moderate enhancement nobody new 2010-08-10T11:39:49+12:00 2011-06-07T16:39:44+12:00 "Invent a nice way to handle deprecation - deprecation flag, auto output a message? GLI handle it properly? In some cases, it will be a rename, in other cases more involved." kjdon Possible 2.88 Release 707 plugin/classifier arg dependency Collection Building major moderate enhancement nobody new 2010-08-10T11:43:04+12:00 2011-06-07T16:39:39+12:00 "Invent a nice way to handle dependencies between args. eg thumbheight isn't needed unless create_thumbnail is set. show grayed out? indented? in pluginfo on screen, and also in GLI. GLI deactivate if dependency not active. tooltip needs to display what it depends on. " kjdon Possible 2.88 Release 708 convert_to arg paged image option Collection Building major moderate enhancement nobody new 2010-08-10T11:44:26+12:00 2011-06-07T16:39:33+12:00 "Currently pagedimg_*, want to rename to pagedimage_*. Do this once deprecation has been handled properly (#706) This is in several plugins, incl PDFPlugin." kjdon Possible 2.88 Release 711 fix gc=2 badness Greenstone2 Runtime enhancement moderate defect kjdon new 2010-08-30T14:41:47+12:00 2011-06-07T16:38:58+12:00 "If you set gc=2 (in main.cfg), then when browsing classifiers, when you open a folder, the parent folder stays open. This is very useful. However, it goes bad when there is a horizontal AZ list - get the a-z bits repeated a large number of times. FIX! Maybe make the a-z bit ignore gs=2?? Also, while we are there, can we make a separate arg for classifiers vs documents? you may want classifiers to be expanded but not document table of contents." kjdon DEC Release 2.86 716 finish manifest collection Sample Collections major moderate documentation kjdon new 2010-10-18T14:34:02+13:00 2010-10-18T14:34:02+13:00 I am working on this, need to finish it off and tidy up the text kjdon DEC Release 2.86 717 finish lom demo Sample Collections major moderate documentation nobody new 2010-10-18T14:51:31+13:00 2010-10-18T14:51:31+13:00 Kath is working on this. is it committed?? kjdon Greenstone 2 wishlist 718 improved goto form Greenstone2 Runtime enhancement moderate enhancement nobody new 2010-10-26T11:42:08+13:00 2010-10-26T11:42:08+13:00 "a document table of contents has the open book icon at the top, and clicking on it closes the book and takes you back to the search page or classifier page from where the book was accessed. The goto form doesn't have this link. The only way to get back there is to use browser's back button or go back to the top level classifier page or search page. Can we modify the form to include the book icon? It could on the left of the 'page X' bit." kjdon Collection building wishlist 722 PagedImagePlugin: multiple texts per page Collection Building: Plugins enhancement moderate enhancement nobody new 2010-12-06T11:09:42+13:00 2010-12-06T11:09:42+13:00 "A request from the user list: Have a series of page images, then two text files per image - different translations. Want to link both text files to the one page. " kjdon Possible 2.88 Release 723 search term highlighting Greenstone2 Runtime enhancement moderate enhancement nobody new 2010-12-07T12:08:14+13:00 2011-06-07T16:38:52+12:00 "In some cases, people want to create links in their document text pages. Eg display all the subjects, each one linking to a search on that subject. If the user has reached the document through a search on subjects, and search term highlighting is on, then the subject ends up getting span tags around it, which screws up the links. A suggestion was to not do highlighting on cgisafe metadata. The current code expands the format statement out to text, then does highlighting on it. No way of knowing which bits were cgisafed. Can we make this better? Maybe highlighting could be done when each individual element is retrieved, eg when get [Text] or [dc.Subject], then can not do it for cgisafe ones. " kjdon 3.11 Release 724 Optimizations for gs_ajax_utils.js Greenstone3 Runtime enhancement moderate enhancement zcarter new 2010-12-08T14:18:11+13:00 2021-03-22T09:46:26+13:00 "I am unable to email these changes as I am at school, and I don't have Subversion write access (and the school computers don't have SVN anyway). The changes are: * Changed multiple concatenations into creation of lists (they are joined when returned). This is faster as in Javascript, strings are immutable. * Changed postfix increments to prefix increments where possible; slight performance boost. * Changed multi-line functions into one-liners where possible (avoids variable allocation, etc.) * Fixed inconsistencies in coding style (all four-spaces instead of mixed, whitespace added, etc.) * Removed unneeded whitespace. Included is a diff file, and the actual file itself." zcarter Possible 2.88 Release 730 PDFPlugin: use_sections option turned on generates a Page 0 Collection Building major moderate defect nobody new 2010-12-16T19:51:19+13:00 2011-06-07T16:37:50+12:00 "When the use_sections is turned on in the PDFPlugin, for PDFs containing extractable text, it generates one extra page that contains nothing: a ""page 0"" that precedes all the others. Not only is it not a necessary (but rather confusing) addition, but the Go To field/button on ""page 0"" doesn't work. In contrast, when working with image-only PDFs (ones with no extractable text), and the pagedimg_jpg option is turned on in the PDFPlugin, there is no empty preliminary page generated when viewing such a document in Greenstone. It starts at page 1." ak19 Possible 2.88 Release 732 multiple metadata fields for depositor Depositor enhancement moderate enhancement nobody new 2011-01-06T14:38:25+13:00 2011-06-07T16:38:46+12:00 "Is there a nice javascript or other way to get more of a field in the depositor metadata page? ie have one field for subject, with a ""add more subjects"" button, so that a document can have two or more subjects. nicer than always displaying two subject fields. Requested by SEAFDEC/AQD Library, 22/2/2010" kjdon Collection building wishlist 735 pagedimageplugin modification Collection Building: Plugins enhancement moderate enhancement nobody new 2011-01-10T14:50:19+13:00 2011-01-10T14:50:19+13:00 "Patch sent to mailing list by Yitzchak Schaffer. A modification to PagedImagePlugin to allow pdfs to be specified instead of image files. Is a bit specific to his collection and relies on pdftotext to be installed. Can we make it more generic and commit it? I've always thought that PagedImagePLugin could pass files to the plugin pipeline, and then add them in to the document. maybe that would be an alternate way to go? " kjdon Collection building wishlist 736 blocking problem Collection Building: Plugins minor moderate annoyingness nobody new 2011-01-13T14:01:15+13:00 2011-01-13T14:01:15+13:00 "Problem with current block_exp vs smart block mechanism. For ISIS files, I would like to process the mst file, then block the corresponding fdt and xrf files. I don't want to just use block_exp for fdt and xrf, as if these don't have a corresponding mst then it would be nice to warn the user. Using the store_block_files routine, and empty block_exp works fine on command line. However, not for GLI. - it complains when you drag in fdt and xrf that no plugin can process the files. So, add these two to the process exp instead. Then all three get highlighted as explodable. and the plugin needs to be fixed up so it doesn't try to process the fdt and xrf. can we do anything better? I have left block_exp in for now." kjdon DEC Release 2.86 739 oai-e collection update Sample Collections minor low task nobody new 2011-02-08T14:14:09+13:00 2011-02-08T14:14:09+13:00 "The oai files were downloaded a while ago, and use old dc.Identifier for the source doc path. Nowadays we set gi.Sourcedoc with the path and this is what is used by default by the OAIPlugin. This collection needs to set -document_field dc.Identifier option to plugin. Would be nice to have the files redownloaded so that they use gi.Sourcedoc instead. However, the jcdlpics oai server is no longer working. If we ever get around to putting up a new repository for the oai tutorial, then maybe we could use that instead? Of can we edit by hand the oai files?" kjdon Possible 2.88 Release 740 Linux/Mac allow spaces in paths in .sh files Greenstone2 Runtime major moderate defect nobody new 2011-03-30T15:56:25+13:00 2011-06-07T16:39:28+12:00 In the same way we've fixed up all the places in the .bat files where having spaces in the path is a problem, we should probably do the same for the .sh/.bash files. sjm84 3.11 Release 742 The -d32/-d64 flag for Java on Mac Collection Building major moderate defect nobody new 2011-04-06T17:44:37+12:00 2021-03-22T09:46:26+13:00 Need to look into this. Remember he problem on Dr John Brine's machine where we had to tell it which Java (32 or 64 bit compatible version) to use. ak19 3.11 Release 743 Client-side XSLT Greenstone3 Runtime major moderate task sjm84 new 2011-04-07T11:36:22+12:00 2021-03-22T09:46:26+13:00 Need to remove hard-wired interface names in TransformingReceptionist.java sjm84 Greenstone 3 wishlist 744 JTidy for Realistic Books at runtime Greenstone3 Runtime enhancement moderate enhancement sjm84 new 2011-04-11T10:08:57+12:00 2011-04-11T10:08:57+12:00 Look into using JTidy or similar to serve up XML-compliant internal Greenstone documents in the form that Realistic Books expects. sjm84 Possible 2.88 Release 746 Merging llssite glisite. Reorganising toplevel scripts and references to them Greenstone2&3 minor moderate defect nobody new 2011-04-17T18:46:27+12:00 2011-12-15T22:11:49+13:00 "(1) After a GS2's installation location is moved manually by a user, a few things need to be done to make sure Greenstone can run again thereafter: In gsi-control.sh and gsi-control.bat, need a new, separate target called something like ""reset-gsdlhome"". This target first *forces* configure-cgi (so that gsdlsite.cfg is indeed deleted and replaced with gsdlsite.cfg.in). This part of the target needs to warn the user that ""local changes made to gsdlsite.cfg will be lost"". Second, the ""reset-gsdlhome"" should call configure-apache to make sure apache is updated with the new GSDLHOME. Finally, need *.app.in template files for Mac and need to add code into configure-apache or somewhere to change all the *.app.in references to **GSDLHOME** into the correct GSDLHOME when generating *.app from that template file. (2) GSI-Makefile is not used, it just makes calls to gsi-control.bat. It can simply go. Maybe leave a note in the release_notes that it has been deprecated, telling people to use gsi-control instead. (3) Too many scripts at top level (.bat, .sh, .app). They're to be put into a folder called Applications (or Launch), and references to them need to be updated. E.g. server.jar refers to gs2-server and gs3-server. " ak19 Possible 2.88 Release 747 Update translation Collection Building enhancement moderate enhancement nobody new 2011-04-29T12:02:18+12:00 2011-06-07T16:38:40+12:00 Send a request to translators to update their translations if they want for 2.85. sjm84 Possible 2.88 Release 752 Sorting issue when a collection of emails is processed Collection Building major moderate defect nobody new 2011-05-20T16:14:53+12:00 2011-06-07T16:39:23+12:00 "Renate Morgenstern reported: I have a collection of emails in an archived mail box. Files are processed, but the sorting is not working properly. When the mail boxes are processed, there is an error message - date out of range'. See extract from log file. Is there soemthing I can do about it?" ak19 3.11 Release 756 Unify client side XSLT work with main oran skin Greenstone3 Interface major moderate defect nobody new 2011-06-09T10:32:46+12:00 2021-03-22T09:46:26+13:00 "Steven McTainsh developed a method for pushing XSL transformations to the client's browser if possible. Current issues: - This is currently only for the default skin. We want it in the oran skin. - It is implemented with two different skins (a client version ""default-client-xslt"" and a server version ""default""). We want it to be a single skin. - To enable/disable it requires a flag in web/WEB-INF/web.xml, need a better switch." sjm84 Possible 2.88 Release 757 More GS2 OAI testing stuff Greenstone2 Runtime minor moderate task nobody new 2011-06-09T15:35:30+12:00 2011-06-09T15:35:30+12:00 "http://www.openarchives.org/Register/ValidateSite is the usual (official) OAI validation site that we use. 1. Mariana Pichinini found another useful URL to validate against: http://oval.base-search.net/oval/validate (or http://oval.base-search.net/oval/oval) After setting up GS2 and collections for OAI, plug in the URL: http://HOST.RESEARCH.scms.waikato.ac.nz:PORT/greenstone/cgi-bin/oaiserver.cgi 2. While there are no errors in validating 2.84.5, there is future work in this area, to make our OAI server perfect. Oval defines WARNINGS and RECOMMENDATIONS as follows - WARNING: Protocol violation or malformed content data - RECOMMENDATION: Deprecated protocol feature or content-related issue Of the warnings and recommendations resulting for a test of 2.84.5, the following are pertinent as they can't be fixed by merely tweaking a collection: - RECOMMENDATION: Deleting strategy is ""no"" – recommended is persistent or transient. - RECOMMENDATION: resumptionToken should contain expirationDate information. - RECOMMENDATION: resumptionToken should contain completeListSize information. - WARNING: Requests seem to be redirected to: "" http://akeake.resnet.scms.waikato.ac.nz:8283/greenstone/cgi-bin/oaiserver.cgi"" Not sure what we can do about the following warning, since it's just the way GS3 works: WARNING: Requests seem to be redirected to: ""http://HOST.RESEARCH.scms.waikato.ac.nz:PORT/greenstone/cgi-bin/oaiserver.cgi"" Recommendations and warnings that merely require tweaking GS2 config settings or those of a collection, or just require correct metadata to be present in the OAI-enabled collections (and which are therefore things that GS2 users can control and configure): - RECOMMENDATION: ListRecords batch size too small (5), should be at least 100. - RECOMMENDATION: dc:language should conform to ISO 639, found ""English"" - WARNING: Records should at least contain the DC elements: date, creator, identifier, type, title. Found a record (oai:greenstone:demo:HASH9717c1cfafd71954cc6e79) missing the following DC element(s): date, type, creator." ak19 Possible 2.88 Release 759 PDFBox and GS2 server vs local GS2 server Collection Building major moderate defect nobody new 2011-06-14T21:31:36+12:00 2011-06-14T21:31:44+12:00 "Robert Ntalaka reported (""Re: [greenstone-devel] GS Admin Password"") on the mailing list on 14/05/11 21:00: ""under GS2 server some of the pdf documents are not processed. When you install the extension for pdfs and you use the local server, all pdfs are processed. When you switch to the GS2 server and build the collections, new pdfs are rejected until you configure it to convert to pageimg."" " ak19 Possible 2.88 Release 764 Perl CGI programs Greenstone2 Runtime major moderate defect nobody new 2011-06-22T11:43:42+12:00 2011-06-22T11:43:42+12:00 Perl CGI programs such as metadata-server.pl should live in the runtime-src folder and be copied to cgi-bin/[os] when installed. This means the initial line can be altered to match correctly where the version of Perl is on the system. sjm84 Possible 2.88 Release 767 AbiWord and perl libraries for converting doc(x) to html Collection Building enhancement moderate enhancement nobody new 2011-06-23T20:34:10+12:00 2015-10-08T15:51:09+13:00 "Max suggested looking over CPAN for docx conversion. We then found the following. http://search.cpan.org/~amiri/MSWord-ToHTML-0.003/lib/MSWord/ToHTML.pm (Uses AbiWord.) http://search.cpan.org/search?query=msword+html&mode=all http://search.cpan.org/~amiri/MSWord-ToHTML-0.003/lib/MSWord/ToHTML.pm http://www.abisource.com/wiki/PluginMatrix http://www.abisource.com/release-notes/2.8.0.phtml Open-Office is a large download. AbiWord, which now handles docx (converts complex docx to html really well, tried it just now), is only 8MB. Maybe if the user doesn't have OO and Office/Word 2007+ already installed we can resort to using Perl code to look for Abiword to do the conversion (if the user has that installed). This will also work on Mac and Linux, since AbiWord is available for those platforms." ak19 Possible 2.88 Release 768 GLI's build output says PDFs and RTFs proccessed by HTML- and TextPlugings Collection Building major moderate defect nobody new 2011-06-24T14:33:32+12:00 2011-06-24T14:33:32+12:00 "Professor Witten noticed in 2.84 version 2 that when GLI is set to Librarian mode, the build-output says that input PDFs and RTFs are proccessed by HTMLPlugin and TextPluging respectively, instead of PDFPlugin and RTFPlugin. The correct plugin names should be shown." ak19 Greenstone 3 wishlist 769 xpath 2.0 and xslt 2.0 Greenstone3 Interface enhancement low enhancement nobody new 2011-06-29T14:59:55+12:00 2015-04-20T13:27:04+12:00 "Need a java library to deal with xpath 2.0 and xslt 2.0. Xalan for transformations only supports version 1.0 of XSLT. And so we can't use functions like ""endswith"" yet." ak19 3.11 Release 771 Admin Tool Greenstone3 Runtime major moderate task nobody new 2011-07-12T13:14:22+12:00 2021-03-22T09:46:26+13:00 Make sure it is ready for 3.05 sjm84 3.11 Release 772 Make .project and .classpath files for Greenstone3 and GLI OtherProjects major moderate task sjm84 new 2011-07-20T14:33:50+12:00 2021-03-22T09:46:26+13:00 We would like to have these files in SVN so that it is easy to get the Greenstone3 and GLI Java code into Eclipse sjm84 Possible 2.88 Release 773 Depositor Improvement Depositor major moderate task nobody new 2011-07-20T14:35:32+12:00 2011-07-20T14:35:32+12:00 Allow the Depositor to build correctly the very first time, even if the collection has never been fully built using GLI or import.pl externally before. sjm84 Possible 2.88 Release 774 Installation of CGI scripts Installation major moderate defect nobody new 2011-07-20T14:38:04+12:00 2011-07-20T14:38:04+12:00 Our Perl CGI scripts should really start within a 'src' area and as part of installation move to the 'cgi-bin' area. Setting things up this way will allow for the correct Perl path to be added as the initial line. It will also make it easier to support multiple operating systems at the same time on a single file-system. sjm84 Possible 2.88 Release 776 Use of proprietary APIs in GLI code GLI major moderate defect sjm84 new 2011-07-20T15:47:01+12:00 2011-07-20T15:47:01+12:00 Look at the proprietary API warnings around JPEGs in GLI code and see if it can be updated to avoid these calls. sjm84 Possible 2.88 Release 777 Google Quick View for PDF and other document formats Greenstone2 Runtime major moderate task nobody new 2011-07-28T11:25:12+12:00 2011-07-28T11:25:12+12:00 "Look at http://docs.google.com/viewer and see if we can make a macro that can allow us to ""Quick View"" PDFs and other formats in our collections." sjm84 Possible 2.88 Release 778 Connecting DC metadata element names with labels between GLI and GEMS GEMS minor moderate defect nobody new 2011-07-28T18:10:20+12:00 2011-08-01T16:38:45+12:00 "> when you write with a prefix like dc.Identifier you expect that > the metadata element is that, but then when you go to GLI and look at > the data elements you see dc.Resource Identifier. For consistency I > think you should put either just ""full name"" [without prefix] or > ""prefix.label"" in the left hand pane of GEMS for all of the metadata > elements of a set, no? > 1. The reason that GEMS lists ""dc.Subject"" and ""dc.Identifier"" to the left is because this is exactly the name of the metadata that you need to type into a format statement for Greenstone to display it. So you would write a format statement like the following in GLI: [dc.Subject] (and not [dc.Subject and Keywords] or [Subject]). When you click on a dc metadata name to the left in GEMS, you will see its official DC element name (e.g. ""Subject"" or ""Identifier"") listed at the top of the upper pane to the right and its label (e.g. ""Resource Identifier"") listed in various languages for which this is defined in the lower pane. 2. Since you're concerned that users may not make the connection between dc.Resource Identifier (and its variants in different languages) in GLI and the matching dc.Identifier in GEMS, Dr Bainbridge came up with the idea of a tooltip when clicking on an element to the left of GEMS, which will also be incorporated into the existing tooltip you see when you hover over a dc metadata label in GLI. (In GLI, the message will be: ""Dublin core element name . To use this in a format statement, you'd write [dc.Subject]."" Followed by the existing tooltip message.) There is a drawback to this though: except for predefined metadata sets (the ones you *load* into GEMS), other metadata sets are created in GEMS, which is another purpose GEMS is used for. In the latter usage, there will not yet be any metadata element name and label, as the user has yet to design the metadata set and needs to still type it into the fields. So undefined element names and labels cannot be loaded into tooltips for metadata items on the left side of GEMS. >>> I can indeed now see that the ""label"" for some Dublin Core fields on >>> the >>> official site is different from and more complex than the ""full name"". >>> This is really confusing, but I guess it means that it would be better >>> to call the field dc.Resource Identifier in GEMS (or simply Identifier, >>> the ""full name"") rather than dc.Identifier. >>> >> >> I just had a look at GEMS. It says dc.Identifier (which is the >> programmatic name) and if you then click on it, its label field says >> ""Resource Identifier"". This is no different from how dc.Subject has a >> label field called ""Subject and Keywords"". >> >> See also >> http://dublincore.org/documents/usageguide/elements.shtml#identifier >> again, which defines the Dublin Core metadata element as ""Subject"" but >> its >> label as ""Subject and Keywords"", while the label for element >> ""Identifier"" >> is ""Resource Identifier"". >> >> Since GEMS consistently uses the metadata set name as prefix for all >> other >> elements, including for Subject, it behaves similarly for Identifier. >> Thus we have dc.Subject and dc.Identifier. I think it will be confusing >> to >> change dc.Identifier to plain ""Identifier"" while all the others remain >> prefixed with dc.*. Plain ""Identifier"" is moreover not the element's >> qualified name (qualified by metadata-set name) within Greenstone. >> " ak19 Possible 2.88 Release 781 Section extraction for PDF using Word Collection Building major moderate defect nobody new 2011-08-02T14:07:36+12:00 2011-08-02T14:07:36+12:00 "John Rose wanted to know two things: a) on whether extracting metadata (including section data) from Word documents using a microsoft utility is possible for Macs as well. b) could the facility of automatic section information extraction which exists only for Word be used for pdf files as follows: 1. Generate the document in Word format (in Word or Open Office). 2. Import it into Greenstone Windows version and generate an html file with the section information incorporated but hidden by right clicking on the file in the Gather view. 3. Generate a pdf file from the Word file (for example in Open Office) and put it in the collection. 4. Find a way to tell Greenstone that the pdf document is the scrlink for the html document (initially by adding a link manually in the archives file, but later perhaps by finding a way to set this by a parameter in HTMLPlugin, something like an associated file). One would also have to make sure that the associated pdf file is not treated a second time as the primary file. In this way one could do section searching on the html file and display in the pdf file. The problem is step 4, could you advise?" ak19 Possible 2.88 Release 783 Order dc.Identifier that's shown by OAI server by user preference Greenstone2&3 major moderate enhancement nobody new 2011-08-15T18:22:38+12:00 2011-08-15T18:22:45+12:00 "Mariana Pichinini wanted more control over the dc.Resource Identifier displayed by the OAI server. The idea is now to have two more properties in oai.cfg: - identifier cutoff (threshhold): how many dc.Identifiers get displayed - identifier order (by collection?): user specified order. E.g. dc.Resource Identifier, gs.OAI Resource URL, autogenerated GS URL to source doc. - identifier order default: the default ordering is given above. Or it can be overridden in this property, to specify the default ordering behaviour for all collections, with exceptions given above under identifier order by collection. The code needs to be changed in runtime-src/src/oaiservr/dublincore.cpp's output_custom_metadata() where it works out which URL to display for oai_dc.Identifier. It needs to create a HashMap of (label, value) pairs, where label is dc.Resource Identifier, gs.OAI Resource URL, greenstone autoIdentifier, and their associated (comma-separated) values. Then the code reads the user-defined ordering from oai.cfg and orders the keys of the map accordingly ''for the collection''. Next, the code outputs the values for the ordered elements in the mapping one after the other (stopping output when the cutoff/threshold is reached, which is another property defined in oai.cfg). Note that there may be multiple values per key in the map. For instance, there can be multiple dc.Resource Identifiers, there can be up to 2 GS autogenerated URLs. OAI itself does not seem to specify any ordering of dc.Resource Identifier or any primary preference for URL over ISBN number, for instance. This leaves it open for Greenstone to allow users to specify the ordering. Not just GS2 but also GS3 code needs to be modified to exhibit the same behaviour." ak19 Possible 2.88 Release 784 z3950 and Library of Congress output Collection Building major moderate defect nobody new 2011-08-15T18:45:54+12:00 2011-08-15T18:46:07+12:00 "Library of Congress' Z39.50 and SRU (SRW) server is returning SRU data as per Dr Bainbridge. The matter is complicated because Yaz, which is the tool that GS uses to download data over z39.50, works with both protocols in the same fashion. And we're not sure how to tell Yaz what specific protocol we want to use to connect. Both the old and new connection points at the LoC give the same XML result, we doesn't expect XML returned for Z39.50 requests (neither does the GS perl code expect it): OLD: Host: http://z3950.loc.gov Port: 7090 Database: voyager Find: computer Max Records: 1 NEW: Host: http://lx2.loc.gov Port: 210 Database: LCDB_MARC8 Find: computer Max Records: 1 Where ""Find"" takes the search term(s). Try using Yaz included with GS as follows to execute the same query from the command line instead of GLI's Z39.50 download pane: 1. > source setup.bash 2. > yaz-client 3. > open http://lx2.loc.gov:210 > base LCDB (Can combine as: > open http://lx2.loc.gov:210/LCDB) When it returns with the response ""Opening Connection...OK."" proceed with: 4. > find computer > show 1+1 (> quit) " ak19 Possible 2.88 Release 786 GetDoc over OAI download naming scheme Downloading minor moderate defect nobody new 2011-08-30T16:27:36+12:00 2011-08-30T16:27:36+12:00 " 1. we built a collection available over OAI, and in the Import Options, we set OIDtype to ""assigned"" and OIDmetadata to ""ex.File.FileName"" (although we think you could also try ex.Source, to avoid relying on embeddedmetadata). 2. Then at the end of etc/oai.cfg, the following mapping was inserted: > oaimapping oaipdf:ex.File.FileName oai_dc.identifier 3. Finally, we used GLI to connect to the OAI server to download the 2 documents in our small oai collection made available over the GS OAI server. These documents came out named as: pdf01pdf-1.pdf, pdf03pdf-1.pdf The suffixed ""-1"" numbering scheme is a count to ensure that files downloaded with the same name are stored under unique filename. Dr Bainbride and John Rose independently suggested that the first file of a name need not have -1 suffixed, and that subsequent files of the same name could use the numbering instead." ak19 Possible 2.88 Release 788 GLI: support copy of extracted metadata GLI major moderate defect nobody new 2011-08-30T17:25:02+12:00 2011-08-30T17:25:02+12:00 At present, only editable metadata in GLI's Enrich pane can be copied (cut and pasted), but not extracted metadata. Want it for ex meta also. ak19 Possible 2.88 Release 789 Installer: if set to other languages, GLI and Reader Interface should default to that language Installation major moderate defect nobody new 2011-08-30T17:37:21+12:00 2011-08-30T17:37:37+12:00 At present, even if we install in say Chinese, GLI (and possibly the Reader interface) starts up in English anyway. ak19 Collection building wishlist 791 Journals Collection Building major high documentation nobody new 2011-09-19T20:42:58+12:00 2011-09-19T20:42:58+12:00 Online Journals JCE Library Possible 2.88 Release 793 On GLI attempting to close: list any programs it opened by name GLI major moderate defect nobody new 2011-09-27T18:16:42+13:00 2011-09-27T18:16:42+13:00 "When GLI can't close due to other programs being open, let's say what those programs are!!" ak19 Possible 2.88 Release 794 Allow spaces in filepath on Linux Greenstone2&3 major moderate defect nobody new 2011-09-28T14:07:13+13:00 2011-09-28T14:07:13+13:00 "Allow spaces in filepath on Linux: - Should work in launch of apps like GLI, GEMS, GS2server etc - Build should still work. - Collecthome still recognised and passed around correctly - Remote GS server should still work (file transfer) - Tutorials " ak19 Possible 2.88 Release 795 That annoying Java exception GLI gives out GLI major moderate defect nobody new 2011-09-28T14:18:06+13:00 2011-09-28T14:18:06+13:00 "Dr Bainbridge has found a possible solution. He wrote: I was just pondering our: Exception in thread ""AWT-EventQueue-0"" java.lang.NullPointerException ... type error message. I'm wondering if we have the sort of issue that could be sorted out by using: SwingUtilities.invokeLater(...) I had in particular focused on the article: http://stackoverflow.com/questions/2811795/java-swing-gui-exception-exception-in-thread-awt-eventqueue-0-java-util-nosuc which has links back to the relevant Sun tutorials. Could you please read up on this, and see if we think it is the cause of our issue? An example of the full exception we get in GLI is included below. If you think it's a quick fix to try out, please give it a whirl, otherwise let's discuss this in the lab later this afternoon." ak19 Possible 2.88 Release 796 Standard way to install GS menu icon on Linux systems Installation major moderate defect nobody new 2011-10-03T16:55:31+13:00 2011-10-03T16:59:48+13:00 "xdg-utils appears to work on various window/desktop managers (gnome, kde, lxde?) of various linux distributions (ubuntu,centos,fedora?). xdg-utils includes: (1) xdg-open which will use the default app to launch a file of a particular file extension. This means it is useful as the default Linxu command for GLI's file assocations. Currently, the CentOS here doesn't recognise the xdg-open command, because it's not installed. (2) xdg-utils will allow Greenstone to install menu items/icons on desktops on various Linux desktop managers. http://portland.freedesktop.org/wiki/ Xdg-utils consists of the following installation related tools: *xdg-desktop-menu, install desktop menu items * xdg-desktop-icon, install icons to the desktop The idea is to include xdg-utils with Greenstone and run its operation to create menu/desktop icons as an ant target during installation of GS core. " ak19 Possible 2.88 Release 797 GS installer: counterintuitive behaviour of select dir button Installation major moderate defect nobody new 2011-10-03T16:59:03+13:00 2011-10-03T16:59:03+13:00 "GS installer comes with a browse button to help you choose a directory to install Greenstone into. This dialog behaves differently from expected: you need to select a folder icon, not double click on it (thereby going into the folder) in order for the selected folder to appear in the textfield wherein you would type the location to install GS into. Is there a way to make this behave more intuitively? Any changes would likely need to be made to the underlying installer code which is not written by Greenstone." ak19 Possible 2.88 Release 799 Boolean searching: AND NOT behaviour Greenstone2 Runtime major moderate defect nobody new 2011-11-01T17:36:54+13:00 2011-11-01T17:36:54+13:00 "When using either MG or MGPP (not Lucene, I think), and having advanced fielded searching on, AND NOT results behave weird. Use the TestDocs test collection from SVN. Build the collection and turn on advanced and fielded searching. Search for ""images"" in text and ""de"" in filenames. Then set the boolean operator to be AND NOT (so it is expected to exclude any files with the filename ""de"" among the results where the text contains the word ""images"".) Which of the indexers has this problem? Was AND NOT implemented for MG and MGPP? Is the submit button sending the correct request? May need to trace the request through to find out what is happening." ak19 Possible 2.88 Release 800 Apache compression feature to speed up Lucene retrieval of GS versions of some docs Greenstone2&3 major moderate defect nobody new 2011-11-02T19:44:42+13:00 2011-11-02T19:44:58+13:00 "Lucene retrieval of Greenstone version of document slow in TestDocs' 592Report.doc and one of the 2 xls files. But it retrieves the doc and xls versions fast, and deals fast with displaying the Greenstone versions of other doc and xls files in the collection. Dr Bainbridge thinks this is because Lucene retrieves data from the index in a different manner to MG(PP): that it has to decompress it first. His idea is to let Apache do the decompression, by switching on the Apache compression feature when the user is using Lucene (or allowing the user to turn this on somewhere and letting them know of it). Putting decompression of Lucene-retrieved docs at Apache level may make things faster." ak19 Possible 2.88 Release 801 gli config files should handle multiple OS GLI major moderate defect nobody new 2011-11-03T20:28:05+13:00 2011-11-03T20:28:05+13:00 "Inside the ./gli/config(Remote).xml, need an additional qualifier that sets the property only for the OS. That way the Mac and Linux using the same network here can work with different file associations (open %1 vs firefox %1) and different remote GS URLs etc. The config file would have something like .propname or /propname, whichever is more convenient." ak19 Possible 2.88 Release 803 Export To CD-ROM (setup.exe and other files) not working on 64 bit win 7 Collection Building major moderate defect nobody new 2011-11-25T15:13:15+13:00 2011-11-25T15:16:13+13:00 "2.85 has a serious bug that is preventing distribution of my collection. The OLD 2001 setup.exe in the windows folder will not work on a Windows 7 64 bit machine. When you use the export your collection to a DVD procedure it includes this old 2001 setup.exe file. This setup file simply will not work on modern Windows software. In fact the Windows folder contains a number of old programs that you should consider updating. I include a list below but for sure the 2001 setup.exe needs to be replaced before Greenstone will work on modern 64 bit software. I think it is dangerous for a developer to leave these old programs in their software if newer ones are available. No telling what evil lurks in the bowels of these OLD files as I found out with Windows 7 64 bit. setup.exe 4/19/2001 setpw.exe 5/17/2001 tar.exe 10/10/2001 unzip.exe 10/10/2001 bunzip2.exe 10/10/2001 bzip2.exe 10/10/2001 gunzip.exe 10/10/2001 gzip.exe 10/11/2001 pdftohtml.exe 7/2/2004 pptextract.exe 9/23/2005 pptextract.frm 9/23/2005 pptextract.vbp 9/23/2005 pptextract.vbw 9/23/2005" ak19 Possible 2.88 Release 807 extrameta data structures in its own pm file Collection Building major moderate defect nobody new 2012-01-16T18:27:23+13:00 2012-01-24T16:23:59+13:00 "1. BUGFIX: http://trac.greenstone.org/changeset/24932 Diego noticed how the metadata in a toplevel metadata.xml, which specifies metadata for files in import's subfolders, does not get attached to the files on Windows, while this works on Linux. It had to do with the difference between the file slashes used on the OS versus the URL-type fileslashes used in the metadata.xml Diego had constructed. This has now been fixed and Dr Bainbridge came up with a tidier solution of a new method in util.pm that would handle the details. perllib/plugins/DirectoryPlugin.pm trunk/greenstone2/perllib/util.pm 2. All perlcode that accesses extrametakeys, extrametadata, extrametafile data structures has been moved into a new perl module called extrametautil.pm. The next step will be to ensure that the file_regexes used to index into these data structures are consistent (using consistent slashes, like URL style slashes). http://trac.greenstone.org/changeset/24951 and http://trac.greenstone.org/changeset/24952 extrametautil.pm plugins/DSpacePlugin.pm (3 diffs) plugins/DirectoryPlugin.pm (5 diffs) plugins/EmbeddedMetadataPlugin.pm (3 diffs) plugins/LOMPlugin.pm (2 diffs) plugins/MetadataCSVPlugin.pm (2 diffs) plugins/MetadataXMLPlugin.pm (3 diffs) plugins/OAIMetadataXMLPlugin.pm (4 diffs) plugins/OAIPlugin.pm (2 diffs) and bin/script/explode_metadata_database.pl (2 diffs)" ak19 Possible 2.88 Release 810 PDFbox archive files invalid ExtraPackages enhancement moderate annoyingness nobody new 2012-02-06T04:28:52+13:00 2012-02-06T04:28:52+13:00 "Following the URL that displays when Greenstone GLI runs without the PDFbox extension installed, a user goes to the TRAC site and follows this path: Browse Source, gs2-extensions,pdf-box,trunk two archive files are listed: Pdf-box-java.zip (a zip archive for windows users) Pdf-box-java.tar.gz (a zipped tarball) While they are supposed to be 9.1 mb in size the actual files when downloaded are only 8kb in size and neither are valid archives This means the pdf-box extension cannot be used. The workaround is for users to convert all pdfs to version 1.4 before importing to greenstone." gnickers Possible 2.88 Release 813 Organising ex.dc* metadata GLI enhancement moderate defect nobody new 2012-03-12T14:24:12+13:00 2012-03-12T14:25:13+13:00 "From emails sent by Dr Nichols: I think it is fine to keep ex.dc.* if: - it is separated out and placed directly below the dc.* in the Enrich tab: so the sets go dc, ex.dc, gs, ex otherwise it is not visible, it is hidden in the rest of the ex stuff - the default format statements include ex.dc as one of the fallbacks - the lack of editability is explained in the UI somewhere: - with tooltips (which could include something like ""extracted metadata from file 000metadata.csv, this value can't be changed in GLI, make changes in the the original file', or some better phrasing - maybe with some other graphical marker in the left hand column, just like the inherited/folder metadata icon. Again with useful tooltip text. that is a simpler improvement." ak19 Possible 2.88 Release 814 Get Corba-based java-client for GS2 to work again Collection Building minor moderate defect nobody new 2012-03-15T20:58:42+13:00 2012-03-15T20:58:42+13:00 "The runtime-src/src/corba/corbaiface.idl has been changed (and corbaproto.mpp and many other classes) which will be committed in the near future (as at today, 15 Mar 2012) once testing has concluded. The datastructures defined therein have been expanded with additional member variables and corbaproto has additional methods and more variables to unpack. The corbaiface.idl that the GS2 java-client uses must be the same as that in GS2 runtime-src's corba. This means that the gs2-java-client's unpacking must be similarly be brought up to speed the way GS2's corbaproto.mpp has been. Once the changes have been made, to make sure things are still working, the updated GS2 java-client must be tested against GS2's corbaserver, as explained in the java-client's README file. http://trac.greenstone.org/browser/other-projects/gs2-java-client/ http://trac.greenstone.org/browser/other-projects/gs2-java-client/trunk/README " ak19 3.11 Release 815 Password Reset Greenstone3 Runtime major moderate task sjm84 new 2012-04-02T10:28:01+12:00 2021-03-22T09:46:26+13:00 We want the ability for users to be sent an email containing a reset password. Investigate using Apache James packaged up as an extension for Greenstone 3. sjm84 3.11 Release 816 Save prefs with user accounts Greenstone3 Runtime minor moderate defect nobody new 2012-04-12T09:37:48+12:00 2021-03-22T09:46:26+13:00 It would be good if things like search preferences were stored in the database with the user accounts so they are restored when the user next logs in. sjm84 Greenstone 2 wishlist 817 Search and replace in GLI GLI enhancement moderate annoyingness nobody new 2012-04-12T15:51:39+12:00 2012-04-12T15:51:39+12:00 "In the Edit pane, each item in the list of ""Existing values for "" at the bottom should allow search and replace (deletion and renaming options)." ak19 3.11 Release 818 Upgrade javascript Collection Building enhancement moderate task sjm84 new 2012-04-23T12:52:53+12:00 2021-03-22T09:46:26+13:00 Reorganise Greenstone 3's javascript so that it is object oriented and uses more jQuery. This should help with problems such as editing pages that have dynamically loading sections. sjm84 Possible 2.88 Release 819 OAI exploding and dc.* meta (not ex.dc.*) Greenstone2&3 major moderate defect nobody new 2012-04-27T14:48:51+12:00 2012-04-27T14:52:32+12:00 "John Rose in an email requested that downloaded .oai files' metadata get attached to downloaded documents as dc.* not as ex.dc.* Kathy considered that instead of the usual way that oai meta gets attached to its document, exploding the oai file might produce the desired results. Instead of the metadata getting attached to the document, it resulted in a nul file. The ex.dc.* turning up as dc.* instead is now a secondary problem. The relevant emails by John Rose and Kathy's responses are under the subject heading ""(Re: )Help please on OAI metadata"" in both private and mailing list communication." ak19 Possible 2.88 Release 820 Support for new Database Definition File format for CDS-ISIS database Collection Building: Plugins major moderate defect nobody new 2012-04-30T16:51:16+12:00 2012-04-30T16:51:16+12:00 "Renate wrote the following to the mailing list: I am using CDs-ISIS databases for building collections. In the meantime a web-based system based on the ISIS family was released in 2009. The database definition file (FDT) format differs from the old format. Is is planned that in future the new format can be used for the explosion of the database? The format of master and cross reference are the same." ak19 Possible 2.88 Release 821 Lucene is slow compared to MG Collection Building major moderate defect nobody new 2012-04-30T16:53:00+12:00 2012-04-30T16:53:00+12:00 "Diego wrote: Clacso is a leading case here in Argentina. They have more than 12.000 documents in 150 collections. They have GS 2.52 with mg indexes and when they run a query in the ""general"" collection (one supercollection that queries in all the collections) the performance is ver very good. If you look here: http://sala.clacso.org.ar/gsdl/cgi-bin/library and write something in the query box, like ""pobreza"" you will have the following results in a few seconds: Word counts: pobreza: 23984 4790 documents... Now I´m migrating it to 2.85 using Lucene as indexer. I also have a supercollection but when I run a query cpu goes to 100% and it takes minutes to get the results!. The link is http://sala.clacso.org.ar/gsdl285/cgi-bin/library.cgi?a=p&p=about&c=general&l=es&w=utf-8 Try ""pobreza"" again. You will have to wait a lot!!! I tried many options for Java. I edited lucene_query.pl to change java parameters, i.e: my $java_lucene = ""\""$java\"" -Xms1024m -Xmx1024m -XX:+AggressiveOpts -XX:+UseG1GC -classpath \""$classpath\"" org.greenstone.LuceneWrapper.GS2LuceneQuery""; but nothing change. Where is the problem?. The server resources?. The way Lucene do the queries?. Some specific configuration for Apache?" ak19 Possible 2.88 Release 822 Better processing of epubs Collection Building: Plugins minor moderate defect nobody new 2012-04-30T20:50:29+12:00 2012-04-30T20:54:39+12:00 "Hello Renate, I think the matter may be complicated by the fact that epub files are zips containing multiple files internally. There are two ways I can get Greenstone to do something (in fact, my aim was initially to try both of them together, since I thought that's how I could get things to work): SOLUTION 1. The first way is to simply add the UnknownPlugin and configure it with the following options as specified below: - mime_type: application/xhtml+xml - process_extension: epub SOLUTION 2. The second way does not conflict with the above, so leave your UnknownPlugin with the configuration settings described above in the plugin pipeline. But I also configured the ZIPPlugin's process_exp option to include epub: - process_exp: (?i)\.(gz|tgz|z|taz|bz|bz2|zip|jar|tar|epub)$ GLI wasn't happy building, and by turning the verbosity level up during import, it turned out that the ZIPPlugin's was using gzip on the Linux where I tested it, and gzip simply does not recognize the epub extension and gave up. What I did instead, to understand more of the issue, was go to the Gather panel and there rename the epub file's extension to "".zip"". Now it no longer mattered whether the ZIPPlugin's process_exp included ""epub"", but it doesn't hurt to leave it in. Hitting the Build Collection button, it processed all the individual XML files making up the chapters of the Pride And Prejudice (P&P) epub that I just downloaded for this test. Then in the Titles classifier it displayed every single chapter of P&P as a separate document, since each was a separate XML in the epub file. However, this did mean that the contents were indexed now: I can search on ""Darcy"" (the name of a character in the epub book) and Greenstone happily returned the results of several chapters. Having investigated a bit further now, I find that the ""gunzip"" program, which the ZIPPlugin uses by default for unrecognised extensions of compressed files, can't process .zip files. And epub extensions are essentially .zip files. However, the ZIPPlugin uses ""unzip"" to process .zip files. I merely added ""epub"" to the list of extensions to be processed by ""unzip"" in the ZIPPlugin script and also added it to the list of file extensions recognised by ZIPPlugin, and now I don't need to rename the epub file in the Gather panel to have a zip extension anymore, nor do I need to configure the ZIPPlugin to recognise epub anymore (as it's added to the process_exp by default). So if you wish to have the results of the 2nd solution outlined above, but without having to rename the epub file in the Gather panel and without having to configure the ZIPPlugin, then use a text editor to edit your Greenstone's perllib/plugins/ZIPPlugin.pm file as follows: a. Find the bit where it says: sub get_default_process_exp { return q^(?i)\.(gz|tgz|z|taz|bz|bz2|zip|jar|tar)$^; } And append the ""|epub"" near the end of this, for it to become: sub get_default_process_exp { return q^(?i)\.(gz|tgz|z|taz|bz|bz2|zip|jar|tar|epub)$^; } b. Find the bit where ZIPPlugin.pm says: } elsif ($file =~ /\.(zip|jar)$/i) { $self->unzip ($filename_no_path); And append ""|epub"" once more to get this to look like: } elsif ($file =~ /\.(zip|jar|epub)$/i) { $self->unzip ($filename_no_path); I am going to commit the ZIPPlugin file with only the 2nd change above, so that by default, the epub file is not unzipped by ZIPPlugin and you can use the UnknownPlugin to treat the entire epub as one single file (as seen in Solution 1). However, if a GLI user chooses to configure the process_exp option of the ZIPPlugin to include epub files, then Greenstone will automatically work as in Solution 2 above." ak19 Possible 2.88 Release 823 """Illegal character in URL"" bug when using remote GS between Linux server and win client" Collection Building major moderate defect nobody new 2012-05-03T16:17:49+12:00 2012-05-03T16:17:49+12:00 "Pierre Fillon's message to the mailing list: Dear Greenstone users, We cannot run properly the remote GLI, having the message ""Illegal character in URL"" every time we are moving or deleting files or directories. The GLI (2.85) is installed on PCs / Windows (various versions). A screen capture of the GLI running process window is joined (Err.jpg). The collections are on 64 bits Linux servers (x86_64) : - Greenstone 2.84 on Ubuntu 10.04.3 / Apache 2.2.14 / PERL 5.10.1 - Greenstone 2.85 on Debian 5.0 (Lenny) / Apache 2.2.9 / PERL 5.10.0 The character set is UTF-8 on the Linux servers. The problem remains when all the file names are lower than 32 characters and don't have any special character (like accents). The problem remains when we replace PERL 5.10 by PERL 5.8.9 (http://wiki.greenstone.org/wiki/index.php/2.83_Release_Notes#Perl_5.8.9_for_Linux). The problem remains when we specify the default character set in Apache (AddDefaultCharset On) (which is ISO-8859-1). I suspect it has something to do with PERL scripts launched by Apache with a wrong character set. Does anyone know this problem or has a clue to solve it ? Thank you for your answers," ak19 Possible 2.88 Release 826 CDS/ISIS plugin doesn't work on 64 bit machine Collection Building major moderate defect nobody new 2012-06-07T12:47:03+12:00 2013-10-24T15:50:10+13:00 "Is this easy to fix? Lots of longs throughout the code... I don't think CDS/ISIS works on 64 bit machines, but our isis-gdl should perhaps be modified to handle files from CDS/ISIS." kjdon Collection building wishlist 827 ABCD ISIS enhancement Collection Building enhancement low enhancement nobody new 2012-06-14T11:29:04+12:00 2013-10-24T15:49:54+13:00 "Renate is using databases from ABCD, a variant of ISIS. It has the same database format as CDS/ISIS but a different FDT format. Can we modify the isis-gdl program to support this? ABCD info: http://bvsmodelo.bvsalud.org/php/level.php?lang=en&component=27&item=13 Attaching a sample ABCD fdt file" kjdon Possible 2.88 Release 829 classifying on two metadata simultaneously Collection Building: Classifiers enhancement moderate enhancement nobody new 2012-06-14T14:01:13+12:00 2012-06-14T14:01:13+12:00 "Amos (mailing list May 2011) asked for the following (I think this is what he is meaning) Have metadata - course code - course name Lots of exam papers for each course. Want to have a bookshelf with course codes, and papers underneath. Instead of just having the course code listed, he wants code and name on the bookshelf. I assume that there is a one-to-one relationship between code and name. Could classify on code or name, but the problem is the bookshelf won't have the other metadata. So really you want to classify on concatenated metadata. eg code+name, so then both fields would appear in the bookshelf. This should be easy to do. But what about if there are multiple values? do we do all combinations? or limit to the first value?" kjdon Collection building wishlist 830 List classifier option Collection Building: Classifiers enhancement moderate enhancement nobody new 2012-06-14T14:54:52+12:00 2012-06-14T14:54:52+12:00 "Can we make an option so that only items with a particular value of metadata are included. eg have type metadata, book/newspaper/article for example. Make a classification including only items where type=book. " kjdon Collection building wishlist 831 global metadata editing Collection Building enhancement moderate enhancement nobody new 2012-06-14T15:05:31+12:00 2012-06-14T15:05:31+12:00 Is there a way to globally edit metadata? eg change all dc.Subject Gaelway to Galway. kjdon Collection building wishlist 832 sort metadata display Greenstone2&3 enhancement moderate enhancement nobody new 2012-06-14T15:07:29+12:00 2012-06-14T15:07:29+12:00 "Can we sort alphabetically all subjects before displaying them for a document? eg [sort:sibling('All':dc.Subject)] or some such. or maybe use javascript?? would it be easier with xslt in greenstone 3?" kjdon Possible 2.88 Release 833 export as csv Collection Exporting enhancement moderate feature nobody new 2012-06-14T15:25:26+12:00 2012-06-14T15:25:26+12:00 "Several people have asked on the mailing list for metadata export as csv file, esp excel format. One row per document, giving all the metadata. Should be easy enough to do? A CSVPlugout??? I guess this would need a header row giving the metadata fields. This may be tricky to know in advance. Perhaps this row is written last when we know what fields are available? Or maybe a plugout option gives the metadata fields to be output. " kjdon Possible 2.88 Release 834 List bookshelf sorting Collection Building: Classifiers enhancement moderate enhancement nobody new 2012-06-14T15:28:03+12:00 2012-06-14T15:28:03+12:00 Can we have an option to reverse sort one particular level in a classification hierarchy? eg if one of the levels is Date, often want that in reverse order so newest docs are first. kjdon Possible 2.88 Release 835 cover image thumbnails Collection Building enhancement moderate enhancement nobody new 2012-06-14T15:30:21+12:00 2012-06-14T15:30:21+12:00 Can we add an option to create thumbnails of the cover images to be used in search results? kjdon 3.11 Release 836 Mixed language strings in GS3 installer (Win and Lin) Installation major moderate defect nobody new 2012-07-03T14:40:32+12:00 2021-03-22T09:46:26+13:00 "Diego noticed this problem and explains: I´ve installed GS3 from caveat-emptor and the installer seem to be mixing the text strings. I select english as language, but i.e. when you have to select the installation folder, you have: Installation Folder (english) Seleccione una carpeta para la instalación (spanish) and a button with the label ""Examinar"" (instead of ""Browse"") and every step has mixed texts. I think that it is a mix between the language you select and your operating system´s locale. The same occurs in Linux an Windows. " ak19 3.11 Release 837 Per-site user databases Greenstone3 Runtime major high task sjm84 new 2012-07-12T17:08:42+12:00 2021-03-22T09:46:26+13:00 At the moment each Tomcat context can only refer to one user database. Ideally we would like a separate database for each site. sjm84 Possible 2.88 Release 841 Upgrade to PDFBox 1.7 as it can convert txt pages to images Collection Building major moderate defect nobody new 2012-07-24T16:01:54+12:00 2012-07-24T16:02:00+12:00 "The -pagedimg_FORMAT option is now supported when using the PDFBox extension. Howerver, our pdfbox jar file comes to version 1.5, and only ""generates pages as images"" when PDF pages are actually images. The pdfbox jar version 1.7 is able to generate pages as images from PDFs containing text. However, the output images aren't always clean: sometimes columns of multi column documents overlap. This may be because the PDFToImage command of PDFBox is still in beta. Otherwise, including in terms of line spacing (an issue we had in the past), the 1.7 pdfbox jar file appears to perform like the 1.5 version. Should we upgrade already, or wait until the PDFToImage command works well before bothering to, since not much is gained at present?" ak19 Possible 2.88 Release 842 item file becomes page 0 in pagedimg collection Collection Building major moderate defect nobody new 2012-07-31T16:11:16+12:00 2012-07-31T16:11:16+12:00 "A GS mailing list member made the following useful observation: -------- Original Message -------- Subject: [greenstone-users] .item file Date: Mon, 30 Jul 2012 19:48:48 -0700 (PDT) From: Gayla Garrett Reply-To: Gayla Garrett To: Greenstone Users I have created a collection that uses the PagedImagePlugin with an .item file, and it is working except my item file is counting as the first page of the document. It displays a blank page and I have to click the next arrow to get to the images display. I've been through the Scanned Image and Advanced Scanned Image collection tutorials once a few weeks ago, but I don't recall if that is described in there. I am going to go through them again, but if anyone knows the immediate answer it would be greatly appreciated! Gayla I think this solves the mystery of why we see a page ""0"" in the Go To Page for the pagedimg collection: page 0 counting the item file would explain it." ak19 Possible 2.88 Release 846 "GS2: searching for ""econom*"" in lucene demo collection, section level vs doc level" Greenstone2 Build Code major moderate defect nobody new 2012-09-07T17:15:57+12:00 2012-09-07T17:15:57+12:00 "In GS2, if you build the docs in the demo collection with lucene as indexer (as we do in the indexers tutorial), then: - searching for ""econom*"" at document (book) level says that ""economic"" occurs 507 times. - searching for ""econom*"" at section (chapter) level says that ""economic"" occurs 508 times. - When searching for ""economic"" itself at section or book level still says it occurs 508 times. - In GS3, the same collection says that ""economic"" occurs 508 times, regardless of whether we searched for ""econom*"" or ""economic"" and regardless of whether either search was conducted at section or document level." ak19 Greenstone 2 wishlist 847 Dereferencing pointers to pointers on 64 bit machines Greenstone2&3 major moderate defect nobody new 2012-09-07T17:29:46+12:00 2012-09-07T17:32:50+12:00 "Dereferencing pointers to pointers on 64 bit machines. If not done correctly, this can result pointer addresses getting truncated to 32 bit values during intermediate conversions. At least, this is what happened in suffix.cpp (suffix.exe) used by the Phind browser, resulting in the occasional segmentation fault when the bug became fatal. (The code was shared between GS2 and GS3, so both were affected.) Perhaps we should consider inspecting the code to see if this sort of conversion happens anywhere else. " ak19 3.11 Release 852 GS3 source distributions still have extra demo collections Collection Building major moderate defect nobody new 2012-12-18T17:31:13+13:00 2021-03-22T09:46:26+13:00 "GS3 source distributions still have extra demo collections. Need to remove gs2mgppdemo, gs2mgdemo and infomine and text collections from GS3 source distributions, since the GS3 binary installs just the lucene demo collection." ak19 Possible 2.88 Release 853 URL-encoding forward-slash despite collection groups using this Greenstone2 Runtime major moderate defect nobody new 2013-03-07T15:27:25+13:00 2013-03-07T15:29:12+13:00 "cgiutils.cpp url-encodes several special chars, but its url-encoding of the forward slash has been temporarily disabled because of the forward slash being used in the qualified-collection names of collection groups, which broken when the forward slash was url-encoded. We'd like cgiutils.cpp to still encode it, but for collection groups to still work. The idea is that it should be decoded back to a forward slash at the earliest possible time for the collection subgroups to work. Maybe other chars encoded by cgiutils can be decoded at this point also, in order for things to be back to 'normal'." ak19 Possible 2.88 Release 854 GLI's Enrich panel won't display fields of custom metaset named exam GLI major moderate defect nobody new 2013-04-23T13:50:06+12:00 2013-04-23T13:50:06+12:00 " If a custom metadata set called ""exam"" with namespace ""exam"" is created in GEMS and this metaset is loaded into GLI from the Enrich panel, the fields of the exam metaset do not display. However if a metaset called ""dodo"" with namespace ""dodo"" were created in GLI, its fields do display when loaded into GLI's enrich panel. I think the problem may perhaps have something to do with the prefix of ""exam"" matching the ""ex."" metadata of Greenstone. Thanks to Renate from the mailing list who discovered the problem with the exam metaset." ak19 Possible 2.88 Release 856 Not receiving emails from Scheduling Greenstone2&3 minor moderate defect nobody new 2013-05-15T14:17:39+12:00 2013-05-15T14:17:39+12:00 "Renate sent in the following email: I have setup a collection to automatically rebuild the collection which seems to run at 24 hours. It runs on Linux. An email should be sent to me regarding the status. I have not installed sendmail, but ssmtp, and no message is delivered. Can I use ssmtp instead to send the log files? I tried to set up scheduling locally too, on Linux. It was set up to send hourly emails, but when I checked the next day, I hadn't received any emails. The following were the settings in the Create Pane's Schedule Options section: schedule frequency: hourly (action remained unticked) email toaddress: waikato.ac.nz fromaddress: greenstone mailing list address smtp: smtp.gmail.com (same SMTP as for GS mailing list) " ak19 3.11 Release 858 Many things to do for GS3 and after GS2 release Greenstone2&3 major moderate defect nobody new 2013-05-29T18:04:01+12:00 2021-03-22T09:46:26+13:00 "GS3 windows BINARY: setting metadata fails (not in svn version). Probably because activate.pl doesn't have the necessary supporting perl files in the cut-down version of perl that comes with the windows binary. GS3 Mac: activate.pl first call to special http/lwp functions fails. Need to investigate what perl files mac needs to get this to work. GS3 Mac: libjni error prevents using mg/mgpp and gdbm for collections. These may not have been compiled up well because the version of Java it finds is not what it needs to be. * GS3 user comments feature * GS3: need to be able to move GS3 installation folder, and everything should still work. * downloadfrom.pl is printing a fixed list (when called from GLI?) It ought to dynamically work out what it should print, like pluginfo.pl's -listall does. Perhaps GLI isn't calling existing functionality to dynamically download the information for the listing. * need more harddisk space on 64 bit VM. " ak19 3.11 Release 859 GS3 outstanding (installer and more) Installation major moderate defect ak19 new 2013-06-21T15:27:59+12:00 2021-03-22T09:46:26+13:00 "- installer: The following ant calls in installer ends up under the Imagemagick heading: gen-mac-shortcuts perl-shebangs " ak19 3.11 Release 860 Diffcol ToDo Greenstone2&3 major moderate defect ak19 new 2013-06-28T17:06:31+12:00 2021-03-22T09:46:26+13:00 "* investigate why images are different sizes between the linux-generated image collection and the test collection built on darwin or windows * investigate the cause of the difference in file sizes for index/idx folder, between linux and windows * GS3 diffcol testing * DEC testing with diffcol Also GS3 variant * src component testing with diffcol need DEC generated for GS3 as well Then have GS3 variant of DEC testing * diffcol-caveat and diffcol-stable Also GS3 variant * diffcol with isntaller version of binary (install in text-only mode) " ak19 3.11 Release 861 GS win binary: have a GS ready cmd console Greenstone2&3 major moderate defect ak19 new 2013-06-28T17:22:49+12:00 2021-03-22T09:46:26+13:00 Create a new Start menu shortcut for the Greenstone set of shortcuts on Windows, that will cd into GS home and have done setup.bat ak19 3.11 Release 862 Greenstone doesn't like empty script tags Greenstone3 Interface trivial very low annoyingness nobody new 2013-08-01T10:48:29+12:00 2021-03-22T09:46:26+13:00 " When creating a new interface, I noticed that the XSL doesn't like empty script tags ( is there a way to fix this?" jlwhisler Possible 2.88 Release 863 GLI scheduling feature needs updating Collection Building major moderate defect nobody new 2013-08-01T17:34:44+12:00 2013-08-01T17:34:44+12:00 "Renate wrote in to the mailing list that scheduling had some issues to do with moving building to index. The cron.pl it generates for her does the following: 1. an incremental-import and a full-buildcol.pl. Perhaps it should be an incremental-import and an incremental-buildcol.pl, both are grouped into incremental-rebuild.pl 2. the cron.pl does not use activate but manually moves the contents of building to index: building/* to index/* Need to consider whether it's useful to replace this with a line to separately run the activate.pl script with the -incremental flag. " ak19 3.11 Release 864 Name of SRW protocol deprecated in favor of SRU Downloading minor low defect nobody new 2013-08-06T08:12:34+12:00 2021-03-22T09:46:26+13:00 The name SRW is deprecated, and SRU should be used instead. This has been changed in the Download panel of the GLI, but the scripts downloadfrom.pl and downloadinfo.pl still use SRW and SRWDownload, respectively, which can be confusing for anyone downloading via SRU from the command line. jlwhisler Possible 2.88 Release 865 Absolute URLS in gdb and doc.xml - should the first or both be relative? Collection Building major moderate defect nobody new 2013-08-12T17:39:31+12:00 2013-08-12T17:39:31+12:00 "In doc.xml the URL and UTF8URL are absolute paths with http:// prefixed In the coll.gdb file, these as well as any tmp files generated are stored as absolute file paths. As long as they are used only as keys (as long as these filepaths represent just strings that we do lookups on), it's not a problem. But there was a time when they were relative paths and it makes more sense to store them as relative, it could potentially harm portability. It weakens what's tested for in diffcol, where tmp files (generated in time stamped folders) end up as keys in the gdb file which may occur in a different order in the gdb file from one OS to another because the tmp filepaths are not identical. At present diffcol ignores such tmp files, which weakens the comparison testing." ak19 DEC Release 2.86 867 Pro Greenstone3 Interface major moderate documentation nobody new 2013-09-04T22:19:55+12:00 2013-09-04T22:19:55+12:00 xamutara sutubatkhuat 3.11 Release 869 Windows GLI changing servlets creates a lock on Index dir GLI major moderate defect nobody new 2013-09-12T16:31:28+12:00 2021-03-22T09:46:26+13:00 "Windows GLI changing servlets creates a lock on Index dir In File>Prefs, changing servlets and rebuilding a collection thereafter results in an error that the index directory is in use. Need to exit and restart GLI to be able to successfully rebuild. This seems to consistently reproduce the index-lock error." jlwhisler 3.11 Release 870 GS3 uninstaller needs to preserve sites and interfaces Collection Building major moderate defect nobody new 2013-09-30T18:58:24+13:00 2021-03-22T09:46:26+13:00 Jenny noticed that the GS3 uninstaller only offers to preserve the collections in localsite, but not other sites. And need preserve site and interface modifications (customisations) as well. ak19 Possible 2.88 Release 875 GLI collect home handling GLI major moderate defect nobody new 2013-10-25T18:21:27+13:00 2013-11-01T16:52:09+13:00 "It can get confusing when you've installed a GS and had a collection open in its GLI and then install another GS and then ran GLI and it opened the collection in the old GS installation. Kathy came up with the following solution: * GLI's config.xml needs to remember GLI home If you open GLI and config.xml records a different GLI home had been using the config file, check if the library url is the correc tone for the new GS installation. Else clear config.xml's library_url, general_site_name and general_servlet_path. open_collection too? Bugs found: server.exe can't properly handle changing gsdlhome and does not respond to changing collecthome from within GLI. " ak19 3.11 Release 876 not all gsf:metadata select attribute values work GLI major moderate defect nobody new 2013-11-13T12:13:39+13:00 2021-03-22T09:46:26+13:00 GS3 manual (http://www.greenstone.org/docs/greenstone3/manual.pdf pg. 26) states that there are 7 possible values for gsf:metadata's select attribute: current, root, parent, ancestors, siblings, children, and descendents, which should allow you to refer to metadata for sections other than the current one (in relation to the current section). root, parent and ancestors appear to work correctly. However, current, siblings, children, and descendents don't display anything at all, though they should based on the manual description. jlwhisler Collection building wishlist 877 Tidy up export As in GLI GLI enhancement moderate enhancement nobody new 2013-11-28T11:35:22+13:00 2015-04-20T13:24:01+12:00 GLi export dialog has hard coded the options for the various export types. Make this dynamically display the export.pl options, and the options for the plugouts. All plugout options can be passed to export.pl under the general -saveas_options option. kjdon 3.11 Release 878 block system commands Greenstone3 Runtime enhancement moderate enhancement nobody new 2013-12-03T13:42:28+13:00 2021-03-22T09:46:26+13:00 "Currently anyone can run system commands and reload/delete collections etc. Make this more secure. Simple solution: Make it only available when logged in as admin. More flexible solution: Have a setting for system commands: - allow none - allow all - allow only admin users" kjdon Greenstone 3 catch-up 886 Add metadata formats to GS3 OAI server Greenstone3 Runtime enhancement moderate enhancement nobody new 2014-09-11T10:47:43+12:00 2015-04-20T13:19:17+12:00 "Greenstone 2 OAI server currently supports the following metadata formats: dublin core qualified dublin core rfc1807 Greenstone 3 oai server only supports dublin core. Add the others in. Is this merely a matter of adding a list of elements into OAIConfig with some mapping rules? - I think so for qualified dublin core. Or is the desired output in a different format, so more code changes will be needed? - maybe for rfc1807. Will need to check the metadata format." kjdon 3.11 Release 887 oai list sets response Greenstone3 Runtime enhancement low enhancement nobody new 2014-09-11T13:50:37+12:00 2021-03-22T09:46:26+13:00 "Currently the set description for a set in a ListSets response is just a text string. We should be using a metadata container?? See http://www.openarchives.org/OAI/2.0/guidelines-repository.htm#setDescription Communities are able to develop their own collection description XML schemas for use within the description container and setDescription elements. If all that is desired is the ability to include an unstructured textual description then it is recommended that repositories use the Dublin Core description element. But is it worth the effort?? Applies to greenstone 2 as well as greenstone3." kjdon 3.11 Release 889 GLI's GShell class - change to use StreamGobbler? GLI major moderate task nobody new 2014-10-10T17:39:09+13:00 2021-03-22T09:46:26+13:00 "GLI's GShell class creates and runs a Process. GShell's handling of the Input, Output and Error streams of the Process may not conform to the ""when stdin/stdout/stderr doesn't work"" article that Dr Bainbridge found. The gli/src util folder contains the *StreamGobbler classes to handle the IOStreams of Processes correctly. Should GShell be changed to use StreamGobbler like FormatConversionDialog.java (and GS3ServerThread.java) do at present? If so, the changes to GShell need to be tested extensively by using GLI, since GShell is GLI's way of running perl scripts and GLI is not tested by diffcol." ak19 3.11 Release 893 Upgrade tomcat in GS3 from 7.026 to 7.057 to work with JDK 8? Greenstone3 Runtime enhancement moderate defect nobody new 2014-12-10T17:21:26+13:00 2021-03-22T09:46:26+13:00 "I can get GS3 to work with Java 8, but only by upgrading the tomcat that GS3 uses to 7.0.57: I worked on an ""Ubuntu 14.04 LTS"" 64 bit machine (bedrock) here. I installed JDK1.8, set up the Java environment in my .profile and logged out and back in, just to make doubly sure all processes will use this java and no other. I then checked out gs3 from SVN and compiled it up. The compilation succeeded but running the GS3 tomcat server initially failed with the error: ""The type java.util.Map$Entry cannot be resolved."" Googling the error led to the stackoverflow page http://stackoverflow.com/questions/23825048/jdk8-the-type-java-util-mapentry-cannot-be-resolved The suggested solution was to update tomcat to 7.0.54. I found 7.0.57 at http://tomcat.apache.org/download-70.cgi I extracted it into my new gs3-svn's ""package""s folder and renamed the extracted folder to ""tomcat"", after renaming the old ""tomcat"" folder there to tomcat-7.0.26. The new tomcat contains up-to-date helper jar files for jasper and ecj, as was mentioned at stackoverflow. Restarting the server now allows me to to view the /greenstone3 servlet. I can search the lucene demo collection after rebuilding it. Rebuilding the solr collection from GLI and searching it still works, so on the surface the new tomcat seems to be okay with JDK8. I then repeated the above with the JAVA settings back to JDK 7, to confirm that the newer tomcat works with Java 7. It seems to be fine, so maybe we can make GS3 ship with the newer tomcat by default, in order for GS3 users to be able to work with Java 8 out of the box. Therefore, to try Java 8 with GS3, you will need tomcat 7.0.57. Untar it into GS3/packages and name the folder ""tomcat"" after moving out the old one. " ak19 Possible 2.88 Release 894 GEMS to forbid space in metadata names GEMS minor moderate defect nobody new 2015-03-16T09:24:30+13:00 2015-03-16T09:24:30+13:00 "Renate managed to create a metadata set with 'Paper title' as a metadata name. This breaks the collect.cfg file and other places. GEMS shouldn't allow spaces (or other funny chars???) in metadata names." kjdon 3.11 Release 896 User Sessions need more investigation Greenstone3 Runtime major moderate defect nobody new 2015-04-16T18:03:15+12:00 2021-03-22T09:46:26+13:00 "Some work needs to be done in GS3's LibraryServlet.java around the session related information. The java code maintains a session table, and tomcat provides the session variable too, and we'd like to ensure these are kept in sync. Further, one of the problematic areas identified is the code upon ""clean_all"" in LibraryServlet.java. Firstly, the session_ids_table is entirely cleared (cleaned) upon ANY system action that is not activate or deactivate (or Ping). Maybe this should happen only when a global MessageRouter reconfigure is requested. Also, all sessions other than the current User Session should be marked as invalidated too, since the corresponding data in the session_ids_table in memory was cleared. We will also want to investigate how tomcat stores session data, does it use serialization. And if so, does the user_session_cache table whose hash we put in there also get serialized sensibly, are is it just a memory location that becomes stale upon a server restart." ak19 DEC Release 2.86 897 Died at unicode.pm line 149 Collection Building major moderate defect nobody new 2015-06-05T01:04:05+12:00 2015-06-05T01:04:05+12:00 "good afternoon I'm trying to build a collection and get an error message: import.pl> HTMLPlugin processing E:\Greenstone\tmp\F378.html import.pl> Died at E:\Greenstone/perllib/unicode.pm line 149. import.pl> Error: Failed to run: ""E:\Greenstone\bin\windows\perl\bin\Perl.exe"" -S import.pl -removeold ""-gli"" ""-language"" ""en"" ""-collectdir"" ""E:\Greenstone\collect"" ""tmp"" import.pl> Command failed. what's wrong?" APN 3.11 Release 899 Built-in citation for GS3 in the style of R Collection Building major moderate defect nobody new 2016-01-13T16:46:14+13:00 2021-03-22T09:46:26+13:00 "Commandline -citation can print out how to cite GS3 software in plain text and in bibtex format. The R stats programme wrote a function() that shows an example citation and provides the bibtex too. Dr Nichols suggested that in Greenstone, we could add that in a help feature too." ak19 Possible 2.88 Release 900 empty metadata.xml file triggers reimport Collection Building major high defect nobody new 2016-04-01T13:48:45+13:00 2017-05-11T13:44:12+12:00 "Empty metadata files are assumed to be new, and therefore cause everything to be reimported in an incremental build. Can we either look inside to realise that they are empty? Or maybe we add it into one of the databases so that we know we have seen it before and it hasn't changed since last time." kjdon 3.11 Release 901 Windows release needs newer version of Perl ExtraPackages major moderate defect nobody new 2016-05-06T17:36:54+12:00 2021-03-22T09:46:26+13:00 "For the next release we should make the windows bin release include a newer version of Perl. Later versions of Perl recognise invalid UTF-8 characters as such and doesn't fail on them. This is necessary to work with Georgy's change for processing UTF-16 specific (and non-UTF8) characters in Diego's PDF. See http://trac.greenstone.org/changeset/30491 and http://trac.greenstone.org/changeset/30492" ak19 3.11 Release 903 build.xml issue while collection build is running Collection Building major very high defect nobody new 2016-05-14T04:18:13+12:00 2021-03-22T09:46:26+13:00 "Please help to resolve the below build issue in greenstone3. When i was doing the build process by giving command ant prepare, the following errors shows; '''''E:\GS3-SVN>ant prepare Buildfile: E:\GS3-SVN\build.xml [echo] os.name: Windows 7 accept-properties: [input] The following properties (among others) are being used from a build properties file found in this directory: [input] tomcat.server=localhost [input] tomcat.port=8383 [input] tomcat.installed.path= (this is the location of Tomcat's base dir if it is already installed) [input] proxy.host= [input] proxy.port= [input] disable.collection.building=false [input] If these are not acceptable, please change them and rerun thi target. Continue [y/n]? [input] (y, n) y init: [echo] tomcat.port = 8383 copy-dot-in-files: [copy] Copying 1 file to E:\GS3-SVN\resources\tomcat [copy] Copying 1 file to E:\GS3-SVN\resources\tomcat [copy] Copying 1 file to E:\GS3-SVN\resources\tomcat [copy] Copying 1 file to E:\GS3-SVN\resources\tomcat [copy] Copying 1 file to E:\GS3-SVN\resources\oai [copy] Copying 1 file to E:\GS3-SVN\resources\cgi [copy] Copying 1 file to E:\GS3-SVN\resources\web [copy] Copying 1 file to E:\GS3-SVN\resources\web prepare-core: prepare-packages: prepare-common-src: prepare-gs2build: [echo] os.name: Windows 7 init: [echo] tomcat.port = 8383 checkout-gs2build: [echo] checking out gs2build BUILD FAILED E:\GS3-SVN\build.xml:2740: The following error occurred while executing this lin e: E:\GS3-SVN\build.xml:2752: Execute failed: java.io.IOException: Cannot run progr am ""svn"": CreateProcess error=2, ?w?????t?@? at java.lang.ProcessBuilder.start(ProcessBuilder.java:460) at java.lang.Runtime.exec(Runtime.java:593) at org.apache.tools.ant.taskdefs.launcher.Java13CommandLauncher.exec(Jav a13CommandLauncher.java:58) at org.apache.tools.ant.taskdefs.Execute.launch(Execute.java:428) at org.apache.tools.ant.taskdefs.Execute.execute(Execute.java:442) at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:629) at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:670) at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:496) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293) at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.jav a:106) at org.apache.tools.ant.Task.perform(Task.java:348) at org.apache.tools.ant.Target.execute(Target.java:435) at org.apache.tools.ant.Target.performTasks(Target.java:456) at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1405) at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(Single CheckExecutor.java:38) at org.apache.tools.ant.Project.executeTargets(Project.java:1260) at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:441) at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:105) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293) at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.jav a:106) at org.apache.tools.ant.Task.perform(Task.java:348) at org.apache.tools.ant.Target.execute(Target.java:435) at org.apache.tools.ant.Target.performTasks(Target.java:456) at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1405) at org.apache.tools.ant.Project.executeTarget(Project.java:1376) at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExe cutor.java:41) at org.apache.tools.ant.Project.executeTargets(Project.java:1260) at org.apache.tools.ant.Main.runBuild(Main.java:853) at org.apache.tools.ant.Main.startAnt(Main.java:235) at org.apache.tools.ant.launch.Launcher.run(Launcher.java:285) at org.apache.tools.ant.launch.Launcher.main(Launcher.java:112) Caused by: java.io.IOException: CreateProcess error=2, ?w?????t?@? at java.lang.ProcessImpl.create(Native Method) at java.lang.ProcessImpl.(ProcessImpl.java:81) at java.lang.ProcessImpl.start(ProcessImpl.java:30) at java.lang.ProcessBuilder.start(ProcessBuilder.java:453) ... 36 more Total time: 2 seconds E:\GS3-SVN>'''''" gunasekarank15 3.11 Release 904 Search suggestions for solr collections using ajax Greenstone3 Runtime enhancement moderate feature Georgy new 2016-05-19T14:27:03+12:00 2021-03-22T09:46:26+13:00 "For a solr collection, the search box could provide search completion or suggestions (showing terms already present in the index). The solr server knows about it. The search box can use ajax which could in theory communicate directly with the solr servlet to get the search suggestions. The only hurdle is that we've made the solr servlet inaccessible for non-localhost machines (see the solr.xml file tomcat context file). We want it to be selectively accessibly for these ajax calls being performed from client machines." ak19 3.11 Release 905 remove java hard coded metadata names Greenstone3 Interface enhancement moderate enhancement kjdon new 2016-06-20T12:54:35+12:00 2021-03-22T09:46:26+13:00 "The server reads through xslt files and format statements looking for gsf:metadata etc to work out what metadata should be returned from the server. Things like requires Thumb metadata to be retrieved. Currently this is hard coded in the java code. (GSXSLT.findExtraMetadataNames) We need some way of specifying this in the xslt instead so we don't need to modify the server when we add new things to eg config_format.xsl. * can we apply config_format first before looking for metadata? then we'd need to look for /metadataList/metadata.... * Maybe config format could leave in a tag? * maybe we do a pass through config format which results in a mapping between gsf elements and which metadata they require? and use that to determine what to output for gsf.xxx? in this case, we wouldn't need to hard code gsf:metadata itself in the code as the list of gsf:xxx elements would be dynamic too." kjdon 3.11 Release 908 Update GLI internal XML format GLI enhancement moderate enhancement kjdon assigned 2016-08-29T09:56:14+12:00 2021-03-22T09:46:26+13:00 Make this the same as gs3 config file format. kjdon 3.11 Release 909 Greenstone 3 depositor Depositor enhancement moderate feature nobody new 2016-09-22T13:37:27+12:00 2021-03-22T09:46:26+13:00 "This is partially implemented. Use url arguments a=de to get to the first depositor page. You can go through the sequence of steps, adding metadata, choosing a file, rebuilding etc. The file gets put into import, but not the metadata. Not sure if the rebuild actually happened or not. Need to get this working properly. And when we do that, move the text strings out of xsl/js files and into interface_default.properties file." kjdon 3.11 Release 910 PagedImagePlugin - encoding for text item files Collection Building major moderate defect nobody new 2016-11-16T17:40:13+13:00 2021-03-22T09:46:26+13:00 "A fix was added (revision 31113) for 3.08 that meant text item files that used utf-8 were processed correctly. However, there does not appear to be any code in this plugin that handle this situation where a plugin option is provided to specify that the input file is in a different encoding (e.g. Latin-7). Some additional testing needs to be done to clarify the situation. In particular, have an item file that has some unusual punctuation in it (e.g. Spanish upside down question mark) and add a temporary line into the plugin that eliminates all [:punct:] and make sure that the unusual punctuation does indeed get removed in the built collection. Now repeat the test where the input file is a non-UTF-8 encoding with unusual punctuation." ak19 3.11 Release 911 Improving Javascript interaction with Java GS3 server Collection Building major moderate defect nobody new 2016-11-18T15:19:02+13:00 2021-03-22T09:46:26+13:00 "At present, the JS code is polling to find that collection building and activation has finished. Polling within Java may be solved with event handlers firing events (causing interrupts) and activating event handling code. But when Javascript on the client machine is interacting with java code on the server, to find out when a process is finished, google for: - web long poll (hack to avoid polling by asking the server's OS to do a system file look up process, that blocks until ready and so has the side-effect we want) - Better solution: web sockets. Confirm it is supported by all browsers. But if using web sockets, need graceful fallback for older browsers and platforms that don't support web sockets. For instance, need to check if Android supports web sockets. Google ""shim"", one or more short lines of code, that produce the functionality in a browser that has not yet been officially implemented in that browser, to bring it up to speed in behaviour with other browsers where that feature has been implemented. " ak19 3.11 Release 912 Flag arguments to full-rebuild Collection Building minor moderate defect kjdon new 2016-11-21T13:42:06+13:00 2021-03-22T09:46:26+13:00 "Full-rebuild.pl can take import or build specific arguments using the -import:arg_name arg_value syntax. It is always looking for an arg value, however, so you can't pass flag arguments. Make it check arg names against a list of flag args and not look for a value if its a flag. Make the list of flags from the arg data structure. import.pl has a few args itself (most are in inexport), so may need to have import.pm so that full-rebuild.pl can load in import::arguments. Also, make the -NO_IMPORT option work for gs2 in collect.cfg? or is there no need for it there???" kjdon 3.11 Release 913 edit metadata - disallow for internal ones Greenstone3 Runtime enhancement moderate enhancement kjdon new 2016-11-21T14:04:26+13:00 2021-03-22T09:46:26+13:00 "In gs3 web editor, *all* metadata is displayed and editable. But the user shouldn't be allowed to modify some internal ones, eg docnum. or nothing will work properly anymore. These metadatas should be displayed together (at bottom of list?) in a non editable form. Is this all the ones that start with lowercase letter? Also, what happens if we change Identifier? will that work? It should. would be good to be able to manually change it." kjdon 3.11 Release 915 changing ports in gs3-server Greenstone3 Runtime enhancement moderate enhancement nobody new 2016-11-22T13:21:58+13:00 2021-03-22T09:46:26+13:00 "in the server program, File->Settings allows the user to change the tomcat port. we should add in shutdown port and derby port to here too. Maybe all three are sequential? 8383, 8384, 8385? If the user changes the main one (8383) then the other two will automatically change to be sequential from that. And then there could be a button to allow manual explicit change of the other two?" kjdon 3.11 Release 916 CGI module needed for document editor Greenstone3 Runtime major moderate defect nobody new 2016-12-01T11:11:55+13:00 2021-03-22T09:46:26+13:00 "Make document editor work for a perl without CGI installed in its system folder. Greenstone3 needs a full CGI module in Greenstone cpan area and the corresponding PERL5INC? path set. On research-internal the installed perl had no CGI in it and document editor didn't work. Need CGI.pm, CGI/Util.pm and more??? " kjdon 3.11 Release 919 tidy up gs3 groups Greenstone3 Interface minor moderate enhancement kjdon assigned 2016-12-19T13:30:12+13:00 2021-03-22T09:46:26+13:00 When viewing a 'group' home page, the description should be that of the group, not of the site. And you should get the groups added into the page breadcrumbs so you can easily go back to the next group up. kjdon 3.11 Release 920 solr node content contains section heading Greenstone2 Build Code minor moderate defect kjdo new 2016-12-19T14:31:43+13:00 2021-03-22T09:46:26+13:00 "If you do a search, then view a section of a document, you can see right at the end of the text the section title is repeated. but not necessarily for all sections. a bit weird. it doesn't show up when you browse to a doc then go into it. Is this due to what we are putting into solr? Or to do with search term highlighting - the textQuery returns nodeContent, which contains the title. Can we get rid of it?" kjdon 3.11 Release 924 Not all metadata should be editable or visible in Document Editor Greenstone3 Interface minor moderate defect nobody new 2017-04-06T14:33:36+12:00 2021-03-22T09:46:26+13:00 "On linux, I always added user comments to my own small test collections and did document editing tests on the demo collection. During Windows testing, in order to save time, I tested both user comments then document editing on the demo collection, on the same doc. When editing a document for which comments were already added, noticed that the usercomment related metadata all appeared in the online metadata editor. Do we want people with doc editing powers to be able to edit someone else's usercomments? More generally, Kathy says there is definitely a list of meta which we don't want users to edit, such as assocfilepath, docnum and others. She thought of that having a reserved list of ineditable metadata, and checking against this when displaying the editable meta in the online doc meta editor, would be necessary." ak19 3.11 Release 926 gsf:link in CCS format Greenstone3 Interface enhancement moderate enhancement kjdon new 2017-04-10T11:24:30+12:00 2021-03-22T09:46:26+13:00 "Currently gsf:link only works on single colleciton pages, not in cross collection search results. Make it work in that case too. normal and See the siteConfig.xml format statement for how to construct those links." kjdon Possible 2.88 Release 927 oai classifier Collection Building major moderate enhancement ak19 new 2017-04-13T09:48:29+12:00 2017-04-13T09:48:57+12:00 "When building a collection, you get a message like ""outputting information for classifier: CL7"" . This is presumably generating the oai database?? If so, I think it should say outputting information for OAI. Is the OAI stuff still done as a classifier? its not just the old stuff left over??? Also, this can be very slow for large collections. Some people may never want to support OAI, so there should be a way in the config file to turn off generating the OAI info. " kjdon 3.11 Release 928 GLI adds index displayItems to top level for gs3 colls GLI minor low enhancement nobody new 2017-04-13T13:44:00+12:00 2021-03-22T09:46:26+13:00 They should be inside indexes not in the top level. And is it still adding in the gs2 versions?? FIX kjdon 3.11 Release 929 Case insensitive searches with Lucene Greenstone2&3 major moderate enhancement nobody new 2017-04-19T17:11:26+12:00 2021-03-22T09:46:26+13:00 "A question came in to the mailing list about how to make searches case insensitive. This is possible already in MGPP, so the question may have been about Lucene. From the page http://stackoverflow.com/questions/5512803/how-to-make-lucene-be-case-insensitive It seems that to make Lucene case sensitive would be a programmatic thing. We'll need to think about whether we want to support this and how to tie this into a configuration change that users can make dynamically." ak19 3.11 Release 930 berry basket send mail Greenstone3 Interface minor moderate defect nobody new 2017-06-12T13:10:47+12:00 2021-03-22T09:46:26+13:00 "In the berry basket email interface, you always get a ""sending mail succeeded"" message, even when it has failed. Can we at least make it say ""sending mail failed"" or give an error?" kjdon Greenstone 3 catch-up 931 Security Issues to be fixed Greenstone3 Interface major high enhancement nobody new 2017-07-28T20:56:58+12:00 2017-07-28T20:56:58+12:00 "The following issues are yet to be fixed. 1. Reflected cross site scripting 2. Open Redirection 3. Click jacking vulnerability 4. Brute Force attack in login page 5. Improper error handling attached files" gkka321 3.11 Release 935 Make perllib/IncrementalBuildUtils.pm reuse existing code Collection Building trivial very high defect ak19 new 2017-09-15T21:04:56+12:00 2021-03-22T09:46:26+13:00 "Related to commits 31972 and 31973. Commit http://trac.greenstone.org/changeset/31972 contained a bug fix to perllib/IncrementalBuildUtils.pm by Dr Bainbridge. IncrementalBuildUtils determines the PATH separator based on whether $GSDLOS contained the ""win"" substring denoting Windows. However, for the Mac OS, $GSDLOS has the value ""darwin"", which contains the ""win"" substring too. So Dr Bainbridge fixed the test to check for win as OS name prefix. This part of the code will need to be reworked in future, such as to use util::envvar_prepend() and envvar_append() functions instead of concatenating something that may already be on the PATH. Also, check for any existing methods to test if OS is windows or not to put it all in one place. Make the changes that reuse code BEFORE starting on GS3 release testing." ak19 3.11 Release 936 Remote GS environment and GS3 Imagemagick Env if it conflicts with wget Greenstone2&3 major moderate task ak19 new 2017-09-25T18:55:44+13:00 2021-03-22T09:46:26+13:00 "For the GS287 release (rc1), gsdlCGI.pm::setup_gsdl() started setting PERL_PERTURB_KEYS and WGETRC, newly added environment vars since the previous GS2 release. However, we should think of renaming the current setup_gsdl to old_setup_gsdl. And then have setup_gsdl() call util::setup_greenstone_env(), to set all the env vars that are active upon calling setup.bash. setup_gsdl() would also have to continue setting any remote GS specific env vars." ak19 3.11 Release 937 PDFBox text conversion Collection Building minor moderate defect nobody new 2017-11-23T12:13:10+13:00 2021-03-22T09:46:26+13:00 "Diego has a PDF file. When you convert to text using pdfbox, the output is invalid for Lucene. MGPP handles it ok, I guess not trying to parse the text as XML. &# -> &# in the html case &# -> &# in the text case. In the text case, it ends up as &# going through to lucene which then complains as &# should be the start of an entity. Can we change the output? Maybe it doesn't make sense to have convert to text as we always are putting the output inside XML. A simple lest file containing the content ""Katherine was here &# some chars."" also fails with the same error. org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 231; A decimal representation must immediately follow the ""&# mgpp processes it ok. Maybe this is not a plugin specific thing, but rather we need to encode more before passing through to lucene?? Note the conversion output is actually wrong for diego's document (looks like + '+1&- ,+-.# *# #9+/ #$ /7- *#$ #$.+ 1 #$ /7- *# ), but regardless of that, it shouldn't cause lucene to fail. The file was too big to upload here. I have put it at files.greenstone.org:/greenstone/files/testfiles/Diego-test.pdf" kjdon 3.11 Release 940 Apache Tika - see if Sam's GS2 extension works and write up tutorial Collection Building enhancement moderate defect nobody new 2018-03-27T16:02:38+13:00 2021-03-22T09:46:26+13:00 "One of questions by Tom Ip on the mailing list was whether there was support in GS for Apache Tika's comprehensive document format conversion tool. It turns out that Sam had written an extension for Tika, including a document conversion plugin (pm file), see http://trac.greenstone.org/changeset/22690 1. Try to download his jar http://trac.greenstone.org/browser/gs2-extensions/tika/trunk/tika-java.tar.gz and see if the existing version works 2. Try to get it working otherwise. 3. Maybe upgrade to the latest version of Tika and ensure it still works. 4. Write up a tutorial or else at least a wiki page on how to use this extension with GLI." ak19 3.11 Release 942 New pdftohtml with Xpdf tools - works with newer PDFs too Collection Building major moderate defect ak19 new 2018-06-28T16:01:05+12:00 2021-03-22T09:46:26+13:00 "Kathy found that users on the mailing list wanted more HTML output options with PDFPlugin. PDFBox's pagedimg output option was modified to produce img+text, but Kathy was hoping there were more possibilities for actual PDF to HTML support out there. Dr Bainbridge first found PDFtoDOM which was based on PDFBox. But this produced unsatisfactory HTML (sometimes fonts weren't extracted, often fonts made the display hard to read due to overlapping characters, a
element around every word rather than every line). Then Dr Bainbridge found XPdf Tools, which contained a new pdftohtml, which produced results we liked. Its pdftohtml tool outputs screenshots of each PDF page's background + the text overlaid, all as HTML. One html doc per page was produced, and we'd manipulate these into a single sectionalised HTML doc. To get Xpdf tools to work with GS in this way: 1. Downloaded Xpdf tools binaries for Lin/Win/Mac, eventually to be compiled up for Lin & Mac 2. To manipulate the HTML DOM produced, Dr Bainbridge found the perl module Mojo::DOM, which he compiled up. 3. Then the code was modified to make use of these. The list of commit revisions so far follow below. 4. This led to thinking that PDFPlugin needed to be restructured as its configuration options were already complicated and filled with mutually contradictory options since pdfbox_conversion was included, and now to become more complicated and contradictory with the inclusion of XPDF tools. The commit revisions thus far that make use of Xpdf Tools' pdftohtml and its pdftotext to finally support PDF to text conversion on Windows are as follows. None of these commits concern restructuring the PDFPlugin as yet. http://trac.greenstone.org/changeset/32205 - http://trac.greenstone.org/changeset/32210, http://trac.greenstone.org/changeset/32215, http://trac.greenstone.org/changeset/32219 - http://trac.greenstone.org/changeset/32224 Note that the Xpdf tools binaries for mac have been committed to an svn ignored folder and that they're not yet automatically checked out. Either we get Xpdf tools to compile from src (if we can get past the fact that Xpdf tools use CMake to configure and build rather than autotools' configure script that we're used to) or we find a better SVN location to put the Mac binaries of Xpdf tools. " ak19 3.11 Release 946 Meta and/or fulltext in SQL database as alternative to GreenstoneXML/METS Collection Building major moderate feature ak19 new 2018-11-09T16:37:57+13:00 2021-03-22T09:46:26+13:00 "This was Diego's request and a very useful suggestion: that instead of storing metadata and fulltext in the doc.xml (docmets.xml) files in archives, metadata and/or fulltext will be written out to a MySQL database (the remainder will go into docsql.xml files, which are always generated). Note that only the 'archives' is affected, not the index folder, as import.pl will write to meta/fulltext to the SQLdb, and buildcol will read it back in from the SQLdb into docobjects which are then processed into the index folder as usual. More information at http://wiki.greenstone.org/doku.php?id=en:user_advanced:greenstonesqlplugs The rest of this ticket will list the SVN changesets." ak19 3.11 Release 953 Get rid of Applets Greenstone3 Runtime enhancement moderate enhancement nobody new 2019-07-01T14:16:07+12:00 2021-03-22T09:46:26+13:00 "Applets not supported very well any more. Rewrite Phind and Collage as javascript? " kjdon 3.11 Release 956 Minor changes to config for Images GPS tutorial after 3.09 Greenstone3 Interface enhancement moderate defect ak19 new 2019-09-30T19:20:29+13:00 2021-03-22T09:46:26+13:00 "As at 30 Sep 2019, the Images GPS collection works the same with a caveat emptor GS3 linux binary as with a 3.09 official release GS3 binary. So that's good. Dr Bainbridge would like: 1. siteConfig.xml should have favouriteBasket turned off again by default. But since Kathy turned it on for a reason (see changeset for revision 33491), let's ask her first whether it can be set to off by default again. 2. Find the Classifier settings for the Files and Titles classifiers that will display fewer documents per page and will split them over multiple pages. That way, we can demonstrate documents that are not currently displayed on the page (but are on the next pages in the classifier) with blue markers on the map, while those on the page are still displayed with red markers, as is said to be the case with the Paradise Gardens collection." ak19 3.11 Release 959 allow collecthome change from popup GLI enhancement low enhancement nobody new 2020-09-15T11:28:17+12:00 2020-09-15T11:28:28+12:00 "If you are running different glis, and you start up a new one, and the collecthome is not the standard location, you get a dialog box saying Your current collect home /Scratch/kjdon/gs3-sites-revamp/web/sites/localsite/collect is located outside of your Greenstone installation and is not its default collect directory at /Scratch/kjdon/gs3-pei-jones-plus-twso/web/sites/localsite/collect. To work with the collect directory of your choice, go to File > Preferences > Connection and set the Collect Directory field to a folder. It would be good to have a checkbox/button here to switch to the standard location for the current greenstone, without going through preferences." kjdon Greenstone 3 catch-up 964 Turn Phind applet into webswing Collection Building major moderate defect nobody assigned 2024-03-06T19:47:33+13:00 2024-03-19T21:36:01+13:00 "Webswing supports JApplets (swing) but not Applets (awt). So the steps involved were: 1. Port Phind Applet classes to JApplet. Located in the same folder, the new classes start with J (JPhind.java, JResultTitle.java, JResultBox.java, JResultDisplay.java and possibly more). And src/java/org/greenstone/gsdl3/service/PhindPhraseBrowse.java changed to refer to the new JPhind (JApplet) class now instead of the old Phind (Applet) class. The source code is still in src/java/org/greenstone/applet/phind and it's all still compiled into web/applet where there are still 3 jars: phind.jar, xml-apis.jar and xercesImpl.jar. 2. Get webswing to display the new JPhind JApplet. This required - modifying webswing.config.in to add in the new configuration for webswing phind. JPhind applet jars remain in web/applet, and this location is configured in webswing.config.in as a combination of setting the GS3 web location to be where we run JPhind webswing from and thus to look in its relative folder applet/*.jars as location for the JPhind webswing app(let)'s jar files. webswing.config.in main class to run is set to being an applet class (that of JPhind) and ""parameters"" instead of cmdline ""args"" are set up to be passed to JPhind because it's run as an applet instead of an application. - adding webswing-phind.xsl into default/transform/pages This was supposed to add custom parameters (""collection and ""library"") for the JPhind applet, but these weren't getting through to the JPhind java code, so the java code was modified to work out the collection and library parameters from the docBaseURL instead. Not the ideal solution, but provides a working version for now. - referring to webswing-phind.xsl in interfaceConfig.xml - modifying header.xsl to have the Phind browsing classifier tab link to the webswing JPhind URL at http://:/greenstone3//collection//page/phind (instead of the old phind applet URL) Notes: - Browsers don't support applets any more, but Java's appletviewer executable can be used to view (J)Applets and test them out - The JPhind JApplet has been rewritten to run as application, achieved by adding a main() method. But because of the extensive command line args involved, I prefer to run as applet, which are easier to preconfigure. - Applets use whatever status window the browser provides, if any, to display messages in the status bar with calls to showStatus(). When JPhind is run as an application, a custom status bar widget is added to mimic the status window browsers in the past would provide for applets. When run as webswing, status bar that is seen through the appletviewer is no longer visible when the same applet is run through the web browser, perhaps because Firefox doesn't provide a status window any more (browser don't even support applets any more). As useful information is displayed in JPhind's status bar, when it's run as an applet it also creates and writes to a custom status bar (besides still writing to any status bar a browser may provide). This ensures that the webswing JPhind JApplet has a working status bar. However, as a result, when viewing the JPhind through the appletviewer, two status bars now appear. Greenstone users are anyway less likely to use the appletviewer to interact with the Phind/JPhind applets, so this shouldn't matter. If it does ever begin to matter, the call to super.showStatus() can be commented out so that only the custom status bar is ever used. Changesets: - Unrelated general modification affecting webswing GLI too: https://trac.greenstone.org/changeset/38805 - Porting awt Applet to swing JApplet https://trac.greenstone.org/changeset/38794 to https://trac.greenstone.org/changeset/38796 https://trac.greenstone.org/changeset/38782 https://trac.greenstone.org/changeset/38799 https://trac.greenstone.org/changeset/38800 https://trac.greenstone.org/changeset/38813 https://trac.greenstone.org/changeset/38796 (src/java/org/greenstone/gsdl3/service/PhindPhraseBrowse.java refers to JPhind class now instead of Phind) - Getting the JApplet to run as a webswing app https://trac.greenstone.org/changeset/38806 https://trac.greenstone.org/changeset/38813 https://trac.greenstone.org/changeset/38814 https://trac.greenstone.org/changeset/38817 (JPhind for now works out collection and library parameters from docBase URL) " anupama