Filename encodings again
|Reported by:||ak19||Owned by:||nobody|
- Dr Bainbridge committed some important changes to Greenstone's Perl code, that makes the strings in Perl unicode-aware for regex matching.
- The above required changes in GLI too:
GLI now has a gs.FilenameEncoding metadata field which appears like all the others in GLI's EnrichPane, but is unique in that this metadata (once set, changed or removed) must be applied to the affected filenames in the Collection Tree.
More importantly, the changes made for this are to allow GLI's java code to interact with the recent changes to Perl where strings were made unicode-aware (for proper regex matching) but which required other changes elsewhere. To still support filenames with different encodings Perl used URL encoded versions of filenames representing characters' code point values in URL encoding. This required that GLI write out URL encoded filenames to the metadata.xml files that are associated with each folder level of a collection, so that Perl can read them. In this way, they can both speak of the same filenames.
Note that these changes to GLI only work on unicode 16 (such as latin-1), non-UTF8 systems. The latter is a requirement since Java uses the filesystem encoding from startup. If it is UTF8, non-recognised characters are replaced by the invalid char for UTF8. This process being destructive, we can't get the original filenames' bytecodes back. The changes made to GLI will work on Windows which is UTF-16 (windows codepage 1252), presumably also Macs (some kind of UTF-16) and also works on Native Latin 1 Linux systems. UTF-8 Linux systems need to be reconfigured to Native Latin-1, or if not installed, an administrator can install it easily.
Revision 23433 - GLI files changed :
+ ADDED: src\org\greenstone\gatherer\metadata\FilenameEncoding.java
+ classes\dictionary.properties + metadata\greenstone.mds + src\org\greenstone\gatherer\Gatherer.java + src\org\greenstone\gatherer\collection\CollectionManager.java + src\org\greenstone\gatherer\collection\CollectionTreeNode.java + src\org\greenstone\gatherer\file\FileNode.java + src\org\greenstone\gatherer\gui\EnrichPane.java + src\org\greenstone\gatherer\gui\GUIManager.java + src\org\greenstone\gatherer\metadata\MetadataValueTableModel.java + src\org\greenstone\gatherer\metadata\MetadataXMLFile.java + src\org\greenstone\gatherer\metadata\MetadataXMLFileManager.java + src\org\greenstone\gatherer\util\SynchronizedTreeModelTools.java