Changeset 34415 for main/trunk/gli/src/org/greenstone/gatherer/metadata/FilenameEncoding.java

Timestamp:

2020-09-22T00:57:33+12:00 (4 years ago)

Author:

ak19

Message:

Bugfix for slowdown when assigning meta to multiple gathered docs in GLI's Enrich pane. Tested on Windows. This is the simplest way I could think of to solve the problem: XMLParsing always resolves html entities (unless possibly when using the StAX parser, but that may not return the Document object as code expects). Entities start with ampersand and are resolved upon parsing, so too standalone ampersand signs. The earlier code, a bugfix for metadata not sticking to filenames/import folder structures containing non-ASCII or ampersands or plus signs, had caused the slow-down, as after each XML parse of the current metadata.xml file, the code would loop through each FileName element of the metadata.xml file and reintroduce the resolved html entities. The best and simplest solution that worked is simply to escape ampersands with %26 when writing out values for the FileName element and compare against filenames that have a similar substitution done. Still to test on Linux, but this reincorporates recent ideas for the bugfix that had worked on Linux (but then broke on Windows) so I feel somewhat confident that this commit is likely to largely work on Linux when I test it tomorrow.

File:

• main/trunk/gli/src/org/greenstone/gatherer/metadata/FilenameEncoding.java (modified) (8 diffs)

main/trunk/gli/src/org/greenstone/gatherer/metadata/FilenameEncoding.java

@@ -77 +77 @@
     public static final Pattern HEX_PATTERN = Pattern.compile("(&#x[0-9a-zA-Z]{1,4}+;)");
 
-    /** The hex entity version of the ampersand character.
+    /** The hex version of the ampersand character: previously hex entity (&#x26) now hex url encoded (%26).
      * We use this in place of the ampersand character in filenames in metadata.xml files to
      * preserve the reference to the literal ampersand in the real file name on the file system.
      */
-    public static final String HEX_ENTITY_AMPERSAND = FilenameEncoding.hexEntityForChar("&"); //"&";
+    public static final String HEX_AMPERSAND = "%26"; //= FilenameEncoding.hexEntityForChar("&"); //"&";
     
 
@@ -257 +257 @@
     
     /** URL encoded version of the byte codes of the given file's name */
-    public static String calcURLEncodedFilePath(File file) {    
-        if(!MULTIPLE_FILENAME_ENCODINGS_SUPPORTED) {
-            return file.getAbsolutePath();
-        }
-        else {
-            String filename = fileToURLEncoding(file);
-            return filename;
-        }
+    public static String calcURLEncodedFilePath(File file) {
+        return fileToURLEncoding(file);
     }
 
@@ -380 +374 @@
     // just return input filename param, but with any & in the filename replaced with its hex entity
         if(!MULTIPLE_FILENAME_ENCODINGS_SUPPORTED) {
-            // protect ampersands in filenames by converting it to its hex entity
             String filepath = file.getAbsolutePath();
-            filepath = filepath.replace("&", HEX_ENTITY_AMPERSAND);
             return filepath;
         }
@@ -430 +422 @@
             
             // Before proceeding, protect & in the filename too.
-            // &'s ASCII code is 36 in decimal, and 26 in hex, so replace with & (HEX_ENTITY_AMPERSAND)
+            // &'s ASCII code is 36 in decimal, and 26 in hex, so replace with & (HEX_AMPERSAND)
             // But dangerous to do simple replace if there are &#x...; entities in the filename already!
             // That is, we'll want to protect & by replacing with &'s hex value, but we don't want to replace the & in "&#x....;" with the same!
@@ -445 +437 @@
             //filename_url_encoded = filename_url_encoded.replace("%2B", "+"); // Don't do this, won't get regex escaped when converted back to a + by caller
             filename_url_encoded = filename_url_encoded.replace("%2B", "+"); // + signs are special, as they will need to be escaped since the caller wants the filename representing a regex
-            filename_url_encoded = filename_url_encoded.replace("%26", HEX_ENTITY_AMPERSAND); // convert URL encoding for ampersand into hex entity for ampersand
+            filename_url_encoded = filename_url_encoded.replace("%26", "&"); // now putting back ampersands too, instead of replacing with HEX_ENTITY_AMPERSAND (&)
         }
         catch (Exception e) {
@@ -544 +536 @@
         // just return input filename param, but with any & in the filename replaced with its hex entity
         if(!MULTIPLE_FILENAME_ENCODINGS_SUPPORTED) {
-            return filename.replace("&", HEX_ENTITY_AMPERSAND);
+            return filename; //return filename.replace("&", HEX_AMPERSAND);
         }
         
@@ -567 +559 @@
     public static String relativeFilenameToURLEncoding(String filename) {
         if(!MULTIPLE_FILENAME_ENCODINGS_SUPPORTED) { // on a UTF-8 file system, DO NOT do the stuff below, just return input param
-            return filename.replace("&", HEX_ENTITY_AMPERSAND);
+            return filename; // return filename.replace("&", HEX_AMPERSAND);
         }
         
@@ -580 +572 @@
     public static String filenameToURLEncodingWithPrefixRemoved(String filename, String removeFilePathPrefix) {
         if(!MULTIPLE_FILENAME_ENCODINGS_SUPPORTED) { // on a UTF-8 file system, DO NOT do the stuff below, just return input param
-            return filename.replace("&", HEX_ENTITY_AMPERSAND);
+            return filename; //return filename.replace("&", HEX_AMPERSAND);
         }