Changeset 33737 for main/trunk/gli/src/org/greenstone/gatherer/metadata/MetadataXMLFile.java

Timestamp:

2019-12-02T20:03:57+13:00 (4 years ago)

Author:

ak19

Message:

A larger fix but not complete fix to the problem of attaching and retaining file level assigned meta to filenames containing non-ASCII characters. 1. Committing intermediate version of bugfix containing the idea suggested by Kathy to reuse the steps in fileToURLEncoding(File) for a String parameter as she felt that since the String represents a filename, a URI object should be instantiable on a String. Worked with some massaging. Can't yet get the new fileToURLEncoding(String) to work by calling fileToURLEncoding(File). So am committing the version of fileToURLEncoding(String) that is largely a copy of fileToURLEncoding(File), until I can get the simpler variant working. 2. The new method is called after each successful parseXML call from MetadataXMLFile, so that the decoded entities resulting from parseXML() are reencoded in the DOM. This allows us to retain the correct filenames originally mentioned in metadata.xml files, do proper comparisons against them to attach/modify further metdata and so that the correct values get written out again into metadata.xml files. 3. Still want to get simpler version of fileToURLEncoding(String) to work that reuses fileToURLEncoding(File). 4. Want to get ampersand and plus signs in filenames to work (+ signs in filenames are lost when filenames are converted to URL). 5. Still need to investigate the missing ex. metadata for filenames containing non-ASCII.

File:

• main/trunk/gli/src/org/greenstone/gatherer/metadata/MetadataXMLFile.java (modified) (6 diffs)

main/trunk/gli/src/org/greenstone/gatherer/metadata/MetadataXMLFile.java

@@ -81 +81 @@
         loaded_file = this;
         loaded_file_document = document;
+        reEncodeFilenamesInMetadataXML(loaded_file_document);
+        
     }
 
@@ -234 +236 @@
         loaded_file = this;
         loaded_file_document = document;
+        
+        reEncodeFilenamesInMetadataXML(loaded_file_document);
     }
 
@@ -395 +399 @@
         loaded_file = this;
         loaded_file_document = document;
+        
+        reEncodeFilenamesInMetadataXML(loaded_file_document);
     }
 
@@ -512 +518 @@
         loaded_file = this;
         loaded_file_document = document;
+        
+        reEncodeFilenamesInMetadataXML(loaded_file_document);
     }
 
@@ -629 +637 @@
         XMLTools.writeXMLFile(loaded_file, loaded_file_document, nonEscapingElements);
     
+    /*  // DEBUGGING:
         Document doc = XMLTools.parseXMLFile(loaded_file);
-        //System.err.println("AT END saveLoadedFile(), PARSED loaded_file contains:\n" +  XMLTools.elementToString(doc.getDocumentElement(), true));
-        
+        System.err.println("AT END saveLoadedFile(), PARSED loaded_file contains:\n" +  XMLTools.elementToString(doc.getDocumentElement(), true));
+        
+        reEncodeFilenamesInMetadataXML(doc);
+        System.err.println("AT END saveLoadedFile(), RE-ENCODED loaded_file contains:\n" +  XMLTools.elementToString(doc.getDocumentElement(), true));
+    */  
         loaded_file_changed = false;
     }
+    
+    //System.err.println("@@@@ END of saveLoadedFile()");
+    //Utility.printCaller();
     }
 
-
+    /** 
+     * parseXML(metadata.xml) has the side-effect of resolving html entities.
+     * Although this is not done by the GLIEntityResolver usage in parseXML(), something
+     * in parseXML() is resolving the html entities, including those used in carefully
+     * html-entity-escaped filenames.
+     * We need to get the filenames in the DOM correct after parsing a metadata.xml file
+     * into memory, so that we have the correct filenames and so that we'll write it out correctly.
+     * Therefore, always call this method after a successful parseXML() call on a metadata.xml.
+     * @param doc is the Document where the FILENAME_ELEMENTs need to be re-encoded.
+     * At the end of this function, the doc will be modified with the re-encoded filenames.
+     * 
+    */
+    static private void reEncodeFilenamesInMetadataXML(Document doc) {
+        if(!FilenameEncoding.MULTIPLE_FILENAME_ENCODINGS_SUPPORTED) {
+            return;
+        }
+        
+        //System.err.println("PARSED loaded_file contains:\n" +  XMLTools.elementToString(doc.getDocumentElement(), true));
+        
+        // Read all the FileSet elements in the file
+        NodeList fileset_elements_nodelist = doc.getElementsByTagName(FILESET_ELEMENT);
+        for (int i = 0; i < fileset_elements_nodelist.getLength(); i++) {
+            Element current_fileset_element = (Element) fileset_elements_nodelist.item(i);
+
+            // get the value of all FileName elements
+            NodeList filename_elements_nodelist = current_fileset_element.getElementsByTagName(FILENAME_ELEMENT);
+            for (int j = 0; j < filename_elements_nodelist.getLength(); j++) {
+                Element filename_element = (Element) filename_elements_nodelist.item(j);
+                String filename = XMLTools.getElementTextValue(filename_element);
+                if(!filename.equals(DIRECTORY_FILENAME)) {
+                    //System.err.println("Filename before reencoding was: " + filename);
+                    // reencode filename                    
+                    // can't convert to URI with backslash-escaped chars (backslash used in regexed filename are illegal in URI object
+                    // created by filenameToURLEncoding).
+                    String encoded_filename = filename.replace("\\", "%5C"); 
+                    encoded_filename = FilenameEncoding.filenameToURLEncoding(encoded_filename);
+                    // escape chars for regex again
+                    encoded_filename = encoded_filename.replace("%5C", "\\");
+                    XMLTools.setElementTextValue(filename_element, encoded_filename);
+                    //System.err.println("Filename after reencoding was: " + encoded_filename);
+                }
+            }
+        }       
+        //System.err.println("RE-ENCODED loaded_file contains:\n" +  XMLTools.elementToString(doc.getDocumentElement(), true));
+    }
+    
     /**
      * Every metadata.xml file must be skimmed when a collection is opened, for three very important reasons:
@@ -655 +715 @@
         return;
     }
-
+    // Always call this method after calling parseXMLFile
+    reEncodeFilenamesInMetadataXML(document);
+    
     // Read all the Metadata elements in the file
     HashMap target_metadata_element_name_attrs_cache = new HashMap();