Changeset 29222

Show
Ignore:
Timestamp:
20.08.2014 22:03:28 (5 years ago)
Author:
ak19
Message:

SOLR related. TEMPORARY changes for the GS3 workshop. Owing to the change to Solr 4.7.2, solr collections can't re-build despite activate if the GS3 server is running because there is a conflict with the jetty server launched by buildcol and jetty finds a lock on the index. The result is that one can't search the solr index after such a rebuild. Dr Bainbridge suggested a temporary measure: instead of commandline building solr collections, we will now build them in GLI. GLI will build solr collections with activate on but, for solr collections alone, it will stop the GS3 server before a build and start it again upon completion. In future, we will get rid of the solr jetty server and just have solr running over HTTP from tomcat. The Java GS3 runtime code will have to access Solr as a HTTPSolrServer rather than as an EmbeddedSolrServer? at that point.

Location:
main/trunk/gli/src/org/greenstone/gatherer
Files:
4 modified

Legend:

Unmodified
Added
Removed
  • main/trunk/gli/src/org/greenstone/gatherer/Gatherer.java

    r26331 r29222  
    10411041    // used to send reload coll messages to the tomcat server 
    10421042    static public void configGS3Server(String site, String command) { 
     1043 
     1044        // Do not do configGS3Server for a GS3 solr collection. Not at present at least, when we're stopping 
     1045        // and starting the GS3 server from GLI for solr cols, since this function clears the solr.xml file. 
     1046        if(Gatherer.c_man.isSolrCollection()) { 
     1047        return; 
     1048        } 
     1049 
    10431050        if (Configuration.library_url == null){ 
    10441051            System.out.println("Error: you have not provided the Greenstone Library address.");   
  • main/trunk/gli/src/org/greenstone/gatherer/collection/CollectionManager.java

    r26573 r29222  
    7979import org.w3c.dom.*; 
    8080 
     81import org.greenstone.gatherer.util.GS3ServerThread; 
     82 
    8183/** This class manages many aspects of the collection, from its creation via scripts, data access via methods and its importing and building into the final collection. It is also responsible for firing appropriate event when significant changes have occured within the collection, and for creating a new metadata set manager as necessary. 
    8284 * @author John Thompson 
     
    215217    command_parts_list.add("-language"); 
    216218    command_parts_list.add(Configuration.getLanguage()); 
     219 
     220    // For now, for solr collections, we pass in -activate 
     221    // but GLI won't do moving building to index 
     222     
     223    if(CollectionManager.isSolrCollection()) { 
     224        command_parts_list.add("-activate"); 
     225    } 
    217226 
    218227    if(Gatherer.GS3) { 
     
    17341743             
    17351744        if ( CollectionDesignManager.buildcolWasFull() ) { 
    1736             if(installCollection()) { 
     1745 
     1746        // No installCollection() for GS3 solr collection: activate will take care of that 
     1747        // and no call to configGS3Server for solr collection 
     1748        if(CollectionManager.isSolrCollection()) { 
     1749         
     1750        DebugStream.println("Solr collection build complete: building already moved to index by activate.pl."); 
     1751         
     1752        // Finished building, 
     1753        // For now, for a GS3 solr collection, we'd have stopped the GS3 server before building 
     1754        // and will now need to restart it. 
     1755        GS3ServerThread thread = new GS3ServerThread(Configuration.gsdl3_src_path, "restart"); 
     1756        thread.start(); 
     1757 
     1758        // sleep before returning and doing the configGS3Server: give the GS3 server time to restart     
     1759        try { 
     1760            Thread.sleep(5000); 
     1761        } catch(Exception e) { 
     1762            e.printStackTrace(); 
     1763        } 
     1764         
     1765        } 
     1766         
     1767        else if(installCollection()) { 
    17371768            // If we have a local library running then ask it to add our newly create collection 
    17381769            if (LocalLibraryServer.isRunning() == true) { 
     
    19872018    } 
    19882019 
     2020    public static String getBuildType() { 
     2021    String buildType = (new CollectionMeta( CollectionDesignManager.collect_config.getBuildType() )).getValue(CollectionMeta.TEXT); 
     2022    return buildType; 
     2023    } 
     2024 
     2025    public static boolean isSolrCollection() { 
     2026    return (Gatherer.GS3 && CollectionManager.getBuildType().equals("solr")); 
     2027    } 
     2028 
    19892029 
    19902030    /** Install collection by moving its files from building to index after a successful build. 
     
    19992039    } 
    20002040 
    2001         
     2041    if(CollectionManager.isSolrCollection()) { 
     2042        // No installCollection() for GS3 solr collection: activate will take care of that 
     2043        DebugStream.println("Solr collection build complete: building already moved to index by activate.pl."); 
     2044 
     2045        // Finished building, 
     2046        // For now, for a GS3 solr collection, we'd have stopped the GS3 server before building 
     2047        // and will now need to restart it. 
     2048        GS3ServerThread thread = new GS3ServerThread(Configuration.gsdl3_src_path, "restart"); 
     2049        thread.start(); 
     2050 
     2051        return true; 
     2052    } 
     2053 
     2054 
    20022055    DebugStream.println("Build complete. Moving files."); 
    20032056 
  • main/trunk/gli/src/org/greenstone/gatherer/gui/CreatePane.java

    r20117 r29222  
    6363 
    6464import org.greenstone.gatherer.collection.CollectionManager; 
     65import org.greenstone.gatherer.util.GS3ServerThread; 
    6566 
    6667/** This class provides a GUI view showing some statistics on your current collection, and options for building it. As the collection is built this initial view is replaced with one showing progress bars and a message log of the creation process. This log can be later accessed via the options tree located in the center of the initial pane. This class is also responsible for creating the GShellProgressMonitors that are then attatched to external shell processes, and calling the methods in the CollectionManager for actually importing and building the collection. <br><BR> 
     
    694695            */ 
    695696                prepareForBuild(); 
     697 
     698        // Starting building, 
     699    // For now, for a GS3 solr collection, we need to stop the GS3 server before building 
     700    // and restart it after building is complete. Restart is done in CollectionManager.processComplete() 
     701    if(CollectionManager.isSolrCollection()) { 
     702        GS3ServerThread thread = new GS3ServerThread(Configuration.gsdl3_src_path, "stop"); 
     703        thread.start(); 
     704    } 
     705 
    696706                Gatherer.c_man.importCollection(); //This starts the building process. 
    697707            /* 
     
    860870        // Remove the collection lock. 
    861871        //Gatherer.g_man.lockCollection(false, false); 
     872 
     873        // Cancelling building: 
     874    // For now, for a GS3 solr collection, we need to stop the GS3 server before building 
     875    // and restart it after building is complete. If a cancel is pressed during build 
     876    // then we also restart the GS3 server 
     877    if(CollectionManager.isSolrCollection()) { 
     878        GS3ServerThread thread = new GS3ServerThread(Configuration.gsdl3_src_path, "restart"); 
     879        thread.start(); 
     880    } 
     881 
     882 
    862883    } 
    863884    } 
  • main/trunk/gli/src/org/greenstone/gatherer/util/GS3ServerThread.java

    r24388 r29222  
    5555    try 
    5656        { 
     57        ///System.err.println("**** GS3 server : " + _ant_command); 
     58 
    5759        String shellCommand = null; 
    5860        Process p = null;