Ticket #902 (closed defect: fixed)

Opened 18 months ago

Last modified 4 months ago

Getting incremental building to work from cmdline (again)

Reported by: ak19 Owned by: ak19
Priority: moderate Milestone: 3.08 Release
Component: Collection Building Severity: major
Keywords: Cc:


1. http://trac.greenstone.org/changeset/30499


Getting incremental-rebuild.pl to work with activate.pl. When incremental is on, the building dir IS the index dir. Need to handle this case, instead of clobbering index, always expecting to copy a distinct building dir to index.

( 3. http://trac.greenstone.org/changeset/30513 )

COSMETIC: Fixing tabbing issues

4. http://trac.greenstone.org/changeset/30517


Fixing incremental-rebuild when the database is gdbm. At this point (see buildcolutils.pm), the code needs to deactivate the collection before calling make_infodatabase(), since otherwise there's a lock on the gdbm database which prevents successful incremental-rebuild and activation.

Need to similarly handle a 2nd case of make_infodatabase() in buildcolutils.pm.

5. http://trac.greenstone.org/changeset/30520



Refactoring activate.pl into activate.pm (class, OOP) and activate.pl. Now buildcolutils.pm uses do_deactivate() from activate.pm.

Renaming activate.pm to servercontrol.pm

get_library_URL() called internal to servercontrol.pm even when using servercontrol in activate.pl. And more tidying up: removed lots of variables in OOP class servercontrol.pm that are no longer in use.

6. http://trac.greenstone.org/changeset/30526

servercontrol.pm used by activate.pl (and buildcolutils.pm) now uses wget instead of the perl libraries LWP, as these libraries may not be available in all perl installations, whereas GS binaries include wget.

7. http://trac.greenstone.org/changeset/30527

Fix for getting solr collections to work with incremental-buildcol.pl. Same as in step 1 but for solr collections too this time: needed to not delete the index folder when the index folder IS the buildng folder, as is the case when doing incremental-building.

Change History

Changed 18 months ago by ak19

  • status changed from new to closed
  • resolution set to fixed

Changed 18 months ago by ak19

8. http://trac.greenstone.org/changeset/30531

Fixing up the wget command issued by servercontrol.pm so that it will work on Windows too. Ampersands in URL to wget need to be escaped to work in a subshell. For linux, nesting the ampersand in single quotes suffices. For windows, preceding the ampersand with a hat sign seems to work from DOS prompt. But nesting the entire wget command inside double quotes seems to work on both OS systems.

Changed 18 months ago by ak19

9. http://trac.greenstone.org/changeset/30532

Filepath fix for spaces wasn't a fix but broke functioning of the setup file: would not retrieve tomcat port number after the intended fix.

10. http://trac.greenstone.org/changeset/30533

Adjustments for Windows OS to servercontrol.pm::config() which now uses wget.

11. http://trac.greenstone.org/changeset/30534

1. BUGFIX. When running full-rebuild on a solr collection when the GS3 server is not running, the solrserver.pm needs to start up and stop the tomcat server to get the solr server (off tomcat) to run and ingest the documents. However, once the tomcat is started up solrserver.pm, it does not return (solrserver.pm is stuck in a while loop waiting for further output from ant start). The problem was with JDK version 1.7.0_55, as using the JRE 1.7.0_72 included with the GS307 binary works and so too does JDK 1.8.*. Dr Bainbridge solved the problem more generally by making the while loop exit when the Build Successful message of the ant start command is encountered. Dr Bainbridge also found that the call to fork() (which I'd copied into solrserver from solrjettyserver off which it was based) was no longer applicable and that the child process may similarly have got stuck. The fixes will ensure that the code will work regardless of whether anyone uses it conjunction with the buggy Java 1.7.0_55.

2. With this commit I have tested ON WINDOWS the recent refactoring of activate.pl into servercontrol.pm (and changes to buildcolutils.pm) by testing full-rebuild and incremental-rebuild.pl against both an already running GS3 server and when the server was not running. Works.

Changed 4 months ago by ak19

12. http://trac.greenstone.org/changeset/31753 and http://trac.greenstone.org/changeset/31754

Modifications to:




Two fixes Kathy requested: 1. when running buildcol, ONLY deactivate a collection surrounding the lock-sensitive make_infodatabase() calls IF incremental. 2. Allow buildcol.pl to accept activate parameters like library_url (library_name and skipactivation). full-(re)build and incremental-(re)build scripts already accept additional parameters such as with -activate:skipactivation, but buildcol.pl doesn't work that way.

Changed 4 months ago by ak19

Note: See TracTickets for help on using tickets.