activate.pl to move building to index after getting server to release coll

[ak19]

The new activate.pl script:

The script moves a collection's building folder (or its contents) to the index folder after a build cycle has completed. If the included GS2 or GS3 web server is running and is a persistent server (or if the library_URL provided as parameter to this script is of a running web server), then this script first deactivates the collection before the move operation and subsequently activates it again.

The relevant changesets are:

1. http://trac.greenstone.org/changeset/25549

Added target to GS3's ant build.xml file to print out default library servlet URL so that activate.pl can use it.

Files: main/trunk/greenstone3/build.xml (1 diff)

2. http://trac.greenstone.org/changeset/25554

http://trac.greenstone.org/changeset/25572 (correction to commit of 25554)

Added mv_dir_contents subroutine (which is called from activate.pl) for moving the contents of src dir into target dir, so these can exist alongside of non-identically named items already existing in target dir, rather replacing target dir with src dir.

The newly added mv_dir_contents subroutine was not complete: it did not move the directory contents of subdirectories which I thought was implicit. Added explicit recursion on subfolders and handling of special cases where folders are empty.

File: main/trunk/greenstone2/perllib/util.pm

3. http://trac.greenstone.org/changeset/25559

http://trac.greenstone.org/changeset/25560

1. Dr Bainbridge fixed a bug in pingaction where it returned the message that the ping didn't succeed for any collection, even when the collection existed. 2. The pingaction no longer requires a collection argument. In the case where no collection is specified, pingaction will always return true when you call a=ping on a running GS server.

FILES: main/trunk/greenstone2/runtime-src/src/recpt/pingaction.cpp

Dr Bainbridge has introduced the isPersistentAction (add the a=is-persistent to the library url). It is true for server.exe and when using mod_gsdl, but false for library.cgi which uses the apache web server (when not using mod_gsdl).

FILES: 2 added, 7 modified

LOCATION: main/trunk/greenstone2/runtime-src/src

recpt/Makefile.in (2 diffs)

recpt/cgiwrapper.cpp (2 diffs)

recpt/cgiwrapper.h (1 diff)

recpt/ispersistentaction.cpp

recpt/ispersistentaction.h

recpt/librarymain.cpp (2 diffs)

recpt/mod_gsdl.cpp (2 diffs)

recpt/win32.mak (2 diffs)

w32server/cgiwrapper.cpp (3 diffs)

4. http://trac.greenstone.org/changeset/25571

Added a ping action (subaction to SystemAction) to mirror GS2's updated ability to ping the server as is needed for activate.pl. (GS2 also allows pinging a collection, but that's not been ported to GS3 yet). Call with ?a=s&sa=ping for systemaction subaction ping.

Location: main/trunk/greenstone3/src/java/org/greenstone/gsdl3

Files: 3 modified

action/SystemAction.java (1 diff)

core/MessageRouter.java (1 diff)

util/GSXML.java (1 diff)

5. http://trac.greenstone.org/changeset/25573

Adding in the new activate.pl script to be called at the end of the build process. It moves building to index after first deactivating the collection on the server (if this is running and a persistent server). After the move operation, it then activates the collection on the server again, if the server is still running and is a persistent server.

buildcol.pl is modified to allow buildcol.pl to be called with the new -activate flag which will then run activate.pl at the end of buildcol.

full-rebuild.pl no longer does the moving building to index itself, but calls activate.pl which will also ensure that collections are deactivated on the server first and reactivated at the end if necessary.

Location: main/trunk/greenstone2

Files: 1 added, 3 modified

bin/script/activate.pl

bin/script/buildcol.pl (3 diffs)

bin/script/full-rebuild.pl (8 diffs)

perllib/strings.properties (1 diff)

[ak19]

Further changes:

1. http://trac.greenstone.org/changeset/25577

and http://trac.greenstone.org/changeset/25578

Changed: main/trunk/greenstone2/perllib/util.pm

Moved some more general methods from activate.pl to util.pm and in the recently added sub mv_dir_contents, use rmdir to remove empty dirs rather than using rm_r.

and

Modifying recently introduced mv_dir_contents method again: recursion call to mv_dir_contents on any directory need only happen if the same directory (with the same sub path) also exists in the target directory. For regular files and any directories not duplicated in target, a straightforward call to mv is sufficient. Reducing the amount of recursion would make it more efficient.

2. http://trac.greenstone.org/changeset/25579

1. Introduced the verbosity flag into activate.pl to print out debugging statements depending on verbosity, and this is passed in from buildcol.pl and full-rebuild.pl too now. 2. Cleaned up some debugging code.

In greenstone2 code:

bin/script/activate.pl (15 diffs)

bin/script/buildcol.pl (1 diff)

bin/script/full-rebuild.pl (4 diffs)

perllib/util.pm (4 diffs)

3. http://trac.greenstone.org/changeset/25580

Changes to GS3's SystemAction.java now supports pinging on a collection with ?a=s&sa=ping&st=collection&sn=colName

So now GS3 supports regular ping and pinging on a collection.

Location: main/trunk/greenstone3/src/java/org/greenstone/gsdl3

action/SystemAction.java (1 diff)

core/MessageRouter.java (3 diffs)

4. http://trac.greenstone.org/changeset/25581

activate.pl now has both ping_library and ping_library_collection subroutines. The second one is called to check whether a collection needs to be deactivated or activated (in case it's already active).

main/trunk/greenstone2/bin/script/activate.pl (5 diffs)

Still need to test against GS3 on Windows again with today's changes.

Need to test all of activate.pl (and buildcol and full-rebuild) on Linux.

[ak19]

Test with (when you have a collection Pinky):

ON GS2:

1. perl -S import.pl -keepold mycol

perl -S buildcol.pl -keepold mycol

perl -S activate.pl -verbosity 3 -keepold mycol

(Also try -verbosity 4)

2. Now that it's imported, test buildcol.pl with the new -activate flag:

perl -S buildcol.pl -activate -removeold mycol

3. Test full-rebuild.pl (which forces -removeold)

perl -S full-rebuild.pl -verbosity 2 mycol

ON GS3:

1. perl -S import.pl -collectdir -keepold "full\path\to\GS3\web\sites\localsite\collect" -site "localsite" mycol

perl -S buildcol.pl -collectdir -keepold "full\path\to\GS3\web\sites\localsite\collect" -site "localsite" mycol

perl -S activate.pl -collectdir -verbosity 2 -keepold "full\path\to\GS3\web\sites\localsite\collect" -site "localsite" mycol

2. Full-rebuild always does a removeold:

perl -S full-rebuild.pl -collectdir "full\path\to\GS3\web\sites\localsite\collect" -site "localsite" mycol

3. perl -S buildcol.pl -activate -removeold -verbosity 3 -collectdir "full\path\to\GS3\web\sites\localsite\collect" -site "localsite" mycol

[ak19]

1. http://trac.greenstone.org/changeset/25596

Only check whether the collection on the server is active after the is-persistent check, because it's not necessary to check whether a collection is active on a server that's not persistent, as there's no point in attempting to deactivate or activate cols on servers that aren't persistent. Files:

main/trunk/greenstone2/bin/script/activate.pl (2 diffs)

2. http://trac.greenstone.org/changeset/25597

Setting svn:executable property to * for activate.pl to make the script executable by default

3. http://trac.greenstone.org/changeset/25598

When testing activate.pl with -keepold on Linux, it could not move the contents of the assoc subfolder of building and failed in the call to util::mv in util::mv_dir_contents. While the rename step was succeeding, the oriignal file remained in building, and Sam suspected the errors may be owing to the files being hardlinked. He suggested copying and deleting since renaming (and moving) these failed. Copying was no longer necessary, since the call rename() from mv() returned true for success. But deleting the file from building was necessary just as Sam said.

main/trunk/greenstone2/perllib/util.pm (3 diffs)