Custom Query (424 matches)

GLI wasn't able to download from https URLs because there was no SSL support. Some other surrounding issues were identified and also needed fixing.

I didn't at first know there was no https support and that this was because wget hadn't been compiled with SSL. I only discovered this after the following two fixes, which allowed better debugging:

1. Adding shutdownhook for GS2 to ensure the GS2 server is stopped on Ctrl-C. Tested on Linux.

http://trac.greenstone.org/changeset/31813 - http://trac.greenstone.org/changeset/31815

2. http://trac.greenstone.org/changeset/31816

"Fix to getRedirectURL() being stuck forever if the proxy details are wrong. I still can't work out what my proxy settings should be to get downloading to work from a TSG-adminsitered machine, but at least now GLI doesn't hang requiring a ctrl-C to terminate it."

3. Compiling up wget on Unix with open SSL to support https URLs required upgrading our wget to a newer version (from 1.15 to 1.17.1) and also compiling up OpenSSL now, which we didn't do before. We also got a precompiled windows wget binary of 1.17.1, compiled with SSL support, upgrading from shipping a windows wget 1.11.x binary that didn't include SSL.

• GLI changes:

http://trac.greenstone.org/changeset/31820 - http://trac.greenstone.org/changeset/31823

• OpenSSL:

http://trac.greenstone.org/changeset/31824

http://trac.greenstone.org/changeset/31835 (for Mac)

• Wget upgrade and setting no-check-certificate using Wgetrc conf file, needed in newer wget versions compiled on Linux to be able to download from https urls that don't have valid site certificates:

http://trac.greenstone.org/changeset/31825 - http://trac.greenstone.org/changeset/31842 and http://trac.greenstone.org/changeset/31854

4. Fixes to get proxying to work and surrounding issues

Further modifications to GLI getRedirectURL()

• http://trac.greenstone.org/changeset/31843

New class URLConnectionManager to enable accessing web pages in "no-check-certificate" mode from Java code. Needed by getRedirectURL()

• getRedirectURL() phased out: http://trac.greenstone.org/changeset/31852

To get proxying to work on Windows and other issues:

http://trac.greenstone.org/changeset/31851

http://trac.greenstone.org/changeset/31853

http://trac.greenstone.org/changeset/31856, http://trac.greenstone.org/changeset/31857

5. no-check-certificate supported through GLI Preferences interface rather than wgetrc conf file:

http://trac.greenstone.org/changeset/31860

6. Changes to get helpful messages on connection failures to display.

Only for windows, proxy_on, proxy_port and proxy_host would be set (along with username and pwd), while on Linux none of these would be set as proxy settings were obtained by wget from env vars that were set up.

To get helpful messages to display, proxy_on, proxy_port and proxy_host needed to be setup regardless of the OS.

http://trac.greenstone.org/changeset/31861 - http://trac.greenstone.org/changeset/31865, http://trac.greenstone.org/changeset/31874 - http://trac.greenstone.org/changeset/31876

7. Shifting to Windows using proxy settings set up in the environment (as linux does) rather than passing proxy flags to wget.

http://trac.greenstone.org/changeset/31877, http://trac.greenstone.org/changeset/31878

8. Shifting from single to support for multiple proxy URLs: one each for HTTP, HTTPS and FTP.

http://trac.greenstone.org/changeset/31879 - http://trac.greenstone.org/changeset/31881

9. GLI informs user when existing proxy settings in environment (from before launching GLI) are overridden with custom proxy settings turned on and set up within GLI.

http://trac.greenstone.org/changeset/31882 - http://trac.greenstone.org/changeset/31884

Currently isn't working properly. It was taking no notice of the collection selection option, always searching all collections. Check and fix.

• Background information for implementing the OAI deletion policy

• the changesets that implemented OAI deletion policy on the perl side, the GS3 server side and GS2 server side.

Background info:

• ​http://openarchives.org/OAI/openarchivesprotocol.html#DeletedRecords

• See Kathy's email to Greenstone Team on 2/11/2016. "Re: Reminder: Re: Improvements for the OAI Greenstone Server".

Changesets:

1. PERL side (including preliminary cleanup):

http://trac.greenstone.org/changeset/31188 to http://trac.greenstone.org/changeset/31192

http://trac.greenstone.org/changeset/31210

http://trac.greenstone.org/changeset/31216 to http://trac.greenstone.org/changeset/31219

2. GS3 server side:

http://trac.greenstone.org/changeset/31229 and http://trac.greenstone.org/changeset/31230

http://trac.greenstone.org/changeset/31236 and http://trac.greenstone.org/changeset/31237

http://trac.greenstone.org/changeset/31241

3. GS2 server side:

http://trac.greenstone.org/changeset/31387 to http://trac.greenstone.org/changeset/31390

http://trac.greenstone.org/changeset/31394 and http://trac.greenstone.org/changeset/31395