Changeset 31825 for main/trunk

Timestamp:

27.07.2017 21:55:40 (2 years ago)

Author:

ak19

Message:

The linux system wget 1.17.1 and the Windows binary 1.17.1 both grab the tudors sample page over https without requiring a certificate check. Neither set the flag in the wgetrc config file as far as I can tell (the system wgetrc file does not exist on Windows, and does not set the check_certificate = off.) However, when I compile up 1.17.1 wget on linux, the generated wget binary requires me to still pass in the --no-check-certificate flag. Stackoverflow says I can set this property to off in the wgetrc conf file. But neither the linux system wget nor windows wget binary set the flag in the wgetrc config file as far as I can tell (the system wgetrc file does not exist on Windows, and does not set the check_certificate = off.) Next, no configure flag I set during configuring wget makes a difference to the behaviour of the built binary. I got a different version of openssl (close to the version used to build the win wget binary) and it makes no differenec either. These and other problems are documented in the README. Committing the README so I don't lose track of where I am with compiling wget.

Files:

• main/trunk/greenstone2/build-src/packages/wget/README (modified) (3 diffs)

main/trunk/greenstone2/build-src/packages/wget/README

@@ -45 +45 @@
 > 
 
+These newly added lines should come after the line:
+> CLEANFILES = *~ *.bak core core.[0-9]* build_info.c version.c
+
+and before the command to make 'all':
+> all: config.h
+>   $(MAKE) $(AM_MAKEFLAGS) all-am
+
+
 6) 2013/01/29 - davidb@cs.waikato.ac.nz added some #ifdef __ANDROID__ blocks into src/util.c to cope with the fact that this OS lacks localeconv()
 
@@ -56 +64 @@
 1489a1494
 > #endif
+
+[Now the #endif comes around line 1441 or 1459, immediately before:
+> initialized = true;
+>    }
+>  *sep = cached_sep;
+>  *grouping = cached_grouping;]
+
 
 7) 2014/10/13 - ak19
@@ -81 +96 @@
 The system wget on Ubuntu 16.04 is version 1.17.1 and does not require this flag. Pre-compiled windows binaries are available for version 1.11.4, so that may still require the flag. This will require further investigation. We'd like both unix and windows operating systems to behave similarly, ideally.
 
-
 * http://nebm.ist.utl.pt/~glopes/wget/
 Prebuilt Windows wget binaries (for 32 and 64 bit) version 1.11.4 that includes SSL support
 * http://gnuwin32.sourceforge.net/packages/wget.htm
 GNU's prebuilt Windows binaries of wget v 1.11.4. May not have been built with SSL support.
+
+The wget 1.11.4 we have, compiled without SSL does not work on https pages.
+Neither does the wget from http://nebm.ist.utl.pt/~glopes/wget/, even after including the --no-check-certificate flag in the wget command. So we'll need to upgrade the wget binary on Windows to a later version too.
+
+
+9) We're now shifting to wget-1.17.1 which is installed on Ubuntu 16.04 and which work on https urls without the --no-check-certificate flag being necessary.
+
+This way our perl code can launch wget as before, without always passing that additional flag. Hopefully the output in the Download pane will be the same so that the donwload parsing will work.
+
+LINUX CHANGES:
+- Grabbed wget-1.17.1.tar.gz from https://ftp.gnu.org/gnu/wget/
+- Made the modifications in steps 5 and 6 above and re-tarred as wget-1.17.1-gs.tar.gz, with corresponding changes in build-src/packages' configure, Makefile.in and Makefile
+- The configure step has now changed for wget v 1.17.1. Refer to build-src/packages/configure
+
+The configure step requires setting
+* --with-libssl-prefix to $bindir/openssl, so the wget build process can find openssl's include and lib folders. (Whereas the --with-ssl indicates what type of ssl we're using, which is openssl in our case.)
+* configuring had initially failed, reporting that OPENSSL_CFLAGS and OPENSSL_LIBS need to be set if not wanting to use whatever pkg-config may find. To set LIBS variables, use one of these forms: LIBS="-L/path/to/lib" or LIBS="/path/to/lib/lib.a" or LIBS="-lssl". To combine all three, separate with spaces. See http://trac.greenstone.org/changeset/30948 and https://github.com/tatsuhiro-t/spdylay/issues/43
+
+Can turn off requiring a certificate check for https URLs in wgetrc conf file, as explained here:
+https://superuser.com/questions/508696/wget-without-no-check-certificate
+However, the linux system wget and windows wget binary are not setting this in their wgetrc file, so how is the certificate check off for them by default?
+
+
+WINDOWS
+Windows binaries for wget 1.7.11 and other versions, built with openSSL support, are at:
+https://eternallybored.org/misc/wget/
+
+I downloaded the 32 bit version "wget-1.17.1-win32.zip" from there (at https://eternallybored.org/misc/wget/releases/old/wget-1.17.1-win32.zip)
+
+Unzipping wouldn't succeed, nor copying the zip's wget.exe directly, both producing a windows error message.
+To successfully extract: use 7zip to view the contents of the zip, then rename the wget.exe to wget.not, then copy out the wget.not file and rename it back.
+
+This version of wget on Windows 64 bit worked successfully to retrieve the https page of the Tudors site, and without the --no-check-certificate flag.
+
+# http://osxdaily.com/2012/05/22/install-wget-mac-os-x/
+# https://lists.gnu.org/archive/html/bug-wget/2014-12/msg00104.html
+
+2ND PROBLEM: OpenSSL License, see https://www.openssl.org/source/license.html
+
+QUESTION: If I delete the gs2build/bin/linux/openssl folder, the built wget does not seem to care. Is it finding something else or has it included the openssl somehow? How can I verify this?