Show
Ignore:
Timestamp:
27.07.2017 21:55:40 (2 years ago)
Author:
ak19
Message:

The linux system wget 1.17.1 and the Windows binary 1.17.1 both grab the tudors sample page over https without requiring a certificate check. Neither set the flag in the wgetrc config file as far as I can tell (the system wgetrc file does not exist on Windows, and does not set the check_certificate = off.) However, when I compile up 1.17.1 wget on linux, the generated wget binary requires me to still pass in the --no-check-certificate flag. Stackoverflow says I can set this property to off in the wgetrc conf file. But neither the linux system wget nor windows wget binary set the flag in the wgetrc config file as far as I can tell (the system wgetrc file does not exist on Windows, and does not set the check_certificate = off.) Next, no configure flag I set during configuring wget makes a difference to the behaviour of the built binary. I got a different version of openssl (close to the version used to build the win wget binary) and it makes no differenec either. These and other problems are documented in the README. Committing the README so I don't lose track of where I am with compiling wget.

Files:
1 modified

Legend:

Unmodified
Added
Removed
  • main/trunk/greenstone2/build-src/packages/wget/README

    r31824 r31825  
    4545>  
    4646 
     47These newly added lines should come after the line: 
     48> CLEANFILES = *~ *.bak core core.[0-9]* build_info.c version.c 
     49 
     50and before the command to make 'all': 
     51> all: config.h 
     52>   $(MAKE) $(AM_MAKEFLAGS) all-am 
     53 
     54 
    47556) 2013/01/29 - davidb@cs.waikato.ac.nz added some #ifdef __ANDROID__ blocks into src/util.c to cope with the fact that this OS lacks localeconv() 
    4856 
     
    56641489a1494 
    5765> #endif 
     66 
     67[Now the #endif comes around line 1441 or 1459, immediately before: 
     68> initialized = true; 
     69>    } 
     70>  *sep = cached_sep; 
     71>  *grouping = cached_grouping;] 
     72 
    5873 
    59747) 2014/10/13 - ak19 
     
    8196The system wget on Ubuntu 16.04 is version 1.17.1 and does not require this flag. Pre-compiled windows binaries are available for version 1.11.4, so that may still require the flag. This will require further investigation. We'd like both unix and windows operating systems to behave similarly, ideally. 
    8297 
    83  
    8498* http://nebm.ist.utl.pt/~glopes/wget/ 
    8599Prebuilt Windows wget binaries (for 32 and 64 bit) version 1.11.4 that includes SSL support 
    86100* http://gnuwin32.sourceforge.net/packages/wget.htm 
    87101GNU's prebuilt Windows binaries of wget v 1.11.4. May not have been built with SSL support. 
     102 
     103The wget 1.11.4 we have, compiled without SSL does not work on https pages. 
     104Neither does the wget from http://nebm.ist.utl.pt/~glopes/wget/, even after including the --no-check-certificate flag in the wget command. So we'll need to upgrade the wget binary on Windows to a later version too. 
     105 
     106 
     1079) We're now shifting to wget-1.17.1 which is installed on Ubuntu 16.04 and which work on https urls without the --no-check-certificate flag being necessary. 
     108 
     109This way our perl code can launch wget as before, without always passing that additional flag. Hopefully the output in the Download pane will be the same so that the donwload parsing will work. 
     110 
     111LINUX CHANGES: 
     112- Grabbed wget-1.17.1.tar.gz from https://ftp.gnu.org/gnu/wget/ 
     113- Made the modifications in steps 5 and 6 above and re-tarred as wget-1.17.1-gs.tar.gz, with corresponding changes in build-src/packages' configure, Makefile.in and Makefile 
     114- The configure step has now changed for wget v 1.17.1. Refer to build-src/packages/configure 
     115 
     116The configure step requires setting 
     117* --with-libssl-prefix to $bindir/openssl, so the wget build process can find openssl's include and lib folders. (Whereas the --with-ssl indicates what type of ssl we're using, which is openssl in our case.) 
     118* configuring had initially failed, reporting that OPENSSL_CFLAGS and OPENSSL_LIBS need to be set if not wanting to use whatever pkg-config may find. To set LIBS variables, use one of these forms: LIBS="-L/path/to/lib" or LIBS="/path/to/lib/lib.a" or LIBS="-lssl". To combine all three, separate with spaces. See http://trac.greenstone.org/changeset/30948 and https://github.com/tatsuhiro-t/spdylay/issues/43 
     119 
     120Can turn off requiring a certificate check for https URLs in wgetrc conf file, as explained here: 
     121https://superuser.com/questions/508696/wget-without-no-check-certificate 
     122However, the linux system wget and windows wget binary are not setting this in their wgetrc file, so how is the certificate check off for them by default? 
     123 
     124 
     125WINDOWS 
     126Windows binaries for wget 1.7.11 and other versions, built with openSSL support, are at: 
     127https://eternallybored.org/misc/wget/ 
     128 
     129I downloaded the 32 bit version "wget-1.17.1-win32.zip" from there (at https://eternallybored.org/misc/wget/releases/old/wget-1.17.1-win32.zip) 
     130 
     131Unzipping wouldn't succeed, nor copying the zip's wget.exe directly, both producing a windows error message. 
     132To successfully extract: use 7zip to view the contents of the zip, then rename the wget.exe to wget.not, then copy out the wget.not file and rename it back. 
     133 
     134This version of wget on Windows 64 bit worked successfully to retrieve the https page of the Tudors site, and without the --no-check-certificate flag. 
     135 
     136# http://osxdaily.com/2012/05/22/install-wget-mac-os-x/ 
     137# https://lists.gnu.org/archive/html/bug-wget/2014-12/msg00104.html 
     138 
     1392ND PROBLEM: OpenSSL License, see https://www.openssl.org/source/license.html 
     140 
     141QUESTION: If I delete the gs2build/bin/linux/openssl folder, the built wget does not seem to care. Is it finding something else or has it included the openssl somehow? How can I verify this?