Changeset 31837 for main/trunk


Ignore:
Timestamp:
2017-08-01T20:57:03+12:00 (7 years ago)
Author:
ak19
Message:

Added instructions on how to successfully compile up openssl v 1.1.0f on Linux. It requires wget 1.19.x however. And it makes no difference to being required to add in no-check-certificate when downloading from an HTTPS URL.

File:
1 edited

Legend:

Unmodified
Added
Removed

  • main/trunk/greenstone2/build-src/packages/wget/README

    r31829 r31837  
    79798) 2017/07/27 - ak19 ([email protected]) - still using wget version 1.15, but now compiling wget up with OpenSSL support. Wget needs SSL support in order for it to access pages over HTTPS. In future, the web will be using https.
    8080
    81 We're now compiling up OpenSSL during the configuration phase since wget needs it to exist during its configure phase. We;re building OpenSSL statically, by setting the no-shared flag. The built OpenSSL gets put into gs2build/linux|darwin/openssl, containing lib, include and bin subfolders. When configuring wget, we build wget against our OpenSSl, and make and make install proceed as normal. Refer to gs2build/build-src/packages configure.
     81We're now compiling up OpenSSL during the configuration phase since wget needs it to exist during its configure phase. We're building OpenSSL statically, by setting the no-shared flag. The built OpenSSL gets put into gs2build/linux|darwin/openssl, containing lib, include and bin subfolders. When configuring wget, we build wget against our OpenSSl, and make and make install proceed as normal. Refer to gs2build/build-src/packages configure.
    8282
    83 We weren't compiling up wget statically before either, so we're still not doing so. To compile up wget (statically or not) with openssl, a helpful page was
     83We weren't compiling up wget statically before either, so we're still not doing so. But if that will be necessary in future, see the section on COMPILING WGET UP STATICALLY further below.
     84
     85To compile up wget (statically or not) with openssl, a helpful page was
    8486https://stackoverflow.com/questions/9817337/compiling-wget-with-static-linking-self-compiled-openssl-library-linking-issu
    8587Note, however, that since the CPPFLAGS and LDFLAGS are now set to point to our OpenSSL during the configure stage, the make command needn't additionally set them as well, contrary to the instruction for make on the stackoverflow page. So we just need to do the usual make, make install once the configure is done against OpenSSL.
    86 
    87 If compiling wget up statically, then, in the LDFLAGS prepended to wget's configure command, append -static. Further, the gcc command that gets run needs to have -lpthread in its library listing at the end. The order of the libraries listed also needs to change for static compilation to be successful:
    88 -lprce -lpthread -ldl <remaining -llibs>
    89 
    90 However, warnings appear when compiling wget statically, as it does not make sense to create some programs statically since they may be stuck including a local context (e.g. something related to DNS warnings in compiling up a previous component statically). Linking against some libraries to create a static binary may not make sense either. For instance -ldl, the dynamic loading or linking library, may not make sense if the binary created is static. This seems to imply that wget makes more sense if compiled up as a shared object, .so, than as a static one, .a.
    9188
    9289The existing version of wget, 1.15, works with HTTPS when compiled against OpenSSL. However, this version of the binary needs to be run with the --no-check-certificate flag on to access https pages without a security certificate.
    … …  
    9491e.g. ./wget --no-check-certificate http://englishhistory.net/tudor/citizens/
    9592
    96 The system wget on Ubuntu 16.04 is version 1.17.1 and does not require this flag. Pre-compiled windows binaries are available for version 1.11.4, so that may still require the flag. This will require further investigation. We'd like both unix and windows operating systems to behave similarly, ideally.
     93The system wget on Ubuntu 16.04 is version 1.17.1 and does not require this flag. Pre-compiled windows binaries are available for version 1.11.4 and also don't require the flag. We'd like both unix and windows operating systems to behave similarly, ideally. However, no matter which version of wget we compile up on Unix, 1.15, 1.17 or 1.19, and no matter which compiled version of openssl (1.0.2x or 1.1.0x) we've built it against, the wget binary we generate on unix always requires --no-check-certificate. So this will indeed be different from the wget 1.17+ binary we've downloaded for Windows.
    9794
    9895* http://nebm.ist.utl.pt/~glopes/wget/
    … …  
    126123
    127124
    128 WINDOWS
     125WINDOWS WGET BINARIES WITH OPENSSL SUPPORT
    129126Windows binaries for wget 1.7.11 and other versions, built with openSSL support, are at:
    130127https://eternallybored.org/misc/wget/
    … …  
    147144- https://stackoverflow.com/questions/14344921/wget-for-windows-7-trusted-source
    148145
     146
    149147COMBINING GREENSTONE's GPL with OpenSSL LICENSES
    150148OpenSSL is under a double license, see https://www.openssl.org/source/license.html
    … …  
    154152
    155153
     154TO COMPILE WGET STATICALLY
     155First refer to https://stackoverflow.com/questions/9817337/compiling-wget-with-static-linking-self-compiled-openssl-library-linking-issu
     156
     157If compiling wget up statically, then, in the LDFLAGS prepended to wget's configure command, append -static. Further, the gcc command that gets run needs to have -lpthread in its library listing at the end. The order of the libraries listed also needs to change for static compilation to be successful:
     158-lprce -lpthread -ldl <remaining -llibs>
     159
     160However, warnings appear when compiling wget statically, as it does not make sense to create some programs statically since they may be stuck including a local context (e.g. something related to DNS warnings in compiling up a previous component statically). Linking against some libraries to create a static binary may not make sense either. For instance -ldl, the dynamic loading or linking library, may not make sense if the binary created is static. This seems to imply that wget makes more sense if compiled up as a shared object, .so, than as a static one, .a.
     161
     162
     163TO COMPILE WGET WITH OPENSSL v 1.1.0f
     164At present, we're compiling Wget 1.17 with openSSL v1.0.2l.
     165
     166To compile with OpenSSL 1.1.0x, you'll need
     167* Wget v. 1.19
     168* -lpthread prepended to $LIBS.
     169
     170Note: Also need to update build-src/packages/Makefile.in's distclean command to remove the extra folder "share" and file "openssl.cnf.dist" generated when building openssl v 1.1.0f.
     171
     172So the wget compile command will look like:
     173
     174LIBS="-lpthread $LIBS" OPENSSL_CFLAGS="-I/Scratch/ak19/gs3-svn-13July2017/gs2build/build-src/packages/openssl/include" OPENSSL_LIBS="-L/Scratch/ak19/gs3-svn-13July2017/gs2build/build-src/packages/openssl/lib -lssl -lcrypto" ./configure --prefix=/Scratch/ak19/gs3-svn-13July2017/gs2build/build-src/packages/wget --with-ssl=openssl --with-openssl=auto --with-libssl-prefix="/Scratch/ak19/gs3-svn-13July2017/gs2build/build-src/packages/openssl" --bindir="/Scratch/ak19/gs3-svn-13July2017/gs2build/bin/linux" -disable-nls
     175
     176
     177
     178
    156179TO DO:
    157 - If I delete the gs2build/bin/linux/openssl folder, the built wget still works fine without it. Dr Bainbridge confirmed that this is because, wget is built against OpenSSL's static libraries and therefore no longer needs the OpenSSL stuff we build and have been putting into gs2build/bin/linux/openssl. So we no longer need to put the built OpenSSL there.
     180+ If I delete the gs2build/bin/linux/openssl folder, the built wget still works fine without it. Dr Bainbridge confirmed that this is because, wget is built against OpenSSL's static libraries and therefore no longer needs the OpenSSL stuff we build and have been putting into gs2build/bin/linux/openssl. So we no longer need to put the built OpenSSL there.
    158181
    159182- Add a tick box in GLI > File > Preferences for turning on No Check Certificate over https, this should then replace our wgetrc file and env variable set in GS2's setup.bash. By default leave this flag unticked, so downloading won't work over https. Need to store this user setting in GLI's config.xml. Ensure that when the download over https failed, it results in an error.
    … …  
    165188
    166189Check the warnings on windows. If it's no longer always warning, then do the stuff above on warning too, not just on error.
    167 
Note: See TracChangeset for help on using the changeset viewer.