Changeset 31824

Show
Ignore:
Timestamp:
27.07.2017 18:40:26 (22 months ago)
Author:
ak19
Message:

We now compile up OpenSSL, since wget needs SSL support to download https pages. wget is now compiled up OpenSSL, achieved by changes during wget's configure phase. OpenSSL is configured and compiled up during GS' configure phase. Details are in the updated README at wget level.

Location:
main/trunk/greenstone2/build-src/packages
Files:
2 added
2 modified

Legend:

Unmodified
Added
Removed
  • main/trunk/greenstone2/build-src/packages/configure

    r30896 r31824  
    139139 
    140140 
     141 
     142# The way common-src compiles up expat during the configure stage itself, 
     143# configure openssl (needs to be done before wget) 
     144# and compile openssl during the configure stage itself. 
     145# Needed by wget for https support 
     146 
     147echo "" 
     148echo "Configuring openssl" 
     149echo "" 
     150cd "$PACKAGES/openssl" 
     151openssl_pkgname=openssl-1.0.2l 
     152if test ! -d $openssl_pkgname ; 
     153then 
     154  gzip -dc $openssl_pkgname.tar.gz | tar -xf - 
     155fi 
     156cd openssl-1.0.2l 
     157 
     158# configure 
     159# see the INSTALL file for OpenSSL to see config options for openssl 
     160# --prefix indicates where the generated lib, include and bin folders will go 
     161# --openssldir will contain the manuals and other non-essentials 
     162# we'll put them into $bindir/openssl for wget to use 
     163echo ./config --prefix=$bindir/openssl --openssldir=$PACKAGES/openssl no-shared 
     164./config --prefix=$bindir/openssl --openssldir=$PACKAGES/openssl no-shared 
     165# make 
     166echo make 
     167make 
     168# make install 
     169echo make install 
     170make install 
     171 
     172# now move the include, lib and bin folders to the correct place 
     173 
    141174# configure the wvWare Word document converter 
    142175echo "" 
     
    227260fi 
    228261 
     262#./configure $CACHE_FILE --prefix=$PACKAGES/wget --without-ssl --bindir="$bindir" -disable-nls $HOSTTARGETFLAGS $wget_extraargs 
     263 
     264# We weren't compiling up wget statically before either 
     265# However, to compile up wget (statically or not) with openssl, see 
     266# https://stackoverflow.com/questions/9817337/compiling-wget-with-static-linking-self-compiled-openssl-library-linking-issu 
     267# Note: if the cppflags and ldflags are set during the configure stage, as we now do, 
     268# then the make command needn't additionally set them as well. So just make, make install then works  
     269echo cd "$PACKAGES/wget/wget-1.15-gs" 
     270echo CPPFLAGS="-I/$bindir/openssl/include" LDFLAGS="-L/$bindir/openssl/lib" ./configure $CACHE_FILE --prefix=$PACKAGES/wget --with-ssl=openssl --bindir="$bindir" -disable-nls $HOSTTARGETFLAGS $wget_extraargs 
    229271cd "$PACKAGES/wget/wget-1.15-gs" && \ 
    230 #./configure $CACHE_FILE --prefix=$PACKAGES/wget --with-ssl=openssl --bindir="$bindir" -disable-nls $HOSTTARGETFLAGS $wget_extraargs 
    231 ./configure $CACHE_FILE --prefix=$PACKAGES/wget --without-ssl --bindir="$bindir" -disable-nls $HOSTTARGETFLAGS $wget_extraargs 
     272CPPFLAGS="-I/$bindir/openssl/include" LDFLAGS="-L/$bindir/openssl/lib" ./configure $CACHE_FILE --prefix=$PACKAGES/wget --with-ssl=openssl --bindir="$bindir" -disable-nls $HOSTTARGETFLAGS $wget_extraargs 
     273 
     274 
     275## 
     276##CPPFLAGS="-I/Scratch/ak19/gs3-svn-13July2017/gs2build/bin/linux/openssl/include" LDFLAGS="-L/Scratch/ak19/gs3-svn-13July2017/gs2build/bin/linux/openssl/lib" ./configure --prefix=/Scratch/ak19/gs3-svn-13July2017/gs2build/build-src/packages/wget --with-ssl=openssl --bindir="/Scratch/ak19/gs3-svn-13July2017/gs2build/bin/linux" -disable-nls 
     277## 
     278##make CPPFLAGS="-I/Scratch/ak19/gs3-svn-13July2017/gs2build/bin/linux/openssl/include" LDFLAGS="-L/Scratch/ak19/gs3-svn-13July2017/gs2build/bin/linux/openssl/lib -static" 
    232279 
    233280if test x$? = x1 ; 
  • main/trunk/greenstone2/build-src/packages/wget/README

    r29366 r31824  
    59597) 2014/10/13 - ak19 
    6060Moved to wget version 1.15.  
    61 Only the changes numbered 5 and 6 above have been ported into it, following Dr Bainbridge's instructions, as there were both changes made to the previous wget version Greenstone used (1.13.4). 
    62 The wget tar file name (wget-1.15-gs) now indicates the version number and that it has been modified by Greenstone, so the Makefile and configure file in build-src/packages/ have been updated to reflect this. 
     61Only the changes numbered 5 and 6 above have been ported into it, following Dr Bainbridge's instructions, as they were both changes made to the previous wget version Greenstone used (1.13.4). 
     62The wget tar file name (wget-1.15-gs) now indicates the version number and that it has been modified for Greenstone, so the Makefile and configure file in build-src/packages/ have been updated to reflect this. 
    6363 
     648) 2017/07/27 - ak19 (anupama.krishnan@waikato.ac.nz) - still using wget version 1.15, but now compiling wget up with OpenSSL support. Wget needs SSL support in order for it to access pages over HTTPS. In future, the web will be using https. 
     65 
     66We're now compiling up OpenSSL during the configuration phase since wget needs it to exist during its configure phase. We;re building OpenSSL statically, by setting the no-shared flag. The built OpenSSL gets put into gs2build/linux|darwin/openssl, containing lib, include and bin subfolders. When configuring wget, we build wget against our OpenSSl, and make and make install proceed as normal. Refer to gs2build/build-src/packages configure. 
     67 
     68We weren't compiling up wget statically before either, so we're still not doing so. To compile up wget (statically or not) with openssl, a helpful page was 
     69https://stackoverflow.com/questions/9817337/compiling-wget-with-static-linking-self-compiled-openssl-library-linking-issu 
     70Note, however, that since the CPPFLAGS and LDFLAGS are now set to point to our OpenSSL during the configure stage, the make command needn't additionally set them as well, contrary to the instruction for make on the stackoverflow page. So we just need to do the usual make, make install once the configure is done against OpenSSL. 
     71 
     72If compiling wget up statically, then, in the LDFLAGS prepended to wget's configure command, append -static. Further, the gcc command that gets run needs to have -lpthread in its library listing at the end. The order of the libraries listed also needs to change for static compilation to be successful: 
     73-lprce -lpthread -ldl <remaining -llibs> 
     74 
     75However, warnings appear when compiling wget statically, as it does not make sense to create some programs statically since they may be stuck including a local context (e.g. something related to DNS warnings in compiling up a previous component statically). Linking against some libraries to create a static binary may not make sense either. For instance -ldl, the dynamic loading or linking library, may not make sense if the binary created is static. This seems to imply that wget makes more sense if compiled up as a shared object, .so, than as a static one, .a. 
     76 
     77The existing version of wget, 1.15, works with HTTPS when compiled against OpenSSL. However, this version of the binary needs to be run with the --no-check-certificate flag on to access https pages without a security certificate. 
     78 
     79e.g. ./wget --no-check-certificate http://englishhistory.net/tudor/citizens/ 
     80 
     81The system wget on Ubuntu 16.04 is version 1.17.1 and does not require this flag. Pre-compiled windows binaries are available for version 1.11.4, so that may still require the flag. This will require further investigation. We'd like both unix and windows operating systems to behave similarly, ideally. 
     82 
     83 
     84* http://nebm.ist.utl.pt/~glopes/wget/ 
     85Prebuilt Windows wget binaries (for 32 and 64 bit) version 1.11.4 that includes SSL support 
     86* http://gnuwin32.sourceforge.net/packages/wget.htm 
     87GNU's prebuilt Windows binaries of wget v 1.11.4. May not have been built with SSL support.