Changeset 31824


Ignore:
Timestamp:
07/27/17 18:40:26 (3 years ago)
Author:
ak19
Message:

We now compile up OpenSSL, since wget needs SSL support to download https pages. wget is now compiled up OpenSSL, achieved by changes during wget's configure phase. OpenSSL is configured and compiled up during GS' configure phase. Details are in the updated README at wget level.

Location:
main/trunk/greenstone2/build-src/packages
Files:
2 added
2 edited

Legend:

Unmodified
Added
Removed
  • main/trunk/greenstone2/build-src/packages/configure

    r30896 r31824  
    139139
    140140
     141
     142# The way common-src compiles up expat during the configure stage itself,
     143# configure openssl (needs to be done before wget)
     144# and compile openssl during the configure stage itself.
     145# Needed by wget for https support
     146
     147echo ""
     148echo "Configuring openssl"
     149echo ""
     150cd "$PACKAGES/openssl"
     151openssl_pkgname=openssl-1.0.2l
     152if test ! -d $openssl_pkgname ;
     153then
     154  gzip -dc $openssl_pkgname.tar.gz | tar -xf -
     155fi
     156cd openssl-1.0.2l
     157
     158# configure
     159# see the INSTALL file for OpenSSL to see config options for openssl
     160# --prefix indicates where the generated lib, include and bin folders will go
     161# --openssldir will contain the manuals and other non-essentials
     162# we'll put them into $bindir/openssl for wget to use
     163echo ./config --prefix=$bindir/openssl --openssldir=$PACKAGES/openssl no-shared
     164./config --prefix=$bindir/openssl --openssldir=$PACKAGES/openssl no-shared
     165# make
     166echo make
     167make
     168# make install
     169echo make install
     170make install
     171
     172# now move the include, lib and bin folders to the correct place
     173
    141174# configure the wvWare Word document converter
    142175echo ""
     
    227260fi
    228261
     262#./configure $CACHE_FILE --prefix=$PACKAGES/wget --without-ssl --bindir="$bindir" -disable-nls $HOSTTARGETFLAGS $wget_extraargs
     263
     264# We weren't compiling up wget statically before either
     265# However, to compile up wget (statically or not) with openssl, see
     266# https://stackoverflow.com/questions/9817337/compiling-wget-with-static-linking-self-compiled-openssl-library-linking-issu
     267# Note: if the cppflags and ldflags are set during the configure stage, as we now do,
     268# then the make command needn't additionally set them as well. So just make, make install then works
     269echo cd "$PACKAGES/wget/wget-1.15-gs"
     270echo CPPFLAGS="-I/$bindir/openssl/include" LDFLAGS="-L/$bindir/openssl/lib" ./configure $CACHE_FILE --prefix=$PACKAGES/wget --with-ssl=openssl --bindir="$bindir" -disable-nls $HOSTTARGETFLAGS $wget_extraargs
    229271cd "$PACKAGES/wget/wget-1.15-gs" && \
    230 #./configure $CACHE_FILE --prefix=$PACKAGES/wget --with-ssl=openssl --bindir="$bindir" -disable-nls $HOSTTARGETFLAGS $wget_extraargs
    231 ./configure $CACHE_FILE --prefix=$PACKAGES/wget --without-ssl --bindir="$bindir" -disable-nls $HOSTTARGETFLAGS $wget_extraargs
     272CPPFLAGS="-I/$bindir/openssl/include" LDFLAGS="-L/$bindir/openssl/lib" ./configure $CACHE_FILE --prefix=$PACKAGES/wget --with-ssl=openssl --bindir="$bindir" -disable-nls $HOSTTARGETFLAGS $wget_extraargs
     273
     274
     275##
     276##CPPFLAGS="-I/Scratch/ak19/gs3-svn-13July2017/gs2build/bin/linux/openssl/include" LDFLAGS="-L/Scratch/ak19/gs3-svn-13July2017/gs2build/bin/linux/openssl/lib" ./configure --prefix=/Scratch/ak19/gs3-svn-13July2017/gs2build/build-src/packages/wget --with-ssl=openssl --bindir="/Scratch/ak19/gs3-svn-13July2017/gs2build/bin/linux" -disable-nls
     277##
     278##make CPPFLAGS="-I/Scratch/ak19/gs3-svn-13July2017/gs2build/bin/linux/openssl/include" LDFLAGS="-L/Scratch/ak19/gs3-svn-13July2017/gs2build/bin/linux/openssl/lib -static"
    232279
    233280if test x$? = x1 ;
  • main/trunk/greenstone2/build-src/packages/wget/README

    r29366 r31824  
    59597) 2014/10/13 - ak19
    6060Moved to wget version 1.15.
    61 Only the changes numbered 5 and 6 above have been ported into it, following Dr Bainbridge's instructions, as there were both changes made to the previous wget version Greenstone used (1.13.4).
    62 The wget tar file name (wget-1.15-gs) now indicates the version number and that it has been modified by Greenstone, so the Makefile and configure file in build-src/packages/ have been updated to reflect this.
     61Only the changes numbered 5 and 6 above have been ported into it, following Dr Bainbridge's instructions, as they were both changes made to the previous wget version Greenstone used (1.13.4).
     62The wget tar file name (wget-1.15-gs) now indicates the version number and that it has been modified for Greenstone, so the Makefile and configure file in build-src/packages/ have been updated to reflect this.
    6363
     648) 2017/07/27 - ak19 (anupama.krishnan@waikato.ac.nz) - still using wget version 1.15, but now compiling wget up with OpenSSL support. Wget needs SSL support in order for it to access pages over HTTPS. In future, the web will be using https.
     65
     66We're now compiling up OpenSSL during the configuration phase since wget needs it to exist during its configure phase. We;re building OpenSSL statically, by setting the no-shared flag. The built OpenSSL gets put into gs2build/linux|darwin/openssl, containing lib, include and bin subfolders. When configuring wget, we build wget against our OpenSSl, and make and make install proceed as normal. Refer to gs2build/build-src/packages configure.
     67
     68We weren't compiling up wget statically before either, so we're still not doing so. To compile up wget (statically or not) with openssl, a helpful page was
     69https://stackoverflow.com/questions/9817337/compiling-wget-with-static-linking-self-compiled-openssl-library-linking-issu
     70Note, however, that since the CPPFLAGS and LDFLAGS are now set to point to our OpenSSL during the configure stage, the make command needn't additionally set them as well, contrary to the instruction for make on the stackoverflow page. So we just need to do the usual make, make install once the configure is done against OpenSSL.
     71
     72If compiling wget up statically, then, in the LDFLAGS prepended to wget's configure command, append -static. Further, the gcc command that gets run needs to have -lpthread in its library listing at the end. The order of the libraries listed also needs to change for static compilation to be successful:
     73-lprce -lpthread -ldl <remaining -llibs>
     74
     75However, warnings appear when compiling wget statically, as it does not make sense to create some programs statically since they may be stuck including a local context (e.g. something related to DNS warnings in compiling up a previous component statically). Linking against some libraries to create a static binary may not make sense either. For instance -ldl, the dynamic loading or linking library, may not make sense if the binary created is static. This seems to imply that wget makes more sense if compiled up as a shared object, .so, than as a static one, .a.
     76
     77The existing version of wget, 1.15, works with HTTPS when compiled against OpenSSL. However, this version of the binary needs to be run with the --no-check-certificate flag on to access https pages without a security certificate.
     78
     79e.g. ./wget --no-check-certificate http://englishhistory.net/tudor/citizens/
     80
     81The system wget on Ubuntu 16.04 is version 1.17.1 and does not require this flag. Pre-compiled windows binaries are available for version 1.11.4, so that may still require the flag. This will require further investigation. We'd like both unix and windows operating systems to behave similarly, ideally.
     82
     83
     84* http://nebm.ist.utl.pt/~glopes/wget/
     85Prebuilt Windows wget binaries (for 32 and 64 bit) version 1.11.4 that includes SSL support
     86* http://gnuwin32.sourceforge.net/packages/wget.htm
     87GNU's prebuilt Windows binaries of wget v 1.11.4. May not have been built with SSL support.
Note: See TracChangeset for help on using the changeset viewer.