Changeset 31824

Timestamp:

2017-07-27T18:40:26+12:00 (7 years ago)

Author:

ak19

Message:

We now compile up OpenSSL, since wget needs SSL support to download https pages. wget is now compiled up OpenSSL, achieved by changes during wget's configure phase. OpenSSL is configured and compiled up during GS' configure phase. Details are in the updated README at wget level.

Location:

main/trunk/greenstone2/build-src/packages

Files:

• configure (modified) (2 diffs)
• openssl (added)
• openssl/openssl-1.0.2l.tar.gz (added)
• wget/README (modified) (1 diff)

main/trunk/greenstone2/build-src/packages/configure

@@ -139 +139 @@
 
 
+
+# The way common-src compiles up expat during the configure stage itself,
+# configure openssl (needs to be done before wget)
+# and compile openssl during the configure stage itself.
+# Needed by wget for https support
+
+echo ""
+echo "Configuring openssl"
+echo ""
+cd "$PACKAGES/openssl"
+openssl_pkgname=openssl-1.0.2l
+if test ! -d $openssl_pkgname ;
+then
+  gzip -dc $openssl_pkgname.tar.gz | tar -xf -
+fi
+cd openssl-1.0.2l
+
+# configure
+# see the INSTALL file for OpenSSL to see config options for openssl
+# --prefix indicates where the generated lib, include and bin folders will go
+# --openssldir will contain the manuals and other non-essentials
+# we'll put them into $bindir/openssl for wget to use
+echo ./config --prefix=$bindir/openssl --openssldir=$PACKAGES/openssl no-shared
+./config --prefix=$bindir/openssl --openssldir=$PACKAGES/openssl no-shared
+# make
+echo make
+make
+# make install
+echo make install
+make install
+
+# now move the include, lib and bin folders to the correct place
+
 # configure the wvWare Word document converter
 echo ""
@@ -227 +260 @@
 fi
 
+#./configure $CACHE_FILE --prefix=$PACKAGES/wget --without-ssl --bindir="$bindir" -disable-nls $HOSTTARGETFLAGS $wget_extraargs
+
+# We weren't compiling up wget statically before either
+# However, to compile up wget (statically or not) with openssl, see
+# https://stackoverflow.com/questions/9817337/compiling-wget-with-static-linking-self-compiled-openssl-library-linking-issu
+# Note: if the cppflags and ldflags are set during the configure stage, as we now do,
+# then the make command needn't additionally set them as well. So just make, make install then works 
+echo cd "$PACKAGES/wget/wget-1.15-gs"
+echo CPPFLAGS="-I/$bindir/openssl/include" LDFLAGS="-L/$bindir/openssl/lib" ./configure $CACHE_FILE --prefix=$PACKAGES/wget --with-ssl=openssl --bindir="$bindir" -disable-nls $HOSTTARGETFLAGS $wget_extraargs
 cd "$PACKAGES/wget/wget-1.15-gs" && \
-#./configure $CACHE_FILE --prefix=$PACKAGES/wget --with-ssl=openssl --bindir="$bindir" -disable-nls $HOSTTARGETFLAGS $wget_extraargs
-./configure $CACHE_FILE --prefix=$PACKAGES/wget --without-ssl --bindir="$bindir" -disable-nls $HOSTTARGETFLAGS $wget_extraargs
+CPPFLAGS="-I/$bindir/openssl/include" LDFLAGS="-L/$bindir/openssl/lib" ./configure $CACHE_FILE --prefix=$PACKAGES/wget --with-ssl=openssl --bindir="$bindir" -disable-nls $HOSTTARGETFLAGS $wget_extraargs
+
+
+##
+##CPPFLAGS="-I/Scratch/ak19/gs3-svn-13July2017/gs2build/bin/linux/openssl/include" LDFLAGS="-L/Scratch/ak19/gs3-svn-13July2017/gs2build/bin/linux/openssl/lib" ./configure --prefix=/Scratch/ak19/gs3-svn-13July2017/gs2build/build-src/packages/wget --with-ssl=openssl --bindir="/Scratch/ak19/gs3-svn-13July2017/gs2build/bin/linux" -disable-nls
+##
+##make CPPFLAGS="-I/Scratch/ak19/gs3-svn-13July2017/gs2build/bin/linux/openssl/include" LDFLAGS="-L/Scratch/ak19/gs3-svn-13July2017/gs2build/bin/linux/openssl/lib -static"
 
 if test x$? = x1 ;

main/trunk/greenstone2/build-src/packages/wget/README

@@ -59 +59 @@
 7) 2014/10/13 - ak19
 Moved to wget version 1.15. 
-Only the changes numbered 5 and 6 above have been ported into it, following Dr Bainbridge's instructions, as there were both changes made to the previous wget version Greenstone used (1.13.4).
-The wget tar file name (wget-1.15-gs) now indicates the version number and that it has been modified by Greenstone, so the Makefile and configure file in build-src/packages/ have been updated to reflect this.
+Only the changes numbered 5 and 6 above have been ported into it, following Dr Bainbridge's instructions, as they were both changes made to the previous wget version Greenstone used (1.13.4).
+The wget tar file name (wget-1.15-gs) now indicates the version number and that it has been modified for Greenstone, so the Makefile and configure file in build-src/packages/ have been updated to reflect this.
 
+8) 2017/07/27 - ak19 ([email protected]) - still using wget version 1.15, but now compiling wget up with OpenSSL support. Wget needs SSL support in order for it to access pages over HTTPS. In future, the web will be using https.
+
+We're now compiling up OpenSSL during the configuration phase since wget needs it to exist during its configure phase. We;re building OpenSSL statically, by setting the no-shared flag. The built OpenSSL gets put into gs2build/linux|darwin/openssl, containing lib, include and bin subfolders. When configuring wget, we build wget against our OpenSSl, and make and make install proceed as normal. Refer to gs2build/build-src/packages configure.
+
+We weren't compiling up wget statically before either, so we're still not doing so. To compile up wget (statically or not) with openssl, a helpful page was
+https://stackoverflow.com/questions/9817337/compiling-wget-with-static-linking-self-compiled-openssl-library-linking-issu
+Note, however, that since the CPPFLAGS and LDFLAGS are now set to point to our OpenSSL during the configure stage, the make command needn't additionally set them as well, contrary to the instruction for make on the stackoverflow page. So we just need to do the usual make, make install once the configure is done against OpenSSL.
+
+If compiling wget up statically, then, in the LDFLAGS prepended to wget's configure command, append -static. Further, the gcc command that gets run needs to have -lpthread in its library listing at the end. The order of the libraries listed also needs to change for static compilation to be successful:
+-lprce -lpthread -ldl <remaining -llibs>
+
+However, warnings appear when compiling wget statically, as it does not make sense to create some programs statically since they may be stuck including a local context (e.g. something related to DNS warnings in compiling up a previous component statically). Linking against some libraries to create a static binary may not make sense either. For instance -ldl, the dynamic loading or linking library, may not make sense if the binary created is static. This seems to imply that wget makes more sense if compiled up as a shared object, .so, than as a static one, .a.
+
+The existing version of wget, 1.15, works with HTTPS when compiled against OpenSSL. However, this version of the binary needs to be run with the --no-check-certificate flag on to access https pages without a security certificate.
+
+e.g. ./wget --no-check-certificate http://englishhistory.net/tudor/citizens/
+
+The system wget on Ubuntu 16.04 is version 1.17.1 and does not require this flag. Pre-compiled windows binaries are available for version 1.11.4, so that may still require the flag. This will require further investigation. We'd like both unix and windows operating systems to behave similarly, ideally.
+
+
+* http://nebm.ist.utl.pt/~glopes/wget/
+Prebuilt Windows wget binaries (for 32 and 64 bit) version 1.11.4 that includes SSL support
+* http://gnuwin32.sourceforge.net/packages/wget.htm
+GNU's prebuilt Windows binaries of wget v 1.11.4. May not have been built with SSL support.