Ignore:
Timestamp:
2018-07-09T21:14:55+12:00 (6 years ago)
Author:
ak19
Message:

All the changes to incorporate libtiff and libjpeg into the xpdftools GS2 extension, to get libjpeg compiled up on 64 bit machines, and for both libs to be seen and used by xpdftools when this is built.

File:
1 edited

Legend:

Unmodified
Added
Removed
  • gs2-extensions/xpdf-tools/trunk/src/packages/GS-README.txt

    r32248 r32249  
     1__________________________________________________________
     2CONTENTS
     3__________________________________________________________
     4
     5Xpdf-Tools related
     6A. XPDF
     7B. Mojo::DOM perl package for parsing HTML
     8C. Compiling Xpdf-Tools: statically or dynamically linked
     9D. How we got Xpdf-Tools to compile using CASCADE-MAKE
     10E. Getting more output when running CMake (verbosity)
     11F. APPENDIX - Useful links
     12
     13LIBJPEG related
     14G. LIBJPEG and LIBTIFF
     15- Issues building LIBJPEG on 64 bit machines and the patch
     16
    117__________________________________________________________
    218A. XPDF
     
    84100__________________________________________________________
    85101
    86 As explained in detail in section D below, gs-CMakeLists.txt allows us to compile xpdf-tools statically (as we've now set it up for by default) or dynamically (as its CMake makefiles were originally set up for).
     102As explained in detail in section D below, we have a customised gs-CMakeLists.txt file which replaces the one in the xpf-4.00.tar.gz package's xpdf subfolder after this is untarred. This customised CMake configure/make file now allows us to compile xpdf-tools either statically (as we've now set it up for by default) or dynamically (as its CMake makefiles were originally set up for).
    87103
    881041. To compile Xpdf-Tools statically, packages/CASCADE-MAKE/XPDFTOOLS.sh should contain:
     
    356372
    357373----
    358 /usr/bin/c++  -I/home/greenstone/gs3-svn-26Mar2018/gs2build/ext/xpdf-tools/trunk/src/linux/include  -I/home/greenstone/gs3-svn-26Mar2018/gs2build/ext/xpdf-tools/trunk/src/linux/include -I/home/greenstone/gs3-svn-26Mar2018/gs2build/ext/xpdf-tools/trunk/src/linux/include/libpng15 -O3 -Wall -fPIC  -L/home/greenstone/gs3-svn-26Mar2018/gs2build/ext/xpdf-tools/trunk/src/linux/lib  -L/home/greenstone/gs3-svn-26Mar2018/gs2build/ext/xpdf-tools/trunk/src/linux/lib -static ***** <- HERE ****** -O2 -Wall -fPIC -rdynamic CMakeFiles/pdftohtml.dir/HTMLGen.cc.o CMakeFiles/pdftohtml.dir/SplashOutputDev.cc.o CMakeFiles/pdftohtml.dir/TextOutputDev.cc.o CMakeFiles/pdftohtml.dir/pdftohtml.cc.o CMakeFiles/xpdf_objs.dir/AcroForm.cc.o CMakeFiles/xpdf_objs.dir/Annot.cc.o CMakeFiles/xpdf_objs.dir/Array.cc.o CMakeFiles/xpdf_objs.dir/BuiltinFont.cc.o CMakeFiles/xpdf_objs.dir/BuiltinFontTables.cc.o CMakeFiles/xpdf_objs.dir/Catalog.cc.o CMakeFiles/xpdf_objs.dir/CharCodeToUnicode.cc.o CMakeFiles/xpdf_objs.dir/CMap.cc.o CMakeFiles/xpdf_objs.dir/Decrypt.cc.o CMakeFiles/xpdf_objs.dir/Dict.cc.o CMakeFiles/xpdf_objs.dir/Error.cc.o CMakeFiles/xpdf_objs.dir/FontEncodingTables.cc.o CMakeFiles/xpdf_objs.dir/Form.cc.o CMakeFiles/xpdf_objs.dir/Function.cc.o CMakeFiles/xpdf_objs.dir/Gfx.cc.o CMakeFiles/xpdf_objs.dir/GfxFont.cc.o CMakeFiles/xpdf_objs.dir/GfxState.cc.o CMakeFiles/xpdf_objs.dir/GlobalParams.cc.o CMakeFiles/xpdf_objs.dir/JArithmeticDecoder.cc.o CMakeFiles/xpdf_objs.dir/JBIG2Stream.cc.o CMakeFiles/xpdf_objs.dir/JPXStream.cc.o CMakeFiles/xpdf_objs.dir/Lexer.cc.o CMakeFiles/xpdf_objs.dir/Link.cc.o CMakeFiles/xpdf_objs.dir/NameToCharCode.cc.o CMakeFiles/xpdf_objs.dir/Object.cc.o CMakeFiles/xpdf_objs.dir/OptionalContent.cc.o CMakeFiles/xpdf_objs.dir/Outline.cc.o CMakeFiles/xpdf_objs.dir/OutputDev.cc.o CMakeFiles/xpdf_objs.dir/Page.cc.o CMakeFiles/xpdf_objs.dir/Parser.cc.o CMakeFiles/xpdf_objs.dir/PDFDoc.cc.o CMakeFiles/xpdf_objs.dir/PDFDocEncoding.cc.o CMakeFiles/xpdf_objs.dir/PSTokenizer.cc.o CMakeFiles/xpdf_objs.dir/SecurityHandler.cc.o CMakeFiles/xpdf_objs.dir/Stream.cc.o CMakeFiles/xpdf_objs.dir/TextString.cc.o CMakeFiles/xpdf_objs.dir/UnicodeMap.cc.o CMakeFiles/xpdf_objs.dir/UnicodeTypeTable.cc.o CMakeFiles/xpdf_objs.dir/UTF8.cc.o CMakeFiles/xpdf_objs.dir/XFAForm.cc.o CMakeFiles/xpdf_objs.dir/XRef.cc.o CMakeFiles/xpdf_objs.dir/Zoox.cc.o  -o pdftohtml ../goo/libgoo.a ../fofi/libfofi.a ../splash/libsplash.a -static -lfreetype -lpng -lz -lm -lc -lpthread
     374/usr/bin/c++  -I/home/greenstone/gs3-svn-26Mar2018/gs2build/ext/xpdf-tools/linux/include  -I/home/greenstone/gs3-svn-26Mar2018/gs2build/ext/xpdf-tools/linux/include -I/home/greenstone/gs3-svn-26Mar2018/gs2build/ext/xpdf-tools/linux/include/libpng15 -O3 -Wall -fPIC  -L/home/greenstone/gs3-svn-26Mar2018/gs2build/ext/xpdf-tools/linux/lib  -L/home/greenstone/gs3-svn-26Mar2018/gs2build/ext/xpdf-tools/linux/lib -static ***** <- HERE ****** -O2 -Wall -fPIC -rdynamic CMakeFiles/pdftohtml.dir/HTMLGen.cc.o CMakeFiles/pdftohtml.dir/SplashOutputDev.cc.o CMakeFiles/pdftohtml.dir/TextOutputDev.cc.o CMakeFiles/pdftohtml.dir/pdftohtml.cc.o CMakeFiles/xpdf_objs.dir/AcroForm.cc.o CMakeFiles/xpdf_objs.dir/Annot.cc.o CMakeFiles/xpdf_objs.dir/Array.cc.o CMakeFiles/xpdf_objs.dir/BuiltinFont.cc.o CMakeFiles/xpdf_objs.dir/BuiltinFontTables.cc.o CMakeFiles/xpdf_objs.dir/Catalog.cc.o CMakeFiles/xpdf_objs.dir/CharCodeToUnicode.cc.o CMakeFiles/xpdf_objs.dir/CMap.cc.o CMakeFiles/xpdf_objs.dir/Decrypt.cc.o CMakeFiles/xpdf_objs.dir/Dict.cc.o CMakeFiles/xpdf_objs.dir/Error.cc.o CMakeFiles/xpdf_objs.dir/FontEncodingTables.cc.o CMakeFiles/xpdf_objs.dir/Form.cc.o CMakeFiles/xpdf_objs.dir/Function.cc.o CMakeFiles/xpdf_objs.dir/Gfx.cc.o CMakeFiles/xpdf_objs.dir/GfxFont.cc.o CMakeFiles/xpdf_objs.dir/GfxState.cc.o CMakeFiles/xpdf_objs.dir/GlobalParams.cc.o CMakeFiles/xpdf_objs.dir/JArithmeticDecoder.cc.o CMakeFiles/xpdf_objs.dir/JBIG2Stream.cc.o CMakeFiles/xpdf_objs.dir/JPXStream.cc.o CMakeFiles/xpdf_objs.dir/Lexer.cc.o CMakeFiles/xpdf_objs.dir/Link.cc.o CMakeFiles/xpdf_objs.dir/NameToCharCode.cc.o CMakeFiles/xpdf_objs.dir/Object.cc.o CMakeFiles/xpdf_objs.dir/OptionalContent.cc.o CMakeFiles/xpdf_objs.dir/Outline.cc.o CMakeFiles/xpdf_objs.dir/OutputDev.cc.o CMakeFiles/xpdf_objs.dir/Page.cc.o CMakeFiles/xpdf_objs.dir/Parser.cc.o CMakeFiles/xpdf_objs.dir/PDFDoc.cc.o CMakeFiles/xpdf_objs.dir/PDFDocEncoding.cc.o CMakeFiles/xpdf_objs.dir/PSTokenizer.cc.o CMakeFiles/xpdf_objs.dir/SecurityHandler.cc.o CMakeFiles/xpdf_objs.dir/Stream.cc.o CMakeFiles/xpdf_objs.dir/TextString.cc.o CMakeFiles/xpdf_objs.dir/UnicodeMap.cc.o CMakeFiles/xpdf_objs.dir/UnicodeTypeTable.cc.o CMakeFiles/xpdf_objs.dir/UTF8.cc.o CMakeFiles/xpdf_objs.dir/XFAForm.cc.o CMakeFiles/xpdf_objs.dir/XRef.cc.o CMakeFiles/xpdf_objs.dir/Zoox.cc.o  -o pdftohtml ../goo/libgoo.a ../fofi/libfofi.a ../splash/libsplash.a -static -lfreetype -lpng -lz -lm -lc -lpthread
    359375----
    360376
     
    396412
    397413
     414__________________________________________________________
     415E. Getting more output when running CMake (verbosity)
     416__________________________________________________________
     417See https://www.linuxquestions.org/questions/programming-9/cmake-or-make-debug-output-show-command-624800/
     418To turn on debugging:
     419    export VERBOSE=1
     420    ./CASCADE-MAKE.sh
     421
     422To turn off debugging, need to actually make VERBOSE undefined again (don't set it to 0):
     423    export VERBOSE=
     424    ./CASCADE-MAKE.sh
     425
     426
     427__________________________________________________________
     428F. APPENDIX - Useful links
     429__________________________________________________________
     430A. Helping CMake along. (Not all of this was necessary for compiling xpdftools statically, but they're generally useful links)
     431
     432https://github.com/SynoCommunity/spksrc/issues/1779
     433https://stackoverflow.com/questions/1620918/cmake-and-libpthread
     434https://cmake.org/cmake/help/v3.0/prop_tgt/LINK_FLAGS.html
     435https://cmake.org/cmake/help/v3.11/command/target_link_libraries.html?highlight=target_link_libraries
     436https://stackoverflow.com/questions/24648357/compiling-a-static-executable-with-cmake
     437https://stackoverflow.com/questions/42815420/cmake-cant-find-my-static-libs
     438https://cmake.org/cmake/help/v3.0/command/message.html
     439https://stackoverflow.com/questions/30980383/cmake-compile-options-for-libpng
     440    https://stackoverflow.com/questions/36220123/undefined-reference-to-png-set-longjmp-fn-when-compiling-pcl-source-file
     441
     442
     443B. About the error "bash: no such file or directory" when run on a statically generated binary:
     444
     445https://askubuntu.com/questions/351827/unable-to-run-a-32-bit-program-on-64-bit-vm/353497#353497
     446https://unix.stackexchange.com/questions/13391/getting-not-found-message-when-running-a-32-bit-binary-on-a-64-bit-system/13409#13409
     447https://arstechnica.com/civis/viewtopic.php?f=16&t=1173118
     448https://superuser.com/questions/344533/no-such-file-or-directory-error-in-bash-but-the-file-exists
     449https://unix.stackexchange.com/questions/45277/executing-binary-file-file-not-found
     450
     451C. Other links
     452
     453https://unix.stackexchange.com/questions/279397/ldd-dont-find-path-how-to-add
     454
     455
     456__________________________________________________________
     457G. LIBJPEG and LIBTIFF
     458__________________________________________________________
     459
     4601. Issues building LIBJPEG on 64 bit machines and the patch
     461
     462I copied the LIBJPEG package from http://trac.greenstone.org/browser/other-projects/realistic-books/trunk/packages (also at http://trac.greenstone.org/browser/gs2-extensions/ocr/trunk/packages/cmdline).
     463
     464    * Configuring out of the box produced the following error:
     465       checking host system type... Invalid configuration `x86_64-unknown-linux-gnu': machine `x86_64-unknown' not recognized
     466
     467    * So that, as a consequence, when running make on the libjpeg package, make failed with the error:
     468       ./libtool --mode=compile gcc -I/home/greenstone/gs3-svn-26Mar2018/gs2build/ext/xpdf-tools/linux/include -fPIC  -I/home/greenstone/gs3-svn-26Mar2018/gs2build/ext/xpdf-tools/linux/include  -I. -c ./jcapimin.c
     469       make: ./libtool: Command not found
     470       make: *** [jcapimin.lo] Error 127
     471        Error encountered running *make * stage of ./CASCADE-MAKE/LIBJPEG.sh
     472
     473The same was true when I grabbed the libjpeg from sourceforge (https://sourceforge.net/projects/libjpeg/files/), which was also still version jpeg 6b from 2008.
     474
     475I found the following webpages discussing the above error messages:
     476- https://unix.stackexchange.com/questions/80479/how-to-work-with-libtool
     477- https://github.com/rwestlund/freesweep/issues/1
     478- https://ubuntuforums.org/showthread.php?t=1232714
     479- https://stackoverflow.com/questions/12828687/configure-fails-to-detect-proper-ld-on-a-64-bit-system-with-32-bit-userland
     480- SOLUTION: https://sourceforge.net/p/libjpeg/bugs/12/
     481
     482However, the error only strikes when configure is run with --enable-static.
     483
     484Note also that contrary to the above pages, running configure with the additional options
     485    --host=x86_64-linux-gnu --build=x86_64-linux-gnu --target=x86_64-linux-gnu --disable-shared --enable-static
     486did not help. Nor did adding the above flags get rid of configure attempting to work with host=x86_64-unknown(-unknown)-linux-gnu
     487
     488The SOLUTION, found when searching for the error message along with "enable-static", as it's the combination that is relevant, is described
     489at https://sourceforge.net/p/libjpeg/bugs/12/
     490
     491which was to patch up the config.sub filed included in the jpeg-6b tarball, to also cover x86_64-* machines:
     492        tahoe | i860 | x86_64-* | m32r | m68k | m68000 | m88k | ns32k | arc | arm \
     493
     494The above change is necessary because this libjpeg is outdated and has been superceded by other JPEG libraries, also discussed at https://sourceforge.net/p/libjpeg/bugs/12/
     495I'm not sure if those libraries are compatible with XpdfTools however, so I'm sticking with libjpeg as long as I can get it to build and be recognised by XpdfTools.
     496
     497The solution is once more to have a patch file: CASCADE-MAKE/LIBJPEG.sh replaces the config.sub with in the jpeg-6b package after this is untarred with packages/gs-libjpeg-config.sub, which contains the patch.
     498
     499
     5002. I followed the instructions at http://www.linuxfromscratch.org/blfs/view/6.3/general/libjpeg.html
     501to try to build libjpeg with --enable-static and --enable-shared to produce both libjpeg.a and libjpeg.so.
     502
     503However, nothing I try gets it to generate a libjpeg.so. It seems to always produce a libjpeg.a in xpdf-tools/linux/lib
     504regardless of whether CASCADE-MAKE/LIBJPEG.sh passes the --enable-static flag to the configure command or not, and regardless of whether --enable-shared is additionally or individually passed in.
     505
     506As a consequence, there's no  libjpeg.so file to set the -DJPEG_LIBRARY flag in XPDFTOOLS.sh to for when building xpdf-tools against dynamically linked libraries.
     507
     508I tried the various combinations with the lib jpeg-6b source tarballs from
     509- sourceforge, https://sourceforge.net/projects/libjpeg/files/, the latest tarball of this was from 2008
     510- http://www.linuxfromscratch.org/blfs/view/6.3/general/libjpeg.html, which was last updated in 2007
     511- http://trac.greenstone.org/browser/other-projects/realistic-books/trunk/packages/jpeg-6b.tar.gz, which was added to trac in 2009 but is probably the 2008 or 2007 version too.
     512
     513
     5143. Modifications for using TIFF and JPEG libraries when building Xpdf-Tools:
     515   
     516* CASCADE-MAKE.sh, replaced
     517    PACKAGES="CMAKE LIBZ LIBPNG FREETYPE XPDFTOOLS"
     518with
     519    PACKAGES="CMAKE LIBZ LIBTIFF LIBPNG LIBJPEG FREETYPE XPDFTOOLS"
     520
     521
     522* XPDFTOOLS.sh
     523If compiling statically make sure the CMake command contains the following changes:
     524        -DTIFF_INCLUDE_DIR=$prefix/include \        # <========== new
     525        -DJPEG_INCLUDE_DIR=$prefix/include \        # <========== new
     526        -DZLIB_LIBRARY=$prefix/lib/libz.a \
     527        -DTIFF_LIBRARY=$prefix/lib/libtiff.a \      # <========== new
     528        -DPNG_LIBRARY=$prefix/lib/libpng15.a \
     529        -DJPEG_LIBRARY=$prefix/lib/libjpeg.a \      # <========== new
     530        -DFREETYPE_LIBRARY=$prefix/lib/libfreetype.a \
     531        -DGSDLFLAG_STATIC="$static_flag" \
     532
     533
     534
     535The above flag names were discovered by deleting the untarred xpdf-4.00 folder.
     536Then in a fresh terminal, source devel.bash from xpdf-tools and re-run CASCADE-MAKE.sh without the above modifications:
     537
     538    -- Found FreeType (new-style includes): /home/greenstone/gs3-svn-26Mar2018/gs2build/ext/xpdf-tools/linux/lib/libfreetype.a
     539    -- Found ZLIB: /home/greenstone/gs3-svn-26Mar2018/gs2build/ext/xpdf-tools/linux/lib/libz.a (found version "1.2.8")
     540    -- Found PNG: /home/greenstone/gs3-svn-26Mar2018/gs2build/ext/xpdf-tools/linux/lib/libpng15.a (found version "1.2.50")
     541    -- Could NOT find JPEG (missing: JPEG_LIBRARY JPEG_INCLUDE_DIR)
     542    -- Could NOT find TIFF (missing: TIFF_LIBRARY TIFF_INCLUDE_DIR)
     543    -- lcms2 not found
     544    -- No Qt library found
     545
     546
     547* packages/gs-CMakeLists.txt was modified again,
     548
     549    - this time to also pass:
     550        -ltiff and -ljpeg to all target_link_libraries() commands that run when GSDLFLAG_STATIC is set
     551    and
     552        ${TIFF_LIBRARY} and ${JPEG_LIBRARY} to all target_link_libraries() commands that run when GSDLFLAG_STATIC is not set
     553
     554    - And to add in the include directories and defitions if JPEG/TIFF libraries were provided:
     555        if (JPEG_FOUND)
     556          include_directories("${JPEG_INCLUDE_DIR}")
     557          add_definitions("${JPEG_DEFINITIONS}")
     558          message(STATUS "@@@@@@@@@@@@@@@ JPEG_FOUND (include_dir ; include_dirs): ${JPEG_INCLUDE_DIR} ; ${JPEG_INCLUDE_DIRS}")
     559        else ()
     560          message(STATUS "@@@@@@@@@@@@@@@ NO JPEG_FOUND")
     561        endif ()
     562        if (TIFF_FOUND)
     563          include_directories("${TIFF_INCLUDE_DIRS}")
     564          add_definitions("${TIFF_DEFINITIONS}")
     565          message(STATUS "@@@@@@@@@@@@@@@ TIFF_FOUND ${TIFF_INCLUDE_DIRS}")
     566        else ()
     567          message(STATUS "@@@@@@@@@@@@@@@ NO TIFF_FOUND")
     568        endif ()
     569
     570    Note however that although gs-CMakeLists.txt now knows what the pluralised TIFF_INCLUDE_DIRS is (and TIFF_INCLUDE_DIR)
     571    as for PNG and ZLIB, gs-CMakeLists.txt does not have a value for the pluralised JPEG_INCLUDE_DIRS, only the
     572    JPEG_INCLUDE_DIRS set above. And both the CMAKE flags in XPDFTOOLS.sh for tiff and jpeg libs seem to have been setup
     573    in the same way now. Not sure where these automatically assigned variables come from in order to check up on them.
     574
     575
Note: See TracChangeset for help on using the changeset viewer.