root/gs2-extensions/xpdf-tools/trunk/src/GS-README.txt @ 32255

Revision 32255, 46.9 KB (checked in by ak19, 11 months ago)

Moved the GS-README.txt out from the packages subfolder to the higher level folder of the xpdf-tools extension, so it's more instantly visible

Line 
1__________________________________________________________
2CONTENTS
3__________________________________________________________
4
5Xpdf-Tools related
6A. XPDF
7B. Mojo::DOM perl package for parsing HTML
8C. Compiling Xpdf-Tools: statically or dynamically linked
9D. How we got Xpdf-Tools to compile using CASCADE-MAKE
10E. Getting more output when running CMake (verbosity)
11F. APPENDIX - Useful links
12
13LIBJPEG related
14G. LIBJPEG and LIBTIFF
15- Moving from 2008's libjpeg version 6b to the newer 2018 version 9c
16- Issues building LIBJPEG version 6b on 64 bit machines and the patch
17
18H. Licensing information and making the distributable tarball
19
20I. PDF2DOM
21    unused, replaced by Xpdf-Tools' more suited pdftohtml capabilities
22
23__________________________________________________________
24A. XPDF
25__________________________________________________________
26
27Xpdf's last mod date is in 2017 and it includes its own pdftohtml utility tool, whereas the old "pdftohtml" tool that GS used was last updated 2013 (and itself made use of Xpdf, possible older versions).
28
29The tool takes a PDF and produces an HTML file for each page of the PDF, consisting of selectable HTML text overlaid on top of "screenshot" image of the page. (A page's text is not part of the screenshot.)
30
311. https://www.xpdfreader.com/download.html
32
33As per the Readme file found in the linux binary of Xpdf Tools, the Xpdf Viewer requires the qt toolkit, but not the Xpdf Tools. Have not read the Install file to confirm whether the same is the case for when compiling the command line tools. (But in that case, can't we just include the tools binary available for all 3 OS, instead of compiling on each platform)
34
35    - Using Xpdf's pdftohtml tool:
36    greenstone@bedrock:~/Downloads/xpdf-tools-linux-4.00/bin64$./pdftohtml -z 1.5 ~/Downloads/ApacheLicence.pdf licence
37
38        where licence is a folder.
39
40    - Using Xpdf's pdftotext tool:
41    greenstone@bedrock:~/Downloads/xpdf-tools-linux-4.00/bin64$./pdftotext -nopgbrk ~/Downloads/ApacheLicence.pdf ~/Downloads/ApacheLicence.txt
42
43        where the output text file must be specified with a full path name.
44
45
462. Documentation on Xpdf-Tools:
47- https://www.xpdfreader.com/support.html
48    for example, the pdftohtml man page: https://www.xpdfreader.com/pdftohtml-man.html
49- https://linux.die.net/man/5/xpdfrc
50(Configuration flags you can put into ~/.xpdfrc to use as defaults when running xpdf tool commands)
51
523. We're using Xpdf Tools version: xpdf-tools-linux-4.00
53
544. We started by working with the ready-made Xpdf-tools binaries available for download from the xpdf site for Win, Linux and Mac.
55
565. We're now moving to compiling up Xpdf-tools ourselves using CASCADE-MAKE, which we have so far got to successfully compile statically on Linux (LSB environment inclusive) to build working binaries.
57
58On Mac, I've been unable to get it to produce statically linked libraries: at this stage they're dynamically linked.
59
60
61__________________________________________________________
62B. Mojo::DOM perl package for parsing HTML
63__________________________________________________________
64
65XPDF's pdftohtml conversion of a single PDF document produces multiple HTML files: one for each page in the source PDF.
66We want the output to be "paged_html": a single HTML file that is sectionalised, each section representing a page of the
67original PDF.
68
69We need to be able to parse the many HTML pages produced by XPDF's pdftohtml conversion of a doc, in order to massage the output
70into the single sectionalised HTML file. For this we needed a HTML parser package for Perl.
71
721. Before Dr Bainbridge found Mojo::DOM, he looked at
73* https://en.wikipedia.org/wiki/Comparison_of_HTML_parsers
74* http://radar.oreilly.com/2014/02/parsing-html-with-perl-2.html
75
762. Main links for Mojo::DOM
77* https://mojolicious.org/perldoc/Mojo/DOM
78* https://metacpan.org/pod/Mojo::DOM
79    Dependencies: http://deps.cpantesters.org/?module=Mojo%3A%3ADOM;perl=latest
80
813. Once you've downloaded Mojo::DOM's src, follow Dr Bainbridge's sequence of commands for building the Mojo::DOM CPAN module of perl below.
82We'll be using this module to be used for parsing the HTML output by XPDF tool pdftohtml
83
84
85    mkdir cpan
86     2020  tar xvzf Mojolicious-7.84.tar.gz
87     2021  cd Mojolicious-7.84/
88     2028  perl ./Makefile.PL PREFIX=`pwd`/installed
89     2030  make
90     2031  make install
91     2033  cp -r installed/share/perl/5.18.2 ../cpan
92    cd ..
93     2044  export PERL5LIB=`pwd`/cpan
94
95     2053  emacs -nw test.pl
96
97    #!/usr/bin/perl -w
98    add in 'use v5.10;'
99     
100     2054  chmod a+x test.pl
101     2055  ./test.pl
102
103
104__________________________________________________________
105C. Compiling Xpdf-Tools: statically or dynamically linked
106__________________________________________________________
107
108As explained in detail in section D below, we have a customised gs-CMakeLists.txt file which replaces the one in the xpf-4.00.tar.gz package's xpdf subfolder after this is untarred. This customised CMake configure/make file now allows us to compile xpdf-tools either statically (as we've now set it up for by default) or dynamically (as its CMake makefiles were originally set up for).
109
1101. To compile Xpdf-Tools statically, packages/CASCADE-MAKE/XPDFTOOLS.sh should contain:
111
112    cmake -DCMAKE_BUILD_TYPE=Release \
113        -DCMAKE_INSTALL_PREFIX=$prefix \
114        -DZLIB_LIBRARY=$prefix/lib/libz.a \         # <========= THIS
115        -DPNG_LIBRARY=$prefix/lib/libpng15.a \      # <========= THIS
116        -DFREETYPE_LIBRARY=$prefix/lib/libfreetype.a \  # <========= THIS
117        -DCMAKE_DISABLE_FIND_PACKAGE_Qt4=1 \
118        -DCMAKE_DISABLE_FIND_PACKAGE_Qt5Widgets=1 \
119        -DCMAKE_C_FLAGS="$CFLAGS" \
120        -DCMAKE_CXX_FLAGS="$CXXFLAGS" \
121        -DCMAKE_EXE_LINKER_FLAGS="$LDFLAGS" \
122        -DGSDLFLAG_STATIC="$static_flag" \          # <========= THIS
123        $GEXT_XPDFTOOLS/packages/$package$version
124
125In place of FREETYPE_LIBRARY above, could also try the following,
126        -DFREETYPE_DIR=$prefix \
127but then check the built binaries by running "ldd" and "file" over them, to make sure they're not referencing any .so dynamic link libraries:
128
129
1302. To compile Xpdf-Tools dynamically and make it find *our* dynamically linked libraries for its helper packages zlib, libpng and freetype, edit packages/CASCADE-MAKE/XPDFTOOLS.sh to contain:
131
132    cmake -DCMAKE_BUILD_TYPE=Release \
133        -DCMAKE_INSTALL_PREFIX=$prefix \
134        -DZLIB_LIBRARY=$prefix/lib/libz.so.1.2.7 \          # <========= THIS
135        -DPNG_LIBRARY=$prefix/lib/libpng15.so.15.30.0 \     # <========= THIS
136        -DFREETYPE_LIBRARY=$prefix/lib/libfreetype.so.6.3.20 \  # <========= THIS
137        -DCMAKE_DISABLE_FIND_PACKAGE_Qt4=1 \
138        -DCMAKE_DISABLE_FIND_PACKAGE_Qt5Widgets=1 \
139        -DCMAKE_C_FLAGS="$CFLAGS" \
140        -DCMAKE_CXX_FLAGS="$CXXFLAGS" \
141        -DCMAKE_EXE_LINKER_FLAGS="$LDFLAGS" \
142        $GEXT_XPDFTOOLS/packages/$package$version       # <=== -DGSDLFLAG_STATIC removed
143
144
145
146    (1) In the above, you could also set
147        -DFREETYPE_DIR=$prefix
148    in place of
149        -DGSDLFLAG_STATIC="$static_flag"
150
151    In that case it makes, xpdf-tools compilation find the "libfreetype.so" (no versioning at end) in our gs2-extension.
152    After successfully building, make sure to have sourced the gs2-extension's setup.bash before running "ldd" over the
153    generated xpdf-tools binaries, in order to let it use the $LD_LIBRARY_PATH we set to find our .so files.
154
155    (2) Note that there are no equivalent for ZLIB and LIBPNG: doing -DZLIB_DIR=$prefix or -DPNG_DIR=$prefix will be
156    ineffective, as neither are recognised by xpdf-tools' CMake set up.
157
158__________________________________________________________
159D. How we got Xpdf-Tools to compile using CASCADE-MAKE
160__________________________________________________________
161
162The process:
163
1641. We set up a CASCADE-MAKE GS2-extension "xpdf-tools" at trac.greenstone.org/browser/gs2-extensions/xpdf-tools/trunk/src
165Be aware that its lowercased "cascade-make" subfolder is an svn external, the original is at http://trac.greenstone.org/browser/other-projects/cascade-make/trunk/
166
167So far, this CASCADE-MAKE project includes the Xpdf-Tools source tarball, its helper packages zlib, libpng and freetype, as well as CMake to compile the Xpdf-Tools source code.
168The next step is to include JPEG and TIFF libraries too.
169
1702a. We downloaded the Xpdf-Tools source tarball, xpdf-4.00.tar.gz, from the xpdf site at https://www.xpdfreader.com/download.html under section "Download the Xpdf source code".
171
172The xpdf-tools source code tarball consists of the source for Xpf-tools and Xpdf (Xpdf-Reader). The Xpdf-Reader additionally requires Qt to build and run, but we don't want the Xpdf-Reader, just Xpdf-Tools.
173
174b. Compiling Xpdf-Tools fron source and running them requires the following packages and libraries, as per the xpdf-tools source code INSTALL file:
175
176To build xpdf-tools:
177- CMake 2.8.8 or newer
178
179Libraries to link against and used by xpdf-tools:
180- FreeType 2.0.5 or newer
181- libpng (for pdftoppm and pdftohtml)
182- zlib (for pdftoppm and pdftohtml)
183
184
1853. Compilation of xpdf-tools worked with CMake 3.11.4 on the linux resnet machine. However, CMake 3.11.3 itself failed to compile in the LSB environment and on the Mac Mountain Lion machine because of a version incompatibility between the older g++ installed there and the advanced version of CMake 3.11.4.
186
187CMake version 3.9.6 however is supposed to be compatible with older versions of g++, as per https://stackoverflow.com/questions/47886400/cmake-configure-error-in-3-10-1-but-not-in-3-9-6
188To avoid installing newer versions of g++ and clang in the LSB virtual machine and the Mac, I've shifted the CMake version back to version 3.9.6, still
189
190
1914a. On building xpdf-tools to work with dynamically linked libs found anywhere.
192
193If compiling xpdf-tools against dynamic linked libraries for these packages, then the basic CMake command in packages/CASECADE-MAKE/XPDFTOOLS.sh can look like:
194    cmake -DCMAKE_BUILD_TYPE=Release \
195        -DCMAKE_INSTALL_PREFIX=$prefix \
196        -DCMAKE_DISABLE_FIND_PACKAGE_Qt4=1 \
197        -DCMAKE_DISABLE_FIND_PACKAGE_Qt5Widgets=1 \
198        -DCMAKE_C_FLAGS="$CFLAGS" \
199        -DCMAKE_CXX_FLAGS="$CXXFLAGS" \
200        -DCMAKE_EXE_LINKER_FLAGS="$LDFLAGS" \
201        $GEXT_XPDFTOOLS/packages/$package$version   # Note: no -DGSDLFLAG_STATIC=...
202
203With the above, the xpdf-tools source code and its make files work out of the box.
204
2054b. On building xpdf-tools to work with the dynamically linked libs for freetype libpng, zlib that we produce when cascade-making the xpdf-tools gs2-extension.
206
207Since we're compiling up freetype, libpng and zlib packages as part of the Xpdf-Tools GS2-extension with CASCADE-MAKE, the next step was to compile xpdf-tools by dynamically linking against our .so files for these 3 libraries. To do so, XPDFTOOL.sh should have the following changes
208
209    (1) set up CFLAGS, CXXFLAGS, CPPFLAGS and LDFLAGS to help linkage of xpdf-tools find our .so versions of the necessary libs:
210
211    export CFLAGS="$CFLAGS -I$GEXTXPDFTOOLS_INSTALLED/include -I$GEXTXPDFTOOLS_INSTALLED/include/libpng15"
212    export CPPFLAGS="$CPPFLAGS -I$GEXTXPDFTOOLS_INSTALLED/include -I$GEXTXPDFTOOLS_INSTALLED/include/libpng15"
213    export CXXFLAGS="$CXXFLAGS -I$GEXTXPDFTOOLS_INSTALLED/include -I$GEXTXPDFTOOLS_INSTALLED/include/libpng15"
214    export LDFLAGS="$LDFLAGS -L$GEXTXPDFTOOLS_INSTALLED/lib"
215
216    (2) The CMAKE command we run must pass the full paths to the actual .so library files (the ones with specific
217    versions in their files names) rather than the symbolically linked generally-named .so files (the latter won't
218    be found when building xpdf-tools and CMake will try to look for the .so library files elsewhere on the system):
219
220    cmake -DCMAKE_BUILD_TYPE=Release \
221        -DCMAKE_INSTALL_PREFIX=$prefix \
222        -DZLIB_LIBRARY=$prefix/lib/libz.so.1.2.7 \              # <========= NEW
223        -DPNG_LIBRARY=$prefix/lib/libpng15.so.15.30.0 \         # <========= NEW
224        -DFREETYPE_LIBRARY=$prefix/lib/libfreetype.so.6.3.20 \      # <========= NEW
225        -DCMAKE_DISABLE_FIND_PACKAGE_Qt4=1 \
226        -DCMAKE_DISABLE_FIND_PACKAGE_Qt5Widgets=1 \
227        -DCMAKE_C_FLAGS="$CFLAGS" \
228        -DCMAKE_CXX_FLAGS="$CXXFLAGS" \
229        -DCMAKE_EXE_LINKER_FLAGS="$LDFLAGS" \
230        $GEXT_XPDFTOOLS/packages/$package$version   # Again: no -DGSDLFLAG_STATIC=...
231
232Further, the "xpdf/CMakeLists.txt" file within the xpdf-4.00.tar.gz source code tarball needs to be modified to refer to ZLIB_LIBRARIES when linking pdftops and pdftoppm. The linking commands for *both* the "pdftops" and "pdftoppm" executable targets in xpdf/CMakeLists.txt should look like the following,
233
234        target_link_libraries(pdftoppm goo fofi splash
235                        ${FREETYPE_LIBRARY} ${FREETYPE_OTHER_LIBS}
236                        ${DTYPE_LIBRARY}
237                        ${LCMS_LIBRARY}
238            ${ZLIB_LIBRARIES})              # <========= NEW
239
240
241    (3) Since CMakeLists.txt has been modified, we initially renamed the xpdf src tarball to gs-xpdf-4.00.tar.gz.
242    However, the current version works with the regular downloaded xpdf-4.00.tar.gz tarball. But after extraction,
243    XPDFTOOLS.sh copies across the custom packages/gs-CMakeLists.txt into the extracted tarball's xpdf subdirectory,
244    renaming the file as CMakeLists.txt (so the path to it becomes "xpdf-4.00/xpdf/CMakeLists.txt"). In XPDFTOOLS.sh:   
245
246    # patch the original tarball with our custom makefile
247    if [[ -d "$package$version/xpdf" && -f "gs-CMakeLists.txt" ]]; then
248        echo "*******************************************************************"
249        echo "Using our custom gs-CMakeLists.txt instead of the one included in $package$version"
250        echo "Renaming gs-CMakeLists.txt to $package$version/xpdf/CMakeLists.txt"
251        echo "*******************************************************************"
252
253        cp "gs-CMakeLists.txt" "$package$version/xpdf/CMakeLists.txt"
254    fi
255
256
2574c. On building static xpdf-tools binaries using the static *.a freetype libpng, zlib libraries that we produce when cascade-making the xpdf-tools gs2-extension.
258
259In order to compile up xpdf-tools *statically*, so that it builds against the static *.a libraries of freetype, libpng and zlib that we produce during the gs2-extension's CASCADE-MAKE process, we have to make further modifications.
260
261    (1) First, the XPDFTOOLS.sh cascade-make file should pass the full paths to the actual (non-symbolic link) .a file for each library.
262    A custom GS flag, GSDLFLAG_STATIC, is also invented in gs-CMakeLists.txt and assigned "-static for linux
263    and "-Bstatic" for Mac, to pass in during the linking stage of building xpdf-tools.
264
265    For Mac OSX, when -static is passed in for linking as on linux, this produced the error
266    "ld: library not found for -lcrt0.o" during the build of the xpdf-tools package. For information, see
267    https://stackoverflow.com/questions/3801011/ld-library-not-found-for-lcrt0-o-on-osx-10-6-with-gcc-clang-static-flag
268    The page https://stackoverflow.com/questions/844819/how-to-static-link-on-os-x mentions compiling
269    with -Bstatic on Mac OSX instead. To do so, XPDFTOOLS.sh passes in the GSDLFLAG_STATIC set to either
270    "-static" (for linux) or "-Bstatic" for darwin.
271    However the last mentioned stackoverflow page also says that -Bstatic is a no-op, and this appears to be
272    the case when "otool -L" is run over the generated xpdf-tools binaries: the binaries are all dynamically
273    linked. Although they're finding our .so files of freetype, libpng and zlib, they're not finding the .a
274    versions, even though XPDFTOOLS.sh tries to point gs-CMakeLists.txt to the correct .a files.
275
276    The new modifications to XPDFTOOLS.sh:
277
278    if [ "x$GSDLOS" == "xdarwin" ] ; then
279        static_flag=-Bstatic
280    else
281        static_flag=-static
282    fi
283
284    ...
285    cmake -DCMAKE_BUILD_TYPE=Release \
286        -DCMAKE_INSTALL_PREFIX=$prefix \
287        -DZLIB_LIBRARY=$prefix/lib/libz.a \                 # <========= MODIFIED TO .a
288        -DPNG_LIBRARY=$prefix/lib/libpng15.a \              # <========= MODIFIED TO .a
289        -DFREETYPE_LIBRARY=$prefix/lib/libfreetype.a \          # <========= MODIFIED TO .a
290        -DCMAKE_DISABLE_FIND_PACKAGE_Qt4=1 \
291        -DCMAKE_DISABLE_FIND_PACKAGE_Qt5Widgets=1 \
292        -DCMAKE_C_FLAGS="$CFLAGS" \
293        -DCMAKE_CXX_FLAGS="$CXXFLAGS" \
294        -DCMAKE_EXE_LINKER_FLAGS="$LDFLAGS" \
295        -DGSDLFLAG_STATIC="$static_flag" \                  # <========= NEW
296        $GEXT_XPDFTOOLS/packages/$package$version
297
298    (2) Our customised gs-CMakeLists.txt file now checks for this flag GSDLFLAG_STATIC being set and, if it is,
299    uses it during the linking stage. As in (1) above, it will be set to "-static" for Linux and "-Bstatic" for Mac.
300   
301    - When the flag is set, the linking flags passed into each occurrence of target_link_libraries() in
302    gs-CMakeLists.txt is moreover manually written in the form of "-static -l<libs>" rather than using
303    the default linking commands inherited from the original CMakeLists.txt.
304    - If GSDLFLAG_STATIC isn't set, then we don't build statically, and the linking flags passed to each
305    target_link_libraries() are mostly the original ones.
306
307    For example,
308
309        if(GSDLFLAG_STATIC)
310            target_link_libraries(pdftoppm goo fofi splash
311              ${GSDLFLAG_STATIC} -lfreetype ${DTYPE_LIBRARY} ${LCMS_LIBRARY} -lz -lm -lc -lpthread)
312        else ()
313            target_link_libraries(pdftoppm goo fofi splash
314                            ${FREETYPE_LIBRARY} ${FREETYPE_OTHER_LIBS}
315                            ${DTYPE_LIBRARY}
316                           ${LCMS_LIBRARY}
317                    ${ZLIB_LIBRARIES})
318        endif ()
319
320    DETAILED EXPLANATION:
321    We found that when building *statically*, gs-CMakeLists.txt needed to NOT use the PNG_LIBRARIES, ZLIB_LIBRARIES
322    and FREETYPE_LIBRARY in its linker commands, target_link_libraries(), as doing so produced partially dynamic
323    xpdf-tools executables which were moreover BROKEN. They wouldn't run, and in fact attempting to run an xpdf-tool,
324    like "./pdftohtml", would produce a file not found error. Something like "bash: no such file or directory".
325
326    Online discussions mentioned that this generally happened when attempting to run 32 bit executables on 64 bit
327    linux when 32 bit loaders are not installed. (In such cases, the solution was to apt-get install some 32 bit package.)
328    However, our broken binaries were all 64 bit, as indicated when running the "file" command on them. However, their
329    being further partially dynamically linked executables didn't imply that they would be broken, as we were eventually
330    able to produce partially dynamic executables that did work, before solving static linking altogether.
331
332    The real issue was that including references to  ${FREETYPE_LIBRARY} ${FREETYPE_OTHER_LIBS}, ${PNG_LIBRARIES} and
333    ${ZLIB_LIBRARIES} in any target_link_libraries() resulted in the wrong linking command producing broken binaries.
334
335    Doing the regular target_link_libraries() in static mode results in building with
336    "-Wl,-Bstatic -lfreetype -lpng15 -lz -Wl,-Bdynamic -lpthread" at end of link line
337    and produces broken binaries for pdftohtml/pdftoppm/pdftops/pdftopng.
338
339    Note that PNG_LIBRARIES includes zlib/lz: "-lpng -lz", and along with freetype,
340    these are linked statically. However, Threads/lpthread is included as a dynamically
341    linked library instead of including a .a (regardless of whether it's appended
342    as -lpthread or Threads::Threads in the target_link_libraries()), contributing to
343    the pdfhtml binary produced being a partially static, partially dynamic one,
344    so a dynamic executable overall.
345
346    The order of dynamic .so files listed by ldd in the broken static binary of pdftohtml differs from
347    a manually statically linked working version of pdftohtml, and seems to be the only difference
348    between the two in ldd's output. Not using "-Wl,-Bstatic" and using -static (-Bstatic on Mac)
349    in its place creates a partially static dynamic executable that isn't broken, whereas
350    additionally removing "-Wl,-Bdynamic -lpthread" and replacing it with -lpthread
351    moreover produces a working pdftohtml that is a fully static linked executable.
352
353    The inclusion of the math lib and c lib (lm and lc) in the final link command
354    are to completely bypass the remaining .so dependencies that were present in
355    the executable and produce the fully static executable. The lm and lc libs were referenced
356    by all xpdf-tool binaries (as indicated when generating dynamic ones and running ldd over them)
357    but Dr Bainbridge said that -lm and -lc were some libs passed in by the compiler by default,
358    which would explain why explicitly setting them for some xpdftools and not other may not have
359    mattered.
360
361NOTES:
362Initial attempts at modifying gs-CMakeLists.txt for static compiling that proved to be unnecessary:
363
364    (1) Setting -static globally doesn't have a useful effect.
365
366    # We want to build static xpdf-tools binaries. See
367    # https://stackoverflow.com/questions/24648357/compiling-a-static-executable-with-cmake
368    # Want to make the min number of changes for building statically, so using the way
369    # below. Beware, must *append* "-static" to existing CMAKE_EXE_LINKER_FLAGS=LD_FLAGS
370    ##SET(CMAKE_FIND_LIBRARY_SUFFIXES ".a")
371    ##SET(BUILD_SHARED_LIBS OFF)
372    ##SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -static")
373
374    The above 3 lines just add a -static before the "-O2 -Wall -fPIC -rdynamic ..." during linking, such as below.
375    But they have no further effect on whether static building actually succeeds or not. The only effective static
376    linking command (for Linux so far) was to pass -static in the target_link_libraries() command followed by the
377    "-l<libname>" for each library in the correct order.
378
379----
380/usr/bin/c++  -I/home/greenstone/gs3-svn-26Mar2018/gs2build/ext/xpdf-tools/linux/include  -I/home/greenstone/gs3-svn-26Mar2018/gs2build/ext/xpdf-tools/linux/include -I/home/greenstone/gs3-svn-26Mar2018/gs2build/ext/xpdf-tools/linux/include/libpng15 -O3 -Wall -fPIC  -L/home/greenstone/gs3-svn-26Mar2018/gs2build/ext/xpdf-tools/linux/lib  -L/home/greenstone/gs3-svn-26Mar2018/gs2build/ext/xpdf-tools/linux/lib -static ***** <- HERE ****** -O2 -Wall -fPIC -rdynamic CMakeFiles/pdftohtml.dir/HTMLGen.cc.o CMakeFiles/pdftohtml.dir/SplashOutputDev.cc.o CMakeFiles/pdftohtml.dir/TextOutputDev.cc.o CMakeFiles/pdftohtml.dir/pdftohtml.cc.o CMakeFiles/xpdf_objs.dir/AcroForm.cc.o CMakeFiles/xpdf_objs.dir/Annot.cc.o CMakeFiles/xpdf_objs.dir/Array.cc.o CMakeFiles/xpdf_objs.dir/BuiltinFont.cc.o CMakeFiles/xpdf_objs.dir/BuiltinFontTables.cc.o CMakeFiles/xpdf_objs.dir/Catalog.cc.o CMakeFiles/xpdf_objs.dir/CharCodeToUnicode.cc.o CMakeFiles/xpdf_objs.dir/CMap.cc.o CMakeFiles/xpdf_objs.dir/Decrypt.cc.o CMakeFiles/xpdf_objs.dir/Dict.cc.o CMakeFiles/xpdf_objs.dir/Error.cc.o CMakeFiles/xpdf_objs.dir/FontEncodingTables.cc.o CMakeFiles/xpdf_objs.dir/Form.cc.o CMakeFiles/xpdf_objs.dir/Function.cc.o CMakeFiles/xpdf_objs.dir/Gfx.cc.o CMakeFiles/xpdf_objs.dir/GfxFont.cc.o CMakeFiles/xpdf_objs.dir/GfxState.cc.o CMakeFiles/xpdf_objs.dir/GlobalParams.cc.o CMakeFiles/xpdf_objs.dir/JArithmeticDecoder.cc.o CMakeFiles/xpdf_objs.dir/JBIG2Stream.cc.o CMakeFiles/xpdf_objs.dir/JPXStream.cc.o CMakeFiles/xpdf_objs.dir/Lexer.cc.o CMakeFiles/xpdf_objs.dir/Link.cc.o CMakeFiles/xpdf_objs.dir/NameToCharCode.cc.o CMakeFiles/xpdf_objs.dir/Object.cc.o CMakeFiles/xpdf_objs.dir/OptionalContent.cc.o CMakeFiles/xpdf_objs.dir/Outline.cc.o CMakeFiles/xpdf_objs.dir/OutputDev.cc.o CMakeFiles/xpdf_objs.dir/Page.cc.o CMakeFiles/xpdf_objs.dir/Parser.cc.o CMakeFiles/xpdf_objs.dir/PDFDoc.cc.o CMakeFiles/xpdf_objs.dir/PDFDocEncoding.cc.o CMakeFiles/xpdf_objs.dir/PSTokenizer.cc.o CMakeFiles/xpdf_objs.dir/SecurityHandler.cc.o CMakeFiles/xpdf_objs.dir/Stream.cc.o CMakeFiles/xpdf_objs.dir/TextString.cc.o CMakeFiles/xpdf_objs.dir/UnicodeMap.cc.o CMakeFiles/xpdf_objs.dir/UnicodeTypeTable.cc.o CMakeFiles/xpdf_objs.dir/UTF8.cc.o CMakeFiles/xpdf_objs.dir/XFAForm.cc.o CMakeFiles/xpdf_objs.dir/XRef.cc.o CMakeFiles/xpdf_objs.dir/Zoox.cc.o  -o pdftohtml ../goo/libgoo.a ../fofi/libfofi.a ../splash/libsplash.a -static -lfreetype -lpng -lz -lm -lc -lpthread
381----
382
383    (2) Threads::Threads instead of -lpthread results in a partially dynamic executable.
384
385    # The original, unmodified CMakeLists.txt was not set up sufficiently
386    # for static compilation of xpdf-tools. As a result, compile would first fail
387    # with errors about undefined refs to mutex / lpthread.
388    # When building xpdf-tools statically, need to add the following 2 lines as well
389    # as append "Threads::Threads" to the end of each "target_link_libraries(<list>)"
390    # See https://stackoverflow.com/questions/1620918/cmake-and-libpthread
391    # found googling cmake and "-lpthread" (pthread) after ERRORS to do with this, like:
392    #   undefined reference to `pthread_mutex_unlock'
393    ##set(THREADS_PREFER_PTHREAD_FLAG ON)
394    ##find_package(Threads REQUIRED)
395
396    In instances when compilation was successful, including the above 2 lines in combination with "Threads::Threads"
397    as the final argument to every target_link_libraries(...) occurrence in gs-CMakeLists.txt would only manage to
398    produce partially dynamically linked xpdftools binaries. (Depending on what the linking command was when building
399    Xpdf-Tools, the partially dynamically linked executables may work or may be broken. See explanation further above.)
400    We wanted fully statically linked binaries, for which we needed to pass in "-lpthread" as the trailing argument
401    to each target_link_libraries(...). So without either, compilation will fail. However, with "Threads::Threads"
402    the binaries weren't fully static, whereas with -lpthread the xpdftools executables were fully static as CMake no
403    longer tried to link against a dynamic Threads library.
404
405
406(5) To view the unmodified CMakeLists.txt included in the xpdf-4.00 source code tarball, untar it and look for its "xpdf/CMakeLists.txt" (not the toplevel file of the same name).
407Run a 'diff' against gs-CMakeLists.txt to see further differences, such as debug statements and comments. Most comments have been removed and placed into this readme file instead.
408
409
410(6) When CASCADE-MAKE is run on the xpdf-tools GS2-extension, it first compiles up CMake, needed to compile up xpdf-tools.
411Unlike the library packages like freetype, libpng and zlib that we also build for xpdf-tools as part of this gs2-extension, CMake's build products don't need to be included in the distribution tarball of our built xpdf-tools executables.
412
413There's a "move-cmake.sh" script in the xpdf-tools gs2-extension that can be run with the "away" and "back" options to move the CMake stuff out of the way (into a "devel" folder) after successfully building xpdf binaries and that can also be run to move them back if wanting to recompile.
414
415The script can be run manually, but it's also run by the extension:
416- packages/CASCADE-MAKE/XPDFTOOLS.sh runs "move-cmake.sh away" after xpdf-tools has been built, so that the extension's install location is ready for tarring up for distribution.
417- When recompiling the xpdf-tools extenion, the CASCADE-MAKE process will run packages/CASCADE-MAKE/CMAKE.sh file which in turn runs "move-cmake.sh back" if there's a prebuilt CMake which had earlier been moved out of the way.
418
419
420__________________________________________________________
421E. Getting more output when running CMake (verbosity)
422__________________________________________________________
423See https://www.linuxquestions.org/questions/programming-9/cmake-or-make-debug-output-show-command-624800/
424To turn on debugging:
425    export VERBOSE=1
426    ./CASCADE-MAKE.sh
427
428To turn off debugging, need to actually make VERBOSE undefined again (don't set it to 0):
429    export VERBOSE=
430    ./CASCADE-MAKE.sh
431
432
433__________________________________________________________
434F. APPENDIX - Useful links
435__________________________________________________________
436A. Helping CMake along. (Not all of this was necessary for compiling xpdftools statically, but they're generally useful links)
437
438https://github.com/SynoCommunity/spksrc/issues/1779
439https://stackoverflow.com/questions/1620918/cmake-and-libpthread
440https://cmake.org/cmake/help/v3.0/prop_tgt/LINK_FLAGS.html
441https://cmake.org/cmake/help/v3.11/command/target_link_libraries.html?highlight=target_link_libraries
442https://stackoverflow.com/questions/24648357/compiling-a-static-executable-with-cmake
443https://stackoverflow.com/questions/42815420/cmake-cant-find-my-static-libs
444https://cmake.org/cmake/help/v3.0/command/message.html
445https://stackoverflow.com/questions/30980383/cmake-compile-options-for-libpng
446    https://stackoverflow.com/questions/36220123/undefined-reference-to-png-set-longjmp-fn-when-compiling-pcl-source-file
447
448
449B. About the error "bash: no such file or directory" when run on a statically generated binary:
450
451https://askubuntu.com/questions/351827/unable-to-run-a-32-bit-program-on-64-bit-vm/353497#353497
452https://unix.stackexchange.com/questions/13391/getting-not-found-message-when-running-a-32-bit-binary-on-a-64-bit-system/13409#13409
453https://arstechnica.com/civis/viewtopic.php?f=16&t=1173118
454https://superuser.com/questions/344533/no-such-file-or-directory-error-in-bash-but-the-file-exists
455https://unix.stackexchange.com/questions/45277/executing-binary-file-file-not-found
456
457C. Other links
458
459https://unix.stackexchange.com/questions/279397/ldd-dont-find-path-how-to-add
460
461
462D. On why you can't build static binaries on Mac, but can build static libraries and link against them
463
464https://developer.apple.com/library/archive/qa/qa1118/_index.html (official page on how Mac doesn't support static binaries)
465https://stackoverflow.com/questions/3801011/ld-library-not-found-for-lcrt0-o-on-osx-10-6-with-gcc-clang-static-flag
466https://stackoverflow.com/questions/844819/how-to-static-link-on-os-x (mention of -Bstatic)
467https://www.allegro.cc/forums/thread/610923
468https://stackoverflow.com/questions/5259249/creating-static-mac-os-x-c-build (has some other suggestions)
469    http://www.network-theory.co.uk/docs/gccintro/gccintro_79.html
470Dead end: https://nelsonslog.wordpress.com/2013/04/24/macos-doesnt-support-static-binaries/
471https://dropline.net/2015/10/static-linking-on-mac-os-x/
472    explains that on Mac, .dylibs must be hidden for .a versions of libraries to be selected when linking
473    This must be true for non-system dylibs too.
474    This means that where possible we want to essentially do "--enable-static --disable-shared", or equivalent,
475    when generating freetype, libz, libpng, libjpg, libtiff library files, so that Xpdf-Tools links against the
476    .a files we generated rather than additional .dylib files
477
478http://www.simplesystems.org/libtiff/build.html
479configuration options for building libtiff. Want to turn off the compile process for libtiff producing tiff binaries, but there appears to be no such option.
480
481
482__________________________________________________________
483G. LIBJPEG and LIBTIFF
484__________________________________________________________
485
4861. The first version of LIBJPEG to work out was version 6b, which required some patching up before it could be built, see point 2 below.
487Besides the fact that version 6b needed patching up, it was also from 2008. I've now found a version of libjpeg from Jan 2018, called "jpegsrc.v9c.tar.gz"
488which was downloadable from www.ijg.org at http://www.ijg.org/files/jpegsrc.v9c.tar.gz. Version 9c can build both static and dynamically linked libraries of
489libjpeg, though we only want the former. (The older version 6b could only generate the static libjpeg.a library file, and contrary to online instructions.)
490
491As needed to be done with the older 6b version, this tarball was renamed to jpeg-9c.tar.gz to fit the naming pattern of its folder once extracted.
492
493There was an incompatibility between the existing CASCADE-MAKE/LIBJPEG.sh and the Makefile generated by configuring the Makefile.in/.am in the jpeg-9c tarball.
494The LIBJPEG.sh would run "make install-lib"  at the end, to install the libjpeg.a in the lib folder and to install 4 header files. This is as per the install.txt
495instructions in the older and current version of jpeg src tarball. However, the header files never got installed when doing so, whether in version 6b or the
496current 9c. And install-lib is not a recognised target in 9c's Makefile, where the target is install-libLTLIBRARIES. So LIBJPEG.sh has been modified to use this
497target name and to moreover copy over the header files (even though they weren't necessary when compiling xpdftools against the libjpeg 6b library previously and
498possibly now with 9c).
499
500Since we want to only generate libjpeg.a and not the .so/.dylib dynamically linked versions, the latter is turned off during configure by passing --disable-shared.
501
502A final change made to LIBJPEG.sh was to undo it copying over the patch file "gs-libjpeg-config.sub" into the extracted jpeg tarball, since the patch was only
503necessary for libjpeg version 6b and not for 9c. These steps have been commented out in LIBJPEG.sh now.
504
505
5062. Issues building LIBJPEG VERSION 6b on 64 bit machines and the patch
507
508LIBJPEG version 6b is from 2008.
509
510I copied the LIBJPEG package from http://trac.greenstone.org/browser/other-projects/realistic-books/trunk/packages (also at http://trac.greenstone.org/browser/gs2-extensions/ocr/trunk/packages/cmdline).
511
512    * Configuring out of the box produced the following error:
513       checking host system type... Invalid configuration `x86_64-unknown-linux-gnu': machine `x86_64-unknown' not recognized
514
515    * So that, as a consequence, when running make on the libjpeg package, make failed with the error:
516       ./libtool --mode=compile gcc -I/home/greenstone/gs3-svn-26Mar2018/gs2build/ext/xpdf-tools/linux/include -fPIC  -I/home/greenstone/gs3-svn-26Mar2018/gs2build/ext/xpdf-tools/linux/include  -I. -c ./jcapimin.c
517       make: ./libtool: Command not found
518       make: *** [jcapimin.lo] Error 127
519        Error encountered running *make * stage of ./CASCADE-MAKE/LIBJPEG.sh
520
521The same was true when I grabbed the libjpeg from sourceforge (https://sourceforge.net/projects/libjpeg/files/), which was also still version jpeg 6b from 2008.
522
523I found the following webpages discussing the above error messages:
524- https://unix.stackexchange.com/questions/80479/how-to-work-with-libtool
525- https://github.com/rwestlund/freesweep/issues/1
526- https://ubuntuforums.org/showthread.php?t=1232714
527- https://stackoverflow.com/questions/12828687/configure-fails-to-detect-proper-ld-on-a-64-bit-system-with-32-bit-userland
528- SOLUTION: https://sourceforge.net/p/libjpeg/bugs/12/
529
530However, the error only strikes when configure is run with --enable-static.
531
532Note also that contrary to the above pages, running configure with the additional options
533    --host=x86_64-linux-gnu --build=x86_64-linux-gnu --target=x86_64-linux-gnu --disable-shared --enable-static
534did not help. Nor did adding the above flags get rid of configure attempting to work with host=x86_64-unknown(-unknown)-linux-gnu
535
536The SOLUTION, found when searching for the error message along with "enable-static", as it's the combination that is relevant, is described
537at https://sourceforge.net/p/libjpeg/bugs/12/
538
539which was to patch up the config.sub filed included in the jpeg-6b tarball, to also cover x86_64-* machines:
540        tahoe | i860 | x86_64-* | m32r | m68k | m68000 | m88k | ns32k | arc | arm \
541
542The above change is necessary because this libjpeg is outdated and has been superceded by other JPEG libraries, also discussed at https://sourceforge.net/p/libjpeg/bugs/12/
543I'm not sure if those libraries are compatible with XpdfTools however, so I'm sticking with libjpeg as long as I can get it to build and be recognised by XpdfTools.
544
545The solution is once more to have a patch file: CASCADE-MAKE/LIBJPEG.sh replaces the config.sub with in the jpeg-6b package after this is untarred with packages/gs-libjpeg-config.sub, which contains the patch.
546
547
5482. I followed the instructions at http://www.linuxfromscratch.org/blfs/view/6.3/general/libjpeg.html
549to try to build libjpeg with --enable-static and --enable-shared to produce both libjpeg.a and libjpeg.so.
550
551However, nothing I try gets it to generate a libjpeg.so. It seems to always produce a libjpeg.a in xpdf-tools/linux/lib
552regardless of whether CASCADE-MAKE/LIBJPEG.sh passes the --enable-static flag to the configure command or not, and regardless of whether --enable-shared is additionally or individually passed in.
553
554As a consequence, there's no  libjpeg.so file to set the -DJPEG_LIBRARY flag in XPDFTOOLS.sh to for when building xpdf-tools against dynamically linked libraries.
555
556I tried the various combinations with the lib jpeg-6b source tarballs from
557- sourceforge, https://sourceforge.net/projects/libjpeg/files/, the latest tarball of this was from 2008
558- http://www.linuxfromscratch.org/blfs/view/6.3/general/libjpeg.html, which was last updated in 2007
559- http://trac.greenstone.org/browser/other-projects/realistic-books/trunk/packages/jpeg-6b.tar.gz, which was added to trac in 2009 but is probably the 2008 or 2007 version too.
560
561
5623. Modifications for using TIFF and JPEG libraries when building Xpdf-Tools:
563   
564* CASCADE-MAKE.sh, replaced
565    PACKAGES="CMAKE LIBZ LIBPNG FREETYPE XPDFTOOLS"
566with
567    PACKAGES="CMAKE LIBZ LIBTIFF LIBPNG LIBJPEG FREETYPE XPDFTOOLS"
568
569
570* XPDFTOOLS.sh
571If compiling statically make sure the CMake command contains the following changes:
572        -DTIFF_INCLUDE_DIR=$prefix/include \        # <========== new
573        -DJPEG_INCLUDE_DIR=$prefix/include \        # <========== new
574        -DZLIB_LIBRARY=$prefix/lib/libz.a \
575        -DTIFF_LIBRARY=$prefix/lib/libtiff.a \      # <========== new
576        -DPNG_LIBRARY=$prefix/lib/libpng15.a \
577        -DJPEG_LIBRARY=$prefix/lib/libjpeg.a \      # <========== new
578        -DFREETYPE_LIBRARY=$prefix/lib/libfreetype.a \
579        -DGSDLFLAG_STATIC="$static_flag" \
580
581
582
583The above flag names were discovered by deleting the untarred xpdf-4.00 folder.
584Then in a fresh terminal, source devel.bash from xpdf-tools and re-run CASCADE-MAKE.sh without the above modifications:
585
586    -- Found FreeType (new-style includes): /home/greenstone/gs3-svn-26Mar2018/gs2build/ext/xpdf-tools/linux/lib/libfreetype.a
587    -- Found ZLIB: /home/greenstone/gs3-svn-26Mar2018/gs2build/ext/xpdf-tools/linux/lib/libz.a (found version "1.2.8")
588    -- Found PNG: /home/greenstone/gs3-svn-26Mar2018/gs2build/ext/xpdf-tools/linux/lib/libpng15.a (found version "1.2.50")
589    -- Could NOT find JPEG (missing: JPEG_LIBRARY JPEG_INCLUDE_DIR)
590    -- Could NOT find TIFF (missing: TIFF_LIBRARY TIFF_INCLUDE_DIR)
591    -- lcms2 not found
592    -- No Qt library found
593
594
595* packages/gs-CMakeLists.txt was modified again,
596
597    - this time to also pass:
598        -ltiff and -ljpeg to all target_link_libraries() commands that run when GSDLFLAG_STATIC is set
599    and
600        ${TIFF_LIBRARY} and ${JPEG_LIBRARY} to all target_link_libraries() commands that run when GSDLFLAG_STATIC is not set
601
602    - And to add in the include directories and defitions if JPEG/TIFF libraries were provided:
603        if (JPEG_FOUND)
604          include_directories("${JPEG_INCLUDE_DIR}")
605          add_definitions("${JPEG_DEFINITIONS}")
606          message(STATUS "@@@@@@@@@@@@@@@ JPEG_FOUND (include_dir ; include_dirs): ${JPEG_INCLUDE_DIR} ; ${JPEG_INCLUDE_DIRS}")
607        else ()
608          message(STATUS "@@@@@@@@@@@@@@@ NO JPEG_FOUND")
609        endif ()
610        if (TIFF_FOUND)
611          include_directories("${TIFF_INCLUDE_DIRS}")
612          add_definitions("${TIFF_DEFINITIONS}")
613          message(STATUS "@@@@@@@@@@@@@@@ TIFF_FOUND ${TIFF_INCLUDE_DIRS}")
614        else ()
615          message(STATUS "@@@@@@@@@@@@@@@ NO TIFF_FOUND")
616        endif ()
617
618    Note however that although gs-CMakeLists.txt now knows what the pluralised TIFF_INCLUDE_DIRS is (and TIFF_INCLUDE_DIR)
619    as for PNG and ZLIB, gs-CMakeLists.txt does not have a value for the pluralised JPEG_INCLUDE_DIRS, only the
620    JPEG_INCLUDE_DIRS set above. And both the CMAKE flags in XPDFTOOLS.sh for tiff and jpeg libs seem to have been setup
621    in the same way now. Not sure where these automatically assigned variables come from in order to check up on them.
622
623__________________________________________________________
624H. Licensing information and making the distributable tarball
625__________________________________________________________
626
627XpdfTools' README lists which files need to be included as per its license when redistributing xpdf-tools binaries.
628
629Running "./CASCADE-MAKE.sh makedist" does some custom steps to create a distribution tarball of the xpdf-tools we compile up:
630it creates a temporary xpdf-tools subdirectory and copies into it the GSDLOS/bin/pdf* statically linked binaries (or dynamic executables linked
631against mostly static libraries in the case of Macs), the GSDLOS/man folder as well as the "redistribute" folder containing further compulsory
632files as required by xpdf-tools' license.
633This temporary xpdf-tools subfolder is then passed as the whitelist folder to the CASCADE-MAKE makedist command, and this folder and its
634contents are then tarred up into the distribution tarball xpdf-tools-GSDLOS.tar.gz. Finally, the temporary xpdf-tools subfolder is deleted.
635
636
637__________________________________________________________
638I. PDF2DOM: tried it out, but wasn't what we wanted
639__________________________________________________________
640Using PDFBox to convert a PDF to full HTML, both images and text and placed correctly with respect to each other, is tricky, see https://stackoverflow.com/questions/9671239/pdfbox-convert-a-pdf-to-text-or-html-including-images-from-the-pdf
641(Google: pdfbox to convert pdf to html with images)
642
643PDF2DOM tool (based on PDFBox) to convert PDF to HTML with images
644* http://cssbox.sourceforge.net/pdf2dom/documentation.php
645* Got the command line jar tool, PDFToHTML.jar version 1.7, from https://sourceforge.net/projects/cssbox/files/Pdf2DOM/
646* Further information and source code at https://github.com/radkovo/Pdf2Dom
647* API: http://cssbox.sourceforge.net/pdf2dom/api/index.html
648
649
6501. Running
651
652java -jar PDFToHTML.jar <infile> [<outfile>]
653
654    greenstone@machine-name:~/Downloads$ java -jar PDFToHTML.jar SampleDoc1.pdf -im=SAVE_TO_DIR -idir=/home/greenstone/Downloads/tmp1 -fm=SAVE_TO_DIR -fdir=/home/greenstone/Downloads/tmp2
655
656
657It will output the page, but you'll see the following output indicating that the logger is not displaying anything:
658    SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
659    SLF4J: Defaulting to no-operation (NOP) logger implementation
660    SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
661
662See https://stackoverflow.com/questions/7421612/slf4j-failed-to-load-class-org-slf4j-impl-staticloggerbinder
663
664To see error output download SLF4J simple jar, run as follows:
665
666    greenstone@machine-name:~/Downloads$ java -classpath slf4j-simple-1.7.25.jar:PDFToHTML.jar org.fit.pdfdom.PDFToHTML ApacheLicencePDFA.pdf -im=SAVE_TO_DIR -idir=/home/greenstone/Downloads/tmp1 -fm=SAVE_TO_DIR -fdir=/home/greenstone/Downloads/tmp2
667
668The above is a MS Word produced PDF (archive format) and works fine: font folder generated containing the extracted fonts
669
670The following is a PDF produced from the same doc file by the latest libreoffice installed on Windows:
671    ApacheLicencePDFA_FromODT.pdf
672But running the same command on it produces the following font errors:
673
674greenstone@machine-name:~/Downloads$ java -classpath slf4j-simple-1.7.25.jar:PDFToHTML.jar org.fit.pdfdom.PDFToHTML ApacheLicencePDFA_FromODT.pdf -im=SAVE_TO_DIR -idir=/home/greenstone/Downloads/tmp1 -fm=SAVE_TO_DIR -fdir=/home/greenstone/Downloads/tmp2
675[main] INFO org.reflections.Reflections - Reflections took 163 ms to scan 1 urls, producing 36 keys and 222 values
676[main] WARN org.fit.pdfdom.FontTable - Error loading font 'BAAAAA+Georgia' Message: FontVerter could not detect the input font's type. class java.io.IOException
677[main] WARN org.fit.pdfdom.FontTable - Error loading font 'CAAAAA+Georgia-Bold' Message: FontVerter could not detect the input font's type. class java.io.IOException
678[main] WARN org.fit.pdfdom.FontTable - Error loading font 'BAAAAA+Georgia' Message: FontVerter could not detect the input font's type. class java.io.IOException
679[main] WARN org.fit.pdfdom.FontTable - Error loading font 'CAAAAA+Georgia-Bold' Message: FontVerter could not detect the input font's type. class java.io.IOException
680
681Fonts get extracted if the source PDF was generated by MS Word's doc to PDF conversion. Fonts didn't get extracted from PDF upon conversion to HTML when libreoffice was used to convert a .doc to the source PDF.
682
6832. Check version of PDF
684https://www.codeproject.com/Questions/167550/How-to-check-different-versions-of-PDF
685
686
6873. pdf to html command line conversion open source
688https://stackoverflow.com/questions/8370014/how-to-convert-pdf-to-html
689
690"Download
691
692    pdfbox-2.0.3.jar
693    fontbox-2.0.3.jar
694    preflight-2.0.3.jar
695    xmpbox-2.0.3.jar
696    pdfbox-tools-2.0.3.jar
697    pdfbox-debugger-2.0.3.jar
698
699from http://pdfbox.apache.org/
700...
701
702PLEASE NOTE: Images do not get pushed to the HTML output."
703
704
7054. Need a way to check if PDF contains images, then use pdf2dom, else basic pdfbox conversion to html (less div tags with inline style markup)?
706https://stackoverflow.com/questions/46215879/count-images-in-pdf-using-pdfbox
707
708
709UNUSED
710Googled for: java tool convert pdf version
711* https://stackoverflow.com/questions/11137912/all-inclusive-tool-to-convert-different-types-of-documents-to-pdf
712* https://www.qoppa.com/pdfprocess/
713jPDFProcess – Java PDF Library to Create, Manipulate PDF
714(appears to be payware)
715* https://www.gnostice.com/nl_article.asp?id=95&t=How_to_Change_the_PDF_Version_of_a_Document
716How to Convert a PDF Document to an Older or Newer Version
717uses .NET
718* http://www.baeldung.com/pdf-conversions-java
719PDF Conversions in Java
720e.g. PDF to html and html to PDF
721
722
723__________________________________________________________
724
725greenstone@machine-name:~/Downloads$ java -classpath slf4j-simple-1.7.25.jar:PDFToHTML.jar org.fit.pdfdom.PDFToHTML SampleDoc1.pdf -im=SAVE_TO_DIR -idir=/home/greenstone/Downloads/tmp1 -fm=SAVE_TO_DIR -fdir=/home/greenstone/Downloads/tmp2
726[main] INFO org.reflections.Reflections - Reflections took 153 ms to scan 1 urls, producing 36 keys and 222 values
727[main] WARN org.fit.pdfdom.FontTable - Error loading font 'BAAAAA+Georgia' Message: FontVerter could not detect the input font's type. class java.io.IOException
728[main] WARN org.fit.pdfdom.FontTable - Error loading font 'CAAAAA+Georgia-Bold' Message: FontVerter could not detect the input font's type. class java.io.IOException
729[main] WARN org.fit.pdfdom.FontTable - Error loading font 'BAAAAA+Georgia' Message: FontVerter could not detect the input font's type. class java.io.IOException
730[main] WARN org.fit.pdfdom.FontTable - Error loading font 'CAAAAA+Georgia-Bold' Message: FontVerter could not detect the input font's type. class java.io.IOException
731
732
733
734greenstone@machine-name:~/Downloads$ java -classpath Pdf2Dom/target/pdf2dom-1.8-SNAPSHOT.jar:pdfbox-app.jar:slf4j-jdk14-1.6.6.jar:log4j-over-slf4j-1.6.6.jar:slf4j-api-1.6.6.jar  org.fit.pdfdom.PDFToHTML SampleDoc1.pdf -im=SAVE_TO_DIR -idir=/home/greenstone/Downloads/tmp1 -fm=SAVE_TO_DIR -fdir=/home/greenstone/Downloads/tmp2
735Exception in thread "main" java.lang.NoClassDefFoundError: org/mabb/fontverter/FontVerter
736    at org.fit.pdfdom.FontTable$Entry.loadTrueTypeFont(FontTable.java:178)
737    at org.fit.pdfdom.FontTable$Entry.getData(FontTable.java:147)
738    at org.fit.pdfdom.FontTable$Entry.isEntryValid(FontTable.java:161)
739    at org.fit.pdfdom.FontTable.addEntry(FontTable.java:48)
740    at org.fit.pdfdom.PDFBoxTree.processFontResources(PDFBoxTree.java:378)
741    at org.fit.pdfdom.PDFBoxTree.updateFontTable(PDFBoxTree.java:361)
742    at org.fit.pdfdom.PDFDomTree.updateFontTable(PDFDomTree.java:544)
743    at org.fit.pdfdom.PDFBoxTree.processPage(PDFBoxTree.java:206)
744    at org.apache.pdfbox.text.PDFTextStripper.processPages(PDFTextStripper.java:319)
745    at org.apache.pdfbox.text.PDFTextStripper.writeText(PDFTextStripper.java:266)
746    at org.fit.pdfdom.PDFDomTree.createDOM(PDFDomTree.java:218)
747    at org.fit.pdfdom.PDFDomTree.writeText(PDFDomTree.java:194)
748    at org.fit.pdfdom.PDFToHTML.main(PDFToHTML.java:77)
749Caused by: java.lang.ClassNotFoundException: org.mabb.fontverter.FontVerter
750    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
751    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
752    at java.security.AccessController.doPrivileged(Native Method)
753    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
754    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
755    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
756    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
757    ... 13 more
758greenstone@machine-name:~/Downloads$
Note: See TracBrowser for help on using the browser.