libextractor

GNU libextractor
Log | Files | Refs | Submodules | README | LICENSE

README (4714B)


      1 About
      2 =====
      3 
      4 GNU libextractor is a library used to extract meta data from files of
      5 arbitrary type.  It is designed to use helper-libraries to perform the
      6 actual extraction, and to be trivially extendable by linking against
      7 external extractors for additional file types.  Common use-cases for
      8 GNU libextractor include detail-views in file managers, detailed
      9 search results in file-sharing networks and general information
     10 gathering in forensics investigations and penetration testing.
     11 
     12 GNU libextractor is a simple C library with a small API.  Bindings for
     13 GNU libextractor exists for many languages in addition to the standard
     14 C/C++ API (we know about bindings for Java, Perl, PHP, Mono, Python,
     15 Ruby).
     16 
     17 libextractor uses a plugin mechanism to enable developers to quickly
     18 add extractors for additional formats.  Plugins are executed
     19 out-of-process and can thus bugs in them (or the libraries that they
     20 use) cannot crash the main application.  libextractor typically ships
     21 with a few dozen plugins that can be used to obtain keywords from
     22 common file types.
     23 
     24 More detailed documentation is available in the GNU libextractor
     25 manual.  libextractor is an official GNU package and available from
     26 http://www.gnu.org/s/libextractor/.
     27 
     28 
     29 Joining GNU
     30 ===========
     31 
     32 This is a GNU program, developed by the GNU Project and part of the
     33 GNU Operating System. If you are the author of an awesome program and
     34 want to join us in writing Free Software, please consider making it an
     35 official GNU program and become a GNU maintainer.  You can find
     36 instructions on how to do so at http://www.gnu.org/help/evaluation.
     37 We are looking forward to hacking with you!
     38 
     39 
     40 extract
     41 =======
     42 
     43 extract is a simple command-line interface to GNU libextractor.
     44 
     45 
     46 Dependencies
     47 ============
     48 
     49 * GNU C/C++ compiler
     50 * libltdl 2.2.x (from GNU libtool)
     51 * GNU libtool 2.2 or higher
     52 * GNU gettext
     53 
     54 The following dependencies are all optional, but should be
     55 available in order for maximum coverage:
     56 
     57 * libarchive
     58 * libavformat / libavcodec / libavutil / libswscale / libswresample
     59   (FFmpeg 5.1 or later; for the video thumbnail and audio preview plugins)
     60 * libbz2 (bzip2)
     61 * libexiv2
     62 * libflac
     63 * libgif (giflib)
     64 * libglib (glib)
     65 * libgtk+
     66 * libgsf
     67 * libgstreamer (1.8 or later, with gst-plugins-base and gst-plugins-good;
     68   the video thumbnail and audio preview plugins additionally need the
     69   gstreamer-video, gstreamer-app and gstreamer-controller libraries, and
     70   at run time the "playback", "videoconvertscale", "jpeg", "opus" and
     71   "ogg" GStreamer plugins)
     72 * libjpeg (v8 or later)
     73 * libmagic (file)
     74 * libmpeg2
     75 * librpm
     76 * libsmf
     77 * libtidy
     78 * libtiff
     79 * libvorbis / libogg
     80 * libz (zlib)
     81 
     82 When building libextractor binaries, please make sure all of these
     83 dependencies are available and configure detects a sufficiently recent
     84 installation.  Otherwise the build system may automatically build only
     85 a subset of GNU libextractor resulting in mediocre meta data
     86 production.
     87 
     88 The sixteen plugins meant for a fast pass over a large volume (pecoff,
     89 lnk, sqlite, tar, iso9660, diskimage, heif, webp, plist, id3, gpx, kml,
     90 geotiff, mbox, apk and ebook) parse headers themselves and need none of
     91 the libraries above, so they are always built.  The two exceptions are
     92 apk and ebook, which read deflated members of a zip container and so
     93 need zlib.
     94 
     95 Regenerating the files those plugins are tested against is not part of
     96 a normal build; the scripts under contrib/ that do it note the extra
     97 tools they need (mingw-w64, qemu-utils, lame, ffmpeg).
     98 
     99 Finally, 'zzuf' is a fuzzing tool that can optionally be detected by
    100 the build system and be used for debugging / testing.  It is not required
    101 at runtime or for normal builds.
    102 
    103 
    104 Installing on macOS with Homebrew
    105 =================================
    106 
    107 The libextractor Git repository is also a Homebrew tap.  Its formula
    108 builds the current development version with every plugin whose
    109 library Homebrew provides, which pulls in many other formulae (among
    110 them FFmpeg, GStreamer and GTK):
    111 
    112   brew tap gnu/libextractor git://git.gnunet.org/libextractor.git
    113   brew trust --tap gnu/libextractor
    114   brew install --HEAD gnu/libextractor/libextractor
    115 
    116 Tapping clones the whole repository (about 100 MB).  'brew upgrade'
    117 only updates a HEAD installation when asked to look for new commits:
    118 
    119   brew upgrade --fetch-HEAD gnu/libextractor/libextractor
    120 
    121 Homebrew's own 'libextractor' formula ('brew install libextractor')
    122 installs the latest release instead, with only the plugins that need
    123 no further libraries.  Both install the same files, so uninstall one
    124 before installing the other, with 'brew uninstall libextractor' (add
    125 '--ignore-dependencies' if formulae such as gnunet depend on it).
    126 Formulae that depend on libextractor accept either.