Parallel Ingest with pingest.pl
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
A program named pingest.pl is now installed to allow faster record
ingest.  It performs ingest in parallel so that multiple batches can
be done simultaneously.  It operates by splitting the records to be
ingested up into batches and running all of the ingest methods on each
batch.  You may pass in options to control how many batches are run at
the same time, how many records there are per batch, and which ingest
operations to skip.

NOTE: The browse ingest is presently done in a single process over all
of the input records as it cannot run in parallel with itself.  It
does, however, run in parallel with the other ingests.

Command Line Options
++++++++++++++++++++
pingest.pl accepts the following command line options:

--host::
    The server where PostgreSQL runs (either host name or IP address).
    The default is read from the PGHOST environment variable or
    "localhost."

--port::
    The port that PostgreSQL listens to on host.  The default is read
    from the PGPORT environment variable or 5432.

--db::
    The database to connect to on the host.  The default is read from
    the PGDATABASE environment variable or "evergreen."

--user::
    The username for database connections.  The default is read from
    the PGUSER environment variable or "evergreen."

--password::
    The password for database connections.  The default is read from
    the PGPASSWORD environment variable or "evergreen."

--batch-size::
    Number of records to process per batch.  The default is 10,000.

--max-child::
    Max number of worker processes (i.e. the number of batches to
    process simultaneously).  The default is 8.

--skip-browse::
--skip-attrs::
--skip-search::
--skip-facets::
--skip-display::
    Skip the selected reingest component.

--start-id::
    Start processing at this record ID.

--end-id::
    Stop processing when this record ID is reached.

--pipe::
    Read record IDs to reingest from standard input.  This option
    conflicts with --start-id and/or --end-id.

--max-duration::
    Stop processing after this many total seconds have passed.  The
    default is to run until all records have been processed.

--help::
    Show the help text.