Evergreen.git
2 years agoLP#1373690: add USE_ID_FOR_OWNING_LIB EDI attribute
Galen Charlton [Tue, 22 Aug 2017 20:13:54 +0000 (16:13 -0400)]
LP#1373690: add USE_ID_FOR_OWNING_LIB EDI attribute

This attribute specifies that the copy owning library
should have its ID emitted rather than its shortname, which can
be used for Baker & Taylor, which imposes a five-character limit
on certain fields. This takes effect only when the INCLUDE_OWNING_LIB
EDI attribute is also in effect.

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
2 years agoLP#1373690: fix typos
Galen Charlton [Tue, 22 Aug 2017 20:03:24 +0000 (16:03 -0400)]
LP#1373690: fix typos

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
2 years agoLP#1373690: normalize IMD values better
Galen Charlton [Tue, 22 Aug 2017 20:00:47 +0000 (16:00 -0400)]
LP#1373690: normalize IMD values better

The EDItEUR book ORDERS message allows two 35-character
item description data elements in IMD segments; this patch
accounts for that. It also, for IMD fields, uses the EDIFACT
release character to escape certain characters, better matching
how the Ruby EDI translator did it.

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
2 years agoLP#1373690 EDI 'use_attrs' configuration option
Bill Erickson [Fri, 18 Aug 2017 15:51:39 +0000 (11:51 -0400)]
LP#1373690 EDI 'use_attrs' configuration option

Adds a new configuration option for EDI accounts specifying wether
purchase orders delivered via a given account should be constructed
using the new EDI Attributes in lieu of the traditional JEDI A/T
template.

The PurchaseOrderEDIRequired A/T validator now bypasses orders with
use_attrs=true accounts.  The new edi_order_pusher.pl script only
process providers whose default EDI accounts have use_attrs=true (except
when a specific PO ID is provided).

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
2 years agoLP#1373690 Midwest Library Service EDI attrs
Bill Erickson [Thu, 22 Jun 2017 18:49:14 +0000 (14:49 -0400)]
LP#1373690 Midwest Library Service EDI attrs

Adds a new default collection of EDI attributes for Midwest Library
Service.  This includes 2 new attributes to support their EDI format:

1. BUYER_ID_ONLY_VENDCODE -- similar to BUYER_ID_INCLUDE_VENDCODE,
   except the buyer SAN is excluded from the NAD+BY segement.

2. INCLUDE_EMPTY_IMD_VALUES -- Render IMD+ fields (title, author,
   edition, etc.) as if they contain values even when they don't.  Empty
   values are included as single-space strings.

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
2 years agoLP#1373690 EDI attributes release notes
Bill Erickson [Thu, 3 Nov 2016 15:19:36 +0000 (11:19 -0400)]
LP#1373690 EDI attributes release notes

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
2 years agoLP#1373690 EDI attribute sets admin UI
Bill Erickson [Wed, 8 Mar 2017 17:47:55 +0000 (12:47 -0500)]
LP#1373690 EDI attribute sets admin UI

1. Create new attribute sets
2. Rename attribute Sets.
3. Enable / Disable attributes for each attributes set.

Found under Admin -> Acquisitions -> EDI Attribute Sets.

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
2 years agoLP#1373690 EDI ORDERS generator script
Bill Erickson [Fri, 27 May 2016 19:46:27 +0000 (15:46 -0400)]
LP#1373690 EDI ORDERS generator script

Adds a new edi_order_pusher.pl script for generating EDI ORDERS from
purchase orders via EDIWriter.pm.  This is a replacement for
edi_pusher.pl, but edi_pusher.pl remains for backwards compat.

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
2 years agoLP#1373690 Attribute-based EDI generator
Bill Erickson [Wed, 25 May 2016 21:40:17 +0000 (17:40 -0400)]
LP#1373690 Attribute-based EDI generator

New Perl module Utils::EDIWriter for buliding EDI ORDERS messages.

Vendor-specific toggles live in new database tables (acq.edi_attr,
acq.edi_attr_set, acq.edi_attr_set_map).

The combination of these 2 replaces the current JEDI Action/Trigger
template with toggle embedded in the template.

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
2 years agoLP#1710949: add tests for blocking after failed attempts
Galen Charlton [Fri, 1 Sep 2017 16:31:41 +0000 (12:31 -0400)]
LP#1710949: add tests for blocking after failed attempts

Enhance the live tests to verify that attempts to log in
get blocked after too many failed ones; this assumes
a stock value for block_count.

General testing plan:

[1] Use srfsh to test, e.g.,

srfsh# request open-ils.auth open-ils.auth.login {"identifier":"99999389406","password":"12345","type":"opac","org":1}

[2] Try various scenarios:

    - wrong password
    - type = staff but user doesn't have STAFF_LOGIN
    - repeated failed attempts
    - barcode is inactive
    - using "identifier" in conjunction with opac.barcode_regex set
    - using "username" or "barcode"
    - supplying a workstation name that exists
    - ... or does not
    - user is inactive
    - user is marked barrred

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
2 years agoLP#1710949 Release notes for auth.login
Bill Erickson [Wed, 16 Aug 2017 14:55:26 +0000 (10:55 -0400)]
LP#1710949 Release notes for auth.login

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
2 years agoLP#1710949 Redact open-ils.auth.login params
Bill Erickson [Tue, 15 Aug 2017 21:45:19 +0000 (17:45 -0400)]
LP#1710949 Redact open-ils.auth.login params

Add open-ils.auth.login to the list of API calls whose parameters should
not be logged.

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
2 years agoLP#1710949 auth.login Perl live test script
Bill Erickson [Tue, 15 Aug 2017 19:21:01 +0000 (15:21 -0400)]
LP#1710949 auth.login Perl live test script

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
2 years agoLP#1710949 open-ils.auth.login API
Bill Erickson [Tue, 15 Aug 2017 17:13:03 +0000 (13:13 -0400)]
LP#1710949 open-ils.auth.login API

Adds a new open-ils.auth API call 'open-ils.auth.login' which performs
the combined steps of open-ils.auth.authenticate.init and
open-ils.auth.authenticate.complete so the caller only need call one API
to login.

API params are consistent with open-ils.auth.authenticate.complete with
2 notable excpetions.  The API uses the bare password instead of the
hashed password, so the caller also need not perform the extra hashing
steps.  Also, no 'nonce' parameter is used as it's no longer needed,
because there is no intermediate authentication cache object as with
.init.

Response data is consistent with open-ils.auth.authenticate.complete.

Example:

srfsh# request open-ils.auth open-ils.auth.login {"username":"admin","password":"fakepassword"}

Other changes in the new code:

1. Using the generic "identifier" parameter in combination with the
   "org" parameter allows the API to reliably determine if a value
   is a username or barcode.

2. Once a caller has reached the configured maximum number of login
   failures, no further attempts to track failures occurs, based on the
   idea that no additional cpu/network cycles should be used on a lost
   cause.

3. A failure count object is only added to memcache when failures
   occur, unlike open-ils.auth.authenticate.init which creates a
   failure tracking object for every login.

4. The code avoids use of the jsonParseFmt() and va_list_to_string()
   functions as these functions require extra data cleansing.

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
2 years agoLP#1708291: tweak to subscription selector
Galen Charlton [Tue, 29 Aug 2017 20:11:13 +0000 (16:11 -0400)]
LP#1708291: tweak to subscription selector

When entering the Manage Predictions or Manage Issues tab
and only one subscription is present in the current OU
scope, automatically select it rather than making the operator
have to select it manually.

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Signed-off-by: Dan Wells <dbw2@calvin.edu>
2 years agoLP#1708291: remove initials field for serial notes
Jason Etheridge [Fri, 4 Aug 2017 17:51:36 +0000 (13:51 -0400)]
LP#1708291: remove initials field for serial notes

This was a carry-over from the copy notes modal that doesn't
apply to serial notes.

Signed-off-by: Jason Etheridge <jason@EquinoxInitiative.org>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Signed-off-by: Dan Wells <dbw2@calvin.edu>
2 years agoLP#1708291: add release notes
Galen Charlton [Fri, 4 Aug 2017 18:02:59 +0000 (14:02 -0400)]
LP#1708291: add release notes

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Signed-off-by: Dan Wells <dbw2@calvin.edu>
2 years agoLP#1708291: web staff client serials module
Galen Charlton [Thu, 13 Apr 2017 16:03:52 +0000 (12:03 -0400)]
LP#1708291: web staff client serials module

This patch adds a serials module to the web staff client, implementing
a unified serials interface allowing for the following actions supported
by the XUL staff client:

- creating subscriptions, distributions, and streams
- creating and editing prediction patterns
- receiving serial issues, with or without barcodes (units)
- batch and quick receiving

This module also implements some new features, including

- the ability to save prediction pattern codes as templates
  that can be shared and reused within an Evergreen database
- a more streamlined interface for managing subscriptions,
  distributions, and streams
- it is no longer necessary to create a starting issue in
  order to predict a run of issues; the dialog box for
  generating a set of predicted issues now lets you specify
  the starting point directly.
- the ability to more directly edit MFHDs

The new serials interfaces can be accessed from the record
details page via a Serials drop-down button that links to
a subscription management page, a quick-receive action, and
a MFHD management page. There is also a new Serials Administration
page where prediction pattern and serial copy templates can
be managed.

To test
-------
* Create, edit, and delete subscriptions, distribution streams,
  and routing lists.
* Use the prediction pattern wizard to create patterns.
* Save prediction pattern templates and use them to apply
  a pattern to new subscriptions.
* Verify that sets of issues can be predicted and received.
* Create and apply serial copy templates and verify that
  they are applied when receiving barcoded issues.

This patch represents a group coding effort by Galen Charlton,
Jason Etheridge, and Mike Rylander.

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Conflicts:
Open-ILS/src/sql/Pg/950.data.seed-values.sql
Open-ILS/web/js/ui/default/staff/cat/catalog/app.js

Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Signed-off-by: Dan Wells <dbw2@calvin.edu>
2 years agoLP#1708291: introduce egI18N
Galen Charlton [Thu, 29 Jun 2017 21:09:54 +0000 (17:09 -0400)]
LP#1708291: introduce egI18N

egI18N is a module that will serve as a grab-bag of functions
related to I18N and L10N. The initial function it provides
takes a acpl IDL object and returns a formatted name qualified
by the org unit, with the underlying template accessible
to the translation subsystem.

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Conflicts:
Open-ILS/src/templates/staff/base_js.tt2
Open-ILS/web/js/ui/default/staff/Gruntfile.js

Signed-off-by: Dan Wells <dbw2@calvin.edu>
2 years agoLP#1708291: teach egBasicComboBox and egDatePicker to accept focusMe
Galen Charlton [Thu, 6 Jul 2017 21:14:15 +0000 (17:14 -0400)]
LP#1708291: teach egBasicComboBox and egDatePicker to accept focusMe

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Conflicts:
Open-ILS/web/js/ui/default/staff/services/ui.js

Signed-off-by: Dan Wells <dbw2@calvin.edu>
2 years agoLP#1152753: make it possible for serial units to be added to copy buckets
Mike Rylander [Mon, 24 Apr 2017 16:40:37 +0000 (12:40 -0400)]
LP#1152753: make it possible for serial units to be added to copy buckets

This patch replaces the baseline asset.copy.id fkey constraints with ones
that understand inheritance, and change all existing contstraints to do the
same via upgrade script.

To test
-------
Create some serial units and verify that they can be added to a copy
bucket.

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Signed-off-by: Dan Wells <dbw2@calvin.edu>
2 years agoLP#1708291: add an egHelpPopover directive
Galen Charlton [Tue, 20 Jun 2017 22:10:46 +0000 (18:10 -0400)]
LP#1708291: add an egHelpPopover directive

This directive allows adding help text accessible by clicking
on a question mark icon.  Example usage:

<eg-help-popover help-text="Use the Force, Leia!">

to create simple popover or

<eg-help-popover help-text="Learn MARC21"
                 help-link="https://www.loc.gov/marc"
>

to have the help text hyperlinked.

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Signed-off-by: Dan Wells <dbw2@calvin.edu>
2 years agoLP#1708291: add a join filter for angular templates
Jason Etheridge [Tue, 30 May 2017 15:51:51 +0000 (11:51 -0400)]
LP#1708291: add a join filter for angular templates

Signed-off-by: Jason Etheridge <jason@equinoxinitiative.org>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Signed-off-by: Dan Wells <dbw2@calvin.edu>
2 years agoLP#1708291: teach egEmbedFrame about an afterload function
Mike Rylander [Thu, 25 May 2017 00:37:17 +0000 (20:37 -0400)]
LP#1708291: teach egEmbedFrame about an afterload function

The 'afterload' attribute added by this patch allows specifying
the name of a function (to be supplied by the embedded content)
to run after the frame has been loaded.

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Signed-off-by: Dan Wells <dbw2@calvin.edu>
2 years agoLP#1708291: add API for safe deleting various serial records
Jason Etheridge [Mon, 22 May 2017 21:28:17 +0000 (17:28 -0400)]
LP#1708291: add API for safe deleting various serial records

This adds routines for safely deleting subscriptions,
distributions and streams.

open-ils.serial.subscription.safe_delete
open-ils.serial.distribution.safe_delete
open-ils.serial.stream.safe_delete
open-ils.serial.subscription.safe_delete.dry_run
open-ils.serial.distribution.safe_delete.dry_run
open-ils.serial.stream.safe_delete.dry_run

Won't delete if there are attached serial.item records with a status other
than Expected, or if there are any attached non-deleted serial.unit
records.

The dry_run variants behave exactly the same except they don't actually
delete anything.

Signed-off-by: Jason Etheridge <jason@equinoxinitiative.org>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Signed-off-by: Dan Wells <dbw2@calvin.edu>
2 years agoLP#1708291: teach MARC editor that it can edit MFHDs/SREs
Galen Charlton [Tue, 23 May 2017 21:59:18 +0000 (17:59 -0400)]
LP#1708291: teach MARC editor that it can edit MFHDs/SREs

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Signed-off-by: Dan Wells <dbw2@calvin.edu>
2 years agoLP#1708291: teach egGrid to always show checkbox menu items
Mike Rylander [Thu, 18 May 2017 23:24:45 +0000 (19:24 -0400)]
LP#1708291: teach egGrid to always show checkbox menu items

This patch also tweaks some styles.

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Signed-off-by: Dan Wells <dbw2@calvin.edu>
2 years agoLP#1708291: improvements to egEditFmRecord
Galen Charlton [Wed, 10 May 2017 21:36:44 +0000 (17:36 -0400)]
LP#1708291: improvements to egEditFmRecord

egEditFmRecord now knows how to specify that a custom Angular template
be used to supply the input widget for a given field; the initial use
of this will be allowing the prediction pattern template editor to be
used to set the pattern in a pattern template. The customFieldTemplates
attribute is used for this purpose.

This patch also teaches egEditFmRecord when to allow an org unit
selector to default to the workstation OU. The orgDefaultAllowed
attribute is used for this purpose.

Finally, a fixes a bug that ensures that the Save button is active
only when the entire form is valid.

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Signed-off-by: Dan Wells <dbw2@calvin.edu>
2 years agoLP#1708291: add a egShareDepthSelector directive
Galen Charlton [Tue, 9 May 2017 18:56:04 +0000 (14:56 -0400)]
LP#1708291: add a egShareDepthSelector directive

This directive implements a selector for OU-sharing depths; depths
and names come from the actor.org_unit_type table. If there are
multiple types defined for a given depth, the display value in
the selector is the concatenation of their names.

The initial use of this directive is for specifying how broadly
prediction pattern templates should be seen.

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Signed-off-by: Dan Wells <dbw2@calvin.edu>
2 years agoLP#1708291: teach egIDL a new fieldmapper object representation
Galen Charlton [Fri, 21 Apr 2017 16:05:45 +0000 (12:05 -0400)]
LP#1708291: teach egIDL a new fieldmapper object representation

This patch teaches egIDL how to represent fieldmapper objects
as objects with attributes rather than getter/setters. This
allows FM objects to be more easily bound to common to Angular
input widgets, avoiding the verbosity of getterSetter ng-options
and the fact that some widgets like uib-datepicker-popup don't
support getterSetter in the first place.

Usage is:

  hash = obj.toTypedHash()

and

  obj = new egCore.idl.fromTypedHash(hash);

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Signed-off-by: Dan Wells <dbw2@calvin.edu>
2 years agoLP#1682609: upgrade to angular-ui-bootstrap 1.3.3
Galen Charlton [Thu, 13 Apr 2017 19:33:06 +0000 (15:33 -0400)]
LP#1682609: upgrade to angular-ui-bootstrap 1.3.3

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Signed-off-by: Dan Wells <dbw2@calvin.edu>
2 years agoLP#1709521 Apply recent patrons from checkout
Bill Erickson [Fri, 1 Sep 2017 01:14:32 +0000 (21:14 -0400)]
LP#1709521 Apply recent patrons from checkout

Load the max recent patrons setting directly in the patron service
instead of requring the calling UI to apply it.  This fixes an issue
where recent patrons are not set from the checkout-by-barcode
interface.

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agoLP1655158 Patron Search by Date of Birth
blake [Thu, 31 Aug 2017 20:33:17 +0000 (20:33 +0000)]
LP1655158 Patron Search by Date of Birth

Adds three UI boxes to the WBSC "Show Extra" patron search. One for the year, month and day.
The javascript on the page is altered to deliver group "4" to the backend. Local javascript
strips out non-numeric user entered data. The backend is updated to handle the new group.
SQL is genereated using the DATE_PART postgres function.

1. Open the web based staff client and browse to the patron search UI.
2. Click the show more down arrow button. Notice the lack of birth date field.
3. Apply the patch, repeat step one. Notice the addition of birth date boxes.
4. Type 1975 into the birth year box and press enter. Notice search results.
5. Try searching for partial names and partial birthdates.
6. Try entering non-numeric data into the birth date boxes.
7. Try searching for patrons without including the dob. Try with only the dob. Try a mix.

Signed-off-by: blake <blake@mobiusconsortium.org>
Signed-off-by: Bill Erickson <berickxx@gmail.com>
2 years agoLP#1695029 Patron reg. supports bool opt-in defaults
Bill Erickson [Fri, 21 Jul 2017 18:50:17 +0000 (14:50 -0400)]
LP#1695029 Patron reg. supports bool opt-in defaults

Support default values for boolean user opt-in settings during patron
registration.  A default value of True, true, T, or t (or really
anything starting with a 't') is treated as true.  Any other value
(including null) means false.

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Josh Stompro <stomproj@larl.org>
Signed-off-by: Jason Etheridge <jason@EquinoxInitiative.org>
2 years agoLP#1695029-Webstaff Fix Patron Registration page never loading
Cesar Velez [Tue, 27 Jun 2017 19:06:48 +0000 (15:06 -0400)]
LP#1695029-Webstaff Fix Patron Registration page never loading

It looks like there was a variable mistype causing the page
to not load, but the setting that the lp bug mentions wasn't being
properly bound and needed to be wrapped to a boolean.

Signed-off by: Cesar Velez <cesar.velez@equinoxinitiative.org>
Signed-off-by: Josh Stompro <stomproj@larl.org>
Signed-off-by: Rogan Hamby <rogan.hamby@gmail.com>
Signed-off-by: Jason Etheridge <jason@EquinoxInitiative.org>
2 years agolp1709521 stamping schema upgrade
Jason Etheridge [Thu, 31 Aug 2017 03:48:24 +0000 (23:48 -0400)]
lp1709521 stamping schema upgrade

Signed-off-by: Jason Etheridge <jason@EquinoxInitiative.org>
2 years agoLP#1709521: Tweak description for new OU setting
Kathy Lussier [Tue, 29 Aug 2017 14:29:16 +0000 (10:29 -0400)]
LP#1709521: Tweak description for new OU setting

Update the description for the new Number of Retrievable Recent Patrons OU
Setting to identify the default value if left unset.

Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Signed-off-by: Jason Etheridge <jason@EquinoxInitiative.org>
2 years agoLP#1709521 Release notes for show recent patrons
Bill Erickson [Sat, 12 Aug 2017 15:52:26 +0000 (11:52 -0400)]
LP#1709521 Release notes for show recent patrons

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Signed-off-by: Jason Etheridge <jason@EquinoxInitiative.org>
2 years agoLP#1709521 Webstaff show recent patrons
Bill Erickson [Sat, 12 Aug 2017 15:52:40 +0000 (11:52 -0400)]
LP#1709521 Webstaff show recent patrons

Adds a new library setting 'ui.staff.max_recent_patrons' which
specifices the number of recently accessed patrons staff may quickly
refetch from the browser client interface.  If no value is set, it
defaults to 1 for backwards compatibility.

If the value is 0 or less, no recent patrons may be retrieved and the
'Retrieve Last Patron' action is hidden.  If the value is > 1 a new
'Retrieve Recent Patrons' menu item appears.  When clicked, the list
of recent patrons is displayed in the patron search interface, most
recently accessed patrons sorted to the top.

For privacy, the recent patrons list expires from the browser when the
authenticion token expires.

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Signed-off-by: Jason Etheridge <jason@EquinoxInitiative.org>
2 years agoLP#1683575 - Webstaff fix silent fail of bad barcodes in ItemStatus
Cesar Velez [Wed, 16 Aug 2017 21:39:30 +0000 (17:39 -0400)]
LP#1683575 - Webstaff fix silent fail of bad barcodes in ItemStatus

This address the current issue in master, that causes bad barcodes
to fail silently.

To test
-------
[1] In the web staff client, go to Item Status and enter an item
    barcode that is not present in the database. Note that no
    warning message displayed.
[2] Apply the patch and repeat step 1. This tie, an 'Item Not Found'
    message is displayed.

Signed-off by: Cesar Velez <cesar.velez@equinoxinitiative.org>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
2 years agoLP#1642086: Relase note for jQuery support
Mike Rylander [Wed, 30 Aug 2017 21:03:31 +0000 (17:03 -0400)]
LP#1642086: Relase note for jQuery support

Signed-off-by: Mike Rylander <mrylander@gmail.com>
2 years agoLP#1642086: Adjust offline resources for jquery support
Mike Rylander [Wed, 30 Aug 2017 20:54:59 +0000 (16:54 -0400)]
LP#1642086: Adjust offline resources for jquery support

Offline needs to know about all the resources and their correct locations.
Since we're moving jquery.min.js, tell it about that.

Signed-off-by: Mike Rylander <mrylander@gmail.com>
2 years agoLP#1642086 TPAC Jquery path repair, .gitignore, karma
Bill Erickson [Wed, 30 Aug 2017 15:53:40 +0000 (11:53 -0400)]
LP#1642086 TPAC Jquery path repair, .gitignore, karma

* Repair JS path in TPAC jquery import.

* Tell git to ignore Open-ILS/web/js/ui/default/common/build/ since it
  only contains generated files.

* Add jquery to Karma unit test dependencies.  This is not strictly
  required, since angular includes a slim jquery shim, but having the
  unit tests load the same base JS as the UI is probably best.
  Confirmed tests still pass.

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
2 years agoLP#1642086: Smallest possible JQuery patch that could work...
Mike Rylander [Tue, 15 Aug 2017 16:51:16 +0000 (12:51 -0400)]
LP#1642086: Smallest possible JQuery patch that could work...

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Bill Erickson <berickxx@gmail.com>
2 years agoThe ngToast maintainers decided to trick us with a new directory name. Thanks.
Mike Rylander [Tue, 29 Aug 2017 19:03:51 +0000 (15:03 -0400)]
The ngToast maintainers decided to trick us with a new directory name. Thanks.

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agoFix the "404 asset" test
Mike Rylander [Tue, 29 Aug 2017 18:42:03 +0000 (14:42 -0400)]
Fix the "404 asset" test

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agoReorder the tabs and adjust the default based on logged-in-ness
Mike Rylander [Tue, 29 Aug 2017 18:15:06 +0000 (14:15 -0400)]
Reorder the tabs and adjust the default based on logged-in-ness

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agoRemove confusing "session" tab from the offline menu entry -- the code will figure...
Mike Rylander [Tue, 29 Aug 2017 18:14:34 +0000 (14:14 -0400)]
Remove confusing "session" tab from the offline menu entry -- the code will figure out the correct default tab

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agoAdd moment.js to the offline asset list
Mike Rylander [Tue, 29 Aug 2017 18:13:48 +0000 (14:13 -0400)]
Add moment.js to the offline asset list

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agooffline: Load Lovefield wrapper in addition to the Lovefield framework
Mike Rylander [Tue, 8 Aug 2017 17:25:39 +0000 (13:25 -0400)]
offline: Load Lovefield wrapper in addition to the Lovefield framework

Spotted by Bill Erickson.  Thanks, Bill.

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Conflicts:
Open-ILS/web/js/ui/default/staff/test/karma.conf.js

Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agooffline: add live-test for offline assets
Mike Rylander [Mon, 7 Aug 2017 15:12:53 +0000 (11:12 -0400)]
offline: add live-test for offline assets

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agooffline: Remove reference to nonexistent file
Mike Rylander [Mon, 7 Aug 2017 14:05:05 +0000 (10:05 -0400)]
offline: Remove reference to nonexistent file

When the status bar was removed back in commit 7814064, the reference to its
backing JS file was removed from base_js.tt2.  It looks like that slipped
back in at some point in the combined serials/offline branch.  This removes
that references, which causes offline failure.

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agooffline: Load lovefield in the testing framework
Mike Rylander [Fri, 4 Aug 2017 15:16:17 +0000 (11:16 -0400)]
offline: Load lovefield in the testing framework

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agooffline: Prefer user-supplied param to browser-supplied cookie in the authen proxy
Mike Rylander [Thu, 27 Jul 2017 16:59:43 +0000 (12:59 -0400)]
offline: Prefer user-supplied param to browser-supplied cookie in the authen proxy

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agooffline: Make sure the the field_doc structure exists before writing to it
Mike Rylander [Tue, 25 Jul 2017 21:10:48 +0000 (17:10 -0400)]
offline: Make sure the the field_doc structure exists before writing to it

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agowebstaff: IDL Clone
Mike Rylander [Wed, 31 May 2017 15:29:36 +0000 (11:29 -0400)]
webstaff: IDL Clone

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agoLP#1706107: Offline mode
Mike Rylander [Mon, 20 Mar 2017 20:38:15 +0000 (16:38 -0400)]
LP#1706107: Offline mode

Here is implemented an offline mode interface for the web staff client.

It is made available during both network and server outages by using the
UpUp[1] service worker wrapper.

We leverage Lovefield[2] for local storage of library settings, configuration
data, offline transactions, and the standalone offline block list.

In order to make use of the offline interface, users should first log into
the web staff client and navigate to the "Search -> Search for Patrons"
interface, perform a search, select a user from the results, and open the
Patron Editor interface.  This will allow the offline interface to collect
all the relevant configuration information for the workstation.  In addition,
the offline interface available from the Circulation menu provides a "Download
block list" button when accessed while logged in.

[1]https://www.talater.com/upup/
[2]https://google.github.io/lovefield/

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Conflicts:
Open-ILS/src/templates/staff/base_js.tt2

Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agoLP#1710512 - OPAC js strings need to be i18n-izable
Cesar Velez [Thu, 24 Aug 2017 21:17:11 +0000 (17:17 -0400)]
LP#1710512 - OPAC js strings need to be i18n-izable

Adds a way to translate displayed string messages, in OPAC
JS source code alerts and other outputs. This is done by having
js.tt2 include i18n-strings.tt2 which contains an explosed/global
JS blob variable called eg_opac_i18n. This is intended to work
in similar fashion as base_js.tt2 in the webstaff ng client.

Signed-off by: Cesar Velez <cesar.velez@equinoxinitiative.org>
Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
2 years agolp1582354 stamping schema upgrade
Jason Etheridge [Tue, 29 Aug 2017 21:44:26 +0000 (17:44 -0400)]
lp1582354 stamping schema upgrade

and some other tweaks:

1) removing permacrud stanza from IDL entry
2) changing a tab to spaces
3) making the upgrade script match the extend-reporter.sql change

Signed-off-by: Jason Etheridge <jason@EquinoxInitiative.org>
2 years agoLP#1582354 - Add owning_lib to available links.
Chris Sharp [Thu, 29 Sep 2016 17:08:15 +0000 (13:08 -0400)]
LP#1582354 - Add owning_lib to available links.

Signed-off-by: Chris Sharp <csharp@georgialibraries.org>
Signed-off-by: Jason Etheridge <jason@EquinoxInitiative.org>
2 years agoLP1582354 report able to show bibs where the last copy was deleted cancels
blake [Tue, 17 May 2016 21:27:00 +0000 (16:27 -0500)]
LP1582354 report able to show bibs where the last copy was deleted cancels

Created a view for ease of reporting. This source table allows you to construct
a clever aggregate report template which will report bibliographic ID's where a
library or a group of libraries no longer have a copy attached but had* a copy
attached. This is especially useful when a holdings sync is required with an
external vendor.

Signed-off-by: blake <blake@mobiusconsortium.org>
Signed-off-by: Chris Sharp <csharp@georgialibraries.org>
Signed-off-by: Jason Etheridge <jason@EquinoxInitiative.org>
2 years agoLP#1048822: Release notes entry for fuller simplified pull list title
Kathy Lussier [Tue, 29 Aug 2017 15:27:30 +0000 (11:27 -0400)]
LP#1048822: Release notes entry for fuller simplified pull list title

Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agoLP#1048822: Stamping upgrade script for simplified pull list fuller title
Kathy Lussier [Tue, 29 Aug 2017 15:10:12 +0000 (11:10 -0400)]
LP#1048822: Stamping upgrade script for simplified pull list fuller title

Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agoLP#1048822 - Fuller Title for reporter.super_simple_record
Josh Stompro [Fri, 1 Jul 2016 18:54:54 +0000 (13:54 -0500)]
LP#1048822 - Fuller Title for reporter.super_simple_record

The simplified pull list uses reporter.super_simple_record for
title information.  reporter.super_simple_record currently only
pulls the first 245a value for the title.  For titles that
use 245(n|p) to fully identify the title this leaves out information
that staff need to find the item.

This patch adds all of the 245(n|p) to the 245a in the title.  I've found
this covers most cases where staff need a fuller title.  The order of
the 245(n|p) also matters, so steps are taken to keep the marc ordering
of those fields.

Testing:

Before Patch - find a title that uses a 245(n|p) field to fully identify the
title.  Place a hold on the title.  Use the simplified pull list to view the
location that has that item on the shelf.  Observe that only the 245a is in
the title field.

After Patch - Look at the simplified pull list again and see that the title now
includes more information.

Signed-off-by: Josh Stompro <stomproj@larl.org>
Signed-off-by: Dan Pearl <dpearl@cwmars.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agoLP#1629298: Web Client Checkout Fails For Barcodes With Spaces
Skye Howard [Thu, 17 Aug 2017 21:52:08 +0000 (21:52 +0000)]
LP#1629298: Web Client Checkout Fails For Barcodes With Spaces

Web Client checkout trims spaces from patron barcodes differently than xul client checkout
The web client was not removing the whitespaces before checking barcode searches
this commit includes a step where the whitespaces are removed.

Signed off by Skye Howard <showard@catalyte.io>
Signed off by Cesar Velez <cesar.velez@equinoxinitiative.org>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
2 years agoLP#1668734: stamp DB update
Galen Charlton [Mon, 28 Aug 2017 17:14:13 +0000 (13:14 -0400)]
LP#1668734: stamp DB update

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
2 years agoLP#1668734: set max_delay, similar to stock courtesy email notice
Galen Charlton [Mon, 28 Aug 2017 17:11:29 +0000 (13:11 -0400)]
LP#1668734: set max_delay, similar to stock courtesy email notice

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
2 years agoLP#1668734: pin ID of new A/T definition
Galen Charlton [Mon, 28 Aug 2017 16:55:50 +0000 (12:55 -0400)]
LP#1668734: pin ID of new A/T definition

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
2 years agoLP#1668734 Action Trigger for SMS Preminder
McCanna [Thu, 20 Jul 2017 19:44:28 +0000 (15:44 -0400)]
LP#1668734 Action Trigger for SMS Preminder

New optional SMS text notification to be sent 3 days prior to the due
date of any circulating item for patrons who have an SMS text number
and carrier stored in their accounts. This action trigger is disabled
by default and can be enabled in Admin > Local Administration >
Notifications / Action Triggers.

Signed-off-by: McCanna <tmccanna@georgialibraries.org>
Signed-off-by: Chris Sharp <csharp@georgialibraries.org>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
2 years agoLP#1698206: Stamping upgrade script for copy_vis_attr_cache
Kathy Lussier [Mon, 28 Aug 2017 15:21:16 +0000 (11:21 -0400)]
LP#1698206: Stamping upgrade script for copy_vis_attr_cache

Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agoLP#1698206: basic release notes entry
Galen Charlton [Thu, 17 Aug 2017 22:22:37 +0000 (18:22 -0400)]
LP#1698206: basic release notes entry

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agoLP#1698206: remove now-superfluous include of List::MoreUtils
Galen Charlton [Thu, 17 Aug 2017 22:16:39 +0000 (18:16 -0400)]
LP#1698206: remove now-superfluous include of List::MoreUtils

I've verified during rebasing that the rework introduced by the
eliminate staged search branch happen to include an independent
fix of LP#1624443.

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agoLP#1698206: fix sequence error in schema update script
Galen Charlton [Thu, 17 Aug 2017 16:35:03 +0000 (12:35 -0400)]
LP#1698206: fix sequence error in schema update script

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agoLP#1698206: Remove remaining SQL uses of the old copy visibility cache
Mike Rylander [Wed, 9 Aug 2017 18:14:42 +0000 (14:14 -0400)]
LP#1698206: Remove remaining SQL uses of the old copy visibility cache

There were two remaining uses of the old copy vis cache in SQL functions used
to render OPAC pages.  This commit gets rid of those.

There is still one reference to the asset.opac_visible_copies table in the old
staged-search function, but that is not used anywhere in the code now, so no
need to change that.  Instead, we should start pruning old code.

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agoLP#1698206: Remove remaining uses of the old copy visibility cache
Mike Rylander [Fri, 4 Aug 2017 17:22:24 +0000 (13:22 -0400)]
LP#1698206: Remove remaining uses of the old copy visibility cache

Some Perl was still using the old cache table, so this teaches them the new
style.

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agoLP#1698206: Copy counts generated from new vis cache data
Mike Rylander [Fri, 4 Aug 2017 16:52:05 +0000 (12:52 -0400)]
LP#1698206: Copy counts generated from new vis cache data

The unAPI, erm, API was depending on old copy visibility caching tables.  Here
we teach it to use the new style.

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agoLP#1698206: Reify baseline schema
Mike Rylander [Mon, 31 Jul 2017 15:40:07 +0000 (11:40 -0400)]
LP#1698206: Reify baseline schema

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agoLP#1698206: Make use of current search library in autosuggest
Mike Rylander [Wed, 19 Jul 2017 18:07:39 +0000 (14:07 -0400)]
LP#1698206: Make use of current search library in autosuggest

Here we teach autosuggest how to check the opac search scope.

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agoLP#1698206: Indicate broad searches and heavy facets
Mike Rylander [Mon, 10 Jul 2017 14:35:07 +0000 (10:35 -0400)]
LP#1698206: Indicate broad searches and heavy facets

There is a configurable, pagable limit on hits, defined by the superpage
size and max superpages.  When the hit count equals this (by default,
100000) we add a '+' to the hit count to show that there are even more hits.

Facets are calculated per superpage, and if the facet use count equals the
number of superpages seen so far multiplied by the superpage size, we
likewise add '+' to indicate that there are likely more records matching the
facet.

For facets, a user can page far enough to increase the visible number, if
they cross superpage boundaries.

Signed-off-by: Mike Rylander <miker@esilibrary.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agoLP#1698206: Add TechRef documentation from commit message
Mike Rylander [Wed, 28 Jun 2017 22:13:30 +0000 (18:13 -0400)]
LP#1698206: Add TechRef documentation from commit message

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agoLP#1698206: Remove hit estimation cruft
Mike Rylander [Wed, 28 Jun 2017 22:07:21 +0000 (18:07 -0400)]
LP#1698206: Remove hit estimation cruft

The old code needed to refine the estimated hit count as each superpage was
read, but we don't need any of that anymore.  Also, it was causing hit count
display issues on superpages after the first.  So, we do away with all that.

Signed-off-by: Mike Rylander <miker@esilibrary.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agoLP#1698206: Eliminate Staged Search
Mike Rylander [Thu, 15 Jun 2017 19:54:40 +0000 (15:54 -0400)]
LP#1698206: Eliminate Staged Search

=== Background
Evergreen stores all data, including that useful for patron and staff search,
in a normalized schema that is time and space efficient for transactional use
cases, and provides guarantees on data integrity.  In addition, development is
made simpler than would be the case otherwise and arbitrary reporting is made
possible.

However, this structure is not effective for direct, SQL-only search
functionality in a hierarchical, consortial dataset.  This is a problem that
is relatively unique to Evergreen, as it is most often employed to host and
serve large consortia with overlapping bibliographic datasets and
non-overlapping item and location datasets.  Other search engines, including
those built into other ILSs, do not generally have to account for
hierarchically organized location visibility concerns as a primary use case.
In other words, because it provides functionality that requires a hierarchical
view of non-bibliographic data, a problem space for Evergreen is essentially
nonexistent in competing products.

Evergreen's search infrastructure has evolved over the years.  In its current
form, the software first performs a full text search against extracted
bibliographic data and limits this initial internal result set to a
configurable size.  It then investigates the visibility of each result on
several non-bibliographic axes.  These visibility tests take up the
preponderance of CPU time spent in search, with full text search of the
bibliographic data generally completing within milliseconds. The main reason
this multi-stage mechanism is used is that there are many visibility axes and
attempting to join all required data sources together in a single query will
cause the search use case to perform very poorly.  A previous attempt to
create a pure SQL search mechanism failed for this reason.

A significant drawback of the current approach is that the costs imposed by
visibility filtering search results using normalized non-bibliographic data,
either in-query or separated from the main full-text query as it is today,
make it necessary to place limits on the number of database rows matched by
full-text query constructs.  This in turn can cause searches to omit results
in certain situations, such as a large consortium consisting of a few large
libraries and many small libraries.

However, it has been shown possible to overcome this performance issue by
providing an extensible way to collect all visibility related information
together into a small number of novel data structures with a compact in-memory
representation and very fast comparison functions.  In this way, we are able
to use pure SQL search strategies and therefore avoid result visibility
problems while also benefiting from improvements to the core PostgreSQL
database engine.  Further, this will open the door to indexing improvements,
such as removal of the need for duplicate data storage, or the use of non-XML
data storage schemes, which could reduce resource requirements and have a
direct, positive effect on patron and staff search experience.

=== Overview of existing search logic

. Construct core bibliographic search query
. Collect non-bibliographic filtering criteria
. Pass query and filters to a database function
. Calculate hierarchical location information for visibility testing
. Open cursor over core query, limited to *superpage_size * max_superpages* records
. Core query implements bib-level sorting
. For each result
.. NEXT if not on requested superpage
.. Check deleted flag, based on search type
.. Check transcendence
... Return result if true
.. Check for direct Located URI in scope
... Return result if exists
.. Check copy status + (circ lib | owning lib) based on modifier
.. Check peer bib copy status + (circ lib | owning lib) based on modifier
.. Check copy location based on filter
.. Check peer bib copy location based on filter
.. General copy visibility checks
... If NOT staff
.... Check for OPAC visible copies (trigger-maintained materialization)
.... Check for peer bib OPAC visible copies
... If staff
.... Confirm no copies here
.... Confirm no peer bib map
.... Confirm no copies anywhere
.... Confirm no Located URIs elsewhere
.. Return result if not excluded
. Calculate summary row

=== Overview of new mechanism
Record and copy information (everything checked in *(7)* above) is collected
into a novel data structure that allows all visibility-indicating criteria to
be flattened to integer arrays.  This is facilitated by a database trigger in
much the same way that basic OPAC copy visibility is collected for copies
today.

Most identifiers in Evergreen are stored as signed integers of either 32 or 64
bits.  The smaller 32 bit space allows for approximately two billion positive
entries, but all identifiers for table rows that are used as visibility axes
fall into a range of between one and one million for all applicable use cases,
and all identifiers of interest are positive.  Therefore, we can make use of
the most significant bits in an integer value to create a per-axis namespacing
mask.  When applied to the idenfitifer for a visibility axis identifier, this
mask allows two values that are identical across axis to be identified as
unique within a combined set of all values.

Sepcifically, we retain the four most significant bits of the integer space
and create from that 16 potential bitmasks for per-axis segregation of
identifiers.  Further, we separate copy-centered axes and bibliographic
record-centered attributes into two separate columns for storage purposes,
which means we can use the same four bits for different purposes within each
copy or bib set.

In order to implement existing visibility tests with this infrastructure, six
copy axes and two record axes are used from the possible 16 from each set.
See the search.calculate_visibility_attribute() for details.  By using 32 bit
integers we can collect all of the bitmasked values of each type (copy or bib)
into a single integer array and leverage the Postgres intarray extension to
test all axes at once.

At search time, required and user-requested visibility restrictions are
converted to *query_int* values. Results are directly filtered based on these
calculated *query_int* values.  This works in a way analogous to record
attribute filtering, avoiding the need to test statuses, circ and owning
library visibility, copy locations and location groups, copy OPAC visibility,
peer bibliographic record, Located URIs, or bibliographic record sources
directly.

=== Minimum Postgres version requirement
Due to features, particularly functions, available only in 9.4 and newer that
are key to the performance of the new method, Postgres 9.4 will need to be the
new lowest supported version for use with Evergreen.  While some of the new
features and functions could be implemented as user-defined functions in
PL/PGSQL, they would not be fast enough to make this pure-SQL search viable.

Among the important improvements that Postgres 9.4 and newer versions bring to
Evergreen are:

* Version 9.4 improved GIN indexes in ways that directly benefit Evergreen, as well as how anti-joins are planned which matters for some Evergreen searches.
* Version 9.5 introduced many general performance improvements, especially for joins and sorting, and brought planner improvements that impact complex queries such as those generated by this code.
* Version 9.6 delivered more general performance improvements, particularly for large servers such as those that Evergreen databases tend to live on, as well as more improvements to GIN indexes, executor changes that can avoid unnecessary work in search queries, new built-in full-text phrase searching, and initial parallel query execution.

=== Performance
The cost of the non-bibliographic filter value caching maintenance process is
10-40% faster than existing partial caching logic which it would replace.

The new code achieves up to 10% faster search times than the old, suboptimal
mechanism time for broad searches.  The new code is faster for more selective
searches, often by up to 90% faster.  In both broad and narrow search cases
the new mechanism performs with complete accuracy and does not miss
small-collection hits in large consortia as the existing code does.

Unsurprisingly, and in addition to the above improvements, performance is
improved marginally as each successive Postgres version at and beyond 9.4.

=== Page rendering changes
Previously, Evergreen would request the record details for a user-visible page
of results in parallel, and then, serially, request the facet data for the
result set.  Now, the facet data is requested asyncronously in the background
and then a single feed containing all records on a result page is requested
syncronously.  By parallelizing the result and facet metadata, page rendering
time is cut down significantly.  Concurrent requests of the same bibliographic
record are shared between apache backends to reduce result request time, and by
making one request instead of ten simultaineously, database load is reduced.  A
performance improvement of up to 20% in post-search page rendering time is seen
from this change.

Additionally, cross-apache caching of ancillary data, such as the coded value
map and other data, via memcache significantly reduces the average page
rendering time not just for result pages, but most pages generated by
Evergreen.  An additional performance improvement of up to 50% in post-search
page rendering time is seen from this change.

While these changes are not directly related to the removal staged search, they
touch areas impacted by core search changes and provided enough improvement
that implementing them concurrently with the elimination of staged search
seemed optimal.

=== User visible configuration changes
The stock configuration now provides an increased value for *max_superpages*
in opensrf.xml.  The default is now 100, and the *superpage_size* remains
1000, for a total limit of 100,000 hits per search.  This is not a limit on
visibility per se, as all records are visibility tested and ranked before
limiting, but simply a limit on the number of pages a user could click through
before reaching the end of the presented result list.

=== Tuning sensitivity
User-level timeouts are still possible with both the old and new code, given a
large enough dataset, a broad enough query, and a cold cache.  However, the
*gin_fuzzy_search_limit* GUC can be used to set a time cap on the new
mechanism. See https://www.postgresql.org/docs/9.6/static/gin-tips.html for
background, though the suggested values in the documentation are significantly
lower than would be readily useful for a large Evergreen instance.

Because it uses a more complex query structure, the new mechanism is somewhat
more sensitive to Postgres tuning in general.  In particular, lowering
*random_page_cost* from the default of *4.0* to a more reasonable *2.0* is
important for proper query planning.  For Evergreen use cases where the search
indexes and relevant tables are kept in RAM or SSDs are used for storage, this
value is acceptable and useful in general.

=== Funding and development
This project was funded by MassLNC and developed by Equinox Open Library
Initiative.

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Conflicts:
Open-ILS/src/perlmods/lib/OpenILS/WWW/EGCatLoader/Util.pm

Signed-off-by: Kathy Lussier <klussier@masslnc.org>
2 years agoLP#1689608: stamp schema update
Galen Charlton [Mon, 28 Aug 2017 15:08:22 +0000 (11:08 -0400)]
LP#1689608: stamp schema update

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
2 years agoLP#1689608: add release notes
Galen Charlton [Mon, 28 Aug 2017 15:02:20 +0000 (11:02 -0400)]
LP#1689608: add release notes

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
2 years agoLP#1689608: Minor fixes
Kathy Lussier [Fri, 21 Jul 2017 16:33:03 +0000 (12:33 -0400)]
LP#1689608: Minor fixes

Fixe a typo in the seed data:
TEXT not NEXT

Also, add the user bucket service to karma test so that our tests are happy.

Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
2 years agoLP#1689608: Batch user editing
Mike Rylander [Thu, 2 Feb 2017 20:29:46 +0000 (15:29 -0500)]
LP#1689608: Batch user editing

Summary
-------

Currently, editing and deleting of users must be performed on a user-by-user
basis. There are workflows that would benefit from the ability to act on a
set of users, where the changes to all users in the set are the same.

This commit provides a new interface analogous to the Copy Bucket interface
to record the selection and grouping of a set of users into a User Bucket.
The addition of users to a User Bucket is possible from the Patron Search
interface by the use of a new grid Action, and directly on the User Bucket
interface by user barcode. It is also possible to add users by uploading
a text file that contains a list of user barcodes.

From this interface it is possible to perform a set of specific batch update
operations against users generally.

Editing users
-------------

In order to facilitate the update of user data fields, specifically:

 * Active flag
 * Primary Permission Group (group application permissions consulted)
 * Juvenile flag
 * Home Library (UPDATE_USER checked against both old and new value)
 * Privilege Expiration Date
 * Barred flag (BAR_PATRON permission consulted)
 * Internet Access Level

This commit contains a new set of business logic allowing staff to supply new
values for these fields. Creation and immediate processing of a change set
will be made available through a grid Menu item. If the staff user does not
have the UPDATE_USER permission, this option will be disabled.

Each change set requires a name. Buckets may have multiple change sets. All
users in the Bucket at the time of processing will be updated when the change
set is processed, and change sets are processed immediately upon successful
creation. The interface will deliver progress information regarding the
processing stage and percent of completion.

While processing the users, the original value for each field edited will be
recorded for potential future rollback. Users can examine the success and
failure of applied change sets.

The user will be able to rollback the entire change set, but not parts thereof.
The rollback will affect only those users that were successfully updated by the
original change set and may be different from the current set of users in the
Bucket. Users can manually discard change sets, removing them from the
interface but preventing future rollback.

As a batch process, rather than a direct edit, this mechanism explicitly skips
processing of Action/Trigger event definitions for user update.

Deleting users
--------------

In order to facilitate the batch deletion of users, this commit creates a new
set of business logic allowing staff to set the Deleted flag on users.
Creation and immediate processing of a batch delete is made available through
a grid Menu item. If the staff user does not have both the UPDATE_USER and
DELETE_USER permission, this option is disabled. Because of the potential for
damage and the additional required permission, this field change is
specifically segregated from the general Editing functionally described above.

Each delete set requires a name. Buckets may have multiple delete sets. All
users in the Bucket at the time of processing will be marked as deleted when
the delete set is processed. The interface will deliver progress information
regarding the processing stage and percent of completion.

While processing the users, the original value for the "deleted" field will be
recorded for potential future rollback. Users will be able to examine the
success and failure of applied delete sets in the same interface used for the
above described change sets.

As a batch process, rather than a direct edit, this mechanism explicitly skips
processing of Action/Trigger event definitions for user deletion.

This mechanism does not use the Purge User functionality, but instead simply
marks the users as deleted. Future enhancement could add such functionality.

Editing Statistical Category Entries
------------------------------------

In order to facilitate the batch editing, addition, and removal of
Statistical Category Entries for users, this commit creates a new set of
business logic allowing staff to either remove or add & update Entries for
Statistical Categories to which the staff member has access. Processing of
Statistical Category Entry modifications will are available through a grid
Menu item.

All users in the bucket will have their Statistical Category Entries
modified. Unlike user data field updates, modification of Statistical
Category Entries is permanent and cannot be rolled back. No named change
sets are required. The interface will deliver progress information regarding
the processing stage and percent of completion.

As a batch process, rather than a direct edit, this mechanism explicitly skips
processing of Action/Trigger event definitions for user update.

New service requirement
-----------------------

This new functionality makes use of the QStore service, which was previously
unused in production.  If this service has been removed from the configuration
of a live Evergreen instances, it will need to be added back in order for
batch user editing to succeed.

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Scott Thomas <scott.thomas@sparkpa.org>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Conflicts:
Open-ILS/src/templates/staff/circ/patron/index.tt2
Open-ILS/web/js/ui/default/staff/circ/patron/app.js

2 years agoLP#1712840: extend fix to circ and holds history tabs
Galen Charlton [Fri, 25 Aug 2017 14:29:43 +0000 (10:29 -0400)]
LP#1712840: extend fix to circ and holds history tabs

This patch fixes the remaining two instances of a
bare 'IF ebook_api.enabled' test.

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Ben Shum <ben@evergreener.net>
2 years agoLP#1712840 Ebook tabs always visible in My Account
McCanna [Fri, 25 Aug 2017 13:53:14 +0000 (09:53 -0400)]
LP#1712840 Ebook tabs always visible in My Account

Corrects a problem in My Account where the ebook tabs
on the checkouts and holds pages were always visible
regardless of whether the ebook_api.enabled value was
set to true or false.

Signed-off-by: McCanna <tmccanna@georgialibraries.org>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Ben Shum <ben@evergreener.net>
2 years agoLP#1712657: stamping schema update (fixup)
Ben Shum [Sat, 26 Aug 2017 22:39:03 +0000 (18:39 -0400)]
LP#1712657: stamping schema update (fixup)

Originally committed with the wrong upgrade stamp number. Move that file,
and also update the baseline schema upgrade version.

Signed-off-by: Ben Shum <ben@evergreener.net>
2 years agoLP#1712657: stamping schema update
Jason Etheridge [Fri, 25 Aug 2017 21:40:44 +0000 (17:40 -0400)]
LP#1712657: stamping schema update

Signed-off-by: Jason Etheridge <jason@EquinoxInitiative.org>
2 years agoLP#1712657 Merge patron checkout history
Bill Erickson [Wed, 23 Aug 2017 19:39:16 +0000 (15:39 -0400)]
LP#1712657 Merge patron checkout history

Transfer checkout history rows to the destination user during patron
merge.

To Test:

1. Enable circ history tracking for 2 users, ideally 2 with same
   last name for easy searching.
2. Check out items to both patrons.
3. Load the 2 patrons in the patron search UI and merge one into the other.
4. action.usr_circ_history rows previously linked to the deleted patron
   will now be linked to the active/lead patron.

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Jason Etheridge <jason@EquinoxInitiative.org>
2 years agoLP1713064: String / Float Value ACP Editor Fix
Jason Boyer [Fri, 25 Aug 2017 15:13:22 +0000 (11:13 -0400)]
LP1713064: String / Float Value ACP Editor Fix

Add an Angular directive to the vol/copy editor
to convert and format numbers stored as strings
in acp objects.

Signed-off-by: Jason Boyer <jboyer@library.in.gov>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
2 years agoLP#1685929: (follow-up) fix passing copy when opening volume/copy editor
Galen Charlton [Fri, 25 Aug 2017 19:51:44 +0000 (15:51 -0400)]
LP#1685929: (follow-up) fix passing copy when opening volume/copy editor

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
2 years agoLP#1685929: adjust egItem.print_spine_labels()
Galen Charlton [Fri, 25 Aug 2017 19:17:25 +0000 (15:17 -0400)]
LP#1685929: adjust egItem.print_spine_labels()

Make egItem.print_spine_labels() accept a list of copy
IDs and make the caller responsible for putting that list together.

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
2 years agoLP#1685929: (follow-up) fix rebase error
Galen Charlton [Fri, 25 Aug 2017 15:49:31 +0000 (11:49 -0400)]
LP#1685929: (follow-up) fix rebase error

Intentionally not putting two actions in the Show group.

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
2 years agoLP#1685929: add implementation print spine labels to checkin grid
Cesar Velez [Tue, 1 Aug 2017 21:42:00 +0000 (22:42 +0100)]
LP#1685929: add implementation print spine labels to checkin grid

Signed-off by: Cesar Velez <cesar.velez@equinoxinitiative.org>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
2 years agoLP#1685929 - Extract itemSvc from item module to core
Cesar Velez [Fri, 11 Aug 2017 15:11:45 +0000 (16:11 +0100)]
LP#1685929 - Extract itemSvc from item module to core

Refactored Missing Pieces module and CheckIn to use
shared functionally of itemSvc, which is now it's own
separate file.

Signed-off by: Cesar Velez <cesar.velez@equinoxinitiative.org>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Conflicts:
Open-ILS/web/js/ui/default/staff/cat/item/app.js

2 years agoLP#1685929-Adds more XUL parity to Webstaff checkIn
Cesar Velez [Fri, 11 Aug 2017 15:00:29 +0000 (16:00 +0100)]
LP#1685929-Adds more XUL parity to Webstaff checkIn

Checkin Grid context actions like Add Items to Bucket, Show Holds on Bib, Show
Last Few Circulations, Edit Item Attributes, Mark Item Missing Pieces,
Print Spine Label, were missing from CheckIn in webstaff.

Signed-off by: Cesar Velez <cesar.velez@equinoxinitiative.org>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Conflicts:
Open-ILS/web/js/ui/default/staff/cat/item/app.js

2 years agoLP#1684988: add opt-in check to patron service
Jeff Davis [Mon, 21 Aug 2017 23:12:46 +0000 (16:12 -0700)]
LP#1684988: add opt-in check to patron service

Signed-off-by: Jeff Davis <jdavis@sitka.bclibraries.ca>
Signed-off-by: Jason Etheridge <jason@EquinoxInitiative.org>
2 years agoLP1712922: Exclude Deleted ACPL in Vol/Copy Editor
Jason Boyer [Thu, 24 Aug 2017 21:34:02 +0000 (17:34 -0400)]
LP1712922: Exclude Deleted ACPL in Vol/Copy Editor

Just what it says on the tin. When loading the list of ACPL
in the editor only retrieve the non-deleted locations.

Tester's note: there's no database contraint that strictly
prevents a copy from linking to a deleted location, so for
items like that, the copy location selector will be blank. This
is an uncommon situation, but Evergreen admins can identify
such items by running the following query:

select acp.id, acp.barcode, acpl.name
from asset.copy acp
join asset.copy_location acpl on (acpl.id = acp.location)
where not acp.deleted
and acpl.deleted
order by acp.id;

Signed-off-by: Jason Boyer <jboyer@library.in.gov>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>