Suzanne Paterno [Thu, 10 Aug 2017 17:15:39 +0000 (13:15 -0400)]
LP#1688398: revamp public catalog bill payment page
- Changed table layout of the credit card charges on the Account screen.
Moved checkbox to left of screen, added a column for Billing Type.
- Add button to Pay Selected vs All Charges.
- Include image of credit cards accepted (need to update with one with
AmEx)
- Reformatted page for inputting credit card
- Updated confirmation screen to bold amount, and list the titles as
the ones the user is paying for.
- Changed Print reciept template in seed data to look more readable and
like other itemized reciepts.
Suzanne Paterno [Thu, 10 Aug 2017 15:38:53 +0000 (11:38 -0400)]
LP#1688398: change wording on OPAC bill payment pages
On the credit card screens replaced the word "Fines" with "Charges"
since not all charges are fines. Can be cost of replacement materials,
as well as other non-circulation fees.
Galen Charlton [Tue, 22 Aug 2017 20:13:54 +0000 (16:13 -0400)]
LP#1373690: add USE_ID_FOR_OWNING_LIB EDI attribute
This attribute specifies that the copy owning library
should have its ID emitted rather than its shortname, which can
be used for Baker & Taylor, which imposes a five-character limit
on certain fields. This takes effect only when the INCLUDE_OWNING_LIB
EDI attribute is also in effect.
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org> Signed-off-by: Bill Erickson <berickxx@gmail.com> Signed-off-by: Mike Rylander <mrylander@gmail.com>
The EDItEUR book ORDERS message allows two 35-character
item description data elements in IMD segments; this patch
accounts for that. It also, for IMD fields, uses the EDIFACT
release character to escape certain characters, better matching
how the Ruby EDI translator did it.
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org> Signed-off-by: Bill Erickson <berickxx@gmail.com> Signed-off-by: Mike Rylander <mrylander@gmail.com>
Adds a new configuration option for EDI accounts specifying wether
purchase orders delivered via a given account should be constructed
using the new EDI Attributes in lieu of the traditional JEDI A/T
template.
The PurchaseOrderEDIRequired A/T validator now bypasses orders with
use_attrs=true accounts. The new edi_order_pusher.pl script only
process providers whose default EDI accounts have use_attrs=true (except
when a specific PO ID is provided).
Signed-off-by: Bill Erickson <berickxx@gmail.com> Signed-off-by: Mike Rylander <mrylander@gmail.com>
Adds a new default collection of EDI attributes for Midwest Library
Service. This includes 2 new attributes to support their EDI format:
1. BUYER_ID_ONLY_VENDCODE -- similar to BUYER_ID_INCLUDE_VENDCODE,
except the buyer SAN is excluded from the NAD+BY segement.
2. INCLUDE_EMPTY_IMD_VALUES -- Render IMD+ fields (title, author,
edition, etc.) as if they contain values even when they don't. Empty
values are included as single-space strings.
Signed-off-by: Bill Erickson <berickxx@gmail.com> Signed-off-by: Mike Rylander <mrylander@gmail.com>
Adds a new edi_order_pusher.pl script for generating EDI ORDERS from
purchase orders via EDIWriter.pm. This is a replacement for
edi_pusher.pl, but edi_pusher.pl remains for backwards compat.
Signed-off-by: Bill Erickson <berickxx@gmail.com> Signed-off-by: Mike Rylander <mrylander@gmail.com>
- wrong password
- type = staff but user doesn't have STAFF_LOGIN
- repeated failed attempts
- barcode is inactive
- using "identifier" in conjunction with opac.barcode_regex set
- using "username" or "barcode"
- supplying a workstation name that exists
- ... or does not
- user is inactive
- user is marked barrred
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org> Signed-off-by: Mike Rylander <mrylander@gmail.com>
Adds a new open-ils.auth API call 'open-ils.auth.login' which performs
the combined steps of open-ils.auth.authenticate.init and
open-ils.auth.authenticate.complete so the caller only need call one API
to login.
API params are consistent with open-ils.auth.authenticate.complete with
2 notable excpetions. The API uses the bare password instead of the
hashed password, so the caller also need not perform the extra hashing
steps. Also, no 'nonce' parameter is used as it's no longer needed,
because there is no intermediate authentication cache object as with
.init.
Response data is consistent with open-ils.auth.authenticate.complete.
1. Using the generic "identifier" parameter in combination with the
"org" parameter allows the API to reliably determine if a value
is a username or barcode.
2. Once a caller has reached the configured maximum number of login
failures, no further attempts to track failures occurs, based on the
idea that no additional cpu/network cycles should be used on a lost
cause.
3. A failure count object is only added to memcache when failures
occur, unlike open-ils.auth.authenticate.init which creates a
failure tracking object for every login.
4. The code avoids use of the jsonParseFmt() and va_list_to_string()
functions as these functions require extra data cleansing.
Signed-off-by: Bill Erickson <berickxx@gmail.com> Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org> Signed-off-by: Mike Rylander <mrylander@gmail.com>
When entering the Manage Predictions or Manage Issues tab
and only one subscription is present in the current OU
scope, automatically select it rather than making the operator
have to select it manually.
This patch adds a serials module to the web staff client, implementing
a unified serials interface allowing for the following actions supported
by the XUL staff client:
- creating subscriptions, distributions, and streams
- creating and editing prediction patterns
- receiving serial issues, with or without barcodes (units)
- batch and quick receiving
This module also implements some new features, including
- the ability to save prediction pattern codes as templates
that can be shared and reused within an Evergreen database
- a more streamlined interface for managing subscriptions,
distributions, and streams
- it is no longer necessary to create a starting issue in
order to predict a run of issues; the dialog box for
generating a set of predicted issues now lets you specify
the starting point directly.
- the ability to more directly edit MFHDs
The new serials interfaces can be accessed from the record
details page via a Serials drop-down button that links to
a subscription management page, a quick-receive action, and
a MFHD management page. There is also a new Serials Administration
page where prediction pattern and serial copy templates can
be managed.
To test
-------
* Create, edit, and delete subscriptions, distribution streams,
and routing lists.
* Use the prediction pattern wizard to create patterns.
* Save prediction pattern templates and use them to apply
a pattern to new subscriptions.
* Verify that sets of issues can be predicted and received.
* Create and apply serial copy templates and verify that
they are applied when receiving barcoded issues.
This patch represents a group coding effort by Galen Charlton,
Jason Etheridge, and Mike Rylander.
egI18N is a module that will serve as a grab-bag of functions
related to I18N and L10N. The initial function it provides
takes a acpl IDL object and returns a formatted name qualified
by the org unit, with the underlying template accessible
to the translation subsystem.
Mike Rylander [Mon, 24 Apr 2017 16:40:37 +0000 (12:40 -0400)]
LP#1152753: make it possible for serial units to be added to copy buckets
This patch replaces the baseline asset.copy.id fkey constraints with ones
that understand inheritance, and change all existing contstraints to do the
same via upgrade script.
To test
-------
Create some serial units and verify that they can be added to a copy
bucket.
Signed-off-by: Mike Rylander <mrylander@gmail.com> Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org> Signed-off-by: Kathy Lussier <klussier@masslnc.org> Signed-off-by: Dan Wells <dbw2@calvin.edu>
Mike Rylander [Thu, 25 May 2017 00:37:17 +0000 (20:37 -0400)]
LP#1708291: teach egEmbedFrame about an afterload function
The 'afterload' attribute added by this patch allows specifying
the name of a function (to be supplied by the embedded content)
to run after the frame has been loaded.
Signed-off-by: Mike Rylander <mrylander@gmail.com> Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org> Signed-off-by: Kathy Lussier <klussier@masslnc.org> Signed-off-by: Dan Wells <dbw2@calvin.edu>
Won't delete if there are attached serial.item records with a status other
than Expected, or if there are any attached non-deleted serial.unit
records.
The dry_run variants behave exactly the same except they don't actually
delete anything.
Signed-off-by: Jason Etheridge <jason@equinoxinitiative.org> Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org> Signed-off-by: Kathy Lussier <klussier@masslnc.org> Signed-off-by: Dan Wells <dbw2@calvin.edu>
egEditFmRecord now knows how to specify that a custom Angular template
be used to supply the input widget for a given field; the initial use
of this will be allowing the prediction pattern template editor to be
used to set the pattern in a pattern template. The customFieldTemplates
attribute is used for this purpose.
This patch also teaches egEditFmRecord when to allow an org unit
selector to default to the workstation OU. The orgDefaultAllowed
attribute is used for this purpose.
Finally, a fixes a bug that ensures that the Save button is active
only when the entire form is valid.
This directive implements a selector for OU-sharing depths; depths
and names come from the actor.org_unit_type table. If there are
multiple types defined for a given depth, the display value in
the selector is the concatenation of their names.
The initial use of this directive is for specifying how broadly
prediction pattern templates should be seen.
LP#1708291: teach egIDL a new fieldmapper object representation
This patch teaches egIDL how to represent fieldmapper objects
as objects with attributes rather than getter/setters. This
allows FM objects to be more easily bound to common to Angular
input widgets, avoiding the verbosity of getterSetter ng-options
and the fact that some widgets like uib-datepicker-popup don't
support getterSetter in the first place.
Load the max recent patrons setting directly in the patron service
instead of requring the calling UI to apply it. This fixes an issue
where recent patrons are not set from the checkout-by-barcode
interface.
Signed-off-by: Bill Erickson <berickxx@gmail.com> Signed-off-by: Kathy Lussier <klussier@masslnc.org>
blake [Thu, 31 Aug 2017 20:33:17 +0000 (20:33 +0000)]
LP1655158 Patron Search by Date of Birth
Adds three UI boxes to the WBSC "Show Extra" patron search. One for the year, month and day.
The javascript on the page is altered to deliver group "4" to the backend. Local javascript
strips out non-numeric user entered data. The backend is updated to handle the new group.
SQL is genereated using the DATE_PART postgres function.
1. Open the web based staff client and browse to the patron search UI.
2. Click the show more down arrow button. Notice the lack of birth date field.
3. Apply the patch, repeat step one. Notice the addition of birth date boxes.
4. Type 1975 into the birth year box and press enter. Notice search results.
5. Try searching for partial names and partial birthdates.
6. Try entering non-numeric data into the birth date boxes.
7. Try searching for patrons without including the dob. Try with only the dob. Try a mix.
Signed-off-by: blake <blake@mobiusconsortium.org> Signed-off-by: Bill Erickson <berickxx@gmail.com>
Support default values for boolean user opt-in settings during patron
registration. A default value of True, true, T, or t (or really
anything starting with a 't') is treated as true. Any other value
(including null) means false.
Signed-off-by: Bill Erickson <berickxx@gmail.com> Signed-off-by: Josh Stompro <stomproj@larl.org> Signed-off-by: Jason Etheridge <jason@EquinoxInitiative.org>
Cesar Velez [Tue, 27 Jun 2017 19:06:48 +0000 (15:06 -0400)]
LP#1695029-Webstaff Fix Patron Registration page never loading
It looks like there was a variable mistype causing the page
to not load, but the setting that the lp bug mentions wasn't being
properly bound and needed to be wrapped to a boolean.
Signed-off by: Cesar Velez <cesar.velez@equinoxinitiative.org> Signed-off-by: Josh Stompro <stomproj@larl.org> Signed-off-by: Rogan Hamby <rogan.hamby@gmail.com> Signed-off-by: Jason Etheridge <jason@EquinoxInitiative.org>
Adds a new library setting 'ui.staff.max_recent_patrons' which
specifices the number of recently accessed patrons staff may quickly
refetch from the browser client interface. If no value is set, it
defaults to 1 for backwards compatibility.
If the value is 0 or less, no recent patrons may be retrieved and the
'Retrieve Last Patron' action is hidden. If the value is > 1 a new
'Retrieve Recent Patrons' menu item appears. When clicked, the list
of recent patrons is displayed in the patron search interface, most
recently accessed patrons sorted to the top.
For privacy, the recent patrons list expires from the browser when the
authenticion token expires.
Signed-off-by: Bill Erickson <berickxx@gmail.com> Signed-off-by: Kathy Lussier <klussier@masslnc.org> Signed-off-by: Jason Etheridge <jason@EquinoxInitiative.org>
Cesar Velez [Wed, 16 Aug 2017 21:39:30 +0000 (17:39 -0400)]
LP#1683575 - Webstaff fix silent fail of bad barcodes in ItemStatus
This address the current issue in master, that causes bad barcodes
to fail silently.
To test
-------
[1] In the web staff client, go to Item Status and enter an item
barcode that is not present in the database. Note that no
warning message displayed.
[2] Apply the patch and repeat step 1. This tie, an 'Item Not Found'
message is displayed.
Signed-off by: Cesar Velez <cesar.velez@equinoxinitiative.org> Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
* Tell git to ignore Open-ILS/web/js/ui/default/common/build/ since it
only contains generated files.
* Add jquery to Karma unit test dependencies. This is not strictly
required, since angular includes a slim jquery shim, but having the
unit tests load the same base JS as the UI is probably best.
Confirmed tests still pass.
Signed-off-by: Bill Erickson <berickxx@gmail.com> Signed-off-by: Mike Rylander <mrylander@gmail.com>
Mike Rylander [Mon, 7 Aug 2017 14:05:05 +0000 (10:05 -0400)]
offline: Remove reference to nonexistent file
When the status bar was removed back in commit 7814064, the reference to its
backing JS file was removed from base_js.tt2. It looks like that slipped
back in at some point in the combined serials/offline branch. This removes
that references, which causes offline failure.
Signed-off-by: Mike Rylander <mrylander@gmail.com> Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Here is implemented an offline mode interface for the web staff client.
It is made available during both network and server outages by using the
UpUp[1] service worker wrapper.
We leverage Lovefield[2] for local storage of library settings, configuration
data, offline transactions, and the standalone offline block list.
In order to make use of the offline interface, users should first log into
the web staff client and navigate to the "Search -> Search for Patrons"
interface, perform a search, select a user from the results, and open the
Patron Editor interface. This will allow the offline interface to collect
all the relevant configuration information for the workstation. In addition,
the offline interface available from the Circulation menu provides a "Download
block list" button when accessed while logged in.
Cesar Velez [Thu, 24 Aug 2017 21:17:11 +0000 (17:17 -0400)]
LP#1710512 - OPAC js strings need to be i18n-izable
Adds a way to translate displayed string messages, in OPAC
JS source code alerts and other outputs. This is done by having
js.tt2 include i18n-strings.tt2 which contains an explosed/global
JS blob variable called eg_opac_i18n. This is intended to work
in similar fashion as base_js.tt2 in the webstaff ng client.
Signed-off by: Cesar Velez <cesar.velez@equinoxinitiative.org> Signed-off-by: Jason Stephenson <jason@sigio.com> Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
blake [Tue, 17 May 2016 21:27:00 +0000 (16:27 -0500)]
LP1582354 report able to show bibs where the last copy was deleted cancels
Created a view for ease of reporting. This source table allows you to construct
a clever aggregate report template which will report bibliographic ID's where a
library or a group of libraries no longer have a copy attached but had* a copy
attached. This is especially useful when a holdings sync is required with an
external vendor.
Signed-off-by: blake <blake@mobiusconsortium.org> Signed-off-by: Chris Sharp <csharp@georgialibraries.org> Signed-off-by: Jason Etheridge <jason@EquinoxInitiative.org>
LP#1048822 - Fuller Title for reporter.super_simple_record
The simplified pull list uses reporter.super_simple_record for
title information. reporter.super_simple_record currently only
pulls the first 245a value for the title. For titles that
use 245(n|p) to fully identify the title this leaves out information
that staff need to find the item.
This patch adds all of the 245(n|p) to the 245a in the title. I've found
this covers most cases where staff need a fuller title. The order of
the 245(n|p) also matters, so steps are taken to keep the marc ordering
of those fields.
Testing:
Before Patch - find a title that uses a 245(n|p) field to fully identify the
title. Place a hold on the title. Use the simplified pull list to view the
location that has that item on the shelf. Observe that only the 245a is in
the title field.
After Patch - Look at the simplified pull list again and see that the title now
includes more information.
Signed-off-by: Josh Stompro <stomproj@larl.org> Signed-off-by: Dan Pearl <dpearl@cwmars.org> Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Skye Howard [Thu, 17 Aug 2017 21:52:08 +0000 (21:52 +0000)]
LP#1629298: Web Client Checkout Fails For Barcodes With Spaces
Web Client checkout trims spaces from patron barcodes differently than xul client checkout
The web client was not removing the whitespaces before checking barcode searches
this commit includes a step where the whitespaces are removed.
Signed off by Skye Howard <showard@catalyte.io>
Signed off by Cesar Velez <cesar.velez@equinoxinitiative.org> Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
New optional SMS text notification to be sent 3 days prior to the due
date of any circulating item for patrons who have an SMS text number
and carrier stored in their accounts. This action trigger is disabled
by default and can be enabled in Admin > Local Administration >
Notifications / Action Triggers.
Mike Rylander [Wed, 9 Aug 2017 18:14:42 +0000 (14:14 -0400)]
LP#1698206: Remove remaining SQL uses of the old copy visibility cache
There were two remaining uses of the old copy vis cache in SQL functions used
to render OPAC pages. This commit gets rid of those.
There is still one reference to the asset.opac_visible_copies table in the old
staged-search function, but that is not used anywhere in the code now, so no
need to change that. Instead, we should start pruning old code.
Mike Rylander [Mon, 10 Jul 2017 14:35:07 +0000 (10:35 -0400)]
LP#1698206: Indicate broad searches and heavy facets
There is a configurable, pagable limit on hits, defined by the superpage
size and max superpages. When the hit count equals this (by default,
100000) we add a '+' to the hit count to show that there are even more hits.
Facets are calculated per superpage, and if the facet use count equals the
number of superpages seen so far multiplied by the superpage size, we
likewise add '+' to indicate that there are likely more records matching the
facet.
For facets, a user can page far enough to increase the visible number, if
they cross superpage boundaries.
The old code needed to refine the estimated hit count as each superpage was
read, but we don't need any of that anymore. Also, it was causing hit count
display issues on superpages after the first. So, we do away with all that.
=== Background
Evergreen stores all data, including that useful for patron and staff search,
in a normalized schema that is time and space efficient for transactional use
cases, and provides guarantees on data integrity. In addition, development is
made simpler than would be the case otherwise and arbitrary reporting is made
possible.
However, this structure is not effective for direct, SQL-only search
functionality in a hierarchical, consortial dataset. This is a problem that
is relatively unique to Evergreen, as it is most often employed to host and
serve large consortia with overlapping bibliographic datasets and
non-overlapping item and location datasets. Other search engines, including
those built into other ILSs, do not generally have to account for
hierarchically organized location visibility concerns as a primary use case.
In other words, because it provides functionality that requires a hierarchical
view of non-bibliographic data, a problem space for Evergreen is essentially
nonexistent in competing products.
Evergreen's search infrastructure has evolved over the years. In its current
form, the software first performs a full text search against extracted
bibliographic data and limits this initial internal result set to a
configurable size. It then investigates the visibility of each result on
several non-bibliographic axes. These visibility tests take up the
preponderance of CPU time spent in search, with full text search of the
bibliographic data generally completing within milliseconds. The main reason
this multi-stage mechanism is used is that there are many visibility axes and
attempting to join all required data sources together in a single query will
cause the search use case to perform very poorly. A previous attempt to
create a pure SQL search mechanism failed for this reason.
A significant drawback of the current approach is that the costs imposed by
visibility filtering search results using normalized non-bibliographic data,
either in-query or separated from the main full-text query as it is today,
make it necessary to place limits on the number of database rows matched by
full-text query constructs. This in turn can cause searches to omit results
in certain situations, such as a large consortium consisting of a few large
libraries and many small libraries.
However, it has been shown possible to overcome this performance issue by
providing an extensible way to collect all visibility related information
together into a small number of novel data structures with a compact in-memory
representation and very fast comparison functions. In this way, we are able
to use pure SQL search strategies and therefore avoid result visibility
problems while also benefiting from improvements to the core PostgreSQL
database engine. Further, this will open the door to indexing improvements,
such as removal of the need for duplicate data storage, or the use of non-XML
data storage schemes, which could reduce resource requirements and have a
direct, positive effect on patron and staff search experience.
=== Overview of existing search logic
. Construct core bibliographic search query
. Collect non-bibliographic filtering criteria
. Pass query and filters to a database function
. Calculate hierarchical location information for visibility testing
. Open cursor over core query, limited to *superpage_size * max_superpages* records
. Core query implements bib-level sorting
. For each result
.. NEXT if not on requested superpage
.. Check deleted flag, based on search type
.. Check transcendence
... Return result if true
.. Check for direct Located URI in scope
... Return result if exists
.. Check copy status + (circ lib | owning lib) based on modifier
.. Check peer bib copy status + (circ lib | owning lib) based on modifier
.. Check copy location based on filter
.. Check peer bib copy location based on filter
.. General copy visibility checks
... If NOT staff
.... Check for OPAC visible copies (trigger-maintained materialization)
.... Check for peer bib OPAC visible copies
... If staff
.... Confirm no copies here
.... Confirm no peer bib map
.... Confirm no copies anywhere
.... Confirm no Located URIs elsewhere
.. Return result if not excluded
. Calculate summary row
=== Overview of new mechanism
Record and copy information (everything checked in *(7)* above) is collected
into a novel data structure that allows all visibility-indicating criteria to
be flattened to integer arrays. This is facilitated by a database trigger in
much the same way that basic OPAC copy visibility is collected for copies
today.
Most identifiers in Evergreen are stored as signed integers of either 32 or 64
bits. The smaller 32 bit space allows for approximately two billion positive
entries, but all identifiers for table rows that are used as visibility axes
fall into a range of between one and one million for all applicable use cases,
and all identifiers of interest are positive. Therefore, we can make use of
the most significant bits in an integer value to create a per-axis namespacing
mask. When applied to the idenfitifer for a visibility axis identifier, this
mask allows two values that are identical across axis to be identified as
unique within a combined set of all values.
Sepcifically, we retain the four most significant bits of the integer space
and create from that 16 potential bitmasks for per-axis segregation of
identifiers. Further, we separate copy-centered axes and bibliographic
record-centered attributes into two separate columns for storage purposes,
which means we can use the same four bits for different purposes within each
copy or bib set.
In order to implement existing visibility tests with this infrastructure, six
copy axes and two record axes are used from the possible 16 from each set.
See the search.calculate_visibility_attribute() for details. By using 32 bit
integers we can collect all of the bitmasked values of each type (copy or bib)
into a single integer array and leverage the Postgres intarray extension to
test all axes at once.
At search time, required and user-requested visibility restrictions are
converted to *query_int* values. Results are directly filtered based on these
calculated *query_int* values. This works in a way analogous to record
attribute filtering, avoiding the need to test statuses, circ and owning
library visibility, copy locations and location groups, copy OPAC visibility,
peer bibliographic record, Located URIs, or bibliographic record sources
directly.
=== Minimum Postgres version requirement
Due to features, particularly functions, available only in 9.4 and newer that
are key to the performance of the new method, Postgres 9.4 will need to be the
new lowest supported version for use with Evergreen. While some of the new
features and functions could be implemented as user-defined functions in
PL/PGSQL, they would not be fast enough to make this pure-SQL search viable.
Among the important improvements that Postgres 9.4 and newer versions bring to
Evergreen are:
* Version 9.4 improved GIN indexes in ways that directly benefit Evergreen, as well as how anti-joins are planned which matters for some Evergreen searches.
* Version 9.5 introduced many general performance improvements, especially for joins and sorting, and brought planner improvements that impact complex queries such as those generated by this code.
* Version 9.6 delivered more general performance improvements, particularly for large servers such as those that Evergreen databases tend to live on, as well as more improvements to GIN indexes, executor changes that can avoid unnecessary work in search queries, new built-in full-text phrase searching, and initial parallel query execution.
=== Performance
The cost of the non-bibliographic filter value caching maintenance process is
10-40% faster than existing partial caching logic which it would replace.
The new code achieves up to 10% faster search times than the old, suboptimal
mechanism time for broad searches. The new code is faster for more selective
searches, often by up to 90% faster. In both broad and narrow search cases
the new mechanism performs with complete accuracy and does not miss
small-collection hits in large consortia as the existing code does.
Unsurprisingly, and in addition to the above improvements, performance is
improved marginally as each successive Postgres version at and beyond 9.4.
=== Page rendering changes
Previously, Evergreen would request the record details for a user-visible page
of results in parallel, and then, serially, request the facet data for the
result set. Now, the facet data is requested asyncronously in the background
and then a single feed containing all records on a result page is requested
syncronously. By parallelizing the result and facet metadata, page rendering
time is cut down significantly. Concurrent requests of the same bibliographic
record are shared between apache backends to reduce result request time, and by
making one request instead of ten simultaineously, database load is reduced. A
performance improvement of up to 20% in post-search page rendering time is seen
from this change.
Additionally, cross-apache caching of ancillary data, such as the coded value
map and other data, via memcache significantly reduces the average page
rendering time not just for result pages, but most pages generated by
Evergreen. An additional performance improvement of up to 50% in post-search
page rendering time is seen from this change.
While these changes are not directly related to the removal staged search, they
touch areas impacted by core search changes and provided enough improvement
that implementing them concurrently with the elimination of staged search
seemed optimal.
=== User visible configuration changes
The stock configuration now provides an increased value for *max_superpages*
in opensrf.xml. The default is now 100, and the *superpage_size* remains
1000, for a total limit of 100,000 hits per search. This is not a limit on
visibility per se, as all records are visibility tested and ranked before
limiting, but simply a limit on the number of pages a user could click through
before reaching the end of the presented result list.
=== Tuning sensitivity
User-level timeouts are still possible with both the old and new code, given a
large enough dataset, a broad enough query, and a cold cache. However, the
*gin_fuzzy_search_limit* GUC can be used to set a time cap on the new
mechanism. See https://www.postgresql.org/docs/9.6/static/gin-tips.html for
background, though the suggested values in the documentation are significantly
lower than would be readily useful for a large Evergreen instance.
Because it uses a more complex query structure, the new mechanism is somewhat
more sensitive to Postgres tuning in general. In particular, lowering
*random_page_cost* from the default of *4.0* to a more reasonable *2.0* is
important for proper query planning. For Evergreen use cases where the search
indexes and relevant tables are kept in RAM or SSDs are used for storage, this
value is acceptable and useful in general.
=== Funding and development
This project was funded by MassLNC and developed by Equinox Open Library
Initiative.
Currently, editing and deleting of users must be performed on a user-by-user
basis. There are workflows that would benefit from the ability to act on a
set of users, where the changes to all users in the set are the same.
This commit provides a new interface analogous to the Copy Bucket interface
to record the selection and grouping of a set of users into a User Bucket.
The addition of users to a User Bucket is possible from the Patron Search
interface by the use of a new grid Action, and directly on the User Bucket
interface by user barcode. It is also possible to add users by uploading
a text file that contains a list of user barcodes.
From this interface it is possible to perform a set of specific batch update
operations against users generally.
Editing users
-------------
In order to facilitate the update of user data fields, specifically:
* Active flag
* Primary Permission Group (group application permissions consulted)
* Juvenile flag
* Home Library (UPDATE_USER checked against both old and new value)
* Privilege Expiration Date
* Barred flag (BAR_PATRON permission consulted)
* Internet Access Level
This commit contains a new set of business logic allowing staff to supply new
values for these fields. Creation and immediate processing of a change set
will be made available through a grid Menu item. If the staff user does not
have the UPDATE_USER permission, this option will be disabled.
Each change set requires a name. Buckets may have multiple change sets. All
users in the Bucket at the time of processing will be updated when the change
set is processed, and change sets are processed immediately upon successful
creation. The interface will deliver progress information regarding the
processing stage and percent of completion.
While processing the users, the original value for each field edited will be
recorded for potential future rollback. Users can examine the success and
failure of applied change sets.
The user will be able to rollback the entire change set, but not parts thereof.
The rollback will affect only those users that were successfully updated by the
original change set and may be different from the current set of users in the
Bucket. Users can manually discard change sets, removing them from the
interface but preventing future rollback.
As a batch process, rather than a direct edit, this mechanism explicitly skips
processing of Action/Trigger event definitions for user update.
Deleting users
--------------
In order to facilitate the batch deletion of users, this commit creates a new
set of business logic allowing staff to set the Deleted flag on users.
Creation and immediate processing of a batch delete is made available through
a grid Menu item. If the staff user does not have both the UPDATE_USER and
DELETE_USER permission, this option is disabled. Because of the potential for
damage and the additional required permission, this field change is
specifically segregated from the general Editing functionally described above.
Each delete set requires a name. Buckets may have multiple delete sets. All
users in the Bucket at the time of processing will be marked as deleted when
the delete set is processed. The interface will deliver progress information
regarding the processing stage and percent of completion.
While processing the users, the original value for the "deleted" field will be
recorded for potential future rollback. Users will be able to examine the
success and failure of applied delete sets in the same interface used for the
above described change sets.
As a batch process, rather than a direct edit, this mechanism explicitly skips
processing of Action/Trigger event definitions for user deletion.
This mechanism does not use the Purge User functionality, but instead simply
marks the users as deleted. Future enhancement could add such functionality.
In order to facilitate the batch editing, addition, and removal of
Statistical Category Entries for users, this commit creates a new set of
business logic allowing staff to either remove or add & update Entries for
Statistical Categories to which the staff member has access. Processing of
Statistical Category Entry modifications will are available through a grid
Menu item.
All users in the bucket will have their Statistical Category Entries
modified. Unlike user data field updates, modification of Statistical
Category Entries is permanent and cannot be rolled back. No named change
sets are required. The interface will deliver progress information regarding
the processing stage and percent of completion.
As a batch process, rather than a direct edit, this mechanism explicitly skips
processing of Action/Trigger event definitions for user update.
New service requirement
-----------------------
This new functionality makes use of the QStore service, which was previously
unused in production. If this service has been removed from the configuration
of a live Evergreen instances, it will need to be added back in order for
batch user editing to succeed.
Signed-off-by: Mike Rylander <mrylander@gmail.com> Signed-off-by: Scott Thomas <scott.thomas@sparkpa.org> Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Conflicts:
Open-ILS/src/templates/staff/circ/patron/index.tt2
Open-ILS/web/js/ui/default/staff/circ/patron/app.js
McCanna [Fri, 25 Aug 2017 13:53:14 +0000 (09:53 -0400)]
LP#1712840 Ebook tabs always visible in My Account
Corrects a problem in My Account where the ebook tabs
on the checkouts and holds pages were always visible
regardless of whether the ebook_api.enabled value was
set to true or false.
Transfer checkout history rows to the destination user during patron
merge.
To Test:
1. Enable circ history tracking for 2 users, ideally 2 with same
last name for easy searching.
2. Check out items to both patrons.
3. Load the 2 patrons in the patron search UI and merge one into the other.
4. action.usr_circ_history rows previously linked to the deleted patron
will now be linked to the active/lead patron.
Signed-off-by: Bill Erickson <berickxx@gmail.com> Signed-off-by: Jason Etheridge <jason@EquinoxInitiative.org>
projects / working / Evergreen.git / log