Jason Stephenson [Thu, 24 Sep 2020 16:31:23 +0000 (12:31 -0400)]
LP#1482757: Speed Up the Delete of Orphaned URIs in upgrade script
Modify the code that deletes orphaned asset.uri table entries in the
upgrade script to use IN logic on the subquery, because IN is much
faster than NOT IN. Also take into account the link between
serial.item and asset.uri so that we don't accidentally delete URIs
used by serial items.
Signed-off-by: Jason Stephenson <jason@sigio.com> Signed-off-by: Jessica Woolford <jwoolford@biblio.org> Signed-off-by: Michele Morgan <mmorgan@noblenet.org>
Michele Morgan [Wed, 29 Jul 2020 20:40:06 +0000 (16:40 -0400)]
LP#1482757: Delete URIs and call numbers when all 856 fields are removed
This patch assures that URIs and call numbers are deleted as appropriate
when all 856 fields are removed from the MARC.
Also provides for tracking editor and edit_date when call numbers are
deleted.
Signed-off-by: Michele Morgan <mmorgan@noblenet.org> Signed-off-by: Jason Stephenson <jason@sigio.com> Signed-off-by: Michele Morgan <mmorgan@noblenet.org>
Michele Morgan [Fri, 24 Apr 2020 17:03:23 +0000 (13:03 -0400)]
LP#1482757: Amend upgrade script to remove existing orphaned URIs
Remove orphaned URIs from the database that have accumulated from past
additions, updates and deletions of records with located URIs.
Signed-off-by: Michele Morgan <mmorgan@noblenet.org> Signed-off-by: Jason Stephenson <jason@sigio.com> Signed-off-by: Michele Morgan <mmorgan@noblenet.org>
Michele Morgan [Fri, 24 Apr 2020 16:43:39 +0000 (12:43 -0400)]
LP#1482757: Remove 'orphaned_uri_list' and 'NOT cn.deleted' criteria
Per Dan Wells' suggestions.
1) One possible "crack" I see for data to fall through is the dead map
selector. The "AND cn.label = '##URI##' AND NOT cn.deleted" seems overly
restrictive. Now, I realize we *shouldn't* have mappings to deleted or
non-##URI## call numbers, but it isn't impossible, and I don't think we'd
have harm in leaving those restrictions out. If a map isn't used, let's
get rid of it regardless, I think.
2) (side note) orphaned_uri_list doesn't seems to be used any more, so
should be removed.
Signed-off-by: Michele Morgan <mmorgan@noblenet.org> Signed-off-by: Jason Stephenson <jason@sigio.com> Signed-off-by: Michele Morgan <mmorgan@noblenet.org>
As reported by Michele Morgan, URIs can be orphaned in some cases, such
as when a location drops one of several 856 entries. This commit tracks
objects at the mapping level, to identify disused URIs even where the
call number level is not changed.
Signed-off-by: Mike Rylander <mrylander@gmail.com> Signed-off-by: Michele Morgan <mmorgan@noblenet.org> Signed-off-by: Jason Stephenson <jason@sigio.com> Signed-off-by: Michele Morgan <mmorgan@noblenet.org>
Instead of recreating all Located URI mappings, we will add any new
ones and remember all those (existing and new) that are in use. Any
existing maps that we don't see in this record, and whose URI is only
used once previously and therefore about to be orphaned, we remove.
Signed-off-by: Mike Rylander <mrylander@gmail.com> Signed-off-by: Michele Morgan <mmorgan@noblenet.org> Signed-off-by: Jason Stephenson <jason@sigio.com> Signed-off-by: Michele Morgan <mmorgan@noblenet.org>
Jeff Godin [Fri, 28 Jan 2022 22:54:38 +0000 (17:54 -0500)]
Avoid fleshing standing penalty org unit children
When fleshing the org unit on standing penalties, exclude child org
units. We don't need them, and they can cause issues when your
standing penalty has a Depth of "Everywhere": you end up with a full
copy of the org unit tree, and you will include this when editing
and saving the user.
Signed-off-by: Jeff Godin <jgodin@tadl.org> Signed-off-by: Chris Sharp <csharp@georgialibraries.org>
Galen Charlton [Mon, 1 Nov 2021 16:36:31 +0000 (12:36 -0400)]
LP#1949389: reduce number of PCRUD requests by subscription manager
This patch changes how the AngularJS subscription manager fetches
serial item templates. In particular, rather than issuing a PCRUD
request for every org unit in the system, it fetches them in one
fell swoop. This fixes an issue where loading the subscription manager
in a large Evergreen consortium could lead to so many requests that
it drives PCRUD into backlog mode, resulting in higher request
latency across the board.
To test
-------
[1] Apply the patch.
[2] Verify that the subscription manager lets you set the receiving
template for each distribution and that the drop-down's contents
changes if the distribution library is changed.
[3] Verify that the Apply Binding Template action works and that
the binding template drop-down(s) include the relevant copy templates.
Note that if a binding template has been set already, the current
binding template is _not_ displayed as the selected value in the drop-down.
This is existing behavior that this patch does not aim to fix.
Sponsored-by: BC Libraries Cooperative Signed-off-by: Galen Charlton <gmc@equinoxOLI.org> Signed-off-by: Mike Rylander <mrylander@gmail.com> Signed-off-by: Jeff Davis <jdavis@sitka.bclibraries.ca>
Fixes the barcode search in the patron search modal in the holds screen.
To test:
1. Navigate to the 'place a hold' screen in a bib record and open
the patron search modal.
2. Enter a patron barcode.
3. The search for the patron fails.
4. Apply the patch and repeat.
Jason Stephenson [Wed, 20 Oct 2021 16:38:46 +0000 (12:38 -0400)]
Lp 1947595: Remove the array_accum aggregate function
Remove the array_accum aggregate from Evergreen because it will need
to be dropped and redefined when upgrading to PostgreSQL 14.
Replace any uses of array_accum with array_agg in the code.
Signed-off-by: Jason Stephenson <jason@sigio.com> Signed-off-by: Jason Boyer <JBoyer@equinoxOLI.org> Signed-off-by: Jane Sandberg <sandbergja@gmail.com>
Galen Charlton [Fri, 7 Jan 2022 23:50:04 +0000 (18:50 -0500)]
LP#1951021: account for change in OverDrive checkout response
The OverDrive circulation API soon will stop providing a direct
download link in favor of displaying a separate fulfillment page
(either as an iframe or a redirect). This patch implements this
change with the redirect option.
Additional information on the API change can be found at
Jane Sandberg [Wed, 27 Oct 2021 17:16:30 +0000 (10:16 -0700)]
LP1903476: Use Github Actions to build antora documentation
This allows documenters to submit a pull request and get a
preview version where they can view their changes. The goal
is to provide quick feedback to documenters on whether their
changes build correctly and appear as desired, without them
having to have a full docs building setup locally.
To test:
1. Use the Github Web site to fork the Evergreen repository.
2. Take the .github/workflows/docs.yml file from this commit.
Add it to any branch of your fork of the Evergreen repository.
3. Click on the Actions tab of your fork in the Github Web
site.
4. Click on the most recent run (at the top of the list).
5. When the build process is done (should take 3-4 minutes),
download and unzip the artifact that appears at the bottom of
the screen. Open index.html in your browser. Confirm that
the docs built properly.
6. On the same branch, make some changes to the documentation
in the docs/modules directory.
7. Repeat steps 3-5. Confirm that your changes are reflected
in the new artifact.
Signed-off-by: Jane Sandberg <sandbej@linnbenton.edu> Signed-off-by: Jason Boyer <JBoyer@equinoxOLI.org>
Jason Stephenson [Mon, 10 Feb 2020 16:09:37 +0000 (11:09 -0500)]
Lp 1862652: pingest.pl reingest record attributes fix
Use the rdeleted parameter when reingesting record attributes in
pingest.pl support script so that passing argument values to the
--attr option makes sense.
Prior to this patch, arguments of the --attr option would be ignored,
and all record attribute fields were reingested for all records,
deleted and not deleted alike.
With this patch, the arguments of the --attr option are respected for
not deleted records. Deleted records still have all attributes
reingested.
Signed-off-by: Jason Stephenson <jason@sigio.com> Signed-off-by: Jane Sandberg <sandbergja@gmail.com>
teaches the Holdings view to determine the number of copies and call
numbers attached to each org unit based on the full data set (via new
API) instead of copies in hand, since we may only have copies in hand
for a subset of child org units.
Signed-off-by: Bill Erickson <berickxx@gmail.com> Signed-off-by: Mary Llewellyn <mllewell@biblio.org> Signed-off-by: Jane Sandberg <sandbergja@gmail.com>
Galen Charlton [Thu, 11 Nov 2021 16:21:11 +0000 (11:21 -0500)]
LP#1949910: serialize deleting items from item bucket
This patch serializes the fetching and fleshing of items to delete
when the delete-from-item-bucket action is invoked, thereby
avoiding possible drone starvation from open-ils.search. It
also throws up the progress dialog for good measure while
the deletion occurs.
To test
-------
[1] Set up an item bucket with 25-50 items.
[2] From the item bucket intervace, invoke the action to delete
all of the items.
[3] Note that the open-ils.search.asset.copy.fleshed2.retrieve calls,
one for each item, are all made simultaneously. Assuming stock
max_children settings, this will cause open-ils.search to log
that some requests are hitting the backlog.
[4] Apply the patch and refresh the item bucket page, then repeat
step 2 (it doesn't matter for this purpose that the items have
already been deleted). This time, the open-ils.search API calls
are made serially and a progress bar is displayed while the
deletion takes place. Verify that no requests hit the backlog.
Signed-off-by: Galen Charlton <gmc@equinoxOLI.org> Signed-off-by: Jeff Davis <jdavis@sitka.bclibraries.ca> Signed-off-by: Jane Sandberg <sandbergja@gmail.com>
Bill Erickson [Tue, 30 Jul 2019 16:12:12 +0000 (12:12 -0400)]
LP1739277 Angular org selector style callback
Allow the caller of an <eg-org-select> to pass a function which is
called against each org unit in the list to determine what CSS class(es)
should be applied to the org unit label in the list.
Includes sandbox example.
Signed-off-by: Bill Erickson <berickxx@gmail.com> Signed-off-by: Jennifer Bruch <jbruch@sparkpa.org> Signed-off-by: Jane Sandberg <sandbej@linnbenton.edu>
Galen Charlton [Mon, 8 Nov 2021 17:00:42 +0000 (12:00 -0500)]
LP#1855780: avoid race condition when deleting env or params
When deleting from a grid using pcrud.autoApply, the grid
should be refreshed only at the very end. Otherwise, even if you're
deleting only one row, the grid reload could occur before the
deletion is committed, yielding a grid fetch error when
pcrud complains about verifying access to a row that is
no longer in the database.
Signed-off-by: Galen Charlton <gmc@equinoxOLI.org> Signed-off-by: Jane Sandberg <sandbergja@gmail.com>
Mike Risher [Thu, 19 Dec 2019 17:56:41 +0000 (17:56 +0000)]
LP#1855780 Angular Notification/Action Triggers port
Port Notification Action Triggers from DOJO to Angular. This consists of 4 grids, each
navigated to by its corresponding tab. The Trigger Event Definitions grid allows
cloning of records. When editing an event defintion one can also edit parameters,
environments, and run tests.
Signed-off-by: Mike Risher <mrisher@catalyte.io> Signed-off-by: Galen Charlton <gmc@equinoxOLI.org> Signed-off-by: Jane Sandberg <sandbergja@gmail.com>
Jason Boyer [Fri, 6 Sep 2019 18:10:45 +0000 (14:10 -0400)]
LP1528301: Hold Notify Phone in Checkin Response
Adds a setting to oils_sip.xml.sample to en/dis-able
the extention of including the hold notify phone number
for captured holds in the SIP Checkin Response.
Defaults to false.
blake [Thu, 9 Jun 2016 14:04:56 +0000 (09:04 -0500)]
LP1528301 Add SIP Support for BF field on type 10 checkin responses
This little patch will append the BF field in the SIP response during
checkin when the item is for another hold. Some self check software vendors
(ahem Bibliotheca) implement the hold phone number for the next patron this way.
You will need this patch as well as the code change to the SIPServer repo
LP1528301_Add_SIP_Support_for_BF_field_on_type_10_checkin_responses
Adjust the eg-basic-combo-box so that it doesn’t give console errors
when typing characters such as [. Fix the bug that causes it to
sometimes show no results when there are valid results that match.
To test
-------
[1] Apply the patch, then create a new report template, add a
display field, then open the Change Transform modal.
[2] Verify that the selection box will accept characters like '['
as input without browser console noise, that the filtering
is case-insensitive, and that filtering continues to work
after clicking on the selector.
[3] Instances of this combo box can also be found in the AngularJS
volume/copy editor.
LP1906842 PLACE_UNFILLABLE_HOLD permission not working if no titles exist
This fix will correct an issue where the screen just keeps looping to
the override screen when trying to place a hold that has no titles
available for it.
Jason Boyer [Fri, 24 Sep 2021 13:12:20 +0000 (09:12 -0400)]
LP1831803: Remove unused data types
The search.search_args and search.search_result types appear to have only
been used in combination with search.query_parser_qts, so they can also
be retired.
Signed-off-by: Jason Boyer <JBoyer@equinoxOLI.org> Signed-off-by: Galen Charlton <gmc@equinoxOLI.org>
Jeff Davis [Fri, 18 Oct 2019 19:44:52 +0000 (12:44 -0700)]
LP#1831803: release note for removing old staged search stuff from db
Signed-off-by: Jeff Davis <jeff.davis@bc.libraries.coop> Signed-off-by: Jason Boyer <JBoyer@equinoxOLI.org> Signed-off-by: Galen Charlton <gmc@equinoxOLI.org>
Jeff Davis [Fri, 18 Oct 2019 19:26:03 +0000 (12:26 -0700)]
LP#1831803: remove asset.refresh_opac_visible_copies_mat_view if it exists too
Signed-off-by: Jeff Davis <jeff.davis@bc.libraries.coop> Signed-off-by: Jason Boyer <JBoyer@equinoxOLI.org> Signed-off-by: Galen Charlton <gmc@equinoxOLI.org>
Jeff Davis [Wed, 5 Jun 2019 21:30:11 +0000 (14:30 -0700)]
LP#1831803: remove obsolete asset.opac_visible_copies and search.query_parser_fts from database
Signed-off-by: Jeff Davis <jeff.davis@bc.libraries.coop> Signed-off-by: Jason Boyer <JBoyer@equinoxOLI.org> Signed-off-by: Galen Charlton <gmc@equinoxOLI.org>
Jeff Davis [Thu, 6 Jun 2019 21:30:34 +0000 (14:30 -0700)]
LP#1830089: close transaction and update copy status on payment or adjust to zero
When you make a payment that sets the balance owed to zero, Evergreen
closes the transaction and sets the item status to Lost & Paid (if
appropriate). Adjust to Zero should do the same thing, but hitherto it
would not update the item status. This commit refactors some code to
ensure that Evergreen gives the same result whether you make a payment
or adjust to zero.
Signed-off-by: Jeff Davis <jeff.davis@bc.libraries.coop> Signed-off-by: Terran McCanna <tmccanna@georgialibraries.org> Signed-off-by: Galen Charlton <gmc@equinoxOLI.org>
Jason Stephenson [Sun, 24 Oct 2021 18:38:52 +0000 (14:38 -0400)]
LP1948594: Fix pgtap live test
Fix the lp1145213_test_func_asset.merge_record_assets.pg pgtap live
test by removing the COMMIT and adding a ROLLBACK at the end. This
permits the test to run multiple times without requiring a database
reload.
To verify the bug:
1. Install the pgtap package appropriate for your PostgreSQL database
version.
2. Load a fresh database with eg_db_config using the --load-all-sample
option.
3. Create the pgtap extension in the evergreen database.
4. Run the Open-ILS/src/sql/Pg/live_t/ tests with pg_prove.
5. Repeat step 4, and if this patch has not been applied, then the
lp1145213_test_func_asset.merge_record_assets.pg tests will fail on
the second run.
To test the fix, repeat steps 2 - 5, above, after pulling this commit.
Step 5 should succeed.
Signed-off-by: Jason Stephenson <jason@sigio.com> Signed-off-by: Galen Charlton <gmc@equinoxOLI.org>
If a MARC record has effectively empty fields (all space or punctuation)
then it's possible to stumble over a NULL text array when parsing
content for the symspell dictionary. This commit checks for that
condition and returns early to avoid the problem.
Signed-off-by: Mike Rylander <mrylander@gmail.com> Signed-off-by: Galen Charlton <gmc@equinoxOLI.org>
For certain data, and certain data set sizes, merging the suggestion
arrays used by the symspell algorithm is noticably expensive. This is
the case for suggestion arrays containing many thousands of entries.
These suggestion sets are not only slow, but generally not useful. We
avoid the creation of such overly long suggestion sets using several
word filters that take advantage of our knowledge of the incoming data
to optimize for what is useful in a bibliographic context. The
mechanisms employed by this patch are:
- Omit suggestions whose length is longer than the max prefix key length
when the prefix key length is less than or equal to the maximum prefix
key length minus the maximum edit distance.
- Omit words that contain a run of 5 or more digits. This will drop most
identifiers from the dictionary while still allowing suggestions to
happen for year values.
- Omit empty keys from the dictionary. This should have been the case
already but is now enforced directly.
- Add a small speedup to evergreen.text_array_merge_unique() by making
it assume that arrays passed to it do not have null values, which we
intentionally avoid, and against which we protect in other ways in the
commit.
Besides improving reingest speed, the patches will also make the
search.symspell_dictionary table significantly smaller.
Signed-off-by: Mike Rylander <mrylander@gmail.com> Signed-off-by: Galen Charlton <gmc@equinoxOLI.org>