working/Evergreen.git
8 years agoSeed data and schema cleanup for merge
Bill Erickson [Wed, 6 Jul 2011 16:52:04 +0000 (12:52 -0400)]
Seed data and schema cleanup for merge

 * Updated A/T event-def ID in seed data to match master
 * Move schema elements into the correct order in the baseline files (mike)

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Bill Erickson <berick@esilibrary.com>
8 years agoLogic error fixes in Vandelay
Bill Erickson [Tue, 31 May 2011 19:24:09 +0000 (15:24 -0400)]
Logic error fixes in Vandelay

* When counting import failures, also count records that have no import
items attached

* Fixed problem w/ not setting import time/as on newly imported records

Signed-off-by: Bill Erickson <berick@esilibrary.com>
8 years agoRepaired typo on schema and upgrade SQL
Bill Erickson [Tue, 31 May 2011 16:16:16 +0000 (12:16 -0400)]
Repaired typo on schema and upgrade SQL

ARRAY_ACUM is not a defined function.  Per Dan Scott, replaced with
ARRAY_AGG instead of the correctly spelled ARRAY_ACCUM, since we'll be
using that function in place of ARRAY_ACCUM going forward.

Thanks, Dan.

Minor syntax/cleanup repairs

Remove tmp seed data file.  T'was replaced with upgrade script

Signed-off-by: Bill Erickson <berick@esilibrary.com>
8 years agoCorrect regression that was held over due to massive moving of functions
Mike Rylander [Thu, 26 May 2011 20:23:18 +0000 (16:23 -0400)]
Correct regression that was held over due to massive moving of functions

upgrade script cleanup

Signed-off-by: Mike Rylander <mrylander@gmail.com>
8 years agoseed data for vandlelay export templates
berick [Thu, 26 May 2011 14:32:15 +0000 (10:32 -0400)]
seed data for vandlelay export templates

8 years agoInitial upgrade script for vandelay improvements
berick [Thu, 26 May 2011 14:30:59 +0000 (10:30 -0400)]
Initial upgrade script for vandelay improvements

TODO:
    * Set up the ALTER TABLE statements
    * Further review
    * Testing

8 years agorepair selector column width (move to width attr) to free up space in queue display
berick [Tue, 24 May 2011 13:16:10 +0000 (09:16 -0400)]
repair selector column width (move to width attr) to free up space in queue display

8 years agodisable sorting on selector column in queue grid
berick [Tue, 24 May 2011 13:15:13 +0000 (09:15 -0400)]
disable sorting on selector column in queue grid

8 years agoavoid pile-up of dojo-attached event handlers for queue upload inputs
berick [Mon, 23 May 2011 18:17:08 +0000 (14:17 -0400)]
avoid pile-up of dojo-attached event handlers for queue upload inputs

8 years agoset imported_as / import_time on import items after successful import
berick [Mon, 23 May 2011 18:16:32 +0000 (14:16 -0400)]
set imported_as / import_time on import items after successful import

8 years agoYet more Vandelay fixes
berick [Mon, 23 May 2011 15:51:05 +0000 (11:51 -0400)]
Yet more Vandelay fixes

* Remove the deprecated .nomatch API call.  no-match is now a runtime
option

* Re-fetch queued record after auto-import to avoid clobbering
imported_as as set by the DB during auto import

8 years agoSpacing and code comments to keep future-miker from becoming confused again by the...
Mike Rylander [Sat, 21 May 2011 13:05:04 +0000 (09:05 -0400)]
Spacing and code comments to keep future-miker from becoming confused again by the lack thereof

Signed-off-by: Mike Rylander <mrylander@gmail.com>
8 years agoShow the "match score," a hopefully clearer term for match point quality,
Lebbeous Fogle-Weekley [Fri, 20 May 2011 22:12:58 +0000 (18:12 -0400)]
Show the "match score," a hopefully clearer term for match point quality,

in the expression tree UI

Signed-off-by: Lebbeous Fogle-Weekley <lebbeous@esilibrary.com>
8 years agoAdd items-imported to VL queue summary
berick [Fri, 20 May 2011 21:13:13 +0000 (17:13 -0400)]
Add items-imported to VL queue summary

Added total_items_imported value to queue summary API call.  Displaying
value in vandelay queue summary area.

8 years agoTrack import time and target copy on import items
berick [Fri, 20 May 2011 21:06:05 +0000 (17:06 -0400)]
Track import time and target copy on import items

Added import_time and imported_as columns to vandelay.import_item

For you branch trackers:

ALTER TABLE vandelay.import_item
    ADD import_time TIMESTAMP WITH TIME ZONE;

ALTER TABLE vandelay.import_item
    ADD imported_as BIGINT REFERENCES
    asset.copy (id) DEFERRABLE INITIALLY DEFERRED;

8 years agomore delineation of quality vs. score in match set config
berick [Fri, 20 May 2011 17:45:52 +0000 (13:45 -0400)]
more delineation of quality vs. score in match set config

8 years agoMatch set point quality terminology change
berick [Fri, 20 May 2011 17:26:03 +0000 (13:26 -0400)]
Match set point quality terminology change

Change "Quality" in the context of a match set
point to "Match Score Value" to avoid overriding the term "quality".

8 years agoReturn to dijit.form.Button for VL upload form
berick [Fri, 20 May 2011 17:23:08 +0000 (13:23 -0400)]
Return to dijit.form.Button for VL upload form

Otherwise the form is not correctly POSTed

8 years agoshow activity dialog on vandelay item export
berick [Fri, 20 May 2011 17:20:40 +0000 (13:20 -0400)]
show activity dialog on vandelay item export

8 years agoMore Vandelay text and style cleanup
berick [Fri, 20 May 2011 13:07:44 +0000 (09:07 -0400)]
More Vandelay text and style cleanup

* More consistent use of the word "match"
* Make upload input more obvious w/ a border
* Move some strings into DTD

8 years agoFlexible TCN match configuration
berick [Thu, 19 May 2011 19:40:30 +0000 (15:40 -0400)]
Flexible TCN match configuration

If users do not want records with duplicate TCN values imported, users
should rely on match-sets to enforce this policy instead of a hard-coded
duplicate tcn block.  This is done by automatically calling the 'override'
version of bib XML import.  The only tcn-related failure that can now
occur is if no free, non-dupe TCN value can be extracted from the record.

8 years agoVandelay import improvements / bug fixes
berick [Thu, 19 May 2011 19:26:49 +0000 (15:26 -0400)]
Vandelay import improvements / bug fixes

 * Allow for quality ratio control during import-on-1match
 * Allow for 901c matching when no match-set is selected
 * Tidy up some display strings
 * Numerous small bug fixes

8 years agoClean up match_set data on delete
berick [Thu, 19 May 2011 16:37:52 +0000 (12:37 -0400)]
Clean up match_set data on delete

Delete via cascade match_set_point's and match_set_quality entries
for a given match_set on delete

8 years agoDisplay org shortname in merge profile editor
berick [Thu, 19 May 2011 14:23:36 +0000 (10:23 -0400)]
Display org shortname in merge profile editor

...instead of org unit ID

8 years agoreturn after Email sent alert
Jason Etheridge [Sat, 14 May 2011 17:44:56 +0000 (13:44 -0400)]
return after Email sent alert

8 years agoThis function also gets used with authority queues, so don't assume import_items()
Jason Etheridge [Sat, 14 May 2011 17:40:36 +0000 (13:40 -0400)]
This function also gets used with authority queues, so don't assume import_items()

8 years agoemail template for import items export
Jason Etheridge [Sat, 14 May 2011 17:30:29 +0000 (13:30 -0400)]
email template for import items export

8 years agotweak formatting for CSV export templates, and add headers
Jason Etheridge [Sat, 14 May 2011 17:21:02 +0000 (13:21 -0400)]
tweak formatting for CSV export templates, and add headers

8 years agoCSV template for import items export
Jason Etheridge [Sat, 14 May 2011 17:15:15 +0000 (13:15 -0400)]
CSV template for import items export

8 years agorobustify exportHandler, assume response of 1 is success for email methods, and make...
Jason Etheridge [Sat, 14 May 2011 16:58:31 +0000 (12:58 -0400)]
robustify exportHandler, assume response of 1 is success for email methods, and make import_item export methods atomic

8 years agotoward Import Item templates for export. May want to consider providing IDL links...
Jason Etheridge [Fri, 13 May 2011 03:18:57 +0000 (23:18 -0400)]
toward Import Item templates for export.  May want to consider providing IDL links for some of these fields so we can flesh them in the environment

8 years agoRemove excised field from IDL
Mike Rylander [Fri, 13 May 2011 03:05:35 +0000 (23:05 -0400)]
Remove excised field from IDL

8 years agofix variable name for Import Items Export
Jason Etheridge [Fri, 13 May 2011 02:35:22 +0000 (22:35 -0400)]
fix variable name for Import Items Export

8 years agochange the group field for these templates, though they weren't breaking anything...
Jason Etheridge [Fri, 13 May 2011 01:42:43 +0000 (21:42 -0400)]
change the group field for these templates, though they weren't breaking anything before (these are called with fire_object_event and instead of create_events_for_hook, if that makes a difference)

8 years agotemplates for exporting authority queues, but needs testing. blockers here include...
Jason Etheridge [Fri, 13 May 2011 01:38:54 +0000 (21:38 -0400)]
templates for exporting authority queues, but needs testing.  blockers here include importing authority records being broken, and possibly selection of an authority queue to inspect being broken

8 years agoemail template for queued bib records
Jason Etheridge [Thu, 12 May 2011 23:50:08 +0000 (19:50 -0400)]
email template for queued bib records

8 years agoIn the event that a bib queue is not using a match_set, bail out of...
senator [Wed, 11 May 2011 15:44:02 +0000 (11:44 -0400)]
In the event that a bib queue is not using a match_set, bail out of...

... match_set_test_marcxml() early.  Needs tested.  Also may need
special check for non-null but empty match_sets.

8 years agoCSV template and file saving worked out
Jason Etheridge [Wed, 11 May 2011 06:42:30 +0000 (02:42 -0400)]
CSV template and file saving worked out

8 years agofirst template and supporting code, printing queued bib records
Jason Etheridge [Wed, 11 May 2011 05:39:32 +0000 (01:39 -0400)]
first template and supporting code, printing queued bib records

8 years agoremove .atomic. from these method names
Jason Etheridge [Tue, 10 May 2011 17:18:18 +0000 (13:18 -0400)]
remove .atomic. from these method names

8 years agoadd .atomic to the end of the queue export API calls where it belongs
berick [Tue, 10 May 2011 14:54:37 +0000 (10:54 -0400)]
add .atomic to the end of the queue export API calls where it belongs

8 years agowire in the event firing, though we still need the templates. Put streaming back...
Jason Etheridge [Tue, 10 May 2011 04:29:56 +0000 (00:29 -0400)]
wire in the event firing, though we still need the templates.  Put streaming back in for the print/csv/email export methods, since we can respond_complete for email and it to doesn't hurt the other two.

8 years agoregister the methods we're expecting for the print/csv/email exports in Vandelay...
Jason Etheridge [Tue, 10 May 2011 03:50:31 +0000 (23:50 -0400)]
register the methods we're expecting for the print/csv/email exports in Vandelay, and remove limit/offset options for such methods, and expectation of streaming

8 years agoinitial export drop-down for items UI
berick [Fri, 6 May 2011 19:37:32 +0000 (15:37 -0400)]
initial export drop-down for items UI

8 years agoinitial export drop-down for queue UI
berick [Fri, 6 May 2011 18:46:25 +0000 (14:46 -0400)]
initial export drop-down for queue UI

8 years agomore import error tracking and reporting bug fixes
berick [Thu, 5 May 2011 18:56:39 +0000 (14:56 -0400)]
more import error tracking and reporting bug fixes

8 years agorun zz_match_bibs_trigger before update/insert so NEW.quality:= will work
berick [Thu, 5 May 2011 18:11:10 +0000 (14:11 -0400)]
run zz_match_bibs_trigger before update/insert so NEW.quality:= will work

8 years agoexpose new import options to queue interface importer dialog; mucho import bug fixes...
berick [Thu, 5 May 2011 17:21:56 +0000 (13:21 -0400)]
expose new import options to queue interface importer dialog; mucho import bug fixes in the middle layer code; expose queued rec quality in matches interface

8 years agoadded edit support for lwm_ratio on merge_profile; default ratio to that of the merge...
berick [Thu, 5 May 2011 14:28:15 +0000 (10:28 -0400)]
added edit support for lwm_ratio on merge_profile; default ratio to that of the merge profile if selected and if a ratio is set on the profile

8 years agoadded missing lwm_ratio field to vandelay.merge_profile
berick [Thu, 5 May 2011 14:27:31 +0000 (10:27 -0400)]
added missing lwm_ratio field to vandelay.merge_profile

8 years agorecord import error handling improvements
berick [Thu, 5 May 2011 14:01:22 +0000 (10:01 -0400)]
record import error handling improvements

8 years agothe vandelay match set editor now does quality metrics too
senator [Wed, 4 May 2011 21:33:29 +0000 (17:33 -0400)]
the vandelay match set editor now does quality metrics too

8 years agoBest Match w/ quality ratio merging
berick [Wed, 4 May 2011 21:15:03 +0000 (17:15 -0400)]
Best Match w/ quality ratio merging

 * Mew Best Match option in merge/overlay
 * support for upload time minimum record quality ratio
 * We now /only/ create new records if no matches exist and the user
   selected a match-free import
 * Selecting a match-free import no longer trumps other merge imports.
   They both work together now.

TODO: find out why queued record quality is not getting set at queued
rec creation time.

8 years agoworking in quality metric editor
senator [Wed, 4 May 2011 20:41:20 +0000 (16:41 -0400)]
working in quality metric editor

8 years agograb attr value from vandelay.extract_rec_attrs hstore to compare to configured value
berick [Wed, 4 May 2011 19:58:37 +0000 (15:58 -0400)]
grab attr value from vandelay.extract_rec_attrs hstore to compare to configured value

8 years agorepaired copy/paste error in match_bib_record
berick [Wed, 4 May 2011 19:45:55 +0000 (15:45 -0400)]
repaired copy/paste error in match_bib_record

8 years agoAllow different quality measures for different values on the same tag+sf/svf
Mike Rylander [Wed, 4 May 2011 19:44:19 +0000 (15:44 -0400)]
Allow different quality measures for different values on the same tag+sf/svf

8 years agoSave incumbent record quality in bib_match.quality, incoming in queued_record.quality
Mike Rylander [Wed, 4 May 2011 18:43:59 +0000 (14:43 -0400)]
Save incumbent record quality in bib_match.quality, incoming in queued_record.quality

8 years agoinitial support for selecting merge-on-best-match and setting minimum quality ratio
berick [Wed, 4 May 2011 18:37:25 +0000 (14:37 -0400)]
initial support for selecting merge-on-best-match and setting minimum quality ratio

8 years agoshow match score/quality in VL record match grid
berick [Wed, 4 May 2011 17:14:38 +0000 (13:14 -0400)]
show match score/quality in VL record match grid

8 years agoadded missing bib_match.match_score field to IDL
berick [Wed, 4 May 2011 17:14:01 +0000 (13:14 -0400)]
added missing bib_match.match_score field to IDL

8 years agoslight mods to vandelay match set page to sync w/ updated bib/auth match table layout
berick [Wed, 4 May 2011 16:03:22 +0000 (12:03 -0400)]
slight mods to vandelay match set page to sync w/ updated bib/auth match table layout

8 years agoGive authority flattening the same treatment as biblio flattening
Mike Rylander [Wed, 4 May 2011 15:52:31 +0000 (11:52 -0400)]
Give authority flattening the same treatment as biblio flattening

8 years agoOnly ingest bibs and items and run matches /before/ we import a bib
Mike Rylander [Wed, 4 May 2011 15:08:13 +0000 (11:08 -0400)]
Only ingest bibs and items and run matches /before/ we import a bib

8 years agomore vandelay menubar slimming
berick [Wed, 4 May 2011 14:45:57 +0000 (10:45 -0400)]
more vandelay menubar slimming

8 years agodon't match on deleted bib records
berick [Wed, 4 May 2011 14:45:40 +0000 (10:45 -0400)]
don't match on deleted bib records

8 years agoalso remove matched_set from IDL
berick [Wed, 4 May 2011 14:13:42 +0000 (10:13 -0400)]
also remove matched_set from IDL

8 years agoClean up uses of vandelay.incoming_record_quality to provide all required params...
Mike Rylander [Wed, 4 May 2011 00:23:57 +0000 (20:23 -0400)]
Clean up uses of vandelay.incoming_record_quality to provide all required params; Remove matched_set on matches, just look at the match_set for the queue of the vandelay record in question

8 years agoAdd MARC::Charset dance to the remaining plperlu function
Mike Rylander [Wed, 4 May 2011 00:06:46 +0000 (20:06 -0400)]
Add MARC::Charset dance to the remaining plperlu function

8 years agomatch set selection support in vl uploage UI, part 1
berick [Tue, 3 May 2011 19:18:55 +0000 (15:18 -0400)]
match set selection support in vl uploage UI, part 1

8 years agodelete match_set_point's starting with leaf nodes to avoid foreign key constraints...
berick [Tue, 3 May 2011 18:42:09 +0000 (14:42 -0400)]
delete match_set_point's starting with leaf nodes to avoid foreign key constraints on nodes w/ children

8 years agocreate an embedded view of the vandelay match-set admin ui inside of vandelay (as...
berick [Tue, 3 May 2011 17:55:22 +0000 (13:55 -0400)]
create an embedded view of the vandelay match-set admin ui inside of vandelay (as one of the tabs)

8 years agoadded support for viewing all import-items related to a queue, with a filter to limit...
berick [Tue, 3 May 2011 15:11:11 +0000 (11:11 -0400)]
added support for viewing all import-items related to a queue, with a filter to limit to those that failed import

8 years agoadded view for all failed item imports for a queue; next up is export options
berick [Mon, 2 May 2011 21:48:29 +0000 (17:48 -0400)]
added view for all failed item imports for a queue; next up is export options

8 years agoadded support for 'limit records to those with any import errors' in vandelay queue...
berick [Mon, 2 May 2011 19:51:15 +0000 (15:51 -0400)]
added support for 'limit records to those with any import errors' in vandelay queue display

8 years agoadded 'with_import_error' option to queued record retrieval; returns recs with eithe...
berick [Mon, 2 May 2011 19:49:33 +0000 (15:49 -0400)]
added 'with_import_error' option to queued record retrieval;  returns recs with either a record or item import failure (or both)

8 years agoVandelay file upload page cleanup
berick [Mon, 2 May 2011 16:52:30 +0000 (12:52 -0400)]
Vandelay file upload page cleanup

 * Separate data configuration parameters from import actions
 * Since import item profile is linked to queue, show the profile for
    the selected queue and dissallow changing the profile for existing
    queues in the UI (which has no effect)

8 years agoadded new stub actions for limit-to-import-errors and show all item import errors...
berick [Mon, 25 Apr 2011 20:34:54 +0000 (16:34 -0400)]
added new stub actions for limit-to-import-errors and show all item import errors; displaying new summary info; much rearrangement to bettet fit the UI pieces

8 years agoadd error and item import/error summary info to queue summary api call
berick [Mon, 25 Apr 2011 20:33:48 +0000 (16:33 -0400)]
add error and item import/error summary info to queue summary api call

8 years agoNo longer limit to non-imported items by default in queue display
berick [Mon, 25 Apr 2011 18:16:35 +0000 (14:16 -0400)]
No longer limit to non-imported items by default in queue display

8 years agohide item import error grid when not in use
berick [Mon, 25 Apr 2011 17:45:21 +0000 (13:45 -0400)]
hide item import error grid when not in use

misc. bug fixes

8 years agoshow rec summary and import failure reason when appropriate
berick [Mon, 25 Apr 2011 15:38:52 +0000 (11:38 -0400)]
show rec summary and import failure reason when appropriate

8 years agoshow item import error counts and rec import error as link to error page; initial...
berick [Mon, 25 Apr 2011 15:02:55 +0000 (11:02 -0400)]
show item import error counts and rec import error as link to error page;  initial error page

8 years agorecord / item import failures summar links
berick [Mon, 25 Apr 2011 13:58:19 +0000 (09:58 -0400)]
record / item import failures summar links

8 years agoadded virtual field for import_items to queued record
berick [Mon, 25 Apr 2011 13:57:13 +0000 (09:57 -0400)]
added virtual field for import_items to queued record

8 years agoadded option to flesh import items on queued record retrieval
berick [Mon, 25 Apr 2011 13:56:58 +0000 (09:56 -0400)]
added option to flesh import items on queued record retrieval

8 years agoMake sort order of imported item retrieval stable
Mike Rylander [Sat, 23 Apr 2011 15:47:27 +0000 (11:47 -0400)]
Make sort order of imported item retrieval stable

8 years agoadded api call open-ils.vandelay.import_item.queue.retrieve for fetching import-items...
berick [Fri, 22 Apr 2011 20:58:15 +0000 (16:58 -0400)]
added api call open-ils.vandelay.import_item.queue.retrieve for fetching import-items by queue, w/ optional has-import-failure filter

8 years agoadded with_rec_import_error and with_item_import_error filters to records-by-queue...
berick [Fri, 22 Apr 2011 20:14:47 +0000 (16:14 -0400)]
added with_rec_import_error and with_item_import_error filters to records-by-queue retrieval

8 years agorec import bug error capturing bug fixes
berick [Fri, 22 Apr 2011 17:42:08 +0000 (13:42 -0400)]
rec import bug error capturing bug fixes

8 years agoremove unused match_attr and actually capture eg_record into eg_id
Mike Rylander [Fri, 22 Apr 2011 17:19:22 +0000 (13:19 -0400)]
remove unused match_attr and actually capture eg_record into eg_id

8 years agoSecondary sort by quality ratio (higher means incoming is more "good" than existing...
Mike Rylander [Fri, 22 Apr 2011 16:55:37 +0000 (12:55 -0400)]
Secondary sort by quality ratio (higher means incoming is more "good" than existing) instead of simple incoming quality, which is a no-op (all the same)

8 years agofunctions for performing queue-wide vandelay.auto_overlay_bib_record_with_best
Mike Rylander [Fri, 22 Apr 2011 16:33:16 +0000 (12:33 -0400)]
functions for performing queue-wide vandelay.auto_overlay_bib_record_with_best

8 years agomatch_score column for recording matchiness; functions for auto-overlay based on...
Mike Rylander [Fri, 22 Apr 2011 16:30:17 +0000 (12:30 -0400)]
match_score column for recording matchiness; functions for auto-overlay based on "bestness"; low-water-mark boundary for best-auto-overlay

8 years agoBugfix for negative matches
senator [Fri, 22 Apr 2011 16:12:30 +0000 (12:12 -0400)]
Bugfix for negative matches

8 years agosome changes in the UI to reflect DB side changes
senator [Fri, 22 Apr 2011 15:45:49 +0000 (11:45 -0400)]
some changes in the UI to reflect DB side changes

Negation and quality are now ignored for operator nodes, and are not
settable in the UI.

Also in the UI, you can no longer replace the root node of the tree with
a non-op node, meaning the root of a tree is always AND or OR.

This is due to some bug that hangs the UI now when you try to save a
single-non-op-node tree, and it's not worth debugging: just remove the
feature.

8 years agoReplace vandelay.match_bib_record() with the new tree-y version
senator [Fri, 22 Apr 2011 15:29:24 +0000 (11:29 -0400)]
Replace vandelay.match_bib_record() with the new tree-y version

8 years agoWe don't have a matched_attr column anymore, because we're using the fancy expression...
Mike Rylander [Fri, 22 Apr 2011 15:14:21 +0000 (11:14 -0400)]
We don't have a matched_attr column anymore, because we're using the fancy expression tree, so test for 901c match directly

8 years agoinitial cut of capturing bib/auth import errors
berick [Fri, 22 Apr 2011 14:38:38 +0000 (10:38 -0400)]
initial cut of capturing bib/auth import errors

8 years agoinitial item-import-failure reporting; needs testing
berick [Thu, 21 Apr 2011 20:55:26 +0000 (16:55 -0400)]
initial item-import-failure reporting;  needs testing