1 Evergreen 3.1 Release Notes
2 ===========================
8 This release contains bug fixes improving on Evergreen 3.1.0.
10 * Fixes a performance issue with the Patron Billing History screen and
11 other screens that cause Flattener.pm to re-create joins
13 * Fixes an issue that prevented patron alerts from showing to staff at
15 * Corrects the "Holdable" attribute display on the Item Status detailed
17 * Fixes the ability to delete multiple copies from Item Status.
21 We would like to thank the following individuals who contributed code,
22 tests and documentation patches to the 3.1.1 point release of
37 Like many major Evergreen upgrades, 3.1 requires a full reingest of your
38 bibliographic records before the system is usable again. While a basic reingest
39 is included at the end of the upgrade script, it happens after the main
40 COMMIT, so it is safe to cancel that and run the required reingest as you see
41 fit (e.g. via pingest.pl).
50 New Latency Tester Tool
51 ^^^^^^^^^^^^^^^^^^^^^^^
52 The Evergreen Web Staff Client now includes a section called *Tests* linked from
53 *Administration -> Workstation*. The *Tests* page houses a simple tool
54 that can be used to test the latency of the websocket connection between the
55 client and the server (via the `opensrf.echo` service).
57 This page displays which Evergreen host server is being queried. Upon hitting
58 the blue "Start Test" button for the first time, it will issue 10 sequentially
59 fired requests in order to get a solid initial average. Clicking the button a
60 second time will take one more measurement and recalculate the average
61 latency. The results can be copied to clipboard for troubleshooting purposes
62 and also cleared from display.
64 marc_export --uris option
65 ^^^^^^^^^^^^^^^^^^^^^^^^^
66 The marc_export support script now has a `--uris` option (short form:
67 `-u`) to export records with located URIs (i.e. electronic resources). When
68 used by itself, it will export only records that have located URIs. When
69 used in conjunction with `--items`, it will add records with located URIs
70 but no items/copies to the output. If combined with a `--library` or
71 `--descendants` option, this option will limit its output to those
72 records with URIs at the designated libraries. The best way to use
73 this option is in combination with the `--items` and one of the
74 `--library` or `--descendants` options to export *all* of a library's
75 holdings both physical and electronic.
81 Sample Data Includes Surveys
82 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
83 The Concerto sample data set now includes patron surveys, questions,
84 answers, and responses.
86 Virtual Index Definitions
87 ^^^^^^^^^^^^^^^^^^^^^^^^^
88 The practical purpose of Virtual Index Definitions is to supply an Evergreen
89 administrator with the ability to control the weighting and field inclusion of
90 values in the general keyword index, commonly referred to as "the blob,"
91 without requiring tricky configuration that has subtle semantics, an
92 over-abundance of index definitions which can slow search generally, or the
93 need to reingest all records on a regular basis as experiments are performed
94 and the configuration refined. Significant results of recasting keyword indexes
95 as a set of one or more Virtual Index Definitions will be simpler search
96 configuration management, faster search speed overall, and more practical
97 reconfiguration and adjustment as needed.
99 Previously, in order to provide field-specific weighting to
100 keyword matches against titles or authors, an administrator must duplicate many
101 other index definitions and supply overriding weights to those duplicates. This
102 not only complicates configuration, but slows down record ingest as well as
103 search. It is also fairly ineffective at achieving the goal of weighted keyword
104 fields. Virtual Index Definitions will substantially alleviate the need for
105 these workarounds and their consequences.
107 * A Virtual Index Definition does not require any configuration for
108 extracting bibliographic data from records, but instead can become a sink for
109 data collected by other index definitions, which is then colocated together to
110 supply a search target made up of the separately extracted data. Virtual Index
111 Definitions are effectively treated as aggregate definitions, matching across
112 all values extracted from constituent non-virtual index definitions. They can
113 further make use of the Combined class functionality to colocate all values in a
114 class together for matching even across virtual fields.
116 * Configuration allows for weighting of constituent index definitions that
117 participate in a Virtual Index Definition. This weighting is separate from the
118 weighting supplied when the index definition itself is a search target.
120 * The Evergreen QueryParser driver returns the list of fields actually
121 searched using every user-supplied term set, including constituent expansion
122 when a Virtual Index Definition is searched. In particular, this will facilitate
123 Search Term Highlighting described below.
125 * Stock configuration changes make use of pre-existing, non-virtual index
126 definitions mapped to new a Virtual Index Definition that implements the
127 functionality provided by the `keyword|keyword` index definition. The
128 `keyword|keyword` definition is left in place for the time being, until more data
129 can be gathered about the real-world effect of removing it entirely and
130 replacing it with Virtual Index Definition mappings.
132 * New system administration functions will be created to facilitate
133 modification of Virtual Index Definition mapping, avoiding the need for a full
134 reingest when existing index definitions are added or removed from a virtual
137 Increased use of Metabib Display Fields
138 +++++++++++++++++++++++++++++++++++++++
139 We use Metabib Display Fields (newly available in 3.0) to render catalog search
140 results, intermediate metarecord results, and record detail pages. This requires
141 the addition of several new Metabib Display Field definitions, as well as Perl
142 services to gather and render the data.
144 We also use more Metabib Display Fields in the client. As a result,
145 bibliographic fields will display in proper case in more client interfaces and
146 in Evergreen reports.
150 A new AngularJS "MARC Search/Facet Fields" interface has been created to replace
151 the Dojo version, and both have been extended to support Virtual Index
152 Definition data supplier mapping and weighting.
154 Settings & Permissions
155 ++++++++++++++++++++++
156 The new Virtual Index Definition data supplier mapping table,
157 `config.metabib_field_virtual_map`, requires the same permissions as the
158 MARC Search/Facet Fields interface: CREATE_METABIB_FIELD, UPDATE_METABIB_FIELD,
159 DELETE_METABIB_FIELD, or ADMIN_METABIB_FIELD for all actions
163 There now exist several new database tables and functions primarily in support
164 of search highlighting. Additionally, the QueryParser driver for Evergreen has
165 been augmented to be able to return a data structure describing how the search
166 was performed, in a way that allows a separate support API to gather a
167 highlighted version of the Display Field data for a given record.
171 By default, the following fields will be weighted more heavily in keyword
172 searches. Administrators can change these defaults by changing the values in the
173 "All searchable fields" virtual index in the "MARC Search/Facet Fields"
177 * Main title (a new index limited to the words in the 245a)
181 In addition, note indexes and the physical description index will receive
182 less weight in default keyword searches.
184 Re-ingest or Indexing Dependencies
185 ++++++++++++++++++++++++++++++++++
186 With the addition and modification of many Index Definitions, a full reingest is
187 recommended. However, search will continue to work as it did previously
188 for those records that have not yet been reingested. Therefore a slow, rolling
189 reingest is recommended.
191 Performance Implications or Concerns
192 ++++++++++++++++++++++++++++++++++++
193 Because the Metabib Display Fields infrastructure will eventually replace
194 functionality that is significantly more CPU-intensive in the various forms of
195 XML parsing, XSLT transformation, XPath calculation, and
196 Metabib Virtual Record construction, it is expected that the overall CPU load
197 will be reduced by this development, and ideally the overall time required to
198 perform and render a search will likewise drop. It is unlikely that the speed
199 increase will be visible to users on a per-search basis, but that search in
200 aggregate will become a smaller consumer of resources.
208 When 2 or more bib records are merged, all records involved are stamped
209 with a new `merge_date` value. For any bib record, this field indicates
210 the last time it was involved in a merge. At the same time, all
211 subordinate records (i.e. those deleted as a product of the merge) are
212 stamped with a `merged_to` value indicating which bib record the source
213 record was merged with.
215 In the browser client bib record display, a warning alert now appears
216 along the top of the page (below the Deleted alert) indicating when a
217 record was used in a merge, when it was merged, and which record it was
218 merge with, rendered as a link to the target record.
224 Alternate Patron Hold Pickup
225 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
226 This feature adds a bit of convenience to a common task: checking out
227 an item on hold to another patron (typically a family member or helper).
229 When you checkout the item, you will get a pop-up window with warnings associated
230 with this item. The "ITEM_ON_HOLDS_SHELF" message is now expanded to
232 * Let you know the name of the person who had placed the hold.
233 * Give you the option (in the form of a checkbox) of cancelling the
234 hold placed by the above-named patron. (Checked = Cancel the hold;
235 Unchecked = Leave the hold in place)
237 The initial value of the checkbox is derived from the
238 `circ.clear_hold_on_checkout` organizational setting.
240 If the operator has CANCEL_HOLD privilege, then if the checkbox is checked and
241 the checkout is allowed to proceed, the hold will be cancelled with a note that
242 the item was checked out to another patron.
244 This feature is available in the browser-based staff client.
246 New Patron Billing Statement
247 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
248 The Evergreen web staff client now includes a patron billing statement,
249 which summarizes a patron's bills, credits and payments in a familiar
250 layout. This can be found on the "Statement" tab of the Patron Bill
251 Details page. (From the Patron Bills page, double-click a row to view
252 its details, or choose "Full Details" from the Actions menu.)
254 Enhanced Billing Timestamp Support
255 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
256 Previously, billings had to make do with a single timestamp attempting
257 to fill two different roles. In the case of an overdue fine, the
258 timestamp represented the *end* of the fine period for that billing,
259 while for all other fines, the timestamp was merely the time the bill
260 was created. This setup generally worked, but not without confusion,
261 and limited our ability to understand and process the data.
263 Billings will now have up to three timestamps: a create date, and when
264 applicable, a fine period start and a fine period end. This clarifies
265 and simplifies things like backdating, retrospective fine generation,
266 account balancing for negative balance avoidance, and billing timeline
269 Copy Alerts and Suppression Matrix
270 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
271 The Copy Alerts feature allows library staff to add customized alert
272 messages to copies. The copy alerts will appear when a specific event
273 takes place, such as when the copy is checked in, checked out, or
274 renewed. Alerts can be temporary or persistent: temporary alerts will be
275 disabled after the initial alert and acknowledgement from staff, while
276 persistent alerts will display each time the alert event takes place.
277 Copy Alerts can be configured to display at the circulating or owning
278 library only or, alternatively, when the library at which the alert
279 event takes place is not the circulating or owning library. Copy Alerts
280 can also be configured to provide options for the next copy status that
281 should be applied to an item. Library administrators will have the
282 ability to create and customize Copy Alert Types and to suppress copy
283 alerts at specific org units.
285 Copy alerts can be added via the volume/creator and the check in,
286 check out, and renew pages. Copy alerts can also be managed at the
289 Copy alert types can be managed via the Copy Alert Types page in
290 Local Administration, and suppression of them can be administered
291 via the Copy Alert Suppression page under Local Administration.
293 Place Multiple Holds At Once
294 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
295 Users with the appropriate permissions now have the ability to place multiple
296 title/metarecords holds at once. This feature is especially beneficial for book
297 clubs and reading groups, which need to place holds on multiple copies of a
300 In order to use the feature:
302 * Set the _Maximum number of duplicate holds allowed_ Library Setting
303 (`circ.holds.max_duplicate_holds`) to a number higher than 1
304 * Log in as a user with the CREATE_DUPLICATE_HOLDS
306 When placing a title or metarecord hold, a _Number of copies_ field will
307 display for these users. This field is not available when placing part, volume
310 This feature does not change the way in which the system fills holds. The
311 multiple holds will fill in the same way that they would if the user had placed
312 multiple holds separately.
314 New Notice Columns in Items Out Grid
315 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
316 The grid in the patron "items out" page in the Evergreen web staff client has two new
317 columns indicating the number of notifications generated for a given loan and the date of
318 the most recent notification. These columns will allow circulation staff to better respond to
319 patron questions about whether they were sent notification about an overdue item.
321 The columns are based on the number of completed Action Trigger events on the
322 loan that have a 'checkout.due' hook. In other words, they would include overdue
323 and courtesy notices.
325 A new library setting, "Exclude Courtesy Notices from Patrons Itemsout Notices Count",
326 if set will cause the notice count and date fields to exclude courtesy notices.
328 Patron Email Addresses Now Clickable In Web Staff Client
329 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
330 Adds a mailto link to the patron's email in their profile so it can
331 be clicked to send and email to the patron. No new settings or
332 permissions are included in this feature.
334 Pickup Library for Staff-placed Holds
335 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
336 Adds a new library setting, _circ.staff_placed_holds_fallback_to_ws_ou_,
337 that helps determine the hold pickup library in cases where patrons don't
338 have a preferred hold pickup library in their account and a staff member
339 is placing the hold on their behalf.
341 * When this setting is true and the patron doesn't have a preferred
342 library listed, the hold pickup library will default to the
343 workstation's organizational unit.
344 * When this setting is false and the patron doesn't have a preferred
345 library listed, the hold pickup library will default to the
346 patron's home library.
351 Search Term Highlighting
352 ^^^^^^^^^^^^^^^^^^^^^^^^
353 Evergreen now highlights search terms on the public catalog's main search
354 results page, the record detail page, and intermediate pages such as metarecord
355 grouped results page. Highlighting search terms will help the user determine why
356 a particular record (or set of records) was retrieved.
358 Highlighting of matched terms uses the same stemming used to accomplish the
359 search, as configured per field and class.
361 This feature will help the user more quickly determine the relevance of a
362 particular record by calling their attention to search terms in context. Lastly,
363 it will help familiarize the user with how records are searched, including which
364 fields are searched as well as exposing concepts like stemming.
366 You can turn off search term highlighting by uncommenting the line
367 `search.no_highlight = 1;` in `config.tt2`.
369 When highlighting is generally enabled, it may be turned on or off on a per-page
370 basis through the use of a UI component which will request the page again
371 without highlighting.
373 Highlighting of terms uses Template::Toolkit-driven CSS. A generic CSS class
374 identifying a highlighted term, along with CSS classes identifying the search
375 class and each search field are available for use for customization of the
376 highlighting. A stock CSS template is provided as a baseline upon which sites
380 Copy Location Filter Displays for System Searches
381 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
382 The Shelving Location filter now displays on the advanced search page when
383 a search is scoped to a library system, not just to an individual branch. If
384 a library system is selected as the Search Library, the shelving location
385 limiter will display any shelving location that is owned by the selected system
386 or by the consortium. It will NOT display shelving locations owned by child
389 Multi-source Attributes
390 ^^^^^^^^^^^^^^^^^^^^^^^
391 We now allow record attribute definitions to extract data using more than
392 one strategy (XPath, tag+subfield, fixed field, etc.) as long as the values
393 from various sources would, after normalization, have the same shape.
397 This change allows us to configure multilingual search, by extracting values
398 from both the 008 controlfield and the 041 datafield. Because the values
399 in each can be normalized to the same controlled list (and, in practice, are
400 already from the same normalized value set), catalog searches can now use normal
401 boolean search semantics to find records with various combinations of
404 E.g., in the concerto test data:
406 * `keyword: piano item_lang(eng) item_lang(ita)`
409 Optional Display of Badges in Catalog
410 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
411 A new setting controls whether badges (popularity, etc.) are displayed
412 in the catalog. If you do not wish badges to be displayed, set the
413 `ctx.hide_badge_scores` setting to "true" in `config.tt2`.
419 Fixes to patron name/username search indexes
420 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
421 When using pg_restore to restore an Evergreen database, some of the
422 indexes used to speed up patron searches on names and usernames
425 This release fixes the underlying issue and re-creates the indexes
430 When using pg_restore to restore an affected database, the
431 "unaccent" indexes on actor.usr would not be created due to an
432 unqualified function reference in `evergreen.unaccent_and_squash`.
434 The function will be replaced to resolve the search path issue,
435 and the following indexes on actor.usr will be dropped and then
438 * actor_usr_first_given_name_unaccent_idx;
439 * actor_usr_second_given_name_unaccent_idx;
440 * actor_usr_family_name_unaccent_idx;
441 * actor_usr_usrname_unaccent_idx;
443 This will be done even if the indexes are already present, and may
444 take a few minutes on a database with many patrons.
447 3.1.0 Acknowledgments
448 ---------------------
449 The Evergreen project would like to acknowledge the following
450 organizations that commissioned developments in this release of
453 * Albany Public Library (Oregon)
454 * Consortium of Ohio Libraries
456 * Indiana State Library
457 * Georgia Public Library Service
458 * Hagerstown - Jefferson Township Library
459 * Linn-Benton Community College
461 * Pennsylvania Integrated Library System
462 * Sage Library System
463 * Union County Public Library (Indiana)
465 We would also like to thank the following individuals who contributed
466 code, translations, documentations patches and tests to this release of
493 We also thank the following organizations whose employees contributed
497 * British Columbia Libraries Cooperative
500 * Equinox Open Library Initiative
501 * Georgia Public Library Service
502 * Greater Clarks Hill Regional Library System
503 * Jordanian Library and Information Association
504 * King County Library System
506 * Linn-Benton Community College
509 * Traverse Area District Library
511 We regret any omissions. If a contributor has been inadvertently
512 missed, please open a bug at http://bugs.launchpad.net/evergreen/
projects / Evergreen.git / blob