dbs [Wed, 16 Mar 2011 04:04:19 +0000 (04:04 +0000)]
Set default db schema search path to public,pg_catalog
With the addition of the 'evergreen' schema, we discovered that
all of the unqualified database objects were being created in
the 'evergreen' schema if the database user name was 'evergreen'
(as the default search path is "$user",public we started matching
"$user").
Thomas Berezansky proposed a variation of the function provided
here for changing database settings. By creating the function in
the first schema file, the subsequent connections pick up the
new default search path and all works blissfully as it once did.
Long term, we should probably move towards explicitly qualified
database objects.
dbs [Sat, 12 Mar 2011 23:14:20 +0000 (23:14 +0000)]
Clean up build errors when creating/recreating database schema
Dropping the evergreen schema cascaded and wiped out some previously
defined functions, so move it ahead of the other functions. (And maybe
we can start moving those public functions into the evergreen schema?)
DROP AGGREGATE IF EXISTS exists on any version of PostgreSQL that we
care about for Evergreen 2.0, so we can use it to avoid false errors
during schema creation.
FTS config was complaining about public.english_nostop in the ALTER
statement, so remove the schema qualification. This seems to have
been there for a long time?
dbs [Sat, 12 Mar 2011 03:14:10 +0000 (03:14 +0000)]
Resolve inconsistent results when invoking LOWER() in C vs. UTF8 locale databases
As reported by the Fundamental Science Library of Armenia, patron searches
were not retrieving patrons who were known to be part of the patron database.
While the initial approach to solve this problem used an encode_utf8() call
to encode the data before passing it on to the database, this turned out to
break the patron search function in other environments.
Testing by Dan Wells confirmed that the LOWER() function returned different
results when invoked against text in a database created with LC_CTYPE=C vs
a database created with LC_CTYPE=*.UTF-8. As the patron search function
used a Perl lc() function call to convert the incoming data to lowercase,
the success of the call depended on the LC_CTYPE value of the database.
To avoid this problem in the future, we define our own evergreen.lowercase()
function that can reliably produce lowercase text for characters outside
the Latin1 range, and we convert our indexes and function calls to use that
function consistently.
dbs [Fri, 11 Mar 2011 20:39:03 +0000 (20:39 +0000)]
LANG=C results in SQL_ASCII encoding by default for PostgreSQL, which isn't cool
We want LC_CTYPE=C and LC_COLLATE=C, and SERVER_ENCODING=UTF8.
The docs have been adjusted as 8.4 and up now offer database-level
--lc-ctype / --lc-collate options for createdb command.
dbs [Fri, 11 Mar 2011 16:46:15 +0000 (16:46 +0000)]
Ensure incoming query values to patron search are UTF8-encoded
Addresses the problem reported with searching for patrons with
Armenian names, where Unicode query params were getting corrupted
on its way to the database.
dbs [Wed, 9 Mar 2011 19:08:43 +0000 (19:08 +0000)]
Draw pre-cataloged items (author and title) in My Account checkedout screen
Two problems: FETCH_MODS_FROM_COPY returns an object, so we weren't
getting to the point of fetching the raw copy itself, and there was
some confusion about pre-cataloged vs. non-cataloged items that is
hopefully straightened out now.
dbs [Mon, 7 Mar 2011 15:14:01 +0000 (15:14 +0000)]
Fix cloning of AutoField widgets: LP #728055
Cloning Action/Trigger event definitions was not working properly, as
all values were being copied from the original object using their display
strings rather than the underlying values.
Bill Erickson suggested a fix in IRC that works, so full credit to him
for this change!
dbs [Sat, 5 Mar 2011 03:18:56 +0000 (03:18 +0000)]
Correct search index definitions for subject|complete and identifier|bibcn
Dan Wells reported in LP# 729385 that the index definitions for
subject|complete and identifier|bibcn were not working, and traced
the problem to a change in how indexes are generated in 2.0+. Dropping
the trailing '//text()' on these two indexes addresses the problem.
Sites upgrading from a previous version of Evergreen 2.0 will want
to refresh the indexes after applying this schema change to enable
searches against subject|complete and identifier|bibcn to work.
dbs [Fri, 4 Mar 2011 04:58:41 +0000 (04:58 +0000)]
Backport SRU fixes from trunk (r19572, r19573)
Add complete set of Evergreen search aliases and attributes to SRU explain
Original "pull from database" code had limited the search aliases to those
that were qualified; now we pull all of them, and if they are not qualified,
we qualify them as "eg".
In addition, for some of the non-index attributes (sort, direction, site,
available), restore the descriptive title that was previously available.
Slightly longer term we will add a "description" column to
config.metabib_search_alias to store this information.
By returning explicit index and title elements from the method,
we can backport cleanly to 2.0 and set the stage for pulling the
title element directly from the database in 2.1 and beyond.
In addition, the enriched object enables us to get rid of the
hardcoded nested qualifier map for bibs.
Finally, we also fix the mappings of SRU qualifiers to Evergreen
search aliases (in many cases we had been quietly falling back
to plain kw searches, which was not cool).
dbs [Thu, 3 Mar 2011 06:01:58 +0000 (06:01 +0000)]
Draw SRU search indexes from config.metabib_search_alias
We had hardcoded search indexes for SRU, but thanks to the work
of Mike Rylander, it is fairly easy to remove that brittle code
and replace it with code that polls the contents of
config.metabib_search_alias and dumps it into the explain document.
miker [Wed, 2 Mar 2011 21:15:15 +0000 (21:15 +0000)]
Call number sorting "before" the context needs to be segregated based on directly on the label, "here-and-after" on the bytea version. Testing bears this out, but more eyes would be appreciated.
dbs [Tue, 1 Mar 2011 05:34:17 +0000 (05:34 +0000)]
Add a TT helper to encode email headers in UTF-8
This can and should be used to encode any headers (To, From, Subject, etc)
in an email that might include text outside of the ASCII 7-bit range. Names
of people or libraries, for example, that might otherwise come out garbled.
dbwells [Thu, 24 Feb 2011 20:04:33 +0000 (20:04 +0000)]
Fix regex in maintain 901c
The previous regex would delete any 901 fields *and* any fields which followed. Since the 901 is typically the last field, this problem didn't surface during testing.
senator [Thu, 24 Feb 2011 15:35:51 +0000 (15:35 +0000)]
Per Robert Soulliere, it can be necessary in some cases to clean out bad
data from action.reservation_transit_copy before applying the missing
fkeys to said table.
dbs [Wed, 23 Feb 2011 15:07:17 +0000 (15:07 +0000)]
Add 830 as a controllable bibliographic field in authority linking script
Per http://www.loc.gov/marc/bibliographic/bd830.html, bibliographic 830
should be controlled by authority 130. The "common cataloging practice"
in 830 of omitting non-filing characters may cause some entries to not be
linked that otherwise would be.
dbs [Wed, 23 Feb 2011 02:12:02 +0000 (02:12 +0000)]
Correct the install location for Dojo i18n bundles
We had been converting xx-YY into xx/yy for a directory structure,
but current evidence shows that it should be xx-yy. Hard to believe
we've overlooked this for so long. Also, with the exception of the
en-ca/en-gb/en-us locales, we should probably trim the locale to
just xx - but that's a different problem, and let's just focus on
getting this working to begin with.
If a series-class index definition has been removed, the data remains in the field entry table. This is because there was no fkey previously. We need to clean that data out so that the fkey can be validated during upgrade.
senator [Thu, 17 Feb 2011 20:57:31 +0000 (20:57 +0000)]
Backport r19470 from trunk
Booking: fix the targeter in the I'm-reserving-this-exact-resource-with-this-
exact-barcode case.
It should have always worked like this. This should better mirror the behavior
in the holds targeter when placing a copy-level hold. Basically don't fail
to target a resource for a reservation later just because it's not in an
available status right at this moment.
dbs [Thu, 17 Feb 2011 18:57:02 +0000 (18:57 +0000)]
Avoid mixed-content warnings due to HTTP link for Google Chrome Frame
We're hardcoding this to HTTPS for now to avoid the mixed-content warnings that
occur when content location is https:// and the linked script is at http:// but
a config-aware patch probably wouldn't be a bad idea.
dbs [Thu, 17 Feb 2011 03:31:01 +0000 (03:31 +0000)]
Teach maintain_901 trigger to respect the "Use record ID for TCN" global flag
Rather than relying on the Perl layer to set the correct TCN on the way in,
do it as part of the in-database indexing to ensure that we have consistent
results no matter how the record gets into the database.
dbs [Sun, 13 Feb 2011 03:19:36 +0000 (03:19 +0000)]
Teach authority_control_fields.pl about record ranges
Passing --record 10,000 times at the command line sucks, and
--all is not particularly realistic for large databases. Adding
--start_id and --end_id makes this script more usable.
Also provide reasonably full POD documentation, and rely on
Pod::Usage to provide help based on it when help is requested
or passed in options don't make sense.
dbs [Fri, 11 Feb 2011 21:45:20 +0000 (21:45 +0000)]
Normalize ISBNs by stripping hyphens in search methods and in reporter.materialized_simple_record
We weren't normalizing ISBNs in search or in reporter.materialized_simple_record
which required users to exactly match the form of the ISBN as entered in the
MARC record to get a successful search result - pretty hit or miss.
The longer term fix in 2.0 and above is to make the ISBN search search against
the identifier|isbn index, which also gets ISBN10/ISBN13 equivalence goodness.
But this patch will be relatively easy to backport through to 1.6.1 and plugs
some of the holes in our exposed APIs and search points in the short term.
erickson [Wed, 9 Feb 2011 18:26:06 +0000 (18:26 +0000)]
allow item status update if either status-specific perm is allowed or UPDATE_COPY is allowed. fire off related A/T events after the initial transaction has been committed
dbs [Tue, 8 Feb 2011 06:21:18 +0000 (06:21 +0000)]
Remove syntax error from O:A:Storage:Publisher:authority and add strict pragma
Came across a line of dead code while debugging a related problem, and
noticed that strict/warnings wasn't in place, so I culled the dead code
and added use strict/use warnings accordingly (now that we no longer
have the syntax error caused by the dead code).
dbs [Tue, 8 Feb 2011 06:13:05 +0000 (06:13 +0000)]
Create authorities that only contain controlled subfields (LP 712496)
Selecting "Create authority" on a field containing uncontrolled subfields
would generate an authority record that contained those uncontrolled
subfields. This would, in turn, prevent the "Validate" button from operating
correctly - if you added a 700 with $a and $c subfields, clicked "Create
authority", and then immediately clicked "Validate", the field would show
up as red.
Now we filter out the uncontrolled subfields before they get sent to
the "Create authority" function, avoiding this validation problem.
dbs [Tue, 8 Feb 2011 05:12:28 +0000 (05:12 +0000)]
Ensure new authority ID subfield is inserted in the correct XUL DOM location
Addresses LP 712499. After creating an authority via the context menu
in the MARC editor, the new ID subfield ($0) would be created right after
the subfield on which the context menu was invoked. It turns out that it
was being placed in the wrong location, and one symptom was that the
Validate button would not validate the controlled field against the newly
created authority.
Now we hunt through the parent DOM nodes until we find the 'sf_box' element
and then we append the ID subfield to that node. We also eliminate some
duplicate code by defining a common function so that the problem can be
fixed in one stroke...
dbs [Tue, 8 Feb 2011 03:50:58 +0000 (03:50 +0000)]
Prevent authority context menu on whitespace content
Continuing the refinement of LP 712538, if the subfield contains
only whitespace, treat it as though it were empty for the purposes
of the context menu.
dbs [Tue, 8 Feb 2011 03:50:27 +0000 (03:50 +0000)]
Prevent authority context menu from being displayed on empty content (LP 712538)
It's confusing to have the authority context menu - particularly the
"Create authority" options - appear for empty content. Check to ensure
that the subfield being clicked on actually has some content before
displaying the authority context menu.
erickson [Fri, 4 Feb 2011 00:15:24 +0000 (00:15 +0000)]
ported open-ils.actor.user.transaction.[fleshed.]retrieve to CStoreEditor both for cleanup and to take advantage of .authoritative. Use authoritative version in staff client
dbwells [Thu, 3 Feb 2011 16:49:24 +0000 (16:49 +0000)]
Two binding fixes for manage items
The first fix is more or less cosmetic. When selecting a target unit, both areas need to refresh in order to not show the selected unit's items in the main list.
The second fix involves handling cases where moving items from one unit to another does not empty the 'donor' unit. In those cases, we need to update the contents of both the new and the old unit.
dbs [Thu, 3 Feb 2011 14:16:25 +0000 (14:16 +0000)]
Authorities: control 610/611, do not control 4xx by default
The 4xx fields in bibliographic records have not been controlled
fields for some time (1988 for CAN/MARC, 1999 for USMARC per)
http://www.loc.gov/marc/bibliographic/bd4xx.html; and
http://www.oclc.org/bibformats/en/4xx/default.shtm also lists
these as obsolete. So, remove them from the default controlled
fields in Evergreen.
Also, add 610/611 to the controlled fields list (per LP #712467).
dbs [Wed, 2 Feb 2011 03:29:27 +0000 (03:29 +0000)]
Apply autogenerate barcode trigger to serial.unit to enable serial checkin
Serial checkin tried to use the '@@AUTO' macro to autogenerate barcodes,
but unfortunately the required trigger had only been defined on the parent
table asset.copy and not on the child table serial.unit. Here we define the
trigger on serial.unit to resolve that problem.
erickson [Tue, 1 Feb 2011 23:40:24 +0000 (23:40 +0000)]
make sure that inline overdue fines generation completes before we attempt to void overdues fines in checkin, otherwise it's possible to create multiple services (storage and cstore) competing for the same DB row mid-transaction, resulting in cstore timeouts and rollback of checkin. This could happen with backdated or amnesty checkins
senator [Tue, 1 Feb 2011 21:39:37 +0000 (21:39 +0000)]
Backport r19355 from trunk
Acq: Improving on r19351, don't let showCreatePane() be issued repeatedly to
fill the window with redundant dialogs, since a) it looks silly, and b) the
dialogs don't work when there are more than one of them. Thanks again Dan.
senator [Tue, 1 Feb 2011 21:16:37 +0000 (21:16 +0000)]
Backport r19351 from trunk
Acq: make the "new provider" button under Admin -> Server Admin ->
Acquisitions -> Providers not produce a dialog with a ton of mysterious
whitespace. Spotted by Dan Scott.
dbs [Sun, 30 Jan 2011 05:20:33 +0000 (05:20 +0000)]
Use the browse method instead of startwith for Manage Authorities, too
Now that the browse method is working properly, adopt it for the
Manage Authorities interface as well so that context around the
search term can be provided.
dbs [Sun, 30 Jan 2011 05:19:15 +0000 (05:19 +0000)]
Correct authority browsing for reals
First, restore the >= enable before and after ranges in
authority_tag_sf_browse(), after I mistakenly removed it in
r19131; the second storage request for $after does not
stomp on the prior $before results, it simply gets pushed
onto the carefully constructed list of $before results,
ensuring that our target is in the middle of page 0.
Second, we're treating all of the "tag" members in the
method registration as list references now (for the purpose
of searching against 4xx/5xx in the .refs. variants), but
that was blowing up when we registered just a single tag as
a string and tried to treat the scalar as a list reference.
I could have checked to see if what we had incoming was a
reference and dance accordingly, but opted to just define
all single-tag entries as single-element arrays instead.
Applied the same to startwith.
Finally, in r19331 I had used chop() to ensure that an
exact match for startwith would be returned as element 1 on
page 0, instead of appearing as the last element of page -1.
I had said that the right way to do this would be to naco_normalize()
the value to match the normalized afr.value, and so this is what
I have done. Rather than torturously using O:A:Storage:FTS to get
at the naco_normalize() definition, I moved the function into its
own Utils package and adjusted its usage accordingly through the
affected code. One step closer to single-sourcing the function
in the database, as well?
senator [Fri, 28 Jan 2011 20:53:01 +0000 (20:53 +0000)]
Backport r19325 from trunk
Serials: Add some missing permacrud permissions to IDL
7 classes (sdistn, siss, sunit, sin, sbsum, sssum, sisum) were missing IDL
perms. Appropriate permissions were added to all but sin (serial_item_note).
For the sin class, since it's several leaps away from anything that points at
an org unit, and since no existing code deals with it via PermaCrud, its
PermaCrud section was simply removed.
Some classes retain open <retrieve /> permissions, and others do not.
No new permissions were created for this. All permissions are reused from
"higher" serials objects.
Credit to Mike Rylander for spotting the omissions.