Dan Scott [Tue, 20 Nov 2012 18:55:02 +0000 (13:55 -0500)]
EResource handling improvements
* Add the ability to read (via -F xml) and write (via -T xml) MARCXML
records. Writing XML records will place one per line, making it
ideal for inserting into a database via COPY().
* Make the "Mark ISBN for SFX" functionality optional via the -I flag,
as not all electronic resources have ISBNs. (Hi, journals and music
records).
* Reduce code duplication significantly
Dan Scott [Thu, 25 Oct 2012 14:56:54 +0000 (10:56 -0400)]
Insert fields in order in ebook processing
pymarc 2.8.5 offers the add_ordered_field() method for inserting fields
in the correct numerical order in the record - so use that. Note that
you may have to run "easy_install --upgrade pymarc" to get the latest
version of pymarc installed.
Dan Scott [Mon, 12 Nov 2012 17:58:42 +0000 (12:58 -0500)]
Delete ILL copies as well as ILL call numbers
Evergreen 2.3 likes to show available copies in the holdings maintenance
window even if the parent call numbers are deleted. So, do what we
should have done before and delete the copies too.
Guard against deleting copies that have been renewed rather than having
their due dates extended.
Dan Scott [Thu, 4 Oct 2012 22:17:12 +0000 (18:17 -0400)]
Ebooks: Only output one record, no matter how many dupes
If you specify dupe-checking by TCN, URL, and ISBN, you only want to
generate one output record per input record, no matter how many types of
matches you get - so short-circuit the logic.
Also, handle the case where you're not specifying all dupe checks :)
Dan Scott [Thu, 4 Oct 2012 21:17:04 +0000 (17:17 -0400)]
Ebooks: handle multiple dupe options
process_fields() modifies the base record, and we were inadvertently
running the base record through the ringer every time we checked for
dupes - which would be evidenced by the addition of one 506 and 590 per
consortial member per dupe check we ran.
In the process of cleaning this up, try to dedupe some of our own code
(heh). We can probably go futher but this is a good start.
Dan Scott [Wed, 11 Jul 2012 01:20:41 +0000 (21:20 -0400)]
Use synchronous methods and cleaner results
Rather than calling ldap.result(), which results in awkward lists of
lists which are hard to handle, just iterate over the results from
ldap.search_s() (which, btw, is now a synchronous call).
Dan Scott [Tue, 10 Jul 2012 20:48:21 +0000 (16:48 -0400)]
Revert debug code that showed all LDAP attributes
Not that I mind verbosity, but unless you request the custom attributes
specifically, you only get the default LDAP attributes. And I'm more
interested in the custom ones.
Dan Scott [Tue, 10 Jul 2012 20:25:42 +0000 (16:25 -0400)]
Experimental web.py frontend for creating Conifer accounts
Use web.py to surface a simple Web UI for generating Conifer accounts
based on the Laurentian ID of a given user. Required a little
refactoring of ldap_osrf_sync to make it usable as a module rather than
just as a script, but that's all for the better.
Dan Scott [Fri, 9 Mar 2012 20:35:36 +0000 (15:35 -0500)]
Carry over the leader from the original record
We iterate over every field of the original record to create the new
record. Except... the leader is not an iterable field. So we were
pushing default leaders onto every new record we were creating. That
is... suboptimal to the max.
Dan Scott [Tue, 24 Jan 2012 21:32:31 +0000 (16:32 -0500)]
Generate email addresses where none are in LDAP
Per discussion with MLaferriere, we should expect to see LDAP entries
without a 'mail' attribute, and should build email addresses based on
the CN. For additional flourish, append @laurentian.ca or
@laurentienne.ca based on the language preference.
Dan Scott [Mon, 16 Jan 2012 22:53:27 +0000 (17:53 -0500)]
LDAP sync: If EG user found, reactivate
In theory, we won't be asked to create a user that already exists in
Evergreen if they are already there and active. So, go ahead and set the
"active" flag to true and update the account expiry date when we are
asked to create an existing user.
Dan Scott [Tue, 10 Jan 2012 18:25:38 +0000 (13:25 -0500)]
Add located URI update script (PGSQL function)
Given MARCXML, modifies the 856 $9 subfields to map to owning systems
rather than individual branches to address the located URI visibility
change in 2.1.
Dan Scott [Fri, 7 Oct 2011 17:21:55 +0000 (13:21 -0400)]
Handle "thornloe" profile
Apparently our LDAP system or a system upon which it depends is either
enforcing an 8-char limit for some reason, or we have to deal with
typos. Whatever.
Dan Scott [Wed, 14 Sep 2011 20:47:38 +0000 (16:47 -0400)]
Don't update the barcode if user was not created
Check the return value and bail if the user creation was a failure.
For large updates, one can use --create and --push to update barcodes
for users who already exist.
Dan Scott [Wed, 14 Sep 2011 20:02:14 +0000 (16:02 -0400)]
Add and use a patron search that overrides opt-in invisibility
Something like this is required for API calls that need to operate
against a number of libraries in a given instance that are using opt-in;
otherwise, attempts to search for users will fail and you may end up
creating near-duplicates etc.
The implementation adds an
open-ils.actor.search.patron.advanced.opt_in_override method to
open-ils.actor, which, if invoked, checks to see if the caller has the
OPT_IN_OVERRIDE permission. If so, then the crazy_search ignores the
normal opt-in limits and searches all pertinent users in the database.
As a global permission, OPT_IN_OVERRIDE is a blunt instrument. Others
might want to put together a more refined version that uses OU depths to
define boundaries.
Dan Scott [Wed, 14 Sep 2011 19:00:18 +0000 (15:00 -0400)]
More defensive error handling
When users aren't opted-in, we can't search for them. Fiddlesticks! This
was breaking the script rather annoyingly; now we print out an error and
move on to the next user.
Dan Scott [Fri, 9 Sep 2011 04:45:30 +0000 (00:45 -0400)]
New --push-barcode option updates LDAP with Evergreen barcodes
By default, the --create-users option will update LDAP with the new
barcode that was generated for the new users.
One can also use a --query of some type and pass --push-barcode to
retrieve the barcodes for all of the LDAP users from Evergreen and then
push those barcodes into their respective LDAP accounts.
Dan Scott [Thu, 8 Sep 2011 21:26:58 +0000 (17:26 -0400)]
Generate barcodes and push them into LDAP
Add a custom API to open-ils.actor for generating barcodes; accepts one
argument, user ID, which gets the barcode set to their new value. If no
arg is passed, just generates a new barcode and returns it (should be
useful on the patron editor screen).
As far as pushing the barcodes into LDAP goes, I can't confirm that the
updates are actually working, because I can't see them, but the lack of
errors suggests that it's working.
Dan Scott [Thu, 8 Sep 2011 15:35:31 +0000 (11:35 -0400)]
Command line options for queries and actions
Use the argparse module to enable override of user/password/hostname
options at the command line, as well as to enable different types of
queries (for example, via LDAP cn / sn / id / createTimestamp) and
enable particular actions (display the LDAP data and/or create new users
in Evergreen).
Much better than manually editing the script each time.
Dan Scott [Wed, 7 Sep 2011 07:12:56 +0000 (03:12 -0400)]
Get working OpenSRF integration, genericize a bit
Still need to teach the beast how to provision barcodes, then write
those back to LDAP as well. Pushing the LDAP mapping into the User class
init method seems like a reasonable way to genericize this a bit; sites
could replace the User class implementation.
Dan Scott [Wed, 31 Aug 2011 20:08:48 +0000 (16:08 -0400)]
Break barcode generator into smaller pieces
This should make it easier to expose evergreen.lu_generate_barcode() for
manual purposes (e.g. for creating a new user on the fly in the Register
Patron UI).
Dan Scott [Thu, 11 Aug 2011 20:45:45 +0000 (16:45 -0400)]
Add a few more T&F data uncorruption fixes
Note that we're now seeing conflicting patterns, so some
legitimate characters are getting corrupted by the script,
while others could go either way (l stroke vs modifier prime).
As long as the fix rate is 95%+ then I'm happy.
Dan Scott [Fri, 29 Jul 2011 15:58:36 +0000 (11:58 -0400)]
Begin addressing the specific manglings of the CRKN records
To begin with, correct the mangled macrons and modifier letter half ring
that are popular in the Arabic language. Here's hoping that correcting
these byte sequences doesn't lead to corruption elsewhere.
Dan Scott [Thu, 28 Jul 2011 15:16:42 +0000 (11:16 -0400)]
Check for 020$z, 024$a,$z for ISBNs
Either our MARC record sources are horribly corrupted, or the SFX
knowledge base is questionable, or both, as we're having to search
cancelled/invalid/non-ISBNs to get matches in SFX for our electronic
records.
Also, use 506 $9 to record our institutional ID in authorization notes.
Dan Scott [Wed, 20 Jul 2011 19:23:32 +0000 (15:23 -0400)]
Refine the 506 $3 punctuation decision
From Graham Staffen:
My definitive answer (until I develop a revised definitive answer) is
that a period would always punctuate the end of the subfield except in
two instances: 1) a closing parenthesis, 2) a closing square bracket. I
have this from our MARC coding trainer who could think of no other
instances when a period is not employed.
Dan Scott [Wed, 20 Jul 2011 14:28:30 +0000 (10:28 -0400)]
Include record count in 598 $c, conditional . in 506 $e
We might as well include the record offset in the 598 so that we can
track which record includes a given problem.
Also, according to Shuzhen, we only want to add a period at the end of
the 506 $e subfield in particular cases (like when it ends in a letter
or number; not with punctuation).
Dan Scott [Thu, 23 Jun 2011 17:51:19 +0000 (13:51 -0400)]
Change link text for Windsor to "Available online"
Per proposal #1 of the June 22, 2011 Conifer ebook specification
document, Windsor wanted to change their link text to the simpler
"Available online". Making it so.
Dan Scott [Wed, 22 Jun 2011 18:50:08 +0000 (14:50 -0400)]
Check for a hit in SFX for every library we have enabled
It's possible that one of the libraries has not yet turned on a given
target in SFX, so we don't want any false negatives. And if a library
hasn't opted into a particular package, they shouldn't be one of the
enabled flags passed to this script.
Also, clean up ISBN marking and access note creation, as we were
modifying the wrong MARC record.
Dan Scott [Wed, 22 Jun 2011 16:04:10 +0000 (12:04 -0400)]
Implement ISBN lookup in SFX
Check the SFX knowledgebase for a matching ISBN and mark it via the
magical $9 SFX subfield so that we can key off of that for subsequent
queries via the catalogue.
Given that only one library may be running this script for a set of
ebook records, ensure that we're checking the results.
Dan Scott [Wed, 22 Jun 2011 14:16:55 +0000 (10:16 -0400)]
On closer inspection, 506$e != 710$a
The 710 is being used for the imprint, while the 506$e is being used to
identify the online platform.
It's questionable whether this should be required - for example, a 506
wouldn't be necessary for Project Gutenberg materials - but let's go
with that for now anyway.
Dan Scott [Wed, 22 Jun 2011 14:01:31 +0000 (10:01 -0400)]
Add 506 note handling per Windsor proposal
The 506 note is for access restrictions; we will generate one 506 field
per instutition, with a customized $a message, the consortial license
identified in the $b, and the publisher in $e, along with the library
symbol in $5 to enable the catalogue to control the display of the
message in an appropriate scope.
Mike Rylander [Wed, 10 Oct 2012 21:36:52 +0000 (17:36 -0400)]
Make it possible to suppress IDL fields
Some clients of external services, particularly pcrud and reporter-store,
need to be able to access tables that contain columns we'd rather restrict.
For instance, the passwd field on actor.usr.
To effect this feature we provide a blacklist attribute for fields, called
suppress_controller, which works in the same way as the class controller
attribute but names controllers not allowed to use the field. When the field
is explicitly named in a query (fieldmapper select block or json_query) an
error is thrown, and suppressed fields are ingored in general fieldmapper
search/retreive requests.
Signed-off-by: Mike Rylander <mrylander@gmail.com> Signed-off-by: Bill Erickson <berick@esilibrary.com>
projects / contrib / Conifer.git / log