From a3fb79743829e24090fc9462a1cca73c3dee7b67 Mon Sep 17 00:00:00 2001 From: Robert Soulliere Date: Wed, 27 Oct 2010 13:23:58 -0400 Subject: [PATCH] Add content to migrating bib records - mostly from wiki. Add content to org units sections of server administration. Add entries to style guide. --- 1.6/admin/migratingdata.xml | 198 +++++++++++++++++++++++++++++ 1.6/admin/serveradministration.xml | 49 ++++++- style_guide/glossary.xml | 85 ++++++++++++- 3 files changed, 329 insertions(+), 3 deletions(-) diff --git a/1.6/admin/migratingdata.xml b/1.6/admin/migratingdata.xml index fc524f96c0..8494904d43 100644 --- a/1.6/admin/migratingdata.xml +++ b/1.6/admin/migratingdata.xml @@ -9,6 +9,152 @@ as well as knowledge of how to export data from your current system or access to data export files from your current system. +
+ Migrating Bibliographic Records + + One of the most important and challenging tasks is migrating your bibliographic records to a new system. The procedure may be different depending on the system from which you + are migrating and the content of the marc records exported from the existing system. The proecedures in this section deal with the process once the data from the existing system + is exporterd into marc records. It does not cover exporting data from your existing non-Evergreen system. + Several tools for importing bibliographic records into Evergreen can be found in the Evergreen installation folder + (/home/opensrf/Evergreen-ILS-1.6.1.2/Open-ILS/src/extras/import/ ) and are also available from the Evergreen repository + ( + http://svn.open-ils.org/trac/ILS/browser/branches/rel_1_6_1/Open-ILS/src/extras/import). + + Converting MARC records to Evergreen BRE JSON format + If you are starting with MARC records from your existing system or another source, use the marc2bre.pl script to create the JSON representation of a bibliographic + record entry (hence bre) in Evergreen. marc2bre.pl can perform the following functions: + + Converts MARC-8 encoded records to UTF-8 encoding + Converts MARC21 to MARCXML21 + Select the unique record number field (common choices are '035' or '001'; check your records as you might be surprised how a supposedly unique field + actually has duplicates, though marc2bre.pl will select a unique identifier for subsequent duplicates) + Extracts certain pertinent fields indexing and display purposes (along with the complete MARCXML21 record) + Sets the ID number of the first record from this batch to be imported into the biblio.record_entry table (hint - run the following + SQL to determine what this number should be to avoid conflicts: + +psql -U postgres evergreen + # SELECT MAX(id)+1 FROM biblio.record_entry; + + + + If you are processing multiple sets of MARC records with marc2bre.plbefore loading the records into the database, you will need to keep track + of the starting ID number for each subsequent batch of records that you are importing. For example, if you are processing three files of MARC records with 10000 + records each into a clean database, you would use , , and + parameters for each respective file. + + + Ignore trash fields that you do not want to retain in Evergreen + + + If you use marc2bre.pl to convert your MARC records from the MARC-8 encoding to the UTF-8 encoding, it relies + on the MARC::Charset Perl module to complete the conversion. When importing a large set of items, you can speed up the process by using a + utility like marc4j or marcdumper to convert the records + to MARC21XML and UTF-8 before running them through marc2bre.pl with the + flag to tell marc2bre.pl that the records are already in MARC21XML format with + the UTF-8 encoding. If you take this approach, due to a current limitation of MARC::File::XML you have to do a + horrible thing and ensure that there are no namespace prefixes in front of the element names. marc2bre.pl cannot parse the following + example: + + + + + + 00677nam a2200193 a 4500 + H01-0000844 + t + 060420s1950 xx 000 u fre d + + CaOHCU + fre + +... +]]>; + + + But marc2bre.pl can parse the same example with the namespace prefixes removed: + + + + + 00677nam a2200193 a 4500 + H01-0000844 + t + 060420s1950 xx 000 u fre d + + CaOHCU + fre + +... +]]>; + + + + + + Converting Records for Import into PostgreSQL + Once you have your records in Evergreen's BRE JSON format, you then need to use direct_ingest.pl to convert the records + into the generic ingest JSON format for Open-ILS. + This step uses the open-ils.ingest application to extract the data that will be indexed in the database. + Once you have your records in Open-ILS JSON ingest format, you then need to use pg_loader.pl to convert these records into a + set of SQL statements that you can use to + load the records into PostgreSQL. The and command line options (bre, mrd, mfr, etc) map to class IDs defined in + /openils/conf/fm_IDL.xml. + + + Adding Metarecords to the Database + One you have loaded the records into PostgreSQL, you can create metarecord entries in the metabib.metarecord table by running the following SQL: + +psql evergreen +# \i /home/opensrf/Evergreen-ILS-1.6*/src/extras/import/quick_metarecord_map.sql + + Metarecords are required to place holds on items, among other actions. + +
+
+ Adding Copies to Bibliographic Records + Once you've loaded the bibliographic records in Evergreen, you can search and view the records in the staff client, but they will not be visible in the catalogue. By + default, bibliographic records will not be visible in the catalogue until you add a copy representing a physical manifestation of that resource. You can add a copy manually through + the staff client via the Holdings maintenance screen, but if you're bulk-importing MARC records you probably want to bulk load the associated copies, call numbers, and barcodes as + well. + + Importing volumes and copies from <systemitem>MARC21XML</systemitem> holdings + There is currently no simple method for importing holdings based on the contents of the MARC holdings field (852, as specified by + http://www.loc.gov/marc/holdings/). + However, a more or less automated method could be built that performs the following steps: + + Create a tab-delimited file that contains your holdings information + + Required fields: bibliographic ID, barcode, and call number + Optional fields: shelving location (text) – see the asset.copy table for + possible fields to include + + + Create a staging table that matches the contents of your tab-delimited file. + + Insert the contents of your tab-delimited file into the table. + + Generate SQL scripts for item import to match the staging table that you created. + + Run the SQL scripts to create the holdings in Evergreen. + + + If an ILS has the concept of item categories, these may be mapped to Evergreen via statistical categories in the + asset.stat_cat table . Note that statistical categories cannot be used as search filters; individual branches can define + their own statistical categories; and define their own statistical category entries for individual items - best use case for statistical categories is probably for gifts. + In 2009, Conifer placed their migration tools + in the Conifer ILS-Contrib SVN repository, which might be useful samples augmenting the + basic staging table import approach. + In 2010, Equinox contributed a set of migration utilities + +
Migrating Patron Data @@ -179,4 +325,56 @@ COMMIT; the convenience of patron loads and the cost of processing these loads vs staff adding patrons manually.
+
+ Making electronic resources visible in the catalogue + For electronic resources that should be visible in the catalogue without any copies, you must set the source column value in the record.biblio_entry row for the respective + bibliographic record to a value that matches the corresponding ID of the config.bib_source where the transcendant value is + TRUE. Here's a practical example: + + + Connect to your Evergreen database with psql (substitute username / database name as required): + +psql -U postgres evergreen + + + + Add a source for your electronic resources: + +# INSERT INTO config.bib_source(quality, source, transcendant) VALUES (50, 'Institutional repository', TRUE); + + + + Find the ID that was generated for your new source: + +# SELECT ID FROM config.bib_source WHERE source = 'Institutional repository'; + + + + Update the source column for your bibliographic record for the electronic resource (for the sake of the example, let's assume that the ID returned from the new + source was 4, and that we know that the bib record with ID 75 is an electronic resource from your institutional repository): + +# UPDATE biblio.record_entry SET source = 4 where biblio.record_entry.id=75; + + + +
+
+ Restoring your Evergreen Database to an Empty State + If you've done a test import of records and you want to quickly get Evergreen back to a pristine state, you can create a clean Evergreen database schema by performing the + following: + + + +cd ILS/Open-ILS/src/sql/Pg/ + + + + Rebuild the database schema: + +./build-db.sh [db-hostname> [db-port] [db-name] [db-user] [db-password] [db-version] + + This will remove all of your data from the database and restore the default values. + + +
diff --git a/1.6/admin/serveradministration.xml b/1.6/admin/serveradministration.xml index a88f156e51..982e16a373 100644 --- a/1.6/admin/serveradministration.xml +++ b/1.6/admin/serveradministration.xml @@ -21,6 +21,8 @@ xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:id="serveradministrati Evergreen. Evergreen can support organizations as simple as a single library with one or more branches or as complex as a consortium composed of many independently governed library systems. Organizational unit types should never have proper names since they are only generic types . + It is a good idea to set up all of your organizational types and units before loading other data. In many cases, editing or deleting organizational units and types + may be difficult once you have loaded records or users. The fields in the organizational unit type record include: Type Name - The name of the organization unit type. @@ -55,6 +57,13 @@ xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:id="serveradministrati to be have the organization units of this type as their home unit. Click Save to save your new organization type. + From the server command line, run autogen to apply the changes to the database and scripts. Run the following command as the opensrf user: + +/openils/bin/autogen.sh -c /openils/conf/opensrf_core.xml -u + + + The staff client will need to be restarted for changes to appear. + Deleting Organization Types<indexterm><primary>organization types</primary><secondary>deleting</secondary></indexterm> @@ -62,12 +71,20 @@ xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:id="serveradministrati tree. Click Delete. Click OK on the warning alert box. + From the server command line, run autogen to apply the changes to the database and scripts. Run the following command as the opensrf user: + +/openils/bin/autogen.sh -c /openils/conf/opensrf_core.xml -u + + + The staff client will need to be restarted for changes to appear. + You will not be able to delete organization types if organization units are assigned to that type. Before you can delete the organization Type, you must change the organization type of the units associated with the type or delete the units. + Editing Organization Types<indexterm><primary>organization types</primary><secondary>editing</secondary></indexterm> @@ -75,6 +92,13 @@ xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:id="serveradministrati organization type tree. Make the changes in the right pane. Click Save to save your changes. + From the server command line, run autogen to apply the changes to the database and scripts. Run the following command as the opensrf user: + +/openils/bin/autogen.sh -c /openils/conf/opensrf_core.xml -u + + + The staff client will need to be restarted for changes to appear. + @@ -113,18 +137,32 @@ xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:id="serveradministrati and copies Check the OPAC Visible check box if you want this location to be visible in the OPAC for searching.Organization Unitsopac visible - Click Save to save your new organizational unit. + Click Save to save your new organizational unit. + From the server command line, run autogen to apply the changes to the database and scripts. Run the following command as the opensrf user: + +/openils/bin/autogen.sh -c /openils/conf/opensrf_core.xml -u + + + The staff client will need to be restarted for changes to appear. + Deleting Organizational Units <indexterm><primary>organization units</primary><secondary>deleting</secondary></indexterm> Select the organizational unit you wish to delete from the organizational unit tree in the left pane. ClickDelete. Click OK on the warning alert box. + From the server command line, run autogen to apply the changes to the database and scripts. Run the following command as the opensrf user: + +/openils/bin/autogen.sh -c /openils/conf/opensrf_core.xml -u + + + The staff client will need to be restarted for changes to appear. + You will not be able to delete organizational units if you have users, workstations or copies assigned to the unit. Before you can delete the organizational unit, you must move its users, workstations, copies and other associated resources to other - organizational units units. + organizational units. @@ -133,6 +171,13 @@ xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:id="serveradministrati organizational unit tree in the left pane. Edit the fields in the right pane. Click Save to save your changes. + From the server command line, run autogen to apply the changes to the database and scripts. Run the following command as the opensrf user: + +/openils/bin/autogen.sh -c /openils/conf/opensrf_core.xml -u + + + The staff client will need to be restarted for changes to appear. + diff --git a/style_guide/glossary.xml b/style_guide/glossary.xml index 96ea591e36..af9d6cb940 100644 --- a/style_guide/glossary.xml +++ b/style_guide/glossary.xml @@ -529,6 +529,90 @@ subdivided with section or simplesect elements. + + + systemitem + + + Indicates various system related resources and items. See the Docbook entry for classification possibilities. Use this in place of emphasis where applicable. + + + + command + + + The name of an executable program or other software command. Use this instead of emphasis where applicable. + + + + option + + + Specify an option for a software command. Use this instead of emphasis where applicable. + + + + filename + + + Specify a name of a file or directory. See docbook entrey for class options. + + + + screen + + + Text that a user sees or might see on a computer screen + + + + userinput + + + Data entered by the user. Used with screen to differentiate input from output. + + + + computeroutput + + + Computer generated output. Used with screen to differentiate output from input. + + + + code + + + An inline code fragment. + + + + errortext + + + An error message. + + + + function + + + The name of a function or subroutine, as in a programming language. + + + + prompt + + + A character or string indicating the start of an input field in a computer display. Used with screen to indicate a prompt for user input. + + + + varname + + + The name of a variable. Use this instead of emphasis where applicable. +
@@ -581,7 +665,6 @@ green, red, and yellow respectively. -
-- 2.43.2