1.6/development/datamodelsandaccess.xml

   1 <?xml version="1.0" encoding="UTF-8"?>\r
   2 <chapter xml:id="data_models_and_access" xmlns="http://docbook.org/ns/docbook" version="5.0" xml:lang="EN"\r
   3     xmlns:xi="http://www.w3.org/2001/XInclude" xmlns:xlink="http://www.w3.org/1999/xlink">\r
   4         <chapterinfo>\r
   5         <title>Evergreen Data Models and Access</title>\r
   6         </chapterinfo>\r
   7                 <abstract id="DM_abstract">\r
   8                 <simpara>This chapter was taken from Dan Scott's <emphasis>Developer Workshop</emphasis>, February 2010.</simpara>\r
   9         </abstract>\r
  10         <section id="exploring_database_schema">\r
  11                 <title>Exploring the Database Schema</title>\r
  12                 <simpara>The database schema is tied pretty tightly to PostgreSQL. Although PostgreSQL<indexterm><primary>databases</primary><secondary>PostgreSQL</secondary></indexterm>\r
  13                 adheres closely to ANSI SQL standards, the use of schemas, SQL functions<indexterm><primary>ANSI</primary></indexterm>\r
  14                 implemented in both <systemitem>plpgsql</systemitem> and <systemitem>plperl</systemitem>, and PostgreSQL&#8217;s native full-text\r
  15                 search would make it&#8230; challenging&#8230; to port to other database platforms.</simpara>\r
  16                 <simpara>A few common PostgreSQL interfaces for poking around the schema and\r
  17                 manipulating data are:</simpara>\r
  18                 <itemizedlist>\r
  19                 <listitem>\r
  20                 <simpara>\r
  21                 psql (the command line client)<indexterm><primary>databases</primary><secondary>PostgreSQL</secondary><tertiery>psql</tertiery></indexterm>\r
  22                 </simpara>\r
  23                 </listitem>\r
  24                 <listitem>\r
  25                 <simpara>\r
  26                 pgadminIII (a GUI client).<indexterm><primary>databases</primary><secondary>PostgreSQL</secondary><tertiery>pgadminIII</tertiery></indexterm>\r
  27                 </simpara>\r
  28                 </listitem>\r
  29                 </itemizedlist>\r
  30                 <simpara>Or you can read through the source files in <filename class="directoy">Open-ILS/src/sql/Pg</filename>.</simpara>\r
  31                 <simpara>Let&#8217;s take a quick tour through the schemas, pointing out some highlights\r
  32                 and some key interdependencies:</simpara>\r
  33                 <itemizedlist>\r
  34                 <listitem>\r
  35                 <simpara>\r
  36                 actor.org_unit &#8594; asset.copy_location\r
  37                 </simpara>\r
  38                 </listitem>\r
  39                 <listitem>\r
  40                 <simpara>\r
  41                 actor.usr &#8594; actor.card\r
  42                 </simpara>\r
  43                 </listitem>\r
  44                 <listitem>\r
  45                 <simpara>\r
  46                 biblio.record_entry &#8594; asset.call_number &#8594; asset.copy\r
  47                 </simpara>\r
  48                 </listitem>\r
  49                 <listitem>\r
  50                 <simpara>\r
  51                 config.metabib_field &#8594; metabib.*_field_entry\r
  52                 </simpara>\r
  53                 </listitem>\r
  54                 </itemizedlist>\r
  55                 This documentation also contains an Appendix for the Evergreen <xref linkend="databaseschema"/>.         \r
  56         </section>\r
  57         <section id="_database_access_methods">\r
  58                 <title>Database access methods</title>\r
  59                 <simpara>You could use direct access to the database via Perl DBI, JDBC, etc,\r
  60                 but Evergreen offers several database CRUD services for\r
  61                 creating / retrieving / updating / deleting data. These avoid tying\r
  62                 you too tightly to the current database schema and they funnel database\r
  63                 access through the same mechanism, rather than tying up connections\r
  64                 with other interfaces.</simpara>\r
  65         </section>\r
  66         <section id="_evergreen_interface_definition_language_idl">\r
  67                 <title>Evergreen Interface Definition Language (IDL)</title>\r
  68                 <indexterm><primary>Evergreen Interface Definition Language (IDL)</primary></indexterm>\r
  69                 <simpara>Defines properties and required permissions for Evergreen classes.\r
  70                 To reduce network overhead, a given object is identified via a\r
  71                 class-hint and serialized as a JSON array of properties (no named properties).</simpara>\r
  72                 <simpara>As of 1.6, fields will be serialized in the order in which they appear\r
  73                 in the IDL definition file, and the is_new / is_changed / is_deleted\r
  74                 properties are automatically added. This has greatly reduced the size of\r
  75                 the <literal>fm_IDL.xml</literal> file and makes DRY people happier :)</simpara>\r
  76                 <itemizedlist>\r
  77                 <listitem>\r
  78                 <simpara>\r
  79                 &#8230; oils_persist:readonly tells us, if true, that the data lives in the database, but is pulled from the SELECT statement defined in the &lt;oils_persist:source_definition&gt; \r
  80                 child element\r
  81                 </simpara>\r
  82                 </listitem>\r
  83                 </itemizedlist>\r
  84                 <simplesect id="_idl_basic_example_config_language_map">\r
  85                         <title>IDL basic example (config.language_map)</title>\r
  86 <programlisting language="xml" linenumbering="unnumbered">\r
  87 &lt;class id="clm" controller="open-ils.cstore open-ils.pcrud"\r
  88         oils_obj:fieldmapper="config::language_map"\r
  89         oils_persist:tablename="config.language_map"\r
  90         reporter:label="Language Map" oils_persist:field_safe="true"&gt; <co id="dmCO5-1"/> <co id="dmCO5-2"/> <co id="dmCO5-3"/> <co id="dmCO5-4"/>\r
  91     &lt;fields oils_persist:primary="code" oils_persist:sequence=""&gt;  <co id="dmCO5-5"/>\r
  92         &lt;field reporter:label="Language Code" name="code"\r
  93             reporter:selector="value" reporter:datatype="text"/&gt; <co id="dmCO5-6"/>\r
  94         &lt;field reporter:label="Language" name="value"\r
  95             reporter:datatype="text" oils_persist:i18n="true"/&gt; <co id="dmCO5-7"/>\r
  96     &lt;/fields&gt;\r
  97     &lt;links/&gt;\r
  98     &lt;permacrud xmlns="http://open-ils.org/spec/opensrf/IDL/permacrud/v1"&gt; <co id="dmCO5-8"/>\r
  99         &lt;actions&gt;\r
 100             &lt;create global_required="true" permission="CREATE_MARC_CODE"&gt; <co id="dmCO5-9"/>\r
 101             &lt;retrieve global_required="true"\r
 102                 permission="CREATE_MARC_CODE UPDATE_MARC_CODE DELETE_MARC_CODE"&gt;\r
 103             &lt;update global_required="true" permission="UPDATE_MARC_CODE"&gt;\r
 104             &lt;delete global_required="true" permission="DELETE_MARC_CODE"&gt;\r
 105         &lt;/actions&gt;\r
 106     &lt;/permacrud&gt;\r
 107 &lt;/class&gt;\r
 108 </programlisting>\r
 109                         <calloutlist>\r
 110                         <callout arearefs="dmCO5-1">\r
 111                         <simpara>\r
 112                         The <literal>class</literal> element defines the attributes and permissions for classes,\r
 113                         and relationships between classes.\r
 114                         </simpara>\r
 115                         <indexterm><primary>Evergreen Interface Definition Language (IDL)</primary><secondary>class element</secondary></indexterm>\r
 116                         <itemizedlist>\r
 117                         <listitem>\r
 118                         <simpara>\r
 119                         The <literal>id</literal> attribute on the <literal>class</literal> element defines the class hint that is\r
 120                         used everywhere in Evergreen.\r
 121                         </simpara>\r
 122                         </listitem>\r
 123                         <listitem>\r
 124                         <simpara>\r
 125                         The <literal>controller</literal> attribute defines the OpenSRF\r
 126                         services that provide access to the data for the class objects.\r
 127                         </simpara>\r
 128                         </listitem>\r
 129                         </itemizedlist>\r
 130                         </callout>\r
 131                         <callout arearefs="dmCO5-2">\r
 132                         <simpara>\r
 133                         The <literal>oils_obj::fieldmapper</literal> attribute defines the name of the class that\r
 134                         is generated by <literal>OpenILS::Utils::Fieldmapper</literal>.\r
 135                         </simpara>\r
 136                         </callout>\r
 137                         <callout arearefs="dmCO5-3">\r
 138                         <simpara>\r
 139                         The <literal>oils_persist:tablename</literal> attribute defines the name of the table\r
 140                         that contains the data for the class objects.\r
 141                         </simpara>\r
 142                         </callout>\r
 143                         <callout arearefs="dmCO5-4">\r
 144                         <simpara>\r
 145                         The reporter interface uses <literal>reporter:label</literal> attribute values in\r
 146                         the source list to provide meaningful class and attribute names. The\r
 147                         <literal>open-ils.fielder</literal> service generates a set of methods that provide direct\r
 148                         access to the classes for which <literal>oils_persist:field_safe</literal> is <literal>true</literal>. For\r
 149                         example,\r
 150                         </simpara>\r
 151 <screen>\r
 152 <userinput>\r
 153 srfsh# request open-ils.fielder open-ils.fielder.clm.atomic \\r
 154 {"query":{"code":{"=":"eng"}}}\r
 155 \r
 156 Received Data: [\r
 157         {\r
 158         "value":"English",\r
 159         "code":"eng"\r
 160         }\r
 161 ]\r
 162 </userinput>\r
 163 </screen>\r
 164                         </callout>\r
 165                         <callout arearefs="dmCO5-5">\r
 166                         <simpara>\r
 167                         The <literal>fields</literal> element defines the list of fields for the class.\r
 168                         </simpara>\r
 169                         <indexterm><primary>Evergreen Interface Definition Language (IDL)</primary><secondary>fields element</secondary></indexterm>\r
 170                         <itemizedlist>\r
 171                         <listitem>\r
 172                         <simpara>\r
 173                         The <literal>oils_persist:primary</literal> attribute defines the column that acts as\r
 174                         the primary key for the table.\r
 175                         </simpara>\r
 176                         </listitem>\r
 177                         <listitem>\r
 178                         <simpara>\r
 179                         The <literal>oils_persist:sequence</literal> attribute holds the name of the database\r
 180                         sequence.\r
 181                         </simpara>\r
 182                         </listitem>\r
 183                         </itemizedlist>\r
 184                         </callout>\r
 185                         <callout arearefs="dmCO5-6">\r
 186                         <simpara>\r
 187                         Each <literal>field</literal> element defines one property of the class.\r
 188                         </simpara>\r
 189                         <indexterm><primary>Evergreen Interface Definition Language (IDL)</primary><secondary>field element</secondary></indexterm>\r
 190                         <itemizedlist>\r
 191                         <listitem>\r
 192                         <simpara>\r
 193                         The <literal>name</literal> attribute defines the getter/setter method name for the field.\r
 194                         </simpara>\r
 195                         </listitem>\r
 196                         <listitem>\r
 197                         <simpara>\r
 198                         The <literal>reporter:label</literal> attribute defines the attribute name as used in\r
 199                         the reporter interface.\r
 200                         </simpara>\r
 201                         </listitem>\r
 202                         <listitem>\r
 203                         <simpara>\r
 204                         The <literal>reporter:selector</literal> attribute defines the field used in the reporter\r
 205                         filter interface to provide a selectable list. This gives the user a more\r
 206                         meaningful access point than the raw numeric ID or abstract code.\r
 207                         </simpara>\r
 208                         </listitem>\r
 209                         <listitem>\r
 210                         <simpara>\r
 211                         The <literal>reporter:datatype</literal> attribute defines the type of data held by\r
 212                         this property for the purposes of the reporter.\r
 213                         </simpara>\r
 214                         </listitem>\r
 215                         </itemizedlist>\r
 216                         </callout>\r
 217                         <callout arearefs="dmCO5-7">\r
 218                         <simpara>\r
 219                         The <literal>oils_persist:i18n</literal> attribute, when <literal>true</literal>, means that\r
 220                         translated values for the field&#8217;s contents may be accessible in\r
 221                         different locales.\r
 222                         </simpara>\r
 223                         </callout>\r
 224                         <callout arearefs="dmCO5-8">\r
 225                         <simpara>\r
 226                         <indexterm><primary>Evergreen Interface Definition Language (IDL)</primary><secondary>permacrud element</secondary></indexterm>\r
 227                         The <literal>permacrud</literal> element defines the permissions (if any) required\r
 228                         to <emphasis role="strong">c</emphasis>reate, <emphasis role="strong">r</emphasis>etrieve, <emphasis role="strong">u</emphasis>pdate, \r
 229                         and <emphasis role="strong">d</emphasis>elete data for this\r
 230                         class. <literal>open-ils.permacrud</literal> must be defined as a controller for the class\r
 231                         for the permissions to be applied.\r
 232                         </simpara>\r
 233                         \r
 234                         </callout>\r
 235                         <callout arearefs="dmCO5-9">\r
 236                         <simpara>\r
 237                         Each action requires one or more <literal>permission</literal> values that the\r
 238                         user must possess to perform the action.\r
 239                         </simpara>\r
 240                         <itemizedlist>\r
 241                         <listitem>\r
 242                         <simpara>\r
 243                         If the <literal>global_required</literal> attribute is <literal>true</literal>, then the user must\r
 244                         have been granted that permission globally (depth = 0) to perform\r
 245                         the action.\r
 246                         </simpara>\r
 247                         </listitem>\r
 248                         <listitem>\r
 249                         <simpara>\r
 250                         The <literal>context_field</literal> attribute denotes the <literal>&lt;field&gt;</literal> that identifies\r
 251                         the org_unit at which the user must have the pertinent permission.\r
 252                         </simpara>\r
 253                         </listitem>\r
 254                         <listitem>\r
 255                         <indexterm><primary>Evergreen Interface Definition Language (IDL)</primary><secondary>action element</secondary></indexterm>\r
 256                         <simpara>\r
 257                         An action element may contain a <literal>&lt;context_field&gt;</literal> element that\r
 258                         defines the linked class (identified by the <literal>link</literal> attribute) and\r
 259                         the field in the linked class that identifies the org_unit where\r
 260                         the permission must be held.\r
 261                         </simpara>\r
 262                         <itemizedlist>\r
 263                         <listitem>\r
 264                         <indexterm><primary>Evergreen Interface Definition Language (IDL)</primary><secondary>context_field element</secondary></indexterm>\r
 265                         <simpara>\r
 266                         If the <literal>&lt;context_field&gt;</literal> element contains a <literal>jump</literal> attribute,\r
 267                         then it defines a link to a link to a class with a field identifying\r
 268                         the org_unit where the permission must be held.\r
 269                         </simpara>\r
 270                         </listitem>\r
 271                         </itemizedlist>\r
 272                         </listitem>\r
 273                         </itemizedlist>\r
 274                         </callout>\r
 275                         </calloutlist>\r
 276                 </simplesect>\r
 277                 <simplesect id="_reporter_data_types_and_their_possible_values">\r
 278                         <title>Reporter data types and their possible values</title>\r
 279                         <itemizedlist>\r
 280                         <listitem>\r
 281                         <simpara>\r
 282                         <literal>bool</literal>: Boolean <literal>true</literal> or <literal>false</literal>\r
 283                         </simpara>\r
 284                         </listitem>\r
 285                         <listitem>\r
 286                         <simpara>\r
 287                         <literal>id</literal>: ID of the row in the database\r
 288                         </simpara>\r
 289                         </listitem>\r
 290                         <listitem>\r
 291                         <simpara>\r
 292                         <literal>int</literal>: integer value\r
 293                         </simpara>\r
 294                         </listitem>\r
 295                         <listitem>\r
 296                         <simpara>\r
 297                         <literal>interval</literal>: PostgreSQL time interval\r
 298                         </simpara>\r
 299                         </listitem>\r
 300                         <listitem>\r
 301                         <simpara>\r
 302                         <literal>link</literal>: link to another class, as defined in the <literal>&lt;links&gt;</literal>\r
 303                         element of the class definition\r
 304                         </simpara>\r
 305                         </listitem>\r
 306                         <listitem>\r
 307                         <simpara>\r
 308                         <literal>money</literal>: currency amount\r
 309                         </simpara>\r
 310                         </listitem>\r
 311                         <listitem>\r
 312                         <simpara>\r
 313                         <literal>org_unit</literal>: list of org_units\r
 314                         </simpara>\r
 315                         </listitem>\r
 316                         <listitem>\r
 317                         <simpara>\r
 318                         <literal>text</literal>: text value\r
 319                         </simpara>\r
 320                         </listitem>\r
 321                         <listitem>\r
 322                         <simpara>\r
 323                         <literal>timestamp</literal>: PostgreSQL timestamp\r
 324                         </simpara>\r
 325                         </listitem>\r
 326                         </itemizedlist>\r
 327                 </simplesect>\r
 328                 <simplesect id="_idl_example_with_linked_fields_actor_workstation">\r
 329                         <title>IDL example with linked fields (actor.workstation)</title>\r
 330                         <simpara>Just as tables often include columns with foreign keys that point\r
 331                         to values stored in the column of a different table, IDL classes\r
 332                         can contain fields that link to fields in other classes. The <literal>&lt;links&gt;</literal>\r
 333                         element defines which fields link to fields in other classes, and\r
 334                         the nature of the relationship:</simpara>\r
 335 <programlisting language="xml" linenumbering="unnumbered">\r
 336 &lt;class id="aws" controller="open-ils.cstore"\r
 337         oils_obj:fieldmapper="actor::workstation"\r
 338         oils_persist:tablename="actor.workstation"\r
 339         reporter:label="Workstation"&gt;\r
 340     &lt;fields oils_persist:primary="id"\r
 341             oils_persist:sequence="actor.workstation_id_seq"&gt;\r
 342         &lt;field reporter:label="Workstation ID" name="id"\r
 343                 reporter:datatype="id"/&gt;\r
 344         &lt;field reporter:label="Workstation Name" name="name"\r
 345                 reporter:datatype="text"/&gt;\r
 346         &lt;field reporter:label="Owning Library" name="owning_lib"\r
 347                 reporter:datatype="org_unit"/&gt;\r
 348         &lt;field reporter:label="Circulations" name="circulations"\r
 349                 oils_persist:virtual="true" reporter:datatype="link"/&gt; <co id="dmCO6-1"/>\r
 350     &lt;/fields&gt;\r
 351     &lt;links&gt;  <co id="dmCO6-2"/>\r
 352         &lt;link field="owning_lib" reltype="has_a" key="id"\r
 353                 map="" class="aou"/&gt;  <co id="dmCO6-3"/>\r
 354         &lt;link field="circulations" reltype="has_many" key="workstation"\r
 355                 map="" class="circ"/&gt;\r
 356         &lt;link field="circulation_checkins" reltype="has_many"\r
 357                 key="checkin_workstation" map="" class="circ"/&gt;\r
 358     &lt;/links&gt;\r
 359 &lt;/class&gt;\r
 360 </programlisting>\r
 361                         <calloutlist>\r
 362                         <callout arearefs="dmCO6-1">\r
 363                         <simpara>\r
 364                         This field includes an <literal>oils_persist:virtual</literal> attribute with the value of\r
 365                         <literal>true</literal>, meaning that the linked class <literal>circ</literal> is a virtual class.\r
 366                         </simpara>\r
 367                         </callout>\r
 368                         <callout arearefs="dmCO6-2">\r
 369                         <simpara>\r
 370                         The <literal>&lt;links&gt;</literal> element contains 0 or more <literal>&lt;link&gt;</literal> elements.\r
 371                         </simpara>\r
 372                         </callout>\r
 373                         <callout arearefs="dmCO6-3">\r
 374                         <simpara>\r
 375                         Each <literal>&lt;link&gt;</literal> element defines the field (<literal>field</literal>) that links to a different\r
 376                         class (<literal>class</literal>), the relationship (<literal>rel_type</literal>) between this field and the target\r
 377                         field (<literal>key</literal>). If the field in this class links to a virtual class, the (<literal>map</literal>)\r
 378                         attribute defines the field in the target class that returns a list of matching\r
 379                         objects for each object in this class.\r
 380                         </simpara>\r
 381                         </callout>\r
 382                         </calloutlist>\r
 383                 </simplesect>\r
 384         </section>\r
 385         <section id="open_ils_cstore_literal_data_access_interfaces">\r
 386                 <title><literal>open-ils.cstore</literal> data access interfaces</title>\r
 387                 <indexterm><primary>cstore</primary></indexterm>\r
 388                 <simpara>For each class documented in the IDL, the <literal>open-ils.cstore</literal> service\r
 389                 automatically generates a set of data access methods, based on the\r
 390                 <literal>oils_persist:tablename</literal> class attribute.</simpara>\r
 391                 <simpara>For example, for the class hint <literal>clm</literal>, cstore generates the following\r
 392                 methods with the <literal>config.language_map</literal> qualifer:</simpara>\r
 393                 <itemizedlist>\r
 394                 <listitem>\r
 395                 <simpara>\r
 396                 <literal>open-ils.cstore.direct.config.language_map.id_list {"code" { "like": "e%" } }</literal>\r
 397                 </simpara>\r
 398                 <simpara>Retrieves a list composed only of the IDs that match the query.</simpara>\r
 399                 </listitem>\r
 400                 <listitem>\r
 401                 <simpara>\r
 402                 <literal>open-ils.cstore.direct.config.language_map.retrieve "eng"</literal>\r
 403                 </simpara>\r
 404                 <simpara>Retrieves the object that matches a specific ID.</simpara>\r
 405                 </listitem>\r
 406                 <listitem>\r
 407                 <simpara>\r
 408                 <literal>open-ils.cstore.direct.config.language_map.search {"code" : "eng"}</literal>\r
 409                 </simpara>\r
 410                 <simpara>Retrieves a list of objects that match the query.</simpara>\r
 411                 </listitem>\r
 412                 <listitem>\r
 413                 <simpara>\r
 414                 <literal>open-ils.cstore.direct.config.language_map.create &lt;_object_&gt;</literal>\r
 415                 </simpara>\r
 416                 <simpara>Creates a new object from the passed in object.</simpara>\r
 417                 </listitem>\r
 418                 <listitem>\r
 419                 <simpara>\r
 420                 <literal>open-ils.cstore.direct.config.language_map.update &lt;_object_&gt;</literal>\r
 421                 </simpara>\r
 422                 <simpara>Updates the object that has been passed in.</simpara>\r
 423                 </listitem>\r
 424                 <listitem>\r
 425                 <simpara>\r
 426                 <literal>open-ils.cstore.direct.config.language_map.delete "eng"</literal>\r
 427                 </simpara>\r
 428                 <simpara>Deletes the object that matches the query.</simpara>\r
 429                 </listitem>\r
 430                 </itemizedlist>\r
 431         </section>\r
 432         <section id="_open_ils_pcrud_data_access_interfaces">\r
 433                 <title>open-ils.pcrud data access interfaces</title>\r
 434                 <indexterm><primary>pcrud</primary></indexterm>\r
 435                 <simpara>For each class documented in the IDL, the <literal>open-ils.pcrud</literal> service\r
 436                 automatically generates a set of data access methods, based on the\r
 437                 <literal>oils_persist:tablename</literal> class attribute.</simpara>\r
 438                 <simpara>For example, for the class hint <literal>clm</literal>, <literal>open-ils.pcrud</literal> generates the following\r
 439                 methods that parallel the <literal>open-ils.cstore</literal> interface:</simpara>\r
 440                 <itemizedlist>\r
 441                 <listitem>\r
 442                 <simpara>\r
 443                 <literal>open-ils.pcrud.id_list.clm &lt;_authtoken_&gt;, { "code": { "like": "e%" } }</literal>\r
 444                 </simpara>\r
 445                 </listitem>\r
 446                 <listitem>\r
 447                 <simpara>\r
 448                 <literal>open-ils.pcrud.retrieve.clm &lt;_authtoken_&gt;, "eng"</literal>\r
 449                 </simpara>\r
 450                 </listitem>\r
 451                 <listitem>\r
 452                 <simpara>\r
 453                 <literal>open-ils.pcrud.search.clm &lt;_authtoken_&gt;, { "code": "eng" }</literal>\r
 454                 </simpara>\r
 455                 </listitem>\r
 456                 <listitem>\r
 457                 <simpara>\r
 458                 <literal>open-ils.pcrud.create.clm &lt;_authtoken_&gt;, &lt;_object_&gt;</literal>\r
 459                 </simpara>\r
 460                 </listitem>\r
 461                 <listitem>\r
 462                 <simpara>\r
 463                 <literal>open-ils.pcrud.update.clm &lt;_authtoken_&gt;, &lt;_object_&gt;</literal>\r
 464                 </simpara>\r
 465                 </listitem>\r
 466                 <listitem>\r
 467                 <simpara>\r
 468                 <literal>open-ils.pcrud.delete.clm &lt;_authtoken_&gt;, "eng"</literal>\r
 469                 </simpara>\r
 470                 </listitem>\r
 471                 </itemizedlist>\r
 472         </section>\r
 473         <section id="_transaction_and_savepoint_control">\r
 474                 <title>Transaction and savepoint control</title>\r
 475                 <simpara>Both <literal>open-ils.cstore</literal> and <literal>open-ils.pcrud</literal> enable you to control database transactions\r
 476                 to ensure that a set of operations either all succeed, or all fail,\r
 477                 atomically:</simpara>\r
 478                 <itemizedlist>\r
 479                 <listitem>\r
 480                 <simpara>\r
 481                 <literal>open-ils.cstore.transaction.begin</literal>\r
 482                 </simpara>\r
 483                 </listitem>\r
 484                 <listitem>\r
 485                 <simpara>\r
 486                 <literal>open-ils.cstore.transaction.commit</literal>\r
 487                 </simpara>\r
 488                 </listitem>\r
 489                 <listitem>\r
 490                 <simpara>\r
 491                 <literal>open-ils.cstore.transaction.rollback</literal>\r
 492                 </simpara>\r
 493                 </listitem>\r
 494                 <listitem>\r
 495                 <simpara>\r
 496                 <literal>open-ils.pcrud.transaction.begin</literal>\r
 497                 </simpara>\r
 498                 </listitem>\r
 499                 <listitem>\r
 500                 <simpara>\r
 501                 <literal>open-ils.pcrud.transaction.commit</literal>\r
 502                 </simpara>\r
 503                 </listitem>\r
 504                 <listitem>\r
 505                 <simpara>\r
 506                 <literal>open-ils.pcrud.transaction.rollback</literal>\r
 507                 </simpara>\r
 508                 </listitem>\r
 509                 </itemizedlist>\r
 510                 <simpara>At a more granular level, <literal>open-ils.cstore</literal> and <literal>open-ils.pcrud</literal> enable you to set database\r
 511                 savepoints to ensure that a set of operations either all succeed, or all\r
 512                 fail, atomically, within a given transaction:</simpara>\r
 513                 <itemizedlist>\r
 514                 <listitem>\r
 515                 <simpara>\r
 516                 <literal>open-ils.cstore.savepoint.begin</literal>\r
 517                 </simpara>\r
 518                 </listitem>\r
 519                 <listitem>\r
 520                 <simpara>\r
 521                 <literal>open-ils.cstore.savepoint.commit</literal>\r
 522                 </simpara>\r
 523                 </listitem>\r
 524                 <listitem>\r
 525                 <simpara>\r
 526                 <literal>open-ils.cstore.savepoint.rollback</literal>\r
 527                 </simpara>\r
 528                 </listitem>\r
 529                 <listitem>\r
 530                 <simpara>\r
 531                 <literal>open-ils.pcrud.savepoint.begin</literal>\r
 532                 </simpara>\r
 533                 </listitem>\r
 534                 <listitem>\r
 535                 <simpara>\r
 536                 <literal>open-ils.pcrud.savepoint.commit</literal>\r
 537                 </simpara>\r
 538                 </listitem>\r
 539                 <listitem>\r
 540                 <simpara>\r
 541                 <literal>open-ils.pcrud.savepoint.rollback</literal>\r
 542                 </simpara>\r
 543                 </listitem>\r
 544                 </itemizedlist>\r
 545                 <simpara>Transactions and savepoints must be performed within a stateful\r
 546                 connection to the <literal>open-ils.cstore</literal> and <literal>open-ils.pcrud</literal> services.\r
 547                 In <literal>srfsh</literal>, you can open a stateful connection using the <literal>open</literal>\r
 548                 command, and then close the stateful connection using the <literal>close</literal>\r
 549                 command - for example:</simpara>\r
 550                 <screen>srfsh# open open-ils.cstore\r
 551                 ... perform various transaction-related work\r
 552                 srfsh# close open-ils.cstore</screen>\r
 553                 <simplesect id="_json_queries">\r
 554                         <title>JSON Queries</title>\r
 555                         <indexterm><primary>JSON</primary></indexterm>\r
 556                         <simpara>Beyond simply retrieving objects by their ID using the <literal>\*.retrieve</literal>\r
 557                         methods, you can issue queries against the <literal>\*.delete</literal> and <literal>\*.search</literal>\r
 558                         methods using JSON to filter results with simple or complex search\r
 559                         conditions.</simpara>\r
 560                         <simpara>For example, to generate a list of barcodes that are held in a\r
 561                         copy location that allows holds and is visible in the OPAC:</simpara>\r
 562 <programlisting language="sh" linenumbering="unnumbered">\r
 563 srfsh# request open-ils.cstore open-ils.cstore.json_query  <co id="dmCO7-1"/>\r
 564     {"select": {"acp":["barcode"], "acpl":["name"]}, <co id="dmCO7-2"/>\r
 565      "from":   {"acp":"acpl"},   <co id="dmCO7-3"/>\r
 566      "where":  [     <co id="dmCO7-4"/>\r
 567          {"+acpl": "holdable"},   <co id="dmCO7-5"/>\r
 568          {"+acpl": "opac_visible"}     <co id="dmCO7-6"/>\r
 569      ]}\r
 570 \r
 571 Received Data: {\r
 572   "barcode":"BARCODE1",\r
 573   "name":"Stacks"\r
 574 }\r
 575 \r
 576 Received Data: {\r
 577   "barcode":"BARCODE2",\r
 578   "name":"Stacks"\r
 579 }\r
 580 </programlisting>\r
 581                         <calloutlist>\r
 582                         <callout arearefs="dmCO7-1">\r
 583                         <simpara>\r
 584                         Invoke the <literal>json_query</literal> service.\r
 585                         </simpara>\r
 586                         </callout>\r
 587                         <callout arearefs="dmCO7-2">\r
 588                         <simpara>\r
 589                         Select the <literal>barcode</literal> field from the <literal>acp</literal> class and the <literal>name</literal>\r
 590                         field from the <literal>acpl</literal> class.\r
 591                         </simpara>\r
 592                         </callout>\r
 593                         <callout arearefs="dmCO7-3">\r
 594                         <simpara>\r
 595                         Join the <literal>acp</literal> class to the <literal>acpl</literal> class based on the linked field\r
 596                         defined in the IDL.\r
 597                         </simpara>\r
 598                         </callout>\r
 599                         <callout arearefs="dmCO7-4">\r
 600                         <simpara>\r
 601                         Add a <literal>where</literal> clause to filter the results. We have more than one\r
 602                         condition beginning with the same key, so we wrap the conditions inside\r
 603                         an array.\r
 604                         </simpara>\r
 605                         </callout>\r
 606                         <callout arearefs="dmCO7-5">\r
 607                         <simpara>\r
 608                         The first condition tests whether the boolean value of the <literal>holdable</literal>\r
 609                         field on the <literal>acpl</literal> class is true.\r
 610                         </simpara>\r
 611                         </callout>\r
 612                         <callout arearefs="dmCO7-6">\r
 613                         <simpara>\r
 614                         The second condition tests whether the boolean value of the\r
 615                         <literal>opac_visible</literal> field on the <literal>acpl</literal> class is true.\r
 616                         </simpara>\r
 617                         </callout>\r
 618                         </calloutlist>\r
 619                         <simpara>For thorough coverage of the breadth of support offered by JSON\r
 620                         query syntax, see <ulink url="http://open-ils.org/dokuwiki/doku.php?id=documentation:technical:jsontutorial">JSON Queries: A Tutorial</ulink>.</simpara>\r
 621                 </simplesect>\r
 622                 <simplesect id="_fleshing_linked_objects">\r
 623                         <title>Fleshing linked objects</title>\r
 624                         <simpara>A simplistic approach to retrieving a set of objects that are linked to\r
 625                         an object that you are retrieving - for example, a set of call numbers\r
 626                         linked to the barcodes that a given user has borrowed - would be to:\r
 627                           1. Retrieve the list of circulation objects (<literal>circ</literal> class)\r
 628                         for a given user (<literal>usr</literal> class).\r
 629                           2. For each circulation object, look up the target copy (<literal>target_copy</literal>\r
 630                         field, linked to the <literal>acp</literal> class).\r
 631                           3. For each copy, look up the call number for that copy (<literal>call_number</literal>\r
 632                         field, linked to the <literal>acn</literal> class).</simpara>\r
 633                         <simpara>However, this would result in potentially hundreds of round-trip\r
 634                         queries from the client to the server. Even with low-latency connections,\r
 635                         the network overhead would be considerable. So, built into the <literal>open-ils.cstore</literal> and\r
 636                         <literal>open-ils.pcrud</literal> access methods is the ability to <emphasis>flesh</emphasis> linked fields -\r
 637                         that is, rather than return an identifier to a given linked field,\r
 638                         the method can return the entire object as part of the initial response.</simpara>\r
 639                         <simpara>Most of the interfaces that return class instances from the IDL offer the\r
 640                         ability to flesh returned fields. For example, the\r
 641                         <literal>open-ils.cstore.direct.\*.retrieve</literal> methods allow you to specify a\r
 642                         JSON structure defining the fields you wish to flesh in the returned object.</simpara>\r
 643                         <formalpara><title>Fleshing fields in objects returned by <literal>open-ils.cstore</literal></title><para>\r
 644 <programlisting language="sh" linenumbering="unnumbered">\r
 645 srfsh# request open-ils.cstore open-ils.cstore.direct.asset.copy.retrieve 1, \\r
 646     {\r
 647         "flesh": 1,    <co id="dmCO8-1"/>\r
 648         "flesh_fields": {   <co id="dmCO8-2"/>\r
 649             "acp": ["location"]\r
 650         }\r
 651     }\r
 652 </programlisting>\r
 653                         </para></formalpara>\r
 654                         <calloutlist>\r
 655                         <callout arearefs="dmCO8-1">\r
 656                         <simpara>\r
 657                         The <literal>flesh</literal> argument is the depth at which objects should be fleshed.\r
 658                         For example, to flesh out a field that links to another object that includes\r
 659                         a field that links to another object, you would specify a depth of 2.\r
 660                         </simpara>\r
 661                         </callout>\r
 662                         <callout arearefs="dmCO8-2">\r
 663                         <simpara>\r
 664                         The <literal>flesh_fields</literal> argument contains a list of objects with the fields\r
 665                         to flesh for each object.\r
 666                         </simpara>\r
 667                         </callout>\r
 668                         </calloutlist>\r
 669                         <simpara>Let&#8217;s flesh things a little deeper. In addition to the copy location,\r
 670                         let&#8217;s also flesh the call number attached to the copy, and then flesh\r
 671                         the bibliographic record attached to the call number.</simpara>\r
 672                         <formalpara><title>Fleshing fields in fields of objects returned by <literal>open-ils.cstore</literal></title><para>\r
 673 <programlisting language="java" linenumbering="unnumbered">\r
 674 request open-ils.cstore open-ils.cstore.direct.asset.copy.retrieve 1, \\r
 675     {\r
 676         "flesh": 2,\r
 677         "flesh_fields": {\r
 678             "acp": ["location", "call_number"],\r
 679             "acn": ["record"]\r
 680          }\r
 681     }\r
 682 </programlisting>\r
 683                         </para></formalpara>\r
 684                 </simplesect>\r
 685         </section>\r
 686         <section id="_adding_an_idl_entry_for_resolverresolver">\r
 687                 <title>Adding an IDL entry for ResolverResolver</title>\r
 688                 <simpara>Most OpenSRF methods in Evergreen define their object interface in the\r
 689                 IDL. Without an entry in the IDL, the prospective caller of a given\r
 690                 method is forced to either call the method and inspect the returned\r
 691                 contents, or read the source to work out the structure of the JSON\r
 692                 payload. At this stage of the tutorial, we have not defined an entry\r
 693                 in the IDL to represent the object returned by the\r
 694                 <literal>open-ils.resolver.resolve_holdings</literal> method. It is time to complete\r
 695                 that task.</simpara>\r
 696                 <simpara>The <literal>open-ils.resolver</literal> service is unlike many of the other classes\r
 697                 defined in the IDL because its data is not stored in the Evergreen\r
 698                 database. Instead, the data is requested from an external Web service\r
 699                 and only temporarily cached in <literal>memcached</literal>. Fortunately, the IDL\r
 700                 enables us to represent this kind of class by setting the\r
 701                 <literal>oils_persist:virtual</literal> class attribute to <literal>true</literal>.</simpara>\r
 702                 <simpara>So, let&#8217;s add an entry to the IDL for the <literal>open-ils.resolver.resolve_holdings</literal>\r
 703                 service:</simpara>\r
 704                 <programlisting language="xml" linenumbering="unnumbered"></programlisting>\r
 705                 <simpara>And let&#8217;s make <literal>ResolverResolver.pm</literal> return an array composed of our new\r
 706                 <literal>rhr</literal> classes rather than raw JSON objects:</simpara>\r
 707                 <programlisting language="perl" linenumbering="unnumbered"></programlisting>\r
 708                 <simpara>Once we add the new entry to the IDL and copy the revised <literal>ResolverResolver.pm</literal>\r
 709                 Perl module to <literal>/openils/lib/perl5/OpenILS/Application/</literal>, we need to:</simpara>\r
 710                 <orderedlist numeration="arabic">\r
 711                 <listitem>\r
 712                 <simpara>\r
 713                 Copy the updated IDL to both the <literal>/openils/conf/</literal> and\r
 714                 <literal>/openils/var/web/reports/</literal> directories. The Dojo approach to\r
 715                 parsing the IDL uses the IDL stored in the reports directory.\r
 716                 </simpara>\r
 717                 </listitem>\r
 718                 <listitem>\r
 719                 <simpara>\r
 720                 Restart the Perl services to make the new IDL visible to the services\r
 721                 and refresh the <literal>open-ils.resolver</literal> implementation\r
 722                 </simpara>\r
 723                 </listitem>\r
 724                 <listitem>\r
 725                 <simpara>\r
 726                 Rerun <filename>/openils/bin/autogen.sh</filename> to regenerate the JavaScript versions<indexterm><primary>autogen</primary></indexterm>\r
 727                 of the IDL required by the HTTP translator and gateway.\r
 728                 </simpara>\r
 729                 </listitem>\r
 730                 </orderedlist>\r
 731                 <simpara>We also need to adjust our JavaScript client to use the nifty new<indexterm><primary>JavaScript</primary></indexterm>\r
 732                 objects that <literal>open-ils.resolver.resolve_holdings</literal> now returns.\r
 733                 The best approach is to use the support in Evergreen&#8217;s Dojo extensions<indexterm><primary>Dojo toolkit</primary></indexterm>\r
 734                 to generate the JavaScript classes directly from the IDL XML file.</simpara>\r
 735                 <formalpara><title>Accessing classes defined in the IDL via Fieldmapper</title><para>\r
 736                 <programlisting language="html" linenumbering="unnumbered"></programlisting>\r
 737                 </para></formalpara>\r
 738                 <calloutlist>\r
 739                 <callout arearefs="">\r
 740                 <simpara>\r
 741                 Load the Dojo core.\r
 742                 </simpara>\r
 743                 </callout>\r
 744                 <callout arearefs="">\r
 745                 <simpara>\r
 746                 <literal>fieldmapper.AutoIDL</literal> reads <filename>/openils/var/reports/fm_IDL.xml</filename> to\r
 747                 generate a list of class properties.\r
 748                 </simpara>\r
 749                 </callout>\r
 750                 <callout arearefs="">\r
 751                 <simpara>\r
 752                 <literal>fieldmapper.dojoData</literal> seems to provide a store for Evergreen data\r
 753                 accessed via Dojo.\r
 754                 </simpara>\r
 755                 </callout>\r
 756                 <callout arearefs="">\r
 757                 <simpara>\r
 758                 <literal>fieldmapper.Fieldmapper</literal> converts the list of class properties into\r
 759                 actual classes.\r
 760                 </simpara>\r
 761                 </callout>\r
 762                 <callout arearefs="">\r
 763                 <simpara>\r
 764                 <literal>fieldmapper.standardRequest</literal> invokes an OpenSRF method and returns\r
 765                 an array of objects.\r
 766                 </simpara>\r
 767                 </callout>\r
 768                 <callout arearefs="">\r
 769                 <simpara>\r
 770                 The first argument to <literal>fieldmapper.standardRequest</literal> is an array\r
 771                 containing the OpenSRF service name and method name.\r
 772                 </simpara>\r
 773                 </callout>\r
 774                 <callout arearefs="">\r
 775                 <simpara>\r
 776                 The second argument to <literal>fieldmapper.standardRequest</literal> is an array\r
 777                 containing the arguments to pass to the OpenSRF method.\r
 778                 </simpara>\r
 779                 </callout>\r
 780                 <callout arearefs="">\r
 781                 <simpara>\r
 782                 As Fieldmapper has instantiated the returned objects based on their\r
 783                 class hints, we can invoke getter/setter methods on the objects.\r
 784                 </simpara>\r
 785                 </callout>\r
 786                 </calloutlist>\r
 787         </section>\r
 788         \r
 789 </chapter>\r