]> git.evergreen-ils.org Git - working/Evergreen.git/blob - 1.6/development/OpenSRF_intro.xml
7567d07bad8508739d490e70fcc9860c55d26614
[working/Evergreen.git] / 1.6 / development / OpenSRF_intro.xml
1 <?xml version="1.0" encoding="UTF-8"?>\r
2 <chapter xml:id="opensrf" xmlns="http://docbook.org/ns/docbook" version="5.0" xml:lang="EN"\r
3     xmlns:xi="http://www.w3.org/2001/XInclude" xmlns:xlink="http://www.w3.org/1999/xlink">\r
4         <chapterinfo>\r
5                 <title>OpenSRF</title>\r
6         </chapterinfo>\r
7         <abstract id="_abstract">\r
8                 <simpara>One of the claimed advantages of\r
9                 Evergreen over alternative integrated library systems is the underlying Open\r
10                 Service Request Framework (OpenSRF, pronounced "open surf") architecture. This\r
11                 article introduces OpenSRF, demonstrates how to build OpenSRF services through\r
12                 simple code examples, and explains the technical foundations on which OpenSRF\r
13                 is built.</simpara>\r
14         </abstract>\r
15         <section id="_introducing_opensrf">\r
16                 <title>Introducing OpenSRF</title>\r
17                 <simpara>OpenSRF is a message routing network that offers scalability and failover\r
18                 support for individual services and entire servers with minimal development and\r
19                 deployment overhead. You can use OpenSRF to build loosely-coupled applications\r
20                 that can be deployed on a single server or on clusters of geographically\r
21                 distributed servers using the same code and minimal configuration changes.\r
22                 Although copyright statements on some of the OpenSRF code date back to Mike\r
23                 Rylander&#8217;s original explorations in 2000, Evergreen was the first major\r
24                 application to be developed with, and to take full advantage of, the OpenSRF\r
25                 architecture starting in 2004. The first official release of OpenSRF was 0.1 in\r
26                 February 2005 (<ulink url="http://evergreen-ils.org/blog/?p=21">http://evergreen-ils.org/blog/?p=21</ulink>), but OpenSRF&#8217;s development\r
27                 continues a steady pace of enhancement and refinement, with the release of\r
28                 1.0.0 in October 2008 and the most recent release of 1.2.2 in February 2010.</simpara>\r
29                 <simpara>OpenSRF is a distinct break from the architectural approach used by previous\r
30                 library systems and has more in common with modern Web applications. The\r
31                 traditional "scale-up" approach to serve more transactions is to purchase a\r
32                 server with more CPUs and more RAM, possibly splitting the load between a Web\r
33                 server, a database server, and a business logic server. Evergreen, however, is\r
34                 built on the Open Service Request Framework (OpenSRF) architecture, which\r
35                 firmly embraces the "scale-out" approach of spreading transaction load over\r
36                 cheap commodity servers. The <ulink url="http://evergreen-ils.org/blog/?p=56">initial GPLS\r
37                 PINES hardware cluster</ulink>, while certainly impressive, may have offered the\r
38                 misleading impression that Evergreen requires a lot of hardware to run.\r
39                 However, Evergreen and OpenSRF easily scale down to a single server; many\r
40                 Evergreen libraries run their entire library system on a single server, and\r
41                 most OpenSRF and Evergreen development occurs on a virtual machine running on a\r
42                 single laptop or desktop image.</simpara>\r
43                 <simpara>Another common concern is that the flexibility of OpenSRF&#8217;s distributed\r
44                 architecture makes it complex to configure and to write new applications. This\r
45                 article demonstrates that OpenSRF itself is an extremely simple architecture on\r
46                 which one can easily build applications of many kinds – not just library\r
47                 applications – and that you can use a number of different languages to call and\r
48                 implement OpenSRF methods with a minimal learning curve. With an application\r
49                 built on OpenSRF, when you identify a bottleneck in your application&#8217;s business\r
50                 logic layer, you can adjust the number of the processes serving that particular\r
51                 bottleneck on each of your servers; or if the problem is that your service is\r
52                 resource-hungry, you could add an inexpensive server to your cluster and\r
53                 dedicate it to running that resource-hungry service.</simpara>\r
54                 <simplesect id="_programming_language_support">\r
55                         <title>Programming language support</title>\r
56                         <simpara>If you need to develop an entirely new OpenSRF service, you can choose from a\r
57                         number of different languages in which to implement that service. OpenSRF\r
58                         client language bindings have been written for C, Java, JavaScript, Perl, and\r
59                         Python, and service language bindings have been written for C, Perl, and Python.\r
60                         This article uses Perl examples as a lowest common denominator programming\r
61                         language. Writing an OpenSRF binding for another language is a relatively small\r
62                         task if that language offers libraries that support the core technologies on\r
63                         which OpenSRF depends:</simpara>\r
64                         <itemizedlist>\r
65                         <listitem>\r
66                         <simpara>\r
67                         <ulink url="http://tools.ietf.org/html/rfc3920">Extensible Messaging and Presence\r
68                         Protocol</ulink> (XMPP, sometimes referred to as Jabber) - provides the base messaging\r
69                         infrastructure between OpenSRF clients and services\r
70                         </simpara>\r
71                         </listitem>\r
72                         <listitem>\r
73                         <simpara>\r
74                         <ulink url="http://json.org">JavaScript Object Notation</ulink> (JSON) - serializes the content\r
75                         of each XMPP message in a standardized and concise format\r
76                         </simpara>\r
77                         </listitem>\r
78                         <listitem>\r
79                         <simpara>\r
80                         <ulink url="http://memcached.org">memcached</ulink> - provides the caching service\r
81                         </simpara>\r
82                         </listitem>\r
83                         <listitem>\r
84                         <simpara>\r
85                         <ulink url="http://tools.ietf.org/html/rfc5424">syslog</ulink> - the standard UNIX logging\r
86                         service\r
87                         </simpara>\r
88                         </listitem>\r
89                         </itemizedlist>\r
90                         <simpara>Unfortunately, the\r
91                         <ulink url="http://evergreen-ils.org/dokuwiki/doku.php?id=osrf-devel:primer">OpenSRF\r
92                         reference documentation</ulink>, although augmented by the\r
93                         <ulink url="http://evergreen-ils.org/dokuwiki/doku.php?id=osrf-devel:terms">OpenSRF\r
94                         glossary</ulink>, blog posts like <ulink url="http://evergreen-ils.org/blog/?p=36">the description\r
95                         of OpenSRF and Jabber</ulink>, and even this article, is not a sufficient substitute\r
96                         for a complete specification on which one could implement a language binding.\r
97                         The recommended option for would-be developers of another language binding is\r
98                         to use the Python implementation as the cleanest basis for a port to another\r
99                         language.</simpara>\r
100                 </simplesect>\r
101         </section>\r
102         <section id="writing_an_opensrf_service">\r
103                 <title>Writing an OpenSRF Service</title>\r
104                 <simpara>Imagine an application architecture in which 10 lines of Perl or Python, using\r
105                 the data types native to each language, are enough to implement a method that\r
106                 can then be deployed and invoked seamlessly across hundreds of servers.  You\r
107                 have just imagined developing with OpenSRF – it is truly that simple. Under the\r
108                 covers, of course, the OpenSRF language bindings do an incredible amount of\r
109                 work on behalf of the developer. An OpenSRF application consists of one or more\r
110                 OpenSRF services that expose methods: for example, the <literal>opensrf.simple-text</literal>\r
111                 <ulink url="http://svn.open-ils.org/trac/OpenSRF/browser/trunk/src/perl/lib/OpenSRF/Application/Demo/SimpleText.pm">demonstration\r
112                 service</ulink> exposes the <literal>opensrf.simple-text.split()</literal> and\r
113                 <literal>opensrf.simple-text.reverse()</literal> methods. Each method accepts zero or more\r
114                 arguments and returns zero or one results. The data types supported by OpenSRF\r
115                 arguments and results are typical core language data types: strings, numbers,\r
116                 booleans, arrays, and hashes.</simpara>\r
117                 <simpara>To implement a new OpenSRF service, perform the following steps:</simpara>\r
118                 <orderedlist numeration="arabic">\r
119                 <listitem>\r
120                 <simpara>\r
121                 Include the base OpenSRF support libraries\r
122                 </simpara>\r
123                 </listitem>\r
124                 <listitem>\r
125                 <simpara>\r
126                 Write the code for each of your OpenSRF methods as separate procedures\r
127                 </simpara>\r
128                 </listitem>\r
129                 <listitem>\r
130                 <simpara>\r
131                 Register each method\r
132                 </simpara>\r
133                 </listitem>\r
134                 <listitem>\r
135                 <simpara>\r
136                 Add the service definition to the OpenSRF configuration files\r
137                 </simpara>\r
138                 </listitem>\r
139                 </orderedlist>\r
140                 <simpara>For example, the following code implements an OpenSRF service. The service\r
141                 includes one method named <literal>opensrf.simple-text.reverse()</literal> that accepts one\r
142                 string as input and returns the reversed version of that string:</simpara>\r
143                 <programlisting language="perl" linenumbering="unnumbered">#!/usr/bin/perl\r
144 \r
145                 package OpenSRF::Application::Demo::SimpleText;\r
146 \r
147                 use strict;\r
148 \r
149                 use OpenSRF::Application;\r
150                 use parent qw/OpenSRF::Application/;\r
151 \r
152                 sub text_reverse {\r
153                     my ($self , $conn, $text) = @_;\r
154                     my $reversed_text = scalar reverse($text);\r
155                     return $reversed_text;\r
156                 }\r
157 \r
158                 __PACKAGE__-&gt;register_method(\r
159                     method    =&gt; 'text_reverse',\r
160                     api_name  =&gt; 'opensrf.simple-text.reverse'\r
161                 );</programlisting>\r
162                 <simpara>Ten lines of code, and we have a complete OpenSRF service that exposes a single\r
163                 method and could be deployed quickly on a cluster of servers to meet your\r
164                 application&#8217;s ravenous demand for reversed strings! If you&#8217;re unfamiliar with\r
165                 Perl, the <literal>use OpenSRF::Application; use parent qw/OpenSRF::Application/;</literal>\r
166                 lines tell this package to inherit methods and properties from the\r
167                 <literal>OpenSRF::Application</literal> module. For example, the call to\r
168                 <literal>__PACKAGE__-&gt;register_method()</literal> is defined in <literal>OpenSRF::Application</literal> but due to\r
169                 inheritance is available in this package (named by the special Perl symbol\r
170                 <literal>__PACKAGE__</literal> that contains the current package name). The <literal>register_method()</literal>\r
171                 procedure is how we introduce a method to the rest of the OpenSRF world.</simpara>\r
172                 <simplesect id="serviceRegistration">\r
173                         <title>Registering a service with the OpenSRF configuration files</title>\r
174                         <simpara>Two files control most of the configuration for OpenSRF:</simpara>\r
175                         <itemizedlist>\r
176                         <listitem>\r
177                         <simpara>\r
178                         <literal>opensrf.xml</literal> contains the configuration for the service itself, as well as\r
179                         a list of which application servers in your OpenSRF cluster should start\r
180                         the service.\r
181                         </simpara>\r
182                         </listitem>\r
183                         <listitem>\r
184                         <simpara>\r
185                         <literal>opensrf_core.xml</literal> (often referred to as the "bootstrap configuration"\r
186                         file) contains the OpenSRF networking information, including the XMPP server\r
187                         connection credentials for the public and private routers. You only need to touch\r
188                         this for a new service if the new service needs to be accessible via the\r
189                         public router.\r
190                         </simpara>\r
191                         </listitem>\r
192                         </itemizedlist>\r
193                         <simpara>Begin by defining the service itself in <literal>opensrf.xml</literal>. To register the\r
194                         <literal>opensrf.simple-text</literal> service, add the following section to the <literal>&lt;apps&gt;</literal>\r
195                         element (corresponding to the XPath <literal>/opensrf/default/apps/</literal>):</simpara>\r
196                         <programlisting language="xml" linenumbering="unnumbered">&lt;apps&gt;\r
197                           &lt;opensrf.simple-text&gt; <co id="CO1-1"/> \r
198                             &lt;keepalive&gt;3&lt;/keepalive&gt;<co id="CO1-2"/> \r
199                             &lt;stateless&gt;1&lt;/stateless&gt;<co id="CO1-3"/>\r
200                             &lt;language&gt;perl&lt;/language&gt;<co id="CO1-4"/> \r
201                             &lt;implementation&gt;OpenSRF::Application::Demo::SimpleText&lt;/implementation&gt;<co id="CO1-5"/> \r
202                             &lt;max_requests&gt;100&lt;/max_requests&gt;<co id="CO1-6"/> \r
203                             &lt;unix_config&gt;\r
204                               &lt;max_requests&gt;1000&lt;/max_requests&gt; <co id="CO1-7"/> \r
205                               &lt;unix_log&gt;opensrf.simple-text_unix.log&lt;/unix_log&gt; <co id="CO1-8"/> \r
206                               &lt;unix_sock&gt;opensrf.simple-text_unix.sock&lt;/unix_sock&gt;<co id="CO1-9"/> \r
207                               &lt;unix_pid&gt;opensrf.simple-text_unix.pid&lt;/unix_pid&gt; <co id="CO1-10"/> \r
208                               &lt;min_children&gt;5&lt;/min_children&gt;  <co id="CO1-11"/> \r
209                               &lt;max_children&gt;15&lt;/max_children&gt;<co id="CO1-12"/> \r
210                               &lt;min_spare_children&gt;2&lt;/min_spare_children&gt;<co id="CO1-13"/> \r
211                               &lt;max_spare_children&gt;5&lt;/max_spare_children&gt; <co id="CO1-14"/> \r
212                             &lt;/unix_config&gt;\r
213                           &lt;/opensrf.simple-text&gt;\r
214 \r
215                           &lt;!-- other OpenSRF services registered here... --&gt;\r
216                         &lt;/apps&gt;</programlisting>\r
217                         <calloutlist>\r
218                         <callout arearefs="CO1-1">\r
219                         <simpara>\r
220                         The element name is the name that the OpenSRF control scripts use to refer\r
221                         to the service.\r
222                         </simpara>\r
223                         </callout>\r
224                         <callout arearefs="CO1-2">\r
225                         <simpara>\r
226                         The <literal>&lt;keepalive&gt;</literal> element specifies the interval (in seconds) between\r
227                         checks to determine if the service is still running.\r
228                         </simpara>\r
229                         </callout>\r
230                         <callout arearefs="CO1-3">\r
231                         <simpara>\r
232                         The <literal>&lt;stateless&gt;</literal> element specifies whether OpenSRF clients can call\r
233                         methods from this service without first having to create a connection to a\r
234                         specific service backend process for that service. If the value is <literal>1</literal>, then\r
235                         the client can simply issue a request and the router will forward the request\r
236                         to an available service and the result will be returned directly to the client.\r
237                         </simpara>\r
238                         </callout>\r
239                         <callout arearefs="CO1-4">\r
240                         <simpara>\r
241                         The <literal>&lt;language&gt;</literal> element specifies the programming language in which the\r
242                         service is implemented.\r
243                         </simpara>\r
244                         </callout>\r
245                         <callout arearefs="CO1-5">\r
246                         <simpara>\r
247                         The <literal>&lt;implementation&gt;</literal> element pecifies the name of the library or module\r
248                         in which the service is implemented.\r
249                         </simpara>\r
250                         </callout>\r
251                         <callout arearefs="CO1-6">\r
252                         <simpara>\r
253                         (C implementations only): The <literal>&lt;max_requests&gt;</literal> element, as a direct child\r
254                         of the service element name, specifies the maximum number of requests a process\r
255                         serves before it is killed and replaced by a new process.\r
256                         </simpara>\r
257                         </callout>\r
258                         <callout arearefs="CO1-7">\r
259                         <simpara>\r
260                         (Perl implementations only): The <literal>&lt;max_requests&gt;</literal> element, as a direct\r
261                         child of the <literal>&lt;unix_config&gt;</literal> element, specifies the maximum number of requests\r
262                         a process serves before it is killed and replaced by a new process.\r
263                         </simpara>\r
264                         </callout>\r
265                         <callout arearefs="CO1-8">\r
266                         <simpara>\r
267                         The <literal>&lt;unix_log&gt;</literal> element specifies the name of the log file for\r
268                         language-specific log messages such as syntax warnings.\r
269                         </simpara>\r
270                         </callout>\r
271                         <callout arearefs="CO1-9">\r
272                         <simpara>\r
273                         The <literal>&lt;unix_sock&gt;</literal> element specifies the name of the UNIX socket used for\r
274                         inter-process communications.\r
275                         </simpara>\r
276                         </callout>\r
277                         <callout arearefs="CO1-10">\r
278                         <simpara>\r
279                         The <literal>&lt;unix_pid&gt;</literal> element specifies the name of the PID file for the\r
280                         master process for the service.\r
281                         </simpara>\r
282                         </callout>\r
283                         <callout arearefs="CO1-11">\r
284                         <simpara>\r
285                         The <literal>&lt;min_children&gt;</literal> element specifies the minimum number of child\r
286                         processes that should be running at any given time.\r
287                         </simpara>\r
288                         </callout>\r
289                         <callout arearefs="CO1-12">\r
290                         <simpara>\r
291                         The <literal>&lt;max_children&gt;</literal> element specifies the maximum number of child\r
292                         processes that should be running at any given time.\r
293                         </simpara>\r
294                         </callout>\r
295                         <callout arearefs="CO1-13">\r
296                         <simpara>\r
297                         The <literal>&lt;min_spare_children&gt;</literal> element specifies the minimum number of idle\r
298                         child processes that should be available to handle incoming requests.  If there\r
299                         are fewer than this number of spare child processes, new processes will be\r
300                         spawned.\r
301                         </simpara>\r
302                         </callout>\r
303                         <callout arearefs="CO1-14">\r
304                         <simpara>\r
305                         The`&lt;max_spare_children&gt;` element specifies the maximum number of idle\r
306                         child processes that should be available to handle incoming requests. If there\r
307                         are more than this number of spare child processes, the extra processes will be\r
308                         killed.\r
309                         </simpara>\r
310                         </callout>\r
311                         </calloutlist>\r
312                         <simpara>To make the service accessible via the public router, you must also\r
313                         edit the <literal>opensrf_core.xml</literal> configuration file to add the service to the list\r
314                         of publicly accessible services:</simpara>\r
315                         <formalpara><title>Making a service publicly accessible in <literal>opensrf_core.xml</literal></title><para>\r
316                         <programlisting language="xml" linenumbering="unnumbered">&lt;router&gt;<co id="CO2-1"/> \r
317                             &lt;!-- This is the public router. On this router, we only register applications\r
318                              which should be accessible to everyone on the opensrf network --&gt;\r
319                             &lt;name&gt;router&lt;/name&gt;\r
320                             &lt;domain&gt;public.localhost&lt;/domain&gt;<co id="CO2-2"/>\r
321                             &lt;services&gt;\r
322                                 &lt;service&gt;opensrf.math&lt;/service&gt;\r
323                                 &lt;service&gt;opensrf.simple-text&lt;/service&gt; <co id="CO2-3"/> \r
324                             &lt;/services&gt;\r
325                         &lt;/router&gt;</programlisting>\r
326                         </para></formalpara>\r
327                         <calloutlist>\r
328                         <callout arearefs="CO2-1">\r
329                         <simpara>\r
330                         This section of the <literal>opensrf_core.xml</literal> file is located at XPath\r
331                         <literal>/config/opensrf/routers/</literal>.\r
332                         </simpara>\r
333                         </callout>\r
334                         <callout arearefs="CO2-2">\r
335                         <simpara>\r
336                         <literal>public.localhost</literal> is the canonical public router domain in the OpenSRF\r
337                         installation instructions.\r
338                         </simpara>\r
339                         </callout>\r
340                         <callout arearefs="CO2-3">\r
341                         <simpara>\r
342                         Each <literal>&lt;service&gt;</literal> element contained in the <literal>&lt;services&gt;</literal> element\r
343                         offers their services via the public router as well as the private router.\r
344                         </simpara>\r
345                         </callout>\r
346                         </calloutlist>\r
347                         <simpara>Once you have defined the new service, you must restart the OpenSRF Router\r
348                         to retrieve the new configuration and start or restart the service itself.</simpara>\r
349                         <simpara>Complete working examples of the <link linkend="opensrf-core-xml">opensrf_core.xml</link> and\r
350                         <link linkend="opensrf-xml">opensrf.xml</link> configuration files are included with this article\r
351                         for your reference.</simpara>\r
352                 </simplesect>\r
353                 <simplesect id="_calling_an_opensrf_method">\r
354                         <title>Calling an OpenSRF method</title>\r
355                         <simpara>OpenSRF clients in any supported language can invoke OpenSRF services in any\r
356                         supported language. So let&#8217;s see a few examples of how we can call our fancy\r
357                         new <literal>opensrf.simple-text.reverse()</literal> method:</simpara>\r
358                         <simplesect id="_calling_opensrf_methods_from_the_srfsh_client">\r
359                                 <title>Calling OpenSRF methods from the srfsh client</title>\r
360                                 <simpara><literal>srfsh</literal> is a command-line tool installed with OpenSRF that you can use to call\r
361                                 OpenSRF methods. To call an OpenSRF method, issue the <literal>request</literal> command and\r
362                                 pass the OpenSRF service and method name as the first two arguments; then pass\r
363                                 one or more JSON objects delimited by commas as the arguments to the method\r
364                                 being invoked.</simpara>\r
365                                 <simpara>The following example calls the <literal>opensrf.simple-text.reverse</literal> method of the\r
366                                 <literal>opensrf.simple-text</literal> OpenSRF service, passing the string <literal>"foobar"</literal> as the\r
367                                 only method argument:</simpara>\r
368                                 <programlisting language="sh" linenumbering="unnumbered">$ srfsh\r
369                                 srfsh # request opensrf.simple-text opensrf.simple-text.reverse "foobar"\r
370 \r
371                                 Received Data: "raboof"\r
372 \r
373                                 =------------------------------------\r
374                                 Request Completed Successfully\r
375                                 Request Time in seconds: 0.016718\r
376                                 =------------------------------------</programlisting>\r
377                         </simplesect>\r
378                         <simplesect id="opensrfIntrospection">\r
379                                 <title>Getting documentation for OpenSRF methods from the srfsh client</title>\r
380                                 <simpara>The <literal>srfsh</literal> client also gives you command-line access to retrieving metadata\r
381                                 about OpenSRF services and methods. For a given OpenSRF method, for example,\r
382                                 you can retrieve information such as the minimum number of required arguments,\r
383                                 the data type and a description of each argument, the package or library in\r
384                                 which the method is implemented, and a description of the method. To retrieve\r
385                                 the documentation for an opensrf method from <literal>srfsh</literal>, issue the <literal>introspect</literal>\r
386                                 command, followed by the name of the OpenSRF service and (optionally) the\r
387                                 name of the OpenSRF method. If you do not pass a method name to the <literal>introspect</literal>\r
388                                 command, <literal>srfsh</literal> lists all of the methods offered by the service. If you pass\r
389                                 a partial method name, <literal>srfsh</literal> lists all of the methods that match that portion\r
390                                 of the method name.</simpara>\r
391                                 <note><simpara>The quality and availability of the descriptive information for each\r
392                                 method depends on the developer to register the method with complete and\r
393                                 accurate information. The quality varies across the set of OpenSRF and\r
394                                 Evergreen APIs, although some effort is being put towards improving the\r
395                                 state of the internal documentation.</simpara></note>\r
396                                 <programlisting language="sh" linenumbering="unnumbered">srfsh# introspect opensrf.simple-text "opensrf.simple-text.reverse"\r
397                                 --&gt; opensrf.simple-text\r
398 \r
399                                 Received Data: {\r
400                                   "__c":"opensrf.simple-text",\r
401                                   "__p":{\r
402                                     "api_level":1,\r
403                                     "stream":0,      <co id="CO3-1"/>\r
404                                     "object_hint":"OpenSRF_Application_Demo_SimpleText",\r
405                                     "remote":0,\r
406                                     "package":"OpenSRF::Application::Demo::SimpleText", <co id="CO3-2"/>\r
407                                     "api_name":"opensrf.simple-text.reverse",<co id="CO3-3"/>\r
408                                     "server_class":"opensrf.simple-text",\r
409                                     "signature":{ <co id="CO3-4"/>\r
410                                       "params":[  <co id="CO3-5"/>\r
411                                         {\r
412                                           "desc":"The string to reverse",\r
413                                           "name":"text",\r
414                                           "type":"string"\r
415                                         }\r
416                                       ],\r
417                                       "desc":"Returns the input string in reverse order\n", <co id="CO3-6"/>\r
418                                       "return":{                                            <co id="CO3-7"/>\r
419                                         "desc":"Returns the input string in reverse order",\r
420                                         "type":"string"\r
421                                       }\r
422                                     },\r
423                                     "method":"text_reverse",  <co id="CO3-8"/>\r
424                                     "argc":1 <co id="CO3-9"/>\r
425                                   }\r
426                                 }</programlisting>\r
427                                 <calloutlist>\r
428                                 <callout arearefs="CO3-1">\r
429                                 <simpara>\r
430                                 <literal>stream</literal> denotes whether the method supports streaming responses or not.\r
431                                 </simpara>\r
432                                 </callout>\r
433                                 <callout arearefs="CO3-2">\r
434                                 <simpara>\r
435                                 <literal>package</literal> identifies which package or library implements the method.\r
436                                 </simpara>\r
437                                 </callout>\r
438                                 <callout arearefs="CO3-3">\r
439                                 <simpara>\r
440                                 <literal>api_name</literal> identifies the name of the OpenSRF method.\r
441                                 </simpara>\r
442                                 </callout>\r
443                                 <callout arearefs="CO3-4">\r
444                                 <simpara>\r
445                                 <literal>signature</literal> is a hash that describes the parameters for the method.\r
446                                 </simpara>\r
447                                 </callout>\r
448                                 <callout arearefs="CO3-5">\r
449                                 <simpara>\r
450                                 <literal>params</literal> is an array of hashes describing each parameter in the method;\r
451                                 each parameter has a description (<literal>desc</literal>), name (<literal>name</literal>), and type (<literal>type</literal>).\r
452                                 </simpara>\r
453                                 </callout>\r
454                                 <callout arearefs="CO3-6">\r
455                                 <simpara>\r
456                                 <literal>desc</literal> is a string that describes the method itself.\r
457                                 </simpara>\r
458                                 </callout>\r
459                                 <callout arearefs="CO3-7">\r
460                                 <simpara>\r
461                                 <literal>return</literal> is a hash that describes the return value for the method; it\r
462                                 contains a description of the return value (<literal>desc</literal>) and the type of the\r
463                                 returned value (<literal>type</literal>).\r
464                                 </simpara>\r
465                                 </callout>\r
466                                 <callout arearefs="CO3-8">\r
467                                 <simpara>\r
468                                 <literal>method</literal> identifies the name of the function or method in the source\r
469                                 implementation.\r
470                                 </simpara>\r
471                                 </callout>\r
472                                 <callout arearefs="CO3-9">\r
473                                 <simpara>\r
474                                 <literal>argc</literal> is an integer describing the minimum number of arguments that\r
475                                 must be passed to this method.\r
476                                 </simpara>\r
477                                 </callout>\r
478                                 </calloutlist>\r
479                         </simplesect>\r
480                         <simplesect id="_calling_opensrf_methods_from_perl_applications">\r
481                                 <title>Calling OpenSRF methods from Perl applications</title>\r
482                                 <simpara>To call an OpenSRF method from Perl, you must connect to the OpenSRF service,\r
483                                 issue the request to the method, and then retrieve the results.</simpara>\r
484                                 <programlisting language="perl" linenumbering="unnumbered">#/usr/bin/perl\r
485                                 use strict;\r
486                                 use OpenSRF::AppSession;\r
487                                 use OpenSRF::System;\r
488 \r
489                                 OpenSRF::System-&gt;bootstrap_client(config_file =&gt; '/openils/conf/opensrf_core.xml');<co id="CO4-1"/>\r
490 \r
491                                 my $session = OpenSRF::AppSession-&gt;create("opensrf.simple-text");<co id="CO4-2"/>\r
492 \r
493                                 print "substring: Accepts a string and a number as input, returns a string\n";\r
494                                 my $result = $session-&gt;request("opensrf.simple-text.substring", "foobar", 3);<co id="CO4-3"/>\r
495                                 my $request = $result-&gt;gather(); <co id="CO4-4"/>\r
496                                 print "Substring: $request\n\n";\r
497 \r
498                                 print "split: Accepts two strings as input, returns an array of strings\n";\r
499                                 $request = $session-&gt;request("opensrf.simple-text.split", "This is a test", " ");<co id="CO4-5"/>\r
500                                 my $output = "Split: [";\r
501                                 my $element;\r
502                                 while ($element = $request-&gt;recv()) {   <co id="CO4-6"/>\r
503                                     $output .= $element-&gt;content . ", ";  <co id="CO4-7"/>\r
504                                 }\r
505                                 $output =~ s/, $/]/;\r
506                                 print $output . "\n\n";\r
507 \r
508                                 print "statistics: Accepts an array of strings as input, returns a hash\n";\r
509                                 my @many_strings = [\r
510                                     "First I think I'll have breakfast",\r
511                                     "Then I think that lunch would be nice",\r
512                                     "And then seventy desserts to finish off the day"\r
513                                 ];\r
514 \r
515                                 $result = $session-&gt;request("opensrf.simple-text.statistics", @many_strings); <co id="CO4-8"/>\r
516                                 $request = $result-&gt;gather();    <co id="CO4-9"/>\r
517                                 print "Length: " . $result-&gt;{'length'} . "\n";\r
518                                 print "Word count: " . $result-&gt;{'word_count'} . "\n";\r
519 \r
520                                 $session-&gt;disconnect();       <co id="CO4-10"/></programlisting>\r
521                                 <calloutlist>\r
522                                 <callout arearefs="CO4-1">\r
523                                 <simpara>\r
524                                 The <literal>OpenSRF::System-&gt;bootstrap_client()</literal> method reads the OpenSRF\r
525                                 configuration information from the indicated file and creates an XMPP client\r
526                                 connection based on that information.\r
527                                 </simpara>\r
528                                 </callout>\r
529                                 <callout arearefs="CO4-2">\r
530                                 <simpara>\r
531                                 The <literal>OpenSRF::AppSession-&gt;create()</literal> method accepts one argument - the name\r
532                                 of the OpenSRF service to which you want to want to make one or more requests -\r
533                                 and returns an object prepared to use the client connection to make those\r
534                                 requests.\r
535                                 </simpara>\r
536                                 </callout>\r
537                                 <callout arearefs="CO4-3">\r
538                                 <simpara>\r
539                                 The <literal>OpenSRF::AppSession-&gt;request()</literal> method accepts a minimum of one\r
540                                 argument - the name of the OpenSRF method to which you want to make a request -\r
541                                 followed by zero or more arguments to pass to the OpenSRF method as input\r
542                                 values. This example passes a string and an integer to the\r
543                                 <literal>opensrf.simple-text.substring</literal> method defined by the <literal>opensrf.simple-text</literal>\r
544                                 OpenSRF service.\r
545                                 </simpara>\r
546                                 </callout>\r
547                                 <callout arearefs="CO4-4">\r
548                                 <simpara>\r
549                                 The <literal>gather()</literal> method, called on the result object returned by the\r
550                                 <literal>request()</literal> method, iterates over all of the possible results from the result\r
551                                 object and returns a single variable.\r
552                                 </simpara>\r
553                                 </callout>\r
554                                 <callout arearefs="CO4-5">\r
555                                 <simpara>\r
556                                 This <literal>request()</literal> call passes two strings to the <literal>opensrf.simple-text.split</literal>\r
557                                 method defined by the <literal>opensrf.simple-text</literal> OpenSRF service and returns (via\r
558                                 <literal>gather()</literal>) a reference to an array of results.\r
559                                 </simpara>\r
560                                 </callout>\r
561                                 <callout arearefs="CO4-6">\r
562                                 <simpara>\r
563                                 The <literal>opensrf.simple-text.split()</literal> method is a streaming method that\r
564                                 returns an array of results with one element per <literal>recv()</literal> call on the\r
565                                 result object. We could use the <literal>gather()</literal> method to retrieve all of the\r
566                                 results in a single array reference, but instead we simply iterate over\r
567                                 the result variable until there are no more results to retrieve.\r
568                                 </simpara>\r
569                                 </callout>\r
570                                 <callout arearefs="CO4-7">\r
571                                 <simpara>\r
572                                 While the <literal>gather()</literal> convenience method returns only the content of the\r
573                                 complete set of results for a given request, the <literal>recv()</literal> method returns an\r
574                                 OpenSRF result object with <literal>status</literal>, <literal>statusCode</literal>, and <literal>content</literal> fields as\r
575                                 we saw in <link linkend="OpenSRFOverHTTP">the HTTP results example</link>.\r
576                                 </simpara>\r
577                                 </callout>\r
578                                 <callout arearefs="CO4-8">\r
579                                 <simpara>\r
580                                 This <literal>request()</literal> call passes an array to the\r
581                                 <literal>opensrf.simple-text.statistics</literal> method defined by the <literal>opensrf.simple-text</literal>\r
582                                 OpenSRF service.\r
583                                 </simpara>\r
584                                 </callout>\r
585                                 <callout arearefs="CO4-9">\r
586                                 <simpara>\r
587                                 The result object returns a hash reference via <literal>gather()</literal>. The hash\r
588                                 contains the <literal>length</literal> and <literal>word_count</literal> keys we defined in the method.\r
589                                 </simpara>\r
590                                 </callout>\r
591                                 <callout arearefs="CO4-10">\r
592                                 <simpara>\r
593                                 The <literal>OpenSRF::AppSession-&gt;disconnect()</literal> method closes the XMPP client\r
594                                 connection and cleans up resources associated with the session.\r
595                                 </simpara>\r
596                                 </callout>\r
597                                 </calloutlist>\r
598                         </simplesect>\r
599                 </simplesect>\r
600                 <simplesect id="_accepting_and_returning_more_interesting_data_types">\r
601                         <title>Accepting and returning more interesting data types</title>\r
602                         <simpara>Of course, the example of accepting a single string and returning a single\r
603                         string is not very interesting. In real life, our applications tend to pass\r
604                         around multiple arguments, including arrays and hashes. Fortunately, OpenSRF\r
605                         makes that easy to deal with; in Perl, for example, returning a reference to\r
606                         the data type does the right thing. In the following example of a method that\r
607                         returns a list, we accept two arguments of type string: the string to be split,\r
608                         and the delimiter that should be used to split the string.</simpara>\r
609                         <formalpara><title>Basic text splitting method</title><para>\r
610                         <programlisting language="perl" linenumbering="unnumbered">sub text_split {\r
611                             my $self = shift;\r
612                             my $conn = shift;\r
613                             my $text = shift;\r
614                             my $delimiter = shift || ' ';\r
615 \r
616                             my @split_text = split $delimiter, $text;\r
617                             return \@split_text;\r
618                         }\r
619 \r
620                         __PACKAGE__-&gt;register_method(\r
621                             method    =&gt; 'text_split',\r
622                             api_name  =&gt; 'opensrf.simple-text.split'\r
623                         );</programlisting>\r
624                         </para></formalpara>\r
625                         <simpara>We simply return a reference to the list, and OpenSRF does the rest of the work\r
626                         for us to convert the data into the language-independent format that is then\r
627                         returned to the caller. As a caller of a given method, you must rely on the\r
628                         documentation used to register to determine the data structures - if the developer has\r
629                         added the appropriate documentation.</simpara>\r
630                 </simplesect>\r
631                 <simplesect id="_accepting_and_returning_evergreen_objects">\r
632                         <title>Accepting and returning Evergreen objects</title>\r
633                         <simpara>OpenSRF is agnostic about objects; its role is to pass JSON back and forth\r
634                         between OpenSRF clients and services, and it allows the specific clients and\r
635                         services to define their own semantics for the JSON structures. On top of that\r
636                         infrastructure, Evergreen offers the fieldmapper: an object-relational mapper\r
637                         that provides a complete definition of all objects, their properties, their\r
638                         relationships to other objects, the permissions required to create, read,\r
639                         update, or delete objects of that type, and the database table or view on which\r
640                         they are based.</simpara>\r
641                         <simpara>The Evergreen fieldmapper offers a great deal of convenience for working with\r
642                         complex system objects beyond the basic mapping of classes to database\r
643                         schemas. Although the result is passed over the wire as a JSON object\r
644                         containing the indicated fields, fieldmapper-aware clients then turn those\r
645                         JSON objects into native objects with setter / getter methods for each field.</simpara>\r
646                         <simpara>All of this metadata about Evergreen objects is defined in the\r
647                         fieldmapper configuration file (<literal>/openils/conf/fm_IDL.xml</literal>), and access to\r
648                         these classes is provided by the <literal>open-ils.cstore</literal>, <literal>open-ils.pcrud</literal>, and\r
649                         <literal>open-ils.reporter-store</literal> OpenSRF services which parse the fieldmapper\r
650                         configuration file and dynamically register OpenSRF methods for creating,\r
651                         reading, updating, and deleting all of the defined classes.</simpara>\r
652                         <formalpara><title>Example fieldmapper class definition for "Open User Summary"</title><para>\r
653                         <programlisting language="xml" linenumbering="unnumbered">&lt;class id="mous" controller="open-ils.cstore open-ils.pcrud"\r
654                          oils_obj:fieldmapper="money::open_user_summary"\r
655                          oils_persist:tablename="money.open_usr_summary"\r
656                          reporter:label="Open User Summary"&gt;                                <co id="CO5-1"/>\r
657                             &lt;fields oils_persist:primary="usr" oils_persist:sequence=""&gt; <co id="CO5-2"/> \r
658                                 &lt;field name="balance_owed" reporter:datatype="money" /&gt;  <co id="CO5-3"/> \r
659                                 &lt;field name="total_owed" reporter:datatype="money" /&gt;\r
660                                 &lt;field name="total_paid" reporter:datatype="money" /&gt;\r
661                                 &lt;field name="usr" reporter:datatype="link"/&gt;\r
662                             &lt;/fields&gt;\r
663                             &lt;links&gt;\r
664                                 &lt;link field="usr" reltype="has_a" key="id" map="" class="au"/&gt;<co id="CO5-4"/> \r
665                             &lt;/links&gt;\r
666                             &lt;permacrud xmlns="http://open-ils.org/spec/opensrf/IDL/permacrud/v1"&gt;<co id="CO5-5"/> \r
667                                 &lt;actions&gt;\r
668                                     &lt;retrieve permission="VIEW_USER"&gt;<co id="CO5-6"/> \r
669                                         &lt;context link="usr" field="home_ou"/&gt;<co id="CO5-7"/>\r
670                                     &lt;/retrieve&gt;\r
671                                 &lt;/actions&gt;\r
672                             &lt;/permacrud&gt;\r
673                         &lt;/class&gt;</programlisting>\r
674                         </para></formalpara>\r
675                         <calloutlist>\r
676                         <callout arearefs="CO5-1">\r
677                         <simpara>\r
678                         The <literal>&lt;class&gt;</literal> element defines the class:\r
679                         </simpara>\r
680                         <itemizedlist>\r
681                         <listitem>\r
682                         <simpara>\r
683                         The <literal>id</literal> attribute defines the <emphasis>class hint</emphasis> that identifies the class both\r
684                         elsewhere in the fieldmapper configuration file, such as in the value of the\r
685                         <literal>field</literal> attribute of the <literal>&lt;link&gt;</literal> element, and in the JSON object itself when\r
686                         it is instantiated. For example, an "Open User Summary" JSON object would have\r
687                         the top level property of <literal>"__c":"mous"</literal>.\r
688                         </simpara>\r
689                         </listitem>\r
690                         <listitem>\r
691                         <simpara>\r
692                         The <literal>controller</literal> attribute identifies the services that have direct access\r
693                         to this class. If <literal>open-ils.pcrud</literal> is not listed, for example, then there is\r
694                         no means to directly access members of this class through a public service.\r
695                         </simpara>\r
696                         </listitem>\r
697                         <listitem>\r
698                         <simpara>\r
699                         The <literal>oils_obj:fieldmapper</literal> attribute defines the name of the Perl\r
700                         fieldmapper class that will be dynamically generated to provide setter and\r
701                         getter methods for instances of the class.\r
702                         </simpara>\r
703                         </listitem>\r
704                         <listitem>\r
705                         <simpara>\r
706                         The <literal>oils_persist:tablename</literal> attribute identifies the schema name and table\r
707                         name of the database table that stores the data that represents the instances\r
708                         of this class. In this case, the schema is <literal>money</literal> and the table is\r
709                         <literal>open_usr_summary</literal>.\r
710                         </simpara>\r
711                         </listitem>\r
712                         <listitem>\r
713                         <simpara>\r
714                         The <literal>reporter:label</literal> attribute defines a human-readable name for the class\r
715                         used in the reporting interface to identify the class. These names are defined\r
716                         in English in the fieldmapper configuration file; however, they are extracted\r
717                         so that they can be translated and served in the user&#8217;s language of choice.\r
718                         </simpara>\r
719                         </listitem>\r
720                         </itemizedlist>\r
721                         </callout>\r
722                         <callout arearefs="CO5-2">\r
723                         <simpara>\r
724                         The <literal>&lt;fields&gt;</literal> element lists all of the fields that belong to the object.\r
725                         </simpara>\r
726                         <itemizedlist>\r
727                         <listitem>\r
728                         <simpara>\r
729                         The <literal>oils_persist:primary</literal> attribute identifies the field that acts as the\r
730                         primary key for the object; in this case, the field with the name <literal>usr</literal>.\r
731                         </simpara>\r
732                         </listitem>\r
733                         <listitem>\r
734                         <simpara>\r
735                         The <literal>oils_persist:sequence</literal> attribute identifies the sequence object\r
736                         (if any) in this database provides values for new instances of this class. In\r
737                         this case, the primary key is defined by a field that is linked to a different\r
738                         table, so no sequence is used to populate these instances.\r
739                         </simpara>\r
740                         </listitem>\r
741                         </itemizedlist>\r
742                         </callout>\r
743                         <callout arearefs="CO5-3">\r
744                         <simpara>\r
745                         Each <literal>&lt;field&gt;</literal> element defines a single field with the following attributes:\r
746                         </simpara>\r
747                         <itemizedlist>\r
748                         <listitem>\r
749                         <simpara>\r
750                         The <literal>name</literal> attribute identifies the column name of the field in the\r
751                         underlying database table as well as providing a name for the setter / getter\r
752                         method that can be invoked in the JSON or native version of the object.\r
753                         </simpara>\r
754                         </listitem>\r
755                         <listitem>\r
756                         <simpara>\r
757                         The <literal>reporter:datatype</literal> attribute defines how the reporter should treat\r
758                         the contents of the field for the purposes of querying and display.\r
759                         </simpara>\r
760                         </listitem>\r
761                         <listitem>\r
762                         <simpara>\r
763                         The <literal>reporter:label</literal> attribute can be used to provide a human-readable name\r
764                         for each field; without it, the reporter falls back to the value of the <literal>name</literal>\r
765                         attribute.\r
766                         </simpara>\r
767                         </listitem>\r
768                         </itemizedlist>\r
769                         </callout>\r
770                         <callout arearefs="CO5-4">\r
771                         <simpara>\r
772                         The <literal>&lt;links&gt;</literal> element contains a set of zero or more <literal>&lt;link&gt;</literal> elements,\r
773                         each of which defines a relationship between the class being described and\r
774                         another class.\r
775                         </simpara>\r
776                         <itemizedlist>\r
777                         <listitem>\r
778                         <simpara>\r
779                         The <literal>field</literal> attribute identifies the field named in this class that links\r
780                         to the external class.\r
781                         </simpara>\r
782                         </listitem>\r
783                         <listitem>\r
784                         <simpara>\r
785                         The <literal>reltype</literal> attribute identifies the kind of relationship between the\r
786                         classes; in the case of <literal>has_a</literal>, each value in the <literal>usr</literal> field is guaranteed\r
787                         to have a corresponding value in the external class.\r
788                         </simpara>\r
789                         </listitem>\r
790                         <listitem>\r
791                         <simpara>\r
792                         The <literal>key</literal> attribute identifies the name of the field in the external\r
793                         class to which this field links.\r
794                         </simpara>\r
795                         </listitem>\r
796                         <listitem>\r
797                         <simpara>\r
798                         The rarely-used <literal>map</literal> attribute identifies a second class to which\r
799                         the external class links; it enables this field to define a direct\r
800                         relationship to an external class with one degree of separation, to\r
801                         avoid having to retrieve all of the linked members of an intermediate\r
802                         class just to retrieve the instances from the actual desired target class.\r
803                         </simpara>\r
804                         </listitem>\r
805                         <listitem>\r
806                         <simpara>\r
807                         The <literal>class</literal> attribute identifies the external class to which this field\r
808                         links.\r
809                         </simpara>\r
810                         </listitem>\r
811                         </itemizedlist>\r
812                         </callout>\r
813                         <callout arearefs="CO5-5">\r
814                         <simpara>\r
815                         The <literal>&lt;permacrud&gt;</literal> element defines the permissions that must have been\r
816                         granted to a user to operate on instances of this class.\r
817                         </simpara>\r
818                         </callout>\r
819                         <callout arearefs="CO5-6">\r
820                         <simpara>\r
821                         The <literal>&lt;retrieve&gt;</literal> element is one of four possible children of the\r
822                         <literal>&lt;actions&gt;</literal> element that define the permissions required for each action:\r
823                         create, retrieve, update, and delete.\r
824                         </simpara>\r
825                         <itemizedlist>\r
826                         <listitem>\r
827                         <simpara>\r
828                         The <literal>permission</literal> attribute identifies the name of the permission that must\r
829                         have been granted to the user to perform the action.\r
830                         </simpara>\r
831                         </listitem>\r
832                         <listitem>\r
833                         <simpara>\r
834                         The <literal>contextfield</literal> attribute, if it exists, defines the field in this class\r
835                         that identifies the library within the system for which the user must have\r
836                         prvileges to work. If a user has been granted a given permission, but has not been\r
837                         granted privileges to work at a given library, they can not perform the action\r
838                         at that library.\r
839                         </simpara>\r
840                         </listitem>\r
841                         </itemizedlist>\r
842                         </callout>\r
843                         <callout arearefs="CO5-7">\r
844                         <simpara>\r
845                         The rarely-used <literal>&lt;context&gt;</literal> element identifies a linked field (<literal>link</literal>\r
846                         attribute) in this class which links to an external class that holds the field\r
847                         (<literal>field</literal> attribute) that identifies the library within the system for which the\r
848                         user must have privileges to work.\r
849                         </simpara>\r
850                         </callout>\r
851                         </calloutlist>\r
852                         <simpara>When you retrieve an instance of a class, you can ask for the result to\r
853                         <emphasis>flesh</emphasis> some or all of the linked fields of that class, so that the linked\r
854                         instances are returned embedded directly in your requested instance. In that\r
855                         same request you can ask for the fleshed instances to in turn have their linked\r
856                         fields fleshed. By bundling all of this into a single request and result\r
857                         sequence, you can avoid the network overhead of requiring the client to request\r
858                         the base object, then request each linked object in turn.</simpara>\r
859                         <simpara>You can also iterate over a collection of instances and set the automatically\r
860                         generated <literal>isdeleted</literal>, <literal>isupdated</literal>, or <literal>isnew</literal> properties to indicate that\r
861                         the given instance has been deleted, updated, or created respectively.\r
862                         Evergreen can then act in batch mode over the collection to perform the\r
863                         requested actions on any of the instances that have been flagged for action.</simpara>\r
864                 </simplesect>\r
865                 <simplesect id="_returning_streaming_results">\r
866                         <title>Returning streaming results</title>\r
867                         <simpara>In the previous implementation of the <literal>opensrf.simple-text.split</literal> method, we\r
868                         returned a reference to the complete array of results. For small values being\r
869                         delivered over the network, this is perfectly acceptable, but for large sets of\r
870                         values this can pose a number of problems for the requesting client. Consider a\r
871                         service that returns a set of bibliographic records in response to a query like\r
872                         "all records edited in the past month"; if the underlying database is\r
873                         relatively active, that could result in thousands of records being returned as\r
874                         a single network request. The client would be forced to block until all of the\r
875                         results are returned, likely resulting in a significant delay, and depending on\r
876                         the implementation, correspondingly large amounts of memory might be consumed\r
877                         as all of the results are read from the network in a single block.</simpara>\r
878                         <simpara>OpenSRF offers a solution to this problem. If the method returns results that\r
879                         can be divided into separate meaningful units, you can register the OpenSRF\r
880                         method as a streaming method and enable the client to loop over the results one\r
881                         unit at a time until the method returns no further results. In addition to\r
882                         registering the method with the provided name, OpenSRF also registers an additional\r
883                         method with <literal>.atomic</literal> appended to the method name. The <literal>.atomic</literal> variant gathers\r
884                         all of the results into a single block to return to the client, giving the caller\r
885                         the ability to choose either streaming or atomic results from a single method\r
886                         definition.</simpara>\r
887                         <simpara>In the following example, the text splitting method has been reimplemented to\r
888                         support streaming; very few changes are required:</simpara>\r
889                         <formalpara><title>Text splitting method - streaming mode</title><para>\r
890                         <programlisting language="perl" linenumbering="unnumbered">sub text_split {\r
891                             my $self = shift;\r
892                             my $conn = shift;\r
893                             my $text = shift;\r
894                             my $delimiter = shift || ' ';\r
895 \r
896                             my @split_text = split $delimiter, $text;\r
897                             foreach my $string (@split_text) { <co id="CO6-1"/>\r
898                                 $conn-&gt;respond($string);\r
899                             }\r
900                             return undef;\r
901                         }\r
902 \r
903                         __PACKAGE__-&gt;register_method(\r
904                             method    =&gt; 'text_split',\r
905                             api_name  =&gt; 'opensrf.simple-text.split',\r
906                             stream    =&gt; 1<co id="CO6-2"/>\r
907                         );</programlisting>\r
908                         </para></formalpara>\r
909                         <calloutlist>\r
910                         <callout arearefs="CO6-1">\r
911                         <simpara>\r
912                         Rather than returning a reference to the array, a streaming method loops\r
913                         over the contents of the array and invokes the <literal>respond()</literal> method of the\r
914                         connection object on each element of the array.\r
915                         </simpara>\r
916                         </callout>\r
917                         <callout arearefs="CO6-2">\r
918                         <simpara>\r
919                         Registering the method as a streaming method instructs OpenSRF to also\r
920                         register an atomic variant (<literal>opensrf.simple-text.split.atomic</literal>).\r
921                         </simpara>\r
922                         </callout>\r
923                         </calloutlist>\r
924                 </simplesect>\r
925                 <simplesect id="_error_warning_info_debug">\r
926                         <title>Error! Warning! Info! Debug!</title>\r
927                         <simpara>As hard as it may be to believe, it is true: applications sometimes do not\r
928                         behave in the expected manner, particularly when they are still under\r
929                         development. The service language bindings for OpenSRF include integrated\r
930                         support for logging messages at the levels of ERROR, WARNING, INFO, DEBUG, and\r
931                         the extremely verbose INTERNAL to either a local file or to a syslogger\r
932                         service. The destination of the log files, and the level of verbosity to be\r
933                         logged, is set in the <literal>opensrf_core.xml</literal> configuration file. To add logging to\r
934                         our Perl example, we just have to add the <literal>OpenSRF::Utils::Logger</literal> package to our\r
935                         list of used Perl modules, then invoke the logger at the desired logging level.</simpara>\r
936                         <simpara>You can include many calls to the OpenSRF logger; only those that are higher\r
937                         than your configured logging level will actually hit the log. The following\r
938                         example exercises all of the available logging levels in OpenSRF:</simpara>\r
939                         <programlisting language="perl" linenumbering="unnumbered">use OpenSRF::Utils::Logger;\r
940                         my $logger = OpenSRF::Utils::Logger;\r
941                         # some code in some function\r
942                         {\r
943                             $logger-&gt;error("Hmm, something bad DEFINITELY happened!");\r
944                             $logger-&gt;warn("Hmm, something bad might have happened.");\r
945                             $logger-&gt;info("Something happened.");\r
946                             $logger-&gt;debug("Something happened; here are some more details.");\r
947                             $logger-&gt;internal("Something happened; here are all the gory details.")\r
948                         }</programlisting>\r
949                         <simpara>If you call the mythical OpenSRF method containing the preceding OpenSRF logger\r
950                         statements on a system running at the default logging level of INFO, you will\r
951                         only see the INFO, WARN, and ERR messages, as follows:</simpara>\r
952                         <formalpara><title>Results of logging calls at the default level of INFO</title><para>\r
953                         <screen>[2010-03-17 22:27:30] opensrf.simple-text [ERR :5681:SimpleText.pm:277:] Hmm, something bad DEFINITELY happened!\r
954                         [2010-03-17 22:27:30] opensrf.simple-text [WARN:5681:SimpleText.pm:278:] Hmm, something bad might have happened.\r
955                         [2010-03-17 22:27:30] opensrf.simple-text [INFO:5681:SimpleText.pm:279:] Something happened.</screen>\r
956                         </para></formalpara>\r
957                         <simpara>If you then increase the the logging level to INTERNAL (5), the logs will\r
958                         contain much more information, as follows:</simpara>\r
959                         <formalpara><title>Results of logging calls at the default level of INTERNAL</title><para>\r
960                         <screen>[2010-03-17 22:48:11] opensrf.simple-text [ERR :5934:SimpleText.pm:277:] Hmm, something bad DEFINITELY happened!\r
961                         [2010-03-17 22:48:11] opensrf.simple-text [WARN:5934:SimpleText.pm:278:] Hmm, something bad might have happened.\r
962                         [2010-03-17 22:48:11] opensrf.simple-text [INFO:5934:SimpleText.pm:279:] Something happened.\r
963                         [2010-03-17 22:48:11] opensrf.simple-text [DEBG:5934:SimpleText.pm:280:] Something happened; here are some more details.\r
964                         [2010-03-17 22:48:11] opensrf.simple-text [INTL:5934:SimpleText.pm:281:] Something happened; here are all the gory details.\r
965                         [2010-03-17 22:48:11] opensrf.simple-text [ERR :5934:SimpleText.pm:283:] Resolver did not find a cache hit\r
966                         [2010-03-17 22:48:21] opensrf.simple-text [INTL:5934:Cache.pm:125:] Stored opensrf.simple-text.test_cache.masaa =&gt; "here" in memcached server\r
967                         [2010-03-17 22:48:21] opensrf.simple-text [DEBG:5934:Application.pm:579:] Coderef for [OpenSRF::Application::Demo::SimpleText::test_cache]...\r
968                         [2010-03-17 22:48:21] opensrf.simple-text [DEBG:5934:Application.pm:586:] A top level Request object is responding de nada\r
969                         [2010-03-17 22:48:21] opensrf.simple-text [DEBG:5934:Application.pm:190:] Method duration for [opensrf.simple-text.test_cache]:  10.005\r
970                         [2010-03-17 22:48:21] opensrf.simple-text [INTL:5934:AppSession.pm:780:] Calling queue_wait(0)\r
971                         [2010-03-17 22:48:21] opensrf.simple-text [INTL:5934:AppSession.pm:769:] Resending...0\r
972                         [2010-03-17 22:48:21] opensrf.simple-text [INTL:5934:AppSession.pm:450:] In send\r
973                         [2010-03-17 22:48:21] opensrf.simple-text [DEBG:5934:AppSession.pm:506:] AppSession sending RESULT to opensrf@private.localhost/... \r
974                         [2010-03-17 22:48:21] opensrf.simple-text [DEBG:5934:AppSession.pm:506:] AppSession sending STATUS to opensrf@private.localhost/... \r
975                         ...</screen>\r
976                         </para></formalpara>\r
977                         <simpara>To see everything that is happening in OpenSRF, try leaving your logging level\r
978                         set to INTERNAL for a few minutes - just ensure that you have a lot of free disk\r
979                         space available if you have a moderately busy system!</simpara>\r
980                 </simplesect>\r
981                 <simplesect id="_caching_results_one_secret_of_scalability">\r
982                         <title>Caching results: one secret of scalability</title>\r
983                         <simpara>If you have ever used an application that depends on a remote Web service\r
984                         outside of your control&#8201;&#8212;&#8201;say, if you need to retrieve results from a\r
985                         microblogging service&#8201;&#8212;&#8201;you know the pain of latency and dependability (or the\r
986                         lack thereof). To improve the response time for OpenSRF services, you can take\r
987                         advantage of the support offered by the <literal>OpenSRF::Utils::Cache</literal> module for\r
988                         communicating with a local instance or cluster of <literal>memcache</literal> daemons to store\r
989                         and retrieve persistent values. The following example demonstrates caching\r
990                         by sleeping for 10 seconds the first time it receives a given cache key and\r
991                         cannot retrieve a corresponding value from the cache:</simpara>\r
992                         <formalpara><title>Simple caching OpenSRF service</title><para>\r
993                         <programlisting language="perl" linenumbering="unnumbered">use OpenSRF::Utils::Cache;<co id="CO7-1"/>\r
994                         sub test_cache {\r
995                             my $self = shift;\r
996                             my $conn = shift;\r
997                             my $test_key = shift;\r
998                             my $cache = OpenSRF::Utils::Cache-&gt;new('global'); <co id="CO7-2"/>\r
999                             my $cache_key = "opensrf.simple-text.test_cache.$test_key"; <co id="CO7-3"/>\r
1000                             my $result = $cache-&gt;get_cache($cache_key) || undef; <co id="CO7-4"/>\r
1001                             if ($result) {\r
1002                                 $logger-&gt;info("Resolver found a cache hit");\r
1003                                 return $result;\r
1004                             }\r
1005                             sleep 10; <co id="CO7-5"/>\r
1006                             my $cache_timeout = 300; <co id="CO7-6"/>\r
1007                             $cache-&gt;put_cache($cache_key, "here", $cache_timeout); <co id="CO7-7"/>\r
1008                             return "There was no cache hit.";\r
1009                         }</programlisting>\r
1010                         </para></formalpara>\r
1011                         <calloutlist>\r
1012                         <callout arearefs="CO7-1">\r
1013                         <simpara>\r
1014                         The OpenSRF::Utils::Cache module provides access to the built-in caching\r
1015                         support in OpenSRF.\r
1016                         </simpara>\r
1017                         </callout>\r
1018                         <callout arearefs="CO7-2">\r
1019                         <simpara>\r
1020                         The constructor for the cache object accepts a single argument to define\r
1021                         the cache type for the object. Each cache type can use a separate <literal>memcache</literal>\r
1022                         server to keep the caches separated. Most Evergreen services use the <literal>global</literal>\r
1023                         cache, while the <literal>anon</literal> cache is used for Web sessions.\r
1024                         </simpara>\r
1025                         </callout>\r
1026                         <callout arearefs="CO7-3">\r
1027                         <simpara>\r
1028                         The cache key is simply a string that uniquely identifies the value you\r
1029                         want to store or retrieve. This line creates a cache key based on the OpenSRF\r
1030                         method name and request input value.\r
1031                         </simpara>\r
1032                         </callout>\r
1033                         <callout arearefs="CO7-4">\r
1034                         <simpara>\r
1035                         The <literal>get_cache()</literal> method checks to see if the cache key already exists. If\r
1036                         a matching key is found, the service immediately returns the stored value.\r
1037                         </simpara>\r
1038                         </callout>\r
1039                         <callout arearefs="CO7-5">\r
1040                         <simpara>\r
1041                         If the cache key does not exist, the code sleeps for 10 seconds to\r
1042                         simulate a call to a slow remote Web service or an intensive process.\r
1043                         </simpara>\r
1044                         </callout>\r
1045                         <callout arearefs="CO7-6">\r
1046                         <simpara>\r
1047                         The <literal>$cache_timeout</literal> variable represents a value for the lifetime of the\r
1048                         cache key in seconds.\r
1049                         </simpara>\r
1050                         </callout>\r
1051                         <callout arearefs="CO7-7">\r
1052                         <simpara>\r
1053                         After the code retrieves its value (or, in the case of this example,\r
1054                         finishes sleeping), it creates the cache entry by calling the <literal>put_cache()</literal>\r
1055                         method. The method accepts three arguments: the cache key, the value to be\r
1056                         stored ("here"), and the timeout value in seconds to ensure that we do not\r
1057                         return stale data on subsequent calls.\r
1058                         </simpara>\r
1059                         </callout>\r
1060                         </calloutlist>\r
1061                 </simplesect>\r
1062                 <simplesect id="_initializing_the_service_and_its_children_child_labour">\r
1063                         <title>Initializing the service and its children: child labour</title>\r
1064                         <simpara>When an OpenSRF service is started, it looks for a procedure called\r
1065                         <literal>initialize()</literal> to set up any global variables shared by all of the children of\r
1066                         the service. The <literal>initialize()</literal> procedure is typically used to retrieve\r
1067                         configuration settings from the <literal>opensrf.xml</literal> file.</simpara>\r
1068                         <simpara>An OpenSRF service spawns one or more children to actually do the work\r
1069                         requested by callers of the service. For every child process an OpenSRF service\r
1070                         spawns, the child process clones the parent environment and then each child\r
1071                         process runs the <literal>child_init()</literal> process (if any) defined in the OpenSRF service\r
1072                         to initialize any child-specific settings.</simpara>\r
1073                         <simpara>When the OpenSRF service kills a child process, it invokes the <literal>child_exit()</literal>\r
1074                         procedure (if any) to clean up any resources associated with the child process.\r
1075                         Similarly, when the OpenSRF service is stopped, it calls the <literal>DESTROY()</literal>\r
1076                         procedure to clean up any remaining resources.</simpara>\r
1077                 </simplesect>\r
1078                 <simplesect id="_retrieving_configuration_settings">\r
1079                         <title>Retrieving configuration settings</title>\r
1080                         <simpara>The settings for OpenSRF services are maintained in the <literal>opensrf.xml</literal> XML\r
1081                         configuration file. The structure of the XML document consists of a root\r
1082                         element <literal>&lt;opensrf&gt;</literal> containing two child elements:</simpara>\r
1083                         <itemizedlist>\r
1084                         <listitem>\r
1085                         <simpara>\r
1086                         The <literal>&lt;default&gt;</literal> element contains an <literal>&lt;apps&gt;</literal> element describing all\r
1087                         OpenSRF services running on this system&#8201;&#8212;&#8201;see <xref linkend="serviceRegistration"/> --, as\r
1088                         well as any other arbitrary XML descriptions required for global configuration\r
1089                         purposes. For example, Evergreen uses this section for email notification and\r
1090                         inter-library patron privacy settings.\r
1091                         </simpara>\r
1092                         </listitem>\r
1093                         <listitem>\r
1094                         <simpara>\r
1095                         The <literal>&lt;hosts&gt;</literal> element contains one element per host that participates in\r
1096                         this OpenSRF system. Each host element must include an <literal>&lt;activeapps&gt;</literal> element\r
1097                         that lists all of the services to start on this host when the system starts\r
1098                         up. Each host element can optionally override any of the default settings.\r
1099                         </simpara>\r
1100                         </listitem>\r
1101                         </itemizedlist>\r
1102                         <simpara>OpenSRF includes a service named <literal>opensrf.settings</literal> to provide distributed\r
1103                         cached access to the configuration settings with a simple API:</simpara>\r
1104                         <itemizedlist>\r
1105                         <listitem>\r
1106                         <simpara>\r
1107                         <literal>opensrf.settings.default_config.get</literal> accepts zero arguments and returns\r
1108                         the complete set of default settings as a JSON document.\r
1109                         </simpara>\r
1110                         </listitem>\r
1111                         <listitem>\r
1112                         <simpara>\r
1113                         <literal>opensrf.settings.host_config.get</literal> accepts one argument (hostname) and\r
1114                         returns the complete set of settings, as customized for that hostname, as a\r
1115                         JSON document.\r
1116                         </simpara>\r
1117                         </listitem>\r
1118                         <listitem>\r
1119                         <simpara>\r
1120                         <literal>opensrf.settings.xpath.get</literal> accepts one argument (an\r
1121                         <ulink url="http://www.w3.org/TR/xpath/">XPath</ulink> expression) and returns the portion of\r
1122                         the configuration file that matches the expression as a JSON document.\r
1123                         </simpara>\r
1124                         </listitem>\r
1125                         </itemizedlist>\r
1126                         <simpara>For example, to determine whether an Evergreen system uses the opt-in\r
1127                         support for sharing patron information between libraries, you could either\r
1128                         invoke the <literal>opensrf.settings.default_config.get</literal> method and parse the\r
1129                         JSON document to determine the value, or invoke the <literal>opensrf.settings.xpath.get</literal>\r
1130                         method with the XPath <literal>/opensrf/default/share/user/opt_in</literal> argument to\r
1131                         retrieve the value directly.</simpara>\r
1132                         <simpara>In practice, OpenSRF includes convenience libraries in all of its client\r
1133                         language bindings to simplify access to configuration values. C offers\r
1134                         osrfConfig.c, Perl offers <literal>OpenSRF::Utils::SettingsClient</literal>, Java offers\r
1135                         <literal>org.opensrf.util.SettingsClient</literal>, and Python offers <literal>osrf.set</literal>. These\r
1136                         libraries locally cache the configuration file to avoid network roundtrips for\r
1137                         every request and enable the developer to request specific values without\r
1138                         having to manually construct XPath expressions.</simpara>\r
1139                 </simplesect>\r
1140         </section>\r
1141         <section id="_getting_under_the_covers_with_opensrf">\r
1142                 <title>OpenSRF Communication Flows</title>\r
1143                 <simpara>Now that you have seen that it truly is easy to create an OpenSRF service, we\r
1144                 can take a look at what is going on under the covers to make all of this work\r
1145                 for you.</simpara>\r
1146                 <simplesect id="_get_on_the_messaging_bus_safely">\r
1147                         <title>Get on the messaging bus - safely</title>\r
1148                         <simpara>One of the core innovations of OpenSRF was to use the Extensible Messaging and\r
1149                         Presence Protocol (XMPP, more colloquially known as Jabber) as the messaging\r
1150                         bus that ties OpenSRF services together across servers. XMPP is an "XML\r
1151                         protocol for near-real-time messaging, presence, and request-response services"\r
1152                         (<ulink url="http://www.ietf.org/rfc/rfc3920.txt">http://www.ietf.org/rfc/rfc3920.txt</ulink>) that OpenSRF relies on to handle most of\r
1153                         the complexity of networked communications.  OpenSRF requres an XMPP server\r
1154                         that supports multiple domains such as <ulink url="http://www.ejabberd.im/">ejabberd</ulink>.\r
1155                         Multiple domain support means that a single server can support XMPP virtual\r
1156                         hosts with separate sets of users and access privileges per domain. By\r
1157                         routing communications through separate public and private XMPP domains,\r
1158                         OpenSRF services gain an additional layer of security.</simpara>\r
1159                         <simpara>The <ulink url="http://evergreen-ils.org/dokuwiki/doku.php?id=opensrf:1.2:install">OpenSRF\r
1160                         installation documentation</ulink> instructs you to create two separate hostnames\r
1161                         (<literal>private.localhost</literal> and <literal>public.localhost</literal>) to use as XMPP domains.  OpenSRF\r
1162                         can control access to its services based on the domain of the client and\r
1163                         whether a given service allows access from clients on the public domain.  When\r
1164                         you start OpenSRF, the first XMPP clients that connect to the XMPP server are\r
1165                         the OpenSRF public and private <emphasis>routers</emphasis>. OpenSRF routers maintain a list of\r
1166                         available services and connect clients to available services. When an OpenSRF\r
1167                         service starts, it establishes a connection to the XMPP server and registers\r
1168                         itself with the private router. The OpenSRF configuration contains a list of\r
1169                         public OpenSRF services, each of which must also register with the public\r
1170                         router.</simpara>\r
1171                 </simplesect>\r
1172                 <simplesect id="_opensrf_communication_flows_over_xmpp">\r
1173                         <title>OpenSRF communication flows over XMPP</title>\r
1174                         <simpara>In a minimal OpenSRF deployment, two XMPP users named "router" connect to the\r
1175                         XMPP server, with one connected to the private XMPP domain and one connected to\r
1176                         the public XMPP domain. Similarly, two XMPP users named "opensrf" connect to\r
1177                         the XMPP server via the private and public XMPP domains. When an OpenSRF\r
1178                         service is started, it uses the "opensrf" XMPP user to advertise its\r
1179                         availability with the corresponding router on that XMPP domain; the XMPP server\r
1180                         automatically assigns a Jabber ID (<emphasis>JID</emphasis>) based on the client hostname to each\r
1181                         service&#8217;s listener process and each connected drone process waiting to carry\r
1182                         out requests. When an OpenSRF router receives a request to invoke a method on a\r
1183                         given service, it connects the requester to the next available listener in the\r
1184                         list of registered listeners for that service.</simpara>\r
1185                         <simpara>Services and clients connect to the XMPP server using a single set of XMPP\r
1186                         client credentials (for example, <literal>opensrf@private.localhost</literal>), but use XMPP\r
1187                         resource identifiers to differentiate themselves in the JID for each\r
1188                         connection. For example, the JID for a copy of the <literal>opensrf.simple-text</literal>\r
1189                         service with process ID <literal>6285</literal> that has connected to the <literal>private.localhost</literal>\r
1190                         domain using the <literal>opensrf</literal> XMPP client credentials could be\r
1191                         <literal>opensrf@private.localhost/opensrf.simple-text_drone_at_localhost_6285</literal>.  By\r
1192                         convention, the user name for OpenSRF clients is <literal>opensrf</literal>, and the user name\r
1193                         for OpenSRF routers is <literal>router</literal>, so the XMPP server for OpenSRF will have four\r
1194                         separate users registered:\r
1195                           * <literal>opensrf@private.localhost</literal> is an OpenSRF client that connects with these\r
1196                         credentials and which can access any OpenSRF service.\r
1197                           * <literal>opensrf@public.localhost</literal> is an OpenSRF client that connects with these\r
1198                         credentials and which can only access OpenSRF services that have registered\r
1199                         with the public router.\r
1200                           * <literal>router@private.localhost</literal> is the private OpenSRF router with which all\r
1201                         services register.\r
1202                           * <literal>router@public.localhost</literal> is the public OpenSRF router with which only\r
1203                         services that must be publicly accessible register.</simpara>\r
1204                         <simpara>All OpenSRF services automatically register themselves with the private XMPP\r
1205                         domain, but only those services that register themselves with the public XMPP\r
1206                         domain can be invoked from public OpenSRF clients.  The OpenSRF client and\r
1207                         router user names, passwords, and domain names, along with the list of services\r
1208                         that should be public, are contained in the <literal>opensrf_core.xml</literal> configuration\r
1209                         file.</simpara>\r
1210                 </simplesect>\r
1211                 <simplesect id="OpenSRFOverHTTP">\r
1212                         <title>OpenSRF communication flows over HTTP</title>\r
1213                         <simpara>In some contexts, access to a full XMPP client is not a practical option. For\r
1214                         example, while XMPP clients have been implemented in JavaScript, you might\r
1215                         be concerned about browser compatibility and processing overhead - or you might\r
1216                         want to issue OpenSRF requests from the command line with <literal>curl</literal>. Fortunately,\r
1217                         any OpenSRF service registered with the public router is accessible via the\r
1218                         OpenSRF HTTP Translator. The OpenSRF HTTP Translator implements the\r
1219                         <ulink url="http://www.open-ils.org/dokuwiki/doku.php?id=opensrf_over_http">OpenSRF-over-HTTP\r
1220                         proposed specification</ulink> as an Apache module that translates HTTP requests into\r
1221                         OpenSRF requests and returns OpenSRF results as HTTP results to the initiating\r
1222                         HTTP client.</simpara>\r
1223                         <formalpara><title>Issuing an HTTP POST request to an OpenSRF method via the OpenSRF HTTP Translator</title><para>\r
1224                         <programlisting language="bash" linenumbering="unnumbered"># curl request broken up over multiple lines for legibility\r
1225                         curl -H "X-OpenSRF-service: opensrf.simple-text"<co id="CO8-1"/>\r
1226                             --data 'osrf-msg=[  \<co id="CO8-2"/>\r
1227                                 {"__c":"osrfMessage","__p":{"threadTrace":0,"locale":"en-CA", <co id="CO8-3"/>\r
1228                                     "type":"REQUEST","payload": {"__c":"osrfMethod","__p": \r
1229                                         {"method":"opensrf.simple-text.reverse","params":["foobar"]}   \r
1230                                     }}                                                                  \r
1231                                 }]'                                                                    \r
1232                         http://localhost/osrf-http-translator <co id="CO8-4"/></programlisting>\r
1233                         </para></formalpara>\r
1234                         <calloutlist>\r
1235                         <callout arearefs="CO8-1">\r
1236                         <simpara>\r
1237                         The <literal>X-OpenSRF-service</literal> header identifies the OpenSRF service of interest.\r
1238                         </simpara>\r
1239                         </callout>\r
1240                         <callout arearefs="CO8-2">\r
1241                         <simpara>\r
1242                         The POST request consists of a single parameter, the <literal>osrf-msg</literal> value,\r
1243                         which contains a JSON array.\r
1244                         </simpara>\r
1245                         </callout>\r
1246                         <callout arearefs="CO8-3">\r
1247                         <simpara>\r
1248                         The first object is an OpenSRF message (<literal>"__c":"osrfMessage"</literal>) with a set of\r
1249                         parameters (<literal>"__p":{}</literal>).\r
1250                         </simpara>\r
1251                         <itemizedlist>\r
1252                         <listitem>\r
1253                         <simpara>\r
1254                         The identifier for the request (<literal>"threadTrace":0</literal>); this value is echoed\r
1255                         back in the result.\r
1256                         </simpara>\r
1257                         </listitem>\r
1258                         <listitem>\r
1259                         <simpara>\r
1260                         The message type (<literal>"type":"REQUEST"</literal>).\r
1261                         </simpara>\r
1262                         </listitem>\r
1263                         <listitem>\r
1264                         <simpara>\r
1265                         The locale for the message; if the OpenSRF method is locale-sensitive, it\r
1266                         can check the locale for each OpenSRF request and return different information\r
1267                         depending on the locale.\r
1268                         </simpara>\r
1269                         </listitem>\r
1270                         <listitem>\r
1271                         <simpara>\r
1272                         The payload of the message (<literal>"payload":{}</literal>) containing the OpenSRF method\r
1273                         request (<literal>"__c":"osrfMethod"</literal>) and its parameters (<literal>"__p:"{}</literal>).\r
1274                         </simpara>\r
1275                         <itemizedlist>\r
1276                         <listitem>\r
1277                         <simpara>\r
1278                         The method name for the request (<literal>"method":"opensrf.simple-text.reverse"</literal>).\r
1279                         </simpara>\r
1280                         </listitem>\r
1281                         <listitem>\r
1282                         <simpara>\r
1283                         A set of JSON parameters to pass to the method (<literal>"params":["foobar"]</literal>); in\r
1284                         this case, a single string <literal>"foobar"</literal>.\r
1285                         </simpara>\r
1286                         </listitem>\r
1287                         </itemizedlist>\r
1288                         </listitem>\r
1289                         </itemizedlist>\r
1290                         </callout>\r
1291                         <callout arearefs="CO8-4">\r
1292                         <simpara>\r
1293                         The URL on which the OpenSRF HTTP translator is listening,\r
1294                         <literal>/osrf-http-translator</literal> is the default location in the Apache example\r
1295                         configuration files shipped with the OpenSRF source, but this is configurable.\r
1296                         </simpara>\r
1297                         </callout>\r
1298                         </calloutlist>\r
1299                         <formalpara><title>Results from an HTTP POST request to an OpenSRF method via the OpenSRF HTTP Translator</title><para>\r
1300                         <programlisting language="bash" linenumbering="unnumbered"># HTTP response broken up over multiple lines for legibility\r
1301                         [{"__c":"osrfMessage","__p":   <co id="CO9-1"/>\r
1302                             {"threadTrace":0, "payload": <co id="CO9-2"/>\r
1303                                 {"__c":"osrfResult","__p": <co id="CO9-3"/>\r
1304                                     {"status":"OK","content":"raboof","statusCode":200} <co id="CO9-4"/>\r
1305                                 },"type":"RESULT","locale":"en-CA" <co id="CO9-5"/>\r
1306                             }\r
1307                         },\r
1308                         {"__c":"osrfMessage","__p":   <co id="CO9-6"/>\r
1309                             {"threadTrace":0,"payload":  <co id="CO9-7"/>\r
1310                                 {"__c":"osrfConnectStatus","__p": <co id="CO9-8"/>\r
1311                                     {"status":"Request Complete","statusCode":205}<co id="CO9-9"/>\r
1312                                 },"type":"STATUS","locale":"en-CA"  <co id="CO9-10"/>\r
1313                             }\r
1314                         }]</programlisting>\r
1315                         </para></formalpara>\r
1316                         <calloutlist>\r
1317                         <callout arearefs="CO9-1">\r
1318                         <simpara>\r
1319                         The OpenSRF HTTP Translator returns an array of JSON objects in its\r
1320                         response. Each object in the response is an OpenSRF message\r
1321                         (<literal>"__c":"osrfMessage"</literal>) with a collection of response parameters (<literal>"__p":</literal>).\r
1322                         </simpara>\r
1323                         </callout>\r
1324                         <callout arearefs="CO9-2">\r
1325                         <simpara>\r
1326                         The OpenSRF message identifier (<literal>"threadTrace":0</literal>) confirms that this\r
1327                         message is in response to the request matching the same identifier.\r
1328                         </simpara>\r
1329                         </callout>\r
1330                         <callout arearefs="CO9-3">\r
1331                         <simpara>\r
1332                         The message includes a payload JSON object (<literal>"payload":</literal>) with an OpenSRF\r
1333                         result for the request (<literal>"__c":"osrfResult"</literal>).\r
1334                         </simpara>\r
1335                         </callout>\r
1336                         <callout arearefs="CO9-4">\r
1337                         <simpara>\r
1338                         The result includes a status indicator string (<literal>"status":"OK"</literal>), the content\r
1339                         of the result response - in this case, a single string "raboof"\r
1340                         (<literal>"content":"raboof"</literal>) - and an integer status code for the request\r
1341                         (<literal>"statusCode":200</literal>).\r
1342                         </simpara>\r
1343                         </callout>\r
1344                         <callout arearefs="CO9-5">\r
1345                         <simpara>\r
1346                         The message also includes the message type (<literal>"type":"RESULT"</literal>) and the\r
1347                         message locale (<literal>"locale":"en-CA"</literal>).\r
1348                         </simpara>\r
1349                         </callout>\r
1350                         <callout arearefs="CO9-6">\r
1351                         <simpara>\r
1352                         The second message in the set of results from the response.\r
1353                         </simpara>\r
1354                         </callout>\r
1355                         <callout arearefs="CO9-7">\r
1356                         <simpara>\r
1357                         Again, the message identifier confirms that this message is in response to\r
1358                         a particular request.\r
1359                         </simpara>\r
1360                         </callout>\r
1361                         <callout arearefs="CO9-8">\r
1362                         <simpara>\r
1363                         The payload of the message denotes that this message is an\r
1364                         OpenSRF connection status message (<literal>"__c":"osrfConnectStatus"</literal>), with some\r
1365                         information about the particular OpenSRF connection that was used for this\r
1366                         request.\r
1367                         </simpara>\r
1368                         </callout>\r
1369                         <callout arearefs="CO9-9">\r
1370                         <simpara>\r
1371                         The response parameters for an OpenSRF connection status message include a\r
1372                         verbose status (<literal>"status":"Request Complete"</literal>) and an integer status code for\r
1373                         the connection status (`"statusCode":205).\r
1374                         </simpara>\r
1375                         </callout>\r
1376                         <callout arearefs="CO9-10">\r
1377                         <simpara>\r
1378                         The message also includes the message type (<literal>"type":"RESULT"</literal>) and the\r
1379                         message locale (<literal>"locale":"en-CA"</literal>).\r
1380                         </simpara>\r
1381                         </callout>\r
1382                         </calloutlist>\r
1383                         <tip><simpara>Before adding a new public OpenSRF service, ensure that it does\r
1384                         not introduce privilege escalation or unchecked access to data. For example,\r
1385                         the Evergreen <literal>open-ils.cstore</literal> private service is an object-relational mapper\r
1386                         that provides read and write access to the entire Evergreen database, so it\r
1387                         would be catastrophic to expose that service publicly. In comparison, the\r
1388                         Evergreen <literal>open-ils.pcrud</literal> public service offers the same functionality as\r
1389                         <literal>open-ils.cstore</literal> to any connected HTTP client or OpenSRF client, but the\r
1390                         additional authentication and authorization layer in <literal>open-ils.pcrud</literal> prevents\r
1391                         unchecked access to Evergreen&#8217;s data.</simpara></tip>\r
1392                 </simplesect>\r
1393                 <simplesect id="_stateless_and_stateful_connections">\r
1394                         <title>Stateless and stateful connections</title>\r
1395                         <simpara>OpenSRF supports both <emphasis>stateless</emphasis> and <emphasis>stateful</emphasis> connections.  When an OpenSRF\r
1396                         client issues a <literal>REQUEST</literal> message in a <emphasis>stateless</emphasis> connection, the router\r
1397                         forwards the request to the next available service and the service returns the\r
1398                         result directly to the client.</simpara>\r
1399                         <formalpara><title>REQUEST flow in a stateless connection</title><para><inlinemediaobject>\r
1400                           <imageobject>\r
1401                           <imagedata fileref="REQUEST.png"/>\r
1402                           </imageobject>\r
1403                           <textobject><phrase>REQUEST flow in a stateless connection</phrase></textobject>\r
1404                         </inlinemediaobject></para></formalpara>\r
1405                         <simpara>When an OpenSRF client issues a <literal>CONNECT</literal> message to create a <emphasis>stateful</emphasis> conection, the\r
1406                         router returns the Jabber ID of the next available service to the client so\r
1407                         that the client can issue one or more <literal>REQUEST</literal> message directly to that\r
1408                         particular service and the service will return corresponding <literal>RESULT</literal> messages\r
1409                         directly to the client. Until the client issues a <literal>DISCONNECT</literal> message, that\r
1410                         particular service is only available to the requesting client. Stateful connections\r
1411                         are useful for clients that need to make many requests from a particular service,\r
1412                         as it avoids the intermediary step of contacting the router for each request, as\r
1413                         well as for operations that require a controlled sequence of commands, such as a\r
1414                         set of database INSERT, UPDATE, and DELETE statements within a transaction.</simpara>\r
1415                         <formalpara><title>CONNECT, REQUEST, and DISCONNECT flow in a stateful connection</title><para><inlinemediaobject>\r
1416                           <imageobject>\r
1417                           <imagedata fileref="CONNECT.png"/>\r
1418                           </imageobject>\r
1419                           <textobject><phrase>CONNECT</phrase></textobject>\r
1420                         </inlinemediaobject></para></formalpara>\r
1421                 </simplesect>\r
1422                 <simplesect id="_message_body_format">\r
1423                         <title>Message body format</title>\r
1424                         <simpara>OpenSRF was an early adopter of JavaScript Object Notation (JSON). While XMPP\r
1425                         is an XML protocol, the Evergreen developers recognized that the compactness of\r
1426                         the JSON format offered a significant reduction in bandwidth for the volume of\r
1427                         messages that would be generated in an application of that size. In addition,\r
1428                         the ability of languages such as JavaScript, Perl, and Python to generate\r
1429                         native objects with minimal parsing offered an attractive advantage over\r
1430                         invoking an XML parser for every message. Instead, the body of the XMPP message\r
1431                         is a simple JSON structure. For a simple request, like the following example\r
1432                         that simply reverses a string, it looks like a significant overhead: but we get\r
1433                         the advantages of locale support and tracing the request from the requester\r
1434                         through the listener and responder (drone).</simpara>\r
1435                         <formalpara><title>A request for opensrf.simple-text.reverse("foobar"):</title><para>\r
1436                         <programlisting language="xml" linenumbering="unnumbered">&lt;message from='router@private.localhost/opensrf.simple-text'\r
1437                           to='opensrf@private.localhost/opensrf.simple-text_listener_at_localhost_6275'\r
1438                           router_from='opensrf@private.localhost/_karmic_126678.3719_6288'\r
1439                           router_to='' router_class='' router_command='' osrf_xid=''\r
1440                         &gt;\r
1441                           &lt;thread&gt;1266781414.366573.12667814146288&lt;/thread&gt;\r
1442                           &lt;body&gt;\r
1443                         [\r
1444                           {"__c":"osrfMessage","__p":\r
1445                             {"threadTrace":"1","locale":"en-US","type":"REQUEST","payload":\r
1446                               {"__c":"osrfMethod","__p":\r
1447                                 {"method":"opensrf.simple-text.reverse","params":["foobar"]}\r
1448                               }\r
1449                             }\r
1450                           }\r
1451                         ]\r
1452                           &lt;/body&gt;\r
1453                         &lt;/message&gt;</programlisting>\r
1454                         </para></formalpara>\r
1455                         <formalpara><title>A response from opensrf.simple-text.reverse("foobar")</title><para>\r
1456                         <programlisting language="xml" linenumbering="unnumbered">&lt;message from='opensrf@private.localhost/opensrf.simple-text_drone_at_localhost_6285'\r
1457                           to='opensrf@private.localhost/_karmic_126678.3719_6288'\r
1458                           router_command='' router_class='' osrf_xid=''\r
1459                         &gt;\r
1460                           &lt;thread&gt;1266781414.366573.12667814146288&lt;/thread&gt;\r
1461                           &lt;body&gt;\r
1462                         [\r
1463                           {"__c":"osrfMessage","__p":\r
1464                             {"threadTrace":"1","payload":\r
1465                               {"__c":"osrfResult","__p":\r
1466                                 {"status":"OK","content":"raboof","statusCode":200}\r
1467                               } ,"type":"RESULT","locale":"en-US"}\r
1468                           },\r
1469                           {"__c":"osrfMessage","__p":\r
1470                             {"threadTrace":"1","payload":\r
1471                               {"__c":"osrfConnectStatus","__p":\r
1472                                 {"status":"Request Complete","statusCode":205}\r
1473                               },"type":"STATUS","locale":"en-US"}\r
1474                           }\r
1475                         ]\r
1476                           &lt;/body&gt;\r
1477                         &lt;/message&gt;</programlisting>\r
1478                         </para></formalpara>\r
1479                         <simpara>The content of the <literal>&lt;body&gt;</literal> element of the OpenSRF request and result should\r
1480                         look familiar; they match the structure of the <link linkend="OpenSRFOverHTTP">OpenSRF over HTTP examples</link> that we previously dissected.</simpara>\r
1481                 </simplesect>\r
1482                 <simplesect id="_registering_opensrf_methods_in_depth">\r
1483                         <title>Registering OpenSRF methods in depth</title>\r
1484                         <simpara>Let&#8217;s explore the call to <literal>__PACKAGE__-&gt;register_method()</literal>; most of the members\r
1485                         of the hash are optional, and for the sake of brevity we omitted them in the\r
1486                         previous example. As we have seen in the results of the <link linkend="opensrfIntrospection">introspection call</link>, a\r
1487                         verbose registration method call is recommended to better enable the internal\r
1488                         documentation. Here is the complete set of members that you should pass to\r
1489                         <literal>__PACKAGE__-&gt;register_method()</literal>:</simpara>\r
1490                         <itemizedlist>\r
1491                         <listitem>\r
1492                         <simpara>\r
1493                         The <literal>method</literal> member specifies the name of the procedure in this module that is being registered as an OpenSRF method.\r
1494                         </simpara>\r
1495                         </listitem>\r
1496                         <listitem>\r
1497                         <simpara>\r
1498                         The <literal>api_name</literal> member specifies the invocable name of the OpenSRF method; by convention, the OpenSRF service name is used as the prefix.\r
1499                         </simpara>\r
1500                         </listitem>\r
1501                         <listitem>\r
1502                         <simpara>\r
1503                         The optional <literal>api_level</literal> member can be used for versioning the methods to allow the use of a deprecated API, but in practical use is always 1.\r
1504                         </simpara>\r
1505                         </listitem>\r
1506                         <listitem>\r
1507                         <simpara>\r
1508                         The optional <literal>argc</literal> member specifies the minimal number of arguments that the method expects.\r
1509                         </simpara>\r
1510                         </listitem>\r
1511                         <listitem>\r
1512                         <simpara>\r
1513                         The optional <literal>stream</literal> member, if set to any value, specifies that the method supports returning multiple values from a single call to subsequent requests. OpenSRF automatically creates a corresponding method with ".atomic" appended to its name that returns the complete set of results in a single request. Streaming methods are useful if you are returning hundreds of records and want to act on the results as they return.\r
1514                         </simpara>\r
1515                         </listitem>\r
1516                         <listitem>\r
1517                         <simpara>\r
1518                         The optional <literal>signature</literal> member is a hash that describes the method&#8217;s purpose, arguments, and return value.\r
1519                         </simpara>\r
1520                         <itemizedlist>\r
1521                         <listitem>\r
1522                         <simpara>\r
1523                         The <literal>desc</literal> member of the <literal>signature</literal> hash describes the method&#8217;s purpose.\r
1524                         </simpara>\r
1525                         </listitem>\r
1526                         <listitem>\r
1527                         <simpara>\r
1528                         The <literal>params</literal> member of the <literal>signature</literal> hash is an array of hashes in which each array element describes the corresponding method argument in order.\r
1529                         </simpara>\r
1530                         <itemizedlist>\r
1531                         <listitem>\r
1532                         <simpara>\r
1533                         The <literal>name</literal> member of the argument hash specifies the name of the argument.\r
1534                         </simpara>\r
1535                         </listitem>\r
1536                         <listitem>\r
1537                         <simpara>\r
1538                         The <literal>desc</literal> member of the argument hash describes the argument&#8217;s purpose.\r
1539                         </simpara>\r
1540                         </listitem>\r
1541                         <listitem>\r
1542                         <simpara>\r
1543                         The <literal>type</literal> member of the argument hash specifies the data type of the argument: for example, string, integer, boolean, number, array, or hash.\r
1544                         </simpara>\r
1545                         </listitem>\r
1546                         </itemizedlist>\r
1547                         </listitem>\r
1548                         <listitem>\r
1549                         <simpara>\r
1550                         The <literal>return</literal> member of the <literal>signature</literal> hash is a hash that describes the return value of the method.\r
1551                         </simpara>\r
1552                         <itemizedlist>\r
1553                         <listitem>\r
1554                         <simpara>\r
1555                         The <literal>desc</literal> member of the <literal>return</literal> hash describes the return value.\r
1556                         </simpara>\r
1557                         </listitem>\r
1558                         <listitem>\r
1559                         <simpara>\r
1560                         The <literal>type</literal> member of the <literal>return</literal> hash specifies the data type of the return value: for example, string, integer, boolean, number, array, or hash.\r
1561                         </simpara>\r
1562                         </listitem>\r
1563                         </itemizedlist>\r
1564                         </listitem>\r
1565                         </itemizedlist>\r
1566                         </listitem>\r
1567                         </itemizedlist>\r
1568                 </simplesect>\r
1569         </section>\r
1570         <section id="_evergreen_specific_opensrf_services">\r
1571                 <title>Evergreen-specific OpenSRF services</title>\r
1572                 <simpara>Evergreen is currently the primary showcase for the use of OpenSRF as an\r
1573                 application architecture. Evergreen 1.6.1 includes the following\r
1574                 set of OpenSRF services:</simpara>\r
1575                 <itemizedlist>\r
1576                 <listitem>\r
1577                 <simpara>\r
1578                 The <literal>open-ils.actor</literal> service supports common tasks for working with user\r
1579                      accounts and libraries.\r
1580                 </simpara>\r
1581                 </listitem>\r
1582                 <listitem>\r
1583                 <simpara>\r
1584                 The <literal>open-ils.auth</literal> service supports authentication of Evergreen users.\r
1585                 </simpara>\r
1586                 </listitem>\r
1587                 <listitem>\r
1588                 <simpara>\r
1589                 The <literal>open-ils.booking</literal> service supports the management of reservations\r
1590                     for bookable items.\r
1591                 </simpara>\r
1592                 </listitem>\r
1593                 <listitem>\r
1594                 <simpara>\r
1595                 The <literal>open-ils.cat</literal> service supports common cataloging tasks, such as\r
1596                      creating, modifying, and merging bibliographic and authority records.\r
1597                 </simpara>\r
1598                 </listitem>\r
1599                 <listitem>\r
1600                 <simpara>\r
1601                 The <literal>open-ils.circ</literal> service supports circulation tasks such as checking\r
1602                     out items and calculating due dates.\r
1603                 </simpara>\r
1604                 </listitem>\r
1605                 <listitem>\r
1606                 <simpara>\r
1607                 The <literal>open-ils.collections</literal> service supports tasks that assist collections\r
1608                     agencies in contacting users with outstanding fines above a certain\r
1609                     threshold.\r
1610                 </simpara>\r
1611                 </listitem>\r
1612                 <listitem>\r
1613                 <simpara>\r
1614                 The <literal>open-ils.cstore</literal> private service supports unrestricted access to\r
1615                     Evergreen fieldmapper objects.\r
1616                 </simpara>\r
1617                 </listitem>\r
1618                 <listitem>\r
1619                 <simpara>\r
1620                 The <literal>open-ils.ingest</literal> private service supports tasks for importing\r
1621                     data such as bibliographic and authority records.\r
1622                 </simpara>\r
1623                 </listitem>\r
1624                 <listitem>\r
1625                 <simpara>\r
1626                 The <literal>open-ils.pcrud</literal> service supports permission-based access to Evergreen\r
1627                     fieldmapper objects.\r
1628                 </simpara>\r
1629                 </listitem>\r
1630                 <listitem>\r
1631                 <simpara>\r
1632                 The <literal>open-ils.penalty</literal> penalty service supports the calculation of\r
1633                     penalties for users, such as being blocked from further borrowing, for\r
1634                     conditions such as having too many items checked out or too many unpaid\r
1635                     fines.\r
1636                 </simpara>\r
1637                 </listitem>\r
1638                 <listitem>\r
1639                 <simpara>\r
1640                 The <literal>open-ils.reporter</literal> service supports the creation and scheduling of\r
1641                     reports.\r
1642                 </simpara>\r
1643                 </listitem>\r
1644                 <listitem>\r
1645                 <simpara>\r
1646                 The <literal>open-ils.reporter-store</literal> private service supports access to Evergreen\r
1647                     fieldmapper objects for the reporting service.\r
1648                 </simpara>\r
1649                 </listitem>\r
1650                 <listitem>\r
1651                 <simpara>\r
1652                 The <literal>open-ils.search</literal> service supports searching across bibliographic\r
1653                     records, authority records, serial records, Z39.50 sources, and ZIP codes.\r
1654                 </simpara>\r
1655                 </listitem>\r
1656                 <listitem>\r
1657                 <simpara>\r
1658                 The <literal>open-ils.storage</literal> private service supports a deprecated method of\r
1659                     providing access to Evergreen fieldmapper objects. Implemented in Perl,\r
1660                     this service has largely been replaced by the much faster C-based\r
1661                     <literal>open-ils.cstore</literal> service.\r
1662                 </simpara>\r
1663                 </listitem>\r
1664                 <listitem>\r
1665                 <simpara>\r
1666                 The <literal>open-ils.supercat</literal> service supports transforms of MARC records into\r
1667                     other formats, such as MODS, as well as providing Atom and RSS feeds and\r
1668                     SRU access.\r
1669                 </simpara>\r
1670                 </listitem>\r
1671                 <listitem>\r
1672                 <simpara>\r
1673                 The <literal>open-ils.trigger</literal> private service supports event-based triggers for\r
1674                     actions such as overdue and holds available notification emails.\r
1675                 </simpara>\r
1676                 </listitem>\r
1677                 <listitem>\r
1678                 <simpara>\r
1679                 The <literal>open-ils.vandelay</literal> service supports the import and export of batches of\r
1680                     bibliographic and authority records.\r
1681                 </simpara>\r
1682                 </listitem>\r
1683                 </itemizedlist>\r
1684                 <simpara>Of some interest is that the <literal>open-ils.reporter-store</literal> and <literal>open-ils.cstore</literal>\r
1685                 services have identical implementations. Surfacing them as separate services\r
1686                 enables a deployer of Evergreen to ensure that the reporting service does not\r
1687                 interfere with the performance-critical <literal>open-ils.cstore</literal> service. One can also\r
1688                 direct the reporting service to a read-only database replica to, again, avoid\r
1689                 interference with <literal>open-ils.cstore</literal> which must write to the master database.</simpara>\r
1690                 <simpara>There are only a few significant services that are not built on OpenSRF in\r
1691                 Evergreen 1.6.0, such as the SIP and Z39.50 servers. These services implement\r
1692                 different protocols and build on existing daemon architectures (Simple2ZOOM\r
1693                 for Z39.50), but still rely on the other OpenSRF services to provide access\r
1694                 to the Evergreen data. The non-OpenSRF services are reasonably self-contained\r
1695                 and can be deployed on different servers to deliver the same sort of deployment\r
1696                 flexibility as OpenSRF services, but have the disadvantage of not being\r
1697                 integrated into the same configuration and control infrastructure as the\r
1698                 OpenSRF services.</simpara>\r
1699         </section>\r
1700         <section id="OpenSRF_attribution">\r
1701                 <simpara>This chapter was taken from Dan Scott's <emphasis>Easing gently into OpenSRF</emphasis> article, June, 2010.</simpara>\r
1702         </section>\r
1703 </chapter>\r