Do some clean up of development files. Adjust converting script to
[working/Evergreen.git] / 1.6 / development / OpenSRF_intro.xml
1 <?xml version="1.0" encoding="UTF-8"?>\r
2 <chapter xml:id="opensrf" xmlns="" version="5.0" xml:lang="EN"\r
3     xmlns:xi="" xmlns:xlink="">\r
4         <chapterinfo>\r
5                 <title>OpenSRF</title>\r
6         </chapterinfo>\r
7         <abstract id="openSRF_abstract">\r
8                 <simpara>One of the claimed advantages of\r
9                 Evergreen over alternative integrated library systems is the underlying Open\r
10                 Service Request Framework (OpenSRF, pronounced "open surf") architecture. This\r
11                 article introduces OpenSRF, demonstrates how to build OpenSRF services through\r
12                 simple code examples, and explains the technical foundations on which OpenSRF\r
13                 is built. This chapter was taken from Dan Scott's <emphasis>Easing gently into OpenSRF</emphasis> article, June, 2010.</simpara>\r
14         </abstract>\r
15         <section id="_introducing_opensrf">\r
16                 <title>Introducing OpenSRF</title>\r
17                 <simpara>OpenSRF is a message routing network that offers scalability and failover\r
18                 support for individual services and entire servers with minimal development and\r
19                 deployment overhead. You can use OpenSRF to build loosely-coupled applications\r
20                 that can be deployed on a single server or on clusters of geographically\r
21                 distributed servers using the same code and minimal configuration changes.\r
22                 Although copyright statements on some of the OpenSRF code date back to Mike\r
23                 Rylander&#8217;s original explorations in 2000, Evergreen was the first major\r
24                 application to be developed with, and to take full advantage of, the OpenSRF\r
25                 architecture starting in 2004. The first official release of OpenSRF was 0.1 in\r
26                 February 2005 (<ulink url=""></ulink>), but OpenSRF&#8217;s development\r
27                 continues a steady pace of enhancement and refinement, with the release of\r
28                 1.0.0 in October 2008 and the most recent release of 1.2.2 in February 2010.</simpara>\r
29                 <simpara>OpenSRF is a distinct break from the architectural approach used by previous\r
30                 library systems and has more in common with modern Web applications. The\r
31                 traditional "scale-up" approach to serve more transactions is to purchase a\r
32                 server with more CPUs and more RAM, possibly splitting the load between a Web\r
33                 server, a database server, and a business logic server. Evergreen, however, is\r
34                 built on the Open Service Request Framework (OpenSRF) architecture, which\r
35                 firmly embraces the "scale-out" approach of spreading transaction load over\r
36                 cheap commodity servers. The <ulink url="">initial GPLS\r
37                 PINES hardware cluster</ulink>, while certainly impressive, may have offered the\r
38                 misleading impression that Evergreen requires a lot of hardware to run.\r
39                 However, Evergreen and OpenSRF easily scale down to a single server; many\r
40                 Evergreen libraries run their entire library system on a single server, and\r
41                 most OpenSRF and Evergreen development occurs on a virtual machine running on a\r
42                 single laptop or desktop image.</simpara>\r
43                 <simpara>Another common concern is that the flexibility of OpenSRF&#8217;s distributed\r
44                 architecture makes it complex to configure and to write new applications. This\r
45                 article demonstrates that OpenSRF itself is an extremely simple architecture on\r
46                 which one can easily build applications of many kinds – not just library\r
47                 applications – and that you can use a number of different languages to call and\r
48                 implement OpenSRF methods with a minimal learning curve. With an application\r
49                 built on OpenSRF, when you identify a bottleneck in your application&#8217;s business\r
50                 logic layer, you can adjust the number of the processes serving that particular\r
51                 bottleneck on each of your servers; or if the problem is that your service is\r
52                 resource-hungry, you could add an inexpensive server to your cluster and\r
53                 dedicate it to running that resource-hungry service.</simpara>\r
54                 <simplesect id="_programming_language_support">\r
55                         <title>Programming language support</title>\r
56                         <simpara>If you need to develop an entirely new OpenSRF service, you can choose from a\r
57                         number of different languages in which to implement that service. OpenSRF\r
58                         client language bindings have been written for C, Java, JavaScript, Perl, and\r
59                         Python, and service language bindings have been written for C, Perl, and Python.\r
60                         This article uses Perl examples as a lowest common denominator programming\r
61                         language. Writing an OpenSRF binding for another language is a relatively small\r
62                         task if that language offers libraries that support the core technologies on\r
63                         which OpenSRF depends:</simpara>\r
64                         <itemizedlist>\r
65                         <listitem>\r
66                         <simpara>\r
67                         <ulink url="">Extensible Messaging and Presence\r
68                         Protocol</ulink> (XMPP, sometimes referred to as Jabber) - provides the base messaging\r
69                         infrastructure between OpenSRF clients and services\r
70                         </simpara>\r
71                         </listitem>\r
72                         <listitem>\r
73                         <simpara>\r
74                         <ulink url="">JavaScript Object Notation</ulink> (JSON) - serializes the content\r
75                         of each XMPP message in a standardized and concise format\r
76                         </simpara>\r
77                         </listitem>\r
78                         <listitem>\r
79                         <simpara>\r
80                         <ulink url="">memcached</ulink> - provides the caching service\r
81                         </simpara>\r
82                         </listitem>\r
83                         <listitem>\r
84                         <simpara>\r
85                         <ulink url="">syslog</ulink> - the standard UNIX logging\r
86                         service\r
87                         </simpara>\r
88                         </listitem>\r
89                         </itemizedlist>\r
90                         <simpara>Unfortunately, the\r
91                         <ulink url="">OpenSRF\r
92                         reference documentation</ulink>, although augmented by the\r
93                         <ulink url="">OpenSRF\r
94                         glossary</ulink>, blog posts like <ulink url="">the description\r
95                         of OpenSRF and Jabber</ulink>, and even this article, is not a sufficient substitute\r
96                         for a complete specification on which one could implement a language binding.\r
97                         The recommended option for would-be developers of another language binding is\r
98                         to use the Python implementation as the cleanest basis for a port to another\r
99                         language.</simpara>\r
100                 </simplesect>\r
101         </section>\r
102         <section id="writing_an_opensrf_service">\r
103                 <title>Writing an OpenSRF Service</title>\r
104                 <simpara>Imagine an application architecture in which 10 lines of Perl or Python, using\r
105                 the data types native to each language, are enough to implement a method that\r
106                 can then be deployed and invoked seamlessly across hundreds of servers.  You\r
107                 have just imagined developing with OpenSRF – it is truly that simple. Under the\r
108                 covers, of course, the OpenSRF language bindings do an incredible amount of\r
109                 work on behalf of the developer. An OpenSRF application consists of one or more\r
110                 OpenSRF services that expose methods: for example, the <literal>opensrf.simple-text</literal>\r
111                 <ulink url="">demonstration\r
112                 service</ulink> exposes the <literal>opensrf.simple-text.split()</literal> and\r
113                 <literal>opensrf.simple-text.reverse()</literal> methods. Each method accepts zero or more\r
114                 arguments and returns zero or one results. The data types supported by OpenSRF\r
115                 arguments and results are typical core language data types: strings, numbers,\r
116                 booleans, arrays, and hashes.</simpara>\r
117                 <simpara>To implement a new OpenSRF service, perform the following steps:</simpara>\r
118                 <orderedlist numeration="arabic">\r
119                 <listitem>\r
120                 <simpara>\r
121                 Include the base OpenSRF support libraries\r
122                 </simpara>\r
123                 </listitem>\r
124                 <listitem>\r
125                 <simpara>\r
126                 Write the code for each of your OpenSRF methods as separate procedures\r
127                 </simpara>\r
128                 </listitem>\r
129                 <listitem>\r
130                 <simpara>\r
131                 Register each method\r
132                 </simpara>\r
133                 </listitem>\r
134                 <listitem>\r
135                 <simpara>\r
136                 Add the service definition to the OpenSRF configuration files\r
137                 </simpara>\r
138                 </listitem>\r
139                 </orderedlist>\r
140                 <simpara>For example, the following code implements an OpenSRF service. The service\r
141                 includes one method named <literal>opensrf.simple-text.reverse()</literal> that accepts one\r
142                 string as input and returns the reversed version of that string:</simpara>\r
143 <programlisting language="perl" linenumbering="unnumbered">\r
144 #!/usr/bin/perl\r
145 \r
146 package OpenSRF::Application::Demo::SimpleText;\r
147 \r
148 use strict;\r
149 \r
150 use OpenSRF::Application;\r
151 use parent qw/OpenSRF::Application/;\r
152 \r
153 sub text_reverse {\r
154     my ($self , $conn, $text) = @_;\r
155     my $reversed_text = scalar reverse($text);\r
156     return $reversed_text;\r
157 }\r
158 \r
159 __PACKAGE__-&gt;register_method(\r
160     method    =&gt; 'text_reverse',\r
161     api_name  =&gt; 'opensrf.simple-text.reverse'\r
162 );\r
163 </programlisting>\r
164                 <simpara>Ten lines of code, and we have a complete OpenSRF service that exposes a single\r
165                 method and could be deployed quickly on a cluster of servers to meet your\r
166                 application&#8217;s ravenous demand for reversed strings! If you&#8217;re unfamiliar with\r
167                 Perl, the <literal>use OpenSRF::Application; use parent qw/OpenSRF::Application/;</literal>\r
168                 lines tell this package to inherit methods and properties from the\r
169                 <literal>OpenSRF::Application</literal> module. For example, the call to\r
170                 <literal>__PACKAGE__-&gt;register_method()</literal> is defined in <literal>OpenSRF::Application</literal> but due to\r
171                 inheritance is available in this package (named by the special Perl symbol\r
172                 <literal>__PACKAGE__</literal> that contains the current package name). The <literal>register_method()</literal>\r
173                 procedure is how we introduce a method to the rest of the OpenSRF world.</simpara>\r
174                 <simplesect id="serviceRegistration">\r
175                         <title>Registering a service with the OpenSRF configuration files</title>\r
176                         <simpara>Two files control most of the configuration for OpenSRF:</simpara>\r
177                         <itemizedlist>\r
178                         <listitem>\r
179                         <simpara>\r
180                         <literal>opensrf.xml</literal> contains the configuration for the service itself, as well as\r
181                         a list of which application servers in your OpenSRF cluster should start\r
182                         the service.\r
183                         </simpara>\r
184                         </listitem>\r
185                         <listitem>\r
186                         <simpara>\r
187                         <literal>opensrf_core.xml</literal> (often referred to as the "bootstrap configuration"\r
188                         file) contains the OpenSRF networking information, including the XMPP server\r
189                         connection credentials for the public and private routers. You only need to touch\r
190                         this for a new service if the new service needs to be accessible via the\r
191                         public router.\r
192                         </simpara>\r
193                         </listitem>\r
194                         </itemizedlist>\r
195                         <simpara>Begin by defining the service itself in <literal>opensrf.xml</literal>. To register the\r
196                         <literal>opensrf.simple-text</literal> service, add the following section to the <literal>&lt;apps&gt;</literal>\r
197                         element (corresponding to the XPath <literal>/opensrf/default/apps/</literal>):</simpara>\r
198 <programlisting language="xml" linenumbering="unnumbered">\r
199 &lt;apps&gt;\r
200   &lt;opensrf.simple-text&gt; <co id="CO1-1"/> \r
201     &lt;keepalive&gt;3&lt;/keepalive&gt;<co id="CO1-2"/> \r
202     &lt;stateless&gt;1&lt;/stateless&gt;<co id="CO1-3"/>\r
203     &lt;language&gt;perl&lt;/language&gt;<co id="CO1-4"/> \r
204     &lt;implementation&gt;OpenSRF::Application::Demo::SimpleText&lt;/implementation&gt;<co id="CO1-5"/> \r
205     &lt;max_requests&gt;100&lt;/max_requests&gt;<co id="CO1-6"/> \r
206     &lt;unix_config&gt;\r
207       &lt;max_requests&gt;1000&lt;/max_requests&gt; <co id="CO1-7"/> \r
208       &lt;unix_log&gt;opensrf.simple-text_unix.log&lt;/unix_log&gt; <co id="CO1-8"/> \r
209       &lt;unix_sock&gt;opensrf.simple-text_unix.sock&lt;/unix_sock&gt;<co id="CO1-9"/> \r
210       &lt;unix_pid&gt;;/unix_pid&gt; <co id="CO1-10"/> \r
211       &lt;min_children&gt;5&lt;/min_children&gt;  <co id="CO1-11"/> \r
212       &lt;max_children&gt;15&lt;/max_children&gt;<co id="CO1-12"/> \r
213       &lt;min_spare_children&gt;2&lt;/min_spare_children&gt;<co id="CO1-13"/> \r
214       &lt;max_spare_children&gt;5&lt;/max_spare_children&gt; <co id="CO1-14"/> \r
215     &lt;/unix_config&gt;\r
216   &lt;/opensrf.simple-text&gt;\r
217 \r
218   &lt;!-- other OpenSRF services registered here... --&gt;\r
219 &lt;/apps&gt;\r
220 </programlisting>\r
221                         <calloutlist>\r
222                         <callout arearefs="CO1-1">\r
223                         <simpara>\r
224                         The element name is the name that the OpenSRF control scripts use to refer\r
225                         to the service.\r
226                         </simpara>\r
227                         </callout>\r
228                         <callout arearefs="CO1-2">\r
229                         <simpara>\r
230                         The <literal>&lt;keepalive&gt;</literal> element specifies the interval (in seconds) between\r
231                         checks to determine if the service is still running.\r
232                         </simpara>\r
233                         </callout>\r
234                         <callout arearefs="CO1-3">\r
235                         <simpara>\r
236                         The <literal>&lt;stateless&gt;</literal> element specifies whether OpenSRF clients can call\r
237                         methods from this service without first having to create a connection to a\r
238                         specific service backend process for that service. If the value is <literal>1</literal>, then\r
239                         the client can simply issue a request and the router will forward the request\r
240                         to an available service and the result will be returned directly to the client.\r
241                         </simpara>\r
242                         </callout>\r
243                         <callout arearefs="CO1-4">\r
244                         <simpara>\r
245                         The <literal>&lt;language&gt;</literal> element specifies the programming language in which the\r
246                         service is implemented.\r
247                         </simpara>\r
248                         </callout>\r
249                         <callout arearefs="CO1-5">\r
250                         <simpara>\r
251                         The <literal>&lt;implementation&gt;</literal> element pecifies the name of the library or module\r
252                         in which the service is implemented.\r
253                         </simpara>\r
254                         </callout>\r
255                         <callout arearefs="CO1-6">\r
256                         <simpara>\r
257                         (C implementations only): The <literal>&lt;max_requests&gt;</literal> element, as a direct child\r
258                         of the service element name, specifies the maximum number of requests a process\r
259                         serves before it is killed and replaced by a new process.\r
260                         </simpara>\r
261                         </callout>\r
262                         <callout arearefs="CO1-7">\r
263                         <simpara>\r
264                         (Perl implementations only): The <literal>&lt;max_requests&gt;</literal> element, as a direct\r
265                         child of the <literal>&lt;unix_config&gt;</literal> element, specifies the maximum number of requests\r
266                         a process serves before it is killed and replaced by a new process.\r
267                         </simpara>\r
268                         </callout>\r
269                         <callout arearefs="CO1-8">\r
270                         <simpara>\r
271                         The <literal>&lt;unix_log&gt;</literal> element specifies the name of the log file for\r
272                         language-specific log messages such as syntax warnings.\r
273                         </simpara>\r
274                         </callout>\r
275                         <callout arearefs="CO1-9">\r
276                         <simpara>\r
277                         The <literal>&lt;unix_sock&gt;</literal> element specifies the name of the UNIX socket used for\r
278                         inter-process communications.\r
279                         </simpara>\r
280                         </callout>\r
281                         <callout arearefs="CO1-10">\r
282                         <simpara>\r
283                         The <literal>&lt;unix_pid&gt;</literal> element specifies the name of the PID file for the\r
284                         master process for the service.\r
285                         </simpara>\r
286                         </callout>\r
287                         <callout arearefs="CO1-11">\r
288                         <simpara>\r
289                         The <literal>&lt;min_children&gt;</literal> element specifies the minimum number of child\r
290                         processes that should be running at any given time.\r
291                         </simpara>\r
292                         </callout>\r
293                         <callout arearefs="CO1-12">\r
294                         <simpara>\r
295                         The <literal>&lt;max_children&gt;</literal> element specifies the maximum number of child\r
296                         processes that should be running at any given time.\r
297                         </simpara>\r
298                         </callout>\r
299                         <callout arearefs="CO1-13">\r
300                         <simpara>\r
301                         The <literal>&lt;min_spare_children&gt;</literal> element specifies the minimum number of idle\r
302                         child processes that should be available to handle incoming requests.  If there\r
303                         are fewer than this number of spare child processes, new processes will be\r
304                         spawned.\r
305                         </simpara>\r
306                         </callout>\r
307                         <callout arearefs="CO1-14">\r
308                         <simpara>\r
309                         The`&lt;max_spare_children&gt;` element specifies the maximum number of idle\r
310                         child processes that should be available to handle incoming requests. If there\r
311                         are more than this number of spare child processes, the extra processes will be\r
312                         killed.\r
313                         </simpara>\r
314                         </callout>\r
315                         </calloutlist>\r
316                         <simpara>To make the service accessible via the public router, you must also\r
317                         edit the <literal>opensrf_core.xml</literal> configuration file to add the service to the list\r
318                         of publicly accessible services:</simpara>\r
319                         <formalpara><title>Making a service publicly accessible in <literal>opensrf_core.xml</literal></title><para>\r
320 <programlisting language="xml" linenumbering="unnumbered">\r
321 &lt;router&gt;<co id="CO2-1"/> \r
322     &lt;!-- This is the public router. On this router, we only register applications\r
323      which should be accessible to everyone on the opensrf network --&gt;\r
324     &lt;name&gt;router&lt;/name&gt;\r
325     &lt;domain&gt;public.localhost&lt;/domain&gt;<co id="CO2-2"/>\r
326     &lt;services&gt;\r
327         &lt;service&gt;opensrf.math&lt;/service&gt;\r
328         &lt;service&gt;opensrf.simple-text&lt;/service&gt; <co id="CO2-3"/> \r
329     &lt;/services&gt;\r
330 &lt;/router&gt;\r
331 </programlisting>\r
332                         </para></formalpara>\r
333                         <calloutlist>\r
334                         <callout arearefs="CO2-1">\r
335                         <simpara>\r
336                         This section of the <literal>opensrf_core.xml</literal> file is located at XPath\r
337                         <literal>/config/opensrf/routers/</literal>.\r
338                         </simpara>\r
339                         </callout>\r
340                         <callout arearefs="CO2-2">\r
341                         <simpara>\r
342                         <literal>public.localhost</literal> is the canonical public router domain in the OpenSRF\r
343                         installation instructions.\r
344                         </simpara>\r
345                         </callout>\r
346                         <callout arearefs="CO2-3">\r
347                         <simpara>\r
348                         Each <literal>&lt;service&gt;</literal> element contained in the <literal>&lt;services&gt;</literal> element\r
349                         offers their services via the public router as well as the private router.\r
350                         </simpara>\r
351                         </callout>\r
352                         </calloutlist>\r
353                         <simpara>Once you have defined the new service, you must restart the OpenSRF Router\r
354                         to retrieve the new configuration and start or restart the service itself.</simpara>\r
355                         <simpara>Complete working examples of the <link linkend="opensrf-core-xml">opensrf_core.xml</link> and\r
356                         <link linkend="opensrf-xml">opensrf.xml</link> configuration files are included with this article\r
357                         for your reference.</simpara>\r
358                 </simplesect>\r
359                 <simplesect id="_calling_an_opensrf_method">\r
360                         <title>Calling an OpenSRF method</title>\r
361                         <simpara>OpenSRF clients in any supported language can invoke OpenSRF services in any\r
362                         supported language. So let&#8217;s see a few examples of how we can call our fancy\r
363                         new <literal>opensrf.simple-text.reverse()</literal> method:</simpara>\r
364                         <simplesect id="_calling_opensrf_methods_from_the_srfsh_client">\r
365                                 <title>Calling OpenSRF methods from the srfsh client</title>\r
366                                 <simpara><literal>srfsh</literal> is a command-line tool installed with OpenSRF that you can use to call\r
367                                 OpenSRF methods. To call an OpenSRF method, issue the <literal>request</literal> command and\r
368                                 pass the OpenSRF service and method name as the first two arguments; then pass\r
369                                 one or more JSON objects delimited by commas as the arguments to the method\r
370                                 being invoked.</simpara>\r
371                                 <simpara>The following example calls the <literal>opensrf.simple-text.reverse</literal> method of the\r
372                                 <literal>opensrf.simple-text</literal> OpenSRF service, passing the string <literal>"foobar"</literal> as the\r
373                                 only method argument:</simpara>\r
374 <programlisting language="sh" linenumbering="unnumbered">\r
375 $ srfsh\r
376 srfsh # request opensrf.simple-text opensrf.simple-text.reverse "foobar"\r
377 \r
378 Received Data: "raboof"\r
379 \r
380 =------------------------------------\r
381 Request Completed Successfully\r
382 Request Time in seconds: 0.016718\r
383 =------------------------------------\r
384 </programlisting>\r
385                         </simplesect>\r
386                         <simplesect id="opensrfIntrospection">\r
387                                 <title>Getting documentation for OpenSRF methods from the srfsh client</title>\r
388                                 <simpara>The <literal>srfsh</literal> client also gives you command-line access to retrieving metadata\r
389                                 about OpenSRF services and methods. For a given OpenSRF method, for example,\r
390                                 you can retrieve information such as the minimum number of required arguments,\r
391                                 the data type and a description of each argument, the package or library in\r
392                                 which the method is implemented, and a description of the method. To retrieve\r
393                                 the documentation for an opensrf method from <literal>srfsh</literal>, issue the <literal>introspect</literal>\r
394                                 command, followed by the name of the OpenSRF service and (optionally) the\r
395                                 name of the OpenSRF method. If you do not pass a method name to the <literal>introspect</literal>\r
396                                 command, <literal>srfsh</literal> lists all of the methods offered by the service. If you pass\r
397                                 a partial method name, <literal>srfsh</literal> lists all of the methods that match that portion\r
398                                 of the method name.</simpara>\r
399                                 <note><simpara>The quality and availability of the descriptive information for each\r
400                                 method depends on the developer to register the method with complete and\r
401                                 accurate information. The quality varies across the set of OpenSRF and\r
402                                 Evergreen APIs, although some effort is being put towards improving the\r
403                                 state of the internal documentation.</simpara></note>\r
404 <programlisting language="sh" linenumbering="unnumbered">\r
405 srfsh# introspect opensrf.simple-text "opensrf.simple-text.reverse"\r
406 --&gt; opensrf.simple-text\r
407 \r
408 Received Data: {\r
409   "__c":"opensrf.simple-text",\r
410   "__p":{\r
411     "api_level":1,\r
412     "stream":0,      <co id="CO3-1"/>\r
413     "object_hint":"OpenSRF_Application_Demo_SimpleText",\r
414     "remote":0,\r
415     "package":"OpenSRF::Application::Demo::SimpleText", <co id="CO3-2"/>\r
416     "api_name":"opensrf.simple-text.reverse",<co id="CO3-3"/>\r
417     "server_class":"opensrf.simple-text",\r
418     "signature":{ <co id="CO3-4"/>\r
419       "params":[  <co id="CO3-5"/>\r
420         {\r
421           "desc":"The string to reverse",\r
422           "name":"text",\r
423           "type":"string"\r
424         }\r
425       ],\r
426       "desc":"Returns the input string in reverse order\n", <co id="CO3-6"/>\r
427       "return":{                                            <co id="CO3-7"/>\r
428         "desc":"Returns the input string in reverse order",\r
429         "type":"string"\r
430       }\r
431     },\r
432     "method":"text_reverse",  <co id="CO3-8"/>\r
433     "argc":1 <co id="CO3-9"/>\r
434   }\r
435 }\r
436 </programlisting>\r
437                                 <calloutlist>\r
438                                 <callout arearefs="CO3-1">\r
439                                 <simpara>\r
440                                 <literal>stream</literal> denotes whether the method supports streaming responses or not.\r
441                                 </simpara>\r
442                                 </callout>\r
443                                 <callout arearefs="CO3-2">\r
444                                 <simpara>\r
445                                 <literal>package</literal> identifies which package or library implements the method.\r
446                                 </simpara>\r
447                                 </callout>\r
448                                 <callout arearefs="CO3-3">\r
449                                 <simpara>\r
450                                 <literal>api_name</literal> identifies the name of the OpenSRF method.\r
451                                 </simpara>\r
452                                 </callout>\r
453                                 <callout arearefs="CO3-4">\r
454                                 <simpara>\r
455                                 <literal>signature</literal> is a hash that describes the parameters for the method.\r
456                                 </simpara>\r
457                                 </callout>\r
458                                 <callout arearefs="CO3-5">\r
459                                 <simpara>\r
460                                 <literal>params</literal> is an array of hashes describing each parameter in the method;\r
461                                 each parameter has a description (<literal>desc</literal>), name (<literal>name</literal>), and type (<literal>type</literal>).\r
462                                 </simpara>\r
463                                 </callout>\r
464                                 <callout arearefs="CO3-6">\r
465                                 <simpara>\r
466                                 <literal>desc</literal> is a string that describes the method itself.\r
467                                 </simpara>\r
468                                 </callout>\r
469                                 <callout arearefs="CO3-7">\r
470                                 <simpara>\r
471                                 <literal>return</literal> is a hash that describes the return value for the method; it\r
472                                 contains a description of the return value (<literal>desc</literal>) and the type of the\r
473                                 returned value (<literal>type</literal>).\r
474                                 </simpara>\r
475                                 </callout>\r
476                                 <callout arearefs="CO3-8">\r
477                                 <simpara>\r
478                                 <literal>method</literal> identifies the name of the function or method in the source\r
479                                 implementation.\r
480                                 </simpara>\r
481                                 </callout>\r
482                                 <callout arearefs="CO3-9">\r
483                                 <simpara>\r
484                                 <literal>argc</literal> is an integer describing the minimum number of arguments that\r
485                                 must be passed to this method.\r
486                                 </simpara>\r
487                                 </callout>\r
488                                 </calloutlist>\r
489                         </simplesect>\r
490                         <simplesect id="_calling_opensrf_methods_from_perl_applications">\r
491                                 <title>Calling OpenSRF methods from Perl applications</title>\r
492                                 <simpara>To call an OpenSRF method from Perl, you must connect to the OpenSRF service,\r
493                                 issue the request to the method, and then retrieve the results.</simpara>\r
494 <programlisting language="perl" linenumbering="unnumbered">\r
495 #/usr/bin/perl\r
496 use strict;\r
497 use OpenSRF::AppSession;\r
498 use OpenSRF::System;\r
499 \r
500 OpenSRF::System-&gt;bootstrap_client(config_file =&gt; '/openils/conf/opensrf_core.xml');<co id="CO4-1"/>\r
501 \r
502 my $session = OpenSRF::AppSession-&gt;create("opensrf.simple-text");<co id="CO4-2"/>\r
503 \r
504 print "substring: Accepts a string and a number as input, returns a string\n";\r
505 my $result = $session-&gt;request("opensrf.simple-text.substring", "foobar", 3);<co id="CO4-3"/>\r
506 my $request = $result-&gt;gather(); <co id="CO4-4"/>\r
507 print "Substring: $request\n\n";\r
508 \r
509 print "split: Accepts two strings as input, returns an array of strings\n";\r
510 $request = $session-&gt;request("opensrf.simple-text.split", "This is a test", " ");<co id="CO4-5"/>\r
511 my $output = "Split: [";\r
512 my $element;\r
513 while ($element = $request-&gt;recv()) {   <co id="CO4-6"/>\r
514     $output .= $element-&gt;content . ", ";  <co id="CO4-7"/>\r
515 }\r
516 $output =~ s/, $/]/;\r
517 print $output . "\n\n";\r
518 \r
519 print "statistics: Accepts an array of strings as input, returns a hash\n";\r
520 my @many_strings = [\r
521     "First I think I'll have breakfast",\r
522     "Then I think that lunch would be nice",\r
523     "And then seventy desserts to finish off the day"\r
524 ];\r
525 \r
526 $result = $session-&gt;request("opensrf.simple-text.statistics", @many_strings); <co id="CO4-8"/>\r
527 $request = $result-&gt;gather();    <co id="CO4-9"/>\r
528 print "Length: " . $result-&gt;{'length'} . "\n";\r
529 print "Word count: " . $result-&gt;{'word_count'} . "\n";\r
530 \r
531 $session-&gt;disconnect();       <co id="CO4-10"/>\r
532 </programlisting>\r
533                                 <calloutlist>\r
534                                 <callout arearefs="CO4-1">\r
535                                 <simpara>\r
536                                 The <literal>OpenSRF::System-&gt;bootstrap_client()</literal> method reads the OpenSRF\r
537                                 configuration information from the indicated file and creates an XMPP client\r
538                                 connection based on that information.\r
539                                 </simpara>\r
540                                 </callout>\r
541                                 <callout arearefs="CO4-2">\r
542                                 <simpara>\r
543                                 The <literal>OpenSRF::AppSession-&gt;create()</literal> method accepts one argument - the name\r
544                                 of the OpenSRF service to which you want to want to make one or more requests -\r
545                                 and returns an object prepared to use the client connection to make those\r
546                                 requests.\r
547                                 </simpara>\r
548                                 </callout>\r
549                                 <callout arearefs="CO4-3">\r
550                                 <simpara>\r
551                                 The <literal>OpenSRF::AppSession-&gt;request()</literal> method accepts a minimum of one\r
552                                 argument - the name of the OpenSRF method to which you want to make a request -\r
553                                 followed by zero or more arguments to pass to the OpenSRF method as input\r
554                                 values. This example passes a string and an integer to the\r
555                                 <literal>opensrf.simple-text.substring</literal> method defined by the <literal>opensrf.simple-text</literal>\r
556                                 OpenSRF service.\r
557                                 </simpara>\r
558                                 </callout>\r
559                                 <callout arearefs="CO4-4">\r
560                                 <simpara>\r
561                                 The <literal>gather()</literal> method, called on the result object returned by the\r
562                                 <literal>request()</literal> method, iterates over all of the possible results from the result\r
563                                 object and returns a single variable.\r
564                                 </simpara>\r
565                                 </callout>\r
566                                 <callout arearefs="CO4-5">\r
567                                 <simpara>\r
568                                 This <literal>request()</literal> call passes two strings to the <literal>opensrf.simple-text.split</literal>\r
569                                 method defined by the <literal>opensrf.simple-text</literal> OpenSRF service and returns (via\r
570                                 <literal>gather()</literal>) a reference to an array of results.\r
571                                 </simpara>\r
572                                 </callout>\r
573                                 <callout arearefs="CO4-6">\r
574                                 <simpara>\r
575                                 The <literal>opensrf.simple-text.split()</literal> method is a streaming method that\r
576                                 returns an array of results with one element per <literal>recv()</literal> call on the\r
577                                 result object. We could use the <literal>gather()</literal> method to retrieve all of the\r
578                                 results in a single array reference, but instead we simply iterate over\r
579                                 the result variable until there are no more results to retrieve.\r
580                                 </simpara>\r
581                                 </callout>\r
582                                 <callout arearefs="CO4-7">\r
583                                 <simpara>\r
584                                 While the <literal>gather()</literal> convenience method returns only the content of the\r
585                                 complete set of results for a given request, the <literal>recv()</literal> method returns an\r
586                                 OpenSRF result object with <literal>status</literal>, <literal>statusCode</literal>, and <literal>content</literal> fields as\r
587                                 we saw in <link linkend="OpenSRFOverHTTP">the HTTP results example</link>.\r
588                                 </simpara>\r
589                                 </callout>\r
590                                 <callout arearefs="CO4-8">\r
591                                 <simpara>\r
592                                 This <literal>request()</literal> call passes an array to the\r
593                                 <literal>opensrf.simple-text.statistics</literal> method defined by the <literal>opensrf.simple-text</literal>\r
594                                 OpenSRF service.\r
595                                 </simpara>\r
596                                 </callout>\r
597                                 <callout arearefs="CO4-9">\r
598                                 <simpara>\r
599                                 The result object returns a hash reference via <literal>gather()</literal>. The hash\r
600                                 contains the <literal>length</literal> and <literal>word_count</literal> keys we defined in the method.\r
601                                 </simpara>\r
602                                 </callout>\r
603                                 <callout arearefs="CO4-10">\r
604                                 <simpara>\r
605                                 The <literal>OpenSRF::AppSession-&gt;disconnect()</literal> method closes the XMPP client\r
606                                 connection and cleans up resources associated with the session.\r
607                                 </simpara>\r
608                                 </callout>\r
609                                 </calloutlist>\r
610                         </simplesect>\r
611                 </simplesect>\r
612                 <simplesect id="_accepting_and_returning_more_interesting_data_types">\r
613                         <title>Accepting and returning more interesting data types</title>\r
614                         <simpara>Of course, the example of accepting a single string and returning a single\r
615                         string is not very interesting. In real life, our applications tend to pass\r
616                         around multiple arguments, including arrays and hashes. Fortunately, OpenSRF\r
617                         makes that easy to deal with; in Perl, for example, returning a reference to\r
618                         the data type does the right thing. In the following example of a method that\r
619                         returns a list, we accept two arguments of type string: the string to be split,\r
620                         and the delimiter that should be used to split the string.</simpara>\r
621                         <formalpara><title>Basic text splitting method</title><para>\r
622 <programlisting language="perl" linenumbering="unnumbered">\r
623 sub text_split {\r
624     my $self = shift;\r
625     my $conn = shift;\r
626     my $text = shift;\r
627     my $delimiter = shift || ' ';\r
628 \r
629     my @split_text = split $delimiter, $text;\r
630     return \@split_text;\r
631 }\r
632 \r
633 __PACKAGE__-&gt;register_method(\r
634     method    =&gt; 'text_split',\r
635     api_name  =&gt; 'opensrf.simple-text.split'\r
636 );\r
637 </programlisting>\r
638                         </para></formalpara>\r
639                         <simpara>We simply return a reference to the list, and OpenSRF does the rest of the work\r
640                         for us to convert the data into the language-independent format that is then\r
641                         returned to the caller. As a caller of a given method, you must rely on the\r
642                         documentation used to register to determine the data structures - if the developer has\r
643                         added the appropriate documentation.</simpara>\r
644                 </simplesect>\r
645                 <simplesect id="_accepting_and_returning_evergreen_objects">\r
646                         <title>Accepting and returning Evergreen objects</title>\r
647                         <simpara>OpenSRF is agnostic about objects; its role is to pass JSON back and forth\r
648                         between OpenSRF clients and services, and it allows the specific clients and\r
649                         services to define their own semantics for the JSON structures. On top of that\r
650                         infrastructure, Evergreen offers the fieldmapper: an object-relational mapper\r
651                         that provides a complete definition of all objects, their properties, their\r
652                         relationships to other objects, the permissions required to create, read,\r
653                         update, or delete objects of that type, and the database table or view on which\r
654                         they are based.</simpara>\r
655                         <simpara>The Evergreen fieldmapper offers a great deal of convenience for working with\r
656                         complex system objects beyond the basic mapping of classes to database\r
657                         schemas. Although the result is passed over the wire as a JSON object\r
658                         containing the indicated fields, fieldmapper-aware clients then turn those\r
659                         JSON objects into native objects with setter / getter methods for each field.</simpara>\r
660                         <simpara>All of this metadata about Evergreen objects is defined in the\r
661                         fieldmapper configuration file (<literal>/openils/conf/fm_IDL.xml</literal>), and access to\r
662                         these classes is provided by the <literal>open-ils.cstore</literal>, <literal>open-ils.pcrud</literal>, and\r
663                         <literal>open-ils.reporter-store</literal> OpenSRF services which parse the fieldmapper\r
664                         configuration file and dynamically register OpenSRF methods for creating,\r
665                         reading, updating, and deleting all of the defined classes.</simpara>\r
666                         <formalpara><title>Example fieldmapper class definition for "Open User Summary"</title><para>\r
667 <programlisting language="xml" linenumbering="unnumbered">\r
668 &lt;class id="mous" controller="open-ils.cstore open-ils.pcrud"\r
669  oils_obj:fieldmapper="money::open_user_summary"\r
670  oils_persist:tablename="money.open_usr_summary"\r
671  reporter:label="Open User Summary"&gt;                                <co id="CO5-1"/>\r
672     &lt;fields oils_persist:primary="usr" oils_persist:sequence=""&gt; <co id="CO5-2"/> \r
673         &lt;field name="balance_owed" reporter:datatype="money" /&gt;  <co id="CO5-3"/> \r
674         &lt;field name="total_owed" reporter:datatype="money" /&gt;\r
675         &lt;field name="total_paid" reporter:datatype="money" /&gt;\r
676         &lt;field name="usr" reporter:datatype="link"/&gt;\r
677     &lt;/fields&gt;\r
678     &lt;links&gt;\r
679         &lt;link field="usr" reltype="has_a" key="id" map="" class="au"/&gt;<co id="CO5-4"/> \r
680     &lt;/links&gt;\r
681     &lt;permacrud xmlns=""&gt;<co id="CO5-5"/> \r
682         &lt;actions&gt;\r
683             &lt;retrieve permission="VIEW_USER"&gt;<co id="CO5-6"/> \r
684                 &lt;context link="usr" field="home_ou"/&gt;<co id="CO5-7"/>\r
685             &lt;/retrieve&gt;\r
686         &lt;/actions&gt;\r
687     &lt;/permacrud&gt;\r
688 &lt;/class&gt;\r
689 </programlisting>\r
690                         </para></formalpara>\r
691                         <calloutlist>\r
692                         <callout arearefs="CO5-1">\r
693                         <simpara>\r
694                         The <literal>&lt;class&gt;</literal> element defines the class:\r
695                         </simpara>\r
696                         <itemizedlist>\r
697                         <listitem>\r
698                         <simpara>\r
699                         The <literal>id</literal> attribute defines the <emphasis>class hint</emphasis> that identifies the class both\r
700                         elsewhere in the fieldmapper configuration file, such as in the value of the\r
701                         <literal>field</literal> attribute of the <literal>&lt;link&gt;</literal> element, and in the JSON object itself when\r
702                         it is instantiated. For example, an "Open User Summary" JSON object would have\r
703                         the top level property of <literal>"__c":"mous"</literal>.\r
704                         </simpara>\r
705                         </listitem>\r
706                         <listitem>\r
707                         <simpara>\r
708                         The <literal>controller</literal> attribute identifies the services that have direct access\r
709                         to this class. If <literal>open-ils.pcrud</literal> is not listed, for example, then there is\r
710                         no means to directly access members of this class through a public service.\r
711                         </simpara>\r
712                         </listitem>\r
713                         <listitem>\r
714                         <simpara>\r
715                         The <literal>oils_obj:fieldmapper</literal> attribute defines the name of the Perl\r
716                         fieldmapper class that will be dynamically generated to provide setter and\r
717                         getter methods for instances of the class.\r
718                         </simpara>\r
719                         </listitem>\r
720                         <listitem>\r
721                         <simpara>\r
722                         The <literal>oils_persist:tablename</literal> attribute identifies the schema name and table\r
723                         name of the database table that stores the data that represents the instances\r
724                         of this class. In this case, the schema is <literal>money</literal> and the table is\r
725                         <literal>open_usr_summary</literal>.\r
726                         </simpara>\r
727                         </listitem>\r
728                         <listitem>\r
729                         <simpara>\r
730                         The <literal>reporter:label</literal> attribute defines a human-readable name for the class\r
731                         used in the reporting interface to identify the class. These names are defined\r
732                         in English in the fieldmapper configuration file; however, they are extracted\r
733                         so that they can be translated and served in the user&#8217;s language of choice.\r
734                         </simpara>\r
735                         </listitem>\r
736                         </itemizedlist>\r
737                         </callout>\r
738                         <callout arearefs="CO5-2">\r
739                         <simpara>\r
740                         The <literal>&lt;fields&gt;</literal> element lists all of the fields that belong to the object.\r
741                         </simpara>\r
742                         <itemizedlist>\r
743                         <listitem>\r
744                         <simpara>\r
745                         The <literal>oils_persist:primary</literal> attribute identifies the field that acts as the\r
746                         primary key for the object; in this case, the field with the name <literal>usr</literal>.\r
747                         </simpara>\r
748                         </listitem>\r
749                         <listitem>\r
750                         <simpara>\r
751                         The <literal>oils_persist:sequence</literal> attribute identifies the sequence object\r
752                         (if any) in this database provides values for new instances of this class. In\r
753                         this case, the primary key is defined by a field that is linked to a different\r
754                         table, so no sequence is used to populate these instances.\r
755                         </simpara>\r
756                         </listitem>\r
757                         </itemizedlist>\r
758                         </callout>\r
759                         <callout arearefs="CO5-3">\r
760                         <simpara>\r
761                         Each <literal>&lt;field&gt;</literal> element defines a single field with the following attributes:\r
762                         </simpara>\r
763                         <itemizedlist>\r
764                         <listitem>\r
765                         <simpara>\r
766                         The <literal>name</literal> attribute identifies the column name of the field in the\r
767                         underlying database table as well as providing a name for the setter / getter\r
768                         method that can be invoked in the JSON or native version of the object.\r
769                         </simpara>\r
770                         </listitem>\r
771                         <listitem>\r
772                         <simpara>\r
773                         The <literal>reporter:datatype</literal> attribute defines how the reporter should treat\r
774                         the contents of the field for the purposes of querying and display.\r
775                         </simpara>\r
776                         </listitem>\r
777                         <listitem>\r
778                         <simpara>\r
779                         The <literal>reporter:label</literal> attribute can be used to provide a human-readable name\r
780                         for each field; without it, the reporter falls back to the value of the <literal>name</literal>\r
781                         attribute.\r
782                         </simpara>\r
783                         </listitem>\r
784                         </itemizedlist>\r
785                         </callout>\r
786                         <callout arearefs="CO5-4">\r
787                         <simpara>\r
788                         The <literal>&lt;links&gt;</literal> element contains a set of zero or more <literal>&lt;link&gt;</literal> elements,\r
789                         each of which defines a relationship between the class being described and\r
790                         another class.\r
791                         </simpara>\r
792                         <itemizedlist>\r
793                         <listitem>\r
794                         <simpara>\r
795                         The <literal>field</literal> attribute identifies the field named in this class that links\r
796                         to the external class.\r
797                         </simpara>\r
798                         </listitem>\r
799                         <listitem>\r
800                         <simpara>\r
801                         The <literal>reltype</literal> attribute identifies the kind of relationship between the\r
802                         classes; in the case of <literal>has_a</literal>, each value in the <literal>usr</literal> field is guaranteed\r
803                         to have a corresponding value in the external class.\r
804                         </simpara>\r
805                         </listitem>\r
806                         <listitem>\r
807                         <simpara>\r
808                         The <literal>key</literal> attribute identifies the name of the field in the external\r
809                         class to which this field links.\r
810                         </simpara>\r
811                         </listitem>\r
812                         <listitem>\r
813                         <simpara>\r
814                         The rarely-used <literal>map</literal> attribute identifies a second class to which\r
815                         the external class links; it enables this field to define a direct\r
816                         relationship to an external class with one degree of separation, to\r
817                         avoid having to retrieve all of the linked members of an intermediate\r
818                         class just to retrieve the instances from the actual desired target class.\r
819                         </simpara>\r
820                         </listitem>\r
821                         <listitem>\r
822                         <simpara>\r
823                         The <literal>class</literal> attribute identifies the external class to which this field\r
824                         links.\r
825                         </simpara>\r
826                         </listitem>\r
827                         </itemizedlist>\r
828                         </callout>\r
829                         <callout arearefs="CO5-5">\r
830                         <simpara>\r
831                         The <literal>&lt;permacrud&gt;</literal> element defines the permissions that must have been\r
832                         granted to a user to operate on instances of this class.\r
833                         </simpara>\r
834                         </callout>\r
835                         <callout arearefs="CO5-6">\r
836                         <simpara>\r
837                         The <literal>&lt;retrieve&gt;</literal> element is one of four possible children of the\r
838                         <literal>&lt;actions&gt;</literal> element that define the permissions required for each action:\r
839                         create, retrieve, update, and delete.\r
840                         </simpara>\r
841                         <itemizedlist>\r
842                         <listitem>\r
843                         <simpara>\r
844                         The <literal>permission</literal> attribute identifies the name of the permission that must\r
845                         have been granted to the user to perform the action.\r
846                         </simpara>\r
847                         </listitem>\r
848                         <listitem>\r
849                         <simpara>\r
850                         The <literal>contextfield</literal> attribute, if it exists, defines the field in this class\r
851                         that identifies the library within the system for which the user must have\r
852                         prvileges to work. If a user has been granted a given permission, but has not been\r
853                         granted privileges to work at a given library, they can not perform the action\r
854                         at that library.\r
855                         </simpara>\r
856                         </listitem>\r
857                         </itemizedlist>\r
858                         </callout>\r
859                         <callout arearefs="CO5-7">\r
860                         <simpara>\r
861                         The rarely-used <literal>&lt;context&gt;</literal> element identifies a linked field (<literal>link</literal>\r
862                         attribute) in this class which links to an external class that holds the field\r
863                         (<literal>field</literal> attribute) that identifies the library within the system for which the\r
864                         user must have privileges to work.\r
865                         </simpara>\r
866                         </callout>\r
867                         </calloutlist>\r
868                         <simpara>When you retrieve an instance of a class, you can ask for the result to\r
869                         <emphasis>flesh</emphasis> some or all of the linked fields of that class, so that the linked\r
870                         instances are returned embedded directly in your requested instance. In that\r
871                         same request you can ask for the fleshed instances to in turn have their linked\r
872                         fields fleshed. By bundling all of this into a single request and result\r
873                         sequence, you can avoid the network overhead of requiring the client to request\r
874                         the base object, then request each linked object in turn.</simpara>\r
875                         <simpara>You can also iterate over a collection of instances and set the automatically\r
876                         generated <literal>isdeleted</literal>, <literal>isupdated</literal>, or <literal>isnew</literal> properties to indicate that\r
877                         the given instance has been deleted, updated, or created respectively.\r
878                         Evergreen can then act in batch mode over the collection to perform the\r
879                         requested actions on any of the instances that have been flagged for action.</simpara>\r
880                 </simplesect>\r
881                 <simplesect id="_returning_streaming_results">\r
882                         <title>Returning streaming results</title>\r
883                         <simpara>In the previous implementation of the <literal>opensrf.simple-text.split</literal> method, we\r
884                         returned a reference to the complete array of results. For small values being\r
885                         delivered over the network, this is perfectly acceptable, but for large sets of\r
886                         values this can pose a number of problems for the requesting client. Consider a\r
887                         service that returns a set of bibliographic records in response to a query like\r
888                         "all records edited in the past month"; if the underlying database is\r
889                         relatively active, that could result in thousands of records being returned as\r
890                         a single network request. The client would be forced to block until all of the\r
891                         results are returned, likely resulting in a significant delay, and depending on\r
892                         the implementation, correspondingly large amounts of memory might be consumed\r
893                         as all of the results are read from the network in a single block.</simpara>\r
894                         <simpara>OpenSRF offers a solution to this problem. If the method returns results that\r
895                         can be divided into separate meaningful units, you can register the OpenSRF\r
896                         method as a streaming method and enable the client to loop over the results one\r
897                         unit at a time until the method returns no further results. In addition to\r
898                         registering the method with the provided name, OpenSRF also registers an additional\r
899                         method with <literal>.atomic</literal> appended to the method name. The <literal>.atomic</literal> variant gathers\r
900                         all of the results into a single block to return to the client, giving the caller\r
901                         the ability to choose either streaming or atomic results from a single method\r
902                         definition.</simpara>\r
903                         <simpara>In the following example, the text splitting method has been reimplemented to\r
904                         support streaming; very few changes are required:</simpara>\r
905                         <formalpara><title>Text splitting method - streaming mode</title><para>\r
906 <programlisting language="perl" linenumbering="unnumbered">\r
907 sub text_split {\r
908     my $self = shift;\r
909     my $conn = shift;\r
910     my $text = shift;\r
911     my $delimiter = shift || ' ';\r
912 \r
913     my @split_text = split $delimiter, $text;\r
914     foreach my $string (@split_text) { <co id="CO6-1"/>\r
915         $conn-&gt;respond($string);\r
916     }\r
917     return undef;\r
918 }\r
919 \r
920 __PACKAGE__-&gt;register_method(\r
921     method    =&gt; 'text_split',\r
922     api_name  =&gt; 'opensrf.simple-text.split',\r
923     stream    =&gt; 1<co id="CO6-2"/>\r
924 );\r
925 </programlisting>\r
926                         </para></formalpara>\r
927                         <calloutlist>\r
928                         <callout arearefs="CO6-1">\r
929                         <simpara>\r
930                         Rather than returning a reference to the array, a streaming method loops\r
931                         over the contents of the array and invokes the <literal>respond()</literal> method of the\r
932                         connection object on each element of the array.\r
933                         </simpara>\r
934                         </callout>\r
935                         <callout arearefs="CO6-2">\r
936                         <simpara>\r
937                         Registering the method as a streaming method instructs OpenSRF to also\r
938                         register an atomic variant (<literal>opensrf.simple-text.split.atomic</literal>).\r
939                         </simpara>\r
940                         </callout>\r
941                         </calloutlist>\r
942                 </simplesect>\r
943                 <simplesect id="_error_warning_info_debug">\r
944                         <title>Error! Warning! Info! Debug!</title>\r
945                         <simpara>As hard as it may be to believe, it is true: applications sometimes do not\r
946                         behave in the expected manner, particularly when they are still under\r
947                         development. The service language bindings for OpenSRF include integrated\r
948                         support for logging messages at the levels of ERROR, WARNING, INFO, DEBUG, and\r
949                         the extremely verbose INTERNAL to either a local file or to a syslogger\r
950                         service. The destination of the log files, and the level of verbosity to be\r
951                         logged, is set in the <literal>opensrf_core.xml</literal> configuration file. To add logging to\r
952                         our Perl example, we just have to add the <literal>OpenSRF::Utils::Logger</literal> package to our\r
953                         list of used Perl modules, then invoke the logger at the desired logging level.</simpara>\r
954                         <simpara>You can include many calls to the OpenSRF logger; only those that are higher\r
955                         than your configured logging level will actually hit the log. The following\r
956                         example exercises all of the available logging levels in OpenSRF:</simpara>\r
957 <programlisting language="perl" linenumbering="unnumbered">\r
958 use OpenSRF::Utils::Logger;\r
959 my $logger = OpenSRF::Utils::Logger;\r
960 # some code in some function\r
961 {\r
962     $logger-&gt;error("Hmm, something bad DEFINITELY happened!");\r
963     $logger-&gt;warn("Hmm, something bad might have happened.");\r
964     $logger-&gt;info("Something happened.");\r
965     $logger-&gt;debug("Something happened; here are some more details.");\r
966     $logger-&gt;internal("Something happened; here are all the gory details.")\r
967 }\r
968 </programlisting>\r
969                         <simpara>If you call the mythical OpenSRF method containing the preceding OpenSRF logger\r
970                         statements on a system running at the default logging level of INFO, you will\r
971                         only see the INFO, WARN, and ERR messages, as follows:</simpara>\r
972                         <formalpara><title>Results of logging calls at the default level of INFO</title><para>\r
973 <screen>\r
974 [2010-03-17 22:27:30] opensrf.simple-text [ERR] Hmm, something bad DEFINITELY happened!\r
975 [2010-03-17 22:27:30] opensrf.simple-text [] Hmm, something bad might have happened.\r
976 [2010-03-17 22:27:30] opensrf.simple-text [] Something happened.\r
977 </screen>\r
978                         </para></formalpara>\r
979                         <simpara>If you then increase the the logging level to INTERNAL (5), the logs will\r
980                         contain much more information, as follows:</simpara>\r
981                         <formalpara><title>Results of logging calls at the default level of INTERNAL</title><para>\r
982 <screen>\r
983 [2010-03-17 22:48:11] opensrf.simple-text [ERR] Hmm, something bad DEFINITELY happened!\r
984 [2010-03-17 22:48:11] opensrf.simple-text [] Hmm, something bad might have happened.\r
985 [2010-03-17 22:48:11] opensrf.simple-text [] Something happened.\r
986 [2010-03-17 22:48:11] opensrf.simple-text [] Something happened; here are some more details.\r
987 [2010-03-17 22:48:11] opensrf.simple-text [] Something happened; here are all the gory details.\r
988 [2010-03-17 22:48:11] opensrf.simple-text [ERR] Resolver did not find a cache hit\r
989 [2010-03-17 22:48:21] opensrf.simple-text [] Stored opensrf.simple-text.test_cache.masaa =&gt; "here" in memcached server\r
990 [2010-03-17 22:48:21] opensrf.simple-text [] Coderef for [OpenSRF::Application::Demo::SimpleText::test_cache]...\r
991 [2010-03-17 22:48:21] opensrf.simple-text [] A top level Request object is responding de nada\r
992 [2010-03-17 22:48:21] opensrf.simple-text [] Method duration for [opensrf.simple-text.test_cache]:  10.005\r
993 [2010-03-17 22:48:21] opensrf.simple-text [] Calling queue_wait(0)\r
994 [2010-03-17 22:48:21] opensrf.simple-text [] Resending...0\r
995 [2010-03-17 22:48:21] opensrf.simple-text [] In send\r
996 [2010-03-17 22:48:21] opensrf.simple-text [] AppSession sending RESULT to opensrf@private.localhost/... \r
997 [2010-03-17 22:48:21] opensrf.simple-text [] AppSession sending STATUS to opensrf@private.localhost/... \r
998 ...\r
999 </screen>\r
1000                         </para></formalpara>\r
1001                         <simpara>To see everything that is happening in OpenSRF, try leaving your logging level\r
1002                         set to INTERNAL for a few minutes - just ensure that you have a lot of free disk\r
1003                         space available if you have a moderately busy system!</simpara>\r
1004                 </simplesect>\r
1005                 <simplesect id="_caching_results_one_secret_of_scalability">\r
1006                         <title>Caching results: one secret of scalability</title>\r
1007                         <simpara>If you have ever used an application that depends on a remote Web service\r
1008                         outside of your control&#8201;&#8212;&#8201;say, if you need to retrieve results from a\r
1009                         microblogging service&#8201;&#8212;&#8201;you know the pain of latency and dependability (or the\r
1010                         lack thereof). To improve the response time for OpenSRF services, you can take\r
1011                         advantage of the support offered by the <literal>OpenSRF::Utils::Cache</literal> module for\r
1012                         communicating with a local instance or cluster of <literal>memcache</literal> daemons to store\r
1013                         and retrieve persistent values. The following example demonstrates caching\r
1014                         by sleeping for 10 seconds the first time it receives a given cache key and\r
1015                         cannot retrieve a corresponding value from the cache:</simpara>\r
1016                         <formalpara><title>Simple caching OpenSRF service</title><para>\r
1017 <programlisting language="perl" linenumbering="unnumbered">\r
1018 use OpenSRF::Utils::Cache;<co id="CO7-1"/>\r
1019 sub test_cache {\r
1020     my $self = shift;\r
1021     my $conn = shift;\r
1022     my $test_key = shift;\r
1023     my $cache = OpenSRF::Utils::Cache-&gt;new('global'); <co id="CO7-2"/>\r
1024     my $cache_key = "opensrf.simple-text.test_cache.$test_key"; <co id="CO7-3"/>\r
1025     my $result = $cache-&gt;get_cache($cache_key) || undef; <co id="CO7-4"/>\r
1026     if ($result) {\r
1027         $logger-&gt;info("Resolver found a cache hit");\r
1028         return $result;\r
1029     }\r
1030     sleep 10; <co id="CO7-5"/>\r
1031     my $cache_timeout = 300; <co id="CO7-6"/>\r
1032     $cache-&gt;put_cache($cache_key, "here", $cache_timeout); <co id="CO7-7"/>\r
1033     return "There was no cache hit.";\r
1034 }\r
1035 </programlisting>\r
1036                         </para></formalpara>\r
1037                         <calloutlist>\r
1038                         <callout arearefs="CO7-1">\r
1039                         <simpara>\r
1040                         The OpenSRF::Utils::Cache module provides access to the built-in caching\r
1041                         support in OpenSRF.\r
1042                         </simpara>\r
1043                         </callout>\r
1044                         <callout arearefs="CO7-2">\r
1045                         <simpara>\r
1046                         The constructor for the cache object accepts a single argument to define\r
1047                         the cache type for the object. Each cache type can use a separate <literal>memcache</literal>\r
1048                         server to keep the caches separated. Most Evergreen services use the <literal>global</literal>\r
1049                         cache, while the <literal>anon</literal> cache is used for Web sessions.\r
1050                         </simpara>\r
1051                         </callout>\r
1052                         <callout arearefs="CO7-3">\r
1053                         <simpara>\r
1054                         The cache key is simply a string that uniquely identifies the value you\r
1055                         want to store or retrieve. This line creates a cache key based on the OpenSRF\r
1056                         method name and request input value.\r
1057                         </simpara>\r
1058                         </callout>\r
1059                         <callout arearefs="CO7-4">\r
1060                         <simpara>\r
1061                         The <literal>get_cache()</literal> method checks to see if the cache key already exists. If\r
1062                         a matching key is found, the service immediately returns the stored value.\r
1063                         </simpara>\r
1064                         </callout>\r
1065                         <callout arearefs="CO7-5">\r
1066                         <simpara>\r
1067                         If the cache key does not exist, the code sleeps for 10 seconds to\r
1068                         simulate a call to a slow remote Web service or an intensive process.\r
1069                         </simpara>\r
1070                         </callout>\r
1071                         <callout arearefs="CO7-6">\r
1072                         <simpara>\r
1073                         The <literal>$cache_timeout</literal> variable represents a value for the lifetime of the\r
1074                         cache key in seconds.\r
1075                         </simpara>\r
1076                         </callout>\r
1077                         <callout arearefs="CO7-7">\r
1078                         <simpara>\r
1079                         After the code retrieves its value (or, in the case of this example,\r
1080                         finishes sleeping), it creates the cache entry by calling the <literal>put_cache()</literal>\r
1081                         method. The method accepts three arguments: the cache key, the value to be\r
1082                         stored ("here"), and the timeout value in seconds to ensure that we do not\r
1083                         return stale data on subsequent calls.\r
1084                         </simpara>\r
1085                         </callout>\r
1086                         </calloutlist>\r
1087                 </simplesect>\r
1088                 <simplesect id="_initializing_the_service_and_its_children_child_labour">\r
1089                         <title>Initializing the service and its children: child labour</title>\r
1090                         <simpara>When an OpenSRF service is started, it looks for a procedure called\r
1091                         <literal>initialize()</literal> to set up any global variables shared by all of the children of\r
1092                         the service. The <literal>initialize()</literal> procedure is typically used to retrieve\r
1093                         configuration settings from the <literal>opensrf.xml</literal> file.</simpara>\r
1094                         <simpara>An OpenSRF service spawns one or more children to actually do the work\r
1095                         requested by callers of the service. For every child process an OpenSRF service\r
1096                         spawns, the child process clones the parent environment and then each child\r
1097                         process runs the <literal>child_init()</literal> process (if any) defined in the OpenSRF service\r
1098                         to initialize any child-specific settings.</simpara>\r
1099                         <simpara>When the OpenSRF service kills a child process, it invokes the <literal>child_exit()</literal>\r
1100                         procedure (if any) to clean up any resources associated with the child process.\r
1101                         Similarly, when the OpenSRF service is stopped, it calls the <literal>DESTROY()</literal>\r
1102                         procedure to clean up any remaining resources.</simpara>\r
1103                 </simplesect>\r
1104                 <simplesect id="_retrieving_configuration_settings">\r
1105                         <title>Retrieving configuration settings</title>\r
1106                         <simpara>The settings for OpenSRF services are maintained in the <literal>opensrf.xml</literal> XML\r
1107                         configuration file. The structure of the XML document consists of a root\r
1108                         element <literal>&lt;opensrf&gt;</literal> containing two child elements:</simpara>\r
1109                         <itemizedlist>\r
1110                         <listitem>\r
1111                         <simpara>\r
1112                         The <literal>&lt;default&gt;</literal> element contains an <literal>&lt;apps&gt;</literal> element describing all\r
1113                         OpenSRF services running on this system&#8201;&#8212;&#8201;see <xref linkend="serviceRegistration"/> --, as\r
1114                         well as any other arbitrary XML descriptions required for global configuration\r
1115                         purposes. For example, Evergreen uses this section for email notification and\r
1116                         inter-library patron privacy settings.\r
1117                         </simpara>\r
1118                         </listitem>\r
1119                         <listitem>\r
1120                         <simpara>\r
1121                         The <literal>&lt;hosts&gt;</literal> element contains one element per host that participates in\r
1122                         this OpenSRF system. Each host element must include an <literal>&lt;activeapps&gt;</literal> element\r
1123                         that lists all of the services to start on this host when the system starts\r
1124                         up. Each host element can optionally override any of the default settings.\r
1125                         </simpara>\r
1126                         </listitem>\r
1127                         </itemizedlist>\r
1128                         <simpara>OpenSRF includes a service named <literal>opensrf.settings</literal> to provide distributed\r
1129                         cached access to the configuration settings with a simple API:</simpara>\r
1130                         <itemizedlist>\r
1131                         <listitem>\r
1132                         <simpara>\r
1133                         <literal>opensrf.settings.default_config.get</literal> accepts zero arguments and returns\r
1134                         the complete set of default settings as a JSON document.\r
1135                         </simpara>\r
1136                         </listitem>\r
1137                         <listitem>\r
1138                         <simpara>\r
1139                         <literal>opensrf.settings.host_config.get</literal> accepts one argument (hostname) and\r
1140                         returns the complete set of settings, as customized for that hostname, as a\r
1141                         JSON document.\r
1142                         </simpara>\r
1143                         </listitem>\r
1144                         <listitem>\r
1145                         <simpara>\r
1146                         <literal>opensrf.settings.xpath.get</literal> accepts one argument (an\r
1147                         <ulink url="">XPath</ulink> expression) and returns the portion of\r
1148                         the configuration file that matches the expression as a JSON document.\r
1149                         </simpara>\r
1150                         </listitem>\r
1151                         </itemizedlist>\r
1152                         <simpara>For example, to determine whether an Evergreen system uses the opt-in\r
1153                         support for sharing patron information between libraries, you could either\r
1154                         invoke the <literal>opensrf.settings.default_config.get</literal> method and parse the\r
1155                         JSON document to determine the value, or invoke the <literal>opensrf.settings.xpath.get</literal>\r
1156                         method with the XPath <literal>/opensrf/default/share/user/opt_in</literal> argument to\r
1157                         retrieve the value directly.</simpara>\r
1158                         <simpara>In practice, OpenSRF includes convenience libraries in all of its client\r
1159                         language bindings to simplify access to configuration values. C offers\r
1160                         osrfConfig.c, Perl offers <literal>OpenSRF::Utils::SettingsClient</literal>, Java offers\r
1161                         <literal>org.opensrf.util.SettingsClient</literal>, and Python offers <literal>osrf.set</literal>. These\r
1162                         libraries locally cache the configuration file to avoid network roundtrips for\r
1163                         every request and enable the developer to request specific values without\r
1164                         having to manually construct XPath expressions.</simpara>\r
1165                 </simplesect>\r
1166         </section>\r
1167         <section id="_getting_under_the_covers_with_opensrf">\r
1168                 <title>OpenSRF Communication Flows</title>\r
1169                 <simpara>Now that you have seen that it truly is easy to create an OpenSRF service, we\r
1170                 can take a look at what is going on under the covers to make all of this work\r
1171                 for you.</simpara>\r
1172                 <simplesect id="_get_on_the_messaging_bus_safely">\r
1173                         <title>Get on the messaging bus - safely</title>\r
1174                         <simpara>One of the core innovations of OpenSRF was to use the Extensible Messaging and\r
1175                         Presence Protocol (XMPP, more colloquially known as Jabber) as the messaging\r
1176                         bus that ties OpenSRF services together across servers. XMPP is an "XML\r
1177                         protocol for near-real-time messaging, presence, and request-response services"\r
1178                         (<ulink url=""></ulink>) that OpenSRF relies on to handle most of\r
1179                         the complexity of networked communications.  OpenSRF requres an XMPP server\r
1180                         that supports multiple domains such as <ulink url="">ejabberd</ulink>.\r
1181                         Multiple domain support means that a single server can support XMPP virtual\r
1182                         hosts with separate sets of users and access privileges per domain. By\r
1183                         routing communications through separate public and private XMPP domains,\r
1184                         OpenSRF services gain an additional layer of security.</simpara>\r
1185                         <simpara>The <ulink url="">OpenSRF\r
1186                         installation documentation</ulink> instructs you to create two separate hostnames\r
1187                         (<literal>private.localhost</literal> and <literal>public.localhost</literal>) to use as XMPP domains.  OpenSRF\r
1188                         can control access to its services based on the domain of the client and\r
1189                         whether a given service allows access from clients on the public domain.  When\r
1190                         you start OpenSRF, the first XMPP clients that connect to the XMPP server are\r
1191                         the OpenSRF public and private <emphasis>routers</emphasis>. OpenSRF routers maintain a list of\r
1192                         available services and connect clients to available services. When an OpenSRF\r
1193                         service starts, it establishes a connection to the XMPP server and registers\r
1194                         itself with the private router. The OpenSRF configuration contains a list of\r
1195                         public OpenSRF services, each of which must also register with the public\r
1196                         router.</simpara>\r
1197                 </simplesect>\r
1198                 <simplesect id="_opensrf_communication_flows_over_xmpp">\r
1199                         <title>OpenSRF communication flows over XMPP</title>\r
1200                         <simpara>In a minimal OpenSRF deployment, two XMPP users named "router" connect to the\r
1201                         XMPP server, with one connected to the private XMPP domain and one connected to\r
1202                         the public XMPP domain. Similarly, two XMPP users named "opensrf" connect to\r
1203                         the XMPP server via the private and public XMPP domains. When an OpenSRF\r
1204                         service is started, it uses the "opensrf" XMPP user to advertise its\r
1205                         availability with the corresponding router on that XMPP domain; the XMPP server\r
1206                         automatically assigns a Jabber ID (<emphasis>JID</emphasis>) based on the client hostname to each\r
1207                         service&#8217;s listener process and each connected drone process waiting to carry\r
1208                         out requests. When an OpenSRF router receives a request to invoke a method on a\r
1209                         given service, it connects the requester to the next available listener in the\r
1210                         list of registered listeners for that service.</simpara>\r
1211                         <simpara>Services and clients connect to the XMPP server using a single set of XMPP\r
1212                         client credentials (for example, <literal>opensrf@private.localhost</literal>), but use XMPP\r
1213                         resource identifiers to differentiate themselves in the JID for each\r
1214                         connection. For example, the JID for a copy of the <literal>opensrf.simple-text</literal>\r
1215                         service with process ID <literal>6285</literal> that has connected to the <literal>private.localhost</literal>\r
1216                         domain using the <literal>opensrf</literal> XMPP client credentials could be\r
1217                         <literal>opensrf@private.localhost/opensrf.simple-text_drone_at_localhost_6285</literal>.  By\r
1218                         convention, the user name for OpenSRF clients is <literal>opensrf</literal>, and the user name\r
1219                         for OpenSRF routers is <literal>router</literal>, so the XMPP server for OpenSRF will have four\r
1220                         separate users registered:\r
1221                           * <literal>opensrf@private.localhost</literal> is an OpenSRF client that connects with these\r
1222                         credentials and which can access any OpenSRF service.\r
1223                           * <literal>opensrf@public.localhost</literal> is an OpenSRF client that connects with these\r
1224                         credentials and which can only access OpenSRF services that have registered\r
1225                         with the public router.\r
1226                           * <literal>router@private.localhost</literal> is the private OpenSRF router with which all\r
1227                         services register.\r
1228                           * <literal>router@public.localhost</literal> is the public OpenSRF router with which only\r
1229                         services that must be publicly accessible register.</simpara>\r
1230                         <simpara>All OpenSRF services automatically register themselves with the private XMPP\r
1231                         domain, but only those services that register themselves with the public XMPP\r
1232                         domain can be invoked from public OpenSRF clients.  The OpenSRF client and\r
1233                         router user names, passwords, and domain names, along with the list of services\r
1234                         that should be public, are contained in the <literal>opensrf_core.xml</literal> configuration\r
1235                         file.</simpara>\r
1236                 </simplesect>\r
1237                 <simplesect id="OpenSRFOverHTTP">\r
1238                         <title>OpenSRF communication flows over HTTP</title>\r
1239                         <simpara>In some contexts, access to a full XMPP client is not a practical option. For\r
1240                         example, while XMPP clients have been implemented in JavaScript, you might\r
1241                         be concerned about browser compatibility and processing overhead - or you might\r
1242                         want to issue OpenSRF requests from the command line with <literal>curl</literal>. Fortunately,\r
1243                         any OpenSRF service registered with the public router is accessible via the\r
1244                         OpenSRF HTTP Translator. The OpenSRF HTTP Translator implements the\r
1245                         <ulink url="">OpenSRF-over-HTTP\r
1246                         proposed specification</ulink> as an Apache module that translates HTTP requests into\r
1247                         OpenSRF requests and returns OpenSRF results as HTTP results to the initiating\r
1248                         HTTP client.</simpara>\r
1249                         <formalpara><title>Issuing an HTTP POST request to an OpenSRF method via the OpenSRF HTTP Translator</title><para>\r
1250 <programlisting language="bash" linenumbering="unnumbered">\r
1251 # curl request broken up over multiple lines for legibility\r
1252 curl -H "X-OpenSRF-service: opensrf.simple-text"<co id="CO8-1"/>\r
1253     --data 'osrf-msg=[  \<co id="CO8-2"/>\r
1254         {"__c":"osrfMessage","__p":{"threadTrace":0,"locale":"en-CA", <co id="CO8-3"/>\r
1255             "type":"REQUEST","payload": {"__c":"osrfMethod","__p": \r
1256                 {"method":"opensrf.simple-text.reverse","params":["foobar"]}   \r
1257             }}                                                                  \r
1258         }]'                                                                    \r
1259 http://localhost/osrf-http-translator <co id="CO8-4"/>\r
1260 </programlisting>\r
1261                         </para></formalpara>\r
1262                         <calloutlist>\r
1263                         <callout arearefs="CO8-1">\r
1264                         <simpara>\r
1265                         The <literal>X-OpenSRF-service</literal> header identifies the OpenSRF service of interest.\r
1266                         </simpara>\r
1267                         </callout>\r
1268                         <callout arearefs="CO8-2">\r
1269                         <simpara>\r
1270                         The POST request consists of a single parameter, the <literal>osrf-msg</literal> value,\r
1271                         which contains a JSON array.\r
1272                         </simpara>\r
1273                         </callout>\r
1274                         <callout arearefs="CO8-3">\r
1275                         <simpara>\r
1276                         The first object is an OpenSRF message (<literal>"__c":"osrfMessage"</literal>) with a set of\r
1277                         parameters (<literal>"__p":{}</literal>).\r
1278                         </simpara>\r
1279                         <itemizedlist>\r
1280                         <listitem>\r
1281                         <simpara>\r
1282                         The identifier for the request (<literal>"threadTrace":0</literal>); this value is echoed\r
1283                         back in the result.\r
1284                         </simpara>\r
1285                         </listitem>\r
1286                         <listitem>\r
1287                         <simpara>\r
1288                         The message type (<literal>"type":"REQUEST"</literal>).\r
1289                         </simpara>\r
1290                         </listitem>\r
1291                         <listitem>\r
1292                         <simpara>\r
1293                         The locale for the message; if the OpenSRF method is locale-sensitive, it\r
1294                         can check the locale for each OpenSRF request and return different information\r
1295                         depending on the locale.\r
1296                         </simpara>\r
1297                         </listitem>\r
1298                         <listitem>\r
1299                         <simpara>\r
1300                         The payload of the message (<literal>"payload":{}</literal>) containing the OpenSRF method\r
1301                         request (<literal>"__c":"osrfMethod"</literal>) and its parameters (<literal>"__p:"{}</literal>).\r
1302                         </simpara>\r
1303                         <itemizedlist>\r
1304                         <listitem>\r
1305                         <simpara>\r
1306                         The method name for the request (<literal>"method":"opensrf.simple-text.reverse"</literal>).\r
1307                         </simpara>\r
1308                         </listitem>\r
1309                         <listitem>\r
1310                         <simpara>\r
1311                         A set of JSON parameters to pass to the method (<literal>"params":["foobar"]</literal>); in\r
1312                         this case, a single string <literal>"foobar"</literal>.\r
1313                         </simpara>\r
1314                         </listitem>\r
1315                         </itemizedlist>\r
1316                         </listitem>\r
1317                         </itemizedlist>\r
1318                         </callout>\r
1319                         <callout arearefs="CO8-4">\r
1320                         <simpara>\r
1321                         The URL on which the OpenSRF HTTP translator is listening,\r
1322                         <literal>/osrf-http-translator</literal> is the default location in the Apache example\r
1323                         configuration files shipped with the OpenSRF source, but this is configurable.\r
1324                         </simpara>\r
1325                         </callout>\r
1326                         </calloutlist>\r
1327                         <formalpara><title>Results from an HTTP POST request to an OpenSRF method via the OpenSRF HTTP Translator</title><para>\r
1328 <programlisting language="bash" linenumbering="unnumbered">\r
1329 # HTTP response broken up over multiple lines for legibility\r
1330 [{"__c":"osrfMessage","__p":   <co id="CO9-1"/>\r
1331     {"threadTrace":0, "payload": <co id="CO9-2"/>\r
1332         {"__c":"osrfResult","__p": <co id="CO9-3"/>\r
1333             {"status":"OK","content":"raboof","statusCode":200} <co id="CO9-4"/>\r
1334         },"type":"RESULT","locale":"en-CA" <co id="CO9-5"/>\r
1335     }\r
1336 },\r
1337 {"__c":"osrfMessage","__p":   <co id="CO9-6"/>\r
1338     {"threadTrace":0,"payload":  <co id="CO9-7"/>\r
1339         {"__c":"osrfConnectStatus","__p": <co id="CO9-8"/>\r
1340             {"status":"Request Complete","statusCode":205}<co id="CO9-9"/>\r
1341         },"type":"STATUS","locale":"en-CA"  <co id="CO9-10"/>\r
1342     }\r
1343 }]\r
1344 </programlisting>\r
1345                         </para></formalpara>\r
1346                         <calloutlist>\r
1347                         <callout arearefs="CO9-1">\r
1348                         <simpara>\r
1349                         The OpenSRF HTTP Translator returns an array of JSON objects in its\r
1350                         response. Each object in the response is an OpenSRF message\r
1351                         (<literal>"__c":"osrfMessage"</literal>) with a collection of response parameters (<literal>"__p":</literal>).\r
1352                         </simpara>\r
1353                         </callout>\r
1354                         <callout arearefs="CO9-2">\r
1355                         <simpara>\r
1356                         The OpenSRF message identifier (<literal>"threadTrace":0</literal>) confirms that this\r
1357                         message is in response to the request matching the same identifier.\r
1358                         </simpara>\r
1359                         </callout>\r
1360                         <callout arearefs="CO9-3">\r
1361                         <simpara>\r
1362                         The message includes a payload JSON object (<literal>"payload":</literal>) with an OpenSRF\r
1363                         result for the request (<literal>"__c":"osrfResult"</literal>).\r
1364                         </simpara>\r
1365                         </callout>\r
1366                         <callout arearefs="CO9-4">\r
1367                         <simpara>\r
1368                         The result includes a status indicator string (<literal>"status":"OK"</literal>), the content\r
1369                         of the result response - in this case, a single string "raboof"\r
1370                         (<literal>"content":"raboof"</literal>) - and an integer status code for the request\r
1371                         (<literal>"statusCode":200</literal>).\r
1372                         </simpara>\r
1373                         </callout>\r
1374                         <callout arearefs="CO9-5">\r
1375                         <simpara>\r
1376                         The message also includes the message type (<literal>"type":"RESULT"</literal>) and the\r
1377                         message locale (<literal>"locale":"en-CA"</literal>).\r
1378                         </simpara>\r
1379                         </callout>\r
1380                         <callout arearefs="CO9-6">\r
1381                         <simpara>\r
1382                         The second message in the set of results from the response.\r
1383                         </simpara>\r
1384                         </callout>\r
1385                         <callout arearefs="CO9-7">\r
1386                         <simpara>\r
1387                         Again, the message identifier confirms that this message is in response to\r
1388                         a particular request.\r
1389                         </simpara>\r
1390                         </callout>\r
1391                         <callout arearefs="CO9-8">\r
1392                         <simpara>\r
1393                         The payload of the message denotes that this message is an\r
1394                         OpenSRF connection status message (<literal>"__c":"osrfConnectStatus"</literal>), with some\r
1395                         information about the particular OpenSRF connection that was used for this\r
1396                         request.\r
1397                         </simpara>\r
1398                         </callout>\r
1399                         <callout arearefs="CO9-9">\r
1400                         <simpara>\r
1401                         The response parameters for an OpenSRF connection status message include a\r
1402                         verbose status (<literal>"status":"Request Complete"</literal>) and an integer status code for\r
1403                         the connection status (`"statusCode":205).\r
1404                         </simpara>\r
1405                         </callout>\r
1406                         <callout arearefs="CO9-10">\r
1407                         <simpara>\r
1408                         The message also includes the message type (<literal>"type":"RESULT"</literal>) and the\r
1409                         message locale (<literal>"locale":"en-CA"</literal>).\r
1410                         </simpara>\r
1411                         </callout>\r
1412                         </calloutlist>\r
1413                         <tip><simpara>Before adding a new public OpenSRF service, ensure that it does\r
1414                         not introduce privilege escalation or unchecked access to data. For example,\r
1415                         the Evergreen <literal>open-ils.cstore</literal> private service is an object-relational mapper\r
1416                         that provides read and write access to the entire Evergreen database, so it\r
1417                         would be catastrophic to expose that service publicly. In comparison, the\r
1418                         Evergreen <literal>open-ils.pcrud</literal> public service offers the same functionality as\r
1419                         <literal>open-ils.cstore</literal> to any connected HTTP client or OpenSRF client, but the\r
1420                         additional authentication and authorization layer in <literal>open-ils.pcrud</literal> prevents\r
1421                         unchecked access to Evergreen&#8217;s data.</simpara></tip>\r
1422                 </simplesect>\r
1423                 <simplesect id="_stateless_and_stateful_connections">\r
1424                         <title>Stateless and stateful connections</title>\r
1425                         <simpara>OpenSRF supports both <emphasis>stateless</emphasis> and <emphasis>stateful</emphasis> connections.  When an OpenSRF\r
1426                         client issues a <literal>REQUEST</literal> message in a <emphasis>stateless</emphasis> connection, the router\r
1427                         forwards the request to the next available service and the service returns the\r
1428                         result directly to the client.</simpara>\r
1429                         <formalpara><title>REQUEST flow in a stateless connection</title><para><inlinemediaobject>\r
1430                           <imageobject>\r
1431                           <imagedata fileref="REQUEST.png"/>\r
1432                           </imageobject>\r
1433                           <textobject><phrase>REQUEST flow in a stateless connection</phrase></textobject>\r
1434                         </inlinemediaobject></para></formalpara>\r
1435                         <simpara>When an OpenSRF client issues a <literal>CONNECT</literal> message to create a <emphasis>stateful</emphasis> conection, the\r
1436                         router returns the Jabber ID of the next available service to the client so\r
1437                         that the client can issue one or more <literal>REQUEST</literal> message directly to that\r
1438                         particular service and the service will return corresponding <literal>RESULT</literal> messages\r
1439                         directly to the client. Until the client issues a <literal>DISCONNECT</literal> message, that\r
1440                         particular service is only available to the requesting client. Stateful connections\r
1441                         are useful for clients that need to make many requests from a particular service,\r
1442                         as it avoids the intermediary step of contacting the router for each request, as\r
1443                         well as for operations that require a controlled sequence of commands, such as a\r
1444                         set of database INSERT, UPDATE, and DELETE statements within a transaction.</simpara>\r
1445                         <formalpara><title>CONNECT, REQUEST, and DISCONNECT flow in a stateful connection</title><para><inlinemediaobject>\r
1446                           <imageobject>\r
1447                           <imagedata fileref="CONNECT.png"/>\r
1448                           </imageobject>\r
1449                           <textobject><phrase>CONNECT</phrase></textobject>\r
1450                         </inlinemediaobject></para></formalpara>\r
1451                 </simplesect>\r
1452                 <simplesect id="_message_body_format">\r
1453                         <title>Message body format</title>\r
1454                         <simpara>OpenSRF was an early adopter of JavaScript Object Notation (JSON). While XMPP\r
1455                         is an XML protocol, the Evergreen developers recognized that the compactness of\r
1456                         the JSON format offered a significant reduction in bandwidth for the volume of\r
1457                         messages that would be generated in an application of that size. In addition,\r
1458                         the ability of languages such as JavaScript, Perl, and Python to generate\r
1459                         native objects with minimal parsing offered an attractive advantage over\r
1460                         invoking an XML parser for every message. Instead, the body of the XMPP message\r
1461                         is a simple JSON structure. For a simple request, like the following example\r
1462                         that simply reverses a string, it looks like a significant overhead: but we get\r
1463                         the advantages of locale support and tracing the request from the requester\r
1464                         through the listener and responder (drone).</simpara>\r
1465                         <formalpara><title>A request for opensrf.simple-text.reverse("foobar"):</title><para>\r
1466 <programlisting language="xml" linenumbering="unnumbered">\r
1467 &lt;message from='router@private.localhost/opensrf.simple-text'\r
1468   to='opensrf@private.localhost/opensrf.simple-text_listener_at_localhost_6275'\r
1469   router_from='opensrf@private.localhost/_karmic_126678.3719_6288'\r
1470   router_to='' router_class='' router_command='' osrf_xid=''\r
1471 &gt;\r
1472   &lt;thread&gt;1266781414.366573.12667814146288&lt;/thread&gt;\r
1473   &lt;body&gt;\r
1474 [\r
1475   {"__c":"osrfMessage","__p":\r
1476     {"threadTrace":"1","locale":"en-US","type":"REQUEST","payload":\r
1477       {"__c":"osrfMethod","__p":\r
1478         {"method":"opensrf.simple-text.reverse","params":["foobar"]}\r
1479       }\r
1480     }\r
1481   }\r
1482 ]\r
1483   &lt;/body&gt;\r
1484 &lt;/message&gt;\r
1485 </programlisting>\r
1486                         </para></formalpara>\r
1487                         <formalpara><title>A response from opensrf.simple-text.reverse("foobar")</title><para>\r
1488 <programlisting language="xml" linenumbering="unnumbered">\r
1489 &lt;message from='opensrf@private.localhost/opensrf.simple-text_drone_at_localhost_6285'\r
1490   to='opensrf@private.localhost/_karmic_126678.3719_6288'\r
1491   router_command='' router_class='' osrf_xid=''\r
1492 &gt;\r
1493   &lt;thread&gt;1266781414.366573.12667814146288&lt;/thread&gt;\r
1494   &lt;body&gt;\r
1495 [\r
1496   {"__c":"osrfMessage","__p":\r
1497     {"threadTrace":"1","payload":\r
1498       {"__c":"osrfResult","__p":\r
1499         {"status":"OK","content":"raboof","statusCode":200}\r
1500       } ,"type":"RESULT","locale":"en-US"}\r
1501   },\r
1502   {"__c":"osrfMessage","__p":\r
1503     {"threadTrace":"1","payload":\r
1504       {"__c":"osrfConnectStatus","__p":\r
1505         {"status":"Request Complete","statusCode":205}\r
1506       },"type":"STATUS","locale":"en-US"}\r
1507   }\r
1508 ]\r
1509   &lt;/body&gt;\r
1510 &lt;/message&gt;\r
1511 </programlisting>\r
1512                         </para></formalpara>\r
1513                         <simpara>The content of the <literal>&lt;body&gt;</literal> element of the OpenSRF request and result should\r
1514                         look familiar; they match the structure of the <link linkend="OpenSRFOverHTTP">OpenSRF over HTTP examples</link> that we previously dissected.</simpara>\r
1515                 </simplesect>\r
1516                 <simplesect id="_registering_opensrf_methods_in_depth">\r
1517                         <title>Registering OpenSRF methods in depth</title>\r
1518                         <simpara>Let&#8217;s explore the call to <literal>__PACKAGE__-&gt;register_method()</literal>; most of the members\r
1519                         of the hash are optional, and for the sake of brevity we omitted them in the\r
1520                         previous example. As we have seen in the results of the <link linkend="opensrfIntrospection">introspection call</link>, a\r
1521                         verbose registration method call is recommended to better enable the internal\r
1522                         documentation. Here is the complete set of members that you should pass to\r
1523                         <literal>__PACKAGE__-&gt;register_method()</literal>:</simpara>\r
1524                         <itemizedlist>\r
1525                         <listitem>\r
1526                         <simpara>\r
1527                         The <literal>method</literal> member specifies the name of the procedure in this module that is being registered as an OpenSRF method.\r
1528                         </simpara>\r
1529                         </listitem>\r
1530                         <listitem>\r
1531                         <simpara>\r
1532                         The <literal>api_name</literal> member specifies the invocable name of the OpenSRF method; by convention, the OpenSRF service name is used as the prefix.\r
1533                         </simpara>\r
1534                         </listitem>\r
1535                         <listitem>\r
1536                         <simpara>\r
1537                         The optional <literal>api_level</literal> member can be used for versioning the methods to allow the use of a deprecated API, but in practical use is always 1.\r
1538                         </simpara>\r
1539                         </listitem>\r
1540                         <listitem>\r
1541                         <simpara>\r
1542                         The optional <literal>argc</literal> member specifies the minimal number of arguments that the method expects.\r
1543                         </simpara>\r
1544                         </listitem>\r
1545                         <listitem>\r
1546                         <simpara>\r
1547                         The optional <literal>stream</literal> member, if set to any value, specifies that the method supports returning multiple values from a single call to \r
1548                         subsequent requests. OpenSRF automatically creates a corresponding method with ".atomic" appended to its name that returns the complete set of results in a \r
1549                         single request. Streaming methods are useful if you are returning hundreds of records and want to act on the results as they return.\r
1550                         </simpara>\r
1551                         </listitem>\r
1552                         <listitem>\r
1553                         <simpara>\r
1554                         The optional <literal>signature</literal> member is a hash that describes the method&#8217;s purpose, arguments, and return value.\r
1555                         </simpara>\r
1556                         <itemizedlist>\r
1557                         <listitem>\r
1558                         <simpara>\r
1559                         The <literal>desc</literal> member of the <literal>signature</literal> hash describes the method&#8217;s purpose.\r
1560                         </simpara>\r
1561                         </listitem>\r
1562                         <listitem>\r
1563                         <simpara>\r
1564                         The <literal>params</literal> member of the <literal>signature</literal> hash is an array of hashes in which each array element describes the corresponding method \r
1565                         argument in order.\r
1566                         </simpara>\r
1567                         <itemizedlist>\r
1568                         <listitem>\r
1569                         <simpara>\r
1570                         The <literal>name</literal> member of the argument hash specifies the name of the argument.\r
1571                         </simpara>\r
1572                         </listitem>\r
1573                         <listitem>\r
1574                         <simpara>\r
1575                         The <literal>desc</literal> member of the argument hash describes the argument&#8217;s purpose.\r
1576                         </simpara>\r
1577                         </listitem>\r
1578                         <listitem>\r
1579                         <simpara>\r
1580                         The <literal>type</literal> member of the argument hash specifies the data type of the argument: for example, string, integer, boolean, number, array, or hash.\r
1581                         </simpara>\r
1582                         </listitem>\r
1583                         </itemizedlist>\r
1584                         </listitem>\r
1585                         <listitem>\r
1586                         <simpara>\r
1587                         The <literal>return</literal> member of the <literal>signature</literal> hash is a hash that describes the return value of the method.\r
1588                         </simpara>\r
1589                         <itemizedlist>\r
1590                         <listitem>\r
1591                         <simpara>\r
1592                         The <literal>desc</literal> member of the <literal>return</literal> hash describes the return value.\r
1593                         </simpara>\r
1594                         </listitem>\r
1595                         <listitem>\r
1596                         <simpara>\r
1597                         The <literal>type</literal> member of the <literal>return</literal> hash specifies the data type of the return value: for example, string, integer, boolean, number, \r
1598                         array, or hash.\r
1599                         </simpara>\r
1600                         </listitem>\r
1601                         </itemizedlist>\r
1602                         </listitem>\r
1603                         </itemizedlist>\r
1604                         </listitem>\r
1605                         </itemizedlist>\r
1606                 </simplesect>\r
1607         </section>\r
1608         <section id="_evergreen_specific_opensrf_services">\r
1609                 <title>Evergreen-specific OpenSRF services</title>\r
1610                 <simpara>Evergreen is currently the primary showcase for the use of OpenSRF as an\r
1611                 application architecture. Evergreen 1.6.1 includes the following\r
1612                 set of OpenSRF services:</simpara>\r
1613                 <itemizedlist>\r
1614                 <listitem>\r
1615                 <simpara>\r
1616                 The <systemitem class="service"></systemitem> service supports common tasks for working with user\r
1617                      accounts and libraries.\r
1618                 </simpara>\r
1619                 </listitem>\r
1620                 <listitem>\r
1621                 <simpara>\r
1622                 The <systemitem class="service">open-ils.auth</systemitem> service supports authentication of Evergreen users.\r
1623                 </simpara>\r
1624                 </listitem>\r
1625                 <listitem>\r
1626                 <simpara>\r
1627                 The <systemitem class="service"></systemitem> service supports the management of reservations\r
1628                     for bookable items.\r
1629                 </simpara>\r
1630                 </listitem>\r
1631                 <listitem>\r
1632                 <simpara>\r
1633                 The <systemitem class="service"></systemitem> service supports common cataloging tasks, such as\r
1634                      creating, modifying, and merging bibliographic and authority records.\r
1635                 </simpara>\r
1636                 </listitem>\r
1637                 <listitem>\r
1638                 <simpara>\r
1639                 The <systemitem class="service">open-ils.circ</systemitem> service supports circulation tasks such as checking\r
1640                     out items and calculating due dates.\r
1641                 </simpara>\r
1642                 </listitem>\r
1643                 <listitem>\r
1644                 <simpara>\r
1645                 The <systemitem class="service">open-ils.collections</systemitem> service supports tasks that assist collections\r
1646                     agencies in contacting users with outstanding fines above a certain\r
1647                     threshold.\r
1648                 </simpara>\r
1649                 </listitem>\r
1650                 <listitem>\r
1651                 <simpara>\r
1652                 The <systemitem class="service">open-ils.cstore</systemitem> private service supports unrestricted access to\r
1653                     Evergreen fieldmapper objects.\r
1654                 </simpara>\r
1655                 </listitem>\r
1656                 <listitem>\r
1657                 <simpara>\r
1658                 The <systemitem class="service">open-ils.ingest</systemitem> private service supports tasks for importing\r
1659                     data such as bibliographic and authority records.\r
1660                 </simpara>\r
1661                 </listitem>\r
1662                 <listitem>\r
1663                 <simpara>\r
1664                 The <systemitem class="service">open-ils.pcrud</systemitem> service supports permission-based access to Evergreen\r
1665                     fieldmapper objects.\r
1666                 </simpara>\r
1667                 </listitem>\r
1668                 <listitem>\r
1669                 <simpara>\r
1670                 The <systemitem class="service">open-ils.penalty</systemitem> penalty service supports the calculation of\r
1671                     penalties for users, such as being blocked from further borrowing, for\r
1672                     conditions such as having too many items checked out or too many unpaid\r
1673                     fines.\r
1674                 </simpara>\r
1675                 </listitem>\r
1676                 <listitem>\r
1677                 <simpara>\r
1678                 The <systemitem class="service">open-ils.reporter</systemitem> service supports the creation and scheduling of\r
1679                     reports.\r
1680                 </simpara>\r
1681                 </listitem>\r
1682                 <listitem>\r
1683                 <simpara>\r
1684                 The <systemitem class="service">open-ils.reporter-store</systemitem> private service supports access to Evergreen\r
1685                     fieldmapper objects for the reporting service.\r
1686                 </simpara>\r
1687                 </listitem>\r
1688                 <listitem>\r
1689                 <simpara>\r
1690                 The <systemitem class="service"></systemitem> service supports searching across bibliographic\r
1691                     records, authority records, serial records, Z39.50 sources, and ZIP codes.\r
1692                 </simpara>\r
1693                 </listitem>\r
1694                 <listitem>\r
1695                 <simpara>\r
1696                 The <systemitem class="service"></systemitem> private service supports a deprecated method of\r
1697                     providing access to Evergreen fieldmapper objects. Implemented in Perl,\r
1698                     this service has largely been replaced by the much faster C-based\r
1699                     <literal>open-ils.cstore</literal> service.\r
1700                 </simpara>\r
1701                 </listitem>\r
1702                 <listitem>\r
1703                 <simpara>\r
1704                 The <systemitem class="service">open-ils.supercat</systemitem> service supports transforms of MARC records into\r
1705                     other formats, such as MODS, as well as providing Atom and RSS feeds and\r
1706                     SRU access.\r
1707                 </simpara>\r
1708                 </listitem>\r
1709                 <listitem>\r
1710                 <simpara>\r
1711                 The <systemitem class="service">open-ils.trigger</systemitem> private service supports event-based triggers for\r
1712                     actions such as overdue and holds available notification emails.\r
1713                 </simpara>\r
1714                 </listitem>\r
1715                 <listitem>\r
1716                 <simpara>\r
1717                 The <systemitem class="service">open-ils.vandelay</systemitem> service supports the import and export of batches of\r
1718                     bibliographic and authority records.\r
1719                 </simpara>\r
1720                 </listitem>\r
1721                 </itemizedlist>\r
1722                 <simpara>Of some interest is that the <systemitem class="service">open-ils.reporter-store</systemitem> and <systemitem class="service">open-ils.cstore</systemitem>\r
1723                 services have identical implementations. Surfacing them as separate services\r
1724                 enables a deployer of Evergreen to ensure that the reporting service does not\r
1725                 interfere with the performance-critical <systemitem class="service">open-ils.cstore</systemitem> service. One can also\r
1726                 direct the reporting service to a read-only database replica to, again, avoid\r
1727                 interference with <systemitem class="service">open-ils.cstore</systemitem> which must write to the master database.</simpara>\r
1728                 <simpara>There are only a few significant services that are not built on OpenSRF in\r
1729                 Evergreen 1.6.0, such as the SIP and Z39.50 servers. These services implement\r
1730                 different protocols and build on existing daemon architectures (Simple2ZOOM\r
1731                 for Z39.50), but still rely on the other OpenSRF services to provide access\r
1732                 to the Evergreen data. The non-OpenSRF services are reasonably self-contained\r
1733                 and can be deployed on different servers to deliver the same sort of deployment\r
1734                 flexibility as OpenSRF services, but have the disadvantage of not being\r
1735                 integrated into the same configuration and control infrastructure as the\r
1736                 OpenSRF services.</simpara>\r
1737         </section>\r
1738 </chapter>\r