Clean up master. use 2.0 files for now.
[working/Evergreen.git] / development / OpenSRF_intro.xml
1 <?xml version="1.0" encoding="UTF-8"?>\r
2 <chapter xml:id="opensrf" xmlns="" version="5.0" xml:lang="EN"\r
3     xmlns:xi="" xmlns:xlink="">\r
4         <chapterinfo>\r
5                 <title>OpenSRF</title>\r
6         </chapterinfo>\r
7         <abstract id="openSRF_abstract">\r
8                 <simpara>One of the claimed advantages of\r
9                 Evergreen over alternative integrated library systems is the underlying Open\r
10                 Service Request Framework (OpenSRF, pronounced "open surf") architecture. This\r
11                 article introduces OpenSRF, demonstrates how to build OpenSRF services through\r
12                 simple code examples, and explains the technical foundations on which OpenSRF\r
13                 is built. This chapter was taken from Dan Scott's <emphasis>Easing gently into OpenSRF</emphasis> article, June, 2010.</simpara>\r
14         </abstract>\r
15         <section id="_introducing_opensrf">\r
16                 <title>Introducing OpenSRF</title>\r
17         <indexterm><primary>OpenSRF</primary></indexterm>\r
18                 <simpara>OpenSRF is a message routing network that offers scalability and failover\r
19                 support for individual services and entire servers with minimal development and\r
20                 deployment overhead. You can use OpenSRF to build loosely-coupled applications\r
21                 that can be deployed on a single server or on clusters of geographically\r
22                 distributed servers using the same code and minimal configuration changes.\r
23                 Although copyright statements on some of the OpenSRF code date back to Mike\r
24                 Rylander&#8217;s original explorations in 2000, Evergreen was the first major\r
25                 application to be developed with, and to take full advantage of, the OpenSRF\r
26                 architecture starting in 2004. The first official release of OpenSRF was 0.1 in\r
27                 February 2005 (<ulink url=""></ulink>), but OpenSRF&#8217;s development\r
28                 continues a steady pace of enhancement and refinement, with the release of\r
29                 1.0.0 in October 2008 and the most recent release of 1.2.2 in February 2010.</simpara>\r
30                 <simpara>OpenSRF is a distinct break from the architectural approach used by previous\r
31                 library systems and has more in common with modern Web applications. The\r
32                 traditional "scale-up" approach to serve more transactions is to purchase a\r
33                 server with more CPUs and more RAM, possibly splitting the load between a Web\r
34                 server, a database server, and a business logic server. Evergreen, however, is\r
35                 built on the Open Service Request Framework (OpenSRF) architecture, which\r
36                 firmly embraces the "scale-out" approach of spreading transaction load over\r
37                 cheap commodity servers. The <ulink url="">initial GPLS\r
38                 PINES hardware cluster</ulink>, while certainly impressive, may have offered the\r
39                 misleading impression that Evergreen requires a lot of hardware to run.\r
40                 However, Evergreen and OpenSRF easily scale down to a single server; many\r
41                 Evergreen libraries run their entire library system on a single server, and\r
42                 most OpenSRF and Evergreen development occurs on a virtual machine running on a\r
43                 single laptop or desktop image.</simpara>\r
44                 <simpara>Another common concern is that the flexibility of OpenSRF&#8217;s distributed\r
45                 architecture makes it complex to configure and to write new applications. This\r
46                 article demonstrates that OpenSRF itself is an extremely simple architecture on\r
47                 which one can easily build applications of many kinds – not just library\r
48                 applications – and that you can use a number of different languages to call and\r
49                 implement OpenSRF methods with a minimal learning curve. With an application\r
50                 built on OpenSRF, when you identify a bottleneck in your application&#8217;s business\r
51                 logic layer, you can adjust the number of the processes serving that particular\r
52                 bottleneck on each of your servers; or if the problem is that your service is\r
53                 resource-hungry, you could add an inexpensive server to your cluster and\r
54                 dedicate it to running that resource-hungry service.</simpara>\r
55                 <simplesect id="_programming_language_support">\r
56                         <title>Programming language support</title>\r
57                         <simpara>If you need to develop an entirely new OpenSRF service, you can choose from a\r
58                         number of different languages in which to implement that service. OpenSRF\r
59                         client language bindings have been written for C, Java, JavaScript, Perl, and\r
60                         Python, and service language bindings have been written for C, Perl, and Python.\r
61                         This article uses Perl examples as a lowest common denominator programming\r
62                         language. Writing an OpenSRF binding for another language is a relatively small\r
63                         task if that language offers libraries that support the core technologies on\r
64                         which OpenSRF depends:</simpara>\r
65                         <itemizedlist>\r
66                         <listitem>\r
67                         <simpara>\r
68                         <ulink url="">Extensible Messaging and Presence\r
69                         Protocol</ulink> (XMPP, sometimes referred to as Jabber) - provides the base messaging\r
70                         infrastructure between OpenSRF clients and services\r
71                         </simpara>\r
72                         <indexterm><primary>XMPP</primary></indexterm>\r
73                         </listitem>\r
74                         <listitem>\r
75                         <simpara>\r
76                         <ulink url="">JavaScript Object Notation</ulink> (JSON) - serializes the content\r
77                         of each XMPP message in a standardized and concise format\r
78                         </simpara>\r
79                         </listitem>\r
80                         <listitem>\r
81                         <simpara>\r
82                         <ulink url="">memcached</ulink> - provides the caching service\r
83                         </simpara>\r
84                         <indexterm><primary>memcached</primary></indexterm>\r
85                         </listitem>\r
86                         <listitem>\r
87                         <simpara>\r
88                         <ulink url="">syslog</ulink> - the standard UNIX logging\r
89                         service\r
90                         </simpara>\r
91                         <indexterm><primary>syslog</primary></indexterm>\r
92                         </listitem>\r
93                         </itemizedlist>\r
94                         <simpara>Unfortunately, the\r
95                         <ulink url="">OpenSRF\r
96                         reference documentation</ulink>, although augmented by the\r
97                         <ulink url="">OpenSRF\r
98                         glossary</ulink>, blog posts like <ulink url="">the description\r
99                         of OpenSRF and Jabber</ulink>, and even this article, is not a sufficient substitute\r
100                         for a complete specification on which one could implement a language binding.\r
101                         The recommended option for would-be developers of another language binding is\r
102                         to use the Python implementation as the cleanest basis for a port to another\r
103                         language.</simpara>\r
104                         <indexterm><primary>Python</primary></indexterm>\r
105                 </simplesect>\r
106         </section>\r
107         <section id="writing_an_opensrf_service">\r
108                 <title>Writing an OpenSRF Service</title>\r
109                 <simpara>Imagine an application architecture in which 10 lines of Perl or Python, using\r
110                 the data types native to each language, are enough to implement a method that\r
111                 can then be deployed and invoked seamlessly across hundreds of servers.  You\r
112                 have just imagined developing with OpenSRF – it is truly that simple. Under the\r
113                 covers, of course, the OpenSRF language bindings do an incredible amount of\r
114                 work on behalf of the developer. An OpenSRF application consists of one or more\r
115                 OpenSRF services that expose methods: for example, the <literal>opensrf.simple-text</literal>\r
116                 <ulink url="">demonstration\r
117                 service</ulink> exposes the <literal>opensrf.simple-text.split()</literal> and\r
118                 <literal>opensrf.simple-text.reverse()</literal> methods. Each method accepts zero or more\r
119                 arguments and returns zero or one results. The data types supported by OpenSRF\r
120                 arguments and results are typical core language data types: strings, numbers,\r
121                 booleans, arrays, and hashes.</simpara>\r
122                 <simpara>To implement a new OpenSRF service, perform the following steps:</simpara>\r
123                 <orderedlist numeration="arabic">\r
124                 <listitem>\r
125                 <simpara>\r
126                 Include the base OpenSRF support libraries\r
127                 </simpara>\r
128                 </listitem>\r
129                 <listitem>\r
130                 <simpara>\r
131                 Write the code for each of your OpenSRF methods as separate procedures\r
132                 </simpara>\r
133                 </listitem>\r
134                 <listitem>\r
135                 <simpara>\r
136                 Register each method\r
137                 </simpara>\r
138                 </listitem>\r
139                 <listitem>\r
140                 <simpara>\r
141                 Add the service definition to the OpenSRF configuration files\r
142                 </simpara>\r
143                 </listitem>\r
144                 </orderedlist>\r
145                 <simpara>For example, the following code implements an OpenSRF service. The service\r
146                 includes one method named <literal>opensrf.simple-text.reverse()</literal> that accepts one\r
147                 string as input and returns the reversed version of that string:</simpara>\r
148 <programlisting language="perl" linenumbering="unnumbered">\r
149 #!/usr/bin/perl\r
150 \r
151 package OpenSRF::Application::Demo::SimpleText;\r
152 \r
153 use strict;\r
154 \r
155 use OpenSRF::Application;\r
156 use parent qw/OpenSRF::Application/;\r
157 \r
158 sub text_reverse {\r
159     my ($self , $conn, $text) = @_;\r
160     my $reversed_text = scalar reverse($text);\r
161     return $reversed_text;\r
162 }\r
163 \r
164 __PACKAGE__-&gt;register_method(\r
165     method    =&gt; 'text_reverse',\r
166     api_name  =&gt; 'opensrf.simple-text.reverse'\r
167 );\r
168 </programlisting>\r
169                 <simpara>Ten lines of code, and we have a complete OpenSRF service that exposes a single\r
170                 method and could be deployed quickly on a cluster of servers to meet your\r
171                 application&#8217;s ravenous demand for reversed strings! If you&#8217;re unfamiliar with\r
172                 Perl, the <literal>use OpenSRF::Application; use parent qw/OpenSRF::Application/;</literal>\r
173                 lines tell this package to inherit methods and properties from the\r
174                 <literal>OpenSRF::Application</literal> module. For example, the call to\r
175                 <literal>__PACKAGE__-&gt;register_method()</literal> is defined in <literal>OpenSRF::Application</literal> but due to\r
176                 inheritance is available in this package (named by the special Perl symbol\r
177                 <literal>__PACKAGE__</literal> that contains the current package name). The <literal>register_method()</literal>\r
178                 procedure is how we introduce a method to the rest of the OpenSRF world.</simpara>\r
179                 <simplesect id="serviceRegistration">\r
180                         <title>Registering a service with the OpenSRF configuration files</title>\r
181                         <simpara>Two files control most of the configuration for OpenSRF:</simpara>\r
182                         <itemizedlist>\r
183                         <listitem>\r
184                         <simpara>\r
185                         <literal>opensrf.xml</literal> contains the configuration for the service itself, as well as\r
186                         a list of which application servers in your OpenSRF cluster should start\r
187                         the service.\r
188                         </simpara>\r
189                         </listitem>\r
190                         <listitem>\r
191                         <simpara>\r
192                         <literal>opensrf_core.xml</literal> (often referred to as the "bootstrap configuration"\r
193                         file) contains the OpenSRF networking information, including the XMPP server\r
194                         connection credentials for the public and private routers. You only need to touch\r
195                         this for a new service if the new service needs to be accessible via the\r
196                         public router.\r
197                         </simpara>\r
198                         <indexterm><primary>configuration files</primary><secondary>opensrf_core.xml</secondary></indexterm>\r
199                         </listitem>\r
200                         </itemizedlist>\r
201                         <simpara>Begin by defining the service itself in <literal>opensrf.xml</literal>. To register the\r
202                         <literal>opensrf.simple-text</literal> service, add the following section to the <literal>&lt;apps&gt;</literal>\r
203                         element (corresponding to the XPath <literal>/opensrf/default/apps/</literal>):</simpara>\r
204                         <indexterm><primary>configuration files</primary><secondary>opensrf.xml</secondary></indexterm>\r
205 <programlisting language="xml" linenumbering="unnumbered">\r
206 &lt;apps&gt;\r
207   &lt;opensrf.simple-text&gt; <co id="CO1-1"/> \r
208     &lt;keepalive&gt;3&lt;/keepalive&gt;<co id="CO1-2"/> \r
209     &lt;stateless&gt;1&lt;/stateless&gt;<co id="CO1-3"/>\r
210     &lt;language&gt;perl&lt;/language&gt;<co id="CO1-4"/> \r
211     &lt;implementation&gt;OpenSRF::Application::Demo::SimpleText&lt;/implementation&gt;<co id="CO1-5"/> \r
212     &lt;max_requests&gt;100&lt;/max_requests&gt;<co id="CO1-6"/> \r
213     &lt;unix_config&gt;\r
214       &lt;max_requests&gt;1000&lt;/max_requests&gt; <co id="CO1-7"/> \r
215       &lt;unix_log&gt;opensrf.simple-text_unix.log&lt;/unix_log&gt; <co id="CO1-8"/> \r
216       &lt;unix_sock&gt;opensrf.simple-text_unix.sock&lt;/unix_sock&gt;<co id="CO1-9"/> \r
217       &lt;unix_pid&gt;;/unix_pid&gt; <co id="CO1-10"/> \r
218       &lt;min_children&gt;5&lt;/min_children&gt;  <co id="CO1-11"/> \r
219       &lt;max_children&gt;15&lt;/max_children&gt;<co id="CO1-12"/> \r
220       &lt;min_spare_children&gt;2&lt;/min_spare_children&gt;<co id="CO1-13"/> \r
221       &lt;max_spare_children&gt;5&lt;/max_spare_children&gt; <co id="CO1-14"/> \r
222     &lt;/unix_config&gt;\r
223   &lt;/opensrf.simple-text&gt;\r
224 \r
225   &lt;!-- other OpenSRF services registered here... --&gt;\r
226 &lt;/apps&gt;\r
227 </programlisting>\r
228                         <calloutlist>\r
229                         <callout arearefs="CO1-1">\r
230                         <simpara>\r
231                         The element name is the name that the OpenSRF control scripts use to refer\r
232                         to the service.\r
233                         </simpara>\r
234                         </callout>\r
235                         <callout arearefs="CO1-2">\r
236                         <simpara>\r
237                         The <literal>&lt;keepalive&gt;</literal> element specifies the interval (in seconds) between\r
238                         checks to determine if the service is still running.\r
239                         </simpara>\r
240                         </callout>\r
241                         <callout arearefs="CO1-3">\r
242                         <simpara>\r
243                         The <literal>&lt;stateless&gt;</literal> element specifies whether OpenSRF clients can call\r
244                         methods from this service without first having to create a connection to a\r
245                         specific service backend process for that service. If the value is <literal>1</literal>, then\r
246                         the client can simply issue a request and the router will forward the request\r
247                         to an available service and the result will be returned directly to the client.\r
248                         </simpara>\r
249                         </callout>\r
250                         <callout arearefs="CO1-4">\r
251                         <simpara>\r
252                         The <literal>&lt;language&gt;</literal> element specifies the programming language in which the\r
253                         service is implemented.\r
254                         </simpara>\r
255                         </callout>\r
256                         <callout arearefs="CO1-5">\r
257                         <simpara>\r
258                         The <literal>&lt;implementation&gt;</literal> element pecifies the name of the library or module\r
259                         in which the service is implemented.\r
260                         </simpara>\r
261                         </callout>\r
262                         <callout arearefs="CO1-6">\r
263                         <simpara>\r
264                         (C implementations only): The <literal>&lt;max_requests&gt;</literal> element, as a direct child\r
265                         of the service element name, specifies the maximum number of requests a process\r
266                         serves before it is killed and replaced by a new process.\r
267                         </simpara>\r
268                         </callout>\r
269                         <callout arearefs="CO1-7">\r
270                         <simpara>\r
271                         (Perl implementations only): The <literal>&lt;max_requests&gt;</literal> element, as a direct\r
272                         child of the <literal>&lt;unix_config&gt;</literal> element, specifies the maximum number of requests\r
273                         a process serves before it is killed and replaced by a new process.\r
274                         </simpara>\r
275                         </callout>\r
276                         <callout arearefs="CO1-8">\r
277                         <simpara>\r
278                         The <literal>&lt;unix_log&gt;</literal> element specifies the name of the log file for\r
279                         language-specific log messages such as syntax warnings.\r
280                         </simpara>\r
281                         </callout>\r
282                         <callout arearefs="CO1-9">\r
283                         <simpara>\r
284                         The <literal>&lt;unix_sock&gt;</literal> element specifies the name of the UNIX socket used for\r
285                         inter-process communications.\r
286                         </simpara>\r
287                         </callout>\r
288                         <callout arearefs="CO1-10">\r
289                         <simpara>\r
290                         The <literal>&lt;unix_pid&gt;</literal> element specifies the name of the PID file for the\r
291                         master process for the service.\r
292                         </simpara>\r
293                         </callout>\r
294                         <callout arearefs="CO1-11">\r
295                         <simpara>\r
296                         The <literal>&lt;min_children&gt;</literal> element specifies the minimum number of child\r
297                         processes that should be running at any given time.\r
298                         </simpara>\r
299                         </callout>\r
300                         <callout arearefs="CO1-12">\r
301                         <simpara>\r
302                         The <literal>&lt;max_children&gt;</literal> element specifies the maximum number of child\r
303                         processes that should be running at any given time.\r
304                         </simpara>\r
305                         </callout>\r
306                         <callout arearefs="CO1-13">\r
307                         <simpara>\r
308                         The <literal>&lt;min_spare_children&gt;</literal> element specifies the minimum number of idle\r
309                         child processes that should be available to handle incoming requests.  If there\r
310                         are fewer than this number of spare child processes, new processes will be\r
311                         spawned.\r
312                         </simpara>\r
313                         </callout>\r
314                         <callout arearefs="CO1-14">\r
315                         <simpara>\r
316                         The`&lt;max_spare_children&gt;` element specifies the maximum number of idle\r
317                         child processes that should be available to handle incoming requests. If there\r
318                         are more than this number of spare child processes, the extra processes will be\r
319                         killed.\r
320                         </simpara>\r
321                         </callout>\r
322                         </calloutlist>\r
323                         <simpara>To make the service accessible via the public router, you must also\r
324                         edit the <literal>opensrf_core.xml</literal> configuration file to add the service to the list\r
325                         of publicly accessible services:</simpara>\r
326                         <formalpara><title>Making a service publicly accessible in <literal>opensrf_core.xml</literal></title><para>\r
327 <programlisting language="xml" linenumbering="unnumbered">\r
328 &lt;router&gt;<co id="CO2-1"/> \r
329     &lt;!-- This is the public router. On this router, we only register applications\r
330      which should be accessible to everyone on the opensrf network --&gt;\r
331     &lt;name&gt;router&lt;/name&gt;\r
332     &lt;domain&gt;public.localhost&lt;/domain&gt;<co id="CO2-2"/>\r
333     &lt;services&gt;\r
334         &lt;service&gt;opensrf.math&lt;/service&gt;\r
335         &lt;service&gt;opensrf.simple-text&lt;/service&gt; <co id="CO2-3"/> \r
336     &lt;/services&gt;\r
337 &lt;/router&gt;\r
338 </programlisting>\r
339                         </para></formalpara>\r
340                         <calloutlist>\r
341                         <callout arearefs="CO2-1">\r
342                         <simpara>\r
343                         This section of the <literal>opensrf_core.xml</literal> file is located at XPath\r
344                         <literal>/config/opensrf/routers/</literal>.\r
345                         </simpara>\r
346                         </callout>\r
347                         <callout arearefs="CO2-2">\r
348                         <simpara>\r
349                         <literal>public.localhost</literal> is the canonical public router domain in the OpenSRF\r
350                         installation instructions.\r
351                         </simpara>\r
352                         </callout>\r
353                         <callout arearefs="CO2-3">\r
354                         <simpara>\r
355                         Each <literal>&lt;service&gt;</literal> element contained in the <literal>&lt;services&gt;</literal> element\r
356                         offers their services via the public router as well as the private router.\r
357                         </simpara>\r
358                         </callout>\r
359                         </calloutlist>\r
360                         <simpara>Once you have defined the new service, you must restart the OpenSRF Router\r
361                         to retrieve the new configuration and start or restart the service itself.</simpara>\r
362                 </simplesect>\r
363                 <simplesect id="_calling_an_opensrf_method">\r
364                         <title>Calling an OpenSRF method</title>\r
365                         <indexterm><primary>srfsh</primary></indexterm>\r
366                         <simpara>OpenSRF clients in any supported language can invoke OpenSRF services in any\r
367                         supported language. So let&#8217;s see a few examples of how we can call our fancy\r
368                         new <literal>opensrf.simple-text.reverse()</literal> method:</simpara>\r
369                         <simplesect id="_calling_opensrf_methods_from_the_srfsh_client">\r
370                                 <title>Calling OpenSRF methods from the srfsh client</title>\r
371                                 <simpara><literal>srfsh</literal> is a command-line tool installed with OpenSRF that you can use to call\r
372                                 OpenSRF methods. To call an OpenSRF method, issue the <literal>request</literal> command and\r
373                                 pass the OpenSRF service and method name as the first two arguments; then pass\r
374                                 one or more JSON objects delimited by commas as the arguments to the method\r
375                                 being invoked.</simpara>\r
376                                 <simpara>The following example calls the <literal>opensrf.simple-text.reverse</literal> method of the\r
377                                 <literal>opensrf.simple-text</literal> OpenSRF service, passing the string <literal>"foobar"</literal> as the\r
378                                 only method argument:</simpara>\r
379 <programlisting language="sh" linenumbering="unnumbered">\r
380 $ srfsh\r
381 srfsh # request opensrf.simple-text opensrf.simple-text.reverse "foobar"\r
382 \r
383 Received Data: "raboof"\r
384 \r
385 =------------------------------------\r
386 Request Completed Successfully\r
387 Request Time in seconds: 0.016718\r
388 =------------------------------------\r
389 </programlisting>\r
390                         </simplesect>\r
391                         <simplesect id="opensrfIntrospection">\r
392                                 <title>Getting documentation for OpenSRF methods from the srfsh client</title>\r
393                                 <simpara>The <literal>srfsh</literal> client also gives you command-line access to retrieving metadata\r
394                                 about OpenSRF services and methods. For a given OpenSRF method, for example,\r
395                                 you can retrieve information such as the minimum number of required arguments,\r
396                                 the data type and a description of each argument, the package or library in\r
397                                 which the method is implemented, and a description of the method. To retrieve\r
398                                 the documentation for an opensrf method from <literal>srfsh</literal>, issue the <literal>introspect</literal>\r
399                                 command, followed by the name of the OpenSRF service and (optionally) the\r
400                                 name of the OpenSRF method. If you do not pass a method name to the <literal>introspect</literal>\r
401                                 command, <literal>srfsh</literal> lists all of the methods offered by the service. If you pass\r
402                                 a partial method name, <literal>srfsh</literal> lists all of the methods that match that portion\r
403                                 of the method name.</simpara>\r
404                                 <note><simpara>The quality and availability of the descriptive information for each\r
405                                 method depends on the developer to register the method with complete and\r
406                                 accurate information. The quality varies across the set of OpenSRF and\r
407                                 Evergreen APIs, although some effort is being put towards improving the\r
408                                 state of the internal documentation.</simpara></note>\r
409 <programlisting language="sh" linenumbering="unnumbered">\r
410 srfsh# introspect opensrf.simple-text "opensrf.simple-text.reverse"\r
411 --&gt; opensrf.simple-text\r
412 \r
413 Received Data: {\r
414   "__c":"opensrf.simple-text",\r
415   "__p":{\r
416     "api_level":1,\r
417     "stream":0,      <co id="CO3-1"/>\r
418     "object_hint":"OpenSRF_Application_Demo_SimpleText",\r
419     "remote":0,\r
420     "package":"OpenSRF::Application::Demo::SimpleText", <co id="CO3-2"/>\r
421     "api_name":"opensrf.simple-text.reverse",<co id="CO3-3"/>\r
422     "server_class":"opensrf.simple-text",\r
423     "signature":{ <co id="CO3-4"/>\r
424       "params":[  <co id="CO3-5"/>\r
425         {\r
426           "desc":"The string to reverse",\r
427           "name":"text",\r
428           "type":"string"\r
429         }\r
430       ],\r
431       "desc":"Returns the input string in reverse order\n", <co id="CO3-6"/>\r
432       "return":{                                            <co id="CO3-7"/>\r
433         "desc":"Returns the input string in reverse order",\r
434         "type":"string"\r
435       }\r
436     },\r
437     "method":"text_reverse",  <co id="CO3-8"/>\r
438     "argc":1 <co id="CO3-9"/>\r
439   }\r
440 }\r
441 </programlisting>\r
442                                 <calloutlist>\r
443                                 <callout arearefs="CO3-1">\r
444                                 <simpara>\r
445                                 <literal>stream</literal> denotes whether the method supports streaming responses or not.\r
446                                 </simpara>\r
447                                 </callout>\r
448                                 <callout arearefs="CO3-2">\r
449                                 <simpara>\r
450                                 <literal>package</literal> identifies which package or library implements the method.\r
451                                 </simpara>\r
452                                 </callout>\r
453                                 <callout arearefs="CO3-3">\r
454                                 <simpara>\r
455                                 <literal>api_name</literal> identifies the name of the OpenSRF method.\r
456                                 </simpara>\r
457                                 </callout>\r
458                                 <callout arearefs="CO3-4">\r
459                                 <simpara>\r
460                                 <literal>signature</literal> is a hash that describes the parameters for the method.\r
461                                 </simpara>\r
462                                 </callout>\r
463                                 <callout arearefs="CO3-5">\r
464                                 <simpara>\r
465                                 <literal>params</literal> is an array of hashes describing each parameter in the method;\r
466                                 each parameter has a description (<literal>desc</literal>), name (<literal>name</literal>), and type (<literal>type</literal>).\r
467                                 </simpara>\r
468                                 </callout>\r
469                                 <callout arearefs="CO3-6">\r
470                                 <simpara>\r
471                                 <literal>desc</literal> is a string that describes the method itself.\r
472                                 </simpara>\r
473                                 </callout>\r
474                                 <callout arearefs="CO3-7">\r
475                                 <simpara>\r
476                                 <literal>return</literal> is a hash that describes the return value for the method; it\r
477                                 contains a description of the return value (<literal>desc</literal>) and the type of the\r
478                                 returned value (<literal>type</literal>).\r
479                                 </simpara>\r
480                                 </callout>\r
481                                 <callout arearefs="CO3-8">\r
482                                 <simpara>\r
483                                 <literal>method</literal> identifies the name of the function or method in the source\r
484                                 implementation.\r
485                                 </simpara>\r
486                                 </callout>\r
487                                 <callout arearefs="CO3-9">\r
488                                 <simpara>\r
489                                 <literal>argc</literal> is an integer describing the minimum number of arguments that\r
490                                 must be passed to this method.\r
491                                 </simpara>\r
492                                 </callout>\r
493                                 </calloutlist>\r
494                         </simplesect>\r
495                         <simplesect id="_calling_opensrf_methods_from_perl_applications">\r
496                                 <title>Calling OpenSRF methods from Perl applications</title>\r
497                                 <simpara>To call an OpenSRF method from Perl, you must connect to the OpenSRF service,\r
498                                 issue the request to the method, and then retrieve the results.</simpara>\r
499 <programlisting language="perl" linenumbering="unnumbered">\r
500 #/usr/bin/perl\r
501 use strict;\r
502 use OpenSRF::AppSession;\r
503 use OpenSRF::System;\r
504 \r
505 OpenSRF::System-&gt;bootstrap_client(config_file =&gt; '/openils/conf/opensrf_core.xml');<co id="CO4-1"/>\r
506 \r
507 my $session = OpenSRF::AppSession-&gt;create("opensrf.simple-text");<co id="CO4-2"/>\r
508 \r
509 print "substring: Accepts a string and a number as input, returns a string\n";\r
510 my $result = $session-&gt;request("opensrf.simple-text.substring", "foobar", 3);<co id="CO4-3"/>\r
511 my $request = $result-&gt;gather(); <co id="CO4-4"/>\r
512 print "Substring: $request\n\n";\r
513 \r
514 print "split: Accepts two strings as input, returns an array of strings\n";\r
515 $request = $session-&gt;request("opensrf.simple-text.split", "This is a test", " ");<co id="CO4-5"/>\r
516 my $output = "Split: [";\r
517 my $element;\r
518 while ($element = $request-&gt;recv()) {   <co id="CO4-6"/>\r
519     $output .= $element-&gt;content . ", ";  <co id="CO4-7"/>\r
520 }\r
521 $output =~ s/, $/]/;\r
522 print $output . "\n\n";\r
523 \r
524 print "statistics: Accepts an array of strings as input, returns a hash\n";\r
525 my @many_strings = [\r
526     "First I think I'll have breakfast",\r
527     "Then I think that lunch would be nice",\r
528     "And then seventy desserts to finish off the day"\r
529 ];\r
530 \r
531 $result = $session-&gt;request("opensrf.simple-text.statistics", @many_strings); <co id="CO4-8"/>\r
532 $request = $result-&gt;gather();    <co id="CO4-9"/>\r
533 print "Length: " . $result-&gt;{'length'} . "\n";\r
534 print "Word count: " . $result-&gt;{'word_count'} . "\n";\r
535 \r
536 $session-&gt;disconnect();       <co id="CO4-10"/>\r
537 </programlisting>\r
538                                 <calloutlist>\r
539                                 <callout arearefs="CO4-1">\r
540                                 <simpara>\r
541                                 The <literal>OpenSRF::System-&gt;bootstrap_client()</literal> method reads the OpenSRF\r
542                                 configuration information from the indicated file and creates an XMPP client\r
543                                 connection based on that information.\r
544                                 </simpara>\r
545                                 </callout>\r
546                                 <callout arearefs="CO4-2">\r
547                                 <simpara>\r
548                                 The <literal>OpenSRF::AppSession-&gt;create()</literal> method accepts one argument - the name\r
549                                 of the OpenSRF service to which you want to want to make one or more requests -\r
550                                 and returns an object prepared to use the client connection to make those\r
551                                 requests.\r
552                                 </simpara>\r
553                                 </callout>\r
554                                 <callout arearefs="CO4-3">\r
555                                 <simpara>\r
556                                 The <literal>OpenSRF::AppSession-&gt;request()</literal> method accepts a minimum of one\r
557                                 argument - the name of the OpenSRF method to which you want to make a request -\r
558                                 followed by zero or more arguments to pass to the OpenSRF method as input\r
559                                 values. This example passes a string and an integer to the\r
560                                 <literal>opensrf.simple-text.substring</literal> method defined by the <literal>opensrf.simple-text</literal>\r
561                                 OpenSRF service.\r
562                                 </simpara>\r
563                                 </callout>\r
564                                 <callout arearefs="CO4-4">\r
565                                 <simpara>\r
566                                 The <literal>gather()</literal> method, called on the result object returned by the\r
567                                 <literal>request()</literal> method, iterates over all of the possible results from the result\r
568                                 object and returns a single variable.\r
569                                 </simpara>\r
570                                 </callout>\r
571                                 <callout arearefs="CO4-5">\r
572                                 <simpara>\r
573                                 This <literal>request()</literal> call passes two strings to the <literal>opensrf.simple-text.split</literal>\r
574                                 method defined by the <literal>opensrf.simple-text</literal> OpenSRF service and returns (via\r
575                                 <literal>gather()</literal>) a reference to an array of results.\r
576                                 </simpara>\r
577                                 </callout>\r
578                                 <callout arearefs="CO4-6">\r
579                                 <simpara>\r
580                                 The <literal>opensrf.simple-text.split()</literal> method is a streaming method that\r
581                                 returns an array of results with one element per <literal>recv()</literal> call on the\r
582                                 result object. We could use the <literal>gather()</literal> method to retrieve all of the\r
583                                 results in a single array reference, but instead we simply iterate over\r
584                                 the result variable until there are no more results to retrieve.\r
585                                 </simpara>\r
586                                 </callout>\r
587                                 <callout arearefs="CO4-7">\r
588                                 <simpara>\r
589                                 While the <literal>gather()</literal> convenience method returns only the content of the\r
590                                 complete set of results for a given request, the <literal>recv()</literal> method returns an\r
591                                 OpenSRF result object with <literal>status</literal>, <literal>statusCode</literal>, and <literal>content</literal> fields as\r
592                                 we saw in <link linkend="OpenSRFOverHTTP">the HTTP results example</link>.\r
593                                 </simpara>\r
594                                 </callout>\r
595                                 <callout arearefs="CO4-8">\r
596                                 <simpara>\r
597                                 This <literal>request()</literal> call passes an array to the\r
598                                 <literal>opensrf.simple-text.statistics</literal> method defined by the <literal>opensrf.simple-text</literal>\r
599                                 OpenSRF service.\r
600                                 </simpara>\r
601                                 </callout>\r
602                                 <callout arearefs="CO4-9">\r
603                                 <simpara>\r
604                                 The result object returns a hash reference via <literal>gather()</literal>. The hash\r
605                                 contains the <literal>length</literal> and <literal>word_count</literal> keys we defined in the method.\r
606                                 </simpara>\r
607                                 </callout>\r
608                                 <callout arearefs="CO4-10">\r
609                                 <simpara>\r
610                                 The <literal>OpenSRF::AppSession-&gt;disconnect()</literal> method closes the XMPP client\r
611                                 connection and cleans up resources associated with the session.\r
612                                 </simpara>\r
613                                 </callout>\r
614                                 </calloutlist>\r
615                         </simplesect>\r
616                 </simplesect>\r
617                 <simplesect id="_accepting_and_returning_more_interesting_data_types">\r
618                         <title>Accepting and returning more interesting data types</title>\r
619                         <simpara>Of course, the example of accepting a single string and returning a single\r
620                         string is not very interesting. In real life, our applications tend to pass\r
621                         around multiple arguments, including arrays and hashes. Fortunately, OpenSRF\r
622                         makes that easy to deal with; in Perl, for example, returning a reference to\r
623                         the data type does the right thing. In the following example of a method that\r
624                         returns a list, we accept two arguments of type string: the string to be split,\r
625                         and the delimiter that should be used to split the string.</simpara>\r
626                         <formalpara><title>Basic text splitting method</title><para>\r
627 <programlisting language="perl" linenumbering="unnumbered">\r
628 sub text_split {\r
629     my $self = shift;\r
630     my $conn = shift;\r
631     my $text = shift;\r
632     my $delimiter = shift || ' ';\r
633 \r
634     my @split_text = split $delimiter, $text;\r
635     return \@split_text;\r
636 }\r
637 \r
638 __PACKAGE__-&gt;register_method(\r
639     method    =&gt; 'text_split',\r
640     api_name  =&gt; 'opensrf.simple-text.split'\r
641 );\r
642 </programlisting>\r
643                         </para></formalpara>\r
644                         <simpara>We simply return a reference to the list, and OpenSRF does the rest of the work\r
645                         for us to convert the data into the language-independent format that is then\r
646                         returned to the caller. As a caller of a given method, you must rely on the\r
647                         documentation used to register to determine the data structures - if the developer has\r
648                         added the appropriate documentation.</simpara>\r
649                 </simplesect>\r
650                 <simplesect id="_accepting_and_returning_evergreen_objects">\r
651                         <title>Accepting and returning Evergreen objects</title>\r
652                         <simpara>OpenSRF is agnostic about objects; its role is to pass JSON back and forth\r
653                         between OpenSRF clients and services, and it allows the specific clients and\r
654                         services to define their own semantics for the JSON structures. On top of that\r
655                         infrastructure, Evergreen offers the fieldmapper: an object-relational mapper\r
656                         that provides a complete definition of all objects, their properties, their\r
657                         relationships to other objects, the permissions required to create, read,\r
658                         update, or delete objects of that type, and the database table or view on which\r
659                         they are based.</simpara>\r
660                         <indexterm><primary>Fieldmapper</primary></indexterm>\r
661                         <simpara>The Evergreen fieldmapper offers a great deal of convenience for working with\r
662                         complex system objects beyond the basic mapping of classes to database\r
663                         schemas. Although the result is passed over the wire as a JSON object\r
664                         containing the indicated fields, fieldmapper-aware clients then turn those\r
665                         JSON objects into native objects with setter / getter methods for each field.</simpara>\r
666                         <simpara>All of this metadata about Evergreen objects is defined in the\r
667                         fieldmapper configuration file (<literal>/openils/conf/fm_IDL.xml</literal>), and access to\r
668                         these classes is provided by the <literal>open-ils.cstore</literal>, <literal>open-ils.pcrud</literal>, and\r
669                         <literal>open-ils.reporter-store</literal> OpenSRF services which parse the fieldmapper\r
670                         configuration file and dynamically register OpenSRF methods for creating,\r
671                         reading, updating, and deleting all of the defined classes.</simpara>\r
672                         <formalpara><title>Example fieldmapper class definition for "Open User Summary"</title><para>\r
673 <programlisting language="xml" linenumbering="unnumbered">\r
674 &lt;class id="mous" controller="open-ils.cstore open-ils.pcrud"\r
675  oils_obj:fieldmapper="money::open_user_summary"\r
676  oils_persist:tablename="money.open_usr_summary"\r
677  reporter:label="Open User Summary"&gt;                                <co id="CO5-1"/>\r
678     &lt;fields oils_persist:primary="usr" oils_persist:sequence=""&gt; <co id="CO5-2"/> \r
679         &lt;field name="balance_owed" reporter:datatype="money" /&gt;  <co id="CO5-3"/> \r
680         &lt;field name="total_owed" reporter:datatype="money" /&gt;\r
681         &lt;field name="total_paid" reporter:datatype="money" /&gt;\r
682         &lt;field name="usr" reporter:datatype="link"/&gt;\r
683     &lt;/fields&gt;\r
684     &lt;links&gt;\r
685         &lt;link field="usr" reltype="has_a" key="id" map="" class="au"/&gt;<co id="CO5-4"/> \r
686     &lt;/links&gt;\r
687     &lt;permacrud xmlns=""&gt;<co id="CO5-5"/> \r
688         &lt;actions&gt;\r
689             &lt;retrieve permission="VIEW_USER"&gt;<co id="CO5-6"/> \r
690                 &lt;context link="usr" field="home_ou"/&gt;<co id="CO5-7"/>\r
691             &lt;/retrieve&gt;\r
692         &lt;/actions&gt;\r
693     &lt;/permacrud&gt;\r
694 &lt;/class&gt;\r
695 </programlisting>\r
696                         </para></formalpara>\r
697                         <calloutlist>\r
698                         <callout arearefs="CO5-1">\r
699                         <simpara>\r
700                         The <literal>&lt;class&gt;</literal> element defines the class:\r
701                         </simpara>\r
702                         <itemizedlist>\r
703                         <listitem>\r
704                         <simpara>\r
705                         The <literal>id</literal> attribute defines the <emphasis>class hint</emphasis> that identifies the class both\r
706                         elsewhere in the fieldmapper configuration file, such as in the value of the\r
707                         <literal>field</literal> attribute of the <literal>&lt;link&gt;</literal> element, and in the JSON object itself when\r
708                         it is instantiated. For example, an "Open User Summary" JSON object would have\r
709                         the top level property of <literal>"__c":"mous"</literal>.\r
710                         </simpara>\r
711                         </listitem>\r
712                         <listitem>\r
713                         <simpara>\r
714                         The <literal>controller</literal> attribute identifies the services that have direct access\r
715                         to this class. If <literal>open-ils.pcrud</literal> is not listed, for example, then there is\r
716                         no means to directly access members of this class through a public service.\r
717                         </simpara>\r
718                         </listitem>\r
719                         <listitem>\r
720                         <simpara>\r
721                         The <literal>oils_obj:fieldmapper</literal> attribute defines the name of the Perl\r
722                         fieldmapper class that will be dynamically generated to provide setter and\r
723                         getter methods for instances of the class.\r
724                         </simpara>\r
725                         </listitem>\r
726                         <listitem>\r
727                         <simpara>\r
728                         The <literal>oils_persist:tablename</literal> attribute identifies the schema name and table\r
729                         name of the database table that stores the data that represents the instances\r
730                         of this class. In this case, the schema is <literal>money</literal> and the table is\r
731                         <literal>open_usr_summary</literal>.\r
732                         </simpara>\r
733                         </listitem>\r
734                         <listitem>\r
735                         <simpara>\r
736                         The <literal>reporter:label</literal> attribute defines a human-readable name for the class\r
737                         used in the reporting interface to identify the class. These names are defined\r
738                         in English in the fieldmapper configuration file; however, they are extracted\r
739                         so that they can be translated and served in the user&#8217;s language of choice.\r
740                         </simpara>\r
741                         </listitem>\r
742                         </itemizedlist>\r
743                         </callout>\r
744                         <callout arearefs="CO5-2">\r
745                         <simpara>\r
746                         The <literal>&lt;fields&gt;</literal> element lists all of the fields that belong to the object.\r
747                         </simpara>\r
748                         <itemizedlist>\r
749                         <listitem>\r
750                         <simpara>\r
751                         The <literal>oils_persist:primary</literal> attribute identifies the field that acts as the\r
752                         primary key for the object; in this case, the field with the name <literal>usr</literal>.\r
753                         </simpara>\r
754                         </listitem>\r
755                         <listitem>\r
756                         <simpara>\r
757                         The <literal>oils_persist:sequence</literal> attribute identifies the sequence object\r
758                         (if any) in this database provides values for new instances of this class. In\r
759                         this case, the primary key is defined by a field that is linked to a different\r
760                         table, so no sequence is used to populate these instances.\r
761                         </simpara>\r
762                         </listitem>\r
763                         </itemizedlist>\r
764                         </callout>\r
765                         <callout arearefs="CO5-3">\r
766                         <simpara>\r
767                         Each <literal>&lt;field&gt;</literal> element defines a single field with the following attributes:\r
768                         </simpara>\r
769                         <itemizedlist>\r
770                         <listitem>\r
771                         <simpara>\r
772                         The <literal>name</literal> attribute identifies the column name of the field in the\r
773                         underlying database table as well as providing a name for the setter / getter\r
774                         method that can be invoked in the JSON or native version of the object.\r
775                         </simpara>\r
776                         </listitem>\r
777                         <listitem>\r
778                         <simpara>\r
779                         The <literal>reporter:datatype</literal> attribute defines how the reporter should treat\r
780                         the contents of the field for the purposes of querying and display.\r
781                         </simpara>\r
782                         </listitem>\r
783                         <listitem>\r
784                         <simpara>\r
785                         The <literal>reporter:label</literal> attribute can be used to provide a human-readable name\r
786                         for each field; without it, the reporter falls back to the value of the <literal>name</literal>\r
787                         attribute.\r
788                         </simpara>\r
789                         </listitem>\r
790                         </itemizedlist>\r
791                         </callout>\r
792                         <callout arearefs="CO5-4">\r
793                         <simpara>\r
794                         The <literal>&lt;links&gt;</literal> element contains a set of zero or more <literal>&lt;link&gt;</literal> elements,\r
795                         each of which defines a relationship between the class being described and\r
796                         another class.\r
797                         </simpara>\r
798                         <itemizedlist>\r
799                         <listitem>\r
800                         <simpara>\r
801                         The <literal>field</literal> attribute identifies the field named in this class that links\r
802                         to the external class.\r
803                         </simpara>\r
804                         </listitem>\r
805                         <listitem>\r
806                         <simpara>\r
807                         The <literal>reltype</literal> attribute identifies the kind of relationship between the\r
808                         classes; in the case of <literal>has_a</literal>, each value in the <literal>usr</literal> field is guaranteed\r
809                         to have a corresponding value in the external class.\r
810                         </simpara>\r
811                         </listitem>\r
812                         <listitem>\r
813                         <simpara>\r
814                         The <literal>key</literal> attribute identifies the name of the field in the external\r
815                         class to which this field links.\r
816                         </simpara>\r
817                         </listitem>\r
818                         <listitem>\r
819                         <simpara>\r
820                         The rarely-used <literal>map</literal> attribute identifies a second class to which\r
821                         the external class links; it enables this field to define a direct\r
822                         relationship to an external class with one degree of separation, to\r
823                         avoid having to retrieve all of the linked members of an intermediate\r
824                         class just to retrieve the instances from the actual desired target class.\r
825                         </simpara>\r
826                         </listitem>\r
827                         <listitem>\r
828                         <simpara>\r
829                         The <literal>class</literal> attribute identifies the external class to which this field\r
830                         links.\r
831                         </simpara>\r
832                         </listitem>\r
833                         </itemizedlist>\r
834                         </callout>\r
835                         <callout arearefs="CO5-5">\r
836                         <simpara>\r
837                         The <literal>&lt;permacrud&gt;</literal> element defines the permissions that must have been\r
838                         granted to a user to operate on instances of this class.\r
839                         </simpara>\r
840                         </callout>\r
841                         <callout arearefs="CO5-6">\r
842                         <simpara>\r
843                         The <literal>&lt;retrieve&gt;</literal> element is one of four possible children of the\r
844                         <literal>&lt;actions&gt;</literal> element that define the permissions required for each action:\r
845                         create, retrieve, update, and delete.\r
846                         </simpara>\r
847                         <itemizedlist>\r
848                         <listitem>\r
849                         <simpara>\r
850                         The <literal>permission</literal> attribute identifies the name of the permission that must\r
851                         have been granted to the user to perform the action.\r
852                         </simpara>\r
853                         </listitem>\r
854                         <listitem>\r
855                         <simpara>\r
856                         The <literal>contextfield</literal> attribute, if it exists, defines the field in this class\r
857                         that identifies the library within the system for which the user must have\r
858                         prvileges to work. If a user has been granted a given permission, but has not been\r
859                         granted privileges to work at a given library, they can not perform the action\r
860                         at that library.\r
861                         </simpara>\r
862                         </listitem>\r
863                         </itemizedlist>\r
864                         </callout>\r
865                         <callout arearefs="CO5-7">\r
866                         <simpara>\r
867                         The rarely-used <literal>&lt;context&gt;</literal> element identifies a linked field (<literal>link</literal>\r
868                         attribute) in this class which links to an external class that holds the field\r
869                         (<literal>field</literal> attribute) that identifies the library within the system for which the\r
870                         user must have privileges to work.\r
871                         </simpara>\r
872                         </callout>\r
873                         </calloutlist>\r
874                         <simpara>When you retrieve an instance of a class, you can ask for the result to\r
875                         <emphasis>flesh</emphasis> some or all of the linked fields of that class, so that the linked\r
876                         instances are returned embedded directly in your requested instance. In that\r
877                         same request you can ask for the fleshed instances to in turn have their linked\r
878                         fields fleshed. By bundling all of this into a single request and result\r
879                         sequence, you can avoid the network overhead of requiring the client to request\r
880                         the base object, then request each linked object in turn.</simpara>\r
881                         <simpara>You can also iterate over a collection of instances and set the automatically\r
882                         generated <literal>isdeleted</literal>, <literal>isupdated</literal>, or <literal>isnew</literal> properties to indicate that\r
883                         the given instance has been deleted, updated, or created respectively.\r
884                         Evergreen can then act in batch mode over the collection to perform the\r
885                         requested actions on any of the instances that have been flagged for action.</simpara>\r
886                 </simplesect>\r
887                 <simplesect id="_returning_streaming_results">\r
888                         <title>Returning streaming results</title>\r
889                         <simpara>In the previous implementation of the <literal>opensrf.simple-text.split</literal> method, we\r
890                         returned a reference to the complete array of results. For small values being\r
891                         delivered over the network, this is perfectly acceptable, but for large sets of\r
892                         values this can pose a number of problems for the requesting client. Consider a\r
893                         service that returns a set of bibliographic records in response to a query like\r
894                         "all records edited in the past month"; if the underlying database is\r
895                         relatively active, that could result in thousands of records being returned as\r
896                         a single network request. The client would be forced to block until all of the\r
897                         results are returned, likely resulting in a significant delay, and depending on\r
898                         the implementation, correspondingly large amounts of memory might be consumed\r
899                         as all of the results are read from the network in a single block.</simpara>\r
900                         <simpara>OpenSRF offers a solution to this problem. If the method returns results that\r
901                         can be divided into separate meaningful units, you can register the OpenSRF\r
902                         method as a streaming method and enable the client to loop over the results one\r
903                         unit at a time until the method returns no further results. In addition to\r
904                         registering the method with the provided name, OpenSRF also registers an additional\r
905                         method with <literal>.atomic</literal> appended to the method name. The <literal>.atomic</literal> variant gathers\r
906                         all of the results into a single block to return to the client, giving the caller\r
907                         the ability to choose either streaming or atomic results from a single method\r
908                         definition.</simpara>\r
909                         <simpara>In the following example, the text splitting method has been reimplemented to\r
910                         support streaming; very few changes are required:</simpara>\r
911                         <formalpara><title>Text splitting method - streaming mode</title><para>\r
912 <programlisting language="perl" linenumbering="unnumbered">\r
913 sub text_split {\r
914     my $self = shift;\r
915     my $conn = shift;\r
916     my $text = shift;\r
917     my $delimiter = shift || ' ';\r
918 \r
919     my @split_text = split $delimiter, $text;\r
920     foreach my $string (@split_text) { <co id="CO6-1"/>\r
921         $conn-&gt;respond($string);\r
922     }\r
923     return undef;\r
924 }\r
925 \r
926 __PACKAGE__-&gt;register_method(\r
927     method    =&gt; 'text_split',\r
928     api_name  =&gt; 'opensrf.simple-text.split',\r
929     stream    =&gt; 1<co id="CO6-2"/>\r
930 );\r
931 </programlisting>\r
932                         </para></formalpara>\r
933                         <calloutlist>\r
934                         <callout arearefs="CO6-1">\r
935                         <simpara>\r
936                         Rather than returning a reference to the array, a streaming method loops\r
937                         over the contents of the array and invokes the <literal>respond()</literal> method of the\r
938                         connection object on each element of the array.\r
939                         </simpara>\r
940                         </callout>\r
941                         <callout arearefs="CO6-2">\r
942                         <simpara>\r
943                         Registering the method as a streaming method instructs OpenSRF to also\r
944                         register an atomic variant (<literal>opensrf.simple-text.split.atomic</literal>).\r
945                         </simpara>\r
946                         </callout>\r
947                         </calloutlist>\r
948                 </simplesect>\r
949                 <simplesect id="_error_warning_info_debug">\r
950                         <title>Error! Warning! Info! Debug!</title>\r
951                         <simpara>As hard as it may be to believe, it is true: applications sometimes do not\r
952                         behave in the expected manner, particularly when they are still under\r
953                         development. The service language bindings for OpenSRF include integrated\r
954                         support for logging messages at the levels of ERROR, WARNING, INFO, DEBUG, and\r
955                         the extremely verbose INTERNAL to either a local file or to a syslogger\r
956                         service. The destination of the log files, and the level of verbosity to be\r
957                         logged, is set in the <literal>opensrf_core.xml</literal> configuration file. To add logging to\r
958                         our Perl example, we just have to add the <literal>OpenSRF::Utils::Logger</literal> package to our\r
959                         list of used Perl modules, then invoke the logger at the desired logging level.</simpara>\r
960                         <simpara>You can include many calls to the OpenSRF logger; only those that are higher\r
961                         than your configured logging level will actually hit the log. The following\r
962                         example exercises all of the available logging levels in OpenSRF:</simpara>\r
963 <programlisting language="perl" linenumbering="unnumbered">\r
964 use OpenSRF::Utils::Logger;\r
965 my $logger = OpenSRF::Utils::Logger;\r
966 # some code in some function\r
967 {\r
968     $logger-&gt;error("Hmm, something bad DEFINITELY happened!");\r
969     $logger-&gt;warn("Hmm, something bad might have happened.");\r
970     $logger-&gt;info("Something happened.");\r
971     $logger-&gt;debug("Something happened; here are some more details.");\r
972     $logger-&gt;internal("Something happened; here are all the gory details.")\r
973 }\r
974 </programlisting>\r
975                         <simpara>If you call the mythical OpenSRF method containing the preceding OpenSRF logger\r
976                         statements on a system running at the default logging level of INFO, you will\r
977                         only see the INFO, WARN, and ERR messages, as follows:</simpara>\r
978                         <formalpara><title>Results of logging calls at the default level of INFO</title><para>\r
979 <screen>\r
980 [2010-03-17 22:27:30] opensrf.simple-text [ERR] \r
981 [2010-03-17 22:27:30] opensrf.simple-text [] \r
982 [2010-03-17 22:27:30] opensrf.simple-text [] \r
983 </screen>\r
984                         </para></formalpara>\r
985                         <simpara>If you then increase the the logging level to INTERNAL (5), the logs will\r
986                         contain much more information, as follows:</simpara>\r
987                         <formalpara><title>Results of logging calls at the default level of INTERNAL</title><para>\r
988 <screen>\r
989 [2010-03-17 22:48:11] opensrf.simple-text [ERR] \r
990 [2010-03-17 22:48:11] opensrf.simple-text [] \r
991 [2010-03-17 22:48:11] opensrf.simple-text [] \r
992 [2010-03-17 22:48:11] opensrf.simple-text [] \r
993 [2010-03-17 22:48:11] opensrf.simple-text [] \r
994 [2010-03-17 22:48:11] opensrf.simple-text [ERR] \r
995 [2010-03-17 22:48:21] opensrf.simple-text [] \r
996 [2010-03-17 22:48:21] opensrf.simple-text [] \r
997 [2010-03-17 22:48:21] opensrf.simple-text [] \r
998 [2010-03-17 22:48:21] opensrf.simple-text [] \r
999 [2010-03-17 22:48:21] opensrf.simple-text [] Calling queue_wait(0)\r
1000 [2010-03-17 22:48:21] opensrf.simple-text [] Resending...0\r
1001 [2010-03-17 22:48:21] opensrf.simple-text [] In send\r
1002 [2010-03-17 22:48:21] opensrf.simple-text [] \r
1003 [2010-03-17 22:48:21] opensrf.simple-text [] \r
1004 ...\r
1005 </screen>\r
1006                         </para></formalpara>\r
1007                         <simpara>To see everything that is happening in OpenSRF, try leaving your logging level\r
1008                         set to INTERNAL for a few minutes - just ensure that you have a lot of free disk\r
1009                         space available if you have a moderately busy system!</simpara>\r
1010                 </simplesect>\r
1011                 <simplesect id="_caching_results_one_secret_of_scalability">\r
1012                         <title>Caching results: one secret of scalability</title>\r
1013                         <indexterm><primary>search results</primary><secondary>caching</secondary></indexterm>\r
1014                         <simpara>If you have ever used an application that depends on a remote Web service\r
1015                         outside of your control&#8201;&#8212;&#8201;say, if you need to retrieve results from a\r
1016                         microblogging service&#8201;&#8212;&#8201;you know the pain of latency and dependability (or the\r
1017                         lack thereof). To improve the response time for OpenSRF services, you can take\r
1018                         advantage of the support offered by the <literal>OpenSRF::Utils::Cache</literal> module for\r
1019                         communicating with a local instance or cluster of <literal>memcache</literal> daemons to store\r
1020                         and retrieve persistent values. The following example demonstrates caching\r
1021                         by sleeping for 10 seconds the first time it receives a given cache key and\r
1022                         cannot retrieve a corresponding value from the cache:</simpara>\r
1023                         <formalpara><title>Simple caching OpenSRF service</title><para>\r
1024 <programlisting language="perl" linenumbering="unnumbered">\r
1025 use OpenSRF::Utils::Cache;<co id="CO7-1"/>\r
1026 sub test_cache {\r
1027     my $self = shift;\r
1028     my $conn = shift;\r
1029     my $test_key = shift;\r
1030     my $cache = OpenSRF::Utils::Cache-&gt;new('global'); <co id="CO7-2"/>\r
1031     my $cache_key = "opensrf.simple-text.test_cache.$test_key"; <co id="CO7-3"/>\r
1032     my $result = $cache-&gt;get_cache($cache_key) || undef; <co id="CO7-4"/>\r
1033     if ($result) {\r
1034         $logger-&gt;info("Resolver found a cache hit");\r
1035         return $result;\r
1036     }\r
1037     sleep 10; <co id="CO7-5"/>\r
1038     my $cache_timeout = 300; <co id="CO7-6"/>\r
1039     $cache-&gt;put_cache($cache_key, "here", $cache_timeout); <co id="CO7-7"/>\r
1040     return "There was no cache hit.";\r
1041 }\r
1042 </programlisting>\r
1043                         </para></formalpara>\r
1044                         <calloutlist>\r
1045                         <callout arearefs="CO7-1">\r
1046                         <simpara>\r
1047                         The OpenSRF::Utils::Cache module provides access to the built-in caching\r
1048                         support in OpenSRF.\r
1049                         </simpara>\r
1050                         </callout>\r
1051                         <callout arearefs="CO7-2">\r
1052                         <simpara>\r
1053                         The constructor for the cache object accepts a single argument to define\r
1054                         the cache type for the object. Each cache type can use a separate <literal>memcache</literal>\r
1055                         server to keep the caches separated. Most Evergreen services use the <literal>global</literal>\r
1056                         cache, while the <literal>anon</literal> cache is used for Web sessions.\r
1057                         </simpara>\r
1058                         </callout>\r
1059                         <callout arearefs="CO7-3">\r
1060                         <simpara>\r
1061                         The cache key is simply a string that uniquely identifies the value you\r
1062                         want to store or retrieve. This line creates a cache key based on the OpenSRF\r
1063                         method name and request input value.\r
1064                         </simpara>\r
1065                         </callout>\r
1066                         <callout arearefs="CO7-4">\r
1067                         <simpara>\r
1068                         The <literal>get_cache()</literal> method checks to see if the cache key already exists. If\r
1069                         a matching key is found, the service immediately returns the stored value.\r
1070                         </simpara>\r
1071                         </callout>\r
1072                         <callout arearefs="CO7-5">\r
1073                         <simpara>\r
1074                         If the cache key does not exist, the code sleeps for 10 seconds to\r
1075                         simulate a call to a slow remote Web service or an intensive process.\r
1076                         </simpara>\r
1077                         </callout>\r
1078                         <callout arearefs="CO7-6">\r
1079                         <simpara>\r
1080                         The <literal>$cache_timeout</literal> variable represents a value for the lifetime of the\r
1081                         cache key in seconds.\r
1082                         </simpara>\r
1083                         </callout>\r
1084                         <callout arearefs="CO7-7">\r
1085                         <simpara>\r
1086                         After the code retrieves its value (or, in the case of this example,\r
1087                         finishes sleeping), it creates the cache entry by calling the <literal>put_cache()</literal>\r
1088                         method. The method accepts three arguments: the cache key, the value to be\r
1089                         stored ("here"), and the timeout value in seconds to ensure that we do not\r
1090                         return stale data on subsequent calls.\r
1091                         </simpara>\r
1092                         </callout>\r
1093                         </calloutlist>\r
1094                 </simplesect>\r
1095                 <simplesect id="_initializing_the_service_and_its_children_child_labour">\r
1096                         <title>Initializing the service and its children: child labour</title>\r
1097                         <simpara>When an OpenSRF service is started, it looks for a procedure called\r
1098                         <literal>initialize()</literal> to set up any global variables shared by all of the children of\r
1099                         the service. The <literal>initialize()</literal> procedure is typically used to retrieve\r
1100                         configuration settings from the <literal>opensrf.xml</literal> file.</simpara>\r
1101                         <simpara>An OpenSRF service spawns one or more children to actually do the work\r
1102                         requested by callers of the service. For every child process an OpenSRF service\r
1103                         spawns, the child process clones the parent environment and then each child\r
1104                         process runs the <literal>child_init()</literal> process (if any) defined in the OpenSRF service\r
1105                         to initialize any child-specific settings.</simpara>\r
1106                         <simpara>When the OpenSRF service kills a child process, it invokes the <literal>child_exit()</literal>\r
1107                         procedure (if any) to clean up any resources associated with the child process.\r
1108                         Similarly, when the OpenSRF service is stopped, it calls the <literal>DESTROY()</literal>\r
1109                         procedure to clean up any remaining resources.</simpara>\r
1110                 </simplesect>\r
1111                 <simplesect id="_retrieving_configuration_settings">\r
1112                         <title>Retrieving configuration settings</title>\r
1113                         <simpara>The settings for OpenSRF services are maintained in the <literal>opensrf.xml</literal> XML\r
1114                         configuration file. The structure of the XML document consists of a root\r
1115                         element <literal>&lt;opensrf&gt;</literal> containing two child elements:</simpara>\r
1116                         <itemizedlist>\r
1117                         <listitem>\r
1118                         <simpara>\r
1119                         The <literal>&lt;default&gt;</literal> element contains an <literal>&lt;apps&gt;</literal> element describing all\r
1120                         OpenSRF services running on this system&#8201;&#8212;&#8201;see <xref linkend="serviceRegistration"/> --, as\r
1121                         well as any other arbitrary XML descriptions required for global configuration\r
1122                         purposes. For example, Evergreen uses this section for email notification and\r
1123                         inter-library patron privacy settings.\r
1124                         </simpara>\r
1125                         </listitem>\r
1126                         <listitem>\r
1127                         <simpara>\r
1128                         The <literal>&lt;hosts&gt;</literal> element contains one element per host that participates in\r
1129                         this OpenSRF system. Each host element must include an <literal>&lt;activeapps&gt;</literal> element\r
1130                         that lists all of the services to start on this host when the system starts\r
1131                         up. Each host element can optionally override any of the default settings.\r
1132                         </simpara>\r
1133                         </listitem>\r
1134                         </itemizedlist>\r
1135                         <simpara>OpenSRF includes a service named <literal>opensrf.settings</literal> to provide distributed\r
1136                         cached access to the configuration settings with a simple API:</simpara>\r
1137                         <itemizedlist>\r
1138                         <listitem>\r
1139                         <simpara>\r
1140                         <literal>opensrf.settings.default_config.get</literal> accepts zero arguments and returns\r
1141                         the complete set of default settings as a JSON document.\r
1142                         </simpara>\r
1143                         </listitem>\r
1144                         <listitem>\r
1145                         <simpara>\r
1146                         <literal>opensrf.settings.host_config.get</literal> accepts one argument (hostname) and\r
1147                         returns the complete set of settings, as customized for that hostname, as a\r
1148                         JSON document.\r
1149                         </simpara>\r
1150                         </listitem>\r
1151                         <listitem>\r
1152                         <simpara>\r
1153                         <literal>opensrf.settings.xpath.get</literal> accepts one argument (an\r
1154                         <ulink url="">XPath</ulink> expression) and returns the portion of\r
1155                         the configuration file that matches the expression as a JSON document.\r
1156                         </simpara>\r
1157                         </listitem>\r
1158                         </itemizedlist>\r
1159                         <simpara>For example, to determine whether an Evergreen system uses the opt-in\r
1160                         support for sharing patron information between libraries, you could either\r
1161                         invoke the <literal>opensrf.settings.default_config.get</literal> method and parse the\r
1162                         JSON document to determine the value, or invoke the <literal>opensrf.settings.xpath.get</literal>\r
1163                         method with the XPath <literal>/opensrf/default/share/user/opt_in</literal> argument to\r
1164                         retrieve the value directly.</simpara>\r
1165                         <simpara>In practice, OpenSRF includes convenience libraries in all of its client\r
1166                         language bindings to simplify access to configuration values. C offers\r
1167                         osrfConfig.c, Perl offers <literal>OpenSRF::Utils::SettingsClient</literal>, Java offers\r
1168                         <literal>org.opensrf.util.SettingsClient</literal>, and Python offers <literal>osrf.set</literal>. These\r
1169                         libraries locally cache the configuration file to avoid network roundtrips for\r
1170                         every request and enable the developer to request specific values without\r
1171                         having to manually construct XPath expressions.</simpara>\r
1172                 </simplesect>\r
1173         </section>\r
1174         <section id="_getting_under_the_covers_with_opensrf">\r
1175                 <title>OpenSRF Communication Flows</title>\r
1176                 <indexterm><primary>OpenSRF</primary><secondary>Communication Flows</secondary></indexterm>\r
1177                 <simpara>Now that you have seen that it truly is easy to create an OpenSRF service, we\r
1178                 can take a look at what is going on under the covers to make all of this work\r
1179                 for you.</simpara>\r
1180                 <simplesect id="_get_on_the_messaging_bus_safely">\r
1181                         <title>Get on the messaging bus - safely</title>\r
1182                         <simpara>One of the core innovations of OpenSRF was to use the Extensible Messaging and\r
1183                         Presence Protocol (XMPP, more colloquially known as Jabber) as the messaging\r
1184                         bus that ties OpenSRF services together across servers. XMPP is an "XML\r
1185                         protocol for near-real-time messaging, presence, and request-response services"\r
1186                         (<ulink url=""></ulink>) that OpenSRF relies on to handle most of\r
1187                         the complexity of networked communications.  OpenSRF requres an XMPP server\r
1188                         that supports multiple domains such as <ulink url="">ejabberd</ulink>.\r
1189                         Multiple domain support means that a single server can support XMPP virtual\r
1190                         hosts with separate sets of users and access privileges per domain. By\r
1191                         routing communications through separate public and private XMPP domains,\r
1192                         OpenSRF services gain an additional layer of security.</simpara>\r
1193                         <simpara>The <ulink url="">OpenSRF\r
1194                         installation documentation</ulink> instructs you to create two separate hostnames\r
1195                         (<literal>private.localhost</literal> and <literal>public.localhost</literal>) to use as XMPP domains.  OpenSRF\r
1196                         can control access to its services based on the domain of the client and\r
1197                         whether a given service allows access from clients on the public domain.  When\r
1198                         you start OpenSRF, the first XMPP clients that connect to the XMPP server are\r
1199                         the OpenSRF public and private <emphasis>routers</emphasis>. OpenSRF routers maintain a list of\r
1200                         available services and connect clients to available services. When an OpenSRF\r
1201                         service starts, it establishes a connection to the XMPP server and registers\r
1202                         itself with the private router. The OpenSRF configuration contains a list of\r
1203                         public OpenSRF services, each of which must also register with the public\r
1204                         router.</simpara>\r
1205                 </simplesect>\r
1206                 <simplesect id="_opensrf_communication_flows_over_xmpp">\r
1207                         <title>OpenSRF communication flows over XMPP</title>\r
1208                         <indexterm><primary>XMPP</primary></indexterm>\r
1209                         <simpara>In a minimal OpenSRF deployment, two XMPP users named "router" connect to the\r
1210                         XMPP server, with one connected to the private XMPP domain and one connected to\r
1211                         the public XMPP domain. Similarly, two XMPP users named "opensrf" connect to\r
1212                         the XMPP server via the private and public XMPP domains. When an OpenSRF\r
1213                         service is started, it uses the "opensrf" XMPP user to advertise its\r
1214                         availability with the corresponding router on that XMPP domain; the XMPP server\r
1215                         automatically assigns a Jabber ID (<emphasis>JID</emphasis>) based on the client hostname to each\r
1216                         service&#8217;s listener process and each connected drone process waiting to carry\r
1217                         out requests. When an OpenSRF router receives a request to invoke a method on a\r
1218                         given service, it connects the requester to the next available listener in the\r
1219                         list of registered listeners for that service.</simpara>\r
1220                         <simpara>Services and clients connect to the XMPP server using a single set of XMPP\r
1221                         client credentials (for example, <literal>opensrf@private.localhost</literal>), but use XMPP\r
1222                         resource identifiers to differentiate themselves in the JID for each\r
1223                         connection. For example, the JID for a copy of the <literal>opensrf.simple-text</literal>\r
1224                         service with process ID <literal>6285</literal> that has connected to the <literal>private.localhost</literal>\r
1225                         domain using the <literal>opensrf</literal> XMPP client credentials could be\r
1226                         <literal>opensrf@private.localhost/opensrf.simple-text_drone_at_localhost_6285</literal>.  By\r
1227                         convention, the user name for OpenSRF clients is <literal>opensrf</literal>, and the user name\r
1228                         for OpenSRF routers is <literal>router</literal>, so the XMPP server for OpenSRF will have four\r
1229                         separate users registered:\r
1230                           * <literal>opensrf@private.localhost</literal> is an OpenSRF client that connects with these\r
1231                         credentials and which can access any OpenSRF service.\r
1232                           * <literal>opensrf@public.localhost</literal> is an OpenSRF client that connects with these\r
1233                         credentials and which can only access OpenSRF services that have registered\r
1234                         with the public router.\r
1235                           * <literal>router@private.localhost</literal> is the private OpenSRF router with which all\r
1236                         services register.\r
1237                           * <literal>router@public.localhost</literal> is the public OpenSRF router with which only\r
1238                         services that must be publicly accessible register.</simpara>\r
1239                         <simpara>All OpenSRF services automatically register themselves with the private XMPP\r
1240                         domain, but only those services that register themselves with the public XMPP\r
1241                         domain can be invoked from public OpenSRF clients.  The OpenSRF client and\r
1242                         router user names, passwords, and domain names, along with the list of services\r
1243                         that should be public, are contained in the <literal>opensrf_core.xml</literal> configuration\r
1244                         file.</simpara>\r
1245                 </simplesect>\r
1246                 <simplesect id="OpenSRFOverHTTP">\r
1247                         <title>OpenSRF communication flows over HTTP</title>\r
1248                         <indexterm><primary>HTTP</primary><secondary>translator</secondary></indexterm>\r
1249                         <simpara>In some contexts, access to a full XMPP client is not a practical option. For\r
1250                         example, while XMPP clients have been implemented in JavaScript, you might\r
1251                         be concerned about browser compatibility and processing overhead - or you might\r
1252                         want to issue OpenSRF requests from the command line with <literal>curl</literal>. Fortunately,\r
1253                         any OpenSRF service registered with the public router is accessible via the\r
1254                         OpenSRF HTTP Translator. The OpenSRF HTTP Translator implements the\r
1255                         <ulink url="">OpenSRF-over-HTTP\r
1256                         proposed specification</ulink> as an Apache module that translates HTTP requests into\r
1257                         OpenSRF requests and returns OpenSRF results as HTTP results to the initiating\r
1258                         HTTP client.</simpara>\r
1259                         <formalpara><title>Issuing an HTTP POST request to an OpenSRF method via the OpenSRF HTTP Translator</title><para>\r
1260 <programlisting language="bash" linenumbering="unnumbered">\r
1261 # curl request broken up over multiple lines for legibility\r
1262 curl -H "X-OpenSRF-service: opensrf.simple-text"<co id="CO8-1"/>\r
1263     --data 'osrf-msg=[  \<co id="CO8-2"/>\r
1264         {"__c":"osrfMessage","__p":{"threadTrace":0,"locale":"en-CA", <co id="CO8-3"/>\r
1265             "type":"REQUEST","payload": {"__c":"osrfMethod","__p": \r
1266                 {"method":"opensrf.simple-text.reverse","params":["foobar"]}   \r
1267             }}                                                                  \r
1268         }]'                                                                    \r
1269 http://localhost/osrf-http-translator <co id="CO8-4"/>\r
1270 </programlisting>\r
1271                         </para></formalpara>\r
1272                         <calloutlist>\r
1273                         <callout arearefs="CO8-1">\r
1274                         <simpara>\r
1275                         The <literal>X-OpenSRF-service</literal> header identifies the OpenSRF service of interest.\r
1276                         </simpara>\r
1277                         </callout>\r
1278                         <callout arearefs="CO8-2">\r
1279                         <simpara>\r
1280                         The POST request consists of a single parameter, the <literal>osrf-msg</literal> value,\r
1281                         which contains a JSON array.\r
1282                         </simpara>\r
1283                         </callout>\r
1284                         <callout arearefs="CO8-3">\r
1285                         <simpara>\r
1286                         The first object is an OpenSRF message (<literal>"__c":"osrfMessage"</literal>) with a set of\r
1287                         parameters (<literal>"__p":{}</literal>).\r
1288                         </simpara>\r
1289                         <itemizedlist>\r
1290                         <listitem>\r
1291                         <simpara>\r
1292                         The identifier for the request (<literal>"threadTrace":0</literal>); this value is echoed\r
1293                         back in the result.\r
1294                         </simpara>\r
1295                         </listitem>\r
1296                         <listitem>\r
1297                         <simpara>\r
1298                         The message type (<literal>"type":"REQUEST"</literal>).\r
1299                         </simpara>\r
1300                         </listitem>\r
1301                         <listitem>\r
1302                         <simpara>\r
1303                         The locale for the message; if the OpenSRF method is locale-sensitive, it\r
1304                         can check the locale for each OpenSRF request and return different information\r
1305                         depending on the locale.\r
1306                         </simpara>\r
1307                         </listitem>\r
1308                         <listitem>\r
1309                         <simpara>\r
1310                         The payload of the message (<literal>"payload":{}</literal>) containing the OpenSRF method\r
1311                         request (<literal>"__c":"osrfMethod"</literal>) and its parameters (<literal>"__p:"{}</literal>).\r
1312                         </simpara>\r
1313                         <itemizedlist>\r
1314                         <listitem>\r
1315                         <simpara>\r
1316                         The method name for the request (<literal>"method":"opensrf.simple-text.reverse"</literal>).\r
1317                         </simpara>\r
1318                         </listitem>\r
1319                         <listitem>\r
1320                         <simpara>\r
1321                         A set of JSON parameters to pass to the method (<literal>"params":["foobar"]</literal>); in\r
1322                         this case, a single string <literal>"foobar"</literal>.\r
1323                         </simpara>\r
1324                         </listitem>\r
1325                         </itemizedlist>\r
1326                         </listitem>\r
1327                         </itemizedlist>\r
1328                         </callout>\r
1329                         <callout arearefs="CO8-4">\r
1330                         <simpara>\r
1331                         The URL on which the OpenSRF HTTP translator is listening,\r
1332                         <literal>/osrf-http-translator</literal> is the default location in the Apache example\r
1333                         configuration files shipped with the OpenSRF source, but this is configurable.\r
1334                         </simpara>\r
1335                         </callout>\r
1336                         </calloutlist>\r
1337                         <formalpara><title>Results from an HTTP POST request to an OpenSRF method via the OpenSRF HTTP Translator</title><para>\r
1338 <programlisting language="bash" linenumbering="unnumbered">\r
1339 # HTTP response broken up over multiple lines for legibility\r
1340 [{"__c":"osrfMessage","__p":   <co id="CO9-1"/>\r
1341     {"threadTrace":0, "payload": <co id="CO9-2"/>\r
1342         {"__c":"osrfResult","__p": <co id="CO9-3"/>\r
1343             {"status":"OK","content":"raboof","statusCode":200} <co id="CO9-4"/>\r
1344         },"type":"RESULT","locale":"en-CA" <co id="CO9-5"/>\r
1345     }\r
1346 },\r
1347 {"__c":"osrfMessage","__p":   <co id="CO9-6"/>\r
1348     {"threadTrace":0,"payload":  <co id="CO9-7"/>\r
1349         {"__c":"osrfConnectStatus","__p": <co id="CO9-8"/>\r
1350             {"status":"Request Complete","statusCode":205}<co id="CO9-9"/>\r
1351         },"type":"STATUS","locale":"en-CA"  <co id="CO9-10"/>\r
1352     }\r
1353 }]\r
1354 </programlisting>\r
1355                         </para></formalpara>\r
1356                         <calloutlist>\r
1357                         <callout arearefs="CO9-1">\r
1358                         <simpara>\r
1359                         The OpenSRF HTTP Translator returns an array of JSON objects in its\r
1360                         response. Each object in the response is an OpenSRF message\r
1361                         (<literal>"__c":"osrfMessage"</literal>) with a collection of response parameters (<literal>"__p":</literal>).\r
1362                         </simpara>\r
1363                         </callout>\r
1364                         <callout arearefs="CO9-2">\r
1365                         <simpara>\r
1366                         The OpenSRF message identifier (<literal>"threadTrace":0</literal>) confirms that this\r
1367                         message is in response to the request matching the same identifier.\r
1368                         </simpara>\r
1369                         </callout>\r
1370                         <callout arearefs="CO9-3">\r
1371                         <simpara>\r
1372                         The message includes a payload JSON object (<literal>"payload":</literal>) with an OpenSRF\r
1373                         result for the request (<literal>"__c":"osrfResult"</literal>).\r
1374                         </simpara>\r
1375                         </callout>\r
1376                         <callout arearefs="CO9-4">\r
1377                         <simpara>\r
1378                         The result includes a status indicator string (<literal>"status":"OK"</literal>), the content\r
1379                         of the result response - in this case, a single string "raboof"\r
1380                         (<literal>"content":"raboof"</literal>) - and an integer status code for the request\r
1381                         (<literal>"statusCode":200</literal>).\r
1382                         </simpara>\r
1383                         </callout>\r
1384                         <callout arearefs="CO9-5">\r
1385                         <simpara>\r
1386                         The message also includes the message type (<literal>"type":"RESULT"</literal>) and the\r
1387                         message locale (<literal>"locale":"en-CA"</literal>).\r
1388                         </simpara>\r
1389                         </callout>\r
1390                         <callout arearefs="CO9-6">\r
1391                         <simpara>\r
1392                         The second message in the set of results from the response.\r
1393                         </simpara>\r
1394                         </callout>\r
1395                         <callout arearefs="CO9-7">\r
1396                         <simpara>\r
1397                         Again, the message identifier confirms that this message is in response to\r
1398                         a particular request.\r
1399                         </simpara>\r
1400                         </callout>\r
1401                         <callout arearefs="CO9-8">\r
1402                         <simpara>\r
1403                         The payload of the message denotes that this message is an\r
1404                         OpenSRF connection status message (<literal>"__c":"osrfConnectStatus"</literal>), with some\r
1405                         information about the particular OpenSRF connection that was used for this\r
1406                         request.\r
1407                         </simpara>\r
1408                         </callout>\r
1409                         <callout arearefs="CO9-9">\r
1410                         <simpara>\r
1411                         The response parameters for an OpenSRF connection status message include a\r
1412                         verbose status (<literal>"status":"Request Complete"</literal>) and an integer status code for\r
1413                         the connection status (`"statusCode":205).\r
1414                         </simpara>\r
1415                         </callout>\r
1416                         <callout arearefs="CO9-10">\r
1417                         <simpara>\r
1418                         The message also includes the message type (<literal>"type":"RESULT"</literal>) and the\r
1419                         message locale (<literal>"locale":"en-CA"</literal>).\r
1420                         </simpara>\r
1421                         </callout>\r
1422                         </calloutlist>\r
1423                         <tip><simpara>Before adding a new public OpenSRF service, ensure that it does\r
1424                         not introduce privilege escalation or unchecked access to data. For example,\r
1425                         the Evergreen <literal>open-ils.cstore</literal> private service is an object-relational mapper\r
1426                         that provides read and write access to the entire Evergreen database, so it\r
1427                         would be catastrophic to expose that service publicly. In comparison, the\r
1428                         Evergreen <literal>open-ils.pcrud</literal> public service offers the same functionality as\r
1429                         <literal>open-ils.cstore</literal> to any connected HTTP client or OpenSRF client, but the\r
1430                         additional authentication and authorization layer in <literal>open-ils.pcrud</literal> prevents\r
1431                         unchecked access to Evergreen&#8217;s data.</simpara></tip>\r
1432                 </simplesect>\r
1433                 <simplesect id="_stateless_and_stateful_connections">\r
1434                         <title>Stateless and stateful connections</title>\r
1435                         <simpara>OpenSRF supports both <emphasis>stateless</emphasis> and <emphasis>stateful</emphasis> connections.  When an OpenSRF\r
1436                         client issues a <literal>REQUEST</literal> message in a <emphasis>stateless</emphasis> connection, the router\r
1437                         forwards the request to the next available service and the service returns the\r
1438                         result directly to the client.</simpara>\r
1439                         \r
1440                         <simpara>When an OpenSRF client issues a <literal>CONNECT</literal> message to create a <emphasis>stateful</emphasis> conection, the\r
1441                         router returns the Jabber ID of the next available service to the client so\r
1442                         that the client can issue one or more <literal>REQUEST</literal> message directly to that\r
1443                         particular service and the service will return corresponding <literal>RESULT</literal> messages\r
1444                         directly to the client. Until the client issues a <literal>DISCONNECT</literal> message, that\r
1445                         particular service is only available to the requesting client. Stateful connections\r
1446                         are useful for clients that need to make many requests from a particular service,\r
1447                         as it avoids the intermediary step of contacting the router for each request, as\r
1448                         well as for operations that require a controlled sequence of commands, such as a\r
1449                         set of database INSERT, UPDATE, and DELETE statements within a transaction.</simpara>\r
1450                         \r
1451                 </simplesect>\r
1452                 <simplesect id="_message_body_format">\r
1453                         <title>Message body format</title>\r
1454                         <simpara>OpenSRF was an early adopter of JavaScript Object Notation (JSON). While XMPP\r
1455                         is an XML protocol, the Evergreen developers recognized that the compactness of\r
1456                         the JSON format offered a significant reduction in bandwidth for the volume of\r
1457                         messages that would be generated in an application of that size. In addition,\r
1458                         the ability of languages such as JavaScript, Perl, and Python to generate\r
1459                         native objects with minimal parsing offered an attractive advantage over\r
1460                         invoking an XML parser for every message. Instead, the body of the XMPP message\r
1461                         is a simple JSON structure. For a simple request, like the following example\r
1462                         that simply reverses a string, it looks like a significant overhead: but we get\r
1463                         the advantages of locale support and tracing the request from the requester\r
1464                         through the listener and responder (drone).</simpara>\r
1465                         <formalpara><title>A request for opensrf.simple-text.reverse("foobar"):</title><para>\r
1466 <programlisting language="xml" linenumbering="unnumbered">\r
1467 &lt;message from='router@private.localhost/opensrf.simple-text'\r
1468   to='opensrf@private.localhost/opensrf.simple-text_listener_at_localhost_6275'\r
1469   router_from='opensrf@private.localhost/_karmic_126678.3719_6288'\r
1470   router_to='' router_class='' router_command='' osrf_xid=''\r
1471 &gt;\r
1472   &lt;thread&gt;1266781414.366573.12667814146288&lt;/thread&gt;\r
1473   &lt;body&gt;\r
1474 [\r
1475   {"__c":"osrfMessage","__p":\r
1476     {"threadTrace":"1","locale":"en-US","type":"REQUEST","payload":\r
1477       {"__c":"osrfMethod","__p":\r
1478         {"method":"opensrf.simple-text.reverse","params":["foobar"]}\r
1479       }\r
1480     }\r
1481   }\r
1482 ]\r
1483   &lt;/body&gt;\r
1484 &lt;/message&gt;\r
1485 </programlisting>\r
1486                         </para></formalpara>\r
1487                         <formalpara><title>A response from opensrf.simple-text.reverse("foobar")</title><para>\r
1488 <programlisting language="xml" linenumbering="unnumbered">\r
1489 &lt;message from='opensrf@private.localhost/opensrf.simple-text_drone_at_localhost_6285'\r
1490   to='opensrf@private.localhost/_karmic_126678.3719_6288'\r
1491   router_command='' router_class='' osrf_xid=''\r
1492 &gt;\r
1493   &lt;thread&gt;1266781414.366573.12667814146288&lt;/thread&gt;\r
1494   &lt;body&gt;\r
1495 [\r
1496   {"__c":"osrfMessage","__p":\r
1497     {"threadTrace":"1","payload":\r
1498       {"__c":"osrfResult","__p":\r
1499         {"status":"OK","content":"raboof","statusCode":200}\r
1500       } ,"type":"RESULT","locale":"en-US"}\r
1501   },\r
1502   {"__c":"osrfMessage","__p":\r
1503     {"threadTrace":"1","payload":\r
1504       {"__c":"osrfConnectStatus","__p":\r
1505         {"status":"Request Complete","statusCode":205}\r
1506       },"type":"STATUS","locale":"en-US"}\r
1507   }\r
1508 ]\r
1509   &lt;/body&gt;\r
1510 &lt;/message&gt;\r
1511 </programlisting>\r
1512                         </para></formalpara>\r
1513                         <simpara>The content of the <literal>&lt;body&gt;</literal> element of the OpenSRF request and result should\r
1514                         look familiar; they match the structure of the <link linkend="OpenSRFOverHTTP">OpenSRF over HTTP examples</link> that we previously dissected.</simpara>\r
1515                 </simplesect>\r
1516                 <simplesect id="_registering_opensrf_methods_in_depth">\r
1517                         <title>Registering OpenSRF methods in depth</title>\r
1518                         <simpara>Let&#8217;s explore the call to <literal>__PACKAGE__-&gt;register_method()</literal>; most of the members\r
1519                         of the hash are optional, and for the sake of brevity we omitted them in the\r
1520                         previous example. As we have seen in the results of the <link linkend="opensrfIntrospection">introspection call</link>, a\r
1521                         verbose registration method call is recommended to better enable the internal\r
1522                         documentation. Here is the complete set of members that you should pass to\r
1523                         <literal>__PACKAGE__-&gt;register_method()</literal>:</simpara>\r
1524                         <itemizedlist>\r
1525                         <listitem>\r
1526                         <simpara>\r
1527                         The <literal>method</literal> member specifies the name of the procedure in this module that is being registered as an OpenSRF method.\r
1528                         </simpara>\r
1529                         </listitem>\r
1530                         <listitem>\r
1531                         <simpara>\r
1532                         The <literal>api_name</literal> member specifies the invocable name of the OpenSRF method; by convention, the OpenSRF service name is used as the prefix.\r
1533                         </simpara>\r
1534                         </listitem>\r
1535                         <listitem>\r
1536                         <simpara>\r
1537                         The optional <literal>api_level</literal> member can be used for versioning the methods to allow the use of a deprecated API, but in practical use is always 1.\r
1538                         </simpara>\r
1539                         </listitem>\r
1540                         <listitem>\r
1541                         <simpara>\r
1542                         The optional <literal>argc</literal> member specifies the minimal number of arguments that the method expects.\r
1543                         </simpara>\r
1544                         </listitem>\r
1545                         <listitem>\r
1546                         <simpara>\r
1547                         The optional <literal>stream</literal> member, if set to any value, specifies that the method supports returning multiple values from a single call to \r
1548                         subsequent requests. OpenSRF automatically creates a corresponding method with ".atomic" appended to its name that returns the complete set of results in a \r
1549                         single request. Streaming methods are useful if you are returning hundreds of records and want to act on the results as they return.\r
1550                         </simpara>\r
1551                         </listitem>\r
1552                         <listitem>\r
1553                         <simpara>\r
1554                         The optional <literal>signature</literal> member is a hash that describes the method&#8217;s purpose, arguments, and return value.\r
1555                         </simpara>\r
1556                         <itemizedlist>\r
1557                         <listitem>\r
1558                         <simpara>\r
1559                         The <literal>desc</literal> member of the <literal>signature</literal> hash describes the method&#8217;s purpose.\r
1560                         </simpara>\r
1561                         </listitem>\r
1562                         <listitem>\r
1563                         <simpara>\r
1564                         The <literal>params</literal> member of the <literal>signature</literal> hash is an array of hashes in which each array element describes the corresponding method \r
1565                         argument in order.\r
1566                         </simpara>\r
1567                         <itemizedlist>\r
1568                         <listitem>\r
1569                         <simpara>\r
1570                         The <literal>name</literal> member of the argument hash specifies the name of the argument.\r
1571                         </simpara>\r
1572                         </listitem>\r
1573                         <listitem>\r
1574                         <simpara>\r
1575                         The <literal>desc</literal> member of the argument hash describes the argument&#8217;s purpose.\r
1576                         </simpara>\r
1577                         </listitem>\r
1578                         <listitem>\r
1579                         <simpara>\r
1580                         The <literal>type</literal> member of the argument hash specifies the data type of the argument: for example, string, integer, boolean, number, array, or hash.\r
1581                         </simpara>\r
1582                         </listitem>\r
1583                         </itemizedlist>\r
1584                         </listitem>\r
1585                         <listitem>\r
1586                         <simpara>\r
1587                         The <literal>return</literal> member of the <literal>signature</literal> hash is a hash that describes the return value of the method.\r
1588                         </simpara>\r
1589                         <itemizedlist>\r
1590                         <listitem>\r
1591                         <simpara>\r
1592                         The <literal>desc</literal> member of the <literal>return</literal> hash describes the return value.\r
1593                         </simpara>\r
1594                         </listitem>\r
1595                         <listitem>\r
1596                         <simpara>\r
1597                         The <literal>type</literal> member of the <literal>return</literal> hash specifies the data type of the return value: for example, string, integer, boolean, number, \r
1598                         array, or hash.\r
1599                         </simpara>\r
1600                         </listitem>\r
1601                         </itemizedlist>\r
1602                         </listitem>\r
1603                         </itemizedlist>\r
1604                         </listitem>\r
1605                         </itemizedlist>\r
1606                 </simplesect>\r
1607         </section>\r
1608         <section id="_evergreen_specific_opensrf_services">\r
1609                 <title>Evergreen-specific OpenSRF services</title>\r
1610                 <simpara>Evergreen is currently the primary showcase for the use of OpenSRF as an\r
1611                 application architecture. Evergreen 1.6.1 includes the following\r
1612                 set of OpenSRF services:</simpara>\r
1613                 <itemizedlist>\r
1614                 <listitem>\r
1615                 <simpara>\r
1616                 The <systemitem class="service"></systemitem> service supports common tasks for working with user\r
1617                      accounts and libraries.\r
1618                 </simpara>\r
1619                 </listitem>\r
1620                 <listitem>\r
1621                 <simpara>\r
1622                 The <systemitem class="service">open-ils.auth</systemitem> service supports authentication of Evergreen users.\r
1623                 </simpara>\r
1624                 </listitem>\r
1625                 <listitem>\r
1626                 <simpara>\r
1627                 The <systemitem class="service"></systemitem> service supports the management of reservations\r
1628                     for bookable items.\r
1629                 </simpara>\r
1630                 </listitem>\r
1631                 <listitem>\r
1632                 <simpara>\r
1633                 The <systemitem class="service"></systemitem> service supports common cataloging tasks, such as\r
1634                      creating, modifying, and merging bibliographic and authority records.\r
1635                 </simpara>\r
1636                 </listitem>\r
1637                 <listitem>\r
1638                 <simpara>\r
1639                 The <systemitem class="service">open-ils.circ</systemitem> service supports circulation tasks such as checking\r
1640                     out items and calculating due dates.\r
1641                 </simpara>\r
1642                 </listitem>\r
1643                 <listitem>\r
1644                 <simpara>\r
1645                 The <systemitem class="service">open-ils.collections</systemitem> service supports tasks that assist collections\r
1646                     agencies in contacting users with outstanding fines above a certain\r
1647                     threshold.\r
1648                 </simpara>\r
1649                 </listitem>\r
1650                 <listitem>\r
1651                 <simpara>\r
1652                 The <systemitem class="service">open-ils.cstore</systemitem> private service supports unrestricted access to\r
1653                     Evergreen fieldmapper objects.\r
1654                 </simpara>\r
1655                 </listitem>\r
1656                 <listitem>\r
1657                 <simpara>\r
1658                 The <systemitem class="service">open-ils.ingest</systemitem> private service supports tasks for importing\r
1659                     data such as bibliographic and authority records.\r
1660                 </simpara>\r
1661                 </listitem>\r
1662                 <listitem>\r
1663                 <simpara>\r
1664                 The <systemitem class="service">open-ils.pcrud</systemitem> service supports permission-based access to Evergreen\r
1665                     fieldmapper objects.\r
1666                 </simpara>\r
1667                 </listitem>\r
1668                 <listitem>\r
1669                 <simpara>\r
1670                 The <systemitem class="service">open-ils.penalty</systemitem> penalty service supports the calculation of\r
1671                     penalties for users, such as being blocked from further borrowing, for\r
1672                     conditions such as having too many items checked out or too many unpaid\r
1673                     fines.\r
1674                 </simpara>\r
1675                 </listitem>\r
1676                 <listitem>\r
1677                 <simpara>\r
1678                 The <systemitem class="service">open-ils.reporter</systemitem> service supports the creation and scheduling of\r
1679                     reports.\r
1680                 </simpara>\r
1681                 </listitem>\r
1682                 <listitem>\r
1683                 <simpara>\r
1684                 The <systemitem class="service">open-ils.reporter-store</systemitem> private service supports access to Evergreen\r
1685                     fieldmapper objects for the reporting service.\r
1686                 </simpara>\r
1687                 </listitem>\r
1688                 <listitem>\r
1689                 <simpara>\r
1690                 The <systemitem class="service"></systemitem> service supports searching across bibliographic\r
1691                     records, authority records, serial records, Z39.50 sources, and ZIP codes.\r
1692                 </simpara>\r
1693                 </listitem>\r
1694                 <listitem>\r
1695                 <simpara>\r
1696                 The <systemitem class="service"></systemitem> private service supports a deprecated method of\r
1697                     providing access to Evergreen fieldmapper objects. Implemented in Perl,\r
1698                     this service has largely been replaced by the much faster C-based\r
1699                     <literal>open-ils.cstore</literal> service.\r
1700                 </simpara>\r
1701                 </listitem>\r
1702                 <listitem>\r
1703                 <simpara>\r
1704                 The <systemitem class="service">open-ils.supercat</systemitem> service supports transforms of MARC records into\r
1705                     other formats, such as MODS, as well as providing Atom and RSS feeds and\r
1706                     SRU access.\r
1707                 </simpara>\r
1708                 </listitem>\r
1709                 <listitem>\r
1710                 <simpara>\r
1711                 The <systemitem class="service">open-ils.trigger</systemitem> private service supports event-based triggers for\r
1712                     actions such as overdue and holds available notification emails.\r
1713                 </simpara>\r
1714                 </listitem>\r
1715                 <listitem>\r
1716                 <simpara>\r
1717                 The <systemitem class="service">open-ils.vandelay</systemitem> service supports the import and export of batches of\r
1718                     bibliographic and authority records.\r
1719                 </simpara>\r
1720                 </listitem>\r
1721                 </itemizedlist>\r
1722                 <simpara>Of some interest is that the <systemitem class="service">open-ils.reporter-store</systemitem> and <systemitem class="service">open-ils.cstore</systemitem>\r
1723                 services have identical implementations. Surfacing them as separate services\r
1724                 enables a deployer of Evergreen to ensure that the reporting service does not\r
1725                 interfere with the performance-critical <systemitem class="service">open-ils.cstore</systemitem> service. One can also\r
1726                 direct the reporting service to a read-only database replica to, again, avoid\r
1727                 interference with <systemitem class="service">open-ils.cstore</systemitem> which must write to the master database.</simpara>\r
1728                 <simpara>There are only a few significant services that are not built on OpenSRF in\r
1729                 Evergreen 1.6.0, such as the SIP and Z39.50 servers. These services implement\r
1730                 different protocols and build on existing daemon architectures (Simple2ZOOM\r
1731                 for Z39.50), but still rely on the other OpenSRF services to provide access\r
1732                 to the Evergreen data. The non-OpenSRF services are reasonably self-contained\r
1733                 and can be deployed on different servers to deliver the same sort of deployment\r
1734                 flexibility as OpenSRF services, but have the disadvantage of not being\r
1735                 integrated into the same configuration and control infrastructure as the\r
1736                 OpenSRF services.</simpara>\r
1737         </section>\r
1738 </chapter>\r