working/OpenSRF.git
3 months agoLp 1808580: Remove Installation Support for Ubuntu 14.04 master
Jason Stephenson [Fri, 14 Dec 2018 20:35:49 +0000 (15:35 -0500)]
Lp 1808580: Remove Installation Support for Ubuntu 14.04

Remove references to Ubuntu 14.04 from the README and the
Makefile.install prerequisites.

Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Ben Shum <ben@evergreener.net>
5 months agoLP#1729610: extend backlog queue to C apps
Galen Charlton [Wed, 12 Dec 2018 19:35:56 +0000 (14:35 -0500)]
LP#1729610: extend backlog queue to C apps

This patch extends the notion of a backlog queue to C apps and
offers the same functionality as the Perl side of the patch series:

- max_backlog_queue configuration setting
- ability to queue messages up to the configured limit
- ability to drop requests that would overflow the backlog
  queue and send status 503 exceptions back to the client.

This patch also adds a new service, opensrf.cslow, that implements
a opensrf.cslow.wait method similar to the Perl opensrf.slooooooow
service.

To test
-------
[1] Set a low max_backlog_queue for opensrf.cslow and a low
    max_children.
[2] Arrange for srfsh to fire off a bunch of opensrf.cslow.wait
    requests.
[3] Verify that requests that come in after the backlog queue fills
    up immediately get 503 exceptions.

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
5 months agoLP#1729610: return new OpenSRF status if backlog queue fills up
Galen Charlton [Wed, 7 Nov 2018 17:03:17 +0000 (12:03 -0500)]
LP#1729610: return new OpenSRF status if backlog queue fills up

This patch teaches Perl services how to return a new OpenSRF status,
OSRF_STATUS_SERVICEUNAVAILABLE (code 503) if the backlog queue
for a service gets full.

To test
-------
[1] Set a low max_backlog_queue for opensrf.sloooow and a low
    max_children.
[2] Arrange for srfsh to fire off a bunch of opensrf.sloooow.wait
    requests.
[3] Verify that requests that come in after the backlog queue fills
    up immediately get 503 exceptions.

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
5 months agoLP#1729610: make it possible to set max_backlog_queue in opensrf.xml
Galen Charlton [Tue, 6 Nov 2018 20:14:55 +0000 (15:14 -0500)]
LP#1729610: make it possible to set max_backlog_queue in opensrf.xml

This patch adds a new service configuration option, max_backlog_queue,
to allow controlling the size of the backlog queue.

If not otherwise specified in opensrf.xml, max_backlog_queue defaults
to 1000.

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
5 months agoLP#1729610: Fix incorrect param description
Remington Steed [Wed, 31 Jan 2018 21:05:52 +0000 (16:05 -0500)]
LP#1729610: Fix incorrect param description

Mike confirmed that the code expects "pause" to be an integer. This
commit changes the description to reflect that, and to mention the
default value.

Signed-off-by: Remington Steed <rjs7@calvin.edu>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
5 months agoLP#1729610: Add some debug/internal logging to backlog queue
Mike Rylander [Tue, 24 Oct 2017 20:45:09 +0000 (16:45 -0400)]
LP#1729610: Add some debug/internal logging to backlog queue

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
5 months agoLP#1729610: Add a service useful for testing behavior in slow response conditions
Mike Rylander [Wed, 1 Nov 2017 15:11:08 +0000 (11:11 -0400)]
LP#1729610: Add a service useful for testing behavior in slow response conditions

This service, opensrf.slooooooow, offers a opensrf.slooooooow.wait method
that waits for the number of seconds specified in its sole parameter.

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
5 months agoLP#1729610: Allow queuing (for a while) during child backlog
Mike Rylander [Tue, 24 Oct 2017 17:27:37 +0000 (13:27 -0400)]
LP#1729610: Allow queuing (for a while) during child backlog

This patch teaches OpenSRF listeners for Perl services how to maintain
a queue of requests in case no drone process is immediately available
to process a requeust.

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
6 months agoLP#1803182 Websocketd graceful shutdown support
Bill Erickson [Tue, 13 Nov 2018 22:21:18 +0000 (17:21 -0500)]
LP#1803182 Websocketd graceful shutdown support

Give websocketd clients a chance to complete any requests that are
in-process at time of shutdown.

Graceful shutdown is initiated by sending a SIGUSR1 signal to the
websocket-osrf child processes or to the parent process group.

This can be done directly via:

$ kill -s USR1 -$websocketd_parent_pid

Or via systemd for systemd users:

$ sudo systemctl kill -s USR1 websocketd-osrf

Note the websocketd parent process ignores SIGUSR1.

Once initiated, the websocketd backend goes into shutdown mode polling
for a graceful shutdown window, which occurs when all request up to now
have been completed.  Once that moment arrives, the client is kicked off
and must connect to a new websocketd instance to issue any new requests.

The polling period lasts for SHUTDOWN_MAX_GRACEFUL_SECONDS seconds (in
osrf-websocket-stdio.c), which currently defaults to 2 minutes.

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
7 months agoLP#1706147 Perl Force-Recycle drone option
Bill Erickson [Mon, 24 Jul 2017 16:08:56 +0000 (12:08 -0400)]
LP#1706147 Perl Force-Recycle drone option

Creates an API-level option to inform the OpenSRF drone management code
that the running drone should be recycled upon completion of the current
OpenSRF session.  This allows for quicker release of resources consumed
by the drone.

To use:

sub some_api_method {
    my ($self, $client, ...) = @_;
    $self->session->force_recycle(1);
    ...
}

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
7 months agoLP#1684970 Translator compatible with mod_remoteip
Bill Erickson [Mon, 27 Aug 2018 14:12:57 +0000 (10:12 -0400)]
LP#1684970 Translator compatible with mod_remoteip

Teach the OSRF Translator to request the IP address of the user agent
(e.g. web browser) instead of the IP address of the up stream client,
which may be a proxy, using the Apache 2.4 request_rec->useragent_ip
value.

http://httpd.apache.org/docs/2.4/developer/new_api_2_4.html

This make is possible for the translator to access the client IP with
Apache's mod_remoteip enabled and configured.

Includes sample config and install documentation.

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
7 months agoLP#1711145 NGINX sample websocketd configs
Bill Erickson [Thu, 13 Sep 2018 17:08:37 +0000 (13:08 -0400)]
LP#1711145 NGINX sample websocketd configs

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
7 months agoLP#1711145 NGINX sample config security improvements
Bill Erickson [Wed, 11 Jul 2018 16:27:05 +0000 (12:27 -0400)]
LP#1711145 NGINX sample config security improvements

* Adds security recommendations from
https://mozilla.github.io/server-side-tls/ssl-config-generator/
* Enables http2
* Apply a 5-minute proxy read timeout to avoid too-short timeouts on
  long API calls.
* Adds a (commented) section on sending nginx logs to syslog

Includes INSTALL notes on generating the dhparam file.

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
7 months agoReorder Ejabberd configuration steps for Ubuntu 18.04.
Jason Stephenson [Mon, 5 Nov 2018 15:42:32 +0000 (10:42 -0500)]
Reorder Ejabberd configuration steps for Ubuntu 18.04.

Rearrange the steps for modifying /etc/ejabberd/ejabberd.yml on Ubuntu
18.04 so that they are in the same order that they appear in the file.

Signed-off-by: Jason Stephenson <jason@sigio.com>
7 months agoLP#1776510 JS libs handle transport errors
Bill Erickson [Tue, 12 Jun 2018 16:12:45 +0000 (12:12 -0400)]
LP#1776510 JS libs handle transport errors

Teach the websocket client code to look for the transport_error flag
applied to the websocket wrapper message by the websocket gateway when a
request for an unavilable service is made.

When encountered, fire the transport or generic error handler callbacks,
if available.  Avoid any attempts to further process the message.

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
7 months agoUpdate README to include Bionic steps
Ben Shum [Thu, 20 Sep 2018 15:34:45 +0000 (11:34 -0400)]
Update README to include Bionic steps

Note the use of mod_legacy_auth in ejabberd 18.x version for Ubuntu Bionic

Signed-off-by: Ben Shum <ben@evergreener.net>
Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
7 months agoAdd support for Ubuntu 18.04 Bionic Beaver to Makefile.install
Chris Sharp [Tue, 7 Aug 2018 23:43:45 +0000 (19:43 -0400)]
Add support for Ubuntu 18.04 Bionic Beaver to Makefile.install

Signed-off-by: Chris Sharp <csharp@georgialibraries.org>
Signed-off-by: Ben Shum <ben@evergreener.net>
Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
8 months agoLP#1703411 XMPP opensrf element make check repairs
Bill Erickson [Thu, 20 Sep 2018 22:04:09 +0000 (18:04 -0400)]
LP#1703411 XMPP opensrf element make check repairs

Update the transport_message unit tests to check for the new <opensrf/>
element for relaying custom commands.

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
8 months agoLP#1703411 XMPP opensrf sub-element repairs
Bill Erickson [Wed, 19 Sep 2018 19:34:49 +0000 (15:34 -0400)]
LP#1703411 XMPP opensrf sub-element repairs

* Message template typo repair -- missing "'"
* XPath repair on path to opensrf sub-element for Perl
* Move 'type' attribute get/set back up to the <message> for Perl.
* Clean up code duplication in the C message building libs.
* Squash a centuries-old memory leak where xmlFree(sender) was only
  called if a router_from was not supplied.

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
8 months agoLP#1703411: Move OpenSRF XMPP attrs to subelement
Mike Rylander [Tue, 28 Aug 2018 18:32:02 +0000 (14:32 -0400)]
LP#1703411: Move OpenSRF XMPP attrs to subelement

Modern versions of Ejabberd strip custom XML attributes which appear
outside of custom elements.  To support OpenSRF's custom router and
osrf_xid commands, move these attributes into a new custom XML element
<opensrf>.

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Jason Stephenson <jason@sigio.com>
9 months agoLp#1718459: Remove Debian 7 Wheezy installation support.
Jason Stephenson [Wed, 20 Sep 2017 15:56:34 +0000 (11:56 -0400)]
Lp#1718459: Remove Debian 7 Wheezy installation support.

Remove references to Debian 7 Wheezy in the README.

Remove src/extras/Makefile.install targets for Wheezy.

Remove the examples/apache2 directory, since none of the supported
distros come with Apache 2.2 any longer.

To test: Simply install on a supported distribution other than Debian
7 Wheezy and do the usual tests.

Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Ben Shum <ben@evergreener.net>
9 months agoLP#1777180: Add zip/unzip to prerequisites
Ben Shum [Wed, 12 Sep 2018 17:07:18 +0000 (13:07 -0400)]
LP#1777180: Add zip/unzip to prerequisites

Need unzip for new websocketd unpackaging. Also get zip just because.

Signed-off-by: Ben Shum <ben@evergreener.net>
Signed-off-by: Jason Stephenson <jason@sigio.com>
9 months agoLP#1777180 Update README for websocketd
Jason Stephenson [Wed, 12 Sep 2018 14:58:58 +0000 (10:58 -0400)]
LP#1777180 Update README for websocketd

Add optional command to run websocketd without using a proxy.

Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Ben Shum <ben@evergreener.net>
9 months agoLP#1777180 Websocketd gateway and test scripts
Bill Erickson [Sat, 9 Jun 2018 23:05:25 +0000 (19:05 -0400)]
LP#1777180 Websocketd gateway and test scripts

Adds a new OpenSRF binary/program for relaying websockets messages
to and from a websocketd instance.  The new binary
(osrf-websocket-stdio) performs the same tasks as the
osrf_websocket_translator.c Apache module, minus the Apache module glue
and the extra threading required to run within the Apache module.

Commit includes 2 test scripts (tester.pl and test-stateful.pl) for
generating large series of test messages to send to a websockets
instance. tester.pl sends echo requests only, test-stateful.pl sends
connect->echo-request->disconnect batches across a configurable number
of forked processes.

INSTALL document updated to include websocketd setup as an alternative
to Apache websockets.

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Jeff Davis <jeff.davis@bc.libraries.coop>
Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Ben Shum <ben@evergreener.net>
10 months agoLP#1702978: memcache Get methods use key as va_list format
Mike Rylander [Fri, 7 Jul 2017 19:22:21 +0000 (15:22 -0400)]
LP#1702978: memcache Get methods use key as va_list format

And, when a key (composed of, say, a username or barcode) has a % in it,
bad things happen.  We will stop acting as if these are variadic functions
now, and also update Evergreen so that it does not do that either.

TODO: Make these actually non-variadic, but that breaks ABI.

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Cesar Velez <cesar.velez@equinoxinitiative.org>
Signed-off-by: Jason Stephenson <jason@sigio.com>
13 months agoLP#1243841: quiet a misleading indentation warning
Galen Charlton [Mon, 7 May 2018 20:03:16 +0000 (16:03 -0400)]
LP#1243841: quiet a misleading indentation warning

Adjusts a spaces-vs-tabs issue to fix the following compilation
warning:

osrf_prefork.c: In function ‘check_children’:
osrf_prefork.c:1067:5: warning: this ‘if’ clause does not
guard... [-Wmisleading-indentation]
     if( select_ret <= 0 ) // we're done here
     ^~
osrf_prefork.c:1072:2: note: ...this statement, but the latter is
misleadingly indented as if it is guarded by the ‘if’
  cur_child = forker->first_child;
  ^~~~~~~~~

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
13 months agoLP#1243841 - Quiet remaining Make install warnings.
Jason Stephenson [Tue, 7 Nov 2017 21:39:39 +0000 (16:39 -0500)]
LP#1243841 - Quiet remaining Make install warnings.

We silence the following warnings:

apachetools.c:179:15: warning: initialization discards 'const' qualifier
from pointer target type [enabled by default]

apachetools.c:181:8: warning: assignment discards 'const' qualifier from
pointer target type [enabled by default]

apachetools.c:183:8: warning: assignment discards 'const' qualifier from
pointer target type [enabled by default]

./osrf_websocket_translator.c:541:9: warning: return makes integer from
pointer without a cast [enabled by default]

./osrf_http_translator.c:300:25: warning: passing argument 1 of 'free'
discards 'const' qualifier from pointer target type [enabled by default]

Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
13 months agoLP#1243841 - Quiet additional Make warnings and some code cleanup.
Jason Stephenson [Tue, 7 Nov 2017 20:58:05 +0000 (15:58 -0500)]
LP#1243841 - Quiet additional Make warnings and some code cleanup.

We make the following warnings go away:

osrf_legacy_json.c:607:6: warning: variable ‘fourth_dash’ set but
not used [-Wunused-but-set-variable]

osrf_legacy_json.c:836:5: warning: passing argument 3 of ‘makeNode’
discards ‘const’ qualifier from pointer target type [enabled by
default]

utils.c:133:2: warning: format not a string literal and no format
arguments [-Wformat-security]

We also cleanup the while block nested in a do while block around line
63 of osrf_cache.c to be more readable by adding braces and breaking
it across 3 lines.

Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
13 months agoLP#1243841 - Quiet wrong format warnings during make install.
Chris Sharp [Wed, 20 Sep 2017 01:25:12 +0000 (21:25 -0400)]
LP#1243841 - Quiet wrong format warnings during make install.

During make install, the compiler warns that %d expects an int when the
actual value is a long int.  Changing %d to %ld fixes the issue.

Signed-off-by: Chris Sharp <csharp@georgialibraries.org>
Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
13 months agoLP#1243841 - Quiet unused return value warning in srfsh.c
Chris Sharp [Wed, 20 Sep 2017 00:46:28 +0000 (20:46 -0400)]
LP#1243841 - Quiet unused return value warning in srfsh.c

Using the technique described here: https://stackoverflow.com/a/13999461.

Signed-off-by: Chris Sharp <csharp@georgialibraries.org>
Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
13 months agoLP#1243841 - Quiet unused return value warnings.
Chris Sharp [Wed, 20 Sep 2017 00:37:13 +0000 (20:37 -0400)]
LP#1243841 - Quiet unused return value warnings.

Using advice given here: https://stackoverflow.com/a/13999461, "The
only good (if ugly) way to suppress these is to convert the return
value into something that the compiler agrees that you can ignore."

Signed-off-by: Chris Sharp <csharp@georgialibraries.org>
Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
13 months agoLP#1243841 - Wrap truth test using "=" in an extra set of parens.
Chris Sharp [Tue, 19 Sep 2017 22:10:00 +0000 (18:10 -0400)]
LP#1243841 - Wrap truth test using "=" in an extra set of parens.

Referring to https://stackoverflow.com/questions/5476759/compiler-warning-suggest-parentheses-around-assignment-used-as-truth-value,
it is recommended to wrap variable assignments that are used as a truth
test within an extra set of parentheses.

Signed-off-by: Chris Sharp <csharp@georgialibraries.org>
Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
13 months agoLP1340982: Ignore both 'comment' and '#comment' nodes in Settings Parse.
Jason Stephenson [Sun, 18 Feb 2018 17:12:13 +0000 (12:12 -0500)]
LP1340982: Ignore both 'comment' and '#comment' nodes in Settings Parse.

Modify the XML2perl helper function in OpenSRF::Utils::SettingsParser
to ignore comment nodes that have names of comment and #comment.

To reproduce the bug:

1. Stop OpenSRF services.

2. Add a XML comment, any text betwen <!-- and -->, in the list of
MARC templates in the opensrf.xml file.

3. Restart OpenSRF services.

4. In the staff client, go to Cataloging->Create New MARC Record.

5. In the list of templates, you will #comment where you added the
comment. (NOTE: I only tested with 1 comment.  Things may get worse
with more than 1 comment.)

6. There will be an error like the following in the
open-ils.cat_stderr.log if you attempt to choose the #comment entry:

   Caught error from 'run' method: Exception: OpenSRF::EX::ERROR
   2018-02-18T11:24:30 OpenSRF::Application
   /usr/local/share/perl/5.22.1/OpenSRF/Application.pm:240 System
   ERROR: Unable to open MARC template file: HASH(0x2249928) :

To test the fix:

1. Leave the comments in the opensrf.xml file from before.

2. Apply this commit, do make and make install for OpenSRF.

3. Just to make sure, do make and make install for Evergreen.

4. Repeat steps 3 and 4 from above.

5. You should NOT see any templates named #comment.

Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
13 months agoLP#1762815 Empty client TZ defaults to server TZ (Perl)
Bill Erickson [Tue, 10 Apr 2018 19:06:56 +0000 (15:06 -0400)]
LP#1762815 Empty client TZ defaults to server TZ (Perl)

Fixes an issue in the Perl client time zone handling that resulted in
the server defaulting to UTC time instead of the server time zone when
no time zone value was received from the client.

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Jason Stephenson <jason@sigio.com>
16 months agoLP#1744158 Websocket proc exits on ejabberd disconnect
Bill Erickson [Wed, 24 Jan 2018 20:16:14 +0000 (15:16 -0500)]
LP#1744158 Websocket proc exits on ejabberd disconnect

Any errors relaying websocket messages to OpenSRF now result in the WS
client being disconnected, allowing the WS process to exit.  This
prevents the WS gateway from accepting requests it cannot process and
allows the client to connect to a new WS process.

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Jason Stephenson <jason@sigio.com>
16 months agoLP#1746577 Websocket responder exits on jabber disconnect
Bill Erickson [Wed, 31 Jan 2018 18:07:41 +0000 (13:07 -0500)]
LP#1746577 Websocket responder exits on jabber disconnect

Regularly check for Jabber socket disconnects in the websocket gateway
"responder" thread (that relays messages from opensrf to the WS client)
and force a client disconnect when a broken jabber socket is detected.

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Chris Sharp <csharp@georgialibraries.org>
Signed-off-by: Jason Stephenson <jason@sigio.com>
20 months agoand continue to bump up version numbers
Galen Charlton [Thu, 21 Sep 2017 13:51:20 +0000 (09:51 -0400)]
and continue to bump up version numbers

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
20 months agobump up version numbers in master to reflect 3.0 as next major release branch
Galen Charlton [Wed, 20 Sep 2017 14:57:09 +0000 (10:57 -0400)]
bump up version numbers in master to reflect 3.0 as next major release branch

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
20 months agoLP#1708048: Fix ld problems by renaming libraries.
Jason Stephenson [Tue, 12 Sep 2017 15:29:41 +0000 (11:29 -0400)]
LP#1708048: Fix ld problems by renaming libraries.

We add the lib... prefix to all C library names as expected by most
linkers.  This necessitates changes to the opensrf.xml configuration
file.

Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
20 months agoLP#1708048: Add support for Debian 9 Stretch
Ben Shum [Tue, 1 Aug 2017 20:07:16 +0000 (16:07 -0400)]
LP#1708048: Add support for Debian 9 Stretch

  - Add new make target "debian-stretch"
  - Add Stretch instructions to README

Signed-off-by: Ben Shum <ben@evergreener.net>
Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
21 months agoLP#1717350: fix chunking messages sent from Perl services
Galen Charlton [Thu, 14 Sep 2017 21:36:40 +0000 (17:36 -0400)]
LP#1717350: fix chunking messages sent from Perl services

The chunk size adjustments introduced by the patches for bug 1709710
could result in calculating non-integral chunk sizes for Perl services.
This in turn led to fractional offsets and lengths being passed to
Perl's substr() function; rounding in turn can lead to characters
in the response getting silently dropped, which in turn broke
parsing the reconstituted response.

To test
-------
The problem can be reproduced by invoking

  open-ils.storage open-ils.storage.actor.org_unit.descendants.atomic 1, 0

in Evergreen databases with org unit trees large enough that the
response gets chunked.

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Jason Stephenson <jason@sigio.com>
21 months agoLP#1709710 Count Perl chunk/bundle sizes in bytes
Bill Erickson [Fri, 8 Sep 2017 21:53:52 +0000 (17:53 -0400)]
LP#1709710 Count Perl chunk/bundle sizes in bytes

For the purposes of bundling/chunking, count the number of bytes in each
affected string instead of the number of characters.

See also https://perldoc.perl.org/bytes.html and 'perldoc -f length'

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
21 months agoLP#1709710: write unit tests for osrfXmlEscapingLength()
Galen Charlton [Fri, 8 Sep 2017 19:34:29 +0000 (15:34 -0400)]
LP#1709710: write unit tests for osrfXmlEscapingLength()

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
21 months agoLP#1709710: Make chunk sizing smart about XML quoting
Mike Rylander [Fri, 18 Aug 2017 15:43:31 +0000 (11:43 -0400)]
LP#1709710: Make chunk sizing smart about XML quoting

XML inside JSON as a quoted string that's itself inside XML causes quite the
pile up of nested excaping of certain characters in OpenSRF PARTIAL_RESPONSE
messages.  Here we check for the worst offenders (<, >, &, and ") and account
for the cost of escaping them in chunked response stanzas.

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
21 months agoLP#1690206 - remove check of httpd.conf from OpenSRF Makefile.install
Chris Sharp [Thu, 11 May 2017 18:47:32 +0000 (14:47 -0400)]
LP#1690206 - remove check of httpd.conf from OpenSRF Makefile.install

The original src/extras/Makefile.install included a grep of the
file /etc/apache2/httpd.conf, which is no longer installed by default
on supported Debian and Ubuntu releases.  As this check results in
an error message, it makes sense to remove the check altogether.

Signed-off-by: Chris Sharp <csharp@georgialibraries.org>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
21 months agoLP#1697029 Log and exit on write to dead child
Bill Erickson [Fri, 9 Jun 2017 17:01:46 +0000 (13:01 -0400)]
LP#1697029 Log and exit on write to dead child

Confirm that a child process is alive just before attempting to write to
its pipe.  If the child process is dead, log the error, then drop the
message and move on.  This allows the parent to continue servicing
future requests.

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
22 months agoLP#1711194: avoid division by zero errors
Galen Charlton [Fri, 18 Aug 2017 15:11:29 +0000 (11:11 -0400)]
LP#1711194: avoid division by zero errors

Under some circumstances, osrf_control --diagnostic may not
be able to deduce the correct max_children setting for a service,
e.g., if it's run without --localhost on a system that doesn't
have a opensrf.xml config section specifying active apps for a
specific hostname. When that happens, just display the count
of running drones rather than displaying error messages about
undefined $dmax values and divisions by zero.

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Bill Erickson <berickxx@gmail.com>
22 months agoLP#1711194 osrf_config --diagnostic reports max-children
Bill Erickson [Wed, 16 Aug 2017 18:10:13 +0000 (14:10 -0400)]
LP#1711194 osrf_config --diagnostic reports max-children

--diagnostic output shows the maximum number of allowed drones per
service along with the active drone count.  Additionally, a usage
percentage value is displayed.

E.g.

$ /openils/bin/osrf_control -l --diagnostic

* open-ils.acq   [1581] uptime=00:35 cputime=00:00:00 #drones=1/15 6%
* open-ils.actor [1629] uptime=00:35 cputime=00:00:00 #drones=2/15 13%

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Chris Sharp <csharp@georgialibraries.org>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
22 months agoLP#1704090: ensure make install respects DESTDIR
Graham Billiau [Mon, 31 Jul 2017 15:00:25 +0000 (11:00 -0400)]
LP#1704090: ensure make install respects DESTDIR

This patch ensures that 'make install' uses DESTDIR consistently,
making life easier for packagers.

Signed-off-by: Graham Billiau <graham@geeksinthegong.net>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
22 months agoRevert "LP#1635737 Add optional context to interval_to_seconds"
Mike Rylander [Mon, 31 Jul 2017 19:15:09 +0000 (15:15 -0400)]
Revert "LP#1635737 Add optional context to interval_to_seconds"

This reverts commit a481100ef9d5bd9eaad5a87ce29776cb07a8687c.

22 months agoRevert "LP#1635737: Unit tests for DST and date math"
Mike Rylander [Mon, 31 Jul 2017 19:15:02 +0000 (15:15 -0400)]
Revert "LP#1635737: Unit tests for DST and date math"

This reverts commit 316f58375c40087bfae32234c4dd3817260831e5.

22 months agoLP#1635737: Unit tests for DST and date math
Mike Rylander [Mon, 31 Jul 2017 16:53:59 +0000 (12:53 -0400)]
LP#1635737: Unit tests for DST and date math

Signed-off-by: Mike Rylander <mrylander@gmail.com>
22 months agoLP#1635737 Add optional context to interval_to_seconds
Dan Wells [Fri, 21 Jul 2017 18:17:34 +0000 (14:17 -0400)]
LP#1635737 Add optional context to interval_to_seconds

Any given interval (e.g. "1 month") can be a different amount of
seconds depending on the context (i.e. "1 month" after February 1 is
March 1, but "1 month" after March 1 is April 1, yet March is longer
than February).  This affects months all the time, but also can affect
days, hours, and even seconds once you consider DST and "leap" times.

By giving an optional context to interval_to_seconds, you can find the
true number of seconds in, for example, "1 month", when starting from
"February 1" (the context).

Signed-off-by: Dan Wells <dbw2@calvin.edu>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
22 months agoLP 1703958: Update Websockets Intructions for Debian Jessie
Jason Stephenson [Wed, 12 Jul 2017 19:24:48 +0000 (15:24 -0400)]
LP 1703958: Update Websockets Intructions for Debian Jessie

Modify the Websockets installation instructions so that users will
have a working installation on Debian Jessie.  We change the "Debian"
instructions to say "Debian Wheezy" and add "Debian Jessie" to the
list with "Ubuntu Trusty / Xenial" where necessary.

Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
22 months agoLP#1704116: fix intermittant failure of parallel building
Graham Billiau [Thu, 13 Jul 2017 15:29:43 +0000 (11:29 -0400)]
LP#1704116: fix intermittant failure of parallel building

Running make to do a parallel build (e.g., 'make -j 4' or
the like) could fail because of an undeclared dependency; this
patch fixes the issue.

The issue can be reproduced by repeating "make clean; make -j 4"
until the build fails.

Signed-off-by: Graham Billiau <graham@geeksinthegong.net>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
2 years agoLP#1672926: Disable/remove default nginx config in REAMDE steps
Ben Shum [Wed, 15 Mar 2017 03:00:56 +0000 (23:00 -0400)]
LP#1672926: Disable/remove default nginx config in REAMDE steps

Since we will be adding our own custom nginx config, disable/remove the
default one that is installed so there's no confusion later on.

Signed-off-by: Ben Shum <ben@evergreener.net>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
2 years agoLP#1670045 - Correct Nginx README instructions
Chris Sharp [Sat, 4 Mar 2017 20:22:20 +0000 (15:22 -0500)]
LP#1670045 - Correct Nginx README instructions

Previously, following the optional Nginx server setup
instructions resulted in failure for nginx to install
because Apache is already using ports 80 and 443.  Switch
the steps so that we change Apache's listening ports before
APT installation.

Signed-off-by: Chris Sharp <csharp@georgialibraries.org>
Signed-off-by: Ben Shum <ben@evergreener.net>
2 years agoRemove Fedora as a well-tested target
Dan Scott [Thu, 2 Mar 2017 15:31:02 +0000 (10:31 -0500)]
Remove Fedora as a well-tested target

The Fedora install target requires a number of updates and is now
far from a "well-tested target".

Signed-off-by: Dan Scott <dscott@laurentian.ca>
2 years agoLP#1667091 Remove non-SSL websockets sample configs
Bill Erickson [Wed, 22 Feb 2017 21:05:25 +0000 (16:05 -0500)]
LP#1667091 Remove non-SSL websockets sample configs

All WebSockets communication occurs via SSL in the sample configuration
files.  Non-SSL communication is still an option if required with local
configuration.

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
2 years agoLP#1666706: add --with-websockets-port configure option
Galen Charlton [Tue, 21 Feb 2017 21:21:23 +0000 (16:21 -0500)]
LP#1666706: add --with-websockets-port configure option

This patch adds the ability to specify the WebSockets
port as a configure option, avoiding the need to manually
edit a couple JavaScript files if one chooses to proxy
WS traffic.

This patch also cleans up the set of JavaScript files that
are installed so that some test case files are not installed
by default.

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
2 years agoLP#1616501: teach mod_perl handlers how to detect client disconnects
Mike Rylander [Thu, 4 Aug 2016 12:57:44 +0000 (08:57 -0400)]
LP#1616501: teach mod_perl handlers how to detect client disconnects

This patch provides an API so that mod_perl handlers
that act as OpenSRF clients have a way to specify that
if the browser disconnects, to stop trying to receive
results from an XMPP request.

To invoke it, mod_perl handlers can add the following:

use OpenSRF;
...
sub hander {
...
    my $r = shift;
    OpenSRF->OSRF_APACHE_REQUEST_OBJ($r);
...

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
2 years agoLP#1652382: more improvements to cache key munging
Galen Charlton [Wed, 15 Feb 2017 21:58:06 +0000 (16:58 -0500)]
LP#1652382: more improvements to cache key munging

- teach osrfCacheRemove to clean keys
- fix implict declaration compilation warning
- account for fact that iscntrl('\0') returns true

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Jeff Davis <jdavis@sitka.bclibraries.ca>
2 years agoLP#1652382: handle cases where supplied key is longer than 250 bytes
Galen Charlton [Wed, 15 Feb 2017 19:12:34 +0000 (14:12 -0500)]
LP#1652382: handle cases where supplied key is longer than 250 bytes

With this patch, if cache clients want to use a key longer
than the memcached text protocol limit of 250 bytes, the
key is normalized to 'shortened_' + md5_hex(normalized_key).

Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Jeff Davis <jdavis@sitka.bclibraries.ca>
2 years agoLP#1652382: Make use of the clean key just created
Mike Rylander [Mon, 30 Jan 2017 17:54:10 +0000 (12:54 -0500)]
LP#1652382: Make use of the clean key just created

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Jeff Davis <jdavis@sitka.bclibraries.ca>
2 years agoLP#1652382: normalization of memcache keys in C code
Mike Rylander [Fri, 27 Jan 2017 20:25:05 +0000 (15:25 -0500)]
LP#1652382: normalization of memcache keys in C code

Memcache does not allow spaces in keys, so here we will actively strip them
from any key we get from a caller.  Some callers are not very proactive about
sending clean keys, and this patch prevents issues that can poison C-based
OpenSRF service backends.

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Signed-off-by: Galen Charlton <gmc@equinoxinitiative.org>
Signed-off-by: Jeff Davis <jdavis@sitka.bclibraries.ca>
2 years agoLP#1655449: Bundling/chunking limits for SubRquests
Mike Rylander [Tue, 10 Jan 2017 20:30:18 +0000 (15:30 -0500)]
LP#1655449: Bundling/chunking limits for SubRquests

We need to teach subrequests to inherit parent request bundling/chunking
parameters so they don't cause overruns in dispatch mode.

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Jason Stephenson <jason@sigio.com>
2 years agoLP#1652122: fix infinite recursion in opensrf.system.method.all
Galen Charlton [Thu, 22 Dec 2016 10:53:12 +0000 (15:53 +0500)]
LP#1652122: fix infinite recursion in opensrf.system.method.all

Under certain circumstances, calling opensrf.system.method.all on a
Perl service can result in an infinite recursion when attempting to
serialize an OpenSRF::Application object to JSON.  In particular,
this was observed to happen when doing an introspection of
the opensrf.settings service.

This patch avoids the infinite recursion (and consequent memory
leak) by ensuring that the 'session' key is slated for exclusion
from serialization from OpenSRF::Application objects during
bootstrapping.

Note that the problem does not affect all Perl services; if a
Perl service declares at least one streaming method, the auto-registration
of the .atomic method will result in 'session'-stripping.

This patch fixes a regression introduced in bug 1350457.

To test
-------
[1] Run 'introspect opensrf.settings' via srfsh; observe
    that it never returns and that the opensrf.settings drone
    will grow in memory sized until killed.
[2] Apply the patch, then repeat step 1. This time, the
    request should succeed.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
2 years agoLP#1648188: example HAProxy configuration
Galen Charlton [Wed, 7 Dec 2016 18:34:15 +0000 (13:34 -0500)]
LP#1648188: example HAProxy configuration

Add an example configuration for using HAProxy.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoLP#1638651: add instructions for using NGINX
Galen Charlton [Wed, 7 Dec 2016 17:54:05 +0000 (12:54 -0500)]
LP#1638651: add instructions for using NGINX

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoLP#1638651: example Nginx websockets/http(s) proxy config
Bill Erickson [Mon, 31 Oct 2016 19:56:32 +0000 (15:56 -0400)]
LP#1638651: example Nginx websockets/http(s) proxy config

Example using Nginx to proxy all websocket, http, and https Apache
traffic.

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoLP#1382038: clarify one step of installing websockets support
Galen Charlton [Wed, 7 Dec 2016 16:58:35 +0000 (11:58 -0500)]
LP#1382038: clarify one step of installing websockets support

Specify directory to be in before copying the Apache websockets
configuration file.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoLP#1382038: adjustments to download instructions
Galen Charlton [Wed, 7 Dec 2016 16:53:49 +0000 (11:53 -0500)]
LP#1382038: adjustments to download instructions

- Use "OSRFVERSION" in the master branch rather than
  a specific version number; this is meant to be changed
  during the release process.
- Update references to the OpenSRF downloads index
  page.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoLP#1382038: Add instructions for implied download/unpack step
Remington Steed [Mon, 21 Nov 2016 15:58:33 +0000 (10:58 -0500)]
LP#1382038: Add instructions for implied download/unpack step

Occasionally, new users have complained that the installation
instructions making too many assumptions about what a user knows. This
commit explicitly instructs users in downloading the source tarball,
unpacking it, and changing the working directory so they will be ready
for the next instructions.

Signed-off-by: Remington Steed <rjs7@calvin.edu>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoLP#1612771: fix chunking for atomic C methods
Galen Charlton [Thu, 3 Nov 2016 22:08:54 +0000 (18:08 -0400)]
LP#1612771: fix chunking for atomic C methods

This patch also refactors the code that actually does the
splitting and sending of chunked responses into a new
public function, osrfSendChunkedResult().

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Ben Shum <ben@evergreener.net>
Signed-off-by: Mike Rylander <miker@esilibrary.com>
2 years agoLP#1631522: add release notes for ->dispatch
Galen Charlton [Tue, 1 Nov 2016 21:22:48 +0000 (17:22 -0400)]
LP#1631522: add release notes for ->dispatch

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoLP#1631522: dev doc now describes ->dispatch
Galen Charlton [Tue, 1 Nov 2016 21:20:49 +0000 (17:20 -0400)]
LP#1631522: dev doc now describes ->dispatch

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoLP#1631522: include example of ->dispatch in example app
Galen Charlton [Tue, 1 Nov 2016 21:03:03 +0000 (17:03 -0400)]
LP#1631522: include example of ->dispatch in example app

This patch also makes the Perl opensrf.math demo app work
correctly, as it hadn't been constructing opensrf.dbmath
method names correctly.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoLP#1631522: Dispatch mode for method_lookup subrequests
Mike Rylander [Thu, 25 Aug 2016 21:42:31 +0000 (17:42 -0400)]
LP#1631522: Dispatch mode for method_lookup subrequests

There is a pattern in the wild of using OpenSRF's method_lookup() facility
to decide between one of several local methods when delegating to pre-existing
logic.  Often times, we want to simply hand control over to another method,
but the output of a subrequest's run() is an array of results.  The caller has
to know if, and how, to restructure the result for the client.

Instead, we can now call dispatch() instead of run() and have OpenSRF session
control completely passed to the delegate code.  This way, the delegate code
need not know anything about its caller, and vice versa.

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoLP#1612771: add release notes
Galen Charlton [Tue, 1 Nov 2016 20:30:26 +0000 (16:30 -0400)]
LP#1612771: add release notes

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoLP#1612771: fix error in POD
Galen Charlton [Tue, 1 Nov 2016 20:22:10 +0000 (16:22 -0400)]
LP#1612771: fix error in POD

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoLP#1612771: update protocol documentation
Galen Charlton [Tue, 1 Nov 2016 20:03:50 +0000 (16:03 -0400)]
LP#1612771: update protocol documentation

Now that we have PARTIAL and NOCONTENT statuses, let's
mention them in the documentation for the benefit of
folks writing future clients.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoLP#1612771: don't adjust max_stanza_size during installation
Galen Charlton [Tue, 1 Nov 2016 19:46:13 +0000 (15:46 -0400)]
LP#1612771: don't adjust max_stanza_size during installation

As the typical max_stanza_size for ejabberd installations
is larger than what OpenSRF now needs, this patch adjusts
the installation instructions to remove the step to change
max_stanza_size.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoLP#1612771: Add chunking support to JS implementation
Mike Rylander [Sun, 23 Feb 2014 20:55:52 +0000 (15:55 -0500)]
LP#1612771: Add chunking support to JS implementation

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Bill Erickson <berick@esilibrary.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoLP#1612771: implement C max_chunk_size server support
Bill Erickson [Fri, 28 Feb 2014 17:44:11 +0000 (12:44 -0500)]
LP#1612771: implement C max_chunk_size server support

Signed-off-by: Bill Erickson <berick@esilibrary.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoLP#1612771: set Perl / C max_chunk_size default sizes
Bill Erickson [Thu, 27 Feb 2014 20:18:15 +0000 (15:18 -0500)]
LP#1612771: set Perl / C max_chunk_size default sizes

default max bundle size == 25K
default max chunk size  == 50K

Note with Ejabberd using 65536 as the default max stanza size, these
new OpenSRF defaults mean that all messages will fit the default
message size constraints -- i.e. no more need to raise the
max_stanza_size.

Signed-off-by: Bill Erickson <berick@esilibrary.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoLP#1612771: C support for receiving chunked responses
Mike Rylander [Sun, 23 Feb 2014 21:35:17 +0000 (16:35 -0500)]
LP#1612771: C support for receiving chunked responses

* client parsing
* consistent w/ Perl, we now have "bundling" and "chunking"

Signed-off-by: Bill Erickson <berick@esilibrary.com>
2 years agoLP#1612771: Perl max_chunk_size additions
Bill Erickson [Mon, 24 Feb 2014 20:14:19 +0000 (15:14 -0500)]
LP#1612771: Perl max_chunk_size additions

* Added missing max_chunk_size method to AppSession
* Copy API max_chunk_size value into the handler AppRequest
* Fix error where no-chunking resulted in empty responses

Signed-off-by: Bill Erickson <berick@esilibrary.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoLP#1612771: bundling and chunking
Mike Rylander [Sun, 23 Feb 2014 19:51:13 +0000 (14:51 -0500)]
LP#1612771: bundling and chunking

This patch is first in a series of patches that provides the following
features:

* OpenSRF message bundling -- Pack multiple OpenSRF messages together
in a single XMPP envelope, as long as we believe more messages will be
sent in the future and we are below some threshold of combined message size.
The default for that threshold is 25Kb.

 * OpenSRF message chunking -- Break up large OpenSRF messages across
multiple XMPP envelopes. This is implemented with a new OpenSRF message type.
C, Perl, and Javascript libraries are taught how to reconstruct chunked
messages. The default chunking threshold is 50Kb, just a bit below the default
ejabberd max stanza size of 64Kb.

This patch in particular renames "chunking" to "bundling", then
implements message splitting ("chunking") in Perl using two new
oilsResult subclasses

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoLP#1603708: Remove support for Ubuntu 12.04 Precise
Ben Shum [Tue, 5 Jul 2016 16:53:13 +0000 (12:53 -0400)]
LP#1603708: Remove support for Ubuntu 12.04 Precise

With support for Ubuntu 16.04 Xenial in place, remove the oldest LTS, which is
Ubuntu 12.04 Precise.

Signed-off-by: Ben Shum <ben@evergreener.net>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoDocs: Change 14.04 to Trusty in README
Ben Shum [Tue, 5 Jul 2016 16:50:34 +0000 (12:50 -0400)]
Docs: Change 14.04 to Trusty in README

For consistency, change references from numbered "Ubuntu 14.04" to "Ubuntu Trusty"

Signed-off-by: Ben Shum <ben@evergreener.net>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoDocs: Add Xenial references in the websocket setup instructions
Ben Shum [Tue, 5 Jul 2016 16:44:17 +0000 (12:44 -0400)]
Docs: Add Xenial references in the websocket setup instructions

We were missing Xenial references for the websocket setup instructions.

They are the same as Trusty, so group together for now.

Also, change references to "Ubuntu 16.04" to read "Ubuntu Xenial" too.

Signed-off-by: Ben Shum <ben@evergreener.net>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoLP#1473479 Syslog configuration adoption
Bill Erickson [Fri, 10 Jul 2015 14:52:20 +0000 (10:52 -0400)]
LP#1473479 Syslog configuration adoption

When an OpenSRF client is run with syslog enabled and with the
OSRF_ADOPT_SYSLOG environment variable set to a true value, no attempt
is made to modify the syslog configuration, including no calls to
openlog()/closelog() and no modification of the syslog facility when
calling syslog().

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoLP#1631520: configure install location of Perl modules
Mike Rylander [Fri, 7 Oct 2016 16:19:52 +0000 (12:19 -0400)]
LP#1631520: configure install location of Perl modules

Add --with-perlbase configure option to specify
an alternative location for installing the Perl modules. This
can be useful for setups that want to run the Perl modules
from a shared filesystem or environments that need to run
multiple versions of OpenSRF simultaneously.

Users of --with-perlbase are responsible for ensuring that
PERL5LIB is set appropriately.

To test
-------
[1] Use --with-perlbase during the configure step, e.g.,

    ./configure --perl-base /tmp/perl

[2] Run make; make check; sudo make install
[3] Verify that the Perl modules are installed under
    /tmp/perl.
[4] Make a change to a Perl source file, then
    go to src/perl, then run sudo ./Build install. Verify
    that it remembers the --with-perlbase directory
    and installs the updated module there.

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoLP#1585041: Move debian_sys_config target for Debian distributions
Ben Shum [Tue, 24 May 2016 05:24:09 +0000 (01:24 -0400)]
LP#1585041: Move debian_sys_config target for Debian distributions

Similar to how things were reordered for Ubuntu, let's move debian_sys_config
for Debian distributions.

Signed-off-by: Ben Shum <ben@evergreener.net>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoLP#1485371: Release notes for TZ handling in OpenSRF
Mike Rylander [Tue, 23 Feb 2016 16:22:34 +0000 (11:22 -0500)]
LP#1485371: Release notes for TZ handling in OpenSRF

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2 years agoLP#1485371: Use client-supplied TZ
Mike Rylander [Mon, 3 Aug 2015 17:27:56 +0000 (13:27 -0400)]
LP#1485371: Use client-supplied TZ

Currently, there is no protocol-level mechanism for passing the client's
desired timezone to the server. In much the same way we pass the locale,
we can let the server know what timezone it should use when interpreting
time stamps.

To do this we:

 * Teach perl server code to live in the client TZ, if supplied
 * Teach perl client code to send the current $ENV{TZ}
 * Teach javascript library to include client TZ in
   gateway/translator/websocket communication
 * Teach C code to pull the incoming TZ and apply it to outgoing messages
 * Teach srfsh to pull TZ from the environment and pass it with requests

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
3 years agoLP#1551090: Update README for Ubuntu 16.04 (Xenial Xerus).
Jason Stephenson [Wed, 4 May 2016 17:05:31 +0000 (13:05 -0400)]
LP#1551090: Update README for Ubuntu 16.04 (Xenial Xerus).

Add installation steps for Ubuntu 16.04.

Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Ben Shum <ben@evergreener.net>
3 years agoLP#1551090: Enable mod_perl2 on Ubuntu 16.04 (Xenial Xerus).
Jason Stephenson [Tue, 19 Apr 2016 20:56:15 +0000 (16:56 -0400)]
LP#1551090: Enable mod_perl2 on Ubuntu 16.04 (Xenial Xerus).

Ubuntu Xenial Xerus does not automatically enable mod_perl2 for
Apache2 when the package is installed, so we enable it via the
Makefile.install.

Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Ben Shum <ben@evergreener.net>
3 years agoLP#1551090: Adding apache2-dev dependency to xenial and fixing whitespace
Chris Sharp [Fri, 19 Feb 2016 01:12:40 +0000 (20:12 -0500)]
LP#1551090: Adding apache2-dev dependency to xenial and fixing whitespace

Signed-off-by: Chris Sharp <csharp@georgialibraries.org>
Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Ben Shum <ben@evergreener.net>
Conflicts:
src/extras/Makefile.install

3 years agoLP#1551090: Since we move apache to the "install_extra_debs*" targets,
Chris Sharp [Thu, 18 Feb 2016 23:10:34 +0000 (18:10 -0500)]
LP#1551090: Since we move apache to the "install_extra_debs*" targets,

we need to move debian_sys_config further down to compensate.

Signed-off-by: Chris Sharp <csharp@georgialibraries.org>
Signed-off-by: Jason Stephenson <jason@sigio.com>
Signed-off-by: Ben Shum <ben@evergreener.net>