Galen Charlton [Thu, 6 May 2021 20:00:33 +0000 (16:00 -0400)]
LP#1927542: fix incorrect ACS Status response in Multiplex mode
This patch fixes an issue where the serialization and deserialization
of the SIP server object when handling a pending ILS login connection
can cause the ACS Status message handler to not have access to the
institution configuration settings for the timeout and retry policy,
causing it to return 000 for both values. This can lead to some
SIP2 clients believing that the ACS is either offline or impossibly
fast.
To test
-------
[1] Set up SIPServer to use the Multiplex personality.
[2] With an appropriate ILS connect set up, send the following
messages
9900402.00 => response should show default timeout of 600 and
retry limit of 003
9300CN_user_|CO_password_|CP_institution_ => should get 941 response 9900402.00 => response shows a timeout of 000 and retry limit
of 000
[3] Apply the patch and repeat step 2. This time, both message
98 responses should show a timeout of 600 and retry limit of
003.
First, we improve the encoding search by looking in the institution
config proper. The search order for an encoding is now:
1. An encoding attribute specified in the account tag.
2. An encoding tag in the institution tag.
3. An encoding tag in the implementation_config tag of the institution
tag as a last resort.
4. Use ASCII if none is set. (The SIP2 specification practially
requires this.)
The last is done for backward compatibility with existing Evergreen
oils_sip.xml files. This should be deprecated in the near future and
only the account tag and institution tag proper should be supported.
Second, we improve the handling of ASCII output by stealing a code
chunk from Evergreen's OILS::SIP->clean_text routine to remove
combining characters on strings.
Third, we remove the encoding attribute from the sample SIPconfig.xml
account tag for lpl-sc, and put an encoding tag in the institution
tags for LPL and UWOLS.
Improve the previous commit by looking for the encoding in the
institution implementation config in addition to the account
configuration. The account configuration is used first, followed by
that from the institution implemenation config.
We also set a convenience field on the server object in the
_load_ils_handler method implementation. This way, we only need get
the encoding once per session. Hopefully this is correct with the
multiplex server.
This commit adds support for an encoding attribute that can be
added to accounts in the configuration file. When set, SIPServer
will encode response messages in the specified encoding. Valid
encodings are those listed in the Encode::Supported man page.
This feature is useful if you have a vendor who cannot support
the default UTF-8 output by SIPServer.
An example is provided in SIPConfig.xml. The lpl-sc account has
been set to use the ascii encoding.
This particular code was mostly taken from a patch to Koha's
SIPServer by Adrien Saurat <adrien.saurat@biblibre.com>,
Christophe Croullebois <christophe.croullebois@biblibre.com>,
and others.
All subsequent contributors have either already approved of the
license switch or have done work-for-hire coding for organizations
that have approved the change.
Consequently, this patch adjusts all references to the GPLv2
to specify that the current license is GPLv2 or later. This
patch also adds copyright and copying statements to a few
files that lacked them.
Signed-off-by: Galen Charlton <gmc@esilibrary.com> Signed-off-by: Jeff Godin <jgodin@tadl.org>
Galen Charlton [Fri, 12 Jun 2015 19:01:47 +0000 (19:01 +0000)]
LP#1464748: don't toss terminal account information prematurely
This patch ensures that when the PreFork Net::Server personality
is in use, terminal login information is not prematurely cleared
between *_transport() and sip_protocol_loop(). This, in conjunction
with the patch for bug 1463459, fixes a problem where a SIP
client that connects, issues a 93 request, then a 99 request, could
get an incorrect 98 response.
[1] Set up a SIP configuration with at least two institutions
with different capabilities and at least one account
for each institution. Assume that account A1 is for
institution I1 and A2 for I2.
[2] Prior to applying the patches for 1463459 and 1464748,
start the SIP server and try the following sequence:
a. 93 to log in as A2
b. 99 to get SC status
c. Note that the 98 response has values for I1, not I2
[3] Apply the patches and restart SIP, then repeat the steps
in [2]. This time, the 98 response should have the
values for I2.
Signed-off-by: Galen Charlton <gmc@esilibrary.com> Signed-off-by: Jason Stephenson <jstephenson@mvlc.org>
LP#1463459: make SC Status handler recognize logged-in SIP session
Due to MsgType not having an account field, handle_sc_status should
be checking $server, not $self, for account information. As a result
of checking the wrong field handle_sc_status always assumes the
session is not logged in and returns status based on the first account.
When this account is not flagged for some capabilities and the client
expects/needs those capabilities the client may determine it can't
function and disconnect, even though the login actually has those
capabilities. This patch fixes that; it also adjusts how
$server->{account} is checked (definedness, not existence).
Signed-off-by: Thomas Berezansky <tsbere@mvlc.org> Signed-off-by: Galen Charlton <gmc@esilibrary.com> Signed-off-by: Jason Stephenson <jstephenson@mvlc.org>
Ensure that if we are asked for summary information in Patron Information that
we have any form of definition for it *and* we have code to call. If neither
case is true pretend they didn't ask for any to begin with.
Testing notes
-------------
Here is an example of the sort of request that, prior to this patch,
could cause a SIPServer backend to crash:
Log message processing duration with finer granularity. This is
especially useful for diagnosing SIP speed issues for automated
materials handling, where small changes in speed can have a big impact.
New message logs at DEBUG level.
Signed-off-by: Bill Erickson <berickxx@gmail.com> Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Mike Rylander [Wed, 11 Sep 2013 19:52:00 +0000 (15:52 -0400)]
LP#1339190 Add support for the "Multiplex" personality
We use Net::Server::Multiplex to reduce resource consumption. In
this mode, SIPServer will maintain connection state in the main
listening process and fork workers as needed to handle individual
requests.
Initial implementation by Mike Rylander, with a conversion from
File::Queue to Memcache, and LOTS of debugging, by Bill Erickson.
Some highlights:
* A fork fence for max concurrent in-flight requests
* Allow the ILS to save state in IO::Multiplex mode
* Optimistic login using fork-and-check
When a SIP child process is spawned to handle a new connection login,
the pending login is tracked in the parent process (by PID) and the
child indicates to the parent that the login has succeeded by storing
login success/failure plus some state information in memcache. Any time
the parent wakes up to process a message, it checks for completed logins
so they can be resolved as OK in the parent and the state information
is extracted and stored for future conversation with the resolved client.
* Let workers hang around for a bit for higher-rate clients
For some clients, such as AMH (sorters), the per-message connections
cause too much latency. So, instead, we'll let their backends hang
around for a while. This is controlled by a new attribute on the
server-params element, worker-keepalive, as a peer to the personality
value. This value can also be set on the institution and login elements
to support finer-grained keepalive tuning.
This is measured in seconds and the default is 5. A value of 0 here
will disable this feature altogether.
Signed-off-by: Mike Rylander <mrylander@gmail.com> Signed-off-by: Bill Erickson <berick@esilibrary.com> Signed-off-by: Blake GH <blake@mobiusconsortium.org>
LP#1338761: add support for Relais extensions to message 24
The Relais SIP2 client expects that the Patron Status Response
(msg 24) will contain certain non-standard fields that are more
properly included in the patron information response:
- patron home address
- patron email
- patron home phone
In addition, and contrary to the specification, Relais expects that
the variable fields in the response will be given in the order that
they appear in the spec.
This patch ensures that the Patron Status repsonse emits the
institution ID, patron ID, and personal name in that order. It also
adds a new institution option, relais_extensions_to_msg24, to make
the response includes the additional fields. For example:
LP#1338731: support clients that send 99 then 93 when starting a raw connection
This patch adds the option, for raw transports, to allow clients
to send the SC Status (99) message first, then the Login (93)
message. It's arguable whether such behavior is standard or not,
but at least one SIP2 client (Relais), inflexibly requires such
behavior.
This option is enabled using the service-level attribute
"allow_sc_status_then_login".
Some client TCP stacks fail to actually close down their sockets
all the way, leading to a pile up of stale backends that can never
go away.
So, we will use Linux's TCP_KEEPALIVE tuning capabilities to probe
the connection on a regular basis. This should detect the half-
closed situation and let the backend shut itself down.
Signed-off-by: Mike Rylander <mrylander@gmail.com> Signed-off-by: Bill Erickson <berick@esilibrary.com>
Jason Stephenson [Tue, 23 Oct 2012 14:36:48 +0000 (10:36 -0400)]
LP#1070470: Limit variable fields to 255 characters.
The SIP2 protocol document specifies that variable fields are to be from
0 to 255 characters in length, not counting the field identifier or the
delimiter.
This branch adds some defensive coding to Sip::add_field to truncate any
field values that are longer than 255 characters to 255 characters before
adding them to the response.
Signed-off-by: Jason Stephenson <jstephenson@mvlc.org> Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Colin Campbell [Wed, 21 Oct 2009 08:54:18 +0000 (09:54 +0100)]
LP#1180479: Koha Bug 3723: Correct return of Institution in Patron Info Resp
While the comment correctly notes that the order of variable length
fields is not fixed some units expect mandatory fields to
follow the sequence in the protocol definition and fail
parsing otherwise. Moved institution id to its expected
place in the patron information response.
Jason Stephenson [Tue, 23 Oct 2012 17:23:14 +0000 (13:23 -0400)]
LP#1070466: Pack, then unpack total in debug_split_print.
The checksum needs to be a 16bit value, so we pack, then unpack the
checksum total in debug_split_print to guarantee that. Some longer
messages have generated 32-bit checksums.
Signed-off-by: Jason Stephenson <jstephenson@mvlc.org> Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Jason Stephenson [Tue, 23 Oct 2012 14:52:21 +0000 (10:52 -0400)]
LP#1070466: Print to STDERR in Sip/Checksum::debug_split_print.
debug_split_print was printing to STDOUT, which goes to the client. This
causes the client to hang up because it is not receiving the expected
response from the server. Explicitly printing to STDERR in this
function resolves this and you can continue your session while also
debugging checksum calculations.
Signed-off-by: Jason Stephenson <jstephenson@mvlc.org> Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Bill Erickson [Wed, 26 Dec 2012 18:53:39 +0000 (13:53 -0500)]
Set langauge value for patron info response
Ensure the 3-char language field has a value (defaulting to '000') in
the patron info response. Without the value, there is a 3-char offset
in the response message fixed fields.
Signed-off-by: Bill Erickson <berick@esilibrary.com> Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Dan Scott [Tue, 21 Jun 2011 02:11:38 +0000 (22:11 -0400)]
Convert checksum to hex value inside checksum() function
Rather than returning the integer value of the checksum and converting
that to a hex value only when it is appended to the returned message,
return the hex value from checksum() itself. This enables the unit tests
for Sip::Checksum to be run meaningfully.
Dan Scott [Wed, 15 Jun 2011 19:22:09 +0000 (15:22 -0400)]
Restore Unicode checksum algorithm
This checksum algorithm was verified to work with
3M V-series self-check machines running with encoding
Unicode, which is important to sites with large
collections of non-English materials.
Dan Scott [Fri, 17 Jun 2011 17:50:26 +0000 (13:50 -0400)]
encode to UTF8 consistently
In read_SIP_packet(), we decode the incoming packet as UTF8 and then
encode it again before writing to the log, but not before returning the
message. We were instead encoding the incoming message in a subsequent
call, which is not the recommended Perl practice of decoding input and
encoding output.
Dan Scott [Wed, 15 Jun 2011 19:17:31 +0000 (15:17 -0400)]
Prevent corruption of SIP messages in logs
Non-ASCII characters were not being decoded before being
written to the logs, thus resulting in heinous corruption
in the logs. decode_utf8() is our friend.
Signed-off-by: Dan Scott <dscott@laurentian.ca> Signed-off-by: Thomas Berezansky <tsbere@mvlc.org>
Joe Atzberger [Thu, 7 Oct 2010 19:48:56 +0000 (19:48 +0000)]
Allow whitespace "lines" to be skipped during login
This keeps clients who might have bad line-endings in the game.
In particular, this would help systems where an expect-driven login
process might be different than the rest of the client interaction.
Joe Atzberger [Thu, 7 Oct 2010 19:40:38 +0000 (19:40 +0000)]
Die if no listeners in config, give better log messages
Also tighten debug messages: one warning at the end listing all failed is good enough.
Certainly better than just throwaway debug messages listing what we are going to try.
Otherwise if we succeeded, we don't need to know about the ones that failed, just
report the successful one.
Joe Atzberger [Mon, 24 May 2010 22:52:21 +0000 (22:52 +0000)]
Checksum computation overhaul
This adds advanced debugging of checksum computation.
Note: good sample data is lacking. I think most of the actual
checksum lines included in the test are questionable. The only
one that computes accurately seems to be the example from the
3M SIP Developers Guide that specifically addresses checksum
computation. The others from the same guide seem invalid!
I think the debugging output suggests the implementation is
consistent with the specified instructions, despite unavailability
of sample data to confirm.
Joe Atzberger [Tue, 11 May 2010 01:56:37 +0000 (01:56 +0000)]
Get rid of weird evals, useless use's
There are no scripts committed that use DATA handles, so those
eval blocks are not part of a known application. And that
behavior is super sketchy, especially with multiple scripts, likely
in combination!
djfiander [Wed, 13 Jan 2010 01:39:30 +0000 (01:39 +0000)]
Dan Scott's patch:
Item status response 18 was returning CJ0|CM0| which are invalid
according to the SIP2 specification; they should either be
18-char fixed fields or not there at all (as they're optional).
It looks like it was just a reversed test. In looking through
Sip/MsgType.pm in that area, I put together a patch which
prevents those fields from being generated if the incoming value
is 0 - and separates out the value assignment & test into
separate statements, which should make the code a bit easier to
read.
As an aside, I believe the same problem was preventing the due
date (AH) from being returned in the item status response.
projects / working / SIPServer.git / log