erickson [Thu, 18 Dec 2008 20:17:57 +0000 (20:17 +0000)]
correctly capture the JID of the backend server process for the session cache. only create a session cache if there is a CONNECT message in the batch. be more aggressive about removing session caches
miker [Sat, 6 Dec 2008 02:28:54 +0000 (02:28 +0000)]
Patch from Scott McKellar:
These patches eliminate several deprecated identifiers in favor of their
camel case equivalents. All these identifiers have already been
eliminated from elsewhere in the source tree:
With these patches I complete my pet project of eliminating redundant and
deprecated identifiers (unless of course I discover others that I hadn't
noticed before).
miker [Sat, 6 Dec 2008 02:24:53 +0000 (02:24 +0000)]
Patch from Scott McKellar:
This patch eliminates the deprecated typedef osrf_message, replacing it
with the camel case equivalent osrfMessage. All other occurrences have
slready been eliminated from the source tree.
miker [Fri, 5 Dec 2008 20:02:35 +0000 (20:02 +0000)]
Patch from Scott McKellar:
1. Move the declaration of osrf_app_request_struct, and its typedef as
osrfAppRequest, out of the header and into osrf_app_session.c.
2. In the declaration of osrf_app_session_struct: remove an obsolete
and commented-out declaration of request_queue.
3. Abolished _osrf_app_session_free(), moving its contents into
osrfAppSessionFree().
4. In osrfAppSessionCleanup(): after freeing the cache, nullify the
pointer to it, in the interests of good hygiene.
5. In _osrf_app_request_free(): free the messages in the result queue.
6. In _osrf_app_request_recv(): Eliminated the useless intermediate
variables tmp_msg used for dequeuing messages.
7. In osrf_app_session_set_locale: If the existing session_locale is
big enough to hold the new locale, use strcpy() instead of free() and
strdup().
8. In osrf_app_session_set_remote: If the existing remote_id is big
enough to hold the new remote_id, use strcpy() instead of free() and
strdup().
9. To eliminate some duplication of code, call
osrf_app_session_set_locale() amd osrf_app_session_set_remote() in
_osrf_app_request_recv() and osrf_app_session_reset_remote(),
respectively.
10. Performance tweak: in osrfAppRequestRespondComplete: don't create
the payload message unless we're actually going to use it.
11. Make osrfAppSessionCache static, since no other source file
references it.
dbs [Sun, 30 Nov 2008 05:30:44 +0000 (05:30 +0000)]
Merge patch from Scott McKellar for better Unicode handling
The attached files contain a drop-in replacement for the
buffer_append_uescape function that I submitted a few days ago. I regard
this new one as experimental, at least for now.
They also offer some byte-testing functions, and the equivalent macros,
that may be useful in other code that deals with UTF-8 strings.
The new function buffer_append_utf8() differs from buffer_append_uescape()
in the following ways:
1. It treats 0xF7 as a control character, which it is.
2. It is more finicky about recognizing the header byte of multibyte
characters. For example 0xF6 is not a valid UTF-8 header byte.
3. When it sees a nul byte in the middle of a multibyte character, it
stops. In the same situation, the older buffer_append_uescape() and
uescape() functions accumulate the nul byte into the hex codes they
build and then keep going, risking not only misbehavior but undefined
behavior.
4. When it finds invalid UTF-8 characters in the input string, it skips
over the invalid UTF-8 until it finds a valid character, and then
continues to translate the rest. In other words it excises the garbage
and translates the rest intact.
---------
The file osrf_utf8.c includes an array of bitmasks that it uses to look
up the characteristics of each byte. Not trusting myself to do that
much tedious typing by hand, I wrote a program to write the list of
bitmasks. The macros are broadly similar to the standard C functions
isprint(), isalpha(), and so forth.
There is also a collection of functions, equivalent to the macros, with
the same names except using double underscores. These may never find a
use, but they're there in case anyone ever needs a function pointer for
some reason.
The logic uses a finite state machine (FSM) to examine and dispatch each
byte in the input stream. Because it needs to branch on the current
state as well as the type of each character, this logic is a little
slower than buffer_append_uescape(). However pretty much any
implementation of the same behavior would probably incur some such extra
overhead in some form.
-------------
(Regarding the second version of osrf_utf8.c):
When it encounters a code point too big to fit into 16 bits (after
stripping out the packaging bits), it formats it into a surrogate pair
of four hex digits each, rather than a single set of five or six hex
digits.
In addition, this new version no longer uses buffer_fadd() to format
hex values.
The code for constructing surrogate pairs is a slightly simplified version
of a code snippet found at:
http://www.unicode.org/faq/utf_bom.html
The code snippet seems to come from a pretty authoritative source. and
my modifications were minimal, consisting mostly of collecting a couple
of constant expressions into constant values.
In the case of the G clef character (U+1D11E), I verified that my code
translates it to the correct surrogate pair ("\uD834\uDD1E").
Scott McKellar
http://home.swbell.net/mck9/ct/
Developer's Certificate of Origin 1.1 By making a contribution to
this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license indicated
in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source license
and I have the right under that license to submit that work with
modifications, whether created in whole or in part by me, under the
same open source license (unless I am permitted to submit under a
different license), as indicated in the file; or
(c) The contribution was provided directly to me by some other person
who certified (a), (b) or (c) and I have not modified it; and
(d) In the case of each of (a), (b), or (c), I understand and agree
that this project and the contribution are public and that a record
of the contribution (including all personal information I submit
with it, including my sign-off) is maintained indefinitely and may
be redistributed consistent with this project or the open source
license indicated in the file.
erickson [Wed, 26 Nov 2008 17:56:42 +0000 (17:56 +0000)]
Two patches from Scott McKellar:
In some earlier patches I eliminated the old osrf_app_client_session_init
function in favor of its camel-case equivalent. Unfortunately I didn't
notice that the new function now puts the old function's name into an
error message. Here's the fix for that little oversight.
---
These patches replace a couple of deprecated identifiers with their
camel-case equivalents:
dbs [Tue, 25 Nov 2008 03:16:13 +0000 (03:16 +0000)]
Cheap hack to enable OpenSRF to build on recent glibc systems
(already in use in src/libopensrf/osrf_system.c)
Seems to stem from HOST_NAME_MAX moving from /usr/include/sys/param.h
to /usr/include/bits/local_lim.h
dbs [Thu, 20 Nov 2008 01:27:16 +0000 (01:27 +0000)]
Force Class::DBI install, as dependencies have started making test t/11 (triggers) fail;
We don't use trigger support in Class::DBI, so forcing the install should be okay.
erickson [Tue, 18 Nov 2008 22:23:29 +0000 (22:23 +0000)]
2 Patches from Scott McKellar, with slight modification:
This patch adds a new function buffer_append_uescape(), a streamlined
replacement for uescape(). The optimizations are as follows:
1. As described in an earlier post, the new function appends the uescaped
text to an existing growing_buffer, rather than into an internal one that
it must create and destroy. It thereby saves two mallocs and two frees.
In addition, the calling code doesn't have to do a buffer_add().
2. Because it doesn't create an internal growing_buffer, it doesn't need
to do a strlen() to determine how big a buffer to allocate.
3. Since the new version doesn't have a boolean parameter, it doesn't have
to test the boolean.
4. For characters < 128 (i.e. ASCII characters), I rearranged the order of
the tests so that non-control characters are recognized immediately. In
uescape() we first go through a switch/case looking for several specific
control characters like '\b' and '\n'. In practice most characters are
not control characters, so this rearrangement saves a few CPU cycles.
5. For control characters, uescape() uses buffer_fadd() to format the
character into hex. Now, buffer_fadd is slow because it uses vsnprintf()
twice, once to determine a buffer size and once to do the formatting. In
turn vsnprintf() is slow because it has to parse the format string. In
this case we don't need vsnprintf() because we already know exactly how
big the buffer needs to be, and the formatting is simple. I eliminated
buffer_fadd() and formatted the hex manually with a little bit-twiddling.
Some of these optimizations could be applied to uescape(), but I haven't
bothered, partly because I wanted a clean comparison for benchmarking
purposes and partly because I expect uescape() to disappear from use
(though I am leaving it available).
=====
This patch is a rewrite of the jsonObjectToJSON and jsonObjectToJSONRaw functions. It is dependent on my previous patch to utils.c and utils.h,
adding the new buffer_append_uescape function.
One purpose is to replace a call to the uescape function with a call to
the faster buffer_append_uescape function. The other purpose is to
introduce a faster way to translate a jsonObject into a string.
(Also in one spot I broke up a very long string literal into several
smaller pieces so that it wouldn't wrap around in the editor.)
In the existing jsonObjectToJSON function, we receive a pointer to a
jsonObject and return a string of JSON. However we don't translate the
original jsonObject directly. Instead, we create a modified clone of the
original, inserting an additional JSON_HASH node wherever we find a
classname. Then we translate the clone, and finally destroy it.
It always struck me as an egregious waste to create and destroy a whole
parallel object just so that we could turn it into a string. So I looked
for a way to eliminate the cloning.
The result is a modification of add_json_to_buffer(), a local function
that recursively traverses and translates the jsonObject. When it sees a
classname (and it has been asked to expand classnames), the new version
inserts additional gibberish into the output text and then continues the
traversal, without modifying or copying the original jsonObject at all.
In my benchmark, this new approach was faster than the original by a
factor of about 5. When I combined this change with the use of the new
buffer_append_uencode function, it was faster by a factor of about 7.2.
The benchmark used a moderately complex jsonObject about 5 or 6 levels
deep, with both hashes and arrays, with classnames at several levels.
The performance gain will no doubt depend on the contents of the
jsonObject,but I haven't tried to isolate the variables.
The new version is a bit trickier and harder to read than the old. In my
opinion the speedup is worth the obscurity, because a lot of places in
Evergreen will benefit.
erickson [Mon, 17 Nov 2008 03:17:50 +0000 (03:17 +0000)]
Patch from Scott McKellar:
This patch is mostly a couple of tweaks to the growing_buffer code, loosely
related to my previous patch to utils.h. There is also a small tweak to
uescape().
1. in buffer_add() I replaced strcat() with strcpy() for appending the new
string. Since we already know where the end of the old string is, we don't
need to ask strcat() to find it for us.
2. In buffer_reset(), the old code contains the following:
osrf_clearbuf( gb->buf, sizeof(gb->buf) );
The evident intent is to clear the buffer. However sizeof(gb->buf) is not
the size of the buffer, it's the size of the pointer to the buffer. We
were clearing only the first four bytes or so. I changed the line to:
osrf_clearbuf( gb->buf, gb->size );
3. Also in buffer_reset(), I added a line to populate the first byte of
the buffer with a nul, to ensure that the length of the (empty) string matches the n_used member.
4. In uescape(), we were examining the contents of string[] without first
verifying that string was not NULL. The result would be undefined
behavior if string were ever NULL. I added a couple of lines to treat
a NULL pointer as if it were a pointer to an empty string.
erickson [Mon, 17 Nov 2008 02:56:40 +0000 (02:56 +0000)]
Patch from Scott McKellar:
This patch fixes a bug in the OSRF_BUFFER_ADD_CHAR macro.
Like the corresponding buffer_add_char function, this macro appends a
specified character to a growing_buffer. Unlike the function, however, the
existing version of the macro does not also append a terminal nul.
This bug had gone unnoticed because, most of the time, the rest of the
buffer is already filled with nuls, left over from the initial creation of
the growing_buffer. I stumbled across the problem when, in the course of
writing a test harness for some other changes, I called buffer_reset()
in order to reuse an existing growing_buffer instead of destroying and
re-creating it.
With debugging turned on, buffer_reset() fills the buffer with exclamation
points, leaving a nul only in the very last byte. Later, if we use
buffer_add() or buffer_fadd() to extend the string stored in the
growing_buffer, it uses strcat() to append the new characters. The result
is a buffer overflow.
Actually buffer_reset() should place a nul in the first byte of the buffer.
Tomorrow I shall submit a patch to that effect.
dbs [Mon, 27 Oct 2008 05:07:06 +0000 (05:07 +0000)]
Clean up the source tree a little more:
* Delete setup.py.in (as we're not modifying it)
* Make math_client.py be modified with SYSCONFDIR location per other scripts
(although slightly longer term we'll need to stop modifying all of these
in place, because that doesn't work after the first ./configure run)
* Add a few files to automake's tracking so that make dist is a little happier
erickson [Fri, 24 Oct 2008 16:31:07 +0000 (16:31 +0000)]
the pool cleanup handler which was thought to only run on top-level child process exit is running on cloned processes cleanup. this is how mod_cgi runs scripts. disabling cleanup for now. note: this cleanup is new to 1.0
erickson [Mon, 13 Oct 2008 20:44:50 +0000 (20:44 +0000)]
io::socket::inet, somewhere between version 1.29 and 1.31, requires the peerport to be explicitly cast to an int. also updated error handling to use correct error var
added a configurable startup pause delay. after opensrf.settings has been started (which will always be the first service to start), pause the top-level process for the configured amount of time before starting any more services
config file, context, and cache server now have apache config settings. keeping a static version of the translator on hand to reduce a layer of malloc/free
projects / OpenSRF.git / log