erickson [Fri, 16 May 2008 12:45:11 +0000 (12:45 +0000)]
Patch from Scott McKellar:
These patches provide a new and more efficient implementation of
osrfHash, using a hash table for random lookups and a doubly linked
list for iterations.
It is more efficient for two main reasons. First, for iterations
it doesn't maintain a separate copy of the keys, that then have to go
through the hashing algorithm. Second, when updating an existing
item, it updates it in place rather than deleting it and creating
a new one.
erickson [Thu, 15 May 2008 20:55:02 +0000 (20:55 +0000)]
Patch from Scott McKellar:
This patch tweaks a few things.
1. In json_parse_json_string() we declare a character array buff[],
fill it with nuls, and never refer to it again. I eliminated it.
2. A few lines below that, we use memset() to clear a three-character
buffer. I replaced the memset() with an initializer clause.
3. in json_handle_error() we were relying on the osrf_clearbuf macro
to add a terminal nul. However the plan is for this macro to become
a no-op someday. I added the nul manually.
erickson [Thu, 15 May 2008 20:46:15 +0000 (20:46 +0000)]
Patch from Scott McKellar:
This patch tweaks the jid_get_* functions, which copy various
fragments of an jabber id into a buffer supplied by the caller.
If these functions don't find the fragments they're looking for, they
now return an empty string in the buffer. Earlier they would leave
the buffer unchanged, leaving it to the calling code to initialize
the buffer.
erickson [Thu, 15 May 2008 13:36:32 +0000 (13:36 +0000)]
Patch from Scott McKellar:
This patch is mostly a performance tweak, but also tidies up a few
things.
In apacheParseParms() we load POST data and GET data into the same
buffer, with the GET data coming first. However the old code loads
the POST data first. If there is also some GET data, we juggle some
buffers in order to get the GET and POST data into the right order.
The new code loads the GET data first, and then appends the POST
data onto it. Besides being simpler, the new code avoids a layer
of copying, as well as and two round trips through malloc() and
free().
Other details:
1. I rearranged the declarations of the variables sarray, buffer,
key, and val, so as to narrow their scope. In the case of sarray
this rearrangement avoids a potential memory leak in the case of an
error exit (where the POST data is excessive).
2. I append a terminal null to the input buffer instead of using
memset() to fill the entire buffer.
3. Since the bread variable is a long rather than an int, I corrected
the format specifier accordingly in one of the debug messages.
4. I eliminated some redundant casts in a couple of calls to
ap_unescape(), since the variables so cast are already the right
types.
5. The final debug message was being issued only when sarray was
not NULL. However at this spot sarray cannot be NULL anyway, so I
eliminated the test of sarray.
At long last, removing old objson sources. This will not affect the legacy
libobjson.so support, since that's all handled by libopensrf/osrf_legacy_json.c
These patches add two new functions to the osrfHash routines, and
apply them in several modules. THe purpose is to eliminate the
access of an osrfHash's internals by the client code, so that the
internals can be replaced more readily.
Those internals haven't changed yet, but they will.
This patch adds a new function so that code outside of osrf_hash.c
doesn't have to directly access the "current" member of an
osrfHashIterator. This bit of encapsulation will make it easier
to refactor the osrfHash code.
This patch boosts the performance of an osrfHashIterator a bit, by
reusing the "current" buffer whenever possible instead of freeing and
reallocating it on every iteration.
I wrote a benchmark that repeatedly traverses a 27-item hash table.
The new version is about 10% faster. This improvement is not as
dramatic as I had hoped for, but it's easy to get, and it won't hurt.
These three patches are independent of each other, but they all do the
same thing.
In each case, we had been getting the local host name by reading the
environmental variable $HOSTNAME. This approach normally works, but
it is vulnerable to abuse or error by a user who modifies the value
of that variable, or even unsets it altogether.
With these patches we will instead call gethostname(), which is not
affected by changes in the environment.
Patch from Scott McKellar (with commentary on future plans):
This patch applies the const qualifier to several variables, each of
them a copy of the "current" member of an osrfHashIterator.
**********************
While this patch is pretty inconsequential on its own, it is part of a
larger plan to streamline the use of osrfHashIterators.
The "current" member points to a dynamically allocated string. Every
time we advance the iterator, we free the string and allocate another
one to replace it.
My plan is to reuse the buffer whenever possible so as to reduce the
churning of memory through malloc() and free(). This approach will
require an additional member to keep track of the current capacity of
the buffer, rather like the "size" member of a growing_buffer.
It will also require that all code using osrfHashIterators treat the
"current" member as read-only. If somebody frees and replaces the
buffer outside of the proper interface, then buffer management will
get very confused.
I doubt that any code does anything so perfidious, but I'm going
through the files to make sure. Adding the const qualifier is an
easy way not only to verify that nothing bad is happening, but also
to make it less likely that something bad will happen in the future.
This patch elinminates the use of the osrf_clearbuf macro in several
spots. In each case, the buffer needs to be initialized to an empty
string, because the call to jid_get_* doesn't necessarily populate it
with anything else. In debugging mode, osrf_clearbuf() will fill the
field with exclamation points, which is not helpful in this context.
erickson [Mon, 31 Mar 2008 20:47:41 +0000 (20:47 +0000)]
This patch replaces the regex-based XML stream parsing mechanism with an XML::Parser (expat) based parser.
The API and parsing behavior should behave identically
This requires a new OpenSRF Perl dependency -> FreezeThaw
erickson [Tue, 11 Mar 2008 14:40:55 +0000 (14:40 +0000)]
rolling back change in changeset 1277 for default osrf_clearbuf handling. not sure of the exact problem yet, suffice to say the services wont start with the "!!!..." version of clearbuf enabled
miker [Mon, 10 Mar 2008 13:32:28 +0000 (13:32 +0000)]
Patch from Scott McKellar:
1. Pedantic: I changed child_dead from an int to a sig_atomic_t, since
we set it from a signal handler.
2. In check_children() and prefork_child_wait(), we prepare an input
buffer for a read() by calling the osrf_clearbuf macro. Depending on
the existence of NDEBUG, this macro may or may not do the right thing.
If it does the wrong thing, it will pad the input data with
exclamation points, up to the maximimum length.
I added code to explicitly add a terminal nul, thus rendering
the osrf_clearbuf macro both harmless and superfluous.
This issue is the same one that I mentioned yesterday in connection
with socket_bundle.c, so I shall not belabor it here.
3. In addition: in check_children() I arranged to issue the debug
message only if the read is successful.
miker [Mon, 10 Mar 2008 13:10:59 +0000 (13:10 +0000)]
Parially, a patch from Scott McKellar:
1. In socket_open_unix_server() and socket_open_unix_client(), I added
checks to make sure that the path parameter doesn't point to aomething
too big to fit into the receiving buffer of the struct sockaddr_un.
2. In _socket_handle_client_data() I add a terminal nul to the data
received by recv().
Also, reversing the semantics and default of NDEBUG per. To turn on debugging
code, set DEBUG to 1 durring the build:
miker [Mon, 10 Mar 2008 12:04:57 +0000 (12:04 +0000)]
Patch from Scott McKellar:
This patch replaces several calls to fprintf() or printf() with calls
to fputs(), where we don't use conversion specifications.
Since fputs() doesn't have to parse the output text for conversions,
it should be marginally more efficient than fprintf() or printf().
More importantly: in one case the output text comes in part from an
input message, and may conceivably contain conversion specifications,
whether inadvertently or maliciously. In that case, fprintf() would
look for non-existent parameters to format into the output, resulting
in undefined behavior.
miker [Mon, 10 Mar 2008 03:02:07 +0000 (03:02 +0000)]
Patch from Scott McKellar:
This patch plugs a memory leak that I somehow missed earlier, and
adds a few performance tweaks.
1. In osrfRouterFree() I added a line to free the osrfHash to which
the classes member points.
2. I replaced an occurrence of "jsonParseString("[]")" with
"jsonNewObjectType(JSON_ARRAY)", a call that produces the same
result with a lot less overhead.
3. For similar reasons I replaced several occurrences of
"jsonParseString("{}")" with "jsonNewObjectType(JSON_HASH)".
miker [Mon, 10 Mar 2008 02:54:12 +0000 (02:54 +0000)]
Patch from Scott McKellar:
These patches mostly concern the jsonObjectFindPath function in
osrf_json_tools.c, along with a couple of related functions. They
plug some memory leaks and boost performance.
1. In osrf_json.h: I moved the nested #includes inside the compilation
guard.
2. In osrf_json_utils.h: I added a compilation guard.
3. I moved the declarations of _jsonObjectF_jsonObjectFindPathRecurse
and __jsonObjectF_jsonObjectFindPathRecurse out of the header and into
the implementation file, and made them both static.
4. I also renamed those functions to findMultiPath and
findMultiPathRecurse, respectively, so that their names wouldn't be
confusingly similar.
5. In both functions, and in the parent function jsonObjectFindPath(),
I added the const qualifier to the character pointer parameters.
6. In jsonObjectFindPath(): we were leaking pathcopy in the case of
an early return. Besides plugging that leak, I rearranged the logic
so that we don't allocate pathcopy unless we actually have a use for
it.
7. Also in jsonObjectFindPath(): I eliminated a call to strlen(), and
the redundant variable t.
8. In _jsonObjectFindPathRecurse() (now findMultiPath()) we were
leaking newarr in the case of an early return. Besides plugging that
leak, I rearranged the logic so that we don't allocate newarr unless
we actually have a use for it.
9. In a couple of spots I replaced "jsonParseString("[]")" with
"jsonNewObjectType(JSON_ARRAY)", a call that produces the same result
with a lot less overhead.
miker [Mon, 10 Mar 2008 02:21:03 +0000 (02:21 +0000)]
Patch from Scott McKellar:
In _jsonInsertParserItem() I changed a switch/case to an if/else,
eliminating a supposedly unreachable default branch that, if reached,
would leak memory.
With this change, a jsonObject that is neither a JSON_HASH nor a
JSON_ARRAY will be silently converted to a JSON_ARRAY by the call to
jsonObjectPush().
erickson [Thu, 6 Mar 2008 17:22:21 +0000 (17:22 +0000)]
Updated OpenSRF Javascript client library. Currently, it only implements XMLHttpRequest as
a transport layer, and, thus far, only implements the multipart/x-mixed-replace (server push)
content type, which means only Mozilla browsers are support.
Non-multipart support will be added to support all other browsers in the future -- not much
code involved there.
Most of the existing OpenSRF JS files will be deprecated after the mozilla-based jabber
handling is imported into this new lib.
Currently, all you need is opensrf.js and opensrf_xhr.js to use the lib in xmlhttprequest mode
erickson [Wed, 5 Mar 2008 22:36:59 +0000 (22:36 +0000)]
Added support for multi-router registration
replaced unnecessary <domains><domain/></domains> with a single <domain> element in opensrf_core.xml
wrapped the top-level <router> element in a <routers> container element in opensrf_core.xml to
support additional router configuration
Updated the python, C, and Perl to match the above changes
Gave the router the ability to launch more than one router process, based on the config
Added a transport_error flag to C session to indicate a communcation error, which will prevent
client sessions from hanging when making bad requests
miker [Thu, 31 Jan 2008 19:46:21 +0000 (19:46 +0000)]
Patch from Scott McKellar (including commentary):
1. I added the const qualifier to the second parameter of xmlSaxAttr().
2. I moved the prototype of _xmlToJSON() from the header to the
implementation file, and made the function static.
At least in its present form, _xmlToJSON should not be part of the
public interface because it is confusing. Sometimes it allocates a
new jsonObject, which needs to be freed, and sometimes it doesn't.
A better design would be for it to expect to receive a non-NULL pointer
to an existing jsonObject. Since it is called in only one place
(other than a couple of recursive calls), this would be an easy
change to make. However I left it alone -- as long as the function
is visible only from within its own source file, the potential for
confusion is limited.
miker [Thu, 31 Jan 2008 19:39:02 +0000 (19:39 +0000)]
Patch from Scott McKellar:
1. I replaced the deprecated identifier osrf_message with osrfMessage,
except for the typedef defining the former.
2. In the header I commented out the prototypes for
osrf_message_set_request_info() and osrf_message_to_xml(), since these
functions are nowhere defined.
3. I made sure to initialize all members of a newly allocated
osrfMessage.
4. In osrf_message_deserialize(): I changed a series of ifs into a
series of else ifs, in order to avoid needless comparisons after one
comparison finds a match.
5. Also in osrf_message_deserialize(): I introduced a temporary
variable to cache the result of some calls to jsonObjectGetString(),
in order to avoid repeating the identical calls.
miker [Thu, 31 Jan 2008 19:23:27 +0000 (19:23 +0000)]
Patch from Scott McKellar:
1. In osrfSystemBootstrap(): we build an osrfStringArray of children
to spawn, and then spawn them. However we didn't free the
osrfStringArray afterwards. Now we do.
2. In osrfSystemBootstrapClientResc(): we were potentially leaking
the osrfStringArray arr in an early return. I moved the early return
to a point before the allocation of the osrfStringArray and a number
of other things. Not only does the early return not have to free
the osrfStringArray, it also doesn't have to free the other things
any more.
miker [Thu, 31 Jan 2008 19:12:40 +0000 (19:12 +0000)]
Patch from Scott McKellar:
1. In send_request() we allocate a jsonObject o and then immediately
assign the resulting pointer to params. I eliminated the rather
pointless o and allocated params directly.
2. We didn't ever free params. Now we do.
3. I replaced two deprecated identifiers with their camel-case
equivalents:
2. In osrf_app_server_session_init() we were leaking session in the
case of an early return. I stuck in a free().
3. Likewise in osrfAppSessionMakeLocaleRequest() we were leaking req
in the case of an early return. I plugged that one too.
4. In osrfAppRequestRespondComplete() we were leaking payload in one
branch of an if/else. I moved the osrfMessageFree() beyond both
branches so that it would be unconditional.
miker [Thu, 31 Jan 2008 18:23:52 +0000 (18:23 +0000)]
Patch from Scott McKellar:
1. I renamed __osrfRouter simply to router, and made it static. We
had one global variable and one auto variable pointing to the same
object, causing some needless juggling. Now we have just one
pointer.
2. I removed the leading underscores from __setupRouter().
3. I renamed the parameter to routerSignalHandler() from "signal"
to "signo", since "signal" is a reserved identifier. I also added
some code to re-raise the signal caught.
[from a followup email]
> I chose instead to terminate the program by re-raising the signal.
> That way the parent process has a chance to detect that the program
> was terminated by a signal rather than by a normal return.
After posting this I realized that the router program runs as a
daemon, and its adopted parent process doesn't care whether it
terminates by a signal or by a normal return. So there's not
much point in re-raising the signal.
It remains true that the signal handler should contrive to
terminate the program, either by exiting or by setting a flag that
the rest of the program tests.
[ The original patch, re-raising the signal, is applied. ]
4. In main() I moved two calls to free() so that they are reachable.
5. In main() I return either EXIT_SUCCESS or EXIT_FAILURE, which are
portable. Otherwise we could find ourselves returning -1, which is
not portable.
6. In setupRouter() I arranged to free resource, and to free
tservers and tclients in the case of an early return. I also free
router in the unlikely event that osrfRouterRun returns.
7. I reworded an error message to something that I think is more
clear.
miker [Thu, 31 Jan 2008 18:18:29 +0000 (18:18 +0000)]
Patch from Scott McKellar:
1. In the header: I added compilation guards.
2. In the header: I changed leading double underscores to leading
single underscores. These identifiers don't appear elsewhere in the
code base, with either double or single underscores.
3. I moved most of the header into the implementation file. The
headers whose prototypes I moved are now static. Where there were
comments associated with the prototypes, I moved them to the function
definitions.
4. I removed an extra leading underscore from __osrfRouterFillFDSet().
5. I replaced some deprecated identifiers with their camel-case
equivalents:
projects / OpenSRF.git / log