erickson [Mon, 29 Oct 2007 12:23:31 +0000 (12:23 +0000)]
fixed infinite loop bug by hopping out of the loop if timeout gets down to 0, but the code still allows the the loop to occur once if timeout is initially 0
miker [Thu, 25 Oct 2007 12:20:16 +0000 (12:20 +0000)]
Patch from Scott McKellar to increase const correctness in the JSON parser.
Also made the top level legacy_jsonParseString[Fmt] API const-correct, though this is removed within those wrapper calls to avoid potentially breaking the legacy code.
miker [Fri, 19 Oct 2007 23:43:59 +0000 (23:43 +0000)]
Patches from Scott McKellar covering:
1. Eliminating a const-removing cast from jsonObjectToJSON(). This
cast is no longer necessary now that a recent patch has changed the
signature of jsonObjectEncodeClass().
2. Moving the JSON_INIT_CLEAR macro out of the header file and into
the implementation file. No other file invokes this macro -- nor
could it, since the macro refers to two static functions within
osrf_json_object.c. Strictly speaking another file could provide
other functions with the same signatures, but I doubt that any such
usage was ever intended.
3. Adds some sanity checking to prevent undefined behavior when
a floating point payload is not representable as an int.
4. Streamlines the serialization of hashes and arrays by
eliminating a layer of allocation and deallocation.
dbs [Thu, 18 Oct 2007 01:11:10 +0000 (01:11 +0000)]
Install the JavaScript libraries into an OpenSRF library,
in prepration for OpenSRF applications being able to use
one known good JavaScript library for OpenSRF communication.
(I'm looking at you, EG, and your two copies of JSON.js)
miker [Mon, 1 Oct 2007 02:47:44 +0000 (02:47 +0000)]
Patch that:
1) Creates safe_calloc, suggested by Scott McKellar which includes
memset, as calloc does. safe_malloc will eventually lose its call
to memset.
2) Creates a macro in utils.h called osrf_clearbuf which
* under CLFAGS=-DNDEBUG fills the buffer with !s and a trailing nul
* otherwise (currently) uses memset to fill with nuls
The secondary behavior should be changed to a no-op after no more
problems arise under NDEBUG mode. I reversed the suggested semantics
because I'm not ready to completely break trunk. To break everything
(AKA find where we should be providing a terminal nul) just compile
like this:
$ CLFAGS=-DNDEBUG make clean all
3) replaces all memsets (excepting the ones in safe_?alloc) that act
on char bufs with said macro, so they can be spotted and improved to
deal with nul terminators where needed. I didn't touch any memsets on
struct pointers.
4) made jid_get_*() from src/libopensrf/transport_message.c safe for
use with NDEBUG mode. They were depending on the target buffer that
the caller passes in to be nul-filled (they use strncpy/memcpy which
don't guarantee terminal nul) -- now they provide their own terminal
nul.
Broad patch from Dan Scott to move towards better memory management:
* bzero->memset (with sizeof) - except when followed immediately by
snprintf(), in which case the call was deleted completely
* sprintf->snprintf (with sizeof) - for the C99-guaranteed
null-terminated string and avoidance of overwrites
* fgets (with sizeof) - because in at least one case "len - 1" was
being used for the length, even though fgets is defined to retrieve 1
byte less than the requested length - so the code was effectively
retrieving 2 bytes less than the allocated buffer
* in 2 places in srfsh.c, increased buffer size by 1 to allow for
null terminator
* various typo fixes
Continued const-correctness improvement from Scott McKellar:
1. I changed the signature of jsonObjectDecodeClass so that it accepts
a non-const pointer parameter. Since it doesn't change the contents
of the jsonObject, there's no need to confine it to non-const
jsonObjects.
2. I replaced jsonObjectGetKey() with jsonObjectGetKeyConst().
3. In one spot I captured the return value of jsonObjectGetIndex()
with a const pointer instead of a non-const pointer.
Patch from Scott McKellar to improve const-correctness in srfsh:
This patch sprinkles a few const qualifiers over srfsh.c, and also
fixes a couple of things I stumbled across along the way.
The main purpose is to treat jsonObjects in a more const-correct
manner. I introduced a new jsonObjectGetKeyConst function, and I
prepared for the day when jsonObjectGetString() returns a const
pointer instead of a non-const pointer.
This patch relies on a previous patch that defines the new
jsonObjectGetKeyConst function.
1. In handle_login() I changed hash to a const pointer, so that we
can assign the return value of jsonObjectGetString() to it.
2. Later in the same function, I rearranged the code a bit to ensure
that we always free a prior value of login_session before giving it
a new value. The original code potentially leaks memory.
3. In the same area I changed authtoken to a const pointer.
4. In send_request() I eliminated an unnecessary test for the
non-nullness of omsg->_result_content, because we had just tested
it a few lines ago.
5. Also in send_request(): I introduced a layer of strdup() when
we get content from jsonObjectGetString(). This change concerns
more than just const-correctness.
The problem with the original code is that we get content sometimes
from jsonFormatString() and sometimes from jsonObjectGetString().
The former returns a malloc'ed buffer, which we need to free. The
latter returns a pointer to a buffer owned by a jsonObject, and we
should not free it. As it happens, we do free it. If we got the
string from jsonObjectGetString(), then we leave the jsonObject with
an invalid pointer inside it. If we free the jsonObject later, we'll
try to free that same buffer a second time. Oops.
Another issue is that jsonObjectGetString() can return NULL, which
we should handle more carefully.
These problems occur in two different places in the same function.
I fixed them both the same way, with some differences in the details.
Patch from Scott McKellar which introduces a const-accepting and -returning
version of jsonObjectGetKey. This is the first step in a plan to push
const-correctness on the OpenSRF stack when dealing with complex objects. The
hope is that this will increase the utility of compile-time checks in new
client code.
erickson [Sun, 19 Aug 2007 01:16:02 +0000 (01:16 +0000)]
added support for multi-threaded client interactions. much like the java lib, each thread is allowed 1 jabber connection, as opposed to 1 process-wide jabber connection. also did some re-tabbing to force 4-space tabs (not raw tabs) - more of those to come
erickson [Fri, 17 Aug 2007 21:57:16 +0000 (21:57 +0000)]
added multi-threaded client support to the opensrf network/xmpp layer
this is all managed below the covers so that clients can continue to safely
use bootstrapClient and will all "just work"
we now allow 1 xmpp connection per thread, as opposed to 1 per process.
making sure error logs are logged to stderr even if syslogging is not an option -- eventually, it should fall back to stderr logging like the C code does
python only evaluates default function param values once, so mutable types will always be pointers to the original default val. change the default osrfNetworkObject init function to None and setting default manually
1. In handle_print(), I added the ability to display the value of
raw_print. If we can display pretty_print, we should be able to
display raw_print.
2. In a couple of places we display the value of login_session.
However if we aren't logged in, we end up passing NULL to printf(),
thereby invoking undefined behavior. Apparently glibc responds by
printing "(null)". I contrived explicitly to print "(none)" rather
than rely on glibc.
3. Since login_session is assigned from strdup() if not NULL, I added
a free() for it in a couple of places: one at the end of main(), and
one just before the assignment from strdup().
Patch from Scott McKellar to fill buffer overflow holes:
The first overflow can happen with an excessively long username.
The second overflow is more doubtful, because the inputs come from
two other functions. It's not obvious whether an overflow is possible
or not. It may be that those functions will never return strings long
enough to overflow. However it is easier to assume that they might,
and avoid the overflow for sure, than to determine whether an overflow
is possible in the first place.
In each case I declared a variable-length character array with a
calculated length.
Attached is a minor patch to OpenSRF/src/java/Makefile for creating a
opensrf.jar. I was thinking an opensrf.jar would make it easy to
deploy OpenSRF client in a Woodchip/OpenTaps instance. It's not yet
clear to me what the build environment is (or will be) for
Woodchip--but .jar files are used to across the board to manage
external dependencies in Javaland.
- - -
I've also given the depencies their own vars for easier handling and
made the dependency fetching smarter to prevent re-fetching existing deps
This moves OpenSRF to the new JSON wire protocol,
including:
a new C parser
a new osrfList based string_array module
a compatibility layer for mimicking libobjson and legacy JSON I/O in the HTTP gateway
a small JSON test program for profiling, etc.
Patch from Scott McKellar to repair srfsh buffer overflow:
The potential overflow occurs in handle_introspection(), which in the
existing code uses sprintf() to insert some user-supplied input into a
fixed length buffer, without checking the length of the user-supplied
input.
There's always more than one way to fix this sort of thing. For
example I could have formatted into a growing_buffer. Instead, I
chose to calculate the necessary buffer size and then declare a
variable-length buffer. That way I avoid two mallocs and two frees.
I tested this change by inserting some temporary printfs to capture
the commands being sent to parse_request(). The outputs before and
after the change were identical.
There's at least one more such potential overflow that I haven't
addressed yet. It's near the top of handle_login(), where we
use sprintf() to insert a username into a fixed length buffer.
There may be others that I haven't noticed yet.
Patch from Scott McKellar to clean up srfsh's user interface:
1. srfsh exits the main loop in two circumstances. The first is when
it sees the "quit" or "exit" command. The second is when it sees
end-of-file, either because it reads the end of a script or because
the user enters the associated keystrokes (typically Control-D).
In the case of end-of-file, srfsh exits without issuing another
newline. As a result, the next shell prompt shows up concatenated
to the srfsh prompt. This result is harmless but annoying.
I tweaked the code to detect this situation and issue an extra
newline, so that the next shell prompt show up on a line by itself.
2. I strip off leading whitespace before processing the input
command.
As a result, srfsh will now recognize "quit", "exit", or a leading
exclamation point, even when preceded by whitespace. Other commands
are not affected because strtok() already strips off leading
blanks. Tab characters don't matter because readline() removes them.
3. I also strip off trailing whitespace. This measure affects only
the "quit" and "exit" commands.
4. I ignore input lines whose first non-whitespace character is an
octothorpe ('#', also known as hash, pound, or tic-tac-toe). In
other words, scripts can now have comments.
Patch from Scott McKellar, with modifications, to remove unused code and provide some memory protection:
This patch contains one minor change and one less minor change.
--------
The less minor change is the elimination of some old code that once
opened a pipe to send commands to bash. According to Bill Erickson
in a private email, that code is a leftover remnant from an
experiment and may be removed.
Before a patch was applied several days ago, things worked like this:
IF you compiled srfsh with a certain macro #defined, AND you entered
a command that was invalid to srfsh, THEN srfsh would pipe the
command to bash for execution.
It is doubtful that anyone but Bill ever used this variation, but if
anyone wants to keep it around, then this is his or her chance to
protest. Before protesting, however, please note that the shell
escape mechanism now provides shell-like command processing. I.e.
if you start the srfsh command line with an exclamation point, srfsh
passes the rest of the command to the default shell (normally
/bin/sh) for execution. The command so passed can use environmental
variables, pipes, IO redirection, wild card expansion, and most of
the sorts of things you're used to from the shell command line.
Note that not all shells behave the same. If you're used to using
tcsh as your shell, for example, you may find that sh won't work
quite the same way. For now, at least, that's just too bad, but
that's the way shell escapes typically work.
It is probably possible to invoke the user's default shell as
identified by the environmental variable $SHELL. However that
nicety would not be trivial to code.
----------
The minor change is that, after building an array of pointers
pointing to tokens from the input command, I set the next
pointer to NULL so that it mark the end of the token list.
(miker: I changed to calloc/free instead of an array, but left the final
NULL in place.)
Patch from Scott McKellar to fix potential local buffer
overflow attack against srfsh:
This patch fixes a potential buffer overflow in the parse_error
function. The existing code concatenates the strtoked tokens into a
fixed-length buffer, with no check for overflow. It isn't hard to
build an srfsh command that overflows the buffer, with baleful
results.
While it's not likely that anyone would do so by accident from the
command line, an srfsh script might well do so, especially if the
script were generated from another program.
More important, someone sufficiently clever might be able to use
such an overflow to work mischief.
My version of parse_error() uses a growing_buffer to accumulate
the tokens.
Patch from Scott McKellar to remove srfsh.h and push all relevant
delarations into the implementation file, with some additional cleanup
to remove "declared by not used" warnings:
1. We have no further use for the header file. Accordingly I
include a patch to the Makefile, removing a dependency on the header.
2. I declared as static everything that was formerly in the header,
plus a few variables that were already defined in srfsh.c.
3. I commented out the shell_reader variable, since it is no longer
used.
4. I commented out two of the lines printed by the help command.
These lines refer to the "time" command, which srfsh no longer
supports.
5. I commented out the prototype for the handle_time function, since
the function itself is already commented out.
Patch from Scott McKellar for declaration cleanup and improved error handling:
1. I added the const qualifier to several function parameters.
2. I replaced inet_addr() with inet_aton().
According to the man page, inet_addr() is obsolete. It reports an
error by returning -1, which however is a valid IP address
(255.255.255.255). inet_aton reports errors in a more robust way.
3. I check the return value from inet_aton(). If the input address is
invalid, I log a message and return -1.
Without such a check, an invalid listener address leads to a server
that is completely unresponsive. The user has no way to know what's
wrong unless he spots the mistake himself in the relevant
configuration file.
Patch from Dan Scott to move JSON to OpenSRF::Utils::JSON:
I noticed back when I was first installing OpenSRF that it includes a
module, JSON.pm, that exists at the root level of the package
directories. This would be fine, except it conflicts with a CPAN
module that is also named JSON, which confuses the CPAN installer when
you check for upgrades and conceivably could lead to a broken system.
I suggested to Mike that it would probably make sense to move the
OpenSRF version of the module into the OpenSRF/Utils/ package
namespace, and he agreed. Of course, there are ramifications
throughout the code, so I've tried to be extra-careful in catching and
correcting all of the places where the use of this module surfaces in
both OpenSRF and Evergreen.
projects / OpenSRF.git / log