git.evergreen-ils.org Git - working/random.git/log

]> git.evergreen-ils.org Git - working/random.git/log

projects / working / random.git / log

commit | commitdiff | tree

Bill Erickson [Wed, 14 Aug 2013 15:44:57 +0000 (11:44 -0400)]

constrictor : patron names file; qa load script

Signed-off-by: Bill Erickson <berick@esilibrary.com>

commit | commitdiff | tree

Bill Erickson [Tue, 7 May 2013 20:30:52 +0000 (16:30 -0400)]

constrictor SIP2; login repair; item info load tests

* login occurs within a thread lock to prevent multiple in-flight
logins (for a single username), which Evergreen does not handle well.

* add # iterations and task delay settings for load testing scripts

Signed-off-by: Bill Erickson <berick@esilibrary.com>

commit | commitdiff | tree

Bill Erickson [Tue, 7 May 2013 20:29:26 +0000 (16:29 -0400)]

constrictor shared thread lock

Each script has a thread lock (RLock()) object which can be shared by
its threads to manage all its thread locking needs.

Signed-off-by: Bill Erickson <berick@esilibrary.com>

commit | commitdiff | tree

erickson [Thu, 21 Apr 2011 14:40:46 +0000 (14:40 +0000)]

allow for a list of limit_path options for tighter control on which resources to crawl

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@1411 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Wed, 23 Feb 2011 16:31:06 +0000 (16:31 +0000)]

improved url deconstruction; ignore JS and anchor links

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@1236 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Sat, 29 Jan 2011 16:06:52 +0000 (16:06 +0000)]

Web spider sample script

Simple web spider script that visits, reports, parses, collects links,
and continues until it has fetched a configured number of pages. Each page
load is a constrictor Task so timing data can be collected.

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@1215 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Sat, 29 Jan 2011 16:06:51 +0000 (16:06 +0000)]

build datamanager at script init instead of during each call to run() to avoid unnecessary server calls

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@1214 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Fri, 17 Sep 2010 23:23:20 +0000 (23:23 +0000)]

in bib search, create the datamanager on thread init to prevent excessive seed data collection on re-runs

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@997 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Thu, 2 Sep 2010 21:29:53 +0000 (21:29 +0000)]

typo repair patch from James Fournie

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@976 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Sat, 14 Aug 2010 14:40:54 +0000 (14:40 +0000)]

created generic sip request task class to reduce code footprint; added patron info request task w/ script; added mixed batch script

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@959 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Wed, 4 Aug 2010 14:23:31 +0000 (14:23 +0000)]

added patron status test and simple endurance script

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@949 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Wed, 4 Aug 2010 13:33:10 +0000 (13:33 +0000)]

log the thread count in the final summary data

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@948 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Tue, 3 Aug 2010 22:07:29 +0000 (22:07 +0000)]

added socket disconnect; more socket fault tolerance; login re-try's since hitting a sip server with a pile of simultaneous logins is not always successful

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@947 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Tue, 3 Aug 2010 22:07:28 +0000 (22:07 +0000)]

append the script thread to the log messages. import from within to funcs to prevent import loop probs

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@946 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Tue, 3 Aug 2010 15:50:06 +0000 (15:50 +0000)]

new SIP2 client contrib module. currently supports login and item information requests

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@945 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Tue, 3 Aug 2010 15:50:05 +0000 (15:50 +0000)]

exit early when title permit call completely fails

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@944 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Mon, 7 Jun 2010 20:15:44 +0000 (20:15 +0000)]

fixed typo

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@890 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Mon, 7 Jun 2010 19:47:01 +0000 (19:47 +0000)]

added support for optional task set names; name defaults to start time

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@889 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Mon, 7 Jun 2010 19:13:57 +0000 (19:13 +0000)]

filter out copies and org-units that are more likely to cause early exit from circ transactions. log api calls in a srfsh-compliant fashion (iow, no array brackets)

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@888 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Tue, 18 May 2010 21:30:37 +0000 (21:30 +0000)]

only load dynamic test objects when they are not already defined in the properties file; in batch api loader, only load/parse the file once at start (instead of once per thread); removed some unused code

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@887 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Thu, 29 Apr 2010 02:55:47 +0000 (02:55 +0000)]

make a certain percentage of the batch calls hold/checkout calls. repaired some typos

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@875 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Tue, 13 Apr 2010 12:39:10 +0000 (12:39 +0000)]

plugged in some initial dynamic data fetching (todo: move to eg_data or related); additional python style tweaks; fleshing out more seed data for batch call

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@865 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Mon, 12 Apr 2010 02:37:52 +0000 (02:37 +0000)]

fixed bug in property reading from eg_data module. more python style tweaks

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@863 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Sun, 11 Apr 2010 19:33:44 +0000 (19:33 +0000)]

added a batch file processor. the batch file is a set of API calls, with param macros, and a weight which defines how many times the call should be made relative to other calls

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@862 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Sun, 11 Apr 2010 19:33:43 +0000 (19:33 +0000)]

protect against 0-task runs

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@861 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Sun, 11 Apr 2010 14:51:35 +0000 (14:51 +0000)]

added option to bypass inserting timing data into the db. removed old db.py module, which was replaced by data.py

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@860 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Sun, 11 Apr 2010 14:51:35 +0000 (14:51 +0000)]

added mvr retrieval task

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@859 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Thu, 8 Apr 2010 18:04:11 +0000 (18:04 +0000)]

python style tweaks

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@858 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Thu, 8 Apr 2010 18:04:11 +0000 (18:04 +0000)]

mixed batch of circ calls

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@857 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Thu, 8 Apr 2010 18:04:10 +0000 (18:04 +0000)]

don't assume response evt is non-null

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@856 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Thu, 8 Apr 2010 18:04:09 +0000 (18:04 +0000)]

some more expressive logging during data inserts

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@855 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Thu, 25 Feb 2010 05:00:22 +0000 (05:00 +0000)]

more python style tweaks. removed some unnecessary cruft from hold placement script

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@796 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Thu, 25 Feb 2010 05:00:21 +0000 (05:00 +0000)]

log real average task time

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@795 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Wed, 24 Feb 2010 02:41:22 +0000 (02:41 +0000)]

only update end time on current task set (not all). log some basic stats after storing data

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@794 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Wed, 24 Feb 2010 02:35:50 +0000 (02:35 +0000)]

missed some pythonification updates

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@793 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Tue, 23 Feb 2010 03:43:33 +0000 (03:43 +0000)]

Make this look more like python. (thanks, Perl one-liners)
Added a clear cache option to clear the local file cache at startup

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@792 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Mon, 22 Feb 2010 03:49:57 +0000 (03:49 +0000)]

missed the new data file in previous commit. also, track task start time on the task object

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@790 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Mon, 22 Feb 2010 03:10:26 +0000 (03:10 +0000)]

Re-org the data handling.
- data is only written to the DB after all scripts are done. This speeds up the client overall inside the critial code path and removes the requirements for DB semaphores to protect against multi-threaded writes.
- Moved to sqlalchemy instead of the bare mysql layer
- TODO add UI-layer methods back in for generating task summaries, etc.

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@789 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Fri, 15 Jan 2010 20:29:26 +0000 (20:29 +0000)]

updated contrib scripts to use new-style per-thread tasks

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@760 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Fri, 15 Jan 2010 20:04:39 +0000 (20:04 +0000)]

it's now possible to run a single task more than once inside a given script. No need to instantiate each task before calling start()

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@759 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Fri, 15 Jan 2010 19:49:37 +0000 (19:49 +0000)]

updated to use simplified inline tasks

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@758 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Fri, 15 Jan 2010 19:43:17 +0000 (19:43 +0000)]

simplified task management. no longer using a task wrapper and task runner to accomodate threads. approach now is to instantiate tasks inside the script (post thread creation). updated sample script. contrib scripts to follow

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@757 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Thu, 13 Aug 2009 17:12:43 +0000 (17:12 +0000)]

used new hold id returned from place hold method to subsequently cancel placed hold. props_filename is a global var, fixed

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@615 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Fri, 17 Jul 2009 20:14:25 +0000 (20:14 +0000)]

allow overriding of property filename, part 3

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@589 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Fri, 17 Jul 2009 20:10:39 +0000 (20:10 +0000)]

allow overriding of property filename, part 2

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@588 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Fri, 17 Jul 2009 20:06:41 +0000 (20:06 +0000)]

allow overriding of property filename

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@587 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Fri, 10 Jul 2009 20:11:17 +0000 (20:11 +0000)]

script to pull object ids/barcodes from the db for generating a constrictor properties file

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@579 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Fri, 10 Jul 2009 18:00:36 +0000 (18:00 +0000)]

fixed renew log line

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@578 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Fri, 10 Jul 2009 16:07:43 +0000 (16:07 +0000)]

added configurable dictionary file option, fixed comment typo

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@577 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Wed, 1 Jul 2009 15:33:19 +0000 (15:33 +0000)]

added really dumb (for now) user search script. made generic random phrase generator in utils

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@555 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Wed, 1 Jul 2009 03:10:26 +0000 (03:10 +0000)]

added user transaction data retrieval, by various types of transaction

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@554 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Wed, 1 Jul 2009 02:46:25 +0000 (02:46 +0000)]

added basic bib search script, which pulls random words from the dictionary and builds 1-3 word search phrases

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@553 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Fri, 9 Jan 2009 21:04:58 +0000 (21:04 +0000)]

typo. thanks Jeff.

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@96 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Wed, 26 Nov 2008 22:25:46 +0000 (22:25 +0000)]

more API drift repairs. now using checkout.full / removed all traces of checkout.permit

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@57 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Wed, 26 Nov 2008 21:47:39 +0000 (21:47 +0000)]

django requires explicit import of _ now

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@56 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Wed, 26 Nov 2008 21:31:49 +0000 (21:31 +0000)]

repaired some django DPI drift

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@55 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Wed, 26 Nov 2008 21:31:12 +0000 (21:31 +0000)]

django no longer supports the --noinput option. this will probably have a negative effect on the auto windows installer.

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@54 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Wed, 26 Nov 2008 21:22:15 +0000 (21:22 +0000)]

some updates to repair API drift with opensrf/evergreen

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@53 6d9bc8c9-1ec2-4278-b937-99fde70a366f

commit | commitdiff | tree

erickson [Wed, 26 Nov 2008 03:43:40 +0000 (03:43 +0000)]

Adding constrictor (per IRC conversation) source to contrib repo. needs a lot of tuning to fit the latest evergreen/opensrf python bits.

git-svn-id: svn://svn.open-ils.org/ILS-Contrib/constrictor/trunk@51 6d9bc8c9-1ec2-4278-b937-99fde70a366f

Random Contribs

RSS Atom