]> git.evergreen-ils.org Git - Evergreen.git/commit
LP#1931162: DYM needs some situational optimization
authorMike Rylander <mrylander@gmail.com>
Mon, 7 Jun 2021 19:57:32 +0000 (15:57 -0400)
committerJason Boyer <JBoyer@equinoxOLI.org>
Fri, 27 Aug 2021 18:49:13 +0000 (14:49 -0400)
commitf76edc5628ffbe1884e9249376f4d5905b2b1322
tree5e8150af425f40b633461fc289ce80d442f0f5dd
parent96e5f9d3c2a739e22485e07d9738ad52f24e60c9
LP#1931162: DYM needs some situational optimization

For some data sets and some queries the Did You Mean search suggestion
logic can be much too slow. This is mainly in cases where a "misspelled"
word of sufficient length greater than the symspell prefix length is
checked against many short prefixes that have many long suggestions
attached to them.

This commit optimizes for that case in particular by testing the length
of suggestions and prefix keys against the user input to avoid
unnecessary tests.  Futher, it captures the edit distance of suggestions
that pass that test in-line, avoiding expensive retesting, and caches
the short-cutoff edit distance when in low-verbosity mode to avoid
future different-but-not-too-different suggestions coming from the same
prefix key.

It additionally provides a general optimization by batching the capture
of suggest counts to avoid per-suggestion secondary lookups, and a
micro-optimization of ordering suggestions by length at distance cache
time.

Signed-off-by: Mike Rylander <mrylander@gmail.com>
Signed-off-by: Shula Link <slink@gchrl.org>
Signed-off-by: Jason Boyer <JBoyer@equinoxOLI.org>
Open-ILS/src/sql/Pg/300.schema.staged_search.sql
Open-ILS/src/sql/Pg/upgrade/XXXX.function.did_you_mean_optimization.sql [new file with mode: 0644]