| Text based search [message #630682] |
Fri, 02 January 2015 18:48  |
manubatham20
Messages: 566 Registered: September 2010 Location: Seattle, WA, USA
|
Senior Member |

|
|
Hi,
I have to do some sort of search engine like searching, but I don't have text based index installed in our system.
I have 2 tables -
1. NAME_DATA_TBL - contains customer names and other information
2. BAD_WORDS - just 1 column containing abusive words
I have to find bad words in the customer names, it can be anywhere - in first_name, middle_initial, or last_name, or any permutation combination of these columns.
CREATE TABLE NAME_DATA_TBL
(
NAME_ID NUMBER(9) NOT NULL,
LAST_BUSINESS_NAME VARCHAR2(60 BYTE),
FIRST_NAME VARCHAR2(32 BYTE),
MIDDLE_INITIAL VARCHAR2(32 BYTE),
NAME_TITLE VARCHAR2(5 BYTE),
NAME_SUFFIX CHAR(12 BYTE),
ADDITIONAL_TITLE VARCHAR2(60 BYTE),
NAME_FORMAT CHAR(1 BYTE),
CONV_RUN_NO NUMBER(3)
);
SQL> SELECT /*+ parallel(a,12) */ COUNT (*) FROM NAME_DATA;
COUNT(*)
----------
290371372
CREATE TABLE BAD_WORDS
(
X_VALUE VARCHAR2(255 BYTE)
);
SQL> select count(*) from BAD_WORDS;
COUNT(*)
----------
547
SELECT A.rowid row_id, A.*
FROM NAME_DATA_TBL A, BAD_WORDS B
WHERE UPPER (
TRIM (A.FIRST_NAME)
|| TRIM (A.MIDDLE_INITIAL)
|| TRIM (A.LAST_BUSINESS_NAME)) LIKE
'%' || B.X_VALUE || '%'
OR UPPER (
TRIM (A.LAST_BUSINESS_NAME)
|| TRIM (A.MIDDLE_INITIAL)
|| TRIM (A.FIRST_NAME)) LIKE
'%' || B.X_VALUE || '%'
OR UPPER (TRIM (A.LAST_BUSINESS_NAME) || TRIM (A.FIRST_NAME)) LIKE
'%' || B.X_VALUE || '%'
OR UPPER (TRIM (A.FIRST_NAME)) LIKE '%' || B.X_VALUE || '%'
OR UPPER (TRIM (A.LAST_BUSINESS_NAME)) LIKE '%' || B.X_VALUE || '%'
OR UPPER (TRIM (A.MIDDLE_INITIAL)) LIKE '%' || B.X_VALUE || '%'
OR UPPER (TRIM (A.FIRST_NAME) || TRIM (A.LAST_BUSINESS_NAME)) LIKE
'%' || B.X_VALUE || '%';
I am trying above, but it is very slow, as for each row, internally it has to go for multiple searches because of like operator.
Any suggestion to make this query faster will be appreciated.
Thanks,
Manu
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Re: Text based search [message #630704 is a reply to message #630682] |
Sat, 03 January 2015 05:11   |
John Watson
Messages: 9003 Registered: January 2010 Location: Global Village
|
Senior Member |
|
|
|
You are probably getting a nested loop join, which could be driven by either table. What is the join order? I would hope that the outer table is BAD_WORDS.
|
|
|
|
|
|
| Re: Text based search [message #630716 is a reply to message #630685] |
Sat, 03 January 2015 10:01   |
Solomon Yakobson
Messages: 3312 Registered: January 2010 Location: Connecticut, USA
|
Senior Member |
|
|
manubatham20 wrote on Fri, 02 January 2015 20:29what should I change?
Just create a trigger:
CREATE OR REPLACE
TRIGGER NO_SWEARING
BEFORE INSERT
OR UPDATE
ON NAME_DATA_TBL
FOR EACH ROW
DECLARE
v_cnt NUMBER;
BEGIN
SELECT COUNT(*)
FROM BAD_WORDS
WHERE ROWNUM = 1
AND (
INSTR(
UPPER(
RTRIM(:NEW.FIRST_NAME) || TRIM(:NEW.MIDDLE_INITIAL) || LTRIM(:NEW.LAST_BUSINESS_NAME)
),
X_VALUE
) > 0
OR
INSTR(
UPPER(
RTRIM(:NEW.LAST_BUSINESS_NAME) || TRIM(:NEW.MIDDLE_INITIAL) || LTRIM(:NEW.FIRST_NAME)
),
X_VALUE
) > 0
OR
INSTR(
UPPER(
RTRIM(:NEW.LAST_BUSINESS_NAME) || LTRIM(:NEW.FIRST_NAME)
),
X_VALUE
) > 0
OR
INSTR(
UPPER(:NEW.FIRST_NAME),
X_VALUE
) > 0
OR
INSTR(
UPPER(:NEW.LAST_BUSINESS_NAME),
X_VALUE
) > 0
OR
INSTR(
UPPER(:NEW.MIDDLE_INITIAL),
X_VALUE
) > 0
OR
INSTR(
UPPER(
RTRIM(:NEW.FIRST_NAME) || LTRIM(:NEW.LAST_BUSINESS_NAME)
),
X_VALUE
) > 0
);
RAISE_APPLICATION_ERROR(
-20500,
'Watch your language!'
);
END;
/
Now you can prevent users from entering "bad words". You still need to validate existing table. Easiest way is to create error log table and issue:
BEGIN
DBMS_ERRLOG.CREATE_ERROR_LOG(dml_table_name => 'NAME_DATA_TBL');
END;
/
UPDATE NAME_DATA_TBL
SET FIRST_NAME = FIRST_NAME
LOG ERRORS INTO ERR$_NAME_DATA_TBL('VALIDATE') REJECT LIMIT UNLIMITED
/
Now check ERR$_NAME_DATA_TBL for offending row ROWIDs.
SY.
|
|
|
|
| Re: Text based search [message #630718 is a reply to message #630716] |
Sat, 03 January 2015 10:40   |
manubatham20
Messages: 566 Registered: September 2010 Location: Seattle, WA, USA
|
Senior Member |

|
|
Hi John,
You are right, it's nested loops. As I have used all the things (TRIM/UPPER/LIKE) which suppress the use of indexes and hash joins.
Attached is plan (jpg) for your reference.
Michael,
Thanks, but as I already said, we don't have text searches installed, as we don't have license. So moving topic won't help.
SY,
We decided to put this check on front-end, that will be less costly.
Thanks you guys.
Manu
|
|
|
|
| Re: Text based search [message #630720 is a reply to message #630718] |
Sat, 03 January 2015 10:48   |
John Watson
Messages: 9003 Registered: January 2010 Location: Global Village
|
Senior Member |
|
|
Quote:Thanks, but as I already said, we don't have text searches installed, as we don't have license. So moving topic won't help.
Text is free with all editions.
|
|
|
|
|
|
|
|
|
|
| Re: Text based search [message #630725 is a reply to message #630724] |
Sat, 03 January 2015 13:18  |
manubatham20
Messages: 566 Registered: September 2010 Location: Seattle, WA, USA
|
Senior Member |

|
|
I think inverted tables, are just one use case substitution of text searches, but not a replacement.
I don't think inverted table will be useful to resolve wildcards.
Manu | DM
|
|
|
|