Home » SQL & PL/SQL » SQL & PL/SQL » Detecting hidden characters (9.2.0.1.0)
Detecting hidden characters [message #517997] Sun, 31 July 2011 00:55 Go to next message
OraFerro
Messages: 433
Registered: July 2011
Senior Member
Hi All,

I am not sure if the problem is related to hidden characters but its my best guess so far.

I am trying to enhance a part of the ERD by creating a lookup for a column one of the table that uses text (finite set of values).

CREATE TABLE N_AGREEMENT_STATUS   
(
STATUS_ID NUMBER(2) PRIMARY KEY,
STATUS_NAME VARCHAR2(10 BYTE)
);

INSERT ALL
INTO N_AGREEMENT_STATUS VALUES (1, 'INACTIVE')
INTO N_AGREEMENT_STATUS VALUES (2, 'ACTIVE')
INTO N_AGREEMENT_STATUS VALUES (3, 'CLOSED')
INTO N_AGREEMENT_STATUS VALUES (4, 'CANCELLED')
SELECT * FROM DUAL;


when I try to update the source table no update takes place (0 records updated) if I used the following statement:
ALTER TABLE N_AGREEMENT ADD  STATUS_ID NUMBER(2);

UPDATE N_AGREEMENT SET STATUS_ID = 
(
SELECT STATUS_ID
FROM N_AGREEMENT_STATUS
WHERE  N_AGREEMENT.STATUS = STATUS_NAME );


but it works fine only if I used:
UPDATE N_AGREEMENT SET STATUS_ID = 
(
SELECT STATUS_ID
FROM N_AGREEMENT_STATUS
WHERE  N_AGREEMENT.STATUS LIKE  STATUS_NAME || '%'
);


The strange thing is that when I use:
SELECT N_AGREEMENT.STATUS, N_AGREEMENT.STATUS_ID
FROM  N_AGREEMENT
WHERE  N_AGREEMENT.STATUS  =   'ACTIVE';


it returns correct results and all status = 'ACTIVE' appear correctly!
Re: Detecting hidden characters [message #517999 is a reply to message #517997] Sun, 31 July 2011 01:20 Go to previous messageGo to next message
Michel Cadot
Messages: 68765
Registered: March 2007
Location: Saint-Maur, France, https...
Senior Member
Account Moderator
The way to find what is in a field is to use dump column:
select dump(mycol) from mytab;


A way to find any non allowed character is to use translate:
select mycol, dump(mycol) from mytab 
where translate(mycol, '0ABCDEFGHIJKLMNOPQRSTUVWXYZ', '0') is not null;


Regards
Michl
Re: Detecting hidden characters [message #518003 is a reply to message #517999] Sun, 31 July 2011 01:51 Go to previous messageGo to next message
OraFerro
Messages: 433
Registered: July 2011
Senior Member
This is really usefull Michil, thanks a lot.

when I executed the translate query i got:
select STATUS, dump(STATUS) from N_AGREEMENT
where translate(STATUS, '0ABCDEFGHIJKLMNOPQRSTUVWXYZ', '0') is not null ;

ACTIVE	Typ=96 Len=10: 65,67,84,73,86,69,32,32,32,32
CLOSED	Typ=96 Len=10: 67,76,79,83,69,68,32,32,32,32
CLOSED	Typ=96 Len=10: 67,76,79,83,69,68,32,32,32,32
CLOSED	Typ=96 Len=10: 67,76,79,83,69,68,32,32,32,32
CLOSED	Typ=96 Len=10: 67,76,79,83,69,68,32,32,32,32
CLOSED	Typ=96 Len=10: 67,76,79,83,69,68,32,32,32,32
CLOSED	Typ=96 Len=10: 67,76,79,83,69,68,32,32,32,32


just showing a sample but in fact all records are returned!

but this doest explain why my previous query:
select * from agreement where status in
 ('ACTIVE', 'INACTIVE', 'CANCELLED', 'CLOSED');


as it returns all columns!
Re: Detecting hidden characters [message #518007 is a reply to message #518003] Sun, 31 July 2011 04:08 Go to previous messageGo to next message
OraFerro
Messages: 433
Registered: July 2011
Senior Member
So is there any explanation for that?

thanks
Re: Detecting hidden characters [message #518008 is a reply to message #518007] Sun, 31 July 2011 04:48 Go to previous messageGo to next message
Michel Cadot
Messages: 68765
Registered: March 2007
Location: Saint-Maur, France, https...
Senior Member
Account Moderator
32 is space, you have spaces at the end of some data.

My experience is not the same as yours:
SQL> select status_id, '"'||status_name||'"' from N_AGREEMENT_STATUS;
 STATUS_ID '"'||STATUS_
---------- ------------
         1 "INACTIVE"
         2 "ACTIVE"
         3 "CLOSED"
         4 "CANCELLED"
        12 "ACTIVE  "
        13 "CLOSED  "

6 rows selected.

SQL> select status_id, '"'||status_name||'"' from N_AGREEMENT_STATUS 
  2  where status_name in ('ACTIVE', 'INACTIVE', 'CANCELLED', 'CLOSED');
 STATUS_ID '"'||STATUS_
---------- ------------
         1 "INACTIVE"
         2 "ACTIVE"
         3 "CLOSED"
         4 "CANCELLED"

4 rows selected.

Post the result of:
SQL> col parameter format a20
SQL> col value format a20
SQL> select * from v$nls_parameters where parameter in ('NLS_COMP','NLS_SORT');
PARAMETER            VALUE
-------------------- --------------------
NLS_SORT             BINARY
NLS_COMP             BINARY


Regards
Michel
Re: Detecting hidden characters [message #518072 is a reply to message #518008] Mon, 01 August 2011 03:06 Go to previous messageGo to next message
OraFerro
Messages: 433
Registered: July 2011
Senior Member
The same result here:
SQL> col parameter format a20
SQL> col value format a20
SQL> select * from v$nls_parameters where parameter in ('NLS_COMP','NLS_SORT');

PARAMETER            VALUE
-------------------- --------------------
NLS_SORT             BINARY
NLS_COMP             BINARY


I also tried to understand the meaning of the above statement and I found a userfull post of you but at the end you said: "NLS settings for your client is different than from your server.
Query v$nls_parameters in both cases."

I dont understand how can this happen if all application clients are using the same DB user to access the database.
Re: Detecting hidden characters [message #518073 is a reply to message #518072] Mon, 01 August 2011 03:10 Go to previous messageGo to next message
OraFerro
Messages: 433
Registered: July 2011
Senior Member
I also double checked my results again:
the query:
select * from n_agreement where status in
 ('ACTIVE', 'INACTIVE', 'CANCELLED', 'CLOSED');


returns all 1503 rows in the table

while the query:
select STATUS, dump(STATUS) from N_AGREEMENT
where translate(STATUS, '0ABCDEFGHIJKLMNOPQRSTUVWXYZ', '0') is not null ;


also returns 1503 rows!
Also when I used Toad to view the Hex value of the field I couldnt see any spaces.
Re: Detecting hidden characters [message #518074 is a reply to message #518072] Mon, 01 August 2011 03:12 Go to previous messageGo to next message
Michel Cadot
Messages: 68765
Registered: March 2007
Location: Saint-Maur, France, https...
Senior Member
Account Moderator
Quote:
I dont understand how can this happen

What "this" refers to?

Regards
Michel
Re: Detecting hidden characters [message #518075 is a reply to message #518073] Mon, 01 August 2011 03:13 Go to previous messageGo to next message
Michel Cadot
Messages: 68765
Registered: March 2007
Location: Saint-Maur, France, https...
Senior Member
Account Moderator
Quote:
while the query:
...also returns 1503 rows!

With which code?

Quote:
Also when I used Toad to view the Hex value of the field I couldnt see any spaces

I do not trust anything from this tool.

Add a space in the TRANSLATE second parameter, what does it give?

Regards
Michel

[Updated on: Mon, 01 August 2011 03:14]

Report message to a moderator

Re: Detecting hidden characters [message #518084 is a reply to message #518075] Mon, 01 August 2011 03:38 Go to previous messageGo to next message
OraFerro
Messages: 433
Registered: July 2011
Senior Member
when I run the query:
-- added space to 2nd parameter
select STATUS, dump(STATUS) from N_AGREEMENT
where 
translate(STATUS, '0ABCDEFGHIJKLMNOPQRSTUVWXYZ ', '0') 
is not null ;


no values are returned as expected which means that there are spaces at the end and right you are, Toad doesnt detect that (unless there is a configuration for this which is very bad even if true).

but still when I run the query:
SQL> select count(*) from n_agreement where status in
  2   ('ACTIVE', 'INACTIVE', 'CANCELLED', 'CLOSED');

  COUNT(*)
----------
      1503

SQL> select count(*) from n_agreement;

  COUNT(*)
----------
      1503

[Updated on: Mon, 01 August 2011 03:40]

Report message to a moderator

Re: Detecting hidden characters [message #518089 is a reply to message #518084] Mon, 01 August 2011 03:45 Go to previous messageGo to next message
Michel Cadot
Messages: 68765
Registered: March 2007
Location: Saint-Maur, France, https...
Senior Member
Account Moderator
desc n_agreement

Regards
Michel
Re: Detecting hidden characters [message #518093 is a reply to message #518089] Mon, 01 August 2011 03:56 Go to previous messageGo to next message
OraFerro
Messages: 433
Registered: July 2011
Senior Member
Sorry for the big number of columns and the bad ERD structure. In fact the whole problem started when I tried to replace the lookup fields with lookups (type, status, supplement_type...)

SQL> desc n_agreement;
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 NUMBER0                                   NOT NULL NUMBER(4)
 SUB_NUMBER                                NOT NULL CHAR(7)
 TYPE                                      NOT NULL CHAR(10)
 SUPPLEMENT_TYPE                                    CHAR(32)
 EXPIRY_MONTH_COUNT                                 NUMBER(2)
 INTEREST_ANNUAL_RA                                 NUMBER(6,4)
 MATURITY                                  NOT NULL NUMBER(3)
 GRACE_PERIOD                              NOT NULL NUMBER(2)
 ACQUISITION_DAY_CO                                 NUMBER(3)
 STATUS                                    NOT NULL CHAR(10)
 FK_PROJECTNUMBER0                                  NUMBER(4)
 FK_COUNTRYCODE                                     NUMBER(3)
 FK_AGREEMENTNUMBER                                 NUMBER(4)
 FK_AGREEMENTSUB_NU                                 CHAR(7)
 FK_AGREEMENTTYPE                                   CHAR(10)
 FK_DRAFT_AGREEMSER                                 NUMBER(5)
 FK_AGREEMENTNUMBE0                                 NUMBER(4)
 FK_AGREEMENTSUB_N0                                 CHAR(7)
 FK0AGREEMENTTYPE                                   CHAR(10)
 FK_TOPICCODE                                       NUMBER(3)
 TECH_STATUS                                        NUMBER(1)
 DHS_CATEG                                          NUMBER(3)

Re: Detecting hidden characters [message #518098 is a reply to message #518093] Mon, 01 August 2011 04:04 Go to previous messageGo to next message
Michel Cadot
Messages: 68765
Registered: March 2007
Location: Saint-Maur, France, https...
Senior Member
Account Moderator
Quote:
STATUS NOT NULL CHAR(10)

CHAR and not VARCHAR2

So, expected behaviour, CHAR is padded by white spaces as well as everything that is compared to it.

Regards
Michel

[Updated on: Mon, 01 August 2011 04:05]

Report message to a moderator

Re: Detecting hidden characters [message #518107 is a reply to message #518098] Mon, 01 August 2011 05:20 Go to previous messageGo to next message
OraFerro
Messages: 433
Registered: July 2011
Senior Member
Thanks Michel, really appreciated.
Re: Detecting hidden characters [message #518108 is a reply to message #518107] Mon, 01 August 2011 05:28 Go to previous messageGo to next message
Michel Cadot
Messages: 68765
Registered: March 2007
Location: Saint-Maur, France, https...
Senior Member
Account Moderator
Just to show what I said:
SQL> create table t1 (v1 char(10));

Table created.

SQL> select * from t1 where v1='X';
V1
----------
X

1 row selected.

SQL> select * from t1 where v1='X                    ';
V1
----------
X

1 row selected.

SQL> create table t2 (v2 char(15));

Table created.

SQL> insert into t2 values ('X');

1 row created.

SQL> select * from t1, t2 where v1=v2;
V1         V2
---------- ---------------
X          X

1 row selected.

Regards
Michel
Re: Detecting hidden characters [message #518112 is a reply to message #518108] Mon, 01 August 2011 06:24 Go to previous messageGo to next message
OraFerro
Messages: 433
Registered: July 2011
Senior Member
And adding to your example to show the case when Varchar2 is compared to CHAR:
create table t3 (v3 varchar(5));

insert into t3 values ('X');

select * from t2, t3 where v2= v3;



No rows are returned unless padding is used:
select * from t2, t3 where v2= rpad(v3,15, ' ' );


According to many Oracle Documents I have read becuase of this case, the use of CHAR should be strictly limited to columns with fixed length.

Thanks again,
Re: Detecting hidden characters [message #518113 is a reply to message #518112] Mon, 01 August 2011 06:28 Go to previous messageGo to next message
cookiemonster
Messages: 13967
Registered: September 2008
Location: Rainy Manchester
Senior Member
I'd use trim in that case, saves having to worry how long the char column is.
But really I'd just avoid char completely. It has no benefits over varchar, only downsides.
Re: Detecting hidden characters [message #518114 is a reply to message #518113] Mon, 01 August 2011 06:35 Go to previous messageGo to next message
Michel Cadot
Messages: 68765
Registered: March 2007
Location: Saint-Maur, France, https...
Senior Member
Account Moderator
Quote:
But really I'd just avoid char completely. It has no benefits over varchar, only downsides.

I concur.

Regards
Michel
Re: Detecting hidden characters [message #518116 is a reply to message #518114] Mon, 01 August 2011 06:41 Go to previous messageGo to next message
OraFerro
Messages: 433
Registered: July 2011
Senior Member
I agree too, for a simple reason, the cases where you have a fixed length column whos data will never change is extremely limited (ex. possible values for gender were previously thought of as either F/M... see how far from reality this is now) Smile

For using trim, i agree of course, i just used padding to show the difference in a more detailed way.

thanks,

[Updated on: Mon, 01 August 2011 06:42]

Report message to a moderator

Re: Detecting hidden characters [message #518117 is a reply to message #518116] Mon, 01 August 2011 06:48 Go to previous messageGo to next message
OraFerro
Messages: 433
Registered: July 2011
Senior Member
by the way Michel

Quote:

I do not trust anything from this tool.


Do you trust any other tool except Sql plus?
Re: Detecting hidden characters [message #518119 is a reply to message #518117] Mon, 01 August 2011 06:59 Go to previous message
Michel Cadot
Messages: 68765
Registered: March 2007
Location: Saint-Maur, France, https...
Senior Member
Account Moderator
I do not trust any tool that does not show what it does.
SQL*Plus (or any other sql command tool) just sends what you write to Oracle engine, I trust them; any tool that shows what it sends to Oracle I trust it; a tool that hides what it sends I do not trust it (of course I could trace it but I prefer do stay away).

Regards
Michel
Previous Topic: calling procedure into function
Next Topic: Transpose (Not Pivot) a Table
Goto Forum:
  


Current Time: Thu Aug 14 15:23:11 CDT 2025