Re: Oralce 8i Japanese OS UTF8

On 12 Aug 2001, darren_at_virtualvoodoo.net wrote:

> And characters are correctly passed through and retrieved to
> the database when the browser encoding is set to utf-8.

How can you tell this is true? It seems you can't retrieve and display characters from your database, so how do you know they have been stored correctly?

Maybe you can store something with the IME into a file? Then, read that file with java, telling it what character set to read. It will then, internally convert it to UCS2 (which is what java works with internally) then, java can use jdbc thin to enter the character. SQLNet will then convert it to UTF-8 and give it to Oracle which will store the data. Then, retrieving with java, the process is reversed. Now, with java, write that data into another file with the same IME character set. You can compare this saved file with the originally written file and see if they are the same.

Have you used something like the above to prove that the data is stored in Oracle correctly?

> I have an English Windows 2000 Professional workstation,
> configured for Native Japanese support (the locale and default
> language settins are all set for japanese).
>
> When I run Oracle tools such as sqlplus, Toad, etc,

Does sqlplus understand multi-byte? I'm not sure about this one, but I would assume it must. Hm... Are you actually able to type select clauses and somehow use the IME to input the japanese characters all within sqlplus? Thats pretty cool, now that I think about it.

How bout Toad? This one might be less likely to even understand multi-byte.

> and select japanese columns input via the web, the characters
> shown are not correct.

I don't understand what isn't correct. Are you retrieving multi-byte which isn't getting displayed correctly in a browser? Like I asked before, how are you sure that the data is correctly stored within Oracle?

> I assume this is because the operating system is expecting
> shift-JIS characters, and oracle is just returning raw UTF-8
> data...

SQLNet makes this translation happen for you.

> If I manually insert or update columns using sqlplus and the
> Windows IME , the characters come back as entered,

Through SQLPLUS> "select blah from table;"

If so, then I guess the characters are saved correctly in the database and my previous comments might be disregarded.

> however, trying to pull up one of those records through the
> web-app causes oracle to throw an error (cannot convert utf-8
> to ucs2

Hm... Then, I would say that java is asking sqlnet to convert to ucs2 and that is causing problems. All of the translation from the database to the client happens at the sqlnet layer. Can you set the environment for your sqlplus session to UCS2, do a retrieval and see if you get the same error?

> - or something like that). I would like to have consistent
> characters in the database from both web-based and manual
> inserts.

I don't think this is the issue. It seems that converting from UTF-8 to the client environment of UCS2 is the issue, although this seems a bit odd.

> So, my question boils down to the following: Is there a way to
> configure the NLS_LANG value in the registry of the Win2k
> Japanese computer so that it can map between the OS Shift-JIS
> and the database UTF-8?

If you are able to use sqlplus to retrieve the correct characters, then its the UCS2 character set conversion within java that seems to be the issue. Hm... I would expect this to work though.

> I've tried using parameters of JAPANESE_JAPAN.UTF8 and
> JAPANESE_JAPAN.JA16SJIS but neither one seems to help with this
> problem.
>
> Is there something else that I'm missing?

I don't have a multi-lingual environment to play with, but I did quite a bit banging my head working with Oracle 7.3 and multi-byte characters. I think we might be able to figure this out, just don't be shy about continuing this thread.

-- 
Galen Boyer
It seems to me, I remember every single thing I know.