Home » SQL & PL/SQL » SQL & PL/SQL » Batch file to run sql script (11g)
Batch file to run sql script [message #651226] Fri, 13 May 2016 04:18 Go to next message
harshadsp
Messages: 100
Registered: August 2008
Senior Member

Hello Experts, need your thoughts on this!
I have created a .sql script which is currently taking a file at oracle directory and execute the sql code to split the input file on multiple files using utl_file commands(as we have discussed previously). But when I give this code to users they need some another way to execute this.

Users will have the input files(say 10) in their windows directory and they want a utility which will select one file from those 10 files and then run my sql program and they will get the multiple output files in same directory.

I was thinking to create a batch file (which I honestly don't know how it works) which will prompt the input file and then user will select the required file and then the sql program will split it into multiple output files. I am not sure how I could point the windows directory instead of my utl/oracle directory or even I am not sure which is the best way to implement this functionality.

Could you please advise how I can proceed for this?

Thanks!
Re: Batch file to run sql script [message #651235 is a reply to message #651226] Fri, 13 May 2016 06:32 Go to previous messageGo to next message
EdStevens
Messages: 1376
Registered: September 2013
Senior Member
PL/SQL executes within the database and can only access files/directories that are accessible to the database server. So no, you will not be able to have utl_file accesses,create,or manipulate files on the client's PC.

What relation do these files have to the database? From your description, it sounds like no relationship at all and you are simply using utl_file because that's the tool you know.
Re: Batch file to run sql script [message #651239 is a reply to message #651235] Fri, 13 May 2016 07:37 Go to previous messageGo to next message
harshadsp
Messages: 100
Registered: August 2008
Senior Member

yes, there is no relation to database. The file which I need to read is ftp'ed to utl_dir in oracle and my code reads that and generates output files in same utl_directory. I am thinking if I could ftp files from sql script and not from shell script? Is it possible? Please advsie.
Re: Batch file to run sql script [message #651240 is a reply to message #651239] Fri, 13 May 2016 07:49 Go to previous messageGo to next message
harshadsp
Messages: 100
Registered: August 2008
Senior Member

But I may get to interact with database to correct some of the values while writing the output file so I used utl_file.
Re: Batch file to run sql script [message #651241 is a reply to message #651240] Fri, 13 May 2016 08:36 Go to previous messageGo to next message
BlackSwan
Messages: 26766
Registered: January 2009
Location: SoCal
Senior Member
So what do you expect/desire from us?
How will you & I know when correct answer is posted here?
Re: Batch file to run sql script [message #651242 is a reply to message #651241] Fri, 13 May 2016 08:38 Go to previous messageGo to next message
harshadsp
Messages: 100
Registered: August 2008
Senior Member

I just want to know if there is any better way to do this.
Re: Batch file to run sql script [message #651243 is a reply to message #651242] Fri, 13 May 2016 08:50 Go to previous messageGo to next message
BlackSwan
Messages: 26766
Registered: January 2009
Location: SoCal
Senior Member
which metric measure better?
What is unit of measure for better?
Re: Batch file to run sql script [message #651247 is a reply to message #651242] Fri, 13 May 2016 09:39 Go to previous messageGo to next message
Michel Cadot
Messages: 68645
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator

Quote:
The file which I need to read is ftp'ed to utl_dir


Do NOT use UTL_DIR it is a security hole.
Use Oracle DIRECTORY object.

harshadsp wrote on Fri, 13 May 2016 15:38
I just want to know if there is any better way to do this.


Yes.

Re: Batch file to run sql script [message #651248 is a reply to message #651247] Fri, 13 May 2016 09:41 Go to previous messageGo to next message
BlackSwan
Messages: 26766
Registered: January 2009
Location: SoCal
Senior Member
is application 3-tier?
Re: Batch file to run sql script [message #651270 is a reply to message #651248] Sat, 14 May 2016 01:25 Go to previous messageGo to next message
harshadsp
Messages: 100
Registered: August 2008
Senior Member

The Oracle is at another Unix server and windows is at another server. By better way, I mean to say if there is any oracles predefined library which I can use to ftp files from windows to Unix oracle server and vice versa. When I goggled I found some ftp.login, xutl_ftp etc but when I use those inside my code its giving error that identifier must be declared. So just want to know if I can use whatever 11g has already without installing anything more.

@Michael sorry but I didn't understand "Do NOT use UTL_DIR it is a security hole. Use Oracle DIRECTORY object.". Is it advisable that I could mount the windows share drive to Unix and then map this drive in utl directories of oracle and run my sql program?

Another way I am thinking is (If our Unix admin say NO to mount the windows drive on Unix), create a batch file, place this in users windows directory. Batch file will
1) first ftp the input files to utl directory path
2) then it will call sql script and prompt user to enter input file to be processed
3) run the sql script for that input parameter(file)
4) sql will generate output files in utl directory
5) ftp output files again back to users windows directory
All this will be done through batch file when user run the same.

I am not sure if this is correct way. Basically user should drive all the process so I thought to make a batch file like above.

Please advise your opinions, thanks!
Re: Batch file to run sql script [message #651272 is a reply to message #651270] Sat, 14 May 2016 01:53 Go to previous messageGo to next message
Michel Cadot
Messages: 68645
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator

Quote:
@Michael sorry but I didn't understand "Do NOT use UTL_DIR it is a security hole. Use Oracle DIRECTORY object.". Is it advisable that I could mount the windows share drive to Unix and then map this drive in utl directories of oracle and run my sql program?


Typo: it is UTL_FILE_DIR.

Quote:
All users can read or write to all files specified by this parameter. Therefore all PL/SQL users must be trusted with the information in the directories specified by this parameter.


UTL_FILE Security Model

CREATE DIRECTORY

Re: Batch file to run sql script [message #651273 is a reply to message #651272] Sat, 14 May 2016 01:56 Go to previous messageGo to next message
harshadsp
Messages: 100
Registered: August 2008
Senior Member

Thanks Michael. when I see my database, I noticed that both the utl_dir and oracle dir (dba_directories) have the same path entries.
Re: Batch file to run sql script [message #651274 is a reply to message #651273] Sat, 14 May 2016 02:31 Go to previous messageGo to next message
Michel Cadot
Messages: 68645
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator

So remove the UTL_FILE_DIR parameter and ensure that your code uses the Oracle DIRECTORY (that is does not name the OS directory). In addition to be more secure, it allows you to change the OS directory without having to change your code, just the Oracle directory definition.

Re: Batch file to run sql script [message #651293 is a reply to message #651240] Sat, 14 May 2016 18:50 Go to previous messageGo to next message
EdStevens
Messages: 1376
Registered: September 2013
Senior Member
harshadsp wrote on Fri, 13 May 2016 07:49
But I may get to interact with database to correct some of the values while writing the output file so I used utl_file.


So there is a relationship between the file and the database!

Since you are willing to place a batch file on the user's desktop, you could - instead - place an executable written in a real programing language (perl? .net? C#? C++? Java?) and have that app query the database for the necessary information, avoiding all the shipping of files back and forth.
Re: Batch file to run sql script [message #651294 is a reply to message #651270] Sat, 14 May 2016 19:01 Go to previous messageGo to next message
EdStevens
Messages: 1376
Registered: September 2013
Senior Member
harshadsp wrote on Sat, 14 May 2016 01:25
The Oracle is at another Unix server and windows is at another server. By better way, I mean to say if there is any oracles predefined library which I can use to ftp files from windows to Unix oracle server and vice versa. When I goggled I found some ftp.login, xutl_ftp etc but when I use those inside my code its giving error that identifier must be declared. So just want to know if I can use whatever 11g has already without installing anything more.


Did you consider asking for help with that code? Most likely it does NOT involve installing anything more, but simply debugging your code and possibly granting some permissions.



Quote:
@Michael sorry but I didn't understand "Do NOT use UTL_DIR it is a security hole. Use Oracle DIRECTORY object.". Is it advisable that I could mount the windows share drive to Unix and then map this drive in utl directories of oracle and run my sql program?

Another way I am thinking is (If our Unix admin say NO to mount the windows drive on Unix), create a batch file, place this in users windows directory. Batch file will
1) first ftp the input files to utl directory path
2) then it will call sql script and prompt user to enter input file to be processed


If you're asking for the name of the file in step 2, how did step 1 know what file to ftp?

Quote:

3) run the sql script for that input parameter(file)
4) sql will generate output files in utl directory
5) ftp output files again back to users windows directory
All this will be done through batch file when user run the same.

I am not sure if this is correct way. Basically user should drive all the process so I thought to make a batch file like above.

Please advise your opinions, thanks!


I can imagine writing a batch file to do this but you'd have to take a lot of other things into consideration. And at the top of the list is having a fool-proof, 100% guaranteed method on making sure that two users running at the same time don't end up colliding on file names.

And while I can imagine it being technically feasible within the broad outlines of what you describe, I can't imagine it being the best approach, even after tweaking the details. Actually, I'd prefer a web-based app written in java or .net. Then you don't have to install ANYTHING on the user's PC, AND you avoid having to ship the d*** files back and forth.
Re: Batch file to run sql script [message #651302 is a reply to message #651274] Sun, 15 May 2016 05:01 Go to previous messageGo to next message
harshadsp
Messages: 100
Registered: August 2008
Senior Member

@Michael-
I have created a new Oracle directory for the windows directory and grant ALL to public. Now from unix box I could see the input files listed under that oracle directory but when I run my code its saying invalid file operation. This path is not there in UTL_FILE_DIR. When I use the directory which has entry in UTL_FILE_DIR the code is running perfect. For some reason its not reading the file from oracle directory. Could you please advise what I need to verify?

Thanks
Re: Batch file to run sql script [message #651303 is a reply to message #651302] Sun, 15 May 2016 05:13 Go to previous messageGo to next message
Michel Cadot
Messages: 68645
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator

Quote:
I have created a new Oracle directory for the windows directory and grant ALL to public.


The purpose of Oracle directory is NOT to grant ALL to everyone but to grant the specific privilege each one require for his job. I you grant ALL to everyone you did not change anything from the previous behaviour (from a security point of view). Read again what Oracle says:

Quote:
All users can read or write to all files specified by this parameter. Therefore all PL/SQL users must be trusted with the information in the directories specified by this parameter.


Can you trust everyone?

Are your files in the directory itself or in a subdirectory.
What what the previous UTL_FILE_DIR value?
What is the DIRECTORY definition (query result from DBA_DIRECTORIES)?

Re: Batch file to run sql script [message #651304 is a reply to message #651303] Sun, 15 May 2016 06:29 Go to previous messageGo to next message
harshadsp
Messages: 100
Registered: August 2008
Senior Member

Michael, when I move my code to prod, I will take care of specific users to have access this path. Thanks for pointing this.
My files are in directory itself. I can see those when using ls -lrt on unix box under the same directory.
The previous UTL_FILE_DIR has the entries for the path which are on same server where oracle is.
The result from dba_directories:-

OWNER DIRECTORY_NAME DIRECTORY_PATH
1 SYS WCPOLS_DIR /home/hpatil/WCPOLS -- This drive is mapped to my windows directory (G:\WCPOLS)-- this is not working in my code
5 SYS CENGROUP_OUTPUT /home/cengroup/cent/dev0/batch/output -- This Is on unix box where oracle is installed and this works perfect in my code
Re: Batch file to run sql script [message #651305 is a reply to message #651304] Sun, 15 May 2016 07:40 Go to previous messageGo to next message
Michel Cadot
Messages: 68645
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator

If this works in one directory and not in the other one (and if is exists) this means Oracle account (not yours) has not the access to it.

Re: Batch file to run sql script [message #651306 is a reply to message #651305] Sun, 15 May 2016 09:50 Go to previous messageGo to next message
harshadsp
Messages: 100
Registered: August 2008
Senior Member

Michael-
The CENGROUP_OUTPUT directory has entries in both UTL_FILE_DIR and DBA_DIRECTORIES whereas the newly created WCPOLS_DIR is just a oracle directory and not set in UTL_FILE_DIRECTORY and I think that's the reason utl_file package is not working with this directory. Could this will be the reason?

Now this made me to think that I should write a shell script to first ftp the files in CENGROUP_OUTPUT directory and then run the sql based on input file.

@EdStevens -
Your input are valid too for batch file creation. I will consider those while programming. Also I don't know much about creating executable with C, C#, Perl etc Smile So the only option for me is do with Oracle only Wink I may write shell to ftp the files.

Thanks for all your suggestions!
Re: Batch file to run sql script [message #651307 is a reply to message #651306] Sun, 15 May 2016 09:54 Go to previous messageGo to next message
Michel Cadot
Messages: 68645
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator

Quote:
The CENGROUP_OUTPUT directory has entries in both UTL_FILE_DIR and DBA_DIRECTORIES whereas the newly created WCPOLS_DIR is just a oracle directory and not set in UTL_FILE_DIRECTORY and I think that's the reason utl_file package is not working with this directory. Could this will be the reason?


No unless your code is wrong. You do not use both directories in the same way.
Post the code which calls UTL_FILE.FOPEN.

Edit: with the value of the variables.

[Updated on: Sun, 15 May 2016 09:54]

Report message to a moderator

Re: Batch file to run sql script [message #651338 is a reply to message #651307] Mon, 16 May 2016 01:43 Go to previous messageGo to next message
harshadsp
Messages: 100
Registered: August 2008
Senior Member

Here is the code with new directory WCPOLS_DIR-

DECLARE

  w_dir                   VARCHAR2(100);
  w_input_file            VARCHAR2(50);
  w_newline               VARCHAR2(2000);
  input_file    utl_file.file_type; 
BEGIN
   
    w_dir            := 'WCPOLS_DIR'; 
    w_input_file     := '&FileName';          

    input_file    := utl_file.fopen(w_dir, w_input_file, 'r',32000);
    
    IF utl_file.is_open(input_file) THEN
     LOOP
     BEGIN
       utl_file.get_line(input_file, w_newline);
      EXCEPTION WHEN NO_DATA_FOUND THEN
          EXIT;
       END;

      END LOOP;
    END IF; 
   COMMIT;
   utl_file.fclose(input_file);   
END; 


This gives error Ora-29283 Invalid file operation

And this is the code with CENGROUP_OUTPUT directory which works perfect
DECLARE

  w_dir                   VARCHAR2(100);
  w_input_file            VARCHAR2(50);
  w_newline               VARCHAR2(2000);
  input_file    utl_file.file_type; 
BEGIN
   
    w_dir            := 'WCPOLS_DIR'; 
    w_input_file     := '&FileName';          

    input_file    := utl_file.fopen(w_dir, w_input_file, 'r',32000);
    
    IF utl_file.is_open(input_file) THEN
     LOOP
     BEGIN
       utl_file.get_line(input_file, w_newline);
      EXCEPTION WHEN NO_DATA_FOUND THEN
          EXIT;
       END;

      END LOOP;
    END IF; 
   COMMIT;
   utl_file.fclose(input_file);   
END; 


The entries in dba_directories
OWNER DIRECTORY_NAME DIRECTORY_PATH
1 SYS WCPOLS_DIR /home/hpatil/WCPOLS
5 SYS CENGROUP_OUTPUT /home/cengroup/cent/dev0/batch/output

The entry on CENGROUP_OUTPUT in UTL_FILE_DIR
NUM NAME TYPE VALUE DISPLAY_VALUE ISDEFAULT ISSES_MODIFIABLE ISSYS_MODIFIABLE ISINSTANCE_MODIFIABLE ISMODIFIED ISADJUSTED ISDEPRECATED ISBASIC DESCRIPTION UPDATE_COMMENT HASH
2 1656 utl_file_dir 2 /cent/dev0/batch/output /home/arggroup/argt/dv0/batch/output, /cent/dev0/batch/output FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE utl_file accessible directories list 379739569[/b][
Re: Batch file to run sql script [message #651340 is a reply to message #651338] Mon, 16 May 2016 01:58 Go to previous messageGo to next message
Michel Cadot
Messages: 68645
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator

You tell 2 things at the same time and so confuse me.
1/ It works with UTL_FILE_DIR and not with Oracle directory
2/ It works when Oracle directory has a value and not the other one.

I don't see any difference in both codes you posted, so I assume you are talking about point 2 (with a different definition of 'WCPOLS_DIR') for which I said:

Quote:
If this works in one directory and not in the other one (and if is exists) this means Oracle account (not yours) has not the access to it.


What is the value of "&FileName" (I assume you give the same one in both tests, is this true?)

Re: Batch file to run sql script [message #651341 is a reply to message #651340] Mon, 16 May 2016 02:12 Go to previous messageGo to next message
harshadsp
Messages: 100
Registered: August 2008
Senior Member

Yes I gave same file names for both the scripts
Re: Batch file to run sql script [message #651342 is a reply to message #651341] Mon, 16 May 2016 02:14 Go to previous messageGo to next message
harshadsp
Messages: 100
Registered: August 2008
Senior Member

I will check with DBA if oracle account has access to the WCPOLS_DIR directory. The code and input parameter/file is same for both the directories which means there is really some access missing for oracle on wcpols_dir. (?)
Re: Batch file to run sql script [message #651343 is a reply to message #651341] Mon, 16 May 2016 02:17 Go to previous messageGo to next message
Michel Cadot
Messages: 68645
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator

Which is?
Please use SQL*Plus and copy and paste the session, the whole session, with success and failures, with the result of:
set lines 100 trimout on
col directory_name format a15
col directory_path format a80
select directory_name, directory_path from dba_directories order by 1;
col name format a15
col value format a80
select name, value from v$parameter where name='utl_file_dir';

And post all these formatted, using code tags.

Re: Batch file to run sql script [message #651344 is a reply to message #651342] Mon, 16 May 2016 02:19 Go to previous messageGo to next message
Michel Cadot
Messages: 68645
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator
harshadsp wrote on Mon, 16 May 2016 09:14
I will check with DBA if oracle account has access to the WCPOLS_DIR directory. The code and input parameter/file is same for both the directories which means there is really some access missing for oracle on wcpols_dir. (?)


No, the problem is at OS level NOT at Oracle one.
Post the SQL*Plus session as I requested to show us what you actually do.

Re: Batch file to run sql script [message #651345 is a reply to message #651342] Mon, 16 May 2016 02:20 Go to previous messageGo to next message
harshadsp
Messages: 100
Registered: August 2008
Senior Member

Michael,
I have some doubt. The code line "input_file := utl_file.fopen(w_dir, w_input_file, 'r',32000);". Could utl_file.fopen command really access the wcpols_dir directory which is not listed in UTL_FILE_DIR? I am not sure, please advise. The other directory has an entry in UTL_FILE_DIR and may be that's the reason it can be read/write on that one.
Re: Batch file to run sql script [message #651346 is a reply to message #651345] Mon, 16 May 2016 02:21 Go to previous messageGo to next message
harshadsp
Messages: 100
Registered: August 2008
Senior Member

Sorry but the SQL*Plus is not available in my Citrix profile so I just can't show that. I am running my code in pl/sql developer.
Re: Batch file to run sql script [message #651348 is a reply to message #651346] Mon, 16 May 2016 02:23 Go to previous messageGo to next message
harshadsp
Messages: 100
Registered: August 2008
Senior Member

I used the command line in pl/sql developer and here it is:-

Connected to Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 
Connected as ops$cendba
 
SQL> 
SQL> set lines 100 trimout on
Cannot SET LINES
SQL> col directory_name format a15
SQL> col directory_path format a80
SQL> select directory_name, directory_path from dba_directories order by 1;
 
DIRECTORY_NAME  DIRECTORY_PATH
--------------- --------------------------------------------------------------------------------
CENGROUP_OUTPUT /home/cengroup/cent/dev0/batch/output
DATA_PUMP_DIR   /u01/app/oracle/product/11.2.0/dbhome_1/rdbms/log/
DMP1            /backup/DDomain/dev/GCPRMT/export
DMP1_DOMAIN     /backup/DDomain/dev/GCPRMT/export
DMP2            /backup/DDomain/dev/GCPRMT/export
ORACLE_OCM_CONF /u01/app/oracle/product/11.2.0/dbhome_1/ccr/state
IG_DIR          
 
WCPOLS_DIR      /home/hpatil/WCPOLS
XMLDIR          /u01/app/oracle/product/11.2.0/dbhome_1/rdbms/xml
XMLFILES        /home/e000/cengroup/cenp/prod/batch/output/XML
 
9 rows selected
SQL> col name format a15
SQL> col value format a80
SQL> select name, value from v$parameter where name='utl_file_dir';
 
NAME            VALUE
--------------- --------------------------------------------------------------------------------
utl_file_dir    /home/arggroup/argt/dv0/batch/output, /cent/dev0/batch/output
 
SQL> 
Re: Batch file to run sql script [message #651349 is a reply to message #651348] Mon, 16 May 2016 02:32 Go to previous messageGo to next message
Michel Cadot
Messages: 68645
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator

And the execution of the PL/SQL blocks for both it works and not, and everything you execute between.

Note; if you use Oracle directories what's inside UTL_FILE_DIR is irrelevant.

Re: Batch file to run sql script [message #651350 is a reply to message #651349] Mon, 16 May 2016 02:37 Go to previous messageGo to next message
harshadsp
Messages: 100
Registered: August 2008
Senior Member

The pl/sql block with CENGROUP_OUTPUT works but for WCPOLS_DIR doesn't work. If UTL_FILE_DIR doesn't matter then may be there is some access issue for WCPOLS_DIR.
Re: Batch file to run sql script [message #651355 is a reply to message #651306] Mon, 16 May 2016 07:19 Go to previous messageGo to next message
EdStevens
Messages: 1376
Registered: September 2013
Senior Member
harshadsp wrote on Sun, 15 May 2016 09:50

@EdStevens -
Your input are valid too for batch file creation. I will consider those while programming. Also I don't know much about creating executable with C, C#, Perl etc Smile So the only option for me is do with Oracle only Wink I may write shell to ftp the files.


A bit of career advice from somone who's bee doing this for over 35 years ... If your only tool is a hammer, every problem looks like a nail. A hammer is fine for driving nails, but is not very effective for ripping a board to size.
Re: Batch file to run sql script [message #651399 is a reply to message #651355] Tue, 17 May 2016 05:51 Go to previous message
harshadsp
Messages: 100
Registered: August 2008
Senior Member

@EdStevens- You are correct. Smile I need to work on other programing as well. Agree with you Smile Thank you!
Previous Topic: NUMBER to HEX, HEX to NUMBER
Next Topic: oracle server CPU spikes
Goto Forum:
  


Current Time: Fri Apr 26 04:54:08 CDT 2024