Home » RDBMS Server » Server Administration » what is the file ORA_DUMMY_FILE.f in oracle? (Oracle 12.2.0.1)
what is the file ORA_DUMMY_FILE.f in oracle? [message #669568] Mon, 30 April 2018 05:52 Go to next message
tombatossals
Messages: 5
Registered: April 2018
Junior Member
We have a recently installed database oracle 12.2 on linux that is currently used by a developers team (no real data there).
We have found that when executing any parallel query the next error arises:
ORA-12801: error signaled in parallel query server P001
ORA-01116: error in opening database file 132
ORA-01110: data file 132: '/u02/oradata/ora12c/pdbname/tablespacenaname_ts_1.dbf'
ORA-27077: too many files open

After some investigation we have found this:
As you know, these are the linux processes for the parallel servers in oracle:
ora_p000_ora12c
ora_p001_ora12c
....
ora_p???_ora12c
They can be seen also with the view: gv$px_process. The spid for each parallel server can be obtained from there.

Then I look for the open files associated with te parallel server here:
ls -l /proc/<spid>/fd

And I'm obtaining around 65000 file descriptors for several parallel servers equal to this one:
991 -> /u01/app/oracle/admin/ora12c/dpdump/676185682F2D4EA0E0530100007FFF5E/ORA_DUMMY_FILE.f (deleted)
I've checked that the max open files in linux is set to 65536

I've closed them using:
gdb -p <spid>
    gdb> p close(<fd_id>
)
(actually I've create a small script for doing it because there are thousands of them)

When the file descriptors are closed the error in parallel queries does no appear any more, but after some hours the file descriptors start being created again (hundreds every day)

If they are not closed then eventually the linux limit is reached and any parallel query throws again the same error.

Does anyone have any idea of how and why this file descriptors are being created, and how to avoid it?.

Some more information that could be useful.
I've tested that when a new PDB is created a directory DATA_PUMP_DIR is created in it (select * from all_directories) that is pointing to
/u01/app/oracle/admin/ora12c/dpdump/<xxxxxxxxxxxxx>

The linux directory is also created. Also one file descriptor is created pointing to ORA_DUMMY_FILE.f in the new dpdump subdirectory like the ones described initially
lsof | grep "ORA_DUMMY_FILE.f (deleted)"
/u01/app/oracle/admin/ora12c/dpdump/<xxxxxxxxxxxxx>/ORA_DUMMY_FILE.f (deleted)
This may be ok, the problem I face is the continuos growing of the file descriptors pointing to ORA_DUMMY_FILE that reach the linux limits.

Re: what is the file ORA_DUMMY_FILE.f in oracle? [message #669570 is a reply to message #669568] Mon, 30 April 2018 06:12 Go to previous messageGo to next message
Frank Naude
Messages: 4557
Registered: April 1998
Senior Member
You need to open an SR with Oracle Support:
$ oerr ora 27077
27077, 00000, "too many files open"
// *Cause:  internal error, the number of files opened through skgfofi has 
//          reached the limit
// *Action: Check for trace file and contact Oracle Support.
As workaround, increase file-max even further (65536 is way too low). For example, on one of my servers:

cat /proc/sys/fs/file-max
6815744

This document may help:
https://docs.oracle.com/en/database/oracle/oracle-database/12.2/unxar/administering-oracle-database-on-linux.html
Re: what is the file ORA_DUMMY_FILE.f in oracle? [message #669575 is a reply to message #669570] Mon, 30 April 2018 08:16 Go to previous messageGo to next message
tombatossals
Messages: 5
Registered: April 2018
Junior Member
Thank you for your answer Frank!,

The limit 65536 I talked initially is the "per-process number of file descriptors" obtained with: ulimit -n

I've checked the /proc/sys/fs/file-max as you have pointed out and we already had this value ok:
cat /proc/sys/fs/file-max
6815744


Also, I've checked the "processes" values in the instance:
select * from v$parameter where name ='processes';
select resource_name,current_utilization,max_utilization 
  from v$resource_limit 
 where resource_name in ('processes'); 
 
v$parameter.processes=1000 : 
v$resource_limit .current=146	
v$resource_limit.max_utilization=662
and according to the link you have posted, the recommended number of file descriptors for each Oracle instance is 512*PROCESSES
So, 512*1000=512000 that is much smaller than 6815744.

So, it doesn't look that the file descriptors limits are the root cause.
Re: what is the file ORA_DUMMY_FILE.f in oracle? [message #669576 is a reply to message #669575] Mon, 30 April 2018 08:19 Go to previous messageGo to next message
BlackSwan
Messages: 26139
Registered: January 2009
Location: SoCal
Senior Member
27077, 00000, "too many files open"
// *Cause:  internal error, the number of files opened through skgfofi has
//          reached the limit
// *Action: Check for trace file and contact Oracle Support.
[oracle@vbgeneric ~]$ 
Re: what is the file ORA_DUMMY_FILE.f in oracle? [message #669578 is a reply to message #669576] Mon, 30 April 2018 08:37 Go to previous messageGo to next message
EdStevens
Messages: 969
Registered: September 2013
Senior Member
BlackSwan wrote on Mon, 30 April 2018 08:19
27077, 00000, "too many files open"
// *Cause:  internal error, the number of files opened through skgfofi has
//          reached the limit
// *Action: Check for trace file and contact Oracle Support.
[oracle@vbgeneric ~]$ 

With the emphasis on "contact Oracle Support" Razz
Re: what is the file ORA_DUMMY_FILE.f in oracle? [message #669579 is a reply to message #669576] Mon, 30 April 2018 08:46 Go to previous messageGo to next message
Frank Naude
Messages: 4557
Registered: April 1998
Senior Member
Please post output of:
1. select * from dba_directories where directory_name like '%DUMMY%';
2. show parameters parallel;

Re: what is the file ORA_DUMMY_FILE.f in oracle? [message #669580 is a reply to message #669579] Mon, 30 April 2018 09:02 Go to previous messageGo to next message
tombatossals
Messages: 5
Registered: April 2018
Junior Member
Hi Frank,

1.-select * from dba_directories where directory_name like '%DUMMY%';
--> No rows are returned. Neither at CDB nor at PDB.
2.- show parameters parallel;
 NAME                            TYPE    VALUE  
------------------------------- ------- ------ 
containers_parallel_degree      integer 65535  
fast_start_parallel_rollback    string  LOW    
parallel_adaptive_multi_user    boolean FALSE  
parallel_degree_limit           string  CPU    
parallel_degree_policy          string  MANUAL 
parallel_execution_message_size integer 16384  
parallel_force_local            boolean FALSE  
parallel_instance_group         string         
parallel_max_servers            integer 480    
parallel_min_percent            integer 0      
parallel_min_servers            integer 48     
parallel_min_time_threshold     string  AUTO   
parallel_servers_target         integer 1      
parallel_threads_per_cpu        integer 2      
recovery_parallelism            integer 0 
Re: what is the file ORA_DUMMY_FILE.f in oracle? [message #669581 is a reply to message #669580] Mon, 30 April 2018 09:45 Go to previous messageGo to next message
Frank Naude
Messages: 4557
Registered: April 1998
Senior Member
This is what I have in /etc/security/limits.conf
oracle soft nofile 1024
oracle hard nofile 65536

Other than adjusting "nofiles", I'm afraid that you will need the help of an Oracle Support Analyst, as this is most likely a bug.
Re: what is the file ORA_DUMMY_FILE.f in oracle? [message #669582 is a reply to message #669581] Mon, 30 April 2018 09:59 Go to previous message
tombatossals
Messages: 5
Registered: April 2018
Junior Member
ok, thank you very much for your time Frank!,
Previous Topic: Database Upgrade
Next Topic: Meta data for Table Partition (merged 2 CM)
Goto Forum:
  


Current Time: Sun Sep 23 20:30:41 CDT 2018