Home » SQL & PL/SQL » SQL & PL/SQL » Skipping Of records: Oracle External table (Oracle 10 g)
( ) 1 Vote
|
|
Re: Skipping Of records: Oracle External table [message #524571 is a reply to message #524566] |
Sat, 24 September 2011 18:36   |
 |
Roadies99
Messages: 28 Registered: May 2011 Location: Delhi
|
Junior Member |

|
|
Apologies for the formatting issue but got tangled with the Formatting Tools as loads of spaces came in.
Coming to the basic point:
I have flat text file on Unix operating system that is required to be loaded to Oracle tables through Oracle External Tables.
while loading the file points needs to be considered
1 The field deilimeter is a TAB character
2 The record delimeter is a newline character
3 The values in the field are within double quotes
4 The fields are not fixed length, rather they are to be loaded as per length of the datatype is specified of the columns in the table
5 Not all field would always have values. There can be empty values where datatype is either varchar2 or number.
6 The file has values of the fields like #Mi which needs to be ignored; and NULL to be in place of a field having varchar2 datatype and 0 in case of number datatype
The sequence of the columns and their datatypes for the table where data is to be loaded are:
component Varchar2 (79)
cost_category Varchar2 (79)
axis Varchar2 (79)
coordinate Varchar2 (79)
curve_type Varchar2 (79)
measure Varchar2 (79)
Version Varchar2 (79)
FAC Number(30,10)
LRIC Number(30,10)
Marginal Number(30,10)
SACS Number(30,10)
DLRIC Number(30,10)
DSAC Number(30,10)
NSOC Number(30,10)
LRIC_FAC_Ratio Number(30,10)
DLRIC_FAC_Ratio Number(30,10)
DSAC_FAC_Ratio Number(30,10)
FAC_TC Number(30,10)
LRIC_TC Number(30,10)
DLRIC_TC Number(30,10)
DSAC_TC Number(30,10)
NSOC_TC Number(30,10)
NSOC_TC_2 Number(30,10)
No cost_convention Number(30,10)
Cost_convention Number(30,10)
The other points of contention are:
The file starts with the follwing which needs to be ignored that is:
"FAC" "LRIC" "Marginal" "SACS" "DLRIC" "DSAC" "NSOC" "LRIC_FAC Ratio" "DLRIC_FAC Ratio"
"DSAC_FAC Ratio" "FAC_TC" "LRIC_TC" "DLRIC_TC" "DSAC_TC"
"NSOC_TC" "NSOC_TC_2" "No cost_convention" "Cost_convention"
The above would be always be there hence this has to be ignored.
The loading of records would start after the above entry.
The sample data is provided below:
"FAC" "LRIC" "Marginal" "SACS" "DLRIC" "DSAC" "NSOC" "LRIC_FAC Ratio" "DLRIC_FAC Ratio" "DSAC_FAC Ratio" "FAC_TC" "LRIC_TC" "DLRIC_TC" "DSAC_TC" "NSOC_TC" "NSOC_TC_2" "No cost_convention" "Cost_convention"
"ICO371" "CECADRZZZZG2ZZZZ" "No Axis" "No co_ordinate" "No Curve_type" "Component_fac" "V211" 53009
"ICO372" "CECADRZZZZG2ZZZZ" "No Axis" "No co_ordinate" "No Curve_type" "Component_fac" "V211" 10702
"ICO373" "CECADRZZZZG2ZZZZ" "No Axis" "No co_ordinate" "No Curve_type" "Component_fac" "V211" 6300
"ICO551" "CECADRZZZZG2ZZZZ" "No Axis" "No co_ordinate" "No Curve_type" "Component_fac" "V211" 908
"ICO374" "CECADRZZZZG2ZZZZ" "No Axis" "No co_ordinate" "No Curve_type" "Component_fac" "V211" 3436
"ICO375" "CECADRZZZZG2ZZZZ" "No Axis" "No co_ordinate" "No Curve_type" "Component_fac" "V211" 4892
"ICO554" "CECADRZZZZG2ZZZZ" "No Axis" "No co_ordinate" "No Curve_type" "Component_fac" "V211" 465
#Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi 0
"No Component" "No Cost_category" "No Axis" "3" "CV230" "Gradient" "V211" #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi 0.3741053789473685
"No Component" "No Cost_category" "No Axis" "4" "CV230" "Gradient" "V211" #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi 0.40836471
"No Component" "No Cost_category" "No Axis" "5" "CV230" "Gradient" "V211" #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi 0.3991534699999998
"No Component" "No Cost_category" "No Axis" "6" "CV230" "Gradient" "V211" #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi 0.3684493599999995
"No Component" "No Cost_category" "No Axis" "7" "CV230" "Gradient" "V211" #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi 0.7852075900000013
"No Component" "No Cost_category" "No Axis" "8" "CV230" "Gradient" "V211" #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi 0.6765691699999999
"No Component" "No Cost_category" "No Axis" "9" "CV230" "Gradient" "V211" #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #
#Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi 0.5128884800000005
"No Component" "No Cost_category" "No Axis" "3" "cv270" "Gradient" "V211" #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi 0.51288848
"No Component" "No Cost_category" "No Axis" "4" "cv270" "Gradient" "V211" #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi 0.5128884699999993
"No Component" "No Cost_category" "No Axis" "5" "cv270" "Gradient" "V211" #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi 0.5128884799999998
"No Component" "No Cost_category" "No Axis" "6" "cv270" "Gradient" "V211" #Mi #Mi #Mi #Mi
Would send a mail for the sample file. As I found no way to attach a file.
[Updated on: Sun, 25 September 2011 00:34] by Moderator Report message to a moderator
|
|
|
|
|
Re: Skipping Of records: Oracle External table [message #524574 is a reply to message #524573] |
Sat, 24 September 2011 19:24   |
 |
Roadies99
Messages: 28 Registered: May 2011 Location: Delhi
|
Junior Member |

|
|
Just in case I have been nopt able to put it across.
The initial entry in the file needs to be omitted.
"FAC" "LRIC" "Marginal" "SACS" "DLRIC" "DSAC" "NSOC" "LRIC_FAC Ratio" "DLRIC_FAC Ratio"
"DSAC_FAC Ratio" "FAC_TC" "LRIC_TC" "DLRIC_TC" "DSAC_TC" "NSOC_TC"
"NSOC_TC_2" "No cost_convention" "Cost_convention"
The above is an entry that keeps on coming on each file from warehouse hence the respective entry above needs to be omitted.
Do not consider it as a header or column names.
The datainsertion into the table would take place after negation of the above entry, in this case it would start from
"FAC" "LRIC" "Marginal" "SACS" "DLRIC" "DSAC" "NSOC" "LRIC_FAC Ratio" "DLRIC_FAC Ratio"
"DSAC_FAC Ratio" "FAC_TC" "LRIC_TC" "DLRIC_TC" "DSAC_TC" "NSOC_TC"
"NSOC_TC_2" "No cost_convention" "Cost_convention"
"ICO371" "CECADRZZZZG2ZZZZ" "No Axis" "No co_ordinate" "No Curve_type" "Component_fac" "V211" 53009
Start of data entry would happen from the record marked bold.
[Updated on: Sun, 25 September 2011 00:35] by Moderator Report message to a moderator
|
|
|
|
Re: Skipping Of records: Oracle External table [message #524577 is a reply to message #524575] |
Sat, 24 September 2011 21:51   |
 |
Barbara Boehmer
Messages: 9106 Registered: November 2002 Location: California, USA
|
Senior Member |
|
|
I have provided the syntax for each of your requirements below, followed by a demonstration. You will need to replace my Windows directory path 'c:\my_oracle_files' in my example with your Unix directory path.
The initial entry in the file needs to be omitted.
SKIP 1
1 The field deilimeter is a TAB character
FIELDS TERMINATED BY X'09'
However, your datafile seems to use whitespace, not tabs, so:
FIELDS TERMINATED BY WHITESPACE
2 The record delimeter is a newline character
RECORDS DELIMITED BY NEWLINE
3 The values in the field are within double quotes
OPTIONALLY ENCLOSED BY '"'
4 The fields are not fixed length, rather they are to be loaded as per length of the datatype is specified of the columns in the table
FIELDS TERMINATED BY WHITESPACE OPTIONALLY ENCLOSED BY '"'
5 Not all field would always have values. There can be empty values where datatype is either varchar2 or number.
MISSING FIELD VALUES ARE NULL
6 The file has values of the fields like #Mi which needs to be ignored; and NULL to be in place of a field having varchar2 datatype and 0 in case of number datatype
For nulls:
NULLIF ("COLUMN_NAME" = "#Mi")
For 0:
Use NVL(column_name,0) in your queries.
-- c:\my_oracle_files\flat_file.dat:
"FAC" "LRIC" "Marginal" "SACS" "DLRIC" "DSAC" "NSOC" "LRIC_FAC Ratio" "DLRIC_FAC Ratio" "DSAC_FAC Ratio" "FAC_TC" "LRIC_TC" "DLRIC_TC" "DSAC_TC" "NSOC_TC" "NSOC_TC_2" "No cost_convention" "Cost_convention"
"ICO371" "CECADRZZZZG2ZZZZ" "No Axis" "No co_ordinate" "No Curve_type" "Component_fac" "V211" 53009
"ICO372" "CECADRZZZZG2ZZZZ" "No Axis" "No co_ordinate" "No Curve_type" "Component_fac" "V211" 10702
"ICO373" "CECADRZZZZG2ZZZZ" "No Axis" "No co_ordinate" "No Curve_type" "Component_fac" "V211" 6300
"ICO551" "CECADRZZZZG2ZZZZ" "No Axis" "No co_ordinate" "No Curve_type" "Component_fac" "V211" 908
"ICO374" "CECADRZZZZG2ZZZZ" "No Axis" "No co_ordinate" "No Curve_type" "Component_fac" "V211" 3436
"ICO375" "CECADRZZZZG2ZZZZ" "No Axis" "No co_ordinate" "No Curve_type" "Component_fac" "V211" 4892
"ICO554" "CECADRZZZZG2ZZZZ" "No Axis" "No co_ordinate" "No Curve_type" "Component_fac" "V211" 465
#Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi 0
"No Component" "No Cost_category" "No Axis" "3" "CV230" "Gradient" "V211" #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi 0.3741053789473685
"No Component" "No Cost_category" "No Axis" "4" "CV230" "Gradient" "V211" #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi 0.40836471
"No Component" "No Cost_category" "No Axis" "5" "CV230" "Gradient" "V211" #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi 0.3991534699999998
"No Component" "No Cost_category" "No Axis" "6" "CV230" "Gradient" "V211" #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi 0.3684493599999995
"No Component" "No Cost_category" "No Axis" "7" "CV230" "Gradient" "V211" #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi 0.7852075900000013
"No Component" "No Cost_category" "No Axis" "8" "CV230" "Gradient" "V211" #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi 0.6765691699999999
"No Component" "No Cost_category" "No Axis" "9" "CV230" "Gradient" "V211" #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #
#Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi 0.5128884800000005
"No Component" "No Cost_category" "No Axis" "3" "cv270" "Gradient" "V211" #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi 0.51288848
"No Component" "No Cost_category" "No Axis" "4" "cv270" "Gradient" "V211" #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi 0.5128884699999993
"No Component" "No Cost_category" "No Axis" "5" "cv270" "Gradient" "V211" #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi #Mi 0.5128884799999998
"No Component" "No Cost_category" "No Axis" "6" "cv270" "Gradient" "V211" #Mi #Mi #Mi #Mi
-- create table:
SCOTT@orcl_11gR2> CREATE OR REPLACE DIRECTORY my_dir AS 'c:\my_oracle_files'
2 /
Directory created.
SCOTT@orcl_11gR2> CREATE TABLE external_table
2 (
3 "COMPONENT" VARCHAR2(79),
4 "COST_CATEGORY" VARCHAR2(79),
5 "AXIS" VARCHAR2(79),
6 "COORDINATE" VARCHAR2(79),
7 "CURVE_TYPE" VARCHAR2(79),
8 "MEASURE" VARCHAR2(79),
9 "VERSION" VARCHAR2(79),
10 "FAC" NUMBER(30,10),
11 "LRIC" NUMBER(30,10),
12 "MARGINAL" NUMBER(30,10),
13 "SACS" NUMBER(30,10),
14 "DLRIC" NUMBER(30,10),
15 "DSAC" NUMBER(30,10),
16 "NSOC" NUMBER(30,10),
17 "LRIC_FAC_RATIO" NUMBER(30,10),
18 "DLRIC_FAC_RATIO" NUMBER(30,10),
19 "DSAC_FAC_RATIO" NUMBER(30,10),
20 "FAC_TC" NUMBER(30,10),
21 "LRIC_TC" NUMBER(30,10),
22 "DLRIC_TC" NUMBER(30,10),
23 "DSAC_TC" NUMBER(30,10),
24 "NSOC_TC" NUMBER(30,10),
25 "NSOC_TC_2" NUMBER(30,10),
26 "NO_COST_CONVENTION" NUMBER(30,10),
27 "COST_CONVENTION" NUMBER(30,10)
28 )
29 ORGANIZATION external
30 (
31 TYPE oracle_loader
32 DEFAULT DIRECTORY MY_DIR
33 ACCESS PARAMETERS
34 (
35 RECORDS DELIMITED BY NEWLINE
36 BADFILE 'MY_DIR':'flat_file.bad'
37 LOGFILE 'test.log'
38 SKIP 1
39 FIELDS TERMINATED BY WHITESPACE OPTIONALLY ENCLOSED BY '"'
40 MISSING FIELD VALUES ARE NULL
41 REJECT ROWS WITH ALL NULL FIELDS
42 (
43 "COMPONENT" NULLIF ("COMPONENT" = "#Mi"),
44 "COST_CATEGORY" NULLIF ("COST_CATEGORY" = "#Mi"),
45 "AXIS" NULLIF ("AXIS" = "#Mi"),
46 "COORDINATE" NULLIF ("COORDINATE" = "#Mi"),
47 "CURVE_TYPE" NULLIF ("CURVE_TYPE" = "#Mi"),
48 "MEASURE" NULLIF ("MEASURE" = "#Mi"),
49 "VERSION" NULLIF ("MEASURE" = "#Mi"),
50 "FAC" NULLIF ("FAC" = "#Mi"),
51 "LRIC" NULLIF ("LRIC" = "#Mi"),
52 "MARGINAL" NULLIF ("MARGINAL" = "#Mi"),
53 "SACS" NULLIF ("SACS" = "#Mi"),
54 "DLRIC" NULLIF ("DLRIC" = "#Mi"),
55 "DSAC" NULLIF ("DSAC" = "#Mi"),
56 "NSOC" NULLIF ("NSOC" = "#Mi"),
57 "LRIC_FAC_RATIO" NULLIF ("LRIC_FAC_RATIO" = "#Mi"),
58 "DLRIC_FAC_RATIO" NULLIF ("DLRIC_FAC_RATIO" = "#Mi"),
59 "DSAC_FAC_RATIO" NULLIF ("DSAC_FAC_RATIO" = "#Mi"),
60 "FAC_TC" NULLIF ("FAC_TC" = "#Mi"),
61 "LRIC_TC" NULLIF ("LRIC_TC" = "#Mi"),
62 "DLRIC_TC" NULLIF ("DLRIC_TC" = "#Mi"),
63 "DSAC_TC" NULLIF ("DSAC_TC" = "#Mi"),
64 "NSOC_TC" NULLIF ("NSOC_TC" = "#Mi"),
65 "NSOC_TC_2" NULLIF ("NSOC_TC_2" = "#Mi"),
66 "NO_COST_CONVENTION" NULLIF ("NO_COST_CONVENTION" = "#Mi"),
67 "COST_CONVENTION" NULLIF ("COST_CONVENTION" = "#Mi")
68 )
69 )
70 location ('flat_file.dat')
71 )REJECT LIMIT UNLIMITED
72 /
Table created.
-- results:
SCOTT@orcl_11gR2> select * from external_table
2 /
COMPONENT
-------------------------------------------------------------------------------
COST_CATEGORY
-------------------------------------------------------------------------------
AXIS
-------------------------------------------------------------------------------
COORDINATE
-------------------------------------------------------------------------------
CURVE_TYPE
-------------------------------------------------------------------------------
MEASURE
-------------------------------------------------------------------------------
VERSION
-------------------------------------------------------------------------------
FAC LRIC MARGINAL SACS DLRIC DSAC NSOC
---------- ---------- ---------- ---------- ---------- ---------- ----------
LRIC_FAC_RATIO DLRIC_FAC_RATIO DSAC_FAC_RATIO FAC_TC LRIC_TC DLRIC_TC
-------------- --------------- -------------- ---------- ---------- ----------
DSAC_TC NSOC_TC NSOC_TC_2 NO_COST_CONVENTION COST_CONVENTION
---------- ---------- ---------- ------------------ ---------------
ICO371
CECADRZZZZG2ZZZZ
No Axis
No co_ordinate
No Curve_type
Component_fac
V211
53009
ICO372
CECADRZZZZG2ZZZZ
No Axis
No co_ordinate
No Curve_type
Component_fac
V211
10702
ICO373
CECADRZZZZG2ZZZZ
No Axis
No co_ordinate
No Curve_type
Component_fac
V211
6300
ICO551
CECADRZZZZG2ZZZZ
No Axis
No co_ordinate
No Curve_type
Component_fac
V211
908
ICO374
CECADRZZZZG2ZZZZ
No Axis
No co_ordinate
No Curve_type
Component_fac
V211
3436
ICO375
CECADRZZZZG2ZZZZ
No Axis
No co_ordinate
No Curve_type
Component_fac
V211
4892
ICO554
CECADRZZZZG2ZZZZ
No Axis
No co_ordinate
No Curve_type
Component_fac
V211
465
#Mi
0
No Component
No Cost_category
No Axis
3
CV230
Gradient
V211
.374105379
No Component
No Cost_category
No Axis
4
CV230
Gradient
V211
.40836471
No Component
No Cost_category
No Axis
5
CV230
Gradient
V211
.39915347
No Component
No Cost_category
No Axis
6
CV230
Gradient
V211
.36844936
No Component
No Cost_category
No Axis
7
CV230
Gradient
V211
.78520759
No Component
No Cost_category
No Axis
8
CV230
Gradient
V211
.67656917
#Mi
.51288848
No Component
No Cost_category
No Axis
3
cv270
Gradient
V211
.51288848
No Component
No Cost_category
No Axis
4
cv270
Gradient
V211
.51288847
No Component
No Cost_category
No Axis
5
cv270
Gradient
V211
.51288848
No Component
No Cost_category
No Axis
6
cv270
Gradient
V211
19 rows selected.
SCOTT@orcl_11gR2>
|
|
|
|
|
Goto Forum:
Current Time: Sat Aug 09 09:15:30 CDT 2025
|