Home » SQL & PL/SQL » SQL & PL/SQL » Select columns based on first record (Header from flat file) (Oracle 11g, Windows 2012 R2)
Select columns based on first record (Header from flat file) [message #655609] Tue, 06 September 2016 00:08 Go to next message
rahul1982
Messages: 53
Registered: November 2011
Location: Pune
Member
I have an table, where the column sequence is not fix. I want to select only specific column by referring the first record which is a header from flat file.

Below are the sample scripts, I want to select only two columns where first record is name & city.

create table cust
(col1 varchar(10),col2 varchar(10), col3 varchar(10),col4 varchar(10), col5 varchar(10));

insert into cust
values('custno' ,'name' , 'address' ,'city' , 'zip' );
insert into cust
values('1' ,'abc' , '123 street' ,'Houston' , '77004' );
insert into cust
values('2' ,'qwe' , 'test ave' ,'Pune' , '41125' );
insert into cust
values('3' ,'xyz' , '10 floor' ,'New York' , '44547' );
insert into cust
values('4' ,'pqr' , '15 floor' ,'New York' , '44557' );
insert into cust
values('5' ,'hhh' , 'Test' ,'Mumbai' , '77455' );

Thanks!
Re: Select columns based on first record (Header from flat file) [message #655611 is a reply to message #655609] Tue, 06 September 2016 01:14 Go to previous messageGo to next message
Michel Cadot
Messages: 68647
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator

I hope this is a homework, if not then shot the one that designed this.

"First" is meaningless in a relational table as rows are like balls in a basket.
In this very specific case, you can determine the "column names" row as this is the lone one which does not contain a numeric value in "col1".

So the core of the query you are searching is:
SQL> select * from cust where not regexp_like(col1,'^\d+$');
COL1       COL2       COL3       COL4       COL5
---------- ---------- ---------- ---------- ----------
custno     name       address    city       zip
Re: Select columns based on first record (Header from flat file) [message #655612 is a reply to message #655609] Tue, 06 September 2016 02:36 Go to previous messageGo to next message
John Watson
Messages: 8931
Registered: January 2010
Location: Global Village
Senior Member
Further to thisQuote:
I hope this is a homework,
you need to more precise if you want to get good marks. This,
Quote:
I want to select only specific column by referring the first record
is not the way to describe the problem. A relational engineer would say Quote:
I want to PROJECT only specific column by referring the first ROW
Re: Select columns based on first record (Header from flat file) [message #655614 is a reply to message #655609] Tue, 06 September 2016 03:24 Go to previous messageGo to next message
flyboy
Messages: 1903
Registered: November 2006
Senior Member
As far as rahul talks about flat file, I wonder why he wants to use Oracle database for its processing and how the posted table is related to its actual content.

This task looks like a part of some (home-made?) ETL tool, so I would say that any text processing utility (sed/awk/perl or some Windows ones if he uses that system) would serve this purpose - extract text from the position based on the first file row content - better.
Re: Select columns based on first record (Header from flat file) [message #655615 is a reply to message #655614] Tue, 06 September 2016 03:29 Go to previous messageGo to next message
rahul1982
Messages: 53
Registered: November 2011
Location: Pune
Member
thank you everyone for your inputs.
Re: Select columns based on first record (Header from flat file) [message #655638 is a reply to message #655609] Tue, 06 September 2016 18:01 Go to previous messageGo to next message
Barbara Boehmer
Messages: 9090
Registered: November 2002
Location: California, USA
Senior Member
-- test data:
SCOTT@orcl_12.1.0.2.0> select * from cust
  2  /

COL1       COL2       COL3       COL4       COL5
---------- ---------- ---------- ---------- ----------
custno     name       address    city       zip
1          abc        123 street Houston    77004
2          qwe        test ave   Pune       41125
3          xyz        10 floor   New York   44547
4          pqr        15 floor   New York   44557
5          hhh        Test       Mumbai     77455

6 rows selected.

-- extraction of name and city:
SCOTT@orcl_12.1.0.2.0> variable g_name_and_city refcursor
SCOTT@orcl_12.1.0.2.0> declare
  2    v_name_col  varchar2(30);
  3    v_city_col  varchar2(30);
  4  begin
  5    for r in
  6  	 (select column_name
  7  	  from	 user_tab_columns
  8  	  where  table_name = 'CUST')
  9    loop
 10  	 begin
 11  	   execute immediate
 12  	     'select ''' || r.column_name ||
 13  	     ''' from	dual
 14  	      where  exists
 15  		     (select *
 16  		      from   cust
 17  		      where ' || r.column_name || ' = ''name'')'
 18  	     into v_name_col;
 19  	     exit when v_name_col is not null;
 20  	 exception
 21  	   when no_data_found then null;
 22  	 end;
 23    end loop;
 24    for r in
 25  	 (select column_name
 26  	  from	 user_tab_columns
 27  	  where  table_name = 'CUST')
 28    loop
 29  	 begin
 30  	   execute immediate
 31  	     'select ''' || r.column_name ||
 32  	     ''' from	dual
 33  	      where  exists
 34  		     (select *
 35  		      from   cust
 36  		      where ' || r.column_name || ' = ''city'')'
 37  	     into v_city_col;
 38  	     exit when v_city_col is not null;
 39  	 exception
 40  	   when no_data_found then null;
 41  	 end;
 42    end loop;
 43    open :g_name_and_city for
 44  	 'select ' || v_name_col || ' name,' || v_city_col || ' city from cust where '
 45  	 || v_name_col || ' != ''name'' and ' || v_city_col || ' != ''city''';
 46  end;
 47  /

PL/SQL procedure successfully completed.

SCOTT@orcl_12.1.0.2.0> print :g_name_and_city

NAME       CITY
---------- ----------
abc        Houston
qwe        Pune
xyz        New York
pqr        New York
hhh        Mumbai

5 rows selected.
Re: Select columns based on first record (Header from flat file) [message #655644 is a reply to message #655638] Wed, 07 September 2016 00:07 Go to previous messageGo to next message
rahul1982
Messages: 53
Registered: November 2011
Location: Pune
Member
Thank you so much!
Re: Select columns based on first record (Header from flat file) [message #655646 is a reply to message #655644] Wed, 07 September 2016 00:48 Go to previous messageGo to next message
Michel Cadot
Messages: 68647
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator

1/ What did you learn?
2/ What did you try?
3/ You can do it in pure SQL.

Re: Select columns based on first record (Header from flat file) [message #655650 is a reply to message #655646] Wed, 07 September 2016 01:46 Go to previous messageGo to next message
Barbara Boehmer
Messages: 9090
Registered: November 2002
Location: California, USA
Senior Member
Michel Cadot wrote on Tue, 06 September 2016 22:48

...You can do it in pure SQL.


Yes, as shown below.

SCOTT@orcl_12.1.0.2.0> select x.name, x.city
  2  from   (select dbms_xmlgen.getxml
  3  		      ('select '||t1.column_name||' name,'||t2.column_name||' city from cust
  4  			where exists (select * from cust where '||t1.column_name||'=''name'')
  5  			and   exists (select * from cust where '||t2.column_name||'=''city'')') xml_data
  6  	     from   user_tab_columns t1, user_tab_columns t2
  7  	     where  t1.table_name = 'CUST'
  8  	     and    t2.table_name = 'CUST'
  9  	     and    t1.column_name != t2.column_name) t,
 10  	    xmltable
 11  	      ('//ROWSET/ROW'
 12  	       passing xmltype (t.xml_data)
 13  	       columns
 14  		 name  varchar2(10)  path '/ROW/NAME',
 15  		 city  varchar2(10)  path '/ROW/CITY') x
 16  where  t.xml_data is  not null
 17  and    x.name != 'name'
 18  and    x.city != 'city'
 19  /

NAME       CITY
---------- ----------
abc        Houston
qwe        Pune
xyz        New York
pqr        New York
hhh        Mumbai

5 rows selected.
Re: Select columns based on first record (Header from flat file) [message #655651 is a reply to message #655650] Wed, 07 September 2016 01:50 Go to previous messageGo to next message
Michel Cadot
Messages: 68647
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator

Well, dbms_xmlgen is not really SQL. Smile

Re: Select columns based on first record (Header from flat file) [message #655652 is a reply to message #655651] Wed, 07 September 2016 01:51 Go to previous messageGo to next message
Barbara Boehmer
Messages: 9090
Registered: November 2002
Location: California, USA
Senior Member
Michel Cadot wrote on Tue, 06 September 2016 23:50

Well, dbms_xmlgen is not really SQL. :)

Then please post what you consider a pure SQL solution. I am intrigued.

Since the columns can be in any order, as the OP stated, you cannot just select col2 and col4.

I was also considering that there may also be any number of columns with any names.

[Updated on: Wed, 07 September 2016 01:56]

Report message to a moderator

Re: Select columns based on first record (Header from flat file) [message #655655 is a reply to message #655652] Wed, 07 September 2016 01:57 Go to previous messageGo to next message
Michel Cadot
Messages: 68647
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator

I don't understand your last sentences.
A table has a fixed number of columns and if the column names don't match the value positions you can't do nothing.

Re: Select columns based on first record (Header from flat file) [message #655657 is a reply to message #655655] Wed, 07 September 2016 02:04 Go to previous messageGo to next message
Barbara Boehmer
Messages: 9090
Registered: November 2002
Location: California, USA
Senior Member
Michel Cadot wrote on Tue, 06 September 2016 23:57

I don't understand your last sentences.
A table has a fixed number of columns and if the column names don't match the value positions you can't do nothing.

The OP said that the "column sequence is not fix", which I understand to mean that the order of the columns may change with each data set that they receive. So, the columns containing the name and city data and the position of the words name and city in the header row could be in one position in one data set and in another position in another data set. The next time that they receive another data set, the columns could be in different order and position. I have had to deal with something similar in the past, where the source that we obtained the data from kept changing their format from month to month, causing the code to have to be rewritten to load it, unless you write something generic like this. In that situation, sometimes they also added columns or changed column names, as well as shuffling the order, so I allowed for that as well. Is that clearer?


Re: Select columns based on first record (Header from flat file) [message #655658 is a reply to message #655655] Wed, 07 September 2016 02:10 Go to previous messageGo to next message
Michel Cadot
Messages: 68647
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator

SQL> with
  2    names_row as (select * from cust where not regexp_like(col1,'^\d+$')),
  3    names as (
  4      select position,
  5             decode(position, 1,col1, 2,col2, 3,col3, 4,col4) column_name
  6      from names_row,
  7           (select level position from dual connect by level <= 4)
  8    )
  9  select decode(c1.position, 1,col1, 2,col2, 3,col3, 4,col4) name,
 10         decode(c2.position, 1,col1, 2,col2, 3,col3, 4,col4) city
 11  from cust,
 12       (select position from names where column_name='name') c1,
 13       (select position from names where column_name='city') c2
 14  where regexp_like(col1,'^\d+$')
 15  /
NAME       CITY
---------- ----------
abc        Houston
qwe        Pune
xyz        New York
pqr        New York
hhh        Mumbai

5 rows selected.
Re: Select columns based on first record (Header from flat file) [message #655659 is a reply to message #655657] Wed, 07 September 2016 02:13 Go to previous messageGo to next message
Barbara Boehmer
Messages: 9090
Registered: November 2002
Location: California, USA
Senior Member
Michel,

I will try to further clarify with an example. Suppose that the next data set that is loaded from a flat file has the columns in the following order, so that the name data is now in col1 and the city data is now in col3.

SCOTT@orcl_12.1.0.2.0> select * from cust
  2  /

COL1       COL2       COL3       COL4       COL5
---------- ---------- ---------- ---------- ----------
name       address    city       zip        custno
abc        123 street Houston    77004      1
qwe        test ave   Pune       41125      2
xyz        10 floor   New York   44547      3
pqr        15 floor   New York   44557      4
hhh        Test       Mumbai     77455      5

6 rows selected.

Then the following will still be able to select the name and city data based on the words name and city in the header row of the flat file that were loaded into the table. The other method that I posted would also work.
SCOTT@orcl_12.1.0.2.0> select x.name, x.city
  2  from   (select dbms_xmlgen.getxml
  3  		      ('select '||t1.column_name||' name,'||t2.column_name||' city from cust
  4  			where exists (select * from cust where '||t1.column_name||'=''name'')
  5  			and   exists (select * from cust where '||t2.column_name||'=''city'')') xml_data
  6  	     from   user_tab_columns t1, user_tab_columns t2
  7  	     where  t1.table_name = 'CUST'
  8  	     and    t2.table_name = 'CUST'
  9  	     and    t1.column_name != t2.column_name) t,
 10  	    xmltable
 11  	      ('//ROWSET/ROW'
 12  	       passing xmltype (t.xml_data)
 13  	       columns
 14  		 name  varchar2(10)  path '/ROW/NAME',
 15  		 city  varchar2(10)  path '/ROW/CITY') x
 16  where  t.xml_data is  not null
 17  and    x.name != 'name'
 18  and    x.city != 'city'
 19  /

NAME       CITY
---------- ----------
abc        Houston
qwe        Pune
xyz        New York
pqr        New York
hhh        Mumbai

5 rows selected.
Re: Select columns based on first record (Header from flat file) [message #655660 is a reply to message #655657] Wed, 07 September 2016 02:16 Go to previous messageGo to next message
Michel Cadot
Messages: 68647
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator
Barbara Boehmer wrote on Wed, 07 September 2016 09:04
The OP said that the "column sequence is not fix", which I understand to mean that the order of the columns may change with each data set that they receive. So, the columns containing the name and city data and the position of the words name and city in the header row could be in one position in one data set and in another position in another data set. The next time that they receive another data set, the columns could be in different order and position. I have had to deal with something similar in the past, where the source that we obtained the data from kept changing their format from month to month, causing the code to have to be rewritten to load it, unless you write something generic like this. In that situation, sometimes they also added columns or changed column names, as well as shuffling the order, so I allowed for that as well. Is that clearer?
Not sure but the query I gave does not care about the order of the columns but, of course, the number of cust table columns is fixed.
If it can change just change the query to match with it as you'd do it for any application code when you change the definition of a table.
I think the problem is like loading a table with SQL*Loader, if you change the table definition you have to change the control file.

Re: Select columns based on first record (Header from flat file) [message #655661 is a reply to message #655659] Wed, 07 September 2016 02:19 Go to previous messageGo to next message
Michel Cadot
Messages: 68647
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator

Quote:
I will try to further clarify with an example. Suppose that the next data set that is loaded from a flat file has the columns in the following order, so that the name data is now in col1 and the city data is now in col3.
My query handles this case (as long as the number of columns does not change or the selected columns are still in the initial number of columns).

[Updated on: Wed, 07 September 2016 02:21]

Report message to a moderator

Re: Select columns based on first record (Header from flat file) [message #655662 is a reply to message #655660] Wed, 07 September 2016 02:22 Go to previous messageGo to next message
Barbara Boehmer
Messages: 9090
Registered: November 2002
Location: California, USA
Senior Member
Michel Cadot wrote on Wed, 07 September 2016 00:16

... the query I gave does not care about the order of the columns...

Apparently, your query does care about the order of the columns, because when the order of the columns is changed, as shown below, your query returns no rows.

SCOTT@orcl_12.1.0.2.0> select * from cust
  2  /

COL1       COL2       COL3       COL4       COL5
---------- ---------- ---------- ---------- ----------
name       address    city       zip        custno
abc        123 street Houston    77004      1
qwe        test ave   Pune       41125      2
xyz        10 floor   New York   44547      3
pqr        15 floor   New York   44557      4
hhh        Test       Mumbai     77455      5

6 rows selected.

SCOTT@orcl_12.1.0.2.0> with
  2    names_row as (select * from cust where not regexp_like(col1,'^\d+$')),
  3    names as (
  4  	 select position,
  5  		decode(position, 1,col1, 2,col2, 3,col3, 4,col4) column_name
  6  	 from names_row,
  7  	      (select level position from dual connect by level <= 4)
  8    )
  9  select decode(c1.position, 1,col1, 2,col2, 3,col3, 4,col4) name,
 10  	    decode(c2.position, 1,col1, 2,col2, 3,col3, 4,col4) city
 11  from cust,
 12  	  (select position from names where column_name='name') c1,
 13  	  (select position from names where column_name='city') c2
 14  where regexp_like(col1,'^\d+$')
 15  /

no rows selected
Re: Select columns based on first record (Header from flat file) [message #655664 is a reply to message #655662] Wed, 07 September 2016 02:28 Go to previous messageGo to next message
Michel Cadot
Messages: 68647
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator
It does not work in this case because of the prerequisite I mentioned in my first post:

Quote:
"First" is meaningless in a relational table as rows are like balls in a basket.
In this very specific case, you can determine the "column names" row as this is the lone one which does not contain a numeric value in "col1".

You have to define a condition that is met by the row containing the column names and can't be fit by any other row.
Your query does not work either if someone has 'name' for city for instance.

[Updated on: Wed, 07 September 2016 02:28]

Report message to a moderator

Re: Select columns based on first record (Header from flat file) [message #655668 is a reply to message #655664] Wed, 07 September 2016 02:42 Go to previous messageGo to next message
Barbara Boehmer
Messages: 9090
Registered: November 2002
Location: California, USA
Senior Member
Michel Cadot wrote on Wed, 07 September 2016 00:28
It does not work in this case because of the prerequisite I mentioned in my first post:

Quote:
"First" is meaningless in a relational table as rows are like balls in a basket.
In this very specific case, you can determine the "column names" row as this is the lone one which does not contain a numeric value in "col1".
You have to define a condition that is met by the row containing the column names and can't be fit by any other row.
Your query does not work either if someone has 'name' for city for instance.

So, your query does not meet the OP's requirements.

My query will fail if there is a value of "name" or "city" in some row other than the header row. Since this appears to be name and address data, I think that is unlikely. Although there might be something like "City of New York" or "New York City", I doubt there would be just "city" or just "name".

If, when loading the flat file, using either SQL*Loader or an external table, a sequence could be added, then that could be used to determine the header row, which would make things easier.



[Updated on: Wed, 07 September 2016 02:43]

Report message to a moderator

Re: Select columns based on first record (Header from flat file) [message #655670 is a reply to message #655668] Wed, 07 September 2016 03:25 Go to previous message
Michel Cadot
Messages: 68647
Registered: March 2007
Location: Nanterre, France, http://...
Senior Member
Account Moderator

Quote:
So, your query does not meet the OP's requirements.
If everything is free there is no solution.
Note: if it is sure there is a numeric ID in a field, you can check all columns and not just the first one as in my query.

Quote:
My query will fail if there is a value of "name" or "city" in some row other than the header row.
So, your query does not meet the OP's requirements. Smile

Quote:
If, when loading the flat file, using either SQL*Loader or an external table, a sequence could be added, then that could be used to determine the header row, which would make things easier.
Sure, there is an order in a file there is not in a table.

The prime condition for the existence of a solution to OP's problem is:

Quote:
You have to define a condition that is met by the row containing the column names and can't be fit by any other row.
If this requirement is not satisfied, there are no solutions.

Previous Topic: Unpivot or Connect by
Next Topic: Format result set of a union
Goto Forum:
  


Current Time: Fri Apr 26 21:22:19 CDT 2024