Message-Id: <10730.125645@fatcity.com> From: "Mohan, Ross" Date: Wed, 3 Jan 2001 17:18:44 -0500 Subject: RE: UNIX: remove duplicate lines This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C075D3.22934540 Content-Type: text/plain; charset="iso-8859-1" "HP" - A PERL script would do it in about 3 lines ( excluding wrappers, etc. ) Of course, TIMTOWTDI. (ask Jared!) Hell, if it's under several hundred thousand lines, load it via sqlldr into a table, and declare the text as a primary key, enable the key, stash the violations by ROWID into another table, delete as necessary, and spool it back out to a file. - Rube Goldberg -----Original Message----- From: hp [mailto:hp@lechateau.ca] Sent: Wednesday, January 03, 2001 4:56 PM To: Multiple recipients of list ORACLE-L Subject: UNIX: remove duplicate lines does anyone know of a way i can remove duplicate lines from a file. I can use the sort -u command but is there any other way to do it without sorting the file? -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: hp INET: hp@lechateau.ca Fat City Network Services -- (858) 538-5051 FAX: (858) 538-5051 San Diego, California -- Public Internet access / Mailing Lists -------------------------------------------------------------------- To REMOVE yourself from this mailing list, send an E-Mail message to: ListGuru@fatcity.com (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). ------_=_NextPart_001_01C075D3.22934540 Content-Type: text/html; charset="iso-8859-1" RE: UNIX: remove duplicate lines

"HP" -


A PERL script would do it in about 3 lines ( excluding wrappers, etc. )
Of course, TIMTOWTDI. (ask Jared!)

Hell, if it's under several hundred thousand lines, load it via
sqlldr into a table, and declare the text as a primary key, enable
the key, stash the violations by ROWID into another table, delete
as necessary, and spool it back out to a file.

- Rube Goldberg

-----Original Message-----
From: hp [mailto:hp@lechateau.ca]
Sent: Wednesday, January 03, 2001 4:56 PM
To: Multiple recipients of list ORACLE-L
Subject: UNIX: remove duplicate lines


does anyone know of a way i can remove duplicate lines from a file.

I can use the sort -u command but is there any other way to do it without
sorting the file?
--
Please see the official ORACLE-L FAQ: http://www.orafaq.com
--
Author: hp
  INET: hp@lechateau.ca

Fat City Network Services    -- (858) 538-5051  FAX: (858) 538-5051
San Diego, California        -- Public Internet access / Mailing Lists
--------------------------------------------------------------------
To REMOVE yourself from this mailing list, send an E-Mail message
to: ListGuru@fatcity.com (note EXACT spelling of 'ListGuru') and in
the message BODY, include a line containing: UNSUB ORACLE-L
(or the name of mailing list you want to be removed from).  You may
also send the HELP command for other information (like subscribing).