Home » SQL & PL/SQL » SQL & PL/SQL » Create Large .CSV and send it as attachment (4 Merged) (Oracle DB 11g (11.2.0.2.0 ))
Create Large .CSV and send it as attachment (4 Merged) [message #558490] Fri, 22 June 2012 22:42 Go to next message
dev88
Messages: 15
Registered: September 2008
Location: SAN JOSE
Junior Member
Hi Gurus,

Need a help. Can i send a large file( over 1 million) as email attachment as .CSV file...?

I have a requirement to send a large file ".CSV" when the procedure / package is invoked
as email attachment. The data in the CSV file is pulled from a table (as below).

(1) I tried below code to execute "send_email" which is uses utility UTL_SMTP. It is working fine with
100000 records and getting an email attachment with .csv
(2) If more 100000 i am not getting any email / attachment.

I am looking forward to send a huge data like 1 million.

Please share your experience if you had come across.

Thanks in advance.

Dev


>>>>>>>>>>>>>>>>>>>>>
DECLARE
l_clob CLOB;
l_attach_text VARCHAR2 (32767);
l_attach_text_h VARCHAR2 (32767);

cursor c1 is
SELECT LOCATION,PARTY_NAME,ADDRESS1,CITY,STATE_PROV,COUNTRY,POSTAL_CODE FROM emp_table;
BEGIN

l_attach_text_h :=
'LOCATION,PARTY,ADDRESS,CITY,STATE,COUNTRY,POSTAL_CODE';

FOR employee_rec in c1
LOOP
DBMS_OUTPUT.put_line('Before loop COUNT Boss ...'||c1%ROWCOUNT);

l_attach_text :=
employee_rec.LOCATION||','||
employee_rec.PARTY_NAME||','||
employee_rec.ADDRESS1||','||
employee_rec.CITY||','||
employee_rec.STATE_PROV||','||
employee_rec.COUNTRY||','||
employee_rec.POSTAL_CODE||chr(13);

l_clob := l_clob|| l_attach_text;


l_clob := l_attach_text_h ||chr(13) || l_clob;

send_mail(p_to => 'xx@sendmail.com',
p_from => 'xx@sendmail.com',
p_subject => 'Test Message',
p_text_msg => 'This is a test message.',
p_attach_name => 'test.csv',
p_attach_mime => 'text/plain',
p_attach_clob => l_clob,
p_smtp_host => 'maildude.com');
END;
/
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Re: Create Large .CSV and send it as attachement [message #558491 is a reply to message #558490] Fri, 22 June 2012 22:46 Go to previous messageGo to next message
BlackSwan
Messages: 26766
Registered: January 2009
Location: SoCal
Senior Member
Please read and follow the forum guidelines, to enable us to help you:

http://www.orafaq.com/forum/t/88153/0/

UTL_MAIL is designed to handle attachments & I am unaware of any size limitations for the attachments.
Large CSV file as email attachement [message #558495 is a reply to message #558490] Fri, 22 June 2012 22:55 Go to previous messageGo to next message
dev88
Messages: 15
Registered: September 2008
Location: SAN JOSE
Junior Member
Hi Gurus,

Need a help. Can i send a large file( over 1 million) as email attachment as .CSV file.

I have a requirement to send a large file ".CSV" when the procedure / package is invoked
as email attachment. The data in the CSV file is pulled from a table.

(1) I tried below code to execute "send_email" which is used utility UTL_SMTP. It is working fine with
100000 records and getting an email attachment with .csv
(2) If more 100000 i am not getting any email / attachment.

I am looking forward to send a huge data like 1 million.

Please share your experience if you had come across.

Thanks in advance.

Dev


>>>>>>>>>>>>>>>>>>>>>
DECLARE
l_clob CLOB;
l_attach_text VARCHAR2 (32767);
l_attach_text_h VARCHAR2 (32767);

cursor c1 is
SELECT LOCATION,PARTY_NAME,ADDRESS1,CITY,STATE_PROV,COUNTRY,POSTAL_CODE FROM emp_table;
BEGIN

l_attach_text_h :=
'LOCATION,PARTY,ADDRESS,CITY,STATE,COUNTRY,POSTAL_CODE';

FOR employee_rec in c1
LOOP
DBMS_OUTPUT.put_line('Before loop COUNT Boss ...'||c1%ROWCOUNT);

l_attach_text :=
employee_rec.LOCATION||','||
employee_rec.PARTY_NAME||','||
employee_rec.ADDRESS1||','||
employee_rec.CITY||','||
employee_rec.STATE_PROV||','||
employee_rec.COUNTRY||','||
employee_rec.POSTAL_CODE||chr(13);

l_clob := l_clob|| l_attach_text;


l_clob := l_attach_text_h ||chr(13) || l_clob;

send_mail(p_to => 'xx@sendmail.com',
p_from => 'xx@sendmail.com',
p_subject => 'Test Message',
p_text_msg => 'This is a test message.',
p_attach_name => 'test.csv',
p_attach_mime => 'text/plain',
p_attach_clob => l_clob,
p_smtp_host => 'maildude.com');
END;
/
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Re: Large CSV file as email attachement [message #558500 is a reply to message #558495] Fri, 22 June 2012 23:58 Go to previous messageGo to next message
Michel Cadot
Messages: 68767
Registered: March 2007
Location: Saint-Maur, France, https...
Senior Member
Account Moderator
From your previous topics:

BlackSwan wrote on Tue, 25 October 2011 20:54
It would be helpful if you followed Posting Guidelines - http://www.orafaq.com/forum/t/88153/0/


Michel Cadot wrote on Wed, 26 October 2011 07:59
...
Use SQL*Plus and copy and paste your session, the WHOLE session.

Before, Please read OraFAQ Forum Guide and How to use [code] tags and make your code easier to read.
Make sure that lines of code do not exceed 80 characters when you format.
Indent the code, use code tags and align the columns in result.
Use the "Preview Message" button to verify.
Also always post your Oracle version, with 4 decimals.
...


BlackSwan wrote on Wed, 12 October 2011 21:21
It would be helpful if you followed Posting Guidelines - http://www.orafaq.com/forum/t/88153/0/


BlackSwan wrote on Sat, 23 June 2012 06:02
Please read and follow the forum guidelines, to enable us to help you:

http://www.orafaq.com/forum/t/88153/0/
...



[Updated on: Fri, 22 June 2012 23:58]

Report message to a moderator

Re: Large CSV file as email attachement [message #558509 is a reply to message #558495] Sat, 23 June 2012 02:40 Go to previous messageGo to next message
John Watson
Messages: 8988
Registered: January 2010
Location: Global Village
Senior Member
Are you sending the mail within the loop, one mail per line of data? It is hard to see because you haven't formatted it correctly.
But apart from that:
The limit could be within your send_mail procedure, if you provide the code (formatted correctly) perhaps someone will look at it.
You could check whether your mail transfer agent has a size limit.
How will the recipient handle data that includes commas in any of the values?
Re: Large CSV file as email attachement [message #558516 is a reply to message #558509] Sat, 23 June 2012 05:20 Go to previous messageGo to next message
ThomasG
Messages: 3212
Registered: April 2005
Location: Heilbronn, Germany
Senior Member
"I am looking forward to send a huge data like 1 million."

Be ready for mail server administrators with scythes and pitchforks. Most mail servers I see in business environments either have a 2 or 10 MB size limit.
Re: Large CSV file as email attachement [message #558539 is a reply to message #558516] Sat, 23 June 2012 11:47 Go to previous messageGo to next message
matthewmorris68
Messages: 258
Registered: May 2012
Location: Orlando, FL
Senior Member

As ThomasG said, your email server likely has an attachment size limit. I encountered this on a project at the company I currently work for. I had code that worked for a couple of years and then suddenly emails stopped being delivered for one report (but worked for all of my others). Took a while to realize that the particular report being sent had just crossed over the 10MB mark. I add a clause to my program that sent the emails to check the size of the CLOB. For CLOBs over 10MB, rather than emailing it as an attachment, I save it as a file on a shared network drive and the email will contain a note to that effect and the location/name of the file.
Re: Large CSV file as email attachement [message #558689 is a reply to message #558539] Mon, 25 June 2012 16:55 Go to previous messageGo to next message
dev88
Messages: 15
Registered: September 2008
Location: SAN JOSE
Junior Member
Thank you! for all your reply's. It helped me lot to convince the team its hard to achive.

matthewmorris68: Can you pl let me know how to achieve the approach you were mentioning like saving as a large file and send a link to access the file as email. I will try this option

Please reply.

Thanks,
Dev
Re: Large CSV file as email attachement [message #558690 is a reply to message #558689] Mon, 25 June 2012 18:28 Go to previous messageGo to next message
matthewmorris68
Messages: 258
Registered: May 2012
Location: Orlando, FL
Senior Member

The check code is essentially the first block below. l_xml_clob is the CLOB where my report is stored while being built. If it is over 10MB (actually I could add another ~5000 bytes, but I left some leeway for the size of the email/etc.), then I save it as a file through my write_file function (second block) and set the body of my email to notify the recipient. You'll need to set up a directory object pointing to a shared location where your recipients can access the file after it is saved.

  v_size := DBMS_LOB.GETLENGTH(l_xml_clob);

  IF v_size < 10480000 THEN
    send_email
        ( v_From,
          p_recipient,
          v_Subject,
          v_body,
          l_xml_clob,
          v_filename);
  ELSE
    v_body := 'The ' || v_report || ' report produced an attachment > 10MB.' ||
       'It has been saved as a file to [location]' || v_filename;

    write_file(v_filename, l_xml_clob);

    send_email
        ( v_From,
          p_recipient,
          v_Subject,
          v_body);
  END IF;




PROCEDURE write_file(p_filename   IN VARCHAR2,
                     p_clob       IN CLOB,
                     p_directory  IN VARCHAR2) 
IS
    
  c_amount CONSTANT BINARY_INTEGER := 32767;
  l_buffer   VARCHAR2(32767);
  l_chr10    PLS_INTEGER;
  l_cloblen  PLS_INTEGER;
  l_fhandler utl_file.file_type;
  l_pos      PLS_INTEGER := 1;
    
BEGIN
  
  l_cloblen  := dbms_lob.getlength(p_clob);
  l_fhandler := utl_file.fopen(p_directory, p_filename, 'W', c_amount);
    
  WHILE l_pos < l_cloblen LOOP
    l_buffer := dbms_lob.substr(p_clob, c_amount, l_pos);
    EXIT WHEN l_buffer IS NULL;
    l_chr10 := INSTR(l_buffer, chr(10), -1);

    IF l_chr10 != 0 THEN
      l_buffer := SUBSTR(l_buffer, 1, l_chr10 - 1);
    END IF;

    utl_file.put_line(l_fhandler, l_buffer, TRUE);
    l_pos := l_pos + LEAST(LENGTH(l_buffer) + 1, c_amount);
  END LOOP;
    
  utl_file.fclose(l_fhandler);
    
EXCEPTION
  WHEN OTHERS THEN
    IF utl_file.is_open(l_fhandler) THEN
      utl_file.fclose(l_fhandler);
    END IF;
END write_file;

Re: Large CSV file as email attachement [message #558844 is a reply to message #558690] Tue, 26 June 2012 20:18 Go to previous messageGo to next message
dev88
Messages: 15
Registered: September 2008
Location: SAN JOSE
Junior Member
matthewmorris68 : Thank you, For sharing the code.

Hi ALL,
Can any one share how to create ".CSV" file and Zip it to compress its size.
Now I have a requirement to get the data from Oracle table( ~ 2 million records or more, approx 1 GB data ) and create as ".csv" file and compress it. Upload it to a shared folder. I searched in this site and found similar code, But records are stored in CLOB columns are by line by line and i wanted to create as a file.

>>>>>>>>>>>>>>>
>>>>>
CREATE TABLE tab_source_dd (csv_col CLOB)
>>>>>

DECLARE
cursor c1 is
SELECT LOCATION,PARTY_NAME,ADDRESS1,CITY,STATE_PROV,COUNTRY,POSTAL_CODE FROM TABLE_NAME;
BEGIN

FOR employee_rec in c1
LOOP
INSERT INTO tab_source_dd VALUES ( employee_rec.LOCATION||','||employee_rec.PARTY_NAME||','||employee_rec.ADDRESS1||','|| employee_rec.CITY||','|| employee_rec.STATE_PROV||','||employee_rec.COUNTRY);
END LOOP;
commit;
END;

>>>>>>>>>>>>>>>

Please reply.

Thanks,
Dev

Re: Large CSV file as email attachement [message #558845 is a reply to message #558844] Tue, 26 June 2012 20:32 Go to previous messageGo to next message
BlackSwan
Messages: 26766
Registered: January 2009
Location: SoCal
Senior Member
>i wanted to create as a file.
UTL_FILE can write records into an OS file.
Re: Large CSV file as email attachement [message #558846 is a reply to message #558844] Tue, 26 June 2012 21:04 Go to previous message
matthewmorris68
Messages: 258
Registered: May 2012
Location: Orlando, FL
Senior Member

The file I replied with takes a CLOB variable and saves it as a file. If the text in the CLOB is comma-delimited and you save the file with a .CSV extension than you have created a CSV file. Another of my procedures uses the write_file to create CSV files. It was added after my XML reporting function -- which is why my CLOB variable is *called* l_xml_clob. However, the name notwithstanding, it will store CLOB in any text format.

write_file does not compress the resulting file. Offhand, the only way that I could think to do that *from* Oracle would be to create a function that calls an OS executable. I have done that, but it's complex enough that I'm not getting into it in a forum posting... and I'd have to research exactly how I did it in any event. I know part of the solution came from a Java procedure on Tom Kyte's website.

Aside from that, if you're saving to a Unix directory, you could create a cron job to periodically check that directory for .CSV files and if any are found to compress them. That'd be much easier to do but might not meet your requirement.
Previous Topic: Replace String in 9i (3 Merged)
Next Topic: Embedded Ref Cursor
Goto Forum:
  


Current Time: Fri Aug 29 06:04:40 CDT 2025