Return-Path: X-Original-To: oracle-l@orafaq.com Delivered-To: oracle-l@orafaq.com Received: from turing.freelists.org (turing.freelists.org [206.53.239.180]) by malta2546.startdedicated.com (Postfix) with ESMTPS id 6701D100339469 for ; Fri, 28 Aug 2020 16:49:01 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id 1866327FF2; Fri, 28 Aug 2020 10:49:00 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=freelists.org; s=turing; t=1598626140; bh=cvknHi4BTmgWifDsASGB3OA0OSSgA2/HRgVm+o0wSO0=; h=From:Sender:Sender:From; b=SqvXaDIfnzsDfmaR5yIE9bsw9MnWtG9coRjE/+ejJiuizUqTzLDWLjnqj4Q/QuJhA oVc5XTnvDpsPlvUgMKGX1HEeRYtGbvWZ3DreSzxe/3otDi7BgQFBqZDjg+L9WSUgNU HqTlB6qxCyYNhiChATIdVcSYy+OBcA823Ex0wFLc= X-Virus-Scanned: Debian amavisd-new at turing.freelists.org Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 078H64Q0cVjr; Fri, 28 Aug 2020 10:48:59 -0400 (EDT) Received: from turing.freelists.org (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id B09A027FFC; Fri, 28 Aug 2020 10:48:05 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=freelists.org; s=turing; t=1598626134; bh=cvknHi4BTmgWifDsASGB3OA0OSSgA2/HRgVm+o0wSO0=; h=From:Sender:Sender:From; b=Q3JRHIiy4A6BiDrZVv3NBHQvPux5hS8ECf43+6FlDAg5c0MelMKKPoz1cT1P36s3/ CedpE576p5YrmXF13ASxSqnkKFfhqcn50zMWwHAEZARBHAW3LNKUEhhMp43kfA9OIV AFaqb5QCbpBRZ4i4dhSSeZYRvAc5Zw75SjCckS9E= Received: with ECARTIS (v1.0.0; list oracle-l); Fri, 28 Aug 2020 10:47:18 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id D1C972630C for ; Fri, 28 Aug 2020 10:47:17 -0400 (EDT) Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id C8qHZkRZn6pv for ; Fri, 28 Aug 2020 10:47:17 -0400 (EDT) Received: from mx1.tidalhosting.net (mx1.tidalhosting.net [155.130.138.9]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id 1CEF026356 for ; Fri, 28 Aug 2020 10:47:17 -0400 (EDT) Received: from mwf4500 (c-73-238-99-41.hsd1.nh.comcast.net [73.238.99.41]) by mx1.tidalhosting.net with SMTP (version=Tls cipher=Aes256 bits=256); Fri, 28 Aug 2020 10:47:00 -0400 From: "Mark W. Farnham" To: , , "'Tim Gorman'" References: <625546540.6286223.1598412528817.ref@mail.yahoo.com> <625546540.6286223.1598412528817@mail.yahoo.com> <1613067749.6729763.1598479894734@mail.yahoo.com> <2067790113.7736.1598489004999@mail.yahoo.com> <31132203-7dc2-0480-b770-6eb5d054e872@bluewin.ch> <98f95d81-cd31-bc52-8d5b-b553483e0d47@gmail.com> <1691487233.550567.1598581275725@mail.yahoo.com> In-Reply-To: Subject: RE: Big Update/DML Date: Fri, 28 Aug 2020 10:46:51 -0400 Message-ID: <0bfb01d67d4a$11595f50$340c1df0$@rsiz.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0BFC_01D67D28.8A5F8D10" Content-Language: en-us X-Exim-Id: 0bfb01d67d4a$11595f50$340c1df0$ X-archive-position: 77612 X-ecartis-version: Ecartis v1.0.0 Sender: oracle-l-bounce@freelists.org Errors-to: oracle-l-bounce@freelists.org X-original-sender: mwf@rsiz.com Precedence: normal Reply-To: mwf@rsiz.com List-Help: List-Unsubscribe: List-software: Ecartis version 1.0.0 List-Id: oracle-l X-List-ID: oracle-l List-Subscribe: List-Owner: List-post: List-Archive: X-list: oracle-l ------=_NextPart_000_0BFC_01D67D28.8A5F8D10 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Humor me: Run it forced local into a newly created tablespace that is = ONLY the destination of the CTAS and has no other competing dictionary = and/or space management. =20 From: Mark W. Farnham [mailto:mwf@rsiz.com]=20 Sent: Friday, August 28, 2020 10:45 AM To: 'smishra_97@yahoo.com'; 'oracle-l@freelists.org'; 'Tim Gorman' Subject: RE: Big Update/DML =20 =E2=80=9CContention was only seen for Interconnect or some gc = wait=E2=80=9D =20 Humor me: Run it forced local.=20 =20 From: oracle-l-bounce@freelists.org = [mailto:oracle-l-bounce@freelists.org] On Behalf Of Sanjay Mishra = (Redacted sender "smishra_97" for DMARC) Sent: Thursday, August 27, 2020 10:21 PM To: oracle-l@freelists.org; Tim Gorman Subject: Re: Big Update/DML =20 Tim =20 Thanks for the update. Based on the Sqlmonitor data shared earlier , it = was using Parallel for DDL operation but I will give another try again = as working to load another table in Nocompress as existing one is = Compress for OLTP. Tablespace was given 1.5T space as it was initially = extending it heavily and that helped to increase the throughput. = Contention was only seen for Interconnect or some gc wait and CPU = Contention or any high IO wait was also not seen in the AWR report.=20 =20 Do we think Advance compression can be major bottleneck as I earlier = thought that it might help in utilizing buffer space but doesn't look = like the case.=20 =20 Will share details soon =20 Tx Sanjay =20 On Thursday, August 27, 2020, 05:08:41 PM EDT, Tim Gorman = wrote:=20 =20 =20 Agreed! Pardon my unwarranted guess, but perhaps parallel DDL (which is = enabled by default) was not enabled in the session for some reason = before starting CTAS? To verify, consider running the following query = after ALTER SESSION ENABLE PARALLEL DDL and before CTAS? select pddl_status from v$session where sid in (select distinct sid from = v$mystat); ...or better yet, check PDDL_STATUS from another session? Another possible issue is high-water mark management. If the datafiles = in the tablespace in which you're creating the new table in the CTAS = statement has to be autoextended to accommodate the volume of data = you're loading, then things might take longer. If the INCREMENT_BY = (i.e. NEXT) for the datafile(s) is really small, then datafile = autoextension might take a *REALLY* long time; look for waits on "enq: = HV - contention" or "enq: HW - contention"? At any rate, the solution = is to either make INCREMENT_BY really large, or just resize the datafile = to accommodate what you're building. There are other possible issues to be considered. The key is to monitor = the operation and use the information that the Oracle RDBMS provides to = understand why you're bottlenecked and resolve it. If you need suggestions for monitoring specific actions, please ask? It = is frustrating when someone announces "it doesn't work" with no attempt = to understand why. On 8/27/2020 3:41 AM, Lothar Flatz wrote: Hi, with regards to CTAS it is very hard to believe it takes that long. I am = pretty sure that there is something wrong. A sql monitor would be extremly helpfull. Regards Lothar Am 27.08.2020 um 02:43 schrieb Sanjay Mishra (Redacted sender smishra_97 = for DMARC): Sayan =20 Update statement is=20 =20 Update snows.stamp_detail set set = stamp_process_calc=3Dprocessed_calc_amt; =20 Tried to use 1. Parallel DML with 100 --> Taking 20+hrs 2. CTAS was tried using half a billion as well as 1 billion rows with = parallel 50, 75,100 - Almost same result 3. CTAS with nologging using same step 2 but still not much improvement =20 We have 5-10 such big table and so running each with this much = time-frame need high downtime Tx Sanjay =20 On Wednesday, August 26, 2020, 08:23:18 PM EDT, Sayan Malakshinov = wrote:=20 =20 =20 Hi Sanjay,=20 =20 It would be better if you provide more details about your update. Exact = update statement would be helpful. is this column nullable/not null? =20 =20 On Thu, Aug 27, 2020 at 1:12 AM Sanjay Mishra = wrote: Andy =20 Yes look like is an option if we are doing work online and despite take = more time but need not require downtime. In our case multiple DDL are = running to existing environment due to Application upgrade and so all = work has to be done with downtime. So challenge is reduce time of DML = operations on big tables containing few billions rows. =20 Tx Sanjay =20 On Wednesday, August 26, 2020, 11:20:55 AM EDT, Andy Sayer = wrote:=20 =20 =20 It does sound like a virtual column could be the ideal solution. But if = data needs to be physically stored or cannot be calculated = deterministically at any point in time then Connor has a great demo of = using dbms_redefinition to create a new table online with a function to = map the new column. There=E2=80=99s obviously some overhead with context = switching but it may be far better than some of the obstacles you might = be facing at the moment: =20 https://connor-mcdonald.com/2016/11/16/performing-a-large-correlated-upda= te/ (and you might be able to help it with pragma udf in the right = circumstances). =20 Obviously, how helpful this is depends where the work is currently going = and how online this needs to be.=20 =20 =20 Thanks, Andrew =20 On Wed, 26 Aug 2020 at 16:00, Jonathan Lewis = wrote: =20 Is that 3-4 billion rows each, or total ? =20 I would be a little suspicious of an update which populates a new column = with a value derived from existing columns. What options might you have = for declaring a virtual column instead - which you could index if = needed. =20 Be extremely cautious about calculating space requirements - if you're = updating every row on old data might you find that you're causing a = significant fraction of the rows in each block to migrate, and there's a = peculiarity of bulk row migration that can effectively "waste" 25% of = the space in every block that becomes the target of a migrated row.=20 =20 This effects can be MUCH work when the table is compress (even for OLTP) = since the update has to decompress the row before updating and then only = "re-compresses" intermittently as the block becomes full. The CPU cost = can be horrendous and you still have the problem of migration if the = addition means the original rows can no longer fit in the block. =20 If it is necessary to add the column you may want to review "alter table = move online" can do in the latest versions (in case you can make it add = the column as you move) or review the options for dbms_redefinition - = maybe running several redefinitions concurrently rather than trying to = do any parallel update to any single table. =20 Regards Jonathan Lewis =20 =20 =20 =20 =20 =20 --=20 Best regards, Sayan Malakshinov Oracle performance tuning engineer Oracle ACE Associate http://orasql.org =20 ------=_NextPart_000_0BFC_01D67D28.8A5F8D10 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

Humor me: Run it forced local into a newly created tablespace that is = ONLY the destination of the CTAS and has no other competing dictionary = and/or space management.

 

From:= = Mark W. Farnham [mailto:mwf@rsiz.com]
Sent: Friday, August = 28, 2020 10:45 AM
To: 'smishra_97@yahoo.com'; = 'oracle-l@freelists.org'; 'Tim Gorman'
Subject: RE: Big = Update/DML

 

=E2=80=9CContention was only seen for Interconnect or some gc = wait=E2=80=9D

 

Humor me: Run it forced local.

 

From:= = oracle-l-bounce@freelists.o= rg [mailto:oracle-l-bounce@free= lists.org] On Behalf Of Sanjay Mishra (Redacted sender = "smishra_97" for DMARC)
Sent: Thursday, August 27, = 2020 10:21 PM
To: oracle-l@freelists.org; Tim = Gorman
Subject: Re: Big = Update/DML

 

Tim

 

Thanks for the update. Based on the Sqlmonitor data shared = earlier , it was using Parallel for DDL operation but I will give = another try again as working to load another table in Nocompress as = existing one is Compress for OLTP. Tablespace was given 1.5T space as it = was initially extending it heavily and that helped to increase the = throughput. Contention was only seen for Interconnect or some gc wait = and CPU Contention or any high IO wait was also not seen in the AWR = report. 

 

Do we think Advance compression can be major bottleneck as = I earlier thought that it might help in utilizing buffer space but = doesn't look like the case. 

 

Will share details soon

 

Tx

Sanjay

 

On Thursday, August 27, 2020, 05:08:41 PM EDT, Tim Gorman <tim.evdbt@gmail.com> wrote: =

 

 

Agreed!  Pardon my unwarranted guess, but perhaps parallel DDL = (which is enabled by default) was not enabled in the session for some = reason before starting CTAS?  To verify, consider running the = following query after ALTER SESSION ENABLE PARALLEL DDL and before = CTAS?

select pddl_status from = v$session where sid in (select distinct sid from = v$mystat);


...or better yet, check PDDL_STATUS from another = session?

Another possible issue is high-water mark = management.  If the datafiles in the tablespace in which you're = creating the new table in the CTAS statement has to be autoextended to = accommodate the volume of data you're loading, then things might take = longer.  If the INCREMENT_BY (i.e. NEXT) for the datafile(s) is = really small, then datafile autoextension might take a *REALLY* long = time;  look for waits on "enq: HV - contention" or = "enq: HW - contention"?  At any rate, the solution is to = either make INCREMENT_BY really large, or just resize the datafile to = accommodate what you're building.

There are other possible issues = to be considered.  The key is to monitor the operation and use the = information that the Oracle RDBMS provides to understand why you're = bottlenecked and resolve it.

If you need suggestions for = monitoring specific actions, please ask?  It is frustrating when = someone announces "it doesn't work" with no attempt to = understand why.

On 8/27/2020 3:41 AM, Lothar Flatz = wrote:

Hi,

with regards to CTAS it is very hard to believe it takes = that long. I am pretty sure that there is something wrong.
A sql = monitor would be extremly = helpfull.

Regards

Lothar

Am 27.08.2020 um 02:43 = schrieb Sanjay Mishra (Redacted sender smishra_97 for = DMARC):

Sayan

 

Update statement = is 

 

Update snows.stamp_detail set set = stamp_process_calc=3Dprocessed_calc_amt;

=

 

Tried to = use

1. Parallel DML with 100 --> Taking = 20+hrs

2. CTAS was tried using half a billion as = well as 1 billion rows with parallel 50, 75,100 - Almost same = result

3. CTAS with nologging using same step 2 but = still not much improvement

 

We have 5-10 such big table and so running = each with this much time-frame need high = downtime

Tx

Sanjay

 

On Wednesday, August 26, 2020, 08:23:18 PM EDT, Sayan Malakshinov = <xt.and.r@gmail.com> wrote: =

 

 

Hi Sanjay,

 

It would be better if you provide more details about your update. = Exact update statement would be helpful. is this column nullable/not = null?

 

 

On Thu, Aug 27, 2020 at 1:12 AM Sanjay Mishra <dmarc-noreply@freelists.org> = wrote:

Andy

 <= /o:p>

Yes look = like is an option if we are doing work online and despite take more time = but need not require downtime. In our case multiple DDL are running to = existing environment due to Application upgrade and so all work has to = be done with downtime. So challenge is reduce time of DML operations on = big tables containing few billions = rows.

 <= /o:p>

Tx

Sanjay<= /o:p>

 <= /o:p>

On Wednesday, August 26, 2020, 11:20:55 AM EDT, Andy Sayer <andysayer@gmail.com> wrote: =

 

 

It does sound like a virtual column could be the ideal solution. = But if data needs to be physically stored or cannot be calculated = deterministically at any point in time then Connor has a great demo of = using dbms_redefinition to create a new table online with a function to = map the new column. There=E2=80=99s obviously some overhead with context = switching but it may be far better than some of the obstacles you might = be facing at the moment: 

https://connor-mcdonald.com/2016/11/16/performing-a-lar= ge-correlated-update/ (and you might be able to help it with pragma = udf in the right circumstances).

 

Obviously, how helpful this is depends where the work is currently = going and how online this needs to = be. 

 

 

Thanks,

Andrew

=

 

On Wed, 26 Aug 2020 at 16:00, Jonathan Lewis <jlewisoracle@gmail.com> = wrote:

 

Is that 3-4 billion rows each, or total = ?

 

I would be a little suspicious of an update which populates a new = column with a value derived from existing columns. What options might = you have for declaring a virtual column instead - which you could index = if needed.

 

Be extremely cautious about calculating space requirements - if = you're updating every row on old data might you find that you're causing = a significant fraction of the rows in each block to migrate, and there's = a peculiarity of bulk row migration that can effectively = "waste" 25% of the space in every block that becomes the = target of a migrated row.

 

This effects can be MUCH work when the table is compress (even for = OLTP) since the update has to decompress the row before updating and = then only "re-compresses" intermittently as the block becomes = full.  The CPU cost can be horrendous and you still have the = problem of migration if the addition means the original rows can no = longer fit in the block.

 

If it is necessary to add the column you may want to review = "alter table move online" can do in the latest versions (in = case you can make it add the column as you move) or review the options = for dbms_redefinition - maybe running several redefinitions concurrently = rather than trying to do any parallel update to any single = table.

 

Regards

Jonathan Lewis

 

 

 

 

 


 

--

Best regards,
Sayan Malakshinov

Oracle performance tuning engineer

Oracle ACE Associate
http://orasql.org

=

 

------=_NextPart_000_0BFC_01D67D28.8A5F8D10-- -- http://www.freelists.org/webpage/oracle-l