Orion... should we believe what we see?

From: Tornblad, John <JTornblad_at_emdeon.com>
Date: Mon, 25 Feb 2013 00:34:16 +0000
Message-ID: <8A354F8DD7A2CF489BC375F536A4BF4514B1D6_at_zbnaaeex012.na.webmd.net>



We are seeing wildy varying performance in an Orion run meant to simulate a DW workload (large sequential 1MB reads).

The output below seems to be "red flags" all over:

a) Wildly varying results, even counterintuitive results (more concurrent reads = less throughput)
b) Latency on the first test (1 concurrent read at a time) showed 34 IOs at a 1-2 sec latency 
c) In other tests, smaller reads (128KB) seem to perform better but have only yielded a maximum of ~1300 MB/sec

Trying to employ some "USE" (utilization/saturation/errors) methodology but this is time consuming. There is some skepticism of Orion's reliability in our shop. Problem statement: we believe this frame should be producing closer to 2300 MB/sec bandwidth on large sequential IOs.

Any comments regarding weirdness (or normality?) of these results, steps to take next, observations would be greatly appreciated.

Orion's documentation is a little scant and not many (any?) metalink notes regarding effectively using it. I've listened to Alex Gorbachev talk about Orion a couple of times but need more input on what to do or think when "things don't go right".

-john

$ORACLE_HOME/bin/orion -testname test -run advanced -matrix col -num_small 0 -write 0 -type seq -num_streamIO 1 -cache_size 3072 -verbose -num_disks 180 -size_large 1024 -simulate raid0 -stripe 128

Large/Small,      0
          1,  43.50
          2, 197.40
          3,  37.75
          4, 568.16
          5,  24.11
          6,  25.17
          7,  67.05
          8, 100.88
          9,  63.42
         10,  64.42
         11, 402.35
         12, 364.49
         13,  76.10
         14, 158.64
         15, 252.41
         16, 591.30
         17, 574.85
         18, 330.49
         19, 176.74
         20,  46.64
         21, 118.67
         22, 144.45
         23, 508.20
         24, 177.40
         25, 185.84
         26, 524.93
         27, 127.82
         28,  53.91
         29,  80.99
         30,  92.30
         60, 555.11
         90, 988.31
        120, 1019.90
        150, 1117.30
        180, 1130.85
        210, 1119.28
        240, 1132.22
        270, 1123.01
        300, 1146.14
        330, 1119.74
        360, 1124.75

Latency Histogram for large IOs _at_ Small=0 and Large=1
        Latency:                # of IOs (read)          # of IOs (write) 
        0 - 1           us:             0                       0
        2 - 4           us:             0                       0
        4 - 8           us:             0                       0
        8 - 16          us:             0                       0
       16 - 32          us:             0                       0
       32 - 64          us:             0                       0
       64 - 128         us:             0                       0
      128 - 256         us:             0                       0
      256 - 512         us:             0                       0
      512 - 1024        us:             0                       0
     1024 - 2048        us:             2515                    0
     2048 - 4096        us:             45                      0
     4096 - 8192        us:             11                      0
     8192 - 16384       us:             20                      0
    16384 - 32768       us:             20                      0
    32768 - 65536       us:             1                       0
    65536 - 131072      us:             0                       0
   131072 - 262144      us:             0                       0
   262144 - 524288      us:             0                       0
   524288 - 1048576     us:             2                       0
  1048576 - 2097152     us:             34                      0
  2097152 - 4194304     us:             0                       0
  4194304 - 8388608     us:             0                       0
  8388608 - 16777216    us:             0                       0
 16777216 - 33554432    us:             0                       0
 33554432 - 67108864    us:             0                       0
 67108864 - 134217728   us:             0                       0
134217728 - 268435456   us:             0                       0

</pre>This message is confidential, intended only for the named recipient(s) and may contain information that is privileged or exempt from disclosure under applicable law. If you are not the intended recipient(s), you are notified that the dissemination, distribution, or copying of this message is strictly prohibited. If you receive this message in error or are not the named recipient(s), please notify the sender by return email and delete this message. Thank you.
--

http://www.freelists.org/webpage/oracle-l Received on Mon Feb 25 2013 - 01:34:16 CET

Original text of this message