Feed aggregator

Another DBMS_Scheduler Chain Rule Issue

David Aldridge - Thu, 2011-01-20 05:23
Following on from a recent adventure with a problem validating DBMS_Scheduler chain rules, I hit another issue today. A rule was defined with a step name that does not exist. This happened because there is an odd limit of chain step name lengths (27 bytes I think), and the name of the step in the […]
Categories: BI & Warehousing

And Another Thing …

David Aldridge - Thu, 2011-01-20 04:02
Following on from my recent complaint about an all-time low on the Oracle Forums, does anyone else get the impression that the work there is not just answering questions, but seems increasingly to be correcting all of the incorrect answers? Obviously I have an example in mind. Or has it always been thus?
Categories: BI & Warehousing

XML Schema 1.1 – What you need to know

Ramkumar Menon - Wed, 2011-01-19 11:02

Motivation

XML Schemas, more popularly known as XSDs, provide us a way to describe structure and content of XML Documents. Managed by W3C's XML Schema WG, a part of the XML Activity Group, XML Schemas evolved out of the shortcomings of its simpler and light-weight non-XML predecessor - DTDs, as well as through discussions and conferences of various interest groups across the community. The first W3C Recommendation of the XML Schema specification was released on 2 May, 2001.  The initial edition provided the users a strong type system, enabled users to model rich and complex structural constraints and value-spaces, recognized and allowed expression of namespaces, and also defined the intended behavior of Schema aware processors. This was followed up with a Second Edition in 2004, which wasn't really a newer version, but a minor errata driven release to the First Edition.

While XML Schemas matured, so did other XML based specifications like XSL, XQuery, Web Services, XPath, and several others . The XML Schema WG also ensured that these dependencies feed into the requirements for future versions of the language. The WG also provided an email distribution list that allowed developers and architects alike to exchange ideas, discuss issues and shortcomings, providing an open platform for debating the current and direction of the standard's future.  XML Schema 1.0 provided a platform that allowed architects in W3C and outside to clearly think about what is the next right step towards a "Version 2".

In 2005, W3C conducted a workshop on XML Schema 1.0 user experience. This workshop brought together users, developers and vendors into an open dialogue on potential usability issues, inter-op concerns and set the stage for a future version of the specification that addresses these glitches while adding in "must-have" features that were missed in 1.0. The key takeaways from all the following discussions were features related to Extensibility, Versioning and Validation. Apart from these, there were several very specific schema semantics that were long awaiting a change since 1.0 Ed2.

These led to the birth of a newer 1.1 version of the specification, which will be discussed in greater detail within this article.

Current Status of the Specification

The WG has released two Candidate Recommendation drafts of the following documents

The WG also released a non-normative guide to versioning XML Languages using new features of XML Schema 1.1.

Guide to Versioning XML Languages using new XML Schema 1.1 features

There are few key early implementers of the latest version of the specification namely Apache Xerces and SAXON.

Key Changes since 1.0

The changes since version 1.0 till date can be grouped into the following.

  • Validation
    • Rule based Validation - Love Schematron? 1.1 enables users to express cross-field validations and constraints using XPath 2.0.
  • Extensibility and Versioning
    • Wildcard Enhancements
      • Weak wildcard support - XSD 1.0 uses Unique Particle Attribution rule (UPA) to avoid ambiguity associated primarily with the usage of wildcard content. UPA restricts users from defining certain complex content models. XSD 1.1 attempts to address this using Weak wildcards that disambiguate using precedence rules.
      • Negative Wildcards and Multiple Namespaces
      • Open Content
    • <xsd:all> Model Group Changes  - Finally!
    • Vendor Unique Extensions - Enables vendors to ship their own primitive datatypes and facets
    • Conditional Inclusion - Enable creation of extensible schemas.
  • Miscellaneous Enhancements
    • Conditional Type Assignments
    • Inherited Attributes
    • Default Attribute Group - Enable globally shared set of attribute definitions.
    • Substitution Group Enhancements - Multiple element substitution related changes.
    • ID related changes - Enables, amonst others, mutliple ID attributes for elements.
    • Redefine Replaced by Override - Richer overrides, more interoperability.
    • Target Namespace attribute on local elements and attribute declarations
  • New Datatypes and Facets

One significant aspect of the standard that has been kept unchanged across revisions and editions is the Namespace URI for the schemas. What this means is that the version 1.1 schemas that you create will have the same XSD Namespace URI as a version 1.0 schema.

http://www.w3.org/2001/XMLSchema

In addition to this, almost all of the features in 1.0 have been brought into 1.1 in order to ensure backwards compatibility. What this means is that an XML document defined using a 1.0 schema can be validated using a 1.1 schema processor/validator.  This backward compatibility also ensures that the specification can satisfy dependant requirements from other specifications that rely on syntax and semantics of the 1.0 version of the specification.

1. Validations Rule based Validation

XML Schema 1.1 introduces a new Schema component named <assert> to facilitate rule based validation, along similar lines of Schematron.

User can specify an Boolean XPath expression that will result in the assertions to pass or fail, depending on the evaluation of the expression. Assertion failures are equivalent to a  non-assertion validation failure.

Example 1 <xsd:element name="priceRange">
  <xsd:complexType>
     <xsd:sequence>
         <xsd:element name="minPrice" type="xsd:decimal"/>
         <xsd:element name="maxPrice" type="xsd:decimal"/>
     </xsd:sequence>
    <xsd:assert test="minPrice le maxPrice"/>
</xsd:complexType>
</xsd:element>

In the above example, the assertion checks if the decimal value of minPrice element is less than or equal to the maxPrice element. If an instance document fails this assertion, that document is considered invalid w.r.t the Schema, just like a non-assertion validation failure would cause it to be.

Syntax and Namespace Scope

The asserts are always defined at the end of the type declaration. It can hold a valid XPath 2.0 boolean expression. Being related to XPath 2.0, users can leverage the XPath 2.0 specific functions and operators while building the assertions.

The context node used for the expression evaluation is always the element whose type declaration hosts the mentioned assertion. In the above example, the context node would be the priceRange element. Hence all expressions within the assertions will need to be relative to the priceRange element.

One key restriction that is imposed on the path expressions that can be modeled in an assertion is that one cannot refer to parent or ancestor nodes of the context node. Only context node, or any of its descendants can be referred to within the path expressions. Neither it's siblings nor it's ancestors are permitted.

By this rule, the following assertion is invalid.

<assert test="string-length(ancestor::order/id) > 0"/>

so is

<assert test="string-length(following-sibling::shippingDate)  > 0"/>

and so is

<assert test="doc('pub/docs/customers.xml')/customerInfo[2]/id='12345'"/>

Users can model as many assertions as they want on the type declaration. In that case, the instance document will be considered valid only if none of the assertions for that type declaration have failed.

The namespace scope of the path elements in the expression is determined by the value of the xpathDefaultNamespace attribute on the assert element. If the attribute is absent, its value is assumed to be ##local - which translates into null namespace. This attribute can also be declared to hold any URI, or alternatively configured to use the targetNamespace or the default namespace declared for the XML Schema.

Example 2

<assert test="tns:price > 0" xmlns:tns="www.oracle.com" xpathDefaultNamespace="www.oracle.com"/>

<assert test="price > 0" xpathDefaultNamespace="##targetNamespace"/>

<assert test="price > 0" xpathDefaultNamespace="##defaultNamespace"/>

<assert test="tns:price > 0" xmlns:tns="www.oracle.com" xpathDefaultNamespace="##targetNamespace"/>

<assert test="price > 0" xpathDefaultNamespace="##local"/>

<assert test="ns1:value > ns2:netValue" xmlns:ns1="www.oracle.com" xmlns:ns2="www.abc.com"/>

The last assert is equivalent to not having the xpathDefaultNamespace attribute at all.

Rule based Validation of scalar content

As we described above, <assert>s can be used to define rule-based constraints on complex content. In the event that one wishes to model rules for simple content - for instance, against elements of string or decimal datatype, one can use a new facet available for atomic datatypes - named <assertion>.

Original Declaration:
<element name="evenPriceChangePercent" type="decimal"/>

Declaration with assertion:
<element name="evenPriceChangePercent" type="evenPriceChangePercentType"/>
<simpleType name="priceChangePercentType">
  <restriction base="decimal">
       <assertion test="$value <= 100 and $value >= 0"/>
       <assertion test="$value mod 2 = 0"/>
  </restriction>
</simpleType>

As seen above, this facet provides a default "value" variable that holds the text content of the simple content used for the evaluation. You cannot access any other context nodes within the XPath expression.

2. Extensibility and Versioning Wildcard Related Enhancements Weak Wildcards

Wildcards were introduced into XML Schema to build flexibility into schemas that users build. One could build newer schemas that could validate instances that were defined using an older version of the schema [Backward compatibility], or vice versa [Forward compatibility]. XMLSchema 1.0 provided schema constructs like <any/> and <anyAttribute/> for this purpose. These constructs enable elements and attributes that aren't explicitly defined in the schema, to occur in a valid XML instance document, providing way for extensible content models.

For instance,

Example 3 <element name="Address">
  <complexType>
     <sequence>
          <element name="street" type="string"/>
          <element name="city" type="string"/>
          <element name="state" type="string"/>
          <element name="zip" type="string"/>
          <!--allow any element from any namespace to occur -->
         <any namespace="##any" minOccurs="0"/>
     </sequence>
  </complexType>
</element>

1.0 allowed for adding these wildcards at any position within your type declaration,as long as there isn't an ambiguity with explicitly declared elements.

Look at the example below.

Example 4 <element name="Address">
  <complexType>
     <sequence>
          <element name="street" type="string"/>
          <element name="city" type="string"/>
          <element name="state" type="string"/>
          <element name="zip" type="string" minOccurs="0"/>
          <!--allow any element from any namespace to occur -->
         <any namespace="##any" minOccurs="0"/>
     </sequence>
  </complexType>
</element>

With the above schema in mind, consider the following XML document.

Example 5 <Address>
   <street>1, Courthouse Square</street1>
  <city>Hill Valley</city>
  <state>CA</state>
  <zip>91919</zip>
</Address>

With this document, it is impossible to state if zip element should be associated with the named element zip, or the wildcard any element. This schema is thus prohibited in XML Schema 1.0. The rule that prohibits this is termed as Unique Particle Attribution (UPA) Rule.

With 1.1, the UPA restriction is relaxed - it is no longer forbidden to have the kind of XML Schema mentioned in Example 4. [competition between element and wildcard]. When there is such an ambiguity, the named element definition takes precedence. In this case, since there is an element named "zip" that can resolve the ambiguiity, that one is chosen over the wildcard during resolution.

Also considered legal is a competition between two wildcards. However, competition between two elements is still considered a violation of the rule.

For example, the following XML schema is still invalid since the processing application will not be able to disambiguate the <direction> element in the XML Instance if their respective siblings were absent.

Example 6 <element name="TimeTravel">
   <complexType>
      <choice>
      <sequence>
          <element name="direction" type="string"/>
           <element name="year" type="integer" minOccurs="0"/> 
      </sequence>
      <sequence>
          <element name="direction" type="string"/>
           <element name="universeName" type="string" minOccurs="0"/> 
      </sequence>
     </choice>
  </complexType>
</element>

Multiple Namespaces and Negative Wildcards

In XML Schema 1.0, you had the capability to specify the namespace URI for a wildcard element. You could either specify

  • A list of one or more specific URIs
  • ##other - which means any URI other than the targetNamespace
  • ##targetNamespace - which is self explanatory
  • ##any - which means any URI
  • ##local - which indicates null namespace.
Example 7 <any namespace="##other"/>
<any namespace="http://www.delorean.com http://www.fluxcapacitor.com"/>
<any namespace="##targetNamespace"/>
<anyAttribute namespace="##local"/.>

In XML Schema 1.1, you have more ways to control your wildcards. You could in fact state that your wildcards NOT come from a namespace UR or multiple Namespace URIs.

In addition to this, you can also constrain the potential qualified name of the wildcard through the new schema constructs.

Example 8 <any notQName="tns:Almanac tns:IceHockey"/>
<anyAttribute notNamespace="##targetNamespace"/>
<any notNamespace="##defaultNamespace"/>
<any notNamespace="http://www.fluxcapacitor.org"/>

Other constraints you can impose are:

  • Ensure your wildcards do not have the same name as any of it's siblings.
  • Ensure your wildcards do not have the same name as any of the existing global declarations in the schema.
Example 9 <any notQName="##definedSibling"/>
<any notQName="##defined"/>

Take a look at the schema below.

Example 10 <element name="Address">
   <complexType>
      <sequence>
           <element name="street" type="string"/>
           <element name="city" type="string"/>
           <element name="state" type="string"/>
           <element name="zip" type="string"/>
            <any notQName="##definedSibling" namespace="##local"/>
      </sequence>
  </complexType>
</element>

This will fail validation for the following XML instance.
<Address>
   <street>1, Courthouse Square</street1>
  <city>Hill Valley</city>
  <state>CA</state>
  <zip>91919</zip>
  <zip>94321</zip>
</Address>

Open Content

Open Content allows for defining elements to possess child element or attribute content other than the ones explicitly defined in it's content model, and allow them to be available at any position within its content.

In 1.0, you needed to explicitly position your <xsd:any/> declarations within your choice or sequence content models. In 1.1, you can declare open content for your elements at one place - within the complexType definition, or at the schema level. On top of this, you can choose to distribute your open content interleaved between your named child elements [interleaved], choose to have them after all of the child elements. [suffixed], or disable inherited openContent behavior.

Example 11 - Interleaved Open Content <element name="Address">
  <complexType>
    <openContent mode="interleave">
         <any/>
    </openContent>

     <sequence>
          <element name="street" type="string"/>
          <element name="city" type="string"/>
          <element name="state" type="string"/>
          <element name="zip" type="string" minOccurs="0"/>
     </sequence>
  </complexType>
</element>

Valid Instances for the Schema

Example 1:

<Address>
   <street>1, Courthouse Square</street1>
   <boo/>
  <city>Hill Valley</city>
   <foo/>
   <bar/>

  <state>CA</state> 
  <zip>91919</zip>
   <boo2/>
</Address>

Example 2:

<Address>
   <street>1, Courthouse Square</street1>
   <boo/>
   <foo/>
   <bar/>
  <city>Hill Valley</city> 
  <state>CA</state> 
   <boo2/>
  <zip>91919</zip>
</Address>

Note that the maxOccurs="unbounded" and minOccurs="0" attributes on the <any/> is implicit - which explains how one is able to have multiple wildcard elements in your XML instance document.

Example 12: Suffixed Open Content
<element name="Address">
  <complexType>
    <openContent mode="suffix">
       <any/>
    </openContent>

     <sequence>
          <element name="street" type="string"/>
          <element name="city" type="string"/>
          <element name="state" type="string"/>
          <element name="zip" type="string" minOccurs="0"/>
     </sequence>
  </complexType>
</element>

Valid Instance for the Schema

<Address>
   <street>1, Courthouse Square</street1> 
  <city>Hill Valley</city> 
  <state>CA</state> 
  <zip>91919</zip>
   <boo/>
   <foo/>
   <bar/>
   <boo2/>

</Address>

The third use-case - mode="none" is typically used when you are importing and extending a complexType from a foreign schema, but do not wish to inherit the openness of the inherited type definition.

The wildcard <any/> within the <openContent> section can be combined with the wildcard constraints discussed in the earlier section as well. [e.g. <any notQName="fc:doc"/>]

Another variant of of the openContent use-case is the one where user wishes to model adding wildcards within any element content defined in the schema - i.e. a schema-wide openContent.

This is supported through a schema construct <defaultOpenContent> declared as a direct child of the <schema> element. See example below.

Example 14 (I skipped "13") <schema targetNamespace="..." xmlns="....">
<defaultOpenContent mode="interleave" appliesToEmpty="true">
   <any/>
</defaultOpenContent>

<element name="Order">
  ....
</element>

<element name="Info">
   <complexType/>
   <attribute name="id" type="string"/>
</element>

<element name="BillingAddress" type="tns:BillingAddressType"/>

<element name="ShippingAddress" type="tns:ShippingAddressType"/>

  <complexType name="BillingAddressType"> 
     <sequence>
          <element name="street1" type="string"/>
          <element name="street2" type="string"/>
          <element name="city" type="string"/>
          <element name="state" type="string"/>
          <element name="zip" type="string" minOccurs="0"/>
     </sequence>
  </complexType>

  <complexType name="ShippingAddressType"> 
     <sequence>
          <element name="street" type="string"/>
          <element name="city" type="string"/>
          <element name="state" type="string"/>
          <element name="zip" type="string" minOccurs="0"/>
     </sequence>
  </complexType>


</schema>

This schema can be used to generate XML documents where arbitrary elements can occur between the contents of ShippingAddress and BillingAddress. Even the empty element Info can have open content within it attributed to the appliesToEmpty attribute on the <defaultOpenContent> declaration.

Valid Instance for the above XSD:

<Order>
   <Info id="123">
        <HelloWorld/>
   </Info>
  <BillingAddress>
      <Welcome6>123</Welcome6>
      <street1>1, Courthouse Square</street1> 
      <street2>Apt 007</street2> 
      <boo2/> 
      <city>Hill Valley</city> 
      <state>CA</state> 
      <zip>91919</zip> 
      <foo3/> 
</BillingAddress>
  <ShippingAddress>
      <Welcome>123</Welcome>
      <street>1, Courthouse Square</street>  
      <boo/> 
      <city>Hill Valley</city> 
      <state>CA</state> 
      <zip>91919</zip> 
      <foo/> 
</ShippingAddress>
</Order>

<xsd:all> Group Changes

The "All" Model group was bundled with several restrictions when it was introduced in XML Schema 1.0.

1. The group disallowed multiple occurence elements to be a part of the content model.   The following is illegal in XML Schema 1.0

Example 15 <complexType name="FluxCapacitorType">
   <all>
       <element name="timeCalibrator" type="string" maxOccurs="unbounded"/>
       <element name="needsRecharge" type="boolean" minOccurs="3" maxOccurs="unbounded"/>
  </all>
</complexType>

Elements within <all> groups were supposed to have maxOccurs=1 and a minOccurs of 0 or 1.

i.e. elements occur in any order, but each element occurs exactly once, if present.

2. <all> group cannot be nested within another model group. What this means is that you cannot have an <all> nested within a <sequence> or a <choice> model group. Thus the following is illegal in 1.0.

Example 16 <complexType name="FluxCapacitorType">
  <choice>
       <element name="autoFunction" type="AutoFunctionType"/>
       <all>
            <element name="timeCalibrator" type="string" maxOccurs="unbounded"/>
            <element name="needsRecharge" type="boolean" minOccurs="3" maxOccurs="unbounded"/>
      </all>
   </choice>
</complexType>

3. You cannot extend a non-empty BaseType with an <all> content model. The following is illegal in 1.0

Example 17 <complexType name="BaseTimeTravelMachineType">
  <all>
        <element name="timeComputer" type="string"/>
        <element name="carrier" type="CarrierType"/>
  </all>
</complexType>

<complexType name="HighSpeedTimeTravelMachineType">
  <complexContent>
       <extension base="BaseTimeTravelMachineType">
            <all>
                   <element name="TravelerProfileReader" type="ProfileReaderType"/>
          </all>
       </extension>
   </complexContent>
</complexType>

4. You cannot have wildcards within <all> groups.  The following is illegal in 1.0.

Example 18 <complexType name="BaseTimeTravelMachineType">
  <all>
       <any/>
        <element name="timeComputer" type="string"/>
        <element name="carrier" type="CarrierType"/>
  </all>
</complexType>

XML Schema 1.1 has legalized all the above models.

Vendor Unique Extensions

Vendors can now define their own primitive datatypes and facets. For instance, it may provide support for a new primitive datatype called string, with it's own implementation defined facets. These datatypes will be in a vendor defined namespace in order to distinguish them from XSD's predefined datatypes. The treatment of  the vendor defined datatype depends on each vendor - and evidently, it may not be supported by a processor implemented by another vendor.

Example 19 <!--new custom integer datatype -->
<element name="age" type="xdk:integer"/>
<!--new custom 'era' facet -->
<simpleType name="TimeType">
     <extension base="xdk:oracleDateTime">
            <xdk:era="Andromedan"/>
      </extension>
</simpleType> Conditional Inclusion

This feature is primarily used to ensure that XML documents that may be defined down the future using a later version of XML Schema language [say 5.5] can be processed in compatibility mode by a XML Processor that supports a prior version of  XML Schema [starting from 1.1].  For instance, one of the element declarations in a user defined XML Schema may use a new construct that was introduced in XML Schema 5.5. If you need to process this XML Schema using an XML Schema 1.1 compliant processor, you have to have a feature that allows you to pre-process and selectively interpret the constructs that the 1.1 version processor understands.

For this purpose, XML Schema 1.1 introduces a new set of attributes in the Schema Versioning Namespace, as listed below.

The Schema Versioning Namespace has a URI - http://www.w3.org/2007/XMLSchema-versioning. Within this namespace URI are four attributes.

  1. vc:typeAvailable
  2. vc:typeUnavailable
  3. vc:facetAvailable
  4. vc:facetUnavailable
  5. vc:minVersion
  6. vc:maxVersion

Look at the 2 examples below. In the first example, the user intends to define an element named TimeMachine that can be interpreted by an XML Schema Processor that complies with version 1.6, as well as one that supports version 1.1.

Example 20 <!--The below declaration is accepted for pre-processing for XML Schema versions 1.1 [inclusive], but less than 1.6. [exclusive] -->
<element name="TimeMachine" vc:minVersion="1.1" vc:maxVersion="1.6">
   <complexType>
         <sequence>
           .....
</element>
<!--The below declaration is accepted for pre-processing for XML Schema versions 1.6 or later. It may have constructs that are not understood by 1.1 processors.
-->

<element name="TimeMachine" vc:minVersion="1.6">
      <complexPhasedDatatype>
          <prioritizedSequence>
                ...
</element>

In the second example, listed below, the user intends to signal the usage of a vendor defined datatype or facet through the first 4 schema versioning attributes listed above.

Example 21 <!--line 1 is for processors that implement the custom datatype, whereas line 2 is used by other processor implementations -->
<element name="EmmetBrownsLabHeight" vc:typeAvailable="xdk:feet" type="xdk:feet"/>
<element name="EmmetBrownsLabHeight" vc:typeUnavailable="xdk:feet" type="xsd:integer"/>

<!--declaration 1 is used by processor that implements the custom minLength facet. The latter declaration is the fallback used by other processors ->
<element name="ItemCode" vc:facetAvailable="xdk:minLength">
    <simpleType>
       <restriction base="string">
              <xdk:minLength value="1,2"/>
       </restriction>
    </simpleType>
</element>
<element name="ItemCode" vc:facetUnavailable="xdk:minLength" type="string"/>

3. Miscellaneous Enhancements Conditional Type Assignments

XML Schema 1.1 allows for dynamic type assignment based on XPath 2.0 boolean condition expressions.

See example below.

Example 22 <!--inline alternative type definitions -->
<element name="TimeTravel" type="TravelType">
      <alternative test="@direction='Future'">
          <complexType>
              <complexContent>
              <restriction base="TravelType"
                         ....
<!--        some past travel related elements go here -->
            </complexType>
       </alternative>
      <alternative test="@direction='Past'">
          <complexType>
              <complexContent>
              <restriction base="TravelType"
                         ....
   <!--        some future travel related elements go here -->
            </complexType>
       </alternative>
  </element>
                          OR
<!--Named alternative type definitions -->
<element name="TimeTravel" type="TravelType">
   <alternative test="@direction='Future' type="FutureTravelType"/>
   <alternative test="@direction='Past' type="PastTravelType"/>
</element>

Few points to note here.

  • You can only access attributes of the context node within the xpath expression. You cannot use child/descendant elements in the expression as you would have seen in the assert statements earlier in the article, nor anything on the ancestor axis. In the above example, direction is an attribute defined within  the complexType TravelType that is used to define the TimeTravel element.
  • The alternative types should be datatypes that are derived from the actual type defined on the element. In the above example, FutureTravelType and PastTravelType are derived from TravelType.
  • Just as you would model the namespaces in asserts, you could use the xpathDefaultNamespace attribute to qualify the xpath expression on the alternatives.

Inherited Attributes

Inheriting attributes allows access to attributes of ancestor elements within descendant elements. Though inheriting does not allow the attributes to actually appear on the descendant element in the XML instance, they can be effectively used within asserts and type alternatives for the descendant element. An attribute can be declared as inheritable by defining inheritable="true" on the attribute declaration, as illustrated below.

Example 23 <element name="TimeTravelType">
    <complexType>
         <sequence>
               <element name="GadgetType">
                    <alternative test="@mode='Past' type="PastGadgetType"/>
                    <alternative test="@mode='Future' type="FutureGadgetType"/>
              </element>
             <element name="dimension">
                 ....
                <assert test="@length > 200 and @mode='Past'"/>
                <assert test="@length <= 200 and @mode='Future'"/>
        </sequence>
        <attribute name="mode" inheritable="true"/>
    </complexType>
</element>

Note that defining an inheritable attribute does not prevent schema designer to define attributes with the same name on the descendant element datatypes. In that case, the local declaration prevails.

Default Attribute Group

Default attribute group defines a set of global attributes that can be accepted by all complexType definitions directly defined in the schema file. [not applicable to imported/included components]

Example 24 <schema targetNamespace="...."
                   defaultAttributes="myLocalAttrs"
                   ....>
     <element name="AllTimePieces">
               ...
     <element>
     <element name="MyTimePiece">
                   ...
      <element>
      <element name="HisMuchBetterTimePiece">
                ....
         </element>

        <attributeGroup name="myLocalAttrs">
             <attribute name="tpId" type="ID"/>
         </attributeGroup>
</schema>

<AllTimePieces tpId="NONE">
         <MyTimePiece tpId="0">
                        ....
          </MyTimePiece>
          <HisMuchBetterTimePiece tpId="1">
                        ...
           </HisMuchBetterTimePiece>
   </AllTimePieces> 

This eliminates the need for redundant reference to the attribute group within all complexType definitions that needed to share the same set of attributes.

Substitution Group Enhancements

SubstitutionGroups in XML Schema 1.0 allows elements to be substitutable for other elements. To be precise, it allows for replacing an element with another. XML Schema 1.1 enhances this functionality by allowing an element to be substitutable for one or more elements

Example 25 <!--SubstitutionGroups in XML Schema 1.0 -->
<element name="shipComment" substitutionGroup="po:Comment" type="ShipCommentType"/>
<element name="billComment" substitutionGroup="po:Comment" type="BillComentType"/>

<!--SubstitutionGroups in XML Schema 1.1 ->
<element name="Flute" substitutionGroup="po:MusicalInstrument po:WindInstrument" type="FluteType"/>
<element name="MusicalInstrument" type="po:MusicalInstrumentType"/>
<element name="WindInstrument" type="po:WindInstrumentType"/>

As seen in Example 25, the element Flute can be substituted for either the MusicalInstrument element, or the WindInstrument element, provided FluteType datatype is derived from MusicalInstrumentType and WindInstrumentType.

ID related changes

XML Schema 1.1 has revised ID semantics to allow elements to possess more than one xsd:ID attribute. XSD 1.0 had forbidden this usage for ensuring compatibility with DTDs. Schemas that intend to be compatible can choose to not use this feature.  It also allows ID attributes to have default and fixed values - again something that was forbidden in 1.0.

Example 26
<element name="Fruit">
   <complexType>
     <sequence>
          <element name="taste" type="f:TasteType"/>
           ...
      </sequence>
     <!--Illustrating more than one ID on an element -->
     <attribute name="fruitUID" type="ID"/>
     <attribute name="flavourUID" type="ID"/>
   <!--illustrating default value on an ID type attribute -->
    <attribute name="locationCode" type="ID" default="US-CA"/>
</element>

Redefine Replaced by Override

<redefine> construct in XML Schema 1.0 enabled users to extend or restrict element type declarations. What it did not allow was to create a new definition for the same element when redefining it.

Example 27 <!--Legal redefine in XML Schema 1.0 -->
<schema .....>
   <redefine schemaLocation="http://www.greatschemas.org/invoice.xsd">

    <complexType name="BillingAddressType">
<!--BillingAddressType is defined in the included schema above, and is being redefined -->
            <extension base="gsc:BillingAddressType">
                  <sequence>
                        <element name="BillingCode" type="string"/>
                  </sequence>
            </extension>
    </complexType>

    <!--
        Illegal redefine in XML Schema 1.0  where BillingAddressType has been defined in the included schema  above
       -->

      <complexType name="BillingAddressType">
                  <sequence>
                        <element name="TotallyDifferentBillingStuff" type="string"/>
                  </sequence>
            </extension>
    </complexType>
</redefine>

Moreover, XML Schema 1.0 had many conflicting and non-interoperable implementations for <redefine> that 1.1 decided to deprecate the <redefine> functionality and replace it with a richer <override> construct instead.

Example 28 <!--Legal override in XML Schema 1.1 showing extension of types-->
<schema .....>
   <override schemaLocation="http://www.greatschemas.org/invoice.xsd">

    <complexType name="BillingAddressType">
<!--BillingAddressType is defined in the included schema above, and is being redefined -->
            <extension base="gsc:BillingAddressType">
                  <sequence>
                        <element name="BillingCode" type="string"/>
                  </sequence>
            </extension>
    </complexType>

    <!--
        Legal override in XML Schema 1.0  where BillingAddressType has been defined in the included schema  above
       -->

      <complexType name="BillingAddressType">
                  <sequence>
                        <element name="TotallyDifferentBillingStuff" type="string"/>
                  </sequence>
            </extension>
    </complexType>
</override>

Note that <redefine>  has been deprecated, but not removed yet. The WG is still looking for feedback from users on this construct.  It MAY be removed from future versions of the specification.

Target Namespace attribute on local elements and attribute declarations

XML Schema 1.1 allows users to define targetNamespace on local element and attribute declarations. The namespace URI may be different from the targetNamespace defined on its ancestors or the schema element.

Example 29 <schema targetNamespace="http://xmlns.oracle.com/definitions" ..... elementFormDefault="qualifiied">
    <element name="Address">
           <complexType>
               <sequence>
                     <element name="street1" targetNamespace="www.alternatens.com/defns"/>
                       ....
               </sequence>
                <attribute name="id" targetNamespace="http://xmlns.groceries.com/defns"/>
             </complexType>
         </element>
</schema>

<schema targetNamespace="http://xmlns.oracle.com/definitions" ..... elementFormDefault="qualifiied">
    <import namespace="www.foreign.com/defns" schemaLocation="Alt_Objects.xsd"/>
    <element name="Address">
           <complexType>
            <complexContent>
              <extension base="ForeignAddressType">
               <sequence>
                     <element name="street1" targetNamespace="www.foreign.com/defns"/>
                       ....
               </sequence>
                <attribute name="id" targetNamespace="www.foreign.com/defns"/>
             </complexType>
         </element>
</schema>

4. New Datatypes and Facets

XML Schema 1.1 added several new core datatypes into its existing library.

xsd11_datatypes

1. The anyAtomicType datatype

This datatype has been newly introduced as the base type for all atomic datatypes in the Schema's datatype library. [i.e. string, float,boolean etc]. It extends from anySimpleType.

"In XML Schema, a simple type is either a list type, a union type, or an atomic type; however, these three kinds of simple type are not recognized explicitly in the type hierarchy. For XPath, it was necessary to distinguish atomic types from the other kinds of simple type, so xdt:anyAtomicType was invented to achieve this."

                - Michael Kay, SAXONICA

2. New Numeric Datatypes

A new datatype named precisionDecimal has been added to support IEEE-754 floating-point decimal datatypes. precisionDecimals carry a numeric value, an arithmetic precision and a sign. Apart from distinguishing between positive and negative zero, this datatype also holds NaN and positive/negative infinity in it's value space.

Example 30 <price>1200.04</price>
<price>INF</price>
<price>-INF</price>
<price>NaN</price>

3. Date/time Datatypes

The following new datatypes - dateTimestamp, dayTimeDuration and yearMonthDuration have been added to the library. dateTimestamp extends the existing dateTime datatype with a mandatory explicitTimezone facet. [the timezone portion was optional in the dateTime datatype.]

<element name="shipDate" type="dateTimeStamp"/>

<shipDate>2010-02-01T12:03:00-08:00</shipDate>
<shipDate>2010-02-01T12:03:00Z</shipDate>

Both new duration related datatypes extend from the existing duration datatype and does a simple restriction on its lexical space.

<!--2 years and 3 months yearMonthDuration -->
<sampleYearMonthDuration>P2Y3M</sampleYearMonthDuration>
<!--1 day and 6 hours dayTimeDuration-->
<sampleDayTimeDuration>P1D6H</sampleDayTimeDuration>

5. Next Steps

Several implementations are under development. This includes the upcoming versions of Xerces, and the SAXON processor. 

You can contribute to the specification by submitting your comments to the public comments mailing list for the standard - www-xml-schema-comments@w3.org. Here is the archive.

6. Credits

Roger Costello's Tutorial is one of the best you can grab to get to the next level of details on the specification. He has done a fabulous job in documenting all major features to fine levels of detail. You can find a link to his tutorial in the next section.

6. References

1. XML Schema 1.1 Part 1: Structures

2. XML Schema 1.1 Part 2: Datatypes

3. Roger Costello's XML Schema Tutorial

4. XML Schema 1.1, Part 1: An introduction to XML Schema 1.1 - IBM

5. XML Schema 1.1, Part 2: An introduction to XML Schema 1.1 - IBM

6. XML Schema 1.1, Part 3: An introduction to XML Schema 1.1 - IBM

Oooohhh... shiny!

alt.oracle - Tue, 2011-01-18 22:52
I went to last year's Oracle Open World. I'd always wanted to go, but having been a consultant for so many years, those opportunities don't always come your way. In my experience, companies will spring for their own employees to go to Open World, but "no way" to that lousy, overpaid consultant who probably won't even be here next week. That leaves it up to the consulting company, whose take on things is usually, "If you don't already know everything they're talking about at Open World, then why did we hire you? Get back to work!" But since I work for a good consulting company, they offered me the chance to go.

Open World is a blast. If you're a geeky Oracle person like me, it's a complete nerd-o-gasm. First of all, Oracle's always announcing the "next big thing" – this year, it was the Oracle Linux kernel (perhaps the subject of a future post) and the latest in Exadata. Then you have your session speakers, most of which are pretty good. The technology booths full of people trying to sell you stuff are always cool. Of course, best of all is all the free swag you get. I came home with more techie junk than you can shake a datafile at. Let me tell you, it takes some mad ninja skilz to nab 11 t-shirts from Open World and get them home. I had to throw away all my underwear just to get them to fit in my luggage (don't ask me how the flight home was...).

Of course, the real focus of any Open World is same as that of a lot of the software industry – better, faster, stronger, more. Newer and shinier. What you have isn't what you need. I can't fault them for this – they need to keep selling stuff to compete and to stay in business, and innovation is a huge part of what we do. Progress is good. But sometimes a DBA needs to distinguish between something that represents progress and something that represents a big ol' pile of shiny.

I talked last time about how being a good DBA means having a healthy dose of skepticism. That has to apply to "new feature-itis" too. Part of what I do involves evaluating new technologies. Not only do I have to evaluate the tech to verify that it does what it says it does, I need to assess that its benefits are worth the time, risks and cost of adopting it. As an evaluator, there's an implied trust with my employers that if I recommend a shiny, new feature, it's because it will benefit their interests – not necessarily mine. I haven't seen it often, but I can remember working with more than one DBA who didn't share my take on this. I've come to expect non-technical people to fall into the whole "Look! Shiny!" thing when it comes to new tech. But some technical folks in positions of authority see new tech as way to 1) pad their resume ("why yes I've worked with feature X, I helped bring it into my last company"), or 2) make them indispensable, since they adopted it and are the only ones who understand it. When I was a newbie DBA, I knew a senior DBA who did just that - repeatedly. Everybody could see it, but nobody was in a position to do anything about it. Then, he left and the rest of us were put in the position of having to support this big, expensive, shiny nightmare.

Flying along the bleeding edge can be a bumpy ride. Resist the urge to pad your resume at the expense of your employer. Otherwise, your big ol' pile of shiny might become a big ol' pile of something else.
Categories: DBA Blogs

Solving ORA-24172: rule set %s.%s has errors

David Aldridge - Tue, 2011-01-18 09:01
DBMS_Scheduler chains are a joy to use, until they stop being a joy and start being a real pain. I modified the logic of a process, dropping one stored procedure and replacing it with another (instead of writing out a list of files to a control file so that a cron job can scp them […]
Categories: BI & Warehousing

Script for Time Dimension Table

Keith Laker - Mon, 2011-01-17 09:06
Note - This blog post was updated on Nov. 14, 2012 with a new script.  This as been simplified a bit and includes half year.

One of the more common requests I get is a script for creating time dimension tables for Oracle OLAP. The following script will create a time dimension table for a standard calendar. It starts by creating a table with dimension members and labels. The second part of the script fills in end date and time span attributes. The section that creates end date and time span can be easily adapted for completing other calendars (e.g., fiscal) where the members have already been filled in.


--
-- Create time dimension table for a standard calendar year (day, month,
-- quarter, half year and year).
--
-- Drop table.
--
--DROP TABLE time_calendar_dim;
--
-- Create time dimension table for calendar year.
--
-- First day if the next day after TO_DATE('31/12/2010','DD/MM/YYYY').
-- Number of days is set in CONNECT BY level <= 365.
--
-- Values for end date and time span attributes are place holders. They need
-- to be filled in correctly later in this script.
--
CREATE TABLE time_calendar_dim AS
WITH base_calendar AS
  (SELECT CurrDate          AS Day_ID,
    1                       AS Day_Time_Span,
    CurrDate                AS Day_End_Date,
    TO_CHAR(CurrDate,'Day') AS Week_Day_Full,
    TO_CHAR(CurrDate,'DY')  AS Week_Day_Short,
    TO_NUMBER(TRIM(leading '0'
  FROM TO_CHAR(CurrDate,'D'))) AS Day_Num_of_Week,
    TO_NUMBER(TRIM(leading '0'
  FROM TO_CHAR(CurrDate,'DD'))) AS Day_Num_of_Month,
    TO_NUMBER(TRIM(leading '0'
  FROM TO_CHAR(CurrDate,'DDD'))) AS Day_Num_of_Year,
    UPPER(TO_CHAR(CurrDate,'Mon')
    || '-'
    || TO_CHAR(CurrDate,'YYYY')) AS Month_ID,
    TO_CHAR(CurrDate,'Mon')
    || ' '
    || TO_CHAR(CurrDate,'YYYY') AS Month_Short_Desc,
    RTRIM(TO_CHAR(CurrDate,'Month'))
    || ' '
    || TO_CHAR(CurrDate,'YYYY') AS Month_Long_Desc,
    TO_CHAR(CurrDate,'Mon')     AS Month_Short,
    TO_CHAR(CurrDate,'Month')   AS Month_Long,
    TO_NUMBER(TRIM(leading '0'
  FROM TO_CHAR(CurrDate,'MM'))) AS Month_Num_of_Year,
    'Q'
    || UPPER(TO_CHAR(CurrDate,'Q')
    || '-'
    || TO_CHAR(CurrDate,'YYYY'))     AS Quarter_ID,
    TO_NUMBER(TO_CHAR(CurrDate,'Q')) AS Quarter_Num_of_Year,
    CASE
      WHEN TO_NUMBER(TO_CHAR(CurrDate,'Q')) <= 2
      THEN 1
      ELSE 2
    END AS half_num_of_year,
    CASE
      WHEN TO_NUMBER(TO_CHAR(CurrDate,'Q')) <= 2
      THEN 'H'
        || 1
        || '-'
        || TO_CHAR(CurrDate,'YYYY')
      ELSE 'H'
        || 2
        || '-'
        || TO_CHAR(CurrDate,'YYYY')
    END                      AS half_of_year_id,
    TO_CHAR(CurrDate,'YYYY') AS Year_ID
  FROM
    (SELECT level n,
      -- Calendar starts at the day after this date.
      TO_DATE('31/12/2010','DD/MM/YYYY') + NUMTODSINTERVAL(level,'DAY') CurrDate
    FROM dual
      -- Change for the number of days to be added to the table.
      CONNECT BY level <= 365
    )
  )
SELECT day_id,
  day_time_span,
  day_end_date,
  week_day_full,
  week_day_short,
  day_num_of_week,
  day_num_of_month,
  day_num_of_year,
  month_id,
  COUNT(*) OVER (PARTITION BY month_id)    AS Month_Time_Span,
  MAX(day_id) OVER (PARTITION BY month_id) AS Month_End_Date,
  month_short_desc,
  month_long_desc,
  month_short,
  month_long,
  month_num_of_year,
  quarter_id,
  COUNT(*) OVER (PARTITION BY quarter_id)    AS Quarter_Time_Span,
  MAX(day_id) OVER (PARTITION BY quarter_id) AS Quarter_End_Date,
  quarter_num_of_year,
  half_num_of_year,
  half_of_year_id,
  COUNT(*) OVER (PARTITION BY half_of_year_id)    AS Half_Year_Time_Span,
  MAX(day_id) OVER (PARTITION BY half_of_year_id) AS Half_Year_End_Date,
  year_id,
  COUNT(*) OVER (PARTITION BY year_id)    AS Year_Time_Span,
  MAX(day_id) OVER (PARTITION BY year_id) AS Year_End_Date
FROM base_calendar
ORDER BY day_id;
);
--
COMMIT;
Categories: BI & Warehousing

An Axiomatic Approach to Algebra and Other Aspects of Life

Cary Millsap - Fri, 2011-01-14 22:50
Not many days pass that I don’t think a time or two about James R. Harkey. Mr. Harkey was my high school mathematics teacher. He taught me algebra, geometry, analytic geometry, trigonometry, and calculus. What I learned from Mr. Harkey influences—to this day—how I write, how I teach, how I plead a case, how I troubleshoot, .... These are the skills I’ve used to earn everything I own.

Prior to Mr. Harkey’s algebra class, algebra for me just was a morass of tricks to memorize: “Take the constant to the other side...”; “Cancel the common factors...”; “Flip the fraction and multiply...” I could practice for a while and then solve problems just like the ones I had been practicing, by applying memorized transformations to superficial patterns that I recognized, but I didn’t understand what I had been taught to do. Without continual practice, the rules I had memorized would evaporate, and then once more I’d be able to solve only those problems for which I could intuit the answer: “7x + 6 = 20” would have been easy, but “7/x – 6 = 20” would have stumped me. This made, for example, studying for final exams quite difficult.

On the first day of Mr. Harkey’s class, he gave us his rules. First, his strict rules of conduct in the classroom lived up to his quite sinister reputation, which was important. Our studies began with a single 8.5" × 14" sheet of paper that apparently he asked us to label “Properties A” (because that’s what I wrote in the upper right-hand corner; and yes, I still have it). He told us that we could consult this sheet of paper on every homework assignment and every exam he’d give. And here’s how we were to use it: every problem would be executed one step at a time; every step would be written down; and beside every step we would write the name of the rule from Properties A that we invoked to perform that step.

You can still hear us now: Holy cow, that’s going to be a lot of extra work.

Well, that’s how it was going to be. Here’s what each homework and test problem had to look like:


The first few days of class, we spent time reviewing every single item on Properties A. Mr. Harkey made sure we all agreed that each axiom and property was true before we moved on to the real work. He was filling our toolbox.

And then we worked problem after problem after problem.

Throughout the year, we did get to shift gears a few times. Not every ax + b = c problem required fourteen steps all year long. After some sequence of accomplishments (I don’t remember what it was—maybe some set number of ‘A’ grades on homework?), I remember being allowed to write the number of the rule instead of the whole name. (When did you first learn about foreign keys? ☺) Some accomplishments after that, we’d be allowed to combine steps like 3, 4 and 5 into one. But we had to demonstrate a pattern of consistent mastery to earn a privilege like that.

Mr. Harkey taught algebra as most teachers teach geometry or predicate logic. Every problem was a proof, documented one logical step at a time. In Mr. Harkey’s algebra class, your “answer” to a homework problem or test question wasn’t the number that x equals, it was the whole proof of how you arrived at the value of x in your answer. Mr. Harkey wasn’t interested in grading your answers. He was going to grade how you got your answers.

The result? After a whole semester of this, I understood algebra, and I mean thoroughly. You couldn’t make a good grade in Mr. Harkey’s algebra class without creating an intimate comprehension of why algebra works the way it does. Learning that way supplies you for a whole lifetime: I still understand it. I can make dimensioned drawings of the things I’m going to build in my shop. I can calculate the tax implications of my business decisions. I can predict the response time behavior of computer software. I can even help my children with their algebra. Nothing about algebra scares me, because I still understand all the rules.

When I help my boys with their homework, I make them use Mr. Harkey’s axiomatic approach with my own Properties A that I made for them. (I rearranged Mr. Harkey’s rules to better illuminate the symmetries among them. If Mr. Harkey had been handy with the laptop computer, which didn’t exist when I was in school, I imagine he’d have done the same thing.)

Invariably, when my one of boys misses a math problem, it’s for the same stupid reason that I make mistakes in my shop or at work. It’s because he’s tried to do steps in his head instead of writing them all down, and of course he’s accidentally integrated an assumption into his work that’s not true. When you don’t have a neat and orderly audit trail to debug, the only way you can fix your work is to start over, which takes more time (which itself increases frustration levels and degrades learning) and which bypasses perhaps the most important technical skill in all of Life today: the ability to troubleshoot.
Theory: Redoing an n-step math problem instead of learning how to propagate a correction to an error made in step – k through step n is how we get to a society in which our support analysts know only two solutions to any problem: (a) reboot, and (b) reinstall.It’s difficult to teach people the value of mastering the basics. It’s difficult enough with children, and it’s even worse with adults, but great teachers and great coaches understand how important it is. I’m grateful to have met my share, and I love meeting new ones. Actually, I believe my 11-year old son has a baseball practice with one tomorrow. We’ll have to check his blog in about 30 years.

Magical Snapshotty-things

alt.oracle - Thu, 2011-01-13 21:50
Magical Snapshotty-Things

I spent some time with a storage vendor recently. Vendors kill me. No matter what you say, they still cannot for the life of them understand why you are not using their product. And if you are, they are mystified by the fact that you're not using every last bell and whistle. In this case, the vendor was questioning why we weren't using their magical-snapshotty backup solution. Basically the way their backup feature works (similar to most snapshotty type of features) is that when a backup occurs, only the deltas are written out. Then, pointers/vectors (vectors sounds cooler) act as reference points to the delta blocks. If a recovery has to occur, the product is smart enough to The upshot of stuff like this is that recoveries are blazingly fast and the amount of data written is extremely small.

Very attractive - too good to be true right? Maybe a little - which takes me to my conversation with the vendor and my point about the inability of vendors to see past their product.

Me: So, your solution doesn't actually copy the data anywhere, except for the deltas?
Them: Yes, that makes it extremely fast and it uses tiny amounts of space.
Me: Okay, but that means there's not a complete copy of the data on physically separate part of the SAN?
Them: Yes, and it's extremely fast by the way.
Me: Um, yeah. So what if something radical happens? What if you lose more disks in the RAID group than you have parity disks?
Them: --Laughs--. That never happens.
Me: Really? I've seen RAID5 groups where two disks failed simultaneously.
Them: No way. Really?
Me: Yep. I've seen it happen three different times.
Them: --dumbfounded look. crickets chirping--
Me: So, you're willing to sign a form that guarantees that your storage system will never have a failure of that nature?
Them: --exasperated look-- Well, we can't really do that.
Me: Hmm. That's a shame.

In the end, they were probably frustrated with me, and I didn't intend to make them mad, but sometimes a DBA has to call BS on things. There's nothing wrong with their product. It's a very good product and we may end up making use of it in some way. The problem is that they're proceeding from a false assumption: namely, that unlikely failures are impossible failures. They're not.

In my last post, I mentioned that I would talk about the second common problem I see in the DBA world with backups, and that is "shortcuts" – ways to make things better, faster, stronger that ultimately leave you with a noose around your neck. The skeptic in me says, if it sounds too good to be true, it probably is – or at least there are probably some strings attached. If these guys were selling a magical-performance-tuney thing, it would be different. But as a DBA, you need to understand that there is no area where your fannie in the on the line more than the recoverability of the data. If you lose data and can't recover - it's gone - and you may be too.

With all apologies to Harry Potter, the trouble with magic is that it isn't real. Database administration isn't an art – it's a hard, cold science. In the end, there aren't many shortcuts to doing your job. If you're going to use a magical backup solution, you have to be dead sure 1) that you know the exact process as to how you're going to magically recover that data and 2) that you've taken every eventuality into consideration.

So in the end, problem #2 is similar to problem #1. Test things and make sure you know what you're doing. If red flags go up, stop and think. I don't want to see you in the unemployment line.
Categories: DBA Blogs

New paper "Mastering Performance with Extended SQL Trace"

Cary Millsap - Thu, 2011-01-13 11:24
Happy New Year.

It’s been a busy few weeks. I finally have something tangible to show for it: “Mastering Performance with Extended SQL Trace” is the new paper I’ve written for this year’s RMOUG conference. Think of it a 15-page update to chapter 5 of Optimizing Oracle Performance.

There’s lots of new detail in there. Some highlights:
  • How to enable and disable traces, even in un-cooperative applications.
  • How to instrument your application so that tracing the right code path during production operation of your application becomes dead simple.
  • How to make that instrumentation highly scalable (think 100,000+ tps).
  • How timestamps since 10.2 allow you to know your recursive call relationships without guessing.
  • How to create response time profiles for calls and groups of calls, with examples.
  • Why you don’t want to be on Oracle 11g prior to 11.2.0.2.0.
I hope you’ll be able to make productive use of it.

A New Low

David Aldridge - Thu, 2011-01-13 03:30
A new low on the Oracle Technet forums: a person with DBA access who can’t work out how to increase tablespace size. Hopefully not a production system.  
Categories: BI & Warehousing

My first 11g show stopper: Purging the Recyclebin

Jeff Hunter - Tue, 2011-01-11 17:11
I've run into my first issue with 11.2.0.2.  After one of my dbs was upgraded, every time I try to purge the recyclebin, I get an ORA-00600 error: SQL> purge dba_recyclebin; purge dba_recyclebin * ERROR at line 1: ORA-00600: internal error code, arguments: [ktcdso-1], [], [], [], [], [], [], [], [], [], [], [] It doesn't matter if the recyclebin is on or off. I have done about six upgrades and

What’s hot and what’s not in 2011 at Oracle

Andrews Consulting - Tue, 2011-01-11 07:36
 Speculation always seems to be fashionable at the start of a new year.  Why not engage in some regarding where Oracle is likely to focus its attention in 2011?  Before doing so, the obvious observation must be made that Oracle is currently on a roll. It appears to be poised for another year of growth, […]
Categories: APPS Blogs

Tip: Comparing Trace Files

Digital Eagle - Mon, 2011-01-10 22:32
This page has moved.  Please update your links:
http://psst0101.digitaleagle.net/2011/01/10/tip-comparing-trace-files/

Trace files can give a lot of nice troubleshooting information, sometimes too much.  Sometimes, you may have a situation where a problem occurs in one system but not in another.  You could run a trace in both systems, but comparing those trace files is difficult.  Each line has a timestamp, which will be different in each file.

Here is a trick for getting rid of those timestamps.  Here is the source:

sed : Remove first 4 letters in each line

Of course, you need Linux, Unix, or Cygwin for this — something with the sed program.  Here is the command:

sed 's/^.\{51\}//g' "mytrace.tracesql" > modtrace.tracesql

For example, here are the first few lines of a trace file:

PSAPPSRV.248 (181) 	 1-1      10.24.26             Cur#1.248.FSDEV90 RC=0 Dur=0.001000 COM Stmt=SELECT VERSION FROM PSVERSION WHERE OBJECTTYPENAME = 'SYS'
PSAPPSRV.248 (181) 	 1-2      10.24.26    0.003000 Cur#1.248.FSDEV90 RC=0 Dur=0.001000 COM Stmt=SELECT PORTAL_NAME, PORTAL_REFTYPE, PORTAL_OBJNAME FROM PSPRSMDEFN WHERE VERSION > :1 UNION SELECT PORTAL_NAME, PORTAL_REFTYPE, PORTAL_OBJNAME FROM PSPRSMDEL WHERE VERSION > :2
PSAPPSRV.248 (181) 	 1-3      10.24.26    0.000000 Cur#1.248.FSDEV90 RC=0 Dur=0.000000 Bind-1 type=8 length=4 value=214748355

After running the command, here is what it looks like:

Cur#1.248.FSDEV90 RC=0 Dur=0.001000 COM Stmt=SELECT VERSION FROM PSVERSION WHERE OBJECTTYPENAME = 'SYS'
Cur#1.248.FSDEV90 RC=0 Dur=0.001000 COM Stmt=SELECT PORTAL_NAME, PORTAL_REFTYPE, PORTAL_OBJNAME FROM PSPRSMDEFN WHERE VERSION > :1 UNION SELECT PORTAL_NAME, PORTAL_REFTYPE, PORTAL_OBJNAME FROM PSPRSMDEL WHERE VERSION > :2
Cur#1.248.FSDEV90 RC=0 Dur=0.000000 Bind-1 type=8 length=4 value=214748355

Now, you can use a tool like Beyond Compare or Meld to compare the trace files without the timestamp.  If you don’t have a compare tool, look here.

 ~-Downloads_857


Listener Logging in 11g

Jeff Hunter - Mon, 2011-01-10 17:01
By default, the listener logging is not turned on in 11r2.  You have to turn it on by specifying: LOGGING_listener_name=ON in your listener.ora file.

Explorations in Component Interface: Handling Component Interface Errors

Digital Eagle - Fri, 2011-01-07 09:11
This page has moved.  Please update your links:
http://psst0101.digitaleagle.net/2011/01/10/tip-comparing-trace-files/

This is a continuation of the following posts:

The plan this time is to try to see how well we can handle errors.  I have been having trouble with some of my production programs crashing when I have taken all the necessary steps to trap and handle the error.  So, we’ll see how this goes …

Trying a Single Error

First, let’s change that second parameter in our program to true.  This should trigger the part of our code that set’s the birth state to “XX”.  Assuming everything works the way it should, we should catch the Invalid Value message and write it to the log file.  Then, the program should run to completion showing the Program Completed message from the second step.

Selection_831

Again, see the App Engine test post for instructions on how to run the program.  Then, look in the log file generated by the PeopleCode.  As we would expect, we see the Invalid Value messages:

Selection_832

Then, if you look in the program’s standard out log, you see the message as well.  I think this is a bug in the tools because we did trap the message.  If I tell it to only put the message in the message collection, I should be able to do what I want with the message as far as showing it or hiding it:

Selection_833

Keep in mind that we are specifying the collection only option.  If we had chosen “2” or “3” for the messages mode, I would expect the “Invalid Value” message in the standard out, but we are specifying “1”:

Selection_834

Multiple Updates

Now, lets try multiple updates with a single error at the end:

Selection_835

If you want, you can just copy my lines:

updateCI(&emplid, False);
updateCI(&emplid, False);
updateCI(&emplid, False);
updateCI(&emplid, False);
updateCI(&emplid, False);
updateCI(&emplid, False);
updateCI(&emplid, False);
updateCI(&emplid, True);

Both logs look just fine:

Selection_836

The key is that it ran to the end of the PeopleCode hence the “End” message, and it ran all the way through Step 2 hence the “Process Complete” message.

Just to be sure, we should try a few different combinations.  Maybe one with the error on the second to last one:

updateCI(&emplid, False);
updateCI(&emplid, False);
updateCI(&emplid, False);
updateCI(&emplid, False);
updateCI(&emplid, False);
updateCI(&emplid, False);
updateCI(&emplid, True);
updateCI(&emplid, False);

This didn’t cause any problems for me.  So, let’s try one in the middle:

updateCI(&emplid, False);
updateCI(&emplid, False);
updateCI(&emplid, False);
updateCI(&emplid, True);
updateCI(&emplid, False);
updateCI(&emplid, False);
updateCI(&emplid, False);
updateCI(&emplid, False);

Again, this worked fine for me.  So, I wasn’t able to reproduce my problem that I keep seeing in production.

Triggering an “Error()”

So far, our examples haven’t actually triggered an error within the Save.  Let’s try that.

First, the easiest way I could come up with was to add some code to the SavePreChange of our component.  If you open up the Component Interface from your project, you can right click on the Component on the left hand side.  Then, if you choose “View Definition”, it will open the Component.  Then, you can choose View > PeopleCode and navigate to the SavePreChange.

I added the following PeopleCode to the end of the program:

If PERSON.BIRTHSTATE = "CA" Then;
   Error ("California is too far West!");
End-If;

Here is what it looks like:

Selection_837

Then, we need to change the state in our code from “XX” to “CA”.

Selection_838

When our updates look like this:

updateCI(&emplid, True);

We get this in the log:

Selection_839

When our updates look like this:

updateCI(&emplid, False);
updateCI(&emplid, False);
updateCI(&emplid, False);
updateCI(&emplid, True);
updateCI(&emplid, False);
updateCI(&emplid, False);
updateCI(&emplid, False);
updateCI(&emplid, False);

We get this in the log:

Selection_840

So, still no problem!

FieldChange Error

Okay, here is one more shot.  I removed my PeopleCode from the SavePreChange.  Next, I opened the FieldChange on the Birth State and put the PeopleCode in there.

Selection_841

And here is what the log looks like:

Selection_842

Still works right.

Conclusion

In conclusion, my goal was to demonstrate a bug in handling errors.  I think I failed because I wasn’t able to reproduce what I am seeing in production.

The problem I am seeing has to do with multiple updates.  The order matters, too.  We saw one instance where the last update erred, and everything worked fine.  Then on the next run, one of the middle updates erred, and the program stopped.  The weird part is that it finishes the PeopleCode program rather than stopping at the Save or where the error occurred.  Then, it abends the program at the end of that Step rather than finishing the whole App Engine.  So, what I was looking for was for it to not show that “Process Complete” message.

If anyone can figure out how to tweak this to reproduce the problem, please let me know.  I would love to demonstrate it so that maybe Oracle could try to fix the problem.  If nothing else though, maybe someone will find the code examples useful in creating their own programs.


Can You Really Create A Business Logic Layer?

Kenneth Downs - Thu, 2011-01-06 19:21

The past three posts of this little mini-series have gone from a Working definition of business logic to a Rigorous definition of business logic and on to some theorems about business logic. To wrap things up, I'd like to ask the question, is it possible to isolate business logic into a single tier?

Related Reading

There are plenty of opinions out there. For a pretty thorough explanation of how to put everything into the DBMS, check out Toon Koppelaar's description. Mr. Koppelaars has some good material, but you do need to read through his earlier posts to get the definitions of some of his terms. You can also follow his links through to some high quality discussions elsewhere.

Contrasting Mr. Koppelaar's opinion is a piece which does not have nearly the same impact, IMHO, because in Dude, Where's My Business Logic? we get some solid history mixed with normative assertions based on either anecdote or nothing at all. I'm a big believer in anecdote, but when I read a sentence that says, "The database should not have any knowledge of what a customer is, but only of the elements that are used to store a customer." then I figure I'm dealing with somebody who needs to see a bit more of the world.

Starting At the Top: The User Interface

First, let's review that our rigorous definition of business logic includes schema (types and constraints), derived values (timestamps, userstamps, calculations, histories), non-algorithmic compound operations (like batch billing) and algorithmic compound operations, those that require looping in their code. This encompasses everything we might do from the simplest passive things like a constraint that prevents discounts from being over 100% to the most complex hours-long business process, along with everything in between accounted for.

Now I want to start out by using that definition to see a little bit about what is going on in the User Interface. This is not the presentation layer as it is often called but the interaction layer and even the command layer.

Consider an admin interface to a database, where the user is entering or modifying prices for the price list. Now, if the user could enter "Kim Stanley Robinson" as the price, that would be kind of silly, so of course the numeric inputs only allow numeric values. Same goes for dates.

So the foundation of usability for a UI is at very least knowlege of and enforcement of types in the UI layer. Don't be scared off that I am claiming the UI is enforcing anything, we'll get to that a little lower down.

Now consider the case where the user is typing in a discount rate for this or that, and a discount is not allowed to be over 100%. The UI really ought to enforce this, otherwise the user's time is wasted when she enters an invalid value, finishes the entire form, and only then gets an error when she tries to save. In the database world we call this a constraint, so the UI needs to know about constraints to better serve the user.

Now this same user is typing a form where there is an entry for US State. The allowed values are in a table in the database, and it would be nice if the user had a drop-down list, and one that was auto-suggesting as the user typed. Of course the easiest way to do something like this is just make sure the UI form "knows" that this field is a foreign key to the STATES table, so it can generate the list using some generic library function that grabs a couple of columns out of the STATES table. Of course, this kind of lookup thing will be happening all over the place, so it would work well if the UI knew about and enforced foreign keys during entry.

And I suppose the user might at some point be entering a purchase order. The purchase order is automatically stamped with today's date. The user might see it, but not be able to change it, so now our UI knows about system-generated values.

Is this user allowed to delete a customer? If not, the button should either be grayed out or not be there at all. The UI needs to know about and enforce some security.

More About Knowing and Enforcing

So in fact the UI layer not only knows the logic but is enforcing it. It is enforcing it for two reasons, to improve the user experience with date pickers, lists, and so forth, and to prevent the user from entering invalid data and wasting round trips.

And yet, because we cannot trust what comes in to the web server over the wire, we have to enforce every single rule a second time when we commit the data.

You usually do not hear people say that the UI enforces business logic. They usually say the opposite. But the UI does enforce business logic. The problem is, everything the UI enforces has to be enforced again. That may be why we often overlook the fact that it is doing so.

The Application and The Database

Now let's go through the stuff the UI is enforcing, and see what happens in the application and the database.

With respect to type, a strongly typed language will throw an error if the type is wrong, and a weakly typed language is wise to put in a type check anyway. The the DBMS is going to only allow correctly typed values, so, including the UI, type is enforced three times.

With respect to lookups like US state, in a SQL database we always let the server do that with a foreign key, if we know what is good for us. That makes double enforcement for lookups.

So we can see where this is going. As we look at constraints and security and anything else that must be right, we find it will be enforced at least twice, and as much as three times.

You Cannot Isolate What Must be Duplicated

By defining First Order Business Logic, the simplest foundation layer, as including things like types and keys and constraints, we find that the enforcement of this First Order stuff is done 2 or 3 times, but never only once.

This more or less leaves in tatters the idea of a "Business Logic Layer" that is in any way capable of handling all business logic all by its lonesome. The UI layer is completely useless unless it is also enforcing as much logic as possible, and even when we leave the Database Server as the final enforcer of First Order Business Logic (types, constraints, keys), it is still often good engineering to do some checks to prevent expensive wasted trips to the server.

So we are wasting time if we sit around trying to figure out how to get the Business Logic "where it belongs", because it "belongs" in at least two places and sometimes three. Herding the cats into a single pen is a fool's errand, it is at once unnecessary, undesirable, and impossible.

Update: Regular reader Dean Thrasher of Infovark summarizes most of what I'm saying here using an apt industry standard term: Business Logic is a cross-cutting concern.

Some Real Questions

Only when we have smashed the concept that Business Logic can exist in serene isolation in its own layer can we start to ask the questions that would actually speed up development and make for better engineering.

Freed of the illusion of a separate layer, when we look at the higher Third and Fourth Order Business Logic, which always require coding, we can decide where they go based either on engineering or the availability of qualified programmers in particular technologies, but we should not make the mistake of believing they are going where they go because the gods would have it so.

But the real pressing question if we are seeking to create efficient manageable large systems is this: how we distribute the same business logic into 2 or 3 (or more) different places so that it is enforced consistently everywhere. Because a smaller code base is always easier to manage than a large one, and because configuration is always easier than coding, this comes down to meta-data, or if you prefer, a data dictionary. That's the trick that always worked for me.

Is This Only A Matter of Definitions?

Anybody who disagrees with the thesis here has only to say, "Ken, those things are not business logic just because you wrote a blog that says they are. In my world business logic is about code baby!" Well sure, have it your way. After all, the nice thing about definitions is that we can all pick the ones we like.

But these definitions, the theorems I derived on Tuesday, and the multiple-enforcement thesis presented here today should make sense to anbyody struggling with where to put the business logic. That struggle and its frustrations come from the mistake of imposing abstract conceptual responsibilities on each tier instead of using the tiers as each is able to get the job done. Databases are wonderful for type, entity integrity (uniqueness), referential integrity, ACID compliance, and many other things. Use them! Code is often better when the problem at hand cannot be solved with a combination of keys and constraints (Fourth Order Business Logic), but even that code can be put into the DB or in the application.

So beware of paradigms that assign responsibility without compromise to this or that tier. It cannot be done. Don't be afraid to use code for doing things that require structured imperative step-wise operations, and don't be afraid to use the database for what it is good for, and leave the arguments about "where everything belongs" to those with too much time on their hands.

Categories: Development

Great Link: VB.Net posting to PeopleSoft

Digital Eagle - Thu, 2011-01-06 16:09
This page has moved.  Please update your links:
http://psst0101.digitaleagle.net/2011/01/10/tip-comparing-trace-files/

I did some looking into using VB.Net with PeopleSoft.  If you are using VB6, you can look at the Excel to CI macros and find out how to post to an iScript, but VB.Net is different.  Here are some links that might help:


Renault under threat from industrial espionage, intellectual property the target

Simon Thorpe - Thu, 2011-01-06 07:38

renault.jpgLast year we saw news of both General Motors and Ford losing a significant amount of valuable information to competitors overseas. Within weeks of the turn of 2011 we see the European car manufacturer, Renault, also suffering. In a recent news report, French Industry Minister Eric Besson warned the country was facing "economic war" and referenced a serious case of espionage which concerns information pertaining to the development of electric cars.

Renault senior vice president Christian Husson told the AFP news agency that the people concerned were in a "particularly strategic position" in the company. An investigation had uncovered a "body of evidence which shows that the actions of these three colleagues were contrary to the ethics of Renault and knowingly and deliberately placed at risk the company's assets", Mr Husson said.

A source told Reuters on Wednesday the company is worried its flagship electric vehicle program, in which Renault with its partner Nissan is investing 4 billion euros ($5.3 billion), might be threatened. This casts a shadow over the estimated losses of Ford ($50 million) and General Motors ($40 million).

One executive in the corporate intelligence-gathering industry, who spoke on condition of anonymity, said: "It's really difficult to say it's a case of corporate espionage ... It can be carelessness." He cited a hypothetical example of an enthusiastic employee giving away too much information about his job on an online forum.

While information has always been passed and leaked, inadvertently or on purpose, the rise of the Internet and social media means corporate spies or careless employees are now more likely to be found out, he added.

We are seeing more and more examples of where companies like these need to invest in technologies such as Oracle IRM to ensure such important information can be kept under control. It isn't just the recent release of information into the public domain via the Wikileaks website that is of concern, but also the increasing threats of industrial espionage in cases such as these. Information rights management doesn't totally remove the threat, but abilities to control documents no matter where they exist certainly increases the capabilities significantly. Every single time someone opens a sealed document the IRM system audits the activity. This makes identifying a potential source for a leak much easier when you have an absolute record of every person who's had access to the documents.

Oracle IRM can also help with accidental or careless loss. Often people use very sensitive information all the time and forget the importance of handling it correctly. With the ability to protect the information from screen shots and prevent people copy and pasting document information into social networks and other, unsecured documents, Oracle IRM brings a totally new level of information security that would have a significant impact on reducing the risk these organizations face of losing their most valuable information.

last 2010 huzzah

Nuno Souto - Thu, 2011-01-06 05:02
2010 was a bit of a roller coaster... First, the passing of four very good friends and colleagues: - John Alexander Wildgoose, a fellow dba and IT pro as well as my long time golf buddy and friend. - Dr. Paulo Abrantes, a very good adolescence friend I hadn't seen in many years, who quietly passed away recently. I do miss our talks about the future, Paulo. - Noel Vanspaal, my database Noonsnoreply@blogger.com4

Berkeley DB Java Edition 4.1.7

Charles Lamb - Thu, 2011-01-06 02:51

Berkeley DB Java Edition 4.1.7 is a patch release consisting three important fixes. We strongly recommend that users of the 4.1.x upgrade to this release.

These fixes include:

[#19346] - Fix a bug that could cause an EnvironmentFailureException with LOG_FILE_NOT_FOUND during recovery, meaning that the JE environment cannot be opened. We strongly recommend that all applications using JE 4.1.6 upgrade immediately. The bug was introduced in JE 4.1.6 and is not present in earlier releases.

[#19312] - Fixed a bug that prevents using a DPL class converter mutation for a proxy class. Previously, an exception such as the following was thrown:

Exception in thread "main" java.lang.ClassCastException:
com.sleepycat.persist.raw.RawObject cannot be cast to
com.sleepycat.persist.model.PersistentProxy
at com.sleepycat.persist.impl.ProxiedFormat.newInstance(ProxiedFormat.java:91)
at com.sleepycat.persist.impl.RecordInput.readObject(RecordInput.java:174)
at com.sleepycat.persist.impl.ReflectionAccessor$ObjectAccess.read(ReflectionAccessor.java:406)
at com.sleepycat.persist.impl.ReflectionAccessor.readNonKeyFields(ReflectionAccessor.java:285)
...

Thanks to James Li for reporting this on OTN and helping us to identify the problem in the source code.

[#19321] - Fix a bug that caused a hard deadlock when attempting to abort a transaction in one thread, while performing operations using the transaction in another thread. Now, rather than a hard deadlock, an IllegalStateException will be thrown in this circumstance. Thanks to Jervin on OTN for reporting this.


The complete list of changes is in the change log page.
http://download.oracle.com/otndocs/products/berkeleydb/html/je/je-4.1.7_changelog.html

Product documentation can be found at:
http://download.oracle.com/docs/cd/E17277_02/html/index.html

Download the source code including the pre-compiled JAR, complete documentation, and the entire test suite as a single package.

http://download.oracle.com/berkeley-db/je-4.1.7.zip
http://download.oracle.com/berkeley-db/je-4.1.7.tar.gz

Pages

Subscribe to Oracle FAQ aggregator