Feed aggregator

Report Time Execution Prediction with Keras and TensorFlow

Andrejus Baranovski - Thu, 2019-08-01 01:03
The aim of this post is to explain Machine Learning to software developers in hands-on terms. Model is based on a common use case in enterprise systems — predicting wait time until the business report is generated.

Report generation in business applications typically takes time, it can be from a few seconds to minutes. Report generation requires time, because typically it would fetch and process many records, this process needs time. Users often get frustrated, they don’t know how long to wait until the report is finished and may go away by closing browser, etc. If we could inform user, before submitting the report request — how it long it will take to execute it, this would be great usability improvement.

I have implemented Machine Learning model using Keras regression to calculate expected report execution time, based on training data (logged information from the past report executions). Keras is a library which wraps TensorFlow complexity into simple and user-friendly API.

Python source code and training data is available on my GitHub repo. This code is based on Keras tutorial.

Alfresco Clustering – ActiveMQ

Yann Neuhaus - Thu, 2019-08-01 01:00

In previous blogs, I talked about some basis and presented some possible architectures for Alfresco, I talked about the Clustering setup for the Alfresco Repository and the Alfresco Share. In this one, I will work on the ActiveMQ layer. I recently posted something related to the setup of ActiveMQ and some initial configuration. I will therefore extend this topic in this blog with what needs to be done to have a simple Cluster for ActiveMQ. I’m not an ActiveMQ expert, I just started using it a few months ago in relation to Alfresco but still, I learned some things in this timeframe so this might be of some use.

ActiveMQ is a Messaging Server so there are therefore three sides to this component. First, there are Producers which produce messages. These messages are put in the broker’s queue which is the second side and finally there are Consumers which consume the messages from the queue. Producers and Consumers are satellites that are using the JMS broker’s queue: they are both clients. Therefore, in a standalone architecture (one broker), there is no issue because clients will always produce and consume all messages. However, if you start adding more brokers and if you aren’t doing it right, you might have producers talking to a specific broker and consumers talking to another one. To solve that, there are a few things possible:

  • a first solution is to create a Network of Brokers which will allow the different brokers to forward the necessary messages between them. You can see that as an Active/Active Cluster
    • Pros: this allows ActiveMQ to support a huge architecture with potentially hundreds or thousands of brokers
    • Cons: messages are, at any point in time, only owned by one single broker so if this broker goes down, the message is lost (if there is no persistence) or will have to wait for the broker to be restarted (if there is persistence)
  • the second solution that ActiveMQ supports is the Master/Slave one. In this architecture, all messages will be replicated from a Master to all Slave brokers. You can see that as something like an Active/Passive Cluster
    • Pros: messages are always processed and cannot be lost. If the Master broker is going down for any reasons, one of the Slave is instantly taking its place as the new Master with all the previous messages
    • Cons: since all messages are replicated, it’s much harder to support a huge architecture

In case of a Network of Brokers, it’s possible to use either the static or dynamic discovery of brokers:

  • Static discovery: Uses the static protocol to provide a list of all URIs to be tested to discover other connections. E.g.: static:(tcp://mq_n1.domain:61616,tcp://mq_n2.domain:61616)?maxReconnectDelay=3000
  • Dynamic discovery: Uses a multicast discovery agent to check for other connections. This is done using the discoveryUri parameter in the XML configuration file

 

I. Client’s configuration

On the client’s side, using several brokers is very simple since it’s all about using the correct broker URL. To be able to connect to several brokers, you should use the Failover Transport protocol which replaced the Reliable protocol used in ActiveMQ 3. For Alfresco, this broker URL needs to be updated in the alfresco-global.properties file. This is an example for a pretty simple URL with two brokers:

[alfresco@alf_n1 ~]$ cat $CATALINA_HOME/shared/classes/alfresco-global.properties
...
### ActiveMQ
messaging.broker.url=failover:(tcp://mq_n1.domain:61616,tcp://mq_n2.domain:61616)?timeout=3000&randomize=false&nested.daemon=false&nested.dynamicManagement=false
#messaging.username=
#messaging.password=
...
[alfresco@alf_n1 ~]$

 

There are a few things to note. The Failover used above is a transport layer that can be used in combination with any of the other transport methods/protocol. Here it’s used with two TCP protocol. The correct nomenclature is either:

  • failover:uri1,…,uriN
    • E.g.: failover:tcp://mq_n1.domain:61616,tcp://mq_n2.domain:61616 => the simplest broker URL for two brokers with no custom options
  • failover:uri1?URIOptions1,…,uriN?URIOptionsN
    • E.g.: failover:tcp://mq_n1.domain:61616?daemon=false&dynamicManagement=false&trace=false,tcp://mq_n2.domain:61616?daemon=false&dynamicManagement=true&trace=true => a more advanced broker URL with some custom options for each of the TCP protocol URIs
  • failover:(uri1?URIOptions1,…,uriN?URIOptionsN)?FailoverTransportOptions
    • E.g.: failover:(tcp://mq_n1.domain:61616?daemon=false&dynamicManagement=false&trace=false,tcp://mq_n2.domain:61616?daemon=false&dynamicManagement=true&trace=true)?timeout=3000&randomize=false => the same broker URL as above but, in addition, with some Failover Transport options
  • failover:(uri1,…,uriN)?FailoverTransportOptions&NestedURIOptions
    • E.g.: failover:(tcp://mq_n1.domain:61616,tcp://mq_n2.domain:61616)?timeout=3000&randomize=false&nested.daemon=false&nested.dynamicManagement=false&nested.trace=false => since ActiveMQ 5.9, it’s now possible to set the nested URIs options (here the TCP protocol options) at the end of the broker URL, they just need to be preceded by “nested.”. Nested options will apply to all URIs.

There are a lot of interesting parameters, these are some:

  • Failover Transport options:
    • backup=true: initialize and keep a second connection to another broker for faster failover
    • randomize=true: will pick a new URI for the reconnect randomly from the list of URIs
    • timeout=3000: time in ms before timeout on the send operations
    • priorityBackup=true: clients will failover to other brokers in case the “primary” broker isn’t available (that’s always the case) but it will consistently try to reconnect to the “primary” one. It is possible to specify several “primary” brokers with the priorityURIs option (comma separated list)
  • TCP Transport options:
    • daemon=false: specify that ActiveMQ isn’t running in a Spring or Web container
    • dynamicManagement=false: disabling the JMX management
    • trace=false: disabling the tracing

The full list of Failover Transport options is described here and the full list of TCP Transport options here.

II. Messaging Server’s configuration

I believe the simplest setup for Clustering in ActiveMQ is using the Master/Slave setup, that’s what I will talk about here. If you are looking for more information about the Network of Brokers, you can find that here. As mentioned previously, the idea behind the Master/Slave is to replicate somehow the messages to Slave brokers. To do that, there are three possible configurations:

  • Shared File System: use a shared file system
  • JDBC: use a Database Server
  • Replicated LevelDB Store: use a ZooKeeper Server. This has been deprecated in recent versions of ActiveMQ 5 in favour of KahaDB, which is a file-based persistence Database. Therefore, this actually is linked to the first configuration above (Shared File System)

In the scope of Alfresco, you should already have a shared file system as well as a shared Database Server for the Repository Clustering… So, it’s pretty easy to fill the prerequisites for ActiveMQ since you already have them. Of course, you can use a dedicated Shared File System or dedicated Database, that’s up to your requirements.

a. JDBC

For the JDBC configuration, you will need to change the persistenceAdapter to use the dedicated jdbcPersistenceAdapter and create the associated DataSource for your Database. ActiveMQ supports some DBs like Apache Derby, DB2, HSQL, MySQL, Oracle, PostgreSQL, SQLServer or Sybase. You will also need to add the JDBC library at the right location.

[alfresco@mq_n1 ~]$ cat $ACTIVEMQ_HOME/conf/activemq.xml
<beans
  xmlns="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
  http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">
  ...
  <broker xmlns="http://activemq.apache.org/schema/core" brokerName="mq_n1" dataDirectory="${activemq.data}">
    ...
    <persistenceAdapter>
      <jdbcPersistenceAdapter dataDirectory="activemq-data" dataSource="postgresql-ds"/>
    </persistenceAdapter>
    ...
  </broker>
  ...
  <bean id="postgresql-ds" class="org.postgresql.ds.PGPoolingDataSource">
    <property name="serverName" value="db_vip"/>
    <property name="databaseName" value="alfresco"/>
    <property name="portNumber" value="5432"/>
    <property name="user" value="alfresco"/>
    <property name="password" value="My+P4ssw0rd"/>
    <property name="dataSourceName" value="postgres"/>
    <property name="initialConnections" value="1"/>
    <property name="maxConnections" value="10"/>
  </bean>
  ...
</beans>
[alfresco@mq_n1 ~]$

 

b. Shared File System

The Shared File System configuration is, from my point of view, the simplest one to configure but for it to work properly, there are some things to note because you should use a shared file system that supports proper file lock. This means that:

  • you cannot use the Oracle Cluster File System (OCFS/OCFS2) because there is no cluster-aware flock or POSIX locks
  • if you are using NFS v3 or lower, you won’t have automatic failover from Master to Slave because there is no timeout and therefore the lock will never be released. You should therefore use NFS v4 instead

Additionally, you need to share the persistenceAdapter between all brokers but you cannot share the data folder completely otherwise the logs will be overwritten by all brokers (that’s bad but it’s not really an issue) and more importantly, the PID file will also be overwritten which will therefore cause issues to start/stop Slave brokers…

Therefore, configuring properly the Shared File System is all about keeping the “$ACTIVEMQ_DATA” environment variable set to the place where you want the logs and PID files to be stored (i.e. locally) and you need to overwrite the persistenceAdapter path to be on the Shared File System:

[alfresco@mq_n1 ~]$ # Root folder of the ActiveMQ binaries
[alfresco@mq_n1 ~]$ echo $ACTIVEMQ_HOME
/opt/activemq
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ # Location of the logs and PID file
[alfresco@mq_n1 ~]$ echo $ACTIVEMQ_DATA
/opt/activemq/data
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ # Location of the Shared File System
[alfresco@mq_n1 ~]$ echo $ACTIVEMQ_SHARED_DATA
/shared/file/system
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ sudo systemctl stop activemq.service
[alfresco@mq_n1 ~]$ grep -A2 "<persistenceAdapter>" $ACTIVEMQ_HOME/conf/activemq.xml
    <persistenceAdapter>
      <kahaDB directory="${activemq.data}/kahadb"/>
    </persistenceAdapter>
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ # Put the KahaDB into the Shared File System
[alfresco@mq_n1 ~]$ sed -i "s, directory=\"[^\"]*\", directory=\"${ACTIVEMQ_SHARED_DATA}/activemq/kahadb\"," $ACTIVEMQ_HOME/conf/activemq.xml
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ grep -A2 "<persistenceAdapter>" $ACTIVEMQ_HOME/conf/activemq.xml
    <persistenceAdapter>
      <kahaDB directory="/shared/file/system/activemq/kahadb"/>
    </persistenceAdapter>
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ sudo systemctl start activemq.service

 

Starting the Master ActiveMQ will display some information in the log of the node1 showing that it has started properly and it will listen for connections on the different transportConnector:

[alfresco@mq_n1 ~]$ cat $ACTIVEMQ_DATA/activemq.log
2019-07-28 11:34:37,598 | INFO  | Refreshing org.apache.activemq.xbean.XBeanBrokerFactory$1@9f116cc: startup date [Sun Jul 28 11:34:37 CEST 2019]; root of context hierarchy | org.apache.activemq.xbean.XBeanBrokerFactory$1 | main
2019-07-28 11:34:38,289 | INFO  | Using Persistence Adapter: KahaDBPersistenceAdapter[/shared/file/system/activemq/kahadb] | org.apache.activemq.broker.BrokerService | main
2019-07-28 11:34:38,330 | INFO  | KahaDB is version 6 | org.apache.activemq.store.kahadb.MessageDatabase | main
2019-07-28 11:34:38,351 | INFO  | PListStore:[/opt/activemq/data/mq_n1/tmp_storage] started | org.apache.activemq.store.kahadb.plist.PListStoreImpl | main
2019-07-28 11:34:38,479 | INFO  | Apache ActiveMQ 5.15.6 (mq_n1, ID:mq_n1-36925-1564306478360-0:1) is starting | org.apache.activemq.broker.BrokerService | main
2019-07-28 11:34:38,533 | INFO  | Listening for connections at: tcp://mq_n1:61616?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
2019-07-28 11:34:38,542 | INFO  | Connector openwire started | org.apache.activemq.broker.TransportConnector | main
2019-07-28 11:34:38,545 | INFO  | Listening for connections at: amqp://mq_n1:5672?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
2019-07-28 11:34:38,546 | INFO  | Connector amqp started | org.apache.activemq.broker.TransportConnector | main
2019-07-28 11:34:38,552 | INFO  | Listening for connections at: stomp://mq_n1:61613?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
2019-07-28 11:34:38,553 | INFO  | Connector stomp started | org.apache.activemq.broker.TransportConnector | main
2019-07-28 11:34:38,556 | INFO  | Listening for connections at: mqtt://mq_n1:1883?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
2019-07-28 11:34:38,561 | INFO  | Connector mqtt started | org.apache.activemq.broker.TransportConnector | main
2019-07-28 11:34:38,650 | WARN  | ServletContext@o.e.j.s.ServletContextHandler@11841b15{/,null,STARTING} has uncovered http methods for path: / | org.eclipse.jetty.security.SecurityHandler | main
2019-07-28 11:34:38,710 | INFO  | Listening for connections at ws://mq_n1:61614?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.ws.WSTransportServer | main
2019-07-28 11:34:38,712 | INFO  | Connector ws started | org.apache.activemq.broker.TransportConnector | main
2019-07-28 11:34:38,712 | INFO  | Apache ActiveMQ 5.15.6 (mq_n1, ID:mq_n1-36925-1564306478360-0:1) started | org.apache.activemq.broker.BrokerService | main
2019-07-28 11:34:38,714 | INFO  | For help or more information please see: http://activemq.apache.org | org.apache.activemq.broker.BrokerService | main
2019-07-28 11:34:39,118 | INFO  | No Spring WebApplicationInitializer types detected on classpath | /admin | main
2019-07-28 11:34:39,373 | INFO  | ActiveMQ WebConsole available at http://0.0.0.0:8161/ | org.apache.activemq.web.WebConsoleStarter | main
2019-07-28 11:34:39,373 | INFO  | ActiveMQ Jolokia REST API available at http://0.0.0.0:8161/api/jolokia/ | org.apache.activemq.web.WebConsoleStarter | main
2019-07-28 11:34:39,402 | INFO  | Initializing Spring FrameworkServlet 'dispatcher' | /admin | main
2019-07-28 11:34:39,532 | INFO  | No Spring WebApplicationInitializer types detected on classpath | /api | main
2019-07-28 11:34:39,563 | INFO  | jolokia-agent: Using policy access restrictor classpath:/jolokia-access.xml | /api | main
[alfresco@mq_n1 ~]$

 

Then starting a Slave will only display the information on the node2 logs that there is already a Master running and therefore the Slave is just waiting and it’s not listening for now:

[alfresco@mq_n2 ~]$ cat $ACTIVEMQ_DATA/activemq.log
2019-07-28 11:35:53,258 | INFO  | Refreshing org.apache.activemq.xbean.XBeanBrokerFactory$1@9f116cc: startup date [Sun Jul 28 11:35:53 CEST 2019]; root of context hierarchy | org.apache.activemq.xbean.XBeanBrokerFactory$1 | main
2019-07-28 11:35:53,986 | INFO  | Using Persistence Adapter: KahaDBPersistenceAdapter[/shared/file/system/activemq/kahadb] | org.apache.activemq.broker.BrokerService | main
2019-07-28 11:35:53,999 | INFO  | Database /shared/file/system/activemq/kahadb/lock is locked by another server. This broker is now in slave mode waiting a lock to be acquired | org.apache.activemq.store.SharedFileLocker | main
[alfresco@mq_n2 ~]$

 

Finally stopping the Master will automatically transform the Slave into a new Master, without any human interaction. From the node2 logs:

[alfresco@mq_n2 ~]$ cat $ACTIVEMQ_DATA/activemq.log
2019-07-28 11:35:53,258 | INFO  | Refreshing org.apache.activemq.xbean.XBeanBrokerFactory$1@9f116cc: startup date [Sun Jul 28 11:35:53 CEST 2019]; root of context hierarchy | org.apache.activemq.xbean.XBeanBrokerFactory$1 | main
2019-07-28 11:35:53,986 | INFO  | Using Persistence Adapter: KahaDBPersistenceAdapter[/shared/file/system/activemq/kahadb] | org.apache.activemq.broker.BrokerService | main
2019-07-28 11:35:53,999 | INFO  | Database /shared/file/system/activemq/kahadb/lock is locked by another server. This broker is now in slave mode waiting a lock to be acquired | org.apache.activemq.store.SharedFileLocker | main
  # The ActiveMQ Master on node1 has been stopped here (11:37:10)
2019-07-28 11:37:11,166 | INFO  | KahaDB is version 6 | org.apache.activemq.store.kahadb.MessageDatabase | main
2019-07-28 11:37:11,187 | INFO  | PListStore:[/opt/activemq/data/mq_n2/tmp_storage] started | org.apache.activemq.store.kahadb.plist.PListStoreImpl | main
2019-07-28 11:37:11,316 | INFO  | Apache ActiveMQ 5.15.6 (mq_n2, ID:mq_n2-41827-1564306631196-0:1) is starting | org.apache.activemq.broker.BrokerService | main
2019-07-28 11:37:11,370 | INFO  | Listening for connections at: tcp://mq_n2:61616?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
2019-07-28 11:37:11,372 | INFO  | Connector openwire started | org.apache.activemq.broker.TransportConnector | main
2019-07-28 11:37:11,379 | INFO  | Listening for connections at: amqp://mq_n2:5672?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
2019-07-28 11:37:11,381 | INFO  | Connector amqp started | org.apache.activemq.broker.TransportConnector | main
2019-07-28 11:37:11,386 | INFO  | Listening for connections at: stomp://mq_n2:61613?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
2019-07-28 11:37:11,387 | INFO  | Connector stomp started | org.apache.activemq.broker.TransportConnector | main
2019-07-28 11:37:11,390 | INFO  | Listening for connections at: mqtt://mq_n2:1883?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
2019-07-28 11:37:11,391 | INFO  | Connector mqtt started | org.apache.activemq.broker.TransportConnector | main
2019-07-28 11:37:11,485 | WARN  | ServletContext@o.e.j.s.ServletContextHandler@2cfbeac4{/,null,STARTING} has uncovered http methods for path: / | org.eclipse.jetty.security.SecurityHandler | main
2019-07-28 11:37:11,547 | INFO  | Listening for connections at ws://mq_n2:61614?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.ws.WSTransportServer | main
2019-07-28 11:37:11,548 | INFO  | Connector ws started | org.apache.activemq.broker.TransportConnector | main
2019-07-28 11:37:11,556 | INFO  | Apache ActiveMQ 5.15.6 (mq_n2, ID:mq_n2-41827-1564306631196-0:1) started | org.apache.activemq.broker.BrokerService | main
2019-07-28 11:37:11,558 | INFO  | For help or more information please see: http://activemq.apache.org | org.apache.activemq.broker.BrokerService | main
2019-07-28 11:37:11,045 | INFO  | No Spring WebApplicationInitializer types detected on classpath | /admin | main
2019-07-28 11:37:11,448 | INFO  | ActiveMQ WebConsole available at http://0.0.0.0:8161/ | org.apache.activemq.web.WebConsoleStarter | main
2019-07-28 11:37:11,448 | INFO  | ActiveMQ Jolokia REST API available at http://0.0.0.0:8161/api/jolokia/ | org.apache.activemq.web.WebConsoleStarter | main
2019-07-28 11:37:11,478 | INFO  | Initializing Spring FrameworkServlet 'dispatcher' | /admin | main
2019-07-28 11:37:11,627 | INFO  | No Spring WebApplicationInitializer types detected on classpath | /api | main
2019-07-28 11:37:11,664 | INFO  | jolokia-agent: Using policy access restrictor classpath:/jolokia-access.xml | /api | main
[alfresco@mq_n2 ~]$

 

You can of course customize ActiveMQ as per your requirements, remove some connectors, setup SSL, aso… But that’s not really the purpose of this blog.

 

 

Other posts of this series on Alfresco HA/Clustering:

Cet article Alfresco Clustering – ActiveMQ est apparu en premier sur Blog dbi services.

Windows Docker containers, when platform matters

Yann Neuhaus - Wed, 2019-07-31 07:58

A couple of days ago, I got a question from a customer about an issue he ran into when trying to spin up a container on Windows.

The context was as follows:

> docker container run hello-world:nanoserver
Unable to find image 'hello-world:nanoserver' locally
nanoserver: Pulling from library/hello-world
C:\Program Files\Docker\docker.exe: no matching manifest for windows/amd64 10.0.14393 in the manifest list entries.
See 'C:\Program Files\Docker\docker.exe run --help'.

 

I thought that was very interesting because it pointed out some considerations about Docker image architecture design. First, we must bear in mind that containers and the underlying host share a single kernel by design and the container’s base image must match that of the host.

Let’s first begin with containers in a Linux world because it highlights the concept of Kernel sharing between different distros. In this demo, let’s say I’m running a Linux Ubuntu server 16.04 …

$ cat /etc/os-release | grep -i version
VERSION="16.04.6 LTS (Xenial Xerus)"
VERSION_ID="16.04"
VERSION_CODENAME=xenial

 

… and let’s say I want to run a container based on Centos 6.6 …

$ docker run --rm -ti centos:6.6 cat /etc/centos-release
Unable to find image 'centos:6.6' locally
6.6: Pulling from library/centos
5dd797628260: Pull complete
Digest: sha256:32b80b90ba17ed16e9fa3430a49f53ff6de0d4c76ad8631717a1373d5921fa26
Status: Downloaded newer image for centos:6.6
CentOS release 6.6 (Final)

 

You may wonder how it is possible to run different distros between the container and the host and what’s the magic behind the scene? In fact, both the container and the host share the same Linux kernel and even if CentOS 6.6 ships with a kernel version 2.6, while Ubuntu 16.04 ships with 4.4 we usually may upgrade the kernel since it’s backward compatible. The commands below demonstrate the centos container is using the same Kernel than the host.

$ uname -r
4.4.0-142-generic
$ docker run --rm -ti centos:6.6 uname -r
4.4.0-142-generic

 

Let’s say now my docker host is running on the x64 architecture. If we look at the Centos image supported architectures on Docker hub, we notice different ones:

From the output above, we may deduce it should exist a combination of different images and tags for each available architecture and the interesting point is how does Docker pull the correct one regarding my underlying architecture? This is where manifest lists come into play and allow multi-architecture images. A manifest list contains platform segregated references to a single-platform manifest entry. We may inspect a manifest list through the docker manifest command (still in experimental mode at the moment of writing this blog post).

For example, if I want to get a list of manifests and their corresponding architectures for the Centos 7, I can run docker manifest command as follows:

$ docker manifest inspect centos:7 --verbose
[
        {
                "Ref": "docker.io/library/centos:7@sha256:ca58fe458b8d94bc6e3072f1cfbd334855858e05e1fd633aa07cf7f82b048e66",
                "Descriptor": {
                        "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
                        "digest": "sha256:ca58fe458b8d94bc6e3072f1cfbd334855858e05e1fd633aa07cf7f82b048e66",
                        "size": 529,
                        "platform": {
                                "architecture": "amd64",
                                "os": "linux"
                        }
                },
                "SchemaV2Manifest": {
                        "schemaVersion": 2,
                        "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
                        "config": {
                                "mediaType": "application/vnd.docker.container.image.v1+json",
                                "size": 2182,
                                "digest": "sha256:9f38484d220fa527b1fb19747638497179500a1bed8bf0498eb788229229e6e1"
                        },
                        "layers": [
                                {
                                        "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
                                        "size": 75403831,
                                        "digest": "sha256:8ba884070f611d31cb2c42eddb691319dc9facf5e0ec67672fcfa135181ab3df"
                                }
                        ]
                }
        },
        {
                "Ref": "docker.io/library/centos:7@sha256:9fd67116449f225c6ef60d769b5219cf3daa831c5a0a6389bbdd7c952b7b352d",
                "Descriptor": {
                        "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
                        "digest": "sha256:9fd67116449f225c6ef60d769b5219cf3daa831c5a0a6389bbdd7c952b7b352d",
                        "size": 529,
                        "platform": {
                                "architecture": "arm",
                                "os": "linux",
                                "variant": "v7"
                        }
                },
                "SchemaV2Manifest": {
                        "schemaVersion": 2,
                        "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
                        "config": {
                                "mediaType": "application/vnd.docker.container.image.v1+json",
                                "size": 2181,
                                "digest": "sha256:8c52f2d0416faa8009082cf3ebdea85b3bc1314d97925342be83bc9169178efe"
                        },
                        "layers": [
                                {
                                        "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
                                        "size": 70029389,
                                        "digest": "sha256:193bcbf05ff9ae85ac1a58cacd9c07f8f4297dc648808c347cceb3797ae603af"
                                }
                        ]
                }
        },
        {
                "Ref": "docker.io/library/centos:7@sha256:f25f24daae92b5b5fe75bc0d5d9a3d2145906290f25aa434c43bfcefecd10dec",
                "Descriptor": {
                        "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
                        "digest": "sha256:f25f24daae92b5b5fe75bc0d5d9a3d2145906290f25aa434c43bfcefecd10dec",
                        "size": 529,
                        "platform": {
                                "architecture": "arm64",
                                "os": "linux",
                                "variant": "v8"
                        }
                },
                "SchemaV2Manifest": {
                        "schemaVersion": 2,
                        "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
                        "config": {
                                "mediaType": "application/vnd.docker.container.image.v1+json",
                                "size": 2183,
                                "digest": "sha256:7a51de8a65d533b6706fbd63beea13610e5486e49141610e553a3e784c133a37"
                        },
                        "layers": [
                                {
                                        "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
                                        "size": 74163767,
                                        "digest": "sha256:90c48ff53512085fb5adaf9bff8f1999a39ce5e5b897f5dfe333555eb27547a7"
                                }
                        ]
                }
        },
        {
                "Ref": "docker.io/library/centos:7@sha256:1f832b4e3b9ddf67fd77831cdfb591ce5e968548a01581672e5f6b32ce1212fe",
                "Descriptor": {
                        "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
                        "digest": "sha256:1f832b4e3b9ddf67fd77831cdfb591ce5e968548a01581672e5f6b32ce1212fe",
                        "size": 529,
                        "platform": {
                                "architecture": "386",
                                "os": "linux"
                        }
                },
                "SchemaV2Manifest": {
                        "schemaVersion": 2,
                        "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
                        "config": {
                                "mediaType": "application/vnd.docker.container.image.v1+json",
                                "size": 2337,
                                "digest": "sha256:fe70670fcbec5e3b3081c6800cb531002474c36563689b450d678a34a89b62c3"
                        },
                        "layers": [
                                {
                                        "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
                                        "size": 75654099,
                                        "digest": "sha256:39016a8400a36ce04799adba71f8678ae257d9d8dba638d81b8c5755f01fe213"
                                }
                        ]
                }
        },
        {
                "Ref": "docker.io/library/centos:7@sha256:2d9b27e9c89d511a58873254d86ecf96df0f599daae3d555d896fee9f49fedf4",
                "Descriptor": {
                        "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
                        "digest": "sha256:2d9b27e9c89d511a58873254d86ecf96df0f599daae3d555d896fee9f49fedf4",
                        "size": 529,
                        "platform": {
                                "architecture": "ppc64le",
                                "os": "linux"
                        }
                },
                "SchemaV2Manifest": {
                        "schemaVersion": 2,
                        "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
                        "config": {
                                "mediaType": "application/vnd.docker.container.image.v1+json",
                                "size": 2185,
                                "digest": "sha256:c9744f4afb966c58d227eb6ba03ab9885925f9e3314edd01d0e75481bf1c937d"
                        },
                        "layers": [
                                {
                                        "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
                                        "size": 76787221,
                                        "digest": "sha256:deab1c539926c1ca990d5d025c6b37c649bbba025883d4b209e3b52b8fdf514a"
                                }
                        ]
                }
        }
]

 

Each manifest entry contains different information including the image signature digest, the operating system and the supported architecture. Let’s pull the Centos:7 image:

$ docker pull centos:7
7: Pulling from library/centos
8ba884070f61: Pull complete
Digest: sha256:a799dd8a2ded4a83484bbae769d97655392b3f86533ceb7dd96bbac929809f3c
Status: Downloaded newer image for centos:7
docker.io/library/centos:7

 

Let’s have a look at the unique identifier of the centos:7 image:

$ docker inspect --format='{{.Id}}' centos:7sha256:9f38484d220fa527b1fb19747638497179500a1bed8bf0498eb788229229e6e1

 

It corresponds to the SchemaV2Manifest digest value of the manifest entry related to the x64 architecture (please refer to the docker manifest inspect output above). Another official way to query manifest list and architecture is to go through the mplatform/mquery container as follows:

$ docker run mplatform/mquery centos:7
Image: centos:7
 * Manifest List: Yes
 * Supported platforms:
   - linux/amd64
   - linux/arm/v7
   - linux/arm64
   - linux/386
   - linux/ppc64le

 

However, for a Linux Centos 6.6 image (used in my first demo) the architecture support seems to be limited to  the x64 architecture:

$ docker run mplatform/mquery centos:6.6
Image: centos:6.6
 * Manifest List: Yes
 * Supported platforms:
   - linux/amd64

 

Now we are aware of manifest lists and multi-architecture images let’s go back to the initial problem. The customer ran into an platform compatibility issue when trying to spin-up a the hello-world:nanoserver container on a Windows Server 2016 Docker host. As a reminder, the error message was:

no matching manifest for windows/amd64 10.0.14393 in the manifest list entries.

In the way, that may be surprising because Windows host and containers also share a single Kernel. That’s true and it was the root cause of my customer’s issue by the way. The image he wanted to pull supports only the following Windows architecture (queried from the manifest list):

> docker run mplatform/mquery hello-world:nanoserver
Image: hello-world:nanoserver
 * Manifest List: Yes
 * Supported platforms:
   - windows/amd64:10.0.17134.885
   - windows/amd64:10.0.17763.615

 

You may notice several supported Windows platforms but with different operating system versions. Let’s have look at the Docker host version in the context of my customer:

> [System.Environment]::OSVersion.Version
Major  Minor  Build  Revision
-----  -----  -----  --------
10     0      14393  0

 

The tricky part is Windows Server 2016 comes with different branches – 1607/1709 and 1803 – which aren’t technically all the same Windows Server version. Each branch comes with a different build number. Referring to the Microsoft documentation when the build number (3rd column) is changing a new operating system version is published. What it means in that case is the OS version between the Windows Docker host and the Docker image we tried to pull are different hence we experienced this compatibility issue. However let’s precise that images and containers may run with newer versions on the host side but the opposite is not true obviously. You can refer to the same Microsoft link to get a picture of Windows container and host compatibility. 

How to fix this issue? Well, we may go two ways here. The first one consists in re-installing a Docker host platform compatible with the corresponding image. The second one consists in using an image compatible with the current architecture and referring to the hello-world image tags we have one. We may check the architecture compatibility by query the manifest file list as follows:

> docker run mplatform/mquery hello-world:nanoserver-sac2016
Image: hello-world:nanoserver-sac2016
 * Manifest List: Yes
 * Supported platforms:
   - windows/amd64:10.0.14393.2551

 

Let’s try to pull the image with the nanoserver-sac2016 tag:

> docker pull hello-world:nanoserver-sac2016
nanoserver-sac2016: Pulling from library/hello-world
bce2fbc256ea: Already exists
6f2071dcd729: Pull complete
909cdbafc9e1: Pull complete
a43e426cc5c9: Pull complete
Digest: sha256:878fd913010d26613319ec7cc83b400cb92113c314da324681d9fecfb5082edc
Status: Downloaded newer image for hello-world:nanoserver-sac2016
docker.io/library/hello-world:nanoserver-sac2016

 

Here we go!

See you!

 

 

 

 

 

 

Cet article Windows Docker containers, when platform matters est apparu en premier sur Blog dbi services.

Driving vs. Being Driven : The reason you fail to get good at anything!

Tim Hall - Wed, 2019-07-31 01:38

It doesn’t matter how many times I’ve gone somewhere. I only know the route when I’ve driven there myself. Everything makes sense when you see someone else do it. You don’t realise how distracted you are, and how much you’ve missed until you have to do it for yourself.

When we have consultants on site to help us with something new, I assume I’m going to drive and they are going to give directions. I make notes as necessary, but the main thing is *I’ve done it*, not them. If I’m told I have to “observe and make notes”, I say I’m not willing to support it, as experience tells me there will be important stuff that gets missed as the consultant rushes through it. Once again, it’s the difference between driving and being driven.

I’ve written a lot about Learning New Things, and I think it always starts with learning to learn for yourself. If you are always relying on other people to lead the way, they are driving and you are being driven. They are getting better and you are just drifting.

I suppose the obvious retort to this is,

“Only a fool learns from his own mistakes. The wise man learns from the mistakes of others.”

Otto von Bismark

There is some truth in that, but the import thing in the second sentence is the wise person *learns* from the mistakes of others. There is still something active going on here. You are learning, not just being passive and waiting to be told what to do.

Standing on the shoulders of giants requires you to climb up on to the shoulders in the first place!

Cheers

Tim…

Driving vs. Being Driven : The reason you fail to get good at anything! was first posted on July 31, 2019 at 7:38 am.
©2012 "The ORACLE-BASE Blog". Use of this feed is for personal non-commercial use only. If you are not reading this article in your feed reader, then the site is guilty of copyright infringement.

Alfresco Clustering – Share

Yann Neuhaus - Wed, 2019-07-31 01:00

In previous blogs, I talked about some basis and presented some possible architectures for Alfresco and I talked about the Clustering setup for the Alfresco Repository. In this one, I will work on the Alfresco Share layer. Therefore, if you are using another client like a CMIS/REST client or an ADF Application, it won’t work that way, but you might or might not need Clustering at that layer, it depends how the Application is working.

The Alfresco Share Clustering is used only for the caches, so you could technically have multiple Share nodes working with a single Repository or a Repository Cluster without the Share Clustering. For that, you could disable the caches on the Share layer because if you kept it enabled, you would have, eventually, faced issues. Alfresco introduced a Share Clustering which is used to keep the caches in sync, so you don’t have to disable it anymore. When needed, cache invalidation messages are sent from one Share node to all others, that include runtime application properties changes as well as new/existing site/user dashboards changes.

Just like for the Repository part, it’s really easy to setup the Share Clustering so there is really no reasons not to. It’s also using Hazelcast but it’s not based on properties that you need to configure in the alfresco-global.properties (because it’s a Share configuration), this one must be done in an XML file and there is no possibilities to do that in the Alfresco Admin Console, obviously.

All Share configuration/customization are put in the “$CATALINA_HOME/shared/classes/alfresco/web-extension” folder, this one is no exception. There are two possibilities for the Share Clustering communications:

  • Multicast
  • Unicast (TCP-IP in Hazelcast)

 

I. Multicast

If you do not know how many nodes will participate in your Share Cluster or if you want to be able to add more nodes in the future without having to change the previous nodes’ configuration, then you probably want to check and opt for the Multicast option. Just create a new file “$CATALINA_HOME/shared/classes/alfresco/web-extension/custom-slingshot-application-context.xml” and put this content inside it:

[alfresco@share_n1 ~]$ cat $CATALINA_HOME/shared/classes/alfresco/web-extension/custom-slingshot-application-context.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:hz="http://www.hazelcast.com/schema/spring"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
                           http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
                           http://www.hazelcast.com/schema/spring
                           http://www.hazelcast.com/schema/spring/hazelcast-spring-2.4.xsd">

  <hz:topic id="topic" instance-ref="webframework.cluster.slingshot" name="share_hz_test"/>
  <hz:hazelcast id="webframework.cluster.slingshot">
    <hz:config>
      <hz:group name="slingshot" password="Sh4r3_hz_Test_pwd"/>
      <hz:network port="5801" port-auto-increment="false">
        <hz:join>
          <hz:multicast enabled="true" multicast-group="224.2.2.5" multicast-port="54327"/>
          <hz:tcp-ip enabled="false">
            <hz:members></hz:members>
          </hz:tcp-ip>
        </hz:join>
        <hz:interfaces enabled="false">
          <hz:interface></hz:interface>
        </hz:interfaces>
      </hz:network>
    </hz:config>
  </hz:hazelcast>

  <bean id="webframework.cluster.clusterservice" class="org.alfresco.web.site.ClusterTopicService" init-method="init">
    <property name="hazelcastInstance" ref="webframework.cluster.slingshot" />
    <property name="hazelcastTopicName">
      <value>share_hz_test</value>
    </property>
  </bean>

</beans>
[alfresco@share_n1 ~]$

 

In the above configuration, be sure to set a topic name (matching the hazelcastTopicName’s value) as well as a group password that is specific to this environment, so you don’t end-up with a single Cluster with members coming from different environments. For the Share layer, it’s less of an issue than for the Repository layer but still. Be sure also to use a network port that isn’t in use, it will be the port that Hazelcast will bind itself to in the local host. For Alfresco Clustering, we used 5701 so here it’s 5801 for example.

Not much more to say about this configuration, we just enabled the multicast with an IP and a port to be used and we disabled the tcp-ip one.

The interfaces is disabled by default but you can enable it, if you want to. If it’s disabled, Hazelcast will list all local interfaces (127.0.0.1, local_IP1, local_IP2, …) and it will choose one in this list. If you want to force Hazelcast to use a specific local network interface, then enable this section and add that here. In can use the following nomenclature (IP only!):

  • 10.10.10.10: Hazelcast will try to bind on 10.10.10.10 only. If it’s not available, then it won’t start
  • 10.10.10.10-11: Hazelcast will try to bind on any IP within the range 10-11 so in this case 2 IPs: 10.10.10.10 or 10.10.10.11. If you have, let’s say, 5 IPs assigned to the local host and you don’t want Hazelcast to use 3 of these, then specify the ones that it can use and it will pick one from the list. This can also be used to have the same content for the custom-slingshot-application-context.xml on different hosts… One server with IP 10.10.10.10 and a second one with IP 10.10.10.11
  • 10.10.10.* or 10.10.*.*: Hazelcast will try to bind on any IP in this range, this is an extended version of the XX-YY range above

 

For most cases, keeping the interfaces disabled is sufficient since it will just pick one available. You might think that Hazelcast may bind itself to 127.0.0.1, technically it’s possible since it’s a local network interface but I have never seen it do so, so I assume that there is some kind of preferred order if another IP is available.

Membership in Hazelcast is based on “age”, meaning that the oldest member will be the one to lead. There is no predefined Master or Slave members, they are all equal, but the oldest/first member is the one that will check if new members are allowed to join (correct config) and if so, it will send the information to all other members that joined already so they are all aligned. If multicast is enabled, a multicast listener is started to listen for new membership requests.

 

II. Unicast

If you already know how many nodes will participate in your Share Cluster or if you prefer to avoid Multicast messages (there is no real need to overload your network with such things…), then it’s preferable to use Unicast messaging. For that purpose, just create the same file as above (“$CATALINA_HOME/shared/classes/alfresco/web-extension/custom-slingshot-application-context.xml“) but instead, use the tcp-ip section:

[alfresco@share_n1 ~]$ cat $CATALINA_HOME/shared/classes/alfresco/web-extension/custom-slingshot-application-context.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:hz="http://www.hazelcast.com/schema/spring"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
                           http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
                           http://www.hazelcast.com/schema/spring
                           http://www.hazelcast.com/schema/spring/hazelcast-spring-2.4.xsd">

  <hz:topic id="topic" instance-ref="webframework.cluster.slingshot" name="share_hz_test"/>
  <hz:hazelcast id="webframework.cluster.slingshot">
    <hz:config>
      <hz:group name="slingshot" password="Sh4r3_hz_Test_pwd"/>
      <hz:network port="5801" port-auto-increment="false">
        <hz:join>
          <hz:multicast enabled="false" multicast-group="224.2.2.5" multicast-port="54327"/>
          <hz:tcp-ip enabled="true">
            <hz:members>share_n1.domain,share_n2.domain</hz:members>
          </hz:tcp-ip>
        </hz:join>
        <hz:interfaces enabled="false">
          <hz:interface></hz:interface>
        </hz:interfaces>
      </hz:network>
    </hz:config>
  </hz:hazelcast>

  <bean id="webframework.cluster.clusterservice" class="org.alfresco.web.site.ClusterTopicService" init-method="init">
    <property name="hazelcastInstance" ref="webframework.cluster.slingshot" />
    <property name="hazelcastTopicName">
      <value>share_hz_test</value>
    </property>
  </bean>

</beans>
[alfresco@share_n1 ~]$

 

The description is basically the same as for the Multicast part. The main difference is that the multicast was disabled, the tcp-ip was enabled and there is therefore a list of members that needs to be set. This is a comma separated list of hostname or IPs that the Hazelcast will try to contact when it starts. Membership in case of Unicast is managed in the same way except that the oldest/first member will listen for new membership requests on the TCP-IP. Therefore, it’s the same principle, it’s just done differently.

Starting the first Share node in the Cluster will display the following information on the logs:

Jul 28, 2019 11:45:35 AM com.hazelcast.impl.AddressPicker
INFO: Resolving domain name 'share_n1.domain' to address(es): [127.0.0.1, 10.10.10.10]
Jul 28, 2019 11:45:35 AM com.hazelcast.impl.AddressPicker
INFO: Resolving domain name 'share_n2.domain' to address(es): [10.10.10.11]
Jul 28, 2019 11:45:35 AM com.hazelcast.impl.AddressPicker
INFO: Interfaces is disabled, trying to pick one address from TCP-IP config addresses: [share_n1.domain/10.10.10.10, share_n2.domain/10.10.10.11, share_n1.domain/127.0.0.1]
Jul 28, 2019 11:45:35 AM com.hazelcast.impl.AddressPicker
INFO: Prefer IPv4 stack is true.
Jul 28, 2019 11:45:35 AM com.hazelcast.impl.AddressPicker
INFO: Picked Address[share_n1.domain]:5801, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5801], bind any local is true
Jul 28, 2019 11:45:36 AM com.hazelcast.system
INFO: [share_n1.domain]:5801 [slingshot] Hazelcast Community Edition 2.4 (20121017) starting at Address[share_n1.domain]:5801
Jul 28, 2019 11:45:36 AM com.hazelcast.system
INFO: [share_n1.domain]:5801 [slingshot] Copyright (C) 2008-2012 Hazelcast.com
Jul 28, 2019 11:45:36 AM com.hazelcast.impl.LifecycleServiceImpl
INFO: [share_n1.domain]:5801 [slingshot] Address[share_n1.domain]:5801 is STARTING
Jul 28, 2019 11:45:36 AM com.hazelcast.impl.TcpIpJoiner
INFO: [share_n1.domain]:5801 [slingshot] Connecting to possible member: Address[share_n2.domain]:5801
Jul 28, 2019 11:45:36 AM com.hazelcast.nio.SocketConnector
INFO: [share_n1.domain]:5801 [slingshot] Could not connect to: share_n2.domain/10.10.10.11:5801. Reason: ConnectException[Connection refused]
Jul 28, 2019 11:45:37 AM com.hazelcast.nio.SocketConnector
INFO: [share_n1.domain]:5801 [slingshot] Could not connect to: share_n2.domain/10.10.10.11:5801. Reason: ConnectException[Connection refused]
Jul 28, 2019 11:45:37 AM com.hazelcast.impl.TcpIpJoiner
INFO: [share_n1.domain]:5801 [slingshot]

Members [1] {
        Member [share_n1.domain]:5801 this
}

Jul 28, 2019 11:45:37 AM com.hazelcast.impl.LifecycleServiceImpl
INFO: [share_n1.domain]:5801 [slingshot] Address[share_n1.domain]:5801 is STARTED
2019-07-28 11:45:37,164  INFO  [web.site.ClusterTopicService] [localhost-startStop-1] Init complete for Hazelcast cluster - listening on topic: share_hz_test

 

Then starting a second node of the Share Cluster will display the following (still on the node1 logs):

Jul 28, 2019 11:48:31 AM com.hazelcast.nio.SocketAcceptor
INFO: [share_n1.domain]:5801 [slingshot] 5801 is accepting socket connection from /10.10.10.11:34191
Jul 28, 2019 11:48:31 AM com.hazelcast.nio.ConnectionManager
INFO: [share_n1.domain]:5801 [slingshot] 5801 accepted socket connection from /10.10.10.11:34191
Jul 28, 2019 11:48:38 AM com.hazelcast.cluster.ClusterManager
INFO: [share_n1.domain]:5801 [slingshot]

Members [2] {
        Member [share_n1.domain]:5801 this
        Member [share_n2.domain]:5801
}

 

 

Other posts of this series on Alfresco HA/Clustering:

Cet article Alfresco Clustering – Share est apparu en premier sur Blog dbi services.

ServiceManager … Manually start/stop

DBASolved - Tue, 2019-07-30 09:58

Oracle GoldenGate Microservices, starting in 12c (12.3.0.0.1) through 19c (19.1.0.0.1), provide a set of services that you can interact with via a webpage, command line, REST API, and PL/SQL. All of which is great; however, for any of these items to work the ServiceManager has to be up and running.

There are three ways configure ServiceManager when an environment is initally setup. These three ways are:

  • Manually
  • As a daemon
  • Integration with XAG agent (9.1 or later)

For this post, I’ll just show you how to start or stop ServiceManager manually. Manually starting or stopping the ServiceManager is the default setting if you do not select either of the other two options while running Oracle GoldenGate Configuration Assistant (OGGCA.sh).

In order to start or stop the ServiceManager manually, you have to make sure you have two files. These files are:

  • startSM.sh
  • stopSM.sh

Both of these files will be in the $DEPLOYMENT_HOME/bin directory for the ServiceManager. On my system this location is:

/opt/app/oracle/gg_deployments/ServiceManager/bin

Note: If you are running ServiceManager as a daemon, you will not have these files. In the bin directory you will find a file that is used to register ServiceManager as a daemon.

Before you can start or stop the ServiceManager manually, there are two (2) environment variables that need to be set. These environment variables are:

  • OGG_ETC_HOME
  • OGG_VAR_HOME

These environment variables are set to the etc and var directory locations for the ServiceManager deployment. On my system these are set to:

export OGG_ETC_HOME=/opt/app/oracle/gg_deployments/ServiceManager/etc
export OGG_VAR_HOME=/opt/app/oracle/gg_deployments/ServiceManager/var

Now with all these requirements met, I can now go back to the $DEPLOYMENT_HOME/bin directory and start or stop the ServiceManager.

[oracle@ogg19c bin]$ cd /opt/app/oracle/gg_deployments/ServiceManager/bin
[oracle@ogg19c bin]$ sh ./startSM.sh

[oracle@ogg19c bin]$ sh ./startSM.sh
Starting Service Manager process…
Service Manager process started (PID: 376)

In order to stop the ServiceManager manually:

[oracle@ogg19c bin]$ cd /opt/app/oracle/gg_deployments/ServiceManager/bin
[oracle@ogg19c bin]$ sh ./stopSM.sh
Stopping Service Manager process (PID: 376)…
Service Manager stopped

Enjoy!!!

Categories: DBA Blogs

How to add cells in a Column on Excel

VitalSoftTech - Tue, 2019-07-30 09:53

Are you just starting to learn Excel? Are all these boxes confusing you? Adding is the most primary and straightforward function of Excel. Here is a simple and easy guide to help you with different methods of finding the sum of cells in columns. So, here we go! How to add in a Column on […]

The post How to add cells in a Column on Excel appeared first on VitalSoftTech.

Categories: DBA Blogs

Announcement: “Oracle Performance Diagnostics and Tuning” Webinar – 20-23 August 2019.

Richard Foote - Tue, 2019-07-30 06:05
I have just scheduled a new “Oracle Performance Diagnostics and Tuning” webinar to run between 20-23 August 2019, specifically for my friends in New Zealand, but is open to anyone. It will run between 7am-11am AEST each day, which is perfect for those in NZ, but also other parts of the world such as Western […]
Categories: DBA Blogs

PeopleSoft UI Showcase

Duncan Davies - Tue, 2019-07-30 06:00

While commuting this morning I listened to Dan & Kyle’s podcast with Matthew Haavisto and they discussed that fact that the Fluid UI has been around for a while (since Tools 8.54, which was more than 5 years ago). It struck me that most clients will have adopted Fluid by now and branding is getting easier so there should be lots of good examples out there.

My mind drifted back to the company I worked for previously where we had a ‘UI Showcase’ of PeopleSoft branding eye-candy to drop into presentations to get people dreaming of the ‘art of the possible’.

The UI Showcase really started when we wanted to share the work we did for Pret a Manger, the high-street retailer. It looks really dated now, but back in 2009 it was pretty exciting.

The burgundy tabs across the top were role-based homepages, the chunky white buttons at the top were tiles that accessed Nav Collections and the semi-transparent alert boxes over the lower section of the image were alerts where the stars changed depending upon different PS query thresholds. Much of this has parallels to the Fluid UI we have today.

On the Podcast, Matthew said that he’d seen some pretty creative uses of Fluid UI but I’m not sure many of those customers who’ve really pushed the boundaries have shared their work publicly. I’d really like to see them, and I bet many others would too. Maybe we could do something similar to a PS UI Showcase again, but on a wider scale?

I thought perhaps I’d start a series of posts, each highlighting a customer who has done something impressive and eye-catching with Fluid might be fun. Then the community would have something to flick through for inspiration when designing your latest refresh. Get in touch (duncan@peoplesofttipster.com) if you’re proud of your Fluid deployment – it can be anything, a cool tile, a great overall design, etc – and we’ll sort out how to feature you.

MobaXterm & KeePass Updates

Tim Hall - Tue, 2019-07-30 02:52

Yesterday I noticed some updates to a couple of tools I use all the time.

MobaXterm 12.0

Followers of the blog know I’ve had a long term love affair with MobaXterm. If you are using Windows and connect to servers using SSH, this is the best tool I’ve come across.

Downloads and Changelog are in the usual places.

KeePass 2.42.1

I use KeePass as my password manager.

Downloads and Changelog are in the usual places.

You can read about how I use KeePass and KeePassXC on my Windows, Mac and Android devices here.

Cheers

Tim…

MobaXterm & KeePass Updates was first posted on July 30, 2019 at 8:52 am.
©2012 "The ORACLE-BASE Blog". Use of this feed is for personal non-commercial use only. If you are not reading this article in your feed reader, then the site is guilty of copyright infringement.

Java JDK12: JEP 325: Switch Expressions

Yann Neuhaus - Tue, 2019-07-30 02:25
Eclipse setup

In order to test Java JDK12 you will have to download Eclipse 4.11 at least. Then download JDK12 from Oracle’s web site. And configure Eclipse to use this JDK for the project. Also download the JDK12 support with the following repository link:

https://download.eclipse.org/eclipse/updates/4.11-P-builds

Edit the projects properties to use the “Java Compiler” compliance 12. And “Enable preview features” set to TRUE. Now you should be able to run JDK12 examples. Here a link for the complete setup:

https://marketplace.eclipse.org/content/java-12-support-eclipse-2019-03-411

Switch expression

The switch expression receives a new lifting in this version. It allows more readiness and more flexibility. We can now:

  • Get rid of the break word in certain cases
  • Return a value from the switch
  • Put several values in one “case”

With the following example you can see how to get rid of the “break” and use a “case L ->” switch label. In addition, the switch returns a value which can be used after the expression.

String alias = switch (day) {
    case MONDAY, TUESDAY, WEDNESDAY, THURSDAY, FRIDAY 	-> "Week day";
    case SATURDAY, SUNDAY				                -> "Weekend";
};
System.out.println(alias);

The previous few lines replaces the following ones:

switch (day) {
    case MONDAY:
        System.out.println("Week day");
        break;
    case TUESDAY:
        System.out.println("Week day");
        break;
    case WEDNESDAY:
        System.out.println("Week day");
        break;
    case THURSDAY:
        System.out.println("Week day");
        break;
    case FRIDAY:
        System.out.println("Week day");
        break;
    case SATURDAY:
        System.out.println("Weekend");
        break;
    case SUNDAY:
        System.out.println("Weekend");
        break;
}

Here the expression:

T result = switch (arg) {
    case L1 -> e1;
    case L2 -> e2;
    default -> e3;
};

When using a block of code you will still need a “break”:

int i = switch (day) {
     case MONDAY -> {
     System.out.println("Monday");
     break 0; //break is needed here to return a value if switch need a return
     }
     default -> 1;
}

Thanks to this new notation, we can now return values with a SWITCH expression. In addition it makes it more readable fo big switch expressions. Note that you can still use the “old” way, thus old code will still compile with the new JDK. For the moment it’s deployed in JDK12 which is a non-LTS release, but if the new expression is validated by the tests and results, it will definitely be part of future releases like the LTS versions and you will be able to use it then.

Cet article Java JDK12: JEP 325: Switch Expressions est apparu en premier sur Blog dbi services.

Alfresco Clustering – Repository

Yann Neuhaus - Tue, 2019-07-30 01:00

In a previous blog, I talked about some basis and presented some possible architectures for Alfresco. Now that this introduction has been done, let’s dig into the real blogs about how to setup a HA/Clustering Alfresco environment. In this blog in particular, I will talk about the Repository layer.

For the Repository Clustering, there are three prerequisites (and that’s all you need):

  • A valid license which include the Repository Clustering
  • A shared file system which is accessible from all Alfresco nodes in the Cluster. This is usually a NAS accessed via NFS
  • A shared database

 

Clustering the Repository part is really simple to do: you just need to put the correct properties in the alfresco-global.properties file. Of course, you could also manage it all from the Alfresco Admin Console but that’s not recommended, you should really always use the alfresco-global.properties by default. The Alfresco Repository Clustering is using Hazelcast. It was using JGroups and EHCache as well before Alfresco 4.2 but now it’s just Hazelcast. So to define an Alfresco Cluster, simply put the following configuration in the alfresco-global.properties of the Alfresco node1:

[alfresco@alf_n1 ~]$ getent hosts `hostname` | awk '{ print $1 }'
10.10.10.10
[alfresco@alf_n1 ~]$
[alfresco@alf_n1 ~]$ cat $CATALINA_HOME/shared/classes/alfresco-global.properties
...
### Content Store
dir.root=/shared_storage/alf_data
...
### DB
db.username=alfresco
db.password=My+P4ssw0rd
db.name=alfresco
db.host=db_vip
## MySQL
#db.port=3306
#db.driver=com.mysql.jdbc.Driver
#db.url=jdbc:mysql://${db.host}:${db.port}/${db.name}?useUnicode=yes&characterEncoding=UTF-8
#db.pool.validate.query=SELECT 1
## PostgreSQL
db.driver=org.postgresql.Driver
db.port=5432
db.url=jdbc:postgresql://${db.host}:${db.port}/${db.name}
db.pool.validate.query=SELECT 1
## Oracle
#db.driver=oracle.jdbc.OracleDriver
#db.port=1521
#db.url=jdbc:oracle:thin:@${db.host}:${db.port}:${db.name}
#db.pool.validate.query=SELECT 1 FROM DUAL
...
### Clustering
alfresco.cluster.enabled=true
alfresco.cluster.interface=10.10.10.10-11
alfresco.cluster.nodetype=Alfresco_node1
alfresco.hazelcast.password=Alfr3sc0_hz_Test_pwd
alfresco.hazelcast.port=5701
alfresco.hazelcast.autoinc.port=false
alfresco.hazelcast.max.no.heartbeat.seconds=15
...
[alfresco@alf_n1 ~]$

 

And for the Alfresco node2, you can use the same content:

[alfresco@alf_n2 ~]$ getent hosts `hostname` | awk '{ print $1 }'
10.10.10.11
[alfresco@alf_n2 ~]$
[alfresco@alf_n2 ~]$ cat $CATALINA_HOME/shared/classes/alfresco-global.properties
...
### Content Store
dir.root=/shared_storage/alf_data
...
### DB
db.username=alfresco
db.password=My+P4ssw0rd
db.name=alfresco
db.host=db_vip
## MySQL
#db.port=3306
#db.driver=com.mysql.jdbc.Driver
#db.url=jdbc:mysql://${db.host}:${db.port}/${db.name}?useUnicode=yes&characterEncoding=UTF-8
#db.pool.validate.query=SELECT 1
## PostgreSQL
db.driver=org.postgresql.Driver
db.port=5432
db.url=jdbc:postgresql://${db.host}:${db.port}/${db.name}
db.pool.validate.query=SELECT 1
## Oracle
#db.driver=oracle.jdbc.OracleDriver
#db.port=1521
#db.url=jdbc:oracle:thin:@${db.host}:${db.port}:${db.name}
#db.pool.validate.query=SELECT 1 FROM DUAL
...
### Clustering
alfresco.cluster.enabled=true
alfresco.cluster.interface=10.10.10.10-11
alfresco.cluster.nodetype=Alfresco_node2
alfresco.hazelcast.password=Alfr3sc0_hz_Test_pwd
alfresco.hazelcast.port=5701
alfresco.hazelcast.autoinc.port=false
alfresco.hazelcast.max.no.heartbeat.seconds=15
...
[alfresco@alf_n2 ~]$

 

Description of the Clustering parameters:

  • alfresco.cluster.enabled: Whether or not you want to enable the Repository Clustering for the local Alfresco node. The default value is false. You will want to set that to true for all Repository nodes that will be used by Share or any other client. If the Repository is only used for Solr Tracking, you can leave that to false
  • alfresco.cluster.interface: This is the network interface on which Hazelcast will listen for Clustering messages. This has to be an IP, it can’t be a hostname. To keep things simple and to have the same alfresco-global.properties on all Alfresco nodes however, it is possible to use a specific nomenclature:
    • 10.10.10.10: Hazelcast will try to bind on 10.10.10.10 only. If it’s not available, then it won’t start
    • 10.10.10.10-11: Hazelcast will try to bind on any IP within the range 10-11 so in this case 2 IPs: 10.10.10.10 or 10.10.10.11. If you have, let’s say, 4 IPs assigned to the local host and you don’t want Hazelcast to use 2 of these, then specify the ones that it can use and it will pick one from the list. This can also be used to have the same content for the alfresco-global.properties on different hosts… One server with IP 10.10.10.10 and a second one with IP 10.10.10.11
    • 10.10.10.* or 10.10.*.*: Hazelcast will try to bind on any IP in this range, this is an extended version of the XX-YY range above
  • alfresco.cluster.nodetype: A human-friendly string to represent the local Alfresco node. It doesn’t have any use for Alfresco, that’s really more for you. It is for example interesting to put a specific string for Alfresco node that won’t take part in the Clustering but that are still using the same Content Store and Database (like a Repository dedicated for the Solr Tracking, as mentioned above)
  • alfresco.hazelcast.password: The password to use for the Alfresco Repository Cluster. You need to use the same password for all members of the same Cluster. You should as well try to use a different password for each Cluster that you might have if they are in the same network (DEV/TEST/PROD for example), otherwise it will get ugly
  • alfresco.hazelcast.port: The default port that will be used for Clustering messages between the different members of the Cluster
  • alfresco.hazelcast.autoinc.port: Whether or not you want to allow Hazelcast to find another free port in case the default port (“alfresco.hazelcast.port”) is currently used. It will increment the port by 1 each time. You should really set this to false and just use the default port, to have full control over the channels that Clustering communications are using otherwise it might get messy as well
  • alfresco.hazelcast.max.no.heartbeat.seconds: The maximum time in seconds allowed between two heartbeat. If there is no heartbeat in this period of time, Alfresco will assume the remote node isn’t running/available

 

As you can see above, it’s really simple to add Clustering to an Alfresco Repository. Since you can(should?) have the same set of properties (except the nodetype string maybe), then it also really simplifies the deployment… If you are familiar with other Document Management System like Documentum for example, then you understand the complexity of some of these solutions! If you compare that to Alfresco, it’s like walking on the street versus walking on the moon where you obviously first need to go to the moon… Anyway, once it’s done, the logs of the Alfresco Repository node1 will display something like that when you start it:

2019-07-20 15:14:25,401  INFO  [cluster.core.ClusteringBootstrap] [localhost-startStop-1] Cluster started, name: MainRepository-<generated_id>
2019-07-20 15:14:25,405  INFO  [cluster.core.ClusteringBootstrap] [localhost-startStop-1] Current cluster members:
  10.10.10.10:5701 (hostname: alf_n1)

 

Wait for the Repository node1 to be fully started and once done, you can start the Repository node2, it needs to be started sequentially normally. You will see on the logs of the Repository node1 that another node joined automatically the Cluster:

2019-07-20 15:15:06,528  INFO  [cluster.core.MembershipChangeLogger] [hz._hzInstance_1_MainRepository-<generated_id>.event-3] Member joined: 10.10.10.11:5701 (hostname: alf_n2)
2019-07-20 15:15:06,529  INFO  [cluster.core.MembershipChangeLogger] [hz._hzInstance_1_MainRepository-<generated_id>.event-3] Current cluster members:
  10.10.10.10:5701 (hostname: alf_n1)
  10.10.10.11:5701 (hostname: alf_n2)

 

On the logs of the Repository node2, you can see directly at the initialization of the Hazelcast Cluster that the two nodes are available.

If you don’t want to check the logs, you can see pretty much the same thing from the Alfresco Admin Console. By accessing “http(s)://<hostname>:<port>/alfresco/s/enterprise/admin/admin-clustering“, you can see currently available cluster members (online nodes), non-available cluster members (offline nodes) as well as connected non-cluster members (nodes using the same DB & Content Store but with “alfresco.cluster.enabled=false”, for example to dedicate a Repository to Solr Tracking).

Alfresco also provides a small utility to check the health of the cluster which will basically ensure that the communication between each member is successful. This utility can be accessed at “http(s)://<hostname>:<port>/alfresco/s/enterprise/admin/admin-clustering-test“. It is useful to include a quick check using this utility in a monitoring solution for example, to ensure that the cluster is healthy.

 

Cet article Alfresco Clustering – Repository est apparu en premier sur Blog dbi services.

Docker : New Builds Using Oracle Linux 8 (oraclelinux:8-slim)

Tim Hall - Tue, 2019-07-30 00:41

Yesterday I noticed the oraclelinux section on Docker Hub included “oraclelinux:8-slim”, so when I got home a did a quick run through some builds using it.

  • ol8_ords : This build is based on “oraclelinux:8-slim” and includes OpenJDK 12, Tomcat 9, ORDS 19, SQLcl 19 and the APEX 19 images.
  • ol8_19 : This build is based on “oraclelinux:8-slim” and includes the 19c database and APEX 19.
  • ol8_183 : This build is based on “oraclelinux:8-slim” and includes the 18c database and APEX 19.

There are also some new compose files, so I could test database and ORDS containers working together.

Everything worked fine, but here come the inevitable warnings and comments.

  • The Oracle database is not certified on Oracle Linux 8 yet, so the database builds are just for playing around, not a recommendation.
  • The database preinstall packages don’t exist yet, so I installed the main required packages with DNF, but I didn’t do some of the additional manual setup I would normally do, so it’s not a perfect example of an installation. I assume the preinstall packages will eventually be released, and I will substitute them in.
  • The ORDS build is not subject to the same certification restrictions as the database, so as far as I know, I could consider using this, although the build I use for work differs a little to this and is still using Oracle JDK 8 and Tomcat 8.5.

If you are interested in playing around with Docker, you can find my articles on it here, and my public builds here.

Cheers

Tim…

Docker : New Builds Using Oracle Linux 8 (oraclelinux:8-slim) was first posted on July 30, 2019 at 6:41 am.
©2012 "The ORACLE-BASE Blog". Use of this feed is for personal non-commercial use only. If you are not reading this article in your feed reader, then the site is guilty of copyright infringement.

[Video] Oracle Exadata Cloud Service (ExaCS) Networking

Online Apps DBA - Mon, 2019-07-29 20:24

[Video] Oracle Exadata Cloud Service (ExaCS) Networking Before creating an instance of Oracle Exadata Cloud Service one needs to create a VCN and two Subnets namely Client Subnet and Backup Subnet. 1. What is VCN? 2. What are the options for creating VCN? 3. How many IP addresses are required for each subnet? 4. How […]

The post [Video] Oracle Exadata Cloud Service (ExaCS) Networking appeared first on Oracle Trainings for Apps & Fusion DBA.

Categories: APPS Blogs

CloudBees DevOps Playground – Hands On with JenkinsX

Yann Neuhaus - Mon, 2019-07-29 16:32

Last week, we had the chance to attend to the CloudBees DevOps Playground in London. The event was a presentation and a Hands-On on Jenkins X done by one of the most popular guys from the CloudBees, Gareth Evans.

Before taking an interest in Jenkins X, we focused most of our time in the Docker and Kubernetes part. We enhance a lot our skills during the last months on the administration of Kubernetes cluster and the deployment of applications, especially the Documentum stack as well WebLogic.

Jenkins X is quite a new technology in the landscape of automatic deployment, and we face the difficulties to find a workshop /training related to Jenkins X administration and usage. So we decided to go to London for this CloudBees DevOps Hands-on done.

As working in middleware infrastructures between system engineer and applications, Jenkins X is completely making sense for us to automate the creation of Kubernetes infrastructure in terms of cluster and application deployment.

What’s Jenkins X?

Basically, Jenkins X automates the whole development process end to end for containerized applications based on Docker and Kubernetes.

Overview of Jenkins X:

  • Jenkins X provides an automated CI/CD solution for Kubernetes
  • Buildpacks to quickly create new applications
  • Uses GitOps to manage promotion between Environments
  • Creates Preview Environments on Pull Requests
  • Provides control via ChatOps and feedback on Pull Requests
  • Improves developers’ productivity
  • It is open source
  • Microservices architecture
  • Designed for extension
  • Relies on k8s CRDs

JX Topologies:

Jenkins X can work in 2 modes: Static and Serverless.

Cloud-Native approach:
Our goal will be to use Jenkins X to automate the deployment of containerized applications on Kubernetes cluster.
Jenkins X make a real collaboration between system engineer and application teams with a focus on making development teams productive through automation and DevOps best practices.

We will achieve the automation of CI/CD pipelines using Jenkins X as following:

This is how Jenkins X works (big picture) and we will see later how to install JX with the different methods on the cloud or on-premise and how to build CI/CD pipelines.

Cet article CloudBees DevOps Playground – Hands On with JenkinsX est apparu en premier sur Blog dbi services.

Alfresco Clustering – Basis & Architectures

Yann Neuhaus - Mon, 2019-07-29 01:00

This blog will be the first of a series on Alfresco HA/Clustering topics. It’s been too long I haven’t posted anything related to Alfresco so I thought about writing a few blogs about my experience with setting up more or less complex HA/Clustering infrastructures. So, let’s start this first part with an introduction to the Alfresco HA/Clustering.

If you want to setup a HA/Cluster environment, you will have to first think about where you want to go exactly. Alfresco is composed of several components so “what do you want to achieve exactly?”, that would probably be the first question to ask.

Alfresco offers a lot of possibilities, you can more or less do whatever you want. That’s really great, but it also means that you should plan what you want to do first. Do you just want a simple HA architecture for Share+Repository but you can live without Solr for a few minutes/hours (in case of issues) or you absolutely want all components to be always available? Or maybe you want an HA architecture which is better suited for high throughput? Obviously, there might be some costs details that need to be taken into consideration linked to the resources but also the licenses: the Alfresco Clustering license itself but also the Index Engine license if you go for separated Solr Servers.

That’s what you need to define first to avoid losing time changing configurations and adding more components into the picture later. Alternatively (and that’s something I will try to cover as much as I can), it’s also possible to setup an environment which will allow you to add more components (at least some of them…) as needed without having to change your HA/Clustering configuration, if you are doing it right from the start and if you don’t change too much the architecture itself.

I mentioned earlier the components of Alfresco (Alfresco Content Services, not the company), these are the ones we are usually talking about:

  • *Front-end (Apache HTTPD, Nginx, …)
  • *ActiveMQ
  • Alfresco PDF Renderer
  • Database
  • File System
  • ImageMagick
  • Java
  • LibreOffice
  • *Share (Tomcat)
  • *Repository (Tomcat)
  • *Solr6 (Jetty)

 

In this series of blog, I won’t talk about the Alfresco PDF Renderer, ImageMagick & Java because these are just simple binaries/executables that need to be available from the Repository side. For LibreOffice, it’s usually Alfresco that is managing it directly (multi-processes, restart if crash, aso…). It wouldn’t really make sense to talk about these in blogs related to Clustering. I will also disregard the Database and File System ones since they are usually out of my scope. The Database is usually installed & managed by my colleagues which are DBAs, they are much better at that than me. That leaves us with all components with an asterisk (*). I will update this list with links to the different blogs.

Before jumping in the first component, which will be the subject of the next blog, I wanted to go through some possible architectures for Alfresco. There are a lot of schemas available on internet but it’s often the same architecture that is presented so I thought I would take some time to represent, in my own way, what the Alfresco’s architecture could look like.

In the below schemas, I represented the main components: Front-end, Share, Repository, Solr, Database & File System (Data) as little boxes. As mentioned previously, I won’t talk about the Database & File System so I just represented them once to see the communications with these but what is behind their boxes can be anything (with HA/Clustering or not). The arrows represent the way communications are initiated: an arrow in a single direction “A->B” means that B is never initiating a communication with A. Boxes that are glued together represent all components installed on the same host (a physical server, a VM, a container or whatever).

 

Alfresco Architecture 1N°1: This is the simplest architecture for Alfresco. As you can see, it’s not a HA/Clustering architecture but I decided to start small. I added a Front-end (even if it’s not mandatory) because it’s a best practice and I would not install Alfresco without it. Nothing specific to say on this architecture, it’s just simple.

 

Alfresco Architecture 2N°2: The first thing to do if you have the simplest architecture in place (N°1) and you start seeing some resources contention is to split the components and more specifically to install Solr separately. This should really be the minimal architecture to use, whenever possible.

 

Alfresco Architecture 3N°3: This is the first HA/Clustering architecture. It starts small as you can see with just two nodes for each Front-end/Share/Repository stack with a Load Balancer to dispatch the load on each side for an Active/Active solution. The dotted grey lines represent the Clustering communications. In this architecture, there is therefore a Clustering for Share and another one for the Repository layer. The Front-end doesn’t need Clustering since it just forwards the communications but the session itself is on the Tomcat (Share/Repository) side. There is only one Solr node and therefore both Repository boxes will communicate with the Solr node (through the Front-end or not). Between the Repository and Solr, there is one bidirectional arrow and another one unidirectional. That’s because both Repository boxes will initiate searches but the Solr will do tracking to index new content with only one Repository: this isn’t optimal.

 

Alfresco Architecture 4N°4: To solve this small issue with Solr tracking, we can add a second Load Balancer in between so that the Solr tracking can target any Repository node. The first bottleneck you will encounter in Alfresco is usually the Repository because a lot of things are happening in the background at that layer. Therefore, this architecture is usually the simplest HA/Clustering solution that you will want to setup.

 

Alfresco Architecture 5N°5: If you are facing some performance issues with Solr or if you want all components to be in HA, then you will have to duplicate the Solr as well. Between the two Solr nodes, I put a Clustering link, that’s in case you are using Solr Sharding. If you are using the default cores (alfresco and archive), then there is no communication between distinct Solr nodes. If you are using Solr Sharding and if you want a HA architecture, then you will have the same shards on both Solr nodes and in this case, there will be communications between the Solr nodes, it’s not really a Clustering so to speak, that’s how Solr Sharding is working but I still used the same representation.

 

Alfresco Architecture 6N°6: As mentioned previously (for the N°4), the Repository is usually the bottleneck. To reduce the load on this layer, it is possible to do several things. The first possibility is to install another Repository and dedicate it to the Solr Tracking. As you can see above, the communications aren’t bidirectional anymore but only unidirectional. Searches will come from the two Repository that are in Cluster and Solr Tracking will use the separated/dedicated Repository. This third Repository can then be set in read-only, the jobs and services can be disabled, the Clustering can be disabled as well (so it uses the same DB but it’s not part of the Clustering communications because it doesn’t have to), aso… I put this third Repository as a standalone box but obviously you can install it with one of the two Solr nodes.

 

Alfresco Architecture 7N°7: The next step can be to add another read-only Repository and put these two nodes side by side with the Solr nodes. This is to only have localhost communications for the Solr Tracking which is therefore a little bit easier to secure.

 

Alfresco Architecture 8N°8: The previous architectures (N°6 & N°7) introduced a new single point of failure so to fix this, there is only one way: add a new Load Balancer between the Solr and the Repository for the tracking. Behind the Load Balancer, there are two solutions: keep the fourth Repository which is also in read-only or use a fallback to the Repository node1/node2 in case the read-only Repository (node3) isn’t available. For that purpose, the Load Balancer should be in, respectively, Active/Active or Active/Passive. As you can see, I choose to represent the first one.

 

These were a few possible architectures. You can obviously add more nodes if you want to, to handle more load. There are many other solutions so have fun designing the best one, according to your requirements.

 

Cet article Alfresco Clustering – Basis & Architectures est apparu en premier sur Blog dbi services.

Oracle 19c Automatic Indexing: Configuration (All I Need)

Richard Foote - Mon, 2019-07-29 00:12
In this post, I’ll discuss how to configure the new Automatic Indexing capability introduced in Oracle Database 19c. The intent of Oracle here is to make the configuration of Automatic Indexing as simplistic as possible, with as few levers for DBAs to potentially stuff up as possible. The ultimate goal would be to have a […]
Categories: DBA Blogs

dbvisit 9: Automatic Failover

Yann Neuhaus - Sat, 2019-07-27 06:43

dbvisit 9 is released since a few months. And one new feature I tested is the Automatic Failover. In this blog I suppose that dbvisit 9 is already installed and that the standby database is already created. Indeed I will not describe nor dbvisit installation neither the standby creation as it is the same as the previous versions.
For more info about dbvisit installation and/or dbvisit standby creation please see these steps in my previous blog or dbvisit documentation
The new feature Autamatic Failover needs to install an observer which main functions are:
-Provide remote monitoring of existing DDCs, and inform the DBA of problems in close to real-time
-Automatically perform a Failover of the DDC if previously-specified conditions are met.

We will describe observer installation and configuration later

We describe below the configuration we are using
dbvisit1: primary server with Oracle 19c
dbvisit2: standby server with Oracle 19c
dbvisitconsole : Host of the dbvisit console (dbvserver) and for the observer

As specified earlier, we need to install an observer. It is very easy to do this, just launch the install_dbvisit executable and follow the instructions

[oracle@dbvisitconsole installer]$ pwd
/home/oracle/dbvisit/installer
[oracle@dbvisitconsole installer]$ ./install-dbvisit

-----------------------------------------------------------
    Welcome to the Dbvisit software installer.
-----------------------------------------------------------

    It is recommended to make a backup of our current Dbvisit software
    location (Dbvisit Base location) for rollback purposes.

    Installer Directory /home/oracle/dbvisit

>>> Please specify the Dbvisit installation directory (Dbvisit Base).

    The various Dbvisit products and components - such as Dbvisit Standby,
    Dbvisit Dbvnet will be installed in the appropriate subdirectories of
    this path.

    Enter a custom value or press ENTER to accept default [/usr/dbvisit]:
     > /u01/app/dbvisit
    DBVISIT_BASE = /u01/app/dbvisit

    -----------------------------------------------------------
    Component      Installer Version   Installed Version
    -----------------------------------------------------------
    standby        9.0.02_0_gbd40c486                                not installed                 
    dbvnet         9.0.02_0_gbd40c486                                not installed                 
    dbvagent       9.0.02_0_gbd40c486                                not installed                 
    dbvserver      9.0.02_0_gbd40c486                                9.0.02_0_gbd40c486            
    observer       1.02                                              not installed                 

    -----------------------------------------------------------

    What action would you like to perform?
       1 - Install component(s)
       2 - Uninstall component(s)
       3 - Exit

    Your choice: 1

    Choose component(s):
       1 - Core Components (Dbvisit Standby Cli, Dbvnet, Dbvagent)
       2 - Dbvisit Standby Core (Command Line Interface)
       3 - Dbvnet (Dbvisit Network Communication)
       4 - Dbvagent (Dbvisit Agent)
       5 - Dbvserver (Dbvisit Central Console) - Not available on Solaris/AIX
       6 - Dbvisit Observer (Automatic Failover Option) - Not available on Solaris/AIX
    Press ENTER to exit Installer

    Your choice: 6

-----------------------------------------------------------
    Summary of the Dbvisit OBSERVER configuration
-----------------------------------------------------------
    DBVISIT_BASE /u01/app/dbvisit

    Press ENTER to continue

-----------------------------------------------------------
    About to install Dbvisit OBSERVER
-----------------------------------------------------------
    Component observer installed.

    -----------------------------------------------------------
    Component      Installer Version   Installed Version
    -----------------------------------------------------------
    standby        9.0.02_0_gbd40c486                                not installed                 
    dbvnet         9.0.02_0_gbd40c486                                not installed                 
    dbvagent       9.0.02_0_gbd40c486                                not installed                 
    dbvserver      9.0.02_0_gbd40c486                                9.0.02_0_gbd40c486            
    observer       1.02                                              1.02                          

    -----------------------------------------------------------

    What action would you like to perform?
       1 - Install component(s)
       2 - Uninstall component(s)
       3 - Exit

    Your choice: 3

>>> Installation completed
    Install log /tmp/dbvisit_install.log.201907231647.
[oracle@dbvisitconsole installer]$

And once the installation done, we can start it

[oracle@dbvisitconsole observer]$ ./observersvc start &
[1] 2866

[oracle@dbvisitconsole observer]$ ps -ef | grep obser
oracle    2866  2275  0 14:25 pts/0    00:00:01 ./observersvc start
oracle    2921  2275  0 14:29 pts/0    00:00:00 grep --color=auto obser
[oracle@dbvisitconsole observer]$

After starting the observer we have to add the observer server. This is done from the MANAGE CONFIGURATION TAB from the MENU

From the Configuration TAB, choose the NEW on the left to add a dbvisit observer

And then fill the informations. Note that the default passphrase for the observer is admin900 and then save

To monitor our configuration by the observer, let’s click on Monitor

And then specify the poll interval and the number of retries before a failover happens. In our case

The observer will monitor the configuration every 60 s and will retry 5 times if there is any error.
If after 5 minutes (5×60 secondes), the probleme is not fixed, than an automatic failover will happen.

The observer logfile is located on the observer server

[oracle@dbvisitconsole log]$ pwd
/u01/app/dbvisit/observer/log
[oracle@dbvisitconsole log]$ ls -l
total 8
-rw-r--r--. 1 oracle oinstall 1109 Jul 25 15:24 observer.log
-rw-r--r--. 1 oracle oinstall   97 Jul 25 15:24 orcl_1_observer.log
[oracle@dbvisitconsole log]$

[oracle@dbvisitconsole log]$ tail -f orcl_1_observer.log
2019/07/25 13:24:46 DDC: DDC#1(orcl): Started watchdog: Watchdog successfully started monitoring

Now let’s break the primary database and normally a failover should happen after 5 minutes

[oracle@dbvisit1 log]$ ps -ef | grep pmon
oracle    1887     1  0 14:03 ?        00:00:00 ora_pmon_orcl
oracle   18199  1733  0 15:31 pts/0    00:00:00 grep --color=auto pmon
[oracle@dbvisit1 log]$ kill -9 1887
[oracle@dbvisit1 log]$ ps -ef | grep pmon
oracle   18304  1733  0 15:32 pts/0    00:00:00 grep --color=auto pmon
[oracle@dbvisit1 log]$

In the observer logfile we can see that the standby was promoted after 5 retries.

[oracle@dbvisitconsole log]$ tail -f orcl_1_observer.log
2019/07/25 13:24:46 DDC: DDC#1(orcl): Started watchdog: Watchdog successfully started monitoring
2019/07/25 13:33:51 DDC: DDC#1(orcl): rules failing (1/5): primary: error on dbvisit1:7891: unexpected database status for DDC orcl: got: "Database is down", expected: "Regular database open in read write mode"
2019/07/25 13:34:51 DDC: DDC#1(orcl): rules failing (2/5): primary: error on dbvisit1:7891: unexpected database status for DDC orcl: got: "Database is down", expected: "Regular database open in read write mode"
2019/07/25 13:35:51 DDC: DDC#1(orcl): rules failing (3/5): primary: error on dbvisit1:7891: unexpected database status for DDC orcl: got: "Database is down", expected: "Regular database open in read write mode"
2019/07/25 13:36:51 DDC: DDC#1(orcl): rules failing (4/5): primary: error on dbvisit1:7891: unexpected database status for DDC orcl: got: "Database is down", expected: "Regular database open in read write mode"
2019/07/25 13:37:51 DDC: DDC#1(orcl): rules failing (5/5): primary: error on dbvisit1:7891: unexpected database status for DDC orcl: got: "Database is down", expected: "Regular database open in read write mode"
2019/07/25 13:37:51 DDC: DDC#1(orcl): configuration failed after 5 retries: primary: error on dbvisit1:7891: unexpected database status for DDC orcl: got: "Database is down", expected: "Regular database open in read write mode"
2019/07/25 13:37:51 DDC: DDC#1(orcl): watchdog shutting down: activation imminent
2019/07/25 13:37:51 DDC: DDC#1(orcl): ACTIVATION started: conditions for activation satisfied
2019/07/25 13:38:41 DDC: DDC#1(orcl): ACTIVATION successful: ACTIVATION OK: standby activated, activation took: 50.043794192s

And we can verify that the standby is now open in read write mode

[oracle@dbvisit2 trace]$ sqlplus / as sysdba

SQL*Plus: Release 19.0.0.0.0 - Production on Thu Jul 25 15:49:38 2019
Version 19.3.0.0.0

Copyright (c) 1982, 2019, Oracle.  All rights reserved.


Connected to:
Oracle Database 19c Standard Edition 2 Release 19.0.0.0.0 - Production
Version 19.3.0.0.0

SQL> select db_unique_name,open_mode from v$database;

DB_UNIQUE_NAME                 OPEN_MODE
------------------------------ --------------------
orcl                           READ WRITE

SQL>

Note that we can use a user defined script with the observer. For more information please see dbvisit documentation

Cet article dbvisit 9: Automatic Failover est apparu en premier sur Blog dbi services.

Spatial Analytics Made Easy: Oracle Spatial Studio

Rittman Mead Consulting - Fri, 2019-07-26 10:31
 Oracle Spatial Studio

Let's say we need to understand where our company needs to open a new shop. Most of the time the decision is driven by gut feeling and some knowledge of the market and client base, but what if we could have visual insights about where are the high density zones with customers not covered by a shop nearby like in the map below?

 Oracle Spatial Studio

Well... welcome Oracle Spatial Studio!

Spatial Studio is Oracle's new tool for creating spatial analytics with a visual GUI. It uses Oracle Spatial database functions in the backen d exposed with an interface in line with the Oracle Analytics Cloud one. Let's see how it works!

QuickStart Installation

First of all we need to download Spatial Studio from the Oracle web page, for this initial test I downloaded the "Quick Start", a self contained version pre-deployed in a lightweight application server. For more robust applications you may want to download the EAR file deployable in Weblogic.

 Oracle Spatial Studio

Once downloaded and unzipped the file, we just need to verify we have a Java JDK 8 (update 181 or higher) under the hood and we can immediately start Oracle Spatial Studio with the ./start.sh command.

The command will start the service on the local machine that can be accessed at https://localhost:4040/spatialstudio. By default Oracle Spatial Studio Quickstart uses HTTPS protocol with self-signed certificates, thus the first time you access the URL you will need to add a security exception in your browser. The configurations such as port, JVM parameters, host and HTTP/HTTPS protocol can be changed in the conf/server.json file.

We can then login with the default credentials admin/welcome1

 Oracle Spatial Studio

The first step in the Spatial Studio setup is the definition of the metadata connection type. This needs to point to an Oracle database with the spatial option. For my example I initially used an Oracle Autonomous Data Warehouse, for which I had to drop the wallet and specify the schema details.

 Oracle Spatial Studio

Once logged in, the layout and working flows are very similar to Oracle Analytics Cloud making the transition between the two very easy (more details on this later on). In the left menu we can access, like in OAC, Projects (visualizations), Data, Jobs and the Console.

 Oracle Spatial Studio

In order to do Spatial Analysis we need to start from a Dataset, this can be existing tables or views, or we can upload local files. To create a Dataset, click on Create and Dataset

 Oracle Spatial Studio

We have then three options:

  • Upload a Spreadsheet containing spatial information (e.g. Addresses, Postcodes, Regions, Cities etc)
  • Upload a Shapefile containing geometric locations and associated attributes.
  • Use spatial data from one of the existing connections, this can point to any connection containing spatial information (e.g. a table in a database containing customer addresses)
 Oracle Spatial StudioSample Dataset with Mockaroo

I used Mockaroo, a realistic data generator service, to create two excel files: one containing customers with related locations and a second one with shops and related latitude and longitude. All I had to do was to select which fields I wanted to include in my file and the related datatype.

 Oracle Spatial Studio

For example, the list of shop dataset contained the following columns:

  • Id: as row number
  • Shop Name: as concatenation of Shop and the Id
  • Lat: Latitude
  • Long: Longitude
  • Dept: the Department (e.g. Grocery, Books, Health&Beauty)

Mockaroo offers a perfect service and has a free tier of datasets with less than 1000 rows which can be useful for demo purposes. For each column defined, you can select between a good variety of column types. You can also define your own type using regular expressions!

 Oracle Spatial StudioAdding the Datasets to Oracle Spatial Studio

Once we have the two datasources in Excel format, it's time to start playing with Spatial Studio. We first need to upload the datasets, we can do it via Create and Dataset. Starting with the Customer.xlsx one. Once selected the file to upload Spatial Studio provides (as OAC) an overview of the dataset together with options to change configurations like dataset name, target destination (metadata database) and column names.

 Oracle Spatial Studio

Once modified the table name to TEST_CUSTOMERS and clicked on Submit Spatial Studio starts inserting all the rows into the SPATIAL_STUDIO connection with a routine that could take seconds or minutes depending on the dataset volume. When the upload routine finishes I can see the TEST_CUSTOMERS table appearing in the list of datasets.

 Oracle Spatial Studio

We can immediately see the yellow warning sign next to the dataset name, it's due to the fact that we have a dataset with no geo-coded information, we can solve this problem by clicking on the option button and then Prepare and Geocode Addresses

 Oracle Spatial Studio

Oracle Spatial Studio will suggest, based on the column content, some geo-type matching e.g. City Name, Country and Postal Code. We can use the defaults or modify them if we feel they are wrong.

 Oracle Spatial Studio

Once clicked on Apply the geocoding job starts.

 Oracle Spatial Studio

Once the job ends, we can see the location icon next to our dataset name

 Oracle Spatial Studio

We can do the same for the Shops.xlsx dataset, starting by uploading it and store it as TEST_SHOPS dataset.

 Oracle Spatial Studio

Once the dataset is uploaded I can geo-locate the information based on the Latitude and Longitude, I can click on the option button and the selecting Prepare and Create Lon/Lat Index. Then I'll need to assign the Longitude and Latitude column correctly and click on Ok.

 Oracle Spatial StudioSpatial Analytics

Now it's time to do some Spatial Analysis so I can click on Create and Project and I'll face an empty canvas by default

 Oracle Spatial Studio

The first step is to add a Map, I can do that by selecting the visualizations menu and then dragging the map to the canvas.

 Oracle Spatial Studio

Next step is to add some data by clicking on Data Elements and then Add Dataset

 Oracle Spatial Studio

I select the TEST_CUSTOMERS dataset and add it to the project, then I need to drag it on top of the map to visualize my customer data.

 Oracle Spatial Studio

Oracle Spatial Studio Offers several options to change the data visualizations like color, opacity, blur etc.

 Oracle Spatial Studio

Now I can add the TEST_SHOPS dataset and visualize it on the map with the same set of steps followed before.

 Oracle Spatial Studio

It's finally time for spatial analysis! Let's say, as per initial example, that I want to know which of my customers doesn't have any shops in the nearest 200km. In order to achieve that I need to first create buffer areas of 200km around the shops, by selecting the TEST_SHOPS datasource and then clicking on the Spatial Analysis.

 Oracle Spatial Studio

This will open a popup window listing a good number of spatial analysis, by clicking on the Transform tab I can see the Add a buffer of a specified distance option.

 Oracle Spatial Studio

Unfortunately the buffer function is not available in ADW at the moment.

 Oracle Spatial Studio

I had to rely on an Oracle Database Cloud Service 18c Enterprise Edition - High Performance (which includes the Spatial option) to continue for my metadata storage and processing. Few Takeaways:

  • Select 18c (or anything above 12.2): I hit an issue ORA-00972: identifier is too long when importing the data in a 12.1 Database, which (thanks StackOverflow) is fixed as of 12.2.
  • High Performance: This includes the Spatial Option

Once I used the DBCS as metadata store, I can finally use the buffer function and set the parameter of 200km around the shops.

 Oracle Spatial Studio

The TEST_SHOPS_BUFFER is now visible under Analysis and can be added on top of the Map correctly showing the 200km buffer zone.

 Oracle Spatial Studio

I can understand which customers have a shop in the nearest 200k by creating an analysis and select the option "Return shapes within a specified distance of another"

 Oracle Spatial Studio

In the parameters I can select the TEST_CUSTOMERS as Layer to be filtered, the TEST_SHOPS as the Layer to be used as filter and the 200Km as distance.

 Oracle Spatial Studio

I can then visualize the result by adding the TEST_CUSTOMERS_WITHIN_DISTANCE layer in the map.

 Oracle Spatial Studio

TEST_CUSTOMERS_WITHIN_DISTANCE contains the customers already "covered" by a shop in the 200km range, what I may want to do now is remove them from my list of customers in order to do analysis on the remaining ones, how can I do that? Unfortunately in the first Spatial Studio version there is no visual way of doing DATASET_A MINUS DATASET_B but, hey, it's just the first incarnation and we can expect that type of functions and many others to be available in future releases!

The following paragraph is an in-depth analysis in the database of functions that will probably be exposed in Spatial Studio's future version, so if not interested, progress directly to the section named "Progressing in the Spatial Analysis".

A Look in the Database

Since we want to achieve our goal of getting the customers not covered by a shop now, we need to look a bit deeper where the data is stored: in the database. This gives us two opportunities: check how Spatial Studio works under the covers and freely use SQL to achieve our goals (DATASET_A MINUS DATASET_B).

First let's have a look at the tables created by Spatial Studio: we can see some metadata tables used by studio as well as the database representation of our two excel files TEST_CUSTOMERS and TEST_SHOPS.

 Oracle Spatial Studio

Looking in depth at the metadata we can also see a table named SGTECH$TABLE followed by an ID. That table collects the information regarding the geo-coding job we executed against our customers dataset which were located starting from zip-codes and addresses. We can associate the table to the TEST_CUSTOMERS dataset with the following query against the SGTECH_OBJECTS metadata table.

SELECT NAME, 
  JSON_VALUE(data, '$.gcHelperTableName') DATASET  
FROM SGTECH_OBJECT 
WHERE OBJECTTYPE='dataset'
AND NAME='TEST_CUSTOMERS';
 Oracle Spatial Studio

The SGTECH$TABLEA004AA549110B928755FC05F01A3EF89 table contains, as expected, a row for each customer in the dataset, together with the related geometry if the geo-coding was successful and some metadata flags like GC_ATTEMPTED, GC_STATUS and GC_MATCH_CODE stating the accuracy of the geo-coding match.

 Oracle Spatial Studio

What about all the analysis like the buffer and the customers within distance? For each analysis Spatial Studio creates a separate view with the SGTECH$VIEW prefix followed by an ID.

 Oracle Spatial Studio

To understand which view is referring to which analysis we need to query the metadata table SGTECH_OBJECTS with a query like

SELECT NAME, 
  JSON_VALUE(data, '$.tableName') DATASET  
FROM SGTECH_OBJECT 
WHERE OBJECTTYPE='dataset'

With the following result

 Oracle Spatial Studio

We know then that the TEST_CUSTOMERS_WITHIN_DISTANCE can be accessed by the view SGTECH$VIEW0B2B36785A28843F74B58B3CCF1C51E3 and when checking its SQL we can clearly see that it executes the SDO_WITHIN_DISTANCE function using the TEST_CUSTOMERS.GC_GEOMETRY, the TEST_SHOPS columns LONGITUDE and LATITUDE and the distance=200 unit=KILOMETER parameters we set in the front-end.

CREATE OR replace force editionable view "SPATIAL_STUDIO"."SGTECH$VIEW0B2B36785A28843F74B58B3CCF1C51E3"
SELECT
    ...
FROM
    "TEST_CUSTOMERS"   "t1",
    "TEST_SHOPS"       "t2"
WHERE
    sdo_within_distance("t1"."GC_GEOMETRY",
    spatial_studio.sgtech_ptf("t2"."LONGITUDE", "t2"."LATITUDE"), 
    'distance=200 unit=KILOMETER'
    ) = 'TRUE';

Ok, we now understood which view contains the data, thus we can create a new view containing only the customers which are not within the 200km distance with

CREATE VIEW TEST_CUSTOMERS_NOT_WITHIN_DISTANCE AS
SELECT
    t1.id            AS id,
    t1.first_name    AS first_name,
    t1.last_name     AS last_name,
    t1.email         AS email,
    t1.gender        AS gender,
    t1.postal_code   AS postal_code,
    t1.street        AS street,
    t1.country       AS COUNTRY,
    t1.city          AS city,
    t1.studio_id     AS studio_id,
    t1.gc_geometry   AS gc_geometry
FROM
    test_customers t1
WHERE
    id NOT IN (
        SELECT
            id
        FROM
            spatial_studio.sgtech$view0b2b36785a28843f74b58b3ccf1c51e3
    );
Progressing in the Spatial Analysis

In the previous paragraph we created a view in the database named TEST_CUSTOMERS_NOT_WITHIN_DISTANCE containing the customer without a shop in a 200km radius. We can now import it into Spatial Studio by creating a new dataset, selecting the connection to the database (in our case named SPATIAL_STUDIO) as source and then the newly created TEST_CUSTOMERS_NOT_WITHIN_DISTANCE view.

 Oracle Spatial Studio

The dataset is added, but it has a yellow warning icon next to it

 Oracle Spatial Studio

Spatial Studio requests us to define a primary key, we can do that by accessing the properties of the dataset, select the Columns tab, choosing which column acts as primary key and validate it. After this step I can visualize this customer in a map.

 Oracle Spatial Studio

What's next? Well If I want to open a new shop, I may want to do that where there is a concentration of customers, which is easily visualizable with Spatial Studio by changing the Render Style to Heatmap.

 Oracle Spatial Studio

With the following output

 Oracle Spatial Studio

We can clearly see some major concentrations around Dallas, Washington and Minneapolis. Focusing more on Dallas, Spatial Studio also offers the option to simulate a new shop in the map and calculate the 200km buffer around it. I can clearly see that adding a shop halfway between Oklahoma City and Dallas would allow me to cover both clients within the 200km radius.

 Oracle Spatial Studio

Please remember that this is a purely demonstrative analysis, and some of the choices, like the 200km buffer are expressly simplistic. Other factors could come into play when choosing a shop location like the revenue generated by some customers. And here it comes the second beauty of Oracle Spatial Studio, we can export datasets as GeoJSON or CSV and include them in Data Visualization.

 Oracle Spatial Studio

For example I can export the data of TEST_CUSTOMERS_NOT_WITHIN_DISTANCE from Spatial Studio and include then in a Data Visualization Project blending them with the Sales related to the same customers.

 Oracle Spatial Studio

I can now focus not only on the customer's position but also on other metrics like Profit or Sales Amount that I may have in other datasets. For another example of Oracle Spatial Studio and Data Visualization interoperability check out this video from Oracle Analytics Senior Director Philippe Lions.

Conclusions

Spatial analytics made easy: this is the focus of Oracle Spatial Studio. Before spatial queries were locked down at database level with limited access from an analyst point of view. Now we have a visual tool with a simple GUI (in line with OAC) that easily enables spatial queries for everybody!

But this is only the first part of the story: the combination of capabilities achievable when mixing Oracle Spatial Studio and Oracle Analytics Cloud takes any type of analytics to the next level!

Categories: BI & Warehousing

Pages

Subscribe to Oracle FAQ aggregator