Pas Apicella

PCF AWS marketplace provides app direct services and in this example I am going to use the "newrelic" monitoring service to monitor my spring based java application. It's really this simple.

1. Create a service as shown below.

[Tue Mar 04 17:19:34 papicella@:~/cfapps/spring-travel ] $ cf create-service newrelic standard dev-newrelic

2. Create a manifest.yml for my spring application which uses the new relic service above.

applications:
- name: pas-springtravel 
  memory: 1024M 
  instances: 1
  host: pas-springtravel 
  domain: cfapps.io 
  path: ./travel.war
  services:
  - dev-mysql
  - dev-newrelic

3. Push the application

[Tue Mar 04 17:19:34 papicella@:~/cfapps/spring-travel ] $ cf push -f manifest.yml 
Using manifest file manifest.yml

Creating app pas-springtravel in org papicella-org / space development as papicella@gopivotal.com...
OK

Using route pas-springtravel.cfapps.io
Binding pas-springtravel.cfapps.io to pas-springtravel...
OK

Uploading pas-springtravel...
Uploading from: travel.war
5.3M, 2748 files
OK
Binding service dev-mysql to pas-springtravel in org papicella-org / space development as papicella@gopivotal.com
OK
Binding service dev-newrelic to pas-springtravel in org papicella-org / space development as papicella@gopivotal.com
OK

Starting app pas-springtravel in org papicella-org / space development as papicella@gopivotal.com...
OK
-----> Downloaded app package (22M)
-----> Uploading droplet (67M)

0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
1 of 1 instances running

App started

Showing health and status for app pas-springtravel in org papicella-org / space development as papicella@gopivotal.com...
OK

requested state: started
instances: 1/1
usage: 1G x 1 instances
urls: pas-springtravel.cfapps.io

     state     since                    cpu    memory         disk           
#0   running   2014-03-04 05:24:43 PM   0.0%   610.7M of 1G   155.9M of 1G 

4. Under the services listed on AWS click on "Manage" and here are some screen shots of what the newrelic monitoring service provides with just a simple BIND when we pushed the application.





http://feeds.feedburner.com/TheBlasFromPas

I previously showed how to create a connection in IntelliJ IDEA to Cloud Foundry v2 in the post below.

http://theblasfrompas.blogspot.com.au/2014/02/intellij-idea-version-13-now-includes.html

With a Cloud Foundry CLOUD connection we can now PUSH our application directly from the IDE as shown below.

1. Create a run configuration for your project as shown below. We also specify the memory and number of instances on this page as part of the push  / deployment process.



2. Select the created run configuration and deploy the application, output as follows


3. View the deployed application on AWS hosted instance of Cloud Foundry



http://feeds.feedburner.com/TheBlasFromPas

Just installed IntelliJ IDEA version 13 and found that it now includes a CloudFoundry connection type. You define it under IDE settings as shown below.


Will test deploying to the publicly hosted AWS Cloud Foundry using this connection at some stage.http://feeds.feedburner.com/TheBlasFromPas

By creating a manifest as follows we can push multiple applications in one go as shown below.

1. manifest.yml

applications:
- name: pas-props
  memory: 256M
  instances: 1
  host: pas-props
  domain: cfapps.io
  path: ./props.war
- name: pas-httpsession
  memory: 256M
  instances: 1
  host: pas-httpsession
  domain: cfapps.io
  path: ./haclusterdemo.war

2. Push as follows

[Thu Feb 20 14:48:45 papicella@:~/vmware/pivotal/products/cloud-foundry/apps/other ] $ cf push -f manifest-twoapps.yml
Using manifest file manifest-twoapps.yml

Creating app pas-props in org papicella-org / space development as papicella@gopivotal.com...
OK

Using route pas-props.cfapps.io
Binding pas-props.cfapps.io to pas-props...
OK

Uploading pas-props...
Uploading from: props.war
2.7K, 5 files
OK

Starting app pas-props in org papicella-org / space development as papicella@gopivotal.com...
OK

1 of 1 instances running

App started

Showing health and status for app pas-props in org papicella-org / space development as papicella@gopivotal.com...
OK

requested state: started
instances: 1/1
usage: 256M x 1 instances
urls: pas-props.cfapps.io

     state     since                    cpu    memory           disk           
#0   running   2014-02-20 02:51:14 PM   0.0%   193.2M of 256M   110.7M of 1G   
Creating app pas-httpsession in org papicella-org / space development as papicella@gopivotal.com...
OK

Creating route pas-httpsession.cfapps.io...
OK

Binding pas-httpsession.cfapps.io to pas-httpsession...
OK

Uploading pas-httpsession...
Uploading from: haclusterdemo.war
130.2K, 10 files
OK

Starting app pas-httpsession in org papicella-org / space development as papicella@gopivotal.com...
OK

0 of 1 instances running, 1 starting
1 of 1 instances running

App started

Showing health and status for app pas-httpsession in org papicella-org / space development as papicella@gopivotal.com...
OK

requested state: started
instances: 1/1
usage: 256M x 1 instances
urls: pas-httpsession.cfapps.io

     state     since                    cpu    memory           disk           
#0   running   2014-02-20 02:53:53 PM   2.0%   214.3M of 256M   113.3M of 1G   

More Information

http://docs.cloudfoundry.org/devguide/deploy-apps/manifest.htmlhttp://feeds.feedburner.com/TheBlasFromPas

I previously blogged how to PUSH an application into PCF , then bind services as shown below.
http://theblasfrompas.blogspot.com.au/2014/02/i-decided-to-install-new-cf-v6-cli-for.html
If you want to do this in one step simply create a manifest file as shown below. This example isn't the spring books application but shows you what your manifest.yml would look like with services included so you could then create one similar to that below.
1. Create a manifest.yml file as follows
manifest.yml
applications:- name: pas-springtravel   memory: 512M   instances: 1  host: pas-springtravel   domain: cfapps.io   path: ./travel.war  services:  - dev-mysql
2. Push the application and bind to the services at push time.
[Tue Feb 18 20:13:01 papicella@:~/vmware/pivotal/products/cloud-foundry/apps/spring-travel ] $ cf push -f manifest.yml Using manifest file manifest.yml
Creating app pas-springtravel in org papicella-org / space development as papicella@gopivotal.com...OK
Creating route pas-springtravel.cfapps.io...OK
Binding pas-springtravel.cfapps.io to pas-springtravel...OK
Uploading pas-springtravel...Uploading from: travel.war5.3M, 2748 filesOKBinding service dev-mysql to pas-springtravel in org papicella-org / space development as papicella@gopivotal.comOK
Starting app pas-springtravel in org papicella-org / space development as papicella@gopivotal.com...OK-----> Downloaded app package (22M)-----> Uploading droplet (62M)
0 of 1 instances running, 1 starting0 of 1 instances running, 1 starting1 of 1 instances running
App started
Showing health and status for app pas-springtravel in org papicella-org / space development as papicella@gopivotal.com...OK
requested state: startedinstances: 1/1usage: 512M x 1 instancesurls: pas-springtravel.cfapps.io
     state     since                    cpu    memory           disk           #0   running   2014-02-18 08:24:58 PM   0.5%   458.6M of 512M   149.9M of 1G   
3. The application using the MYSQL service

http://feeds.feedburner.com/TheBlasFromPas

I decided to install the new CF v6 CLI for PCF (Pivotal Cloud Foundry). Given this will be the new CLI for PCF here is how it differs to the old Ruby CLI. In this example I deploy / push a spring based application and bind it to existing services.

To download and install the new CLI v6 use the link below.

http://docs.cloudfoundry.com/docs/using/managing-apps/cf/go-cli.html

In this example we have the following 2 services that already exist and I am using the PUBLIC hosted AWS version of PCF at https://console.run.pivotal.io/

• dev-mysql
• dev-redis

1. First lets deploy our application and when we do we DON'T actually start it at this point.

[Sun Feb 16 22:23:22 papicella@:~/vmware/pivotal/products/cloud-foundry/apps/spring-books ] $ cf push pas-books -d cfapps.io -i 1 -m 512M -p springmvc31.war --no-start
Creating app pas-books in org papicella-org / space development as papicella@gopivotal.com...
OK

Creating route pas-books.cfapps.io...
OK

Binding pas-books.cfapps.io to pas-books...
OK

Uploading pas-books...
Uploading from: /Users/papicella/vmware/pivotal/products/cloud-foundry/apps/spring-books/springmvc31.war
511.2K, 44 files
OK

2. Bind the application to the 2 services above.

[Sun Feb 16 22:25:56 papicella@:~/vmware/pivotal/products/cloud-foundry/apps/spring-books ] $ cf bind-service pas-books dev-mysql
Binding service dev-mysql to app pas-books in org papicella-org / space development as papicella@gopivotal.com...
OK
TIP: Use 'cf push' to ensure your env variable changes take effect

[Sun Feb 16 22:25:56 papicella@:~/vmware/pivotal/products/cloud-foundry/apps/spring-books ] $ cf bind-service pas-books dev-redis
Binding service dev-redis to app pas-books in org papicella-org / space development as papicella@gopivotal.com...
OK
TIP: Use 'cf push' to ensure your env variable changes take effect

3. Push the application again and this time start it.

[Sun Feb 16 22:29:26 papicella@:~/vmware/pivotal/products/cloud-foundry/apps/spring-books ] $ cf push pas-books -p springmvc31.war
Updating app pas-books in org papicella-org / space development as papicella@gopivotal.com...
OK

Uploading pas-books...
Uploading from: /Users/papicella/vmware/pivotal/products/cloud-foundry/apps/spring-books/springmvc31.war
511.2K, 44 files
OK

Starting app pas-books in org papicella-org / space development as papicella@gopivotal.com...
-----> Downloaded app package (19M)
OK
-----> Java Buildpack source: system
-----> Downloading Open JDK 1.7.0_51 from http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.7.0_51.tar.gz 
(1.2s)
       Expanding Open JDK to .java-buildpack/open_jdk 
(1.2s)
-----> Downloading Spring Auto Reconfiguration 0.8.7 from http://download.pivotal.io.s3.amazonaws.com/auto-reconfiguration/auto-reconfiguration-0.8.7.jar 
(1.4s)
       Modifying /WEB-INF/web.xml for Auto Reconfiguration
-----> Downloading Tomcat 7.0.50 from http://download.pivotal.io.s3.amazonaws.com/tomcat/tomcat-7.0.50.tar.gz 
(0.3s)
       Expanding Tomcat to .java-buildpack/tomcat 
(0.1s)
-----> Downloading Buildpack Tomcat Support 1.1.1 from http://download.pivotal.io.s3.amazonaws.com/tomcat-buildpack-support/tomcat-buildpack-support-1.1.1.jar 
(0.0s)
-----> Uploading droplet (60M)

0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
1 of 1 instances running

App started

Showing health and status for app pas-books in org papicella-org / space development as papicella@gopivotal.com...
OK

requested state: started
instances: 1/1
usage: 512M x 1 instances
urls: pas-books.cfapps.io

     state     since                    cpu    memory           disk           
#0   running   2014-02-16 10:31:13 PM   0.0%   494.9M of 512M   134.2M of 1G   


If all went well we should see something as follows.


And of course a running application


http://feeds.feedburner.com/TheBlasFromPas

In this example below I show how you would create a user defined service for Oracle 11g R2 which is running outside of the PCF instance and how your applications would bind to that service. The example below is from the spring-music sample application which is a spring based web application.

https://github.com/cloudfoundry-samples/spring-music

1. First lets create our user defined service for Oracle , here we just use a simple URI which the Spring Cloud library knows how to use.

c:\pivotal\spring-music-oracle>cf create-service user-provided
Name?> oracle11gr2

What credential parameters should applications use to connect to this service instance?
(e.g. hostname, port, password)> uri

uri> oracle://scott:tiger@192.168.110.209:1521/XE

Creating service oracle11gr2... OK

c:\pivotal\spring-music-oracle>


The spring cloud project is here.

https://github.com/spring-projects/spring-cloud

2. Now with a previously created WAR file lets push the spring music app to PCF and bind to our oracle service as shown below.

c:\pivotal\spring-music-oracle>cf push --path=spring-music-oracle.war
Name> spring-music-oracle

Instances> 1

1: 128M
2: 256M
3: 512M
4: 1G
Memory Limit> 3

Creating spring-music-oracle... OK

1: spring-music-oracle
2: none
Subdomain> spring-music-oracle

1: system.com
2: none
Domain> system.com

Binding spring-music-oracle.system.com to spring-music-oracle... OK

Create services for application?> n

Bind other services to application?> y

1: mysql-dev
2: rmq
3: user-provided-705bf
4: oracle11gr2
Which service?> 4

Binding oracle11gr2 to spring-music-oracle... OK
Bind another service?> n

Save configuration?> n

Uploading spring-music-oracle... OK
Preparing to start spring-music-oracle... OK
-----> Downloaded app package (22M)
-----> Downloading OpenJDK 1.7.0_51 from http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.7.0_
51.tar.gz (1m 52s)
       Expanding JRE to .java (2.4s)
-----> Downloading Spring Auto-reconfiguration 0.8.4 from http://download.pivotal.io.s3.amazonaws.com/auto-reconfigurati
on/auto-reconfiguration-0.8.4.jar (4.3s)
       Modifying /WEB-INF/web.xml for Auto Reconfiguration
-----> Downloading Tomcat 7.0.50 from http://download.pivotal.io.s3.amazonaws.com/tomcat/tomcat-7.0.50.tar.gz (27.9s)
       Expanding Tomcat to .tomcat (0.3s)
-----> Downloading Buildpack Tomcat Support 1.1.1 from http://download.pivotal.io.s3.amazonaws.com/tomcat-buildpack-supp
ort/tomcat-buildpack-support-1.1.1.jar (0.2s)
-----> Uploading droplet (63M)
Checking status of app 'spring-music-oracle'...
  0 of 1 instances running (1 starting)
  0 of 1 instances running (1 starting)
  0 of 1 instances running (1 starting)
  0 of 1 instances running (1 starting)
  0 of 1 instances running (1 starting)
  0 of 1 instances running (1 starting)
  0 of 1 instances running (1 starting)
  0 of 1 instances running (1 starting)
  0 of 1 instances running (1 starting)
  0 of 1 instances running (1 starting)
  0 of 1 instances running (1 starting)
  1 of 1 instances running (1 running)
Push successful! App 'spring-music-oracle' available at spring-music-oracle.system.com

c:\pivotal\spring-music-oracle>


3. Here are some screenshots showing it deployed , invoking the application, and data within the Oracle 11g r2 schema.






The code to make this happen for Oracle is as follows, we do this to make use of the spring cloud code to bind to the user defined service "oracle11gr2" we created above. The spring cloud Library enables cloud applications to connect to services. The classes below show how we enable our oracle service for the spring music application.

OracleServiceInfo.java

  
package org.cloudfoundry.samples.music.cloud;

import org.springframework.cloud.service.ServiceInfo.ServiceLabel;
import org.springframework.cloud.service.common.RelationalServiceInfo;

@ServiceLabel("oracle")
public class OracleServiceInfo extends RelationalServiceInfo {

    public OracleServiceInfo(String id, String url) 
    {       
     super(id, url, "oracle");
    }

 @Override
 public String getJdbcUrl() 
 {
  return String.format("jdbc:%s:thin:%s/%s@%s:%d/%s",
                              "oracle",
                              super.getUserName(), super.getPassword(),
                              super.getHost(), super.getPort(), super.getPath());
 }
      
} 

OracleDataSourceCreator.java

  
package org.cloudfoundry.samples.music.cloud;

import org.cloudfoundry.samples.music.cloud.OracleServiceInfo;
import org.springframework.cloud.service.relational.DataSourceCreator;

public class OracleDataSourceCreator extends DataSourceCreator<OracleServiceInfo> {
    private static final String[] DRIVERS = new String[]{"oracle.jdbc.OracleDriver"};

    private static final String VALIDATION_QUERY = "SELECT 'Y' from dual";

    public OracleDataSourceCreator() {
     super("spring-cloud.oracle.driver", DRIVERS, VALIDATION_QUERY);
    }
} 

UserProvidedOracleServiceInfoCreator.java

  
package org.cloudfoundry.samples.music.cloud;

import org.springframework.cloud.cloudfoundry.CloudFoundryServiceInfoCreator;

import java.util.Map;

public class UserProvidedOracleServiceInfoCreator extends CloudFoundryServiceInfoCreator<OracleServiceInfo> {
    public UserProvidedOracleServiceInfoCreator() {
        super("user-provided");
    }

    @Override
    @SuppressWarnings("unchecked")
    public boolean accept(Map<String, Object> serviceData) {
        String label = (String) serviceData.get("label");

        if (label.equals(getTag())) {
            Map<String, Object> credentials = (Map<String, Object>) serviceData.get("credentials");
            String uri = (String) credentials.get("uri");

            if (uri != null && uri.startsWith("oracle:")) {
                return true;
            }
        }

        return false;
    }

    @Override
    @SuppressWarnings("unchecked")
    public OracleServiceInfo createServiceInfo(Map<String, Object> serviceData) {
        String id = (String) serviceData.get("name");
        Map<String, Object> credentials = (Map<String, Object>) serviceData.get("credentials");
        String uri = credentials.get("uri").toString();
        return new OracleServiceInfo(id, uri);
    }
}  

OracleRepositoryConfig.java

  
package org.cloudfoundry.samples.music.config.data;

import org.hibernate.dialect.MySQL5Dialect;
import org.hibernate.dialect.Oracle10gDialect;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;

@Configuration
@Profile("oracle")
@EnableJpaRepositories("org.cloudfoundry.samples.music.repositories.jpa")
public class OracleRepositoryConfig extends AbstractJpaRepositoryConfig {

    protected String getHibernateDialect() {
        return Oracle10gDialect.class.getName();
    }

}

RelationalCloudDataSourceConfig.java

  
package org.cloudfoundry.samples.music.config.data;

import org.springframework.cloud.config.java.AbstractCloudConfig;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;

import javax.sql.DataSource;

@Configuration
@Profile({"mysql-cloud", "postgres-cloud", "oracle-cloud"})
public class RelationalCloudDataSourceConfig extends AbstractCloudConfig {

    @Bean
    public DataSource dataSource() {
        return connectionFactory().dataSource();
    }

} 

src/main/resources/META-INF/servicesorg.springframework.cloud.cloudfoundry.CloudFoundryServiceInfoCreator

org.cloudfoundry.samples.music.cloud.UserProvidedMysqlServiceInfoCreator
org.cloudfoundry.samples.music.cloud.UserProvidedOracleServiceInfoCreator

src/main/resources/META-INF/services/org.springframework.cloud.service.ServiceConnectorCreator

org.springframework.cloud.service.relational.MysqlDataSourceCreator
org.cloudfoundry.samples.music.cloud.OracleDataSourceCreator
org.springframework.cloud.service.relational.PostgresqlDataSourceCreator
org.springframework.cloud.service.keyval.RedisConnectionFactoryCreator
org.springframework.cloud.service.document.MongoDbFactoryCreator
org.springframework.cloud.service.messaging.RabbitConnectionFactoryCreator
org.springframework.cloud.service.smtp.MailSenderCreator

Finally you need to ensure that ojdbc6.jar is shipped with the WAR file within "WEB-INF/lib" which is the Oracle JDBC driver to enable the application to connect to oracle.

More Information on Pivotal CF

http://www.gopivotal.com/products/cloud-foundry



http://feeds.feedburner.com/TheBlasFromPas

Thymeleaf offers a set of Spring integrations that allow you to use it as a full-featured substitute for JSP in Spring MVC applications.

So here is a simple example just so we can compare how things differ in HTML versus JSP, does remind me of JSF / Viewlets to be honest.

1. Let start with what our simple project looks like in IntelliJ


2. The web.xml is defined as follows

  
<web-app version="2.4"
 xmlns="http://java.sun.com/xml/ns/j2ee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee 
 http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd">

 <display-name>Spring MVC Application</display-name>

    <servlet>
  <servlet-name>mvc-dispatcher</servlet-name>
  <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
        <load-on-startup>1</load-on-startup>
 </servlet>

    <servlet-mapping>
        <servlet-name>mvc-dispatcher</servlet-name>
        <url-pattern>/</url-pattern>
    </servlet-mapping>

</web-app> 

3. The spring context XML is defined as follows

WEB-INF/mvc-dispatcher-servlet.xml

  
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:context="http://www.springframework.org/schema/context"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd">

    <context:component-scan base-package="pivotal.au.pas.springapp.mvc"/>

    <bean id="templateResolver"
          class="org.thymeleaf.templateresolver.ServletContextTemplateResolver">
        <property name="prefix" value="/WEB-INF/resources/templates/" />
        <property name="suffix" value=".html" />
        <property name="templateMode" value="HTML5" />
    </bean>

    <bean id="templateEngine"
          class="org.thymeleaf.spring3.SpringTemplateEngine">
        <property name="templateResolver" ref="templateResolver" />
    </bean>

    <bean class="org.thymeleaf.spring3.view.ThymeleafViewResolver">
        <property name="templateEngine" ref="templateEngine" />
    </bean>

</beans>  

4. Now the PeopleController.java class looks just like it would when rendering a JSP page view and so it should all we have done here is change what the view page will be which in this case is HTML itself. I have created some mock up data , rather then connecting to a RDBMS here.

  
package pivotal.au.pas.springapp.mvc;

import org.springframework.stereotype.Controller;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.RequestMapping;
import pivotal.au.pas.springapp.beans.Person;

import java.util.ArrayList;
import java.util.List;

@Controller
public class PeopleController
{
    private static List<Person> people = null;

    static
    {
        people = new ArrayList<Person>();
        people.add(new Person(1, "pas"));
        people.add(new Person(2, "lucia"));
        people.add(new Person(3, "lucas"));
        people.add(new Person(4, "siena"));
    }

    @RequestMapping("/showallpeople")
    public String showallpeople(Model model)
    {
        model.addAttribute("people", people);
        return "people";
    }
}    

5. The view page is just a HTML page as follows

WEB-INF/resources/templates/people.html

  
<!DOCTYPE html>
<html xmlns:th="http://www.thymeleaf.org">
<head>
    <title>Spring MVC with Thymeleaf - All people</title>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
</head>
<body>
<h2> Spring MVC with Thymeleaf - All people </h2>

<table border="1">
    <thead>
    <tr>
        <th>Id#</th>
        <th>Name</th>
    </tr>
    </thead>
    <tbody>
    <tr th:each="var : ${people}">
        <td th:text="${var.id}"></td>
        <td th:text="${var.name}"></td>
    </tr>
    </tbody>
</table>

<br />
<hr />
<br />
<address>
    <a href="mailto:papicella@gopivotal.com">Pas Apicella</a>
</address>
</body>
</html>

6. So here is a simple JSP page as the entry page, could of been a HTML page.

index.jsp

  
<%@ page contentType="text/html;charset=UTF-8" language="java" %>
<html>
<head>
    <title>Spring MVC with Thymeleaf</title>
</head>
<body>
<h2>Spring MVC with Thymeleaf</h2>
<ul>
    <li><a href="/greeting">Greeting</a></li>
    <li><a href="/showallpeople">Show All People</a></li>
</ul>
<hr />
<br />
<address>
    <a href="mailto:papicella@gopivotal.com">Pas Apicella</a>
</address>
</body>
</html>   

7. Run index.jsp

8. Finally when run and I click on "Show All People" link we get display as follows


More Information

http://spring.io/guides/gs/serving-web-content/

http://www.thymeleaf.org/index.html

http://feeds.feedburner.com/TheBlasFromPas

Spring XD comes with several batch import and export modules. You can run them out of the box and show using the "Import CSV Files to JDBC (filejdbc)" as shown below.

This example show's inserting data into GemFireXD, you can use any RDBMS which supports a JDBC driver.

Note: XD_BASE = /Users/papicella/vmware/software/spring/spring-xd-1.0.0.M4

1. Configure connection to RDBMS in "$XD_BASE/xd/config/batch-jdbc-import.properties".

# Setting for the JDBC batch import job module
driverClass=com.vmware.sqlfire.jdbc.ClientDriver
url=jdbc:sqlfire://10.32.240.113:1527
username=APP
password=APP

2. Add jdbc jar file to $XD_BASE/xd/lib directory, in this case GemFireXD which uses "sqlfireclient.jar"

3. Start SpringXD single node using "./$XD_BASE/xd/bin/xd-singlenode"

4. Create a file called people.csv with contents as follows

[Fri Jan 10 14:35:00 papicella@:~/vmware/software/spring/spring-xd-1.0.0.M4/files ] $ cat people.csv
1,pas
2,lucia
3,lucas
4,siena

5. Log into SpringXD shell as shown below using "$XD_BASE/shell/bin/xd-shell"
6. Create table in GemFireXD as shown below.

  
sqlf> create table people (id int, name varchar(20));
0 rows inserted/updated/deleted 

7. Create a JOB as shown below

xd:>job create myjob --definition "filejdbc --resources=file:/Users/papicella/vmware/software/spring/spring-xd-1.0.0.M4/files/*.csv --names=id,name --tableName=people"Successfully created and deployed job 'myjob'
8. Start the JOB

xd:>job launch myjob
Successfully launched the job 'myjob'

9. Verify that the CSV data has been inserted into the table PEOPLE in GemFireXD

  
sqlf> select * from people;
ID         |NAME                
--------------------------------
2          |lucia               
1          |pas                 
4          |siena               
3          |lucas               

4 rows selected 

More Information

For more information on SpringXD or GemFireXD see the links below.

http://projects.spring.io/spring-xd/ - SpringXD

http://gopivotal.com/products/pivotal-hd - GemFireXD Betahttp://feeds.feedburner.com/TheBlasFromPas

The following examples shows what we can do out of the box with SpringXD TAIL input source and JDBC sink as the output source. This can all be done without writing and code.

Assumes you have SpringXD installed and GemFireXD installed. See the getting stared guide which explains how to setup SpringXD. As for a database output for JDBC you can use any RDBMS here so you could use MYSQL , ORACLE etc if you don't have GemFireXD installed.

Note: XD_BASE = /Users/papicella/vmware/software/spring/spring-xd-1.0.0.M4

1. Configure connection to RDBMS in "$XD_BASE/xd/config/jdbc.properties".

driverClass=com.vmware.sqlfire.jdbc.ClientDriver
url=jdbc:sqlfire://10.32.240.113:1527
username=APP
password=APP

2. Add jdbc jar file to $XD_BASE/xd/lib directory, in this case GemFireXD which uses "sqlfireclient.jar"

3. Start SpringXD single node using "./$XD_BASE/xd/bin/xd-singlenode"

./$XD_BASE/xd/bin/xd-singlenode
.....

11:29:00,417  INFO main container.XDContainer:175 - started container: 0
11:29:00,424  INFO main launcher.LocalContainerLauncher:95 - 
 _____                           __   _______
/  ___|          (-)             \ \ / /  _  \
\ `--. _ __  _ __ _ _ __   __ _   \ V /| | | |
 `--. \ '_ \| '__| | '_ \ / _` |  / ^ \| | | |
/\__/ / |_) | |  | | | | | (_| | / / \ \ |/ /
\____/| .__/|_|  |_|_| |_|\__, | \/   \/___/
      | |                  __/ |
      |_|                 |___/
1.0.0.M4                         eXtreme Data

Using local mode JMX is disabled for XD components
XD Configuration:
XD_HOME=/Users/papicella/vmware/software/spring/spring-xd-1.0.0.M4/xd
XD_TRANSPORT=local
XD_STORE=memory
XD_ANALYTICS=memory
XD_HADOOP_DISTRO=hadoop12

started container : 85631@localhost
Documentation: https://github.com/SpringSource/spring-xd/wiki

4. Create a empty file that the stream will read from using TAIL as the input

> touch filetest

eg:

[Fri Jan 10 11:29:06 papicella@:/tmp/xd/input ] $ ls -la /tmp/xd/input/filetest 
-rw-r--r--  1 papicella  wheel  0 10 Jan 11:29 /tmp/xd/input/filetest

5. Log into SpringXD shell as shown below using "$XD_BASE/shell/bin/xd-shell"
[Fri Jan 10 10:46:39 papicella@:~/vmware/software/spring/spring-xd-1.0.0.M4/shell/bin ] $ ./xd-shell _____                           __   _______/  ___|          (-)             \ \ / /  _  \\ `--. _ __  _ __ _ _ __   __ _   \ V /| | | | `--. \ '_ \| '__| | '_ \ / _` |  / ^ \| | | |/\__/ / |_) | |  | | | | | (_| | / / \ \ |/ /\____/| .__/|_|  |_|_| |_|\__, | \/   \/___/      | |                  __/ |      |_|                 |___/eXtreme Data1.0.0.M4 | Admin Server Target: http://localhost:9393Welcome to the Spring XD shell. For assistance hit TAB or type "help".
6. Create a STREAM as shown below.

xd:> stream create --name filetest --definition "tail --name=/tmp/xd/input/filetest |  jdbc --columns=id,name"Created new stream 'filetest'
7. Create table in database as follows.

  
sqlf> create table filetest (id int, name varchar(20));

8. Create file as follows called "testfile" we will copy this to stream file soon.
[Fri Jan 10 11:38:44 papicella@:~/vmware/software/spring/spring-xd-1.0.0.M4 ] $ cat filetest{"id":"1","name":"pas"}{"id":"2","name":"lucia"}{"id":"3","name":"lucas"}{"id":"4","name":"siena"}
7. Copy file "filetest" above to /tmp/xd/input/filetest
9. Check base table in RDBMS for data inserts.

  
sqlf> select * from filetest;
ID         |NAME                
--------------------------------
4          |siena               
1          |pas                 
3          |lucas               
2          |lucia               

4 rows selected

More Information

For more information on SpringXD or GemFireXD see the links below.

http://projects.spring.io/spring-xd/ - SpringXD

http://gopivotal.com/products/pivotal-hd - GemFireXD Beta



http://feeds.feedburner.com/TheBlasFromPas

Like GemFire 7 , GemFireXD now includes Pulse. GemFire XD Pulse is a Web Application that provides a graphical dashboard for monitoring vital, real-time health and performance of GemFire XD clusters, members, and tables.

Use Pulse to examine total memory, CPU, and disk space used by members, uptime statistics, client connections, WAN connections, query statistics, and critical notifications. Pulse communicates with a GemFire XD JMX manager to provide a complete view of your GemFire XD deployment. You can drill down from a high-level cluster view to examine individual members and tables within a member.

By default, GemFire XD Pulse runs in a Tomcat server container that is embedded in a GemFire XD JMX manager node which you can enable by starting your locator as follows.

sqlf locator start -peer-discovery-address=localhost -peer-discovery-port=41111 -jmx-manager-start=true -jmx-manager-http-port=7075 -conserve-sockets=false -client-bind-address=localhost -client-port=1527 -dir=locator -sync=false

Then you just need to start a browser, point to http://locator-ip:7075/pulse and login as "admin/admin".

Screen Shot Below.


http://feeds.feedburner.com/TheBlasFromPas

The CREATE TYPE statement creates a user-defined type (UDT). A UDT is a serializable Java class whose instances are stored in columns. The class must implement the java.io.Serializable interface. In this example below we create a TYPE with just one string property to highlight how it's done and then how we can create a FUNCTION to allow us to insert the TYPE using sqlf command line.

1. Create 2 Java classes as shown below. One is our UDT class while the other is used to create an instance of it and expose it as a FUNCTION

SqlfireString.java

 
package pas.au.apples.sqlfire.types;

public class SqlfireString implements java.io.Serializable
{
 public String value;
 
 public SqlfireString()
 { 
 }
 
 public SqlfireString(String value)
 {
  this.value = value;
 }

 @Override
 public String toString() 
 {
  return value;
 }
}

SqlfireStringFactory.java

  
package pas.au.apples.sqlfire.types;

public class SqlfireStringFactory 
{
 public static SqlfireString newInstance (String s)
 {
  return new SqlfireString(s);
 }
}

2. Create TYPE as shown below

  
CREATE TYPE LARGE_STRING 
EXTERNAL NAME 'pas.au.apples.sqlfire.types.SqlfireString' LANGUAGE JAVA;

3. Create FUNCTION to enable us to use TYPE

  
CREATE FUNCTION udt_large_string(VARCHAR(32672)) RETURNS LARGE_STRING 
LANGUAGE JAVA
PARAMETER STYLE JAVA
NO SQL
EXTERNAL NAME 'pas.au.apples.sqlfire.types.SqlfireStringFactory.newInstance';  

4. Create TABLE using TYPE and insert data

  
create table udt_table (id int, text LARGE_STRING);

insert into udt_table values (1, udt_large_string('pas')); 

Note: When using JDBC you would use a PreparedStatement and setObject method to add the TYPE column as shown below.

  
SqlfireString udtLargeString = new SqlfireString("pas");        
pstmt.setObject(1, udtLargeString);

More Information

http://gemfirexd-05.run.pivotal.io/index.jsp?topic=/com.pivotal.gemfirexd.0.5/reference/language_ref/rrefsqljcreatetype.htmlhttp://feeds.feedburner.com/TheBlasFromPas

Previously I blogged about using the HikariCP with Pivotal GemFireXD as per the post below.

http://theblasfrompas.blogspot.com.au/2013/12/hikaricp-connection-pool-with-pivotal.html

In this example I show how you would use this Connection Pool with Spring.

1. Ensure your using version 1.2.1 of HikariCP as it provides the setDataSourceProperties() setter to HikariConfig to allow easier configuration though Spring. Example below is for Maven.

  
<dependencies>
        <dependency>
            <groupId>com.zaxxer</groupId>
            <artifactId>HikariCP</artifactId>
            <version>1.2.1</version>
            <scope>compile</scope>
        </dependency> 

2. Create a spring configuration XML file as shown below.

  
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:util="http://www.springframework.org/schema/util"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
                           http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-3.1.xsd">

    <bean id="hikariConfig" class="com.zaxxer.hikari.HikariConfig">
      <property name="maximumPoolSize" value="10" />
      <property name="minimumPoolSize" value="2" />
      <property name="dataSourceClassName" 
                value="com.vmware.sqlfire.internal.jdbc.ClientDataSource" />
      <property name="dataSourceProperties" ref="props" />
      <property name="poolName" value="springHikariCP" />
    </bean>

    <bean id="dataSource" class="com.zaxxer.hikari.HikariDataSource">
          <constructor-arg ref="hikariConfig" />
    </bean>

    <util:properties id="props" location="classpath:datasource.properties"/>

</beans>

3. Create a datasource.properties as shown below.

portNumber=1527
serverName=192.168.1.6
user=app
password=app

4. Finally access with code as follows

ApplicationContextHolder.java

  
package pivotal.au.gemfirexd.demos.connectionpool.spring;

import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;

public class ApplicationContextHolder
{
    private static ClassPathXmlApplicationContext applicationContext;

    public static ClassPathXmlApplicationContext getInstance()
    {
        if(applicationContext == null)
        {
            applicationContext = new ClassPathXmlApplicationContext("classpath:application-context.xml");
        }
        return applicationContext;
    }

    public static ApplicationContext getInstance(String contextLocation)
    {
        if(applicationContext == null)
        {
            applicationContext = new ClassPathXmlApplicationContext(contextLocation);
        }
        return applicationContext;
    }
}

Accessing as follows

  
package pivotal.au.gemfirexd.demos.connectionpool.spring;

import com.zaxxer.hikari.HikariDataSource;
import org.springframework.context.ApplicationContext;

import java.sql.Connection;
import java.sql.SQLException;
import java.util.logging.Level;
import java.util.logging.Logger;
import java.sql.*;

public class TestHikariCPSpring
{
    private ApplicationContext context;
    private Logger logger = Logger.getLogger(this.getClass().getSimpleName());

    public void run() throws SQLException
    {
        context = getContext();
        HikariDataSource ds = (HikariDataSource) context.getBean("dataSource");
        Connection conn = ds.getConnection();

..... 

http://feeds.feedburner.com/TheBlasFromPas

I decided to try out the HikariCP as per the link below it says it's the fastest Connection Pool and the most lightweight. It probably is so I thought I would set it up for GemFireXD.

Quote: There is nothing faster.1 There is nothing more correct. HikariCP is a "zero-overhead" production-quality connection pool. Coming in at roughly 50Kb, the library is extremely light.

https://github.com/brettwooldridge/HikariCP

Example Below.

1. Create a Pool class as follows

  
package pivotal.au.gemfirexd.demos.connectionpool;

import com.zaxxer.hikari.HikariConfig;
import com.zaxxer.hikari.HikariDataSource;

import java.sql.Connection;
import java.sql.SQLException;

public class HikariGFXDPool
{

    private static HikariGFXDPool instance = null;
    private HikariDataSource ds = null;

    static
    {
        try
        {
            instance = new HikariGFXDPool();
        }
        catch (Exception e)
        {
            throw new RuntimeException(e.getMessage(), e);
        }

    }

    private HikariGFXDPool()
    {
        HikariConfig config = new HikariConfig();
        config.setMaximumPoolSize(10);
        config.setMinimumPoolSize(2);

        config.setDataSourceClassName("com.vmware.sqlfire.internal.jdbc.ClientDataSource");

        config.addDataSourceProperty("portNumber", 1527);
        config.addDataSourceProperty("serverName", "192.168.1.6");
        config.addDataSourceProperty("user", "app");
        config.addDataSourceProperty("password", "app");

        ds = new HikariDataSource(config);
    }

    public static HikariGFXDPool getInstance ()
    {
        return instance;
    }

    public Connection getConnection()  throws SQLException
    {
        return ds.getConnection();
    }

}

2. Create a Test Class as follows

  
package pivotal.au.gemfirexd.demos.connectionpool;

import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.logging.Level;
import java.util.logging.Logger;

public class TestPool
{
    private Logger logger = Logger.getLogger(this.getClass().getSimpleName());

    public void run () throws SQLException
    {
        Statement   stmt = null;
        ResultSet   rset = null;
        Connection  conn = null;

        HikariGFXDPool pool = HikariGFXDPool.getInstance();

        try
        {
            conn = pool.getConnection();
            stmt = conn.createStatement();
            rset = stmt.executeQuery("select * from dept order by 1");
            while (rset.next())
            {
                System.out.println(String.format("Dept[%s, %s, %s]",
                                        rset.getInt(1),
                                        rset.getString(2),
                                        rset.getString(3)));
            }
        }
        catch (SQLException se)
        {
            logger.log(Level.SEVERE, se.getMessage());
        }
        finally
        {
            if (stmt != null)
            {
                stmt.close();
            }
            
            if (rset != null)
            {
                rset.close();
            }
            
            if (conn != null)
            {
                conn.close();
            }

        }

    }

    public static void main(String[] args) throws SQLException
    {
        TestPool test = new TestPool();
        test.run();
    }
} 

3. Run Test class and verify output as follows


SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Dept[10, ACCOUNTING, NEW YORK]
Dept[20, RESEARCH, DALLAS]
Dept[30, SALES, CHICAGO]
Dept[40, OPERATIONS, BRISBANE]
Dept[50, MARKETING, ADELAIDE]
Dept[60, DEV, PERTH]
Dept[70, SUPPORT, SYDNEY]http://feeds.feedburner.com/TheBlasFromPas

We recently released Pivotal GemFireXD BETA with the latest release of Pivotal HD. Pivotal GemFire XD is a memory-optimized, distributed data store that is designed for applications that have demanding scalability and availability requirements. With GemFire XD you can manage data entirely using in-memory tables, or you can persist very large tables to local disk store files or to a Hadoop Distributed File System (HDFS) for big data deployments.

In this post we show how to add the popular INITCAP function to GemFireXD or SQLFire.

1. Create a class with a public static method as shown below , this class method will do the INITCAP for us in SQL terms.

  
package pivotal.au.accelarator.gemfirexd.sql.functions;

import org.apache.commons.lang3.StringUtils;
import org.apache.commons.lang3.text.WordUtils;

public class GemFireXdSqlFunctions
{
    public static String initcap (String input)
    {
        return WordUtils.capitalizeFully(input);
    }
} 

2. Add the class to the classpath of the distributed system.

3. Create a stored function called INITCAP to use the method defined at #1

  
CREATE FUNCTION INITCAP (data VARCHAR(32672)) RETURNS VARCHAR(32672)
LANGUAGE JAVA
EXTERNAL NAME 'pivotal.au.accelarator.gemfirexd.sql.functions.GemFireXdSqlFunctions.initcap'
PARAMETER STYLE JAVA; 

4. Test as shown below.

  
sqlf> connect client 'localhost:1527';
sqlf> select initcap('pas APICELLA') from sysibm.sysdummy1;
1                                                                                                                               
--------------------------------------------------------------------------------------------------------------------------------
Pas Apicella                                                                                                                    

1 row selected  

For more information on Pivotal GemFireXD see the link below.

http://gopivotal.com/products/pivotal-hd

http://feeds.feedburner.com/TheBlasFromPas