Showing posts with label Java. Show all posts
Showing posts with label Java. Show all posts

Wednesday, 18 May 2016

Web Service Client with Basic Authentication and SSL

Web Service Client with Basic Authentication and SSL

Recently, I had to create a web service client for a web service that uses a number of Web Service Policies. In general, the web service utilizes the following policies:
  • Transport: Service uses one way certificates. Client had to download and check server´s certificate in order to prove the server´s  identity.
  • Authentication: Basic authentication is required to access the URL and the service WSDL.
The following steps were used.
  • Creating the TrustStore: Access the Web Service URL, download the web service certificate and create a x509 trustStore to host the server´s certificate.
  • Create the client Stub: Access the Web Service URL and create the client stub by compiling the WSDL with wsimport.
  • Code and complete the service client. This has the following sub tasks:
    • Code the client to use Basic Authentication
    • Code the client to utilize the trustStore in order to setup SSL session with the server
    • Code the client to call the web method.

Creating the SSL Trustore.

During SSL handshake, the trustStore is used to verify server´s id.
Download the Server´s certificate by hitting the Web Service URL. There you will be prompted for login. You can login with the given user/password.

Then, the certificate is stored in your browser. You can export it easy but that depends to you browser. Chrome for example, the certificate can be downloaded directly as a x509 trustStore like the following image illustrates:



If you want to create the a trustStore manually you need to create a X509 keystore file using Java keytool and then import the server´s public certificate in it. The trustStore will be password protected and the certificate inside the trustStore will be password protected using "password" passphrase:

$ keytool -genkey -alias replserver -keyalg RSA -keystore mykeystore.jks -dname "cn=localhost, ou=IT, o=Continuent, c=DE"  -storepass password -keypass password

Now you have the keyStore. Next you need to import the server´s public certificate in it. In the general case, supposing the Server certificate is the following one plain text file server-certificate.txt then do one of the following actions to:

Check the server´s certificate:
openssl x509 -in server-certificate.txt -text -noout

Delete previous certificate version from the trustStore if any:
keytool -delete -alias myserver-name.com  -keystore mykeystore.jks 

Re-import the server certificate to the trustStore:
keytool -import -alias myserver-name.com -keystore mykeystore.jks  -file server-certificate.txt

Access the Web Service URL and create the client stub by compiling the WSDL with wsimport.

After running your wsimport command directly you should get a message complaining about a missing web authorization file.
What you need to do is create an authorization file (usually the default name/location for it is $HOME_DIRECTORY/.metro/auth, but check the previous error message, you'll get the hint from there).
Inside this file you just write the line: "https://username:password@url?wsdl"

 Now create a file called: wsimport_mysvc.bat and code the following commands:
setlocal
set _JAVA_OPTIONS=%_JAVA_OPTIONS% -Djavax.net.ssl.trustStore=mykeystore.jks -Djavax.net.ssl.keyStorePassword=changeit -Djavax.net.ssl.trustStore=mykeystore.jks
wsimport -s . -verbose -keep -p gr.illumine.wsclient.stub  -extension https://myserver-name.com/wsd/alc_interface?wsdl
endlocal

Doing so, you fulfill both conditions for basic authentication and also for transport/SSL by asking wsimport to examine what is been sent from server against to what is stored in mykeystore.jks

Run the wsimport_mysvc.bat and the client stub files will be created in the package gr.illumine.wsclient.stub

C:\>set _JAVA_OPTIONS= -Djavax.net.ssl.trustStore=cacerts -Djavax.
net.ssl.keyStorePassword=changeit -Djavax.net.ssl.trustStore=cacerts

C:\>wsimport -s . -verbose -keep -p gr.illumine.wsclient.stub  -extension https://myserver-name.com/wsd/alc_interface?wsdl
Picked up _JAVA_OPTIONS:  -Djavax.net.ssl.trustStore=cacerts -Djavax.net.ssl.key
StorePassword=changeit -Djavax.net.ssl.trustStore=cacerts
parsing WSDL...

Code the client

The first thing you have to do is to add a static initializer that will provide the username and password for basic authentication:

public class AlcClient {
 
 private static final Logger log= Logger.getLogger( AlcClient.class.getName() );
 
 /* 
  * Use this static initializer to provide Basic Authentication for the Web Service Consumption
  */
 static {
     java.net.Authenticator.setDefault(new java.net.Authenticator() {

         @Override
         protected java.net.PasswordAuthentication getPasswordAuthentication() {
             return new java.net.PasswordAuthentication("happyuser", "mypassword".toCharArray());
         }
     });
 }

Next, configure your SSL settings in the code, by adding the following system parameters:
        /*
         * Use the following settings to specify how this client will utilize the X509 trust store
         * called mykeystore.jks. In this trustore, it is stored the server´s public certificate
         * Also the trustore/keystores are password protected with a password "password"
         */
        System.setProperty("java.protocol.handler.pkgs","com.sun.net.ssl.internal.www.protocol");
        System.setProperty("javax.net.ssl.keyStore","mykeystore.jks");
        System.setProperty("javax.net.ssl.keyStorePassword","password");
        System.setProperty("javax.net.ssl.keyStoreType", "JKS");
        System.setProperty("javax.net.ssl.trustStore","mykeystore.jks");
        System.setProperty("javax.net.ssl.trustStorePassword","password");
        System.setProperty("javax.net.ssl.trustStoreType", "JKS");

Then add some debugging options to debug your SSL session. You are strongly advised to comment out the following code after testing it since it will affect the SSL performance.
        /* Following options enable logging of all communication to the console
         * We are most interested in the request response SOAP Messages   */
        System.setProperty("com.sun.xml.ws.transport.http.client.HttpTransportPipe.dump", "true");
        System.setProperty("com.sun.xml.internal.ws.transport.http.client.HttpTransportPipe.dump", "true");
        System.setProperty("com.sun.xml.ws.transport.http.HttpAdapter.dump", "true");
        System.setProperty("com.sun.xml.internal.ws.transport.http.HttpAdapter.dump", "true");

Now code the Web Service client instance by using the stub you have created with the wsimport:
 ZALCINTERFACE_Service service = new ZALCINTERFACE_Service( new URL("myserver-name.com/wsd/alc_interface?wsdl"),
     new QName("urn:com:myserver-name:document:sap:soap:functions:mc-style", 
                      "ZALC_INTERFACE"));
  
 /*
  * From this service get the proper port
 */
 ZALCINTERFACE port = service.getZALCINTERFACE(); 

        /* Make the web service call */
        String responseMessage = port.callMyWebMethod();

Get the entire web service client java implementation can be downloaded here

Monday, 7 March 2016

Fedora 22 Apache Tomcat and Httpd. Publishing an application in minutes.

Recalling from the previous article "Quest of the Holy Cloud" I got a provider and started a simple VM over there.
One of my first actions was to baptize my server and give it a fancy hostname.
Now lets come to the juicy part. In this article I am going to build a simple application server to handle PDF trans-code to images with a custom Java application I built.
The actions I am going to demonstrate are how to:
  • Setup OpenJKD on Fedora 22
  • Install Ghostscript libraries required for my application.
  • Download, install and configure Apache Tomcat 7
  • Install and configure Apache HTTPd.
  • Installing Open JDK

Install OpenJDK

The first step is really easy. We need a JDK or a JRE in order to run Tomcat that hosts our application. The straight option is to use opensource community JAVA: OpenJDK.
To do so, I entered the following commands:
# dnf install java
Last metadata expiration check performed 1:09:31 ago on Mon Mar  7 12:20:26 2016.
...
To check where java is and what has been installed:
# which java
/bin/java
# java -version
openjdk version "1.8.0_72"
OpenJDK Runtime Environment (build 1.8.0_72-b15)
OpenJDK 64-Bit Server VM (build 25.72-b15, mixed mode)

Install Ghostscript

Most of the software I wrote rely to Ghostscript shared libraries that are called from the corresponding Java API. To install them I entered the following commands:
# dnf install ghostscript
Last metadata expiration check performed 1:15:36 ago on Mon Mar  7 12:20:26 2016.
..
The library got installed at:
# ls -lh /lib64/libgs*
..
-rwxr-xr-x. 1 root root 16M Mar 31  2015 /lib64/libgs.so.9.16

# file  /lib64/libgs.so.9.16
/lib64/libgs.so.9.16: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=6601d742a4829cb3e4fe8197f1b1457f665ce130, stripped

Install Apache Tomcat 7

Apache Tomcat 7 can be downloaded from here as a tar.gz file by picking up a binary distribution as follows:
# cd /opt
# wget http://mirror.serversupportforum.de/apache/tomcat/tomcat-7/v7.0.68/bin/apache-at-7.0.68.tar.gz
# tar -xvf apache-tomcat-7.0.68.tar.gz

Now tomcat is not provided as a service from Fedora. To do so, we need to create a simple start script in /etc/init.d:

# cd /etc/init.d
# vi tomcat
paste the following to the script tomcat:
#!/bin/bash
# start/ stop Tomcat script
# Since you are using OpneJDK put this as your java home
JAVA_HOME=/
export JAVA_HOME
PATH=$JAVA_HOME/bin:$PATH
export PATH
# Where you have placed tomcat
CATALINA_HOME=/opt/apache-tomcat-7.0.68

case $1 in
start)
sh $CATALINA_HOME/bin/startup.sh
;;
stop)
sh $CATALINA_HOME/bin/shutdown.sh
;;
restart)
sh $CATALINA_HOME/bin/shutdown.sh
sh $CATALINA_HOME/bin/startup.sh
;;
esac
exit 0
Now tomcat needs to be registered as a Linux service. To do so add those commands:
# cd /etc/init.d
# chmod 755 tomcat  
# chkconfig --add tomcat  
# chkconfig --level 234 tomcat on  
# chkconfig --list tomcat 

Installing Apache HTTPD

This comes as a standard service supported from Fedora distribution. To install it:
# dnf install httpd
...
For a very fast configuration of http you can edit httpd.conf and add a simple virtual host:
#  vi /etc/httpd/conf/httpd.conf
# add where "Listen 80" is:
Listen My.Host.IP.Here:80

    DocumentRoot "/www/illumineit.com"
    ServerName www.illumineit.com

    # Other directives here

Since in modern Cloud environments the linux firewall IP Tables may block everything, here are the commands to unlock the ports:
iptables -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
iptables -A INPUT -p tcp -m tcp --dport 443 -j ACCEPT
You can start the HTTP service and get its status:

# service httpd start
Redirecting to /bin/systemctl start  httpd.service
# service httpd status
Redirecting to /bin/systemctl status  httpd.service
 httpd.service - The Apache HTTP Server
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled; vendor preset: disabled)
   Active: active (running) since Mon 2016-03-07 14:09:27 UTC; 4s ago
 Main PID: 1760 (httpd)
   Status: "Processing requests..."
   CGroup: /system.slice/httpd.service
           ├─1760 /usr/sbin/httpd -DFOREGROUND
           ├─1761 /usr/sbin/httpd -DFOREGROUND
           ├─1762 /usr/sbin/httpd -DFOREGROUND
           ├─1763 /usr/sbin/httpd -DFOREGROUND
           ├─1764 /usr/sbin/httpd -DFOREGROUND
           └─1765 /usr/sbin/httpd -DFOREGROUND

Mar 07 14:09:27 securepdf.illumineit.com systemd[1]: Starting The Apache HTTP Server...
Mar 07 14:09:27 securepdf.illumineit.com systemd[1]: Started The Apache HTTP Server.
The deployment directory for tomcat where you can place your WAR files is: /opt/apache-tomcat-7.0.68/webapps/ since I have donwloaded and installed tomcat on /opt.
You can use WinSCP to copy your WAR file there:

# ls -lh  /opt/apache-tomcat-7.0.68/webapps/
total 27M
drwxr-xr-x. 14 root root 4.0K Mar  3 11:00 docs
drwxr-xr-x.  7 root root 4.0K Mar  3 11:00 examples
drwxr-xr-x.  5 root root 4.0K Mar  3 11:00 host-manager
drwxr-xr-x.  5 root root 4.0K Mar  3 11:00 manager
drwxr-xr-x.  3 root root 4.0K Mar  3 11:00 ROOT
drwxr-xr-x.  4 root root 4.0K Mar  4 16:59 zsecure-pdf
-rw-r--r--.  1 root root  27M Mar  4 16:59 zsecure-pdf.war

Friday, 14 March 2014

Concurrent mode failure: Tuning JVM GC for Solr

The machine
I have an 8 CPU VM server with 32GB RAM running Solr. My JVM is 1.6.0_37 with the following JVM settings:
-Xms28g
-Xmx28g
-XX:NewSize=6g
-XX:MaxNewSize=6g
-XX:SurvivorRatio=4
-XX:PermSize=512m
-XX:MaxPermSize=512m
-XX:SoftRefLRUPolicyMSPerMB=500
-XX:+PrintCommandLineFlags
-XX:+HeapDumpOnOutOfMemoryError
-XX:+DumpGCHistoryOnOutOfMemory
-XX:+DumpDetailedClassStatisticOnOutOfMemory
-XX:HeapDumpPath=/opt/alfresco/tomcat/dumps
-verbose:gc
-Xloggc:/opt/alfresco/tomcat/dumps/gc-logs/gc-2014-03-13-10-00-07.log
-XX:+GCHistory
-XX:+CMSClassUnloadingEnabled
-XX:+DisableExplicitGC
-XX:+PrintGCDateStamps
-XX:+PrintGCDetails
-XX:+PrintTenuringDistribution
-XX:+UseCompressedOops
-XX:+UseConcMarkSweepGC
-XX:+UseParNewGC

The reason for such huge heap is that the Solr data are about 130 GB and Sorl is heavily utilized from around 100 concurrent threads performing text search on documents.
I notice that Sorl application pauses for some time without responding. I discovered on the GC logs the following problem:
2014-03-17T14:29:23.438+0100: 7991.661: [GC2014-03-17T14:29:23.438+0100: 7991.661: [ParNew (promotion failed)
Desired survivor size 268435456 bytes, new threshold 15 (max 15)
- age   1:  125233576 bytes,  125233576 total
: 2621440K->2228808K(2621440K), 5.9399450 secs]2014-03-17T14:29:29.378+0100: 7997.601: [CMS2014-03-17T14:29:32.715+0100: 8000.938: [CMS-concurrent-sweep: 21.573/33.920 secs] [Times: user=118.80 sys=3.29, real=33.91 secs]
 (concurrent mode failure): 20465547K->11174034K(26214400K), 33.5774490 secs] 22674213K->11174034K(28835840K), [CMS Perm : 47873K->47648K(524288K)], 39.5176760 secs] [Times: user=46.19 sys=2.36, real=39.51 secs]

This issue is summarized in the official ORACLE documentation for JVM v6 as follows:

..a concurrent collection needs to be started at a time such that the collection can finish before the tenured generation becomes full; otherwise the application would observe longer pauses due to concurrent mode failure. There are several ways a concurrent collection can be started. 

See:  Concurrent Mode Failure

The message "concurrent mode failure" signifies that the concurrent collection of the tenured generation did not finish before the tenured generation became full. In other words, the new generation is filling up too fast, it is overflowing to tenured generation but the CMS could not clear out the tenured generation in the background. When a concurrent mode failure happens, the low pause collector does a stop-the-world (STW) collection. All the application threads are stopped, a different algorithm is used to collect the tenured generation (our particular flavor of a mark-sweep-compact), the applications threads are started again, and life goes on....

Seems that a concurrent mode failure is responsible for a "Stop the World" JVM pausing.
See also another wonderful Blog about the same issue here :

In order to treat problem we tune the following JVM flags:
-XX:CMSInitiatingOccupancyFraction=10
Indicating that a concurrent collection will start if the occupancy of the tenured generation exceeds 10% instead of 92% that is the default threshold.
-XX:CMSIncrementalSafetyFactor=100
Indicating to the JVM GC to start a concurrent collection at the next opportunity without any delay.
See also Oracles GC tunning instructions here

Tuesday, 28 January 2014

Overriding finalize() for reference count on active threads

The need for this small article came from the implementation of a custom thread controller. In my design I am using a ThreadGroup that holds threads that dynamically load classes and invoke methods of objects created on the fly.

The problem here comes with the ThreadGroup. I want to have the number of all threads in the group and we mean absolute number not something like myThreadGroup.activeCount() that shows only the active threads of the group. Normally ThreadGroup.activeCount() returns an estimate of the number of active threads in this thread group, so we cannot rely on the this.
One solution is to hold a container of references to all the new threads, but then I have to implement custom synchronized code for accessing the container and blah blah...

The solution I finally followed was to implement an object reference count on the parent class CustomPlugin. When a new object of any descendant class rooted from CustomPlugin, reference count is augmented. When a plugin exits execute() method, the curring thread exits run().
Plugin object reference terminated and destroyed by the JVM GC and the number of concurent plugins is decreased to something less than MAX_TOTAL_PLUGINS. Only then a new plugin can be created.

public abstract class CustomPlugin extends Thread implements Plugin{
 
 public static final int MAX_TOTAL_PLUGINS = 3;
 
 private static int refCount = 0;

 public CustomPlugin(String name) throws Exception{
    if(!allowLoad()){
       throw new Exception("Maximum plugin objects already loaded!");
    }
    Thread.currentThread().setName(name + "-" + Thread.currentThread().getId() );
    refCount++;
 }
 

 @Override
 protected void finalize(){
    try {
      super.finalize();
    } catch (Throwable e) {
      e.printStackTrace();
    }finally{
      refCount--; 
    }
 }

 public static boolean allowLoad() {
    return (refCount>=MAX_TOTAL_PLUGINS?false:true);
 }

 public abstract execute(String [] args);
}
Remember that you have only CustomPlugin.MAX_TOTAL_PLUGINS=3 available threads that you can use for your plugins, meaning that only MAX_TOTAL_PLUGINS can be used in total, regardless if their threads are sleeping! When a plugin exits execute() method, the curring thread exits run(). Plugin object reference terminated and destroyed by the JVM GC and the number of concurrent plugins is decreased to something less than MAX_TOTAL_PLUGINS. Only then a new plugin can be called.

Wednesday, 22 January 2014

Implementing 2-way SSL in Java using TLS and Self Signed Certificates part3

Step 3: The Client (Get the complete code here)

The client also requires the Keystore/Trustore created in Part-1

Again in the client we have to do a couple of things similar to the server:

The first is to specify the Java Keystore/Trustore we created in  Part-1 of this article:

System.setProperty("javax.net.ssl.keyStore","mysystem.jks");
System.setProperty("javax.net.ssl.keyStorePassword","welcome");

System.setProperty("javax.net.ssl.trustStore","mysystem.jks");
System.setProperty("javax.net.ssl.trustStorePassword","welcome");

Similarly with the server side described in Part-2, we have to create the client socket as an SSLSocket:

SSLSocketFactory factory = (SSLSocketFactory) SSLSocketFactory.getDefault();    
SSLSocket sslSock = (SSLSocket) factory.createSocket("localhost",8095);

The entire code of the client can be downloaded here.

Next article, Part-4, of this Blog series will assist you to debug the SSL/TLS client/server communication.

Implementing 2-way SSL in Java using TLS and Self Signed Certificates part2

Step 2: The server (Get the complete server code here)

Requires the Trustore/Keystore created in Step-1.

To write the server process in Java is pretty simple. You just have to do a couple of steps:
Specify a couple of properties so that the Trustore/Keystore can be loaded like the following code fragment shows:

System.setProperty("javax.net.ssl.keyStore","mysystem.jks");
System.setProperty("javax.net.ssl.keyStorePassword","welcome");

System.setProperty("javax.net.ssl.trustStore","mysystem.jks");
System.setProperty("javax.net.ssl.trustStorePassword","welcome");

Create the ServerSocket as anSSLServerSocketlike the following code fragment shows:
 
char ksPass[] = "welcome".toCharArray();
char ctPass[] = "welcome".toCharArray();

//Create and load the Keystore
KeyStore ks = KeyStore.getInstance("JKS");
ks.load(new FileInputStream("ianalyzer.jks"), ksPass);
KeyManagerFactory kmf = KeyManagerFactory.getInstance("SunX509");
kmf.init(ks, ctPass);

//Create the ServerSocket as an SSLServerSocket
SSLContext secureSocket = SSLContext.getInstance("TLS");
secureSocket.init(kmf.getKeyManagers(), null, null);
SSLServerSocketFactory ssf = secureSocket.getServerSocketFactory();
ssocket = (SSLServerSocket) ssf.createServerSocket(8095);
SSLServerSocket ss = (SSLServerSocket) ssocket;

//This explicitly states TLS with 2-way authentication
ss.setNeedClientAuth(true); 

The entire code for server implementation can be downloaded here.

Implementing 2-way SSL in Java using TLS and Self Signed Certificates part1

Consider that we want to implement in Java a secure communication (Transport Layer Security ) for a system called MySystem.

The problem

The security scenario for the implementation of  MySystem is simple:
  • Authentication only between peers that both share the Keystore/Trustore file
  • Session establishment only between peers that have the Keystore/Trustore file
Doing so, the entire communication between client and server requires authentication and is encrypted:



Before going further on this study, pay a visit to this site for Java SSL: ssljavaguide.

To implement the scenario, there are three basic steps:
  1. Create the Java Keystore/Trustore that will be used for Authentication and Encryption of Transport/Session. This will be used from both Client and Server parties. (Current Part)
  2. Implement the Client side: (See blog article Part-2)
  3. Implement the Server side: (See blog article Part-3)
Part-4 deals with debugging the Client/Server SSL/TLS communication.

Step 1: Create the Keystore/Trustore
Following steps of this section, results in the creation of a  Keystore/Trustore .jks file that contains:
  • MySystem Private key 
  • MySystem Selfsigned Certificate
To do so we are going to use the tools openssl  and keytool. We prefer using openssl because it can work silently - without prompt the user to put passwords, domains, server names....

The steps are:
1) Generate RSA 1024 bit private key. The key will be password protected:
openssl genrsa -out mysystem.key 1024 -passin pass:welcome

2) Generate Certificate Request for CA (.csr) using the private key
openssl req -x509 -sha256 -new -subj '/C=GR/ST=Athens/L=Chalandri/CN=mysystem'  -key mysystem.key -out mysystem.csr

3) Generate self signed certificate expiry-time 10 years from the certificate request
openssl x509 -sha256 -days 3652 -in mysystem.csr -signkey mysystem.key -out mysystem.crt


4) Import the pair (private key and selfsigned certificate) in a new JKS (Trustore/Keystore together)
First we need to create PKCS12 keystore from private key and self signed certificate.
openssl pkcs12 -export -name mysystem -in mysystem.crt -inkey mysystem.key -out mysystem.p12 -passin pass:welcome -password pass:welcome

Then we need to convert PKCS12 keystore into a JKS keystore
keytool -importkeystore -destkeystore mysystem.jks -srckeystore mysystem.p12 -srcstoretype pkcs12 -alias mysystem -srcstorepass welcome  -storepass welcome  -noprompt

At this point we have created the Java  Keystore/Trustore mysystem.jks file.

Copy mysystem.jks on both client and server machines.

Download all the commands for the Keystore/Trustore .jks file generation here

Tuesday, 5 November 2013

JBoss Monitoring from JConsole

This was a very simple issue pointed to me by a friend:
"OK.. it would be interesting to restrict jmx-console over http and instead, use JConsole to monitor our JBoss 5.0.1.GA from a VPN tunneled port"

JConsole is a very important tool for monitoring the JVM. It is located in you JAVA_HOME/bin.

This is very simple for the local JBoss process, in other words, when JConsole and JBoss are running on the same machine. You just have to start JBoss and run JConsole.

In case when JBoss is running on a different IP you just have to find the following line in your server.log:

2012-03-16 04:58:30,047 INFO  [org.jboss.mx.remoting.service.JMXConnectorServerService] (main) JMX Connector server: service:jmx:rmi://kandath/jndi/rmi://kandath:1090/jmxconnector

Get the connector server URL:
 service:jmx:rmi://kandath/jndi/rmi://kandath:1090/jmxconnector
paste it to the JConsole  as the following picture instructs:
JBoss as a local process and remote using remote JMX service

JBoss 5.1.0.GA comes preconfigured with the following: 
1.  JMX service is on and port to use for RMI by default is set to 1090.
2. No security on JMX console!
Consider seriously to secure the access to the JMX service.

Also read the following interesting bug on JBoss jira