12/29/2011

Java Dynamic Proxy Example

In this post, I will go over the basic constructs of dynamic proxy, followed by some notes and things to watch out when implementing it. First, the 5 elements of a dynamic proxy as implemented in this example:

TestImpl: the class behind the proxy, not to be directly invoked by the client. In some cases, this type may not exist.

TestIF: the proxy interface implemented by TestImpl, and the dynamic proxy. This is the interface type the client should reference. One or more interfaces are required to create dynamic proxy. If there is no such interface, you may need to dynamically generate them with tools like ASM.

TestInvocationHandler: the InvocationHandler that handles method invocation on the proxy from the client. It contains an instance of TestImpl to delegate method invocations to.

Test: the test main class.

$Proxy0: the dynamic proxy that implements TestIF, and the client-facing implementation of TestIF.

public interface TestIF {
String hello(String name);
}
public class TestImpl implements TestIF {
public String hello(String name) {
return String.format("Hello %s, this is %s", name, this);
}
}
import java.lang.reflect.*;
public class TestInvocationHandler implements InvocationHandler {
private Object testImpl;

public TestInvocationHandler(Object impl) {
this.testImpl = impl;
}

@Override
public Object invoke(Object proxy, Method method, Object[] args)
throws Throwable {
if(Object.class == method.getDeclaringClass()) {
String name = method.getName();
if("equals".equals(name)) {
return proxy == args[0];
} else if("hashCode".equals(name)) {
return System.identityHashCode(proxy);
} else if("toString".equals(name)) {
return proxy.getClass().getName() + "@" +
Integer.toHexString(System.identityHashCode(proxy)) +
", with InvocationHandler " + this;
} else {
throw new IllegalStateException(String.valueOf(method));
}
}
return method.invoke(testImpl, args);
}
}
import java.lang.reflect.*;
public class Test {
public static void main(String... args) {
TestIF t = (TestIF) Proxy.newProxyInstance(TestIF.class.getClassLoader(),
new Class<?>[] {TestIF.class},
new TestInvocationHandler(new TestImpl()));
System.out.printf("t.hello(Duke): %s%n", t.hello("Duke"));
System.out.printf("t.toString(): %s%n", t);
System.out.printf("t.hashCode(): %H%n", t);
System.out.printf("t.equals(t): %B%n", t.equals(t));
System.out.printf("t.equals(new Object()): %B%n", t.equals(new Object()));
System.out.printf("t.equals(null): %B%n", t.equals(null));
}
}
Implementation notes:

When dispatching method invocations on proxy to delegate object, 3 methods from java.lang.Object need special handling: toString(), hashCode() and equals(Object). Since they are related to the proxy object identity, they should be serviced directly by the handler. One option is to base their return values on the proxy param, as in this example.

Other public methods from java.lang.Object are all final, and are not routed to the InvocationHandler by JVM. These methods are:
public final native Class<?> getClass();
public final native void notify();
public final native void notifyAll();
public final native void wait(long timeout) throws InterruptedException;
public final void wait(long timeout, int nanos) throws InterruptedException;
public final void wait() throws InterruptedException
The handler's invoke method should not invoke methods on the proxy object passed in as the first param (except the 6 final methods listed above), to avoid infinite loop and StackOverflowError. For example, this following debug line should not be used:
// AVOID THIS
System.out.printf("proxy=%s, method=%s, args=%s%n",
proxy, method, Arrays.toString(args));
If the handler already knows how to acquire the delegate TestImpl instance, through either direct instantiation or lookup, it can hide it completely from the client. So the only type of TestIF the client knows is the proxy.

The Object[] args passed into handler's invoke method is null, when the invoked method takes no param. This can come as a surprise to many. So a null check is needed before operating on args.

To run this example after compiling all java classes:
$ java Test

t.hello(Duke): Hello Duke, this is TestImpl@a3901c6
t.toString(): $Proxy0@24a37368, with InvocationHandler TestInvocationHandler@66edc3a2
t.hashCode(): 24A37368
t.equals(t): TRUE
t.equals(new Object()): FALSE
t.equals(null): FALSE
To save/keep/dump the generated proxy class file (with non-standard option):
$ java -Dsun.misc.ProxyGenerator.saveGeneratedFiles=true Test
To view the structure of the proxy class:
$ $JAVA_HOME/bin/javap -v \$Proxy0

My Unix Linux Notes

How to extract/expand/explode file.tar.gz, or file.tgz, with any of the following commands:
gtar xzvf file.tar.gz
gnutar xzf file.tar.gz
gunzip < file.tgz | tar xvf -

Note the use of < (gunzip file.tgz | tar xvf - won't work). You can always run gunzip and tar in 2 steps, which creates an extra tar file:

gunzip file.tar.gz
tar xvf file.tar
How to find a port number (8080) in use:
netstat -an | grep 8080
How to kill all java processes:
killall java

How to list all background jobs:
jobs
[1] Running ./standalone.sh


How to kill the current background job:
kill %

How to list all java processes:
$JAVA_HOME/bin/jps
$JAVA_HOME/bin/jps -l
$JAVA_HOME/bin/jps -lv
ps -ef | grep java    # more reliable than jps


How to find a java (jboss) process with jps and kill it:
jps | grep jboss | awk '{print $1}' | xargs kill -9


How to find a jboss process with "ps -ef" and kill it:
ps -ef | grep -v grep | grep jboss | awk '{print $2}' | xargs kill -9


Note that in "ps -ef" output, the PID is the 2nd column, after UID. This is different from "ps" and "jps..." output, where PID is column 1.

How to find a file is opened by which program and user (takes a while to finish):
lsof | vim -


How to list all files in current directory in absolute path (for copying the path):
find $PWD


How to sort a list of items (e.g., file paths) and remove duplicates:
sort a.txt > b.txt
uniq b.txt > c.txt

12/05/2011

Extend Thread vs implement Runnable

There are two ways to create your own thread type: subclass java.lang.Thread class, or implementing java.lang.Runnable and pass it to Thread constructor or java.util.concurrent.ThreadFactory. What is the difference, and which one is better?

1, The practical reason is, a Java class can have only one superclass. So if your thread class extends java.lang.Thread, it cannot inherit from any other classes. This limits how you can reuse your application logic.

2, From a design point of view, there should be a clean separation between how a task is identified and defined, between how it is executed. The former is the responsibility of a Runnalbe impl, and the latter is job of the Thread class.

3, A Runnable instance can be passed to other libraries that accept task submission, e.g., java.util.concurrent.Executors. A Thread subclass inherits all the overhead of thread management and is hard to reuse.

4, Their instances also have different lifecycle. Once a thread is started and completed its work, it's subject to garbage collection. An instance of Runnalbe task can be resubmitted or retried multiple times, though usually new tasks are instantiated for each submission to ease state management.

11/27/2011

My tcsh notes

Settings to remember when I need to use tcsh in a foreign environment:

set prompt="%/ > "
# set prompt="%{\033]0;%M: %c\007%}%/ > "
set complete=enhance
set autolist=true

set colorcat
setenv GREP_OPTIONS --color=auto
setenv GREP_COLOR 32
Copy the above lines to $HOME/.tcshrc. The first 3 are the most important ones to remember. Setting prompt to "%/ > " puts the $cwd in the front of the command prompt. $cwd is the same as the all-upper-case $PWD but is easier to type.

To list current key bindings in tcsh:
$ bindkey
"^A" -> beginning-of-line
"^D" -> delete-char-or-list-or-eof
"^E" -> end-of-line
"^I" -> complete-word
"^J" -> newline
"^K" -> kill-line
"^L" -> clear-screen
"^M" -> newline
"^P" -> up-history
"^T" -> transpose-chars
"^U" -> kill-whole-line
"^W" -> kill-region
... ...
To use vi-mode or emacs-mode command-line editing and navigation:
bindkey -v
bindkey -e

11/14/2011

Why use ThreadFactory

When submitting tasks to java executor service, some thread will be allocated to perform that task. The servicing thread may be assigned from the internal thread pool, or created on-demand. Each executor service has an associated ThreadFactory, and a default ThreadFactory if the application does not specify one. For non-trivial apps, it's always a good idea to set a custom ThreadFactory.

1, To set a more descriptive thread name. With the default ThreadFactory, it gives thread names in the form of pool-m-thread-n, such as pool-1-thread-1, pool-2-thread-1, pool-3-thread-1, etc. When these threads showing up in debugger, profiler, or monitoring tool, it's hard to know their purpose and how they were started.

When analyzing a thread dump, thread name is the only clue to trace to the source where the thread pool or executor service is created.  The stack trace contains all JDK classes with java.lang.Thread.run at the very bottom.  It doesn't tell anything about your application.

2, To set thread daemon status. The default ThreadFactory produces non-daemon threads.

3, To set thread contextClassLoader to appropriate value. Threads created by the default ThreadFactory will inherit the contextClassLoader from the parent thread, which may not be what you want. In general, a pooled thread itself should not be associated with any specific class loader. Instead, the submitted task has the best knowledge about its class loading requirement, so the task Runnable or Callable should manage the thread contextClassLoader for the duration of its run method.

If the new thread does not need to access classes loaded in the parent thead, set its contextClassLoader to null. If a task needs visibility to a class loader, set its contextClassLoader at the beginning of its run method, and restore its contextClassLoader right before exiting run method.

The default ThreadFactory in JDK 6 java.util.concurrent is a package-private static nested class in java.util.concurrent.Executors:

static class DefaultThreadFactory implements ThreadFactory {
        static final AtomicInteger poolNumber = new AtomicInteger(1);
        final ThreadGroup group;
        final AtomicInteger threadNumber = new AtomicInteger(1);
        final String namePrefix;

        DefaultThreadFactory() {
            SecurityManager s = System.getSecurityManager();
            group = (s != null)? s.getThreadGroup() :
                                 Thread.currentThread().getThreadGroup();
            namePrefix = "pool-" +
                          poolNumber.getAndIncrement() +
                         "-thread-";
        }

        public Thread newThread(Runnable r) {
            Thread t = new Thread(group, r,
                                  namePrefix + threadNumber.getAndIncrement(),
                                  0);
            if (t.isDaemon())
                t.setDaemon(false);
            if (t.getPriority() != Thread.NORM_PRIORITY)
                t.setPriority(Thread.NORM_PRIORITY);
            return t;
        }
    }
Note that poolNumber is declared as static final and threadNumber as final. Each instance of ThreadFactory is tied to a unique pool, and a static poolNumber tracks the sequence of all instances of DefaultThreadFactory. An instance of ThreadFactory (and the associated pool) contains multiple threads, and the instance field threadNumber tallies all threads therein.

Both poolNumber and threadNumber fields are of type java.util.concurrent.AtomicInteger to take advantage of its atomic compound operations. Their initial value is 1 and getAndIncrement() is invoked to produce sequence like 1, 2, 3. Alternatively we could set its initial value to 0 and invoke incrementAndGet() to produce the same sequence.

Why poolNumber and threadNumber need to guard against concurrent access and modification? poolNumber is static and thus a shared data among all instances of DefaultThreadFactory. Multiple instances of DefaultThreadFactory may be incrementing it simutaneously, so it needs to be thread-safe.

As for threadNumber, the creation of threads, either initially at execution service startup time, or under heavy load, may be serialized by the execution service. But an instance of ThreadFactory can be passed around and associated with multiple execution service. Therefore, the internal state of ThreadFactory needs to be thread-safe and immutable.

11/09/2011

Notes on Java Debugger (NetBeans)

Some notes on using NetBeans Java Debugger. They also apply in principal to other Java debuggers.

1, Set line breakpoint (the most common breakpoint), optionally make them conditional breakpoint to filter out irelevant hits. Sometimes I found conditional breakpoint doesn't seem to work and the execution always pauses regardless of the condition.

2, Use field breakpoint to intercept field access and/or modification. This is very useful to track all access and mutation of the field without having to inspect its setter and getter. And some fields don't even have setter or getter methods.

3, Use method breakpoint to inspect method entry and exit. This is just a variation of line breakpoint.

4, Use class breakpoint to track class loading and unloading. This is useful when you just have a vague idea that a certain class will be executed but don't know which part. When the class breakpoint is hit, you can then note down the calling stack trace.

5, Exception breakpoint is also available but haven't tried that.

6, Rewind (step back) the calling sequence by popping off the top element in the calling stack. You can do it from Debugger menu, or the context menu of a particular thread.

7, Add debug buttons to tools bar, including "Attach Debugger ..." button. If "Attach Debugger button is not included by default, you can customize the tool bar by dragging it into the bar. This is very convenient for attaching debugger to remote java process.

8, Apply code changes dynamically to the debugging session, without restarting the remote java application. Debug > Apply Code Changes, or Apply Code Changes button in tool bar. Note that the code changes are not effective in current debugging session. To make it permanent, need to rebuild and restart the remote application.

9, You can start multiple debugging sessions by attaching to different java processes. For example, to debug both the client side and server side operations. Pay attention to where the execution is, the client side, or the server side? Sometimes you can't tell where you are by just looking at the current class name, because some classes are loaded by both the client side and the server side programs.

10, To copy the current stack trace, go to Debug > Evaluate Express, enter
Thread.dumpStack();
The stack trace will appear in the application's log file or console.

11, If you need to step into source code outside your main project, get hold of its source code if available, and create a IDE project off it. That may involve checking it out from different repo, with different scm tool, or download source zip/jar files from maven repo. A decompiler may be able to handle some simple classes, but tend to produce incomplete source code for large, complext classes.

12, Always attach JDK source code and javadoc to your IDE. NetBeans Tools > Java Platform > JDK 1.6 > Sources. You will need to download the JDK source installer and install it to disk. Source files for JDK classes are located in j2se/src/share/classes.

Finally, shortcuts:

Step Over        F8
Step Into F7
Continue F5
Breakpoints Ctrl-Shift-5
Variables Ctrl-Shift-1
Watches Ctrl-Shift-2
Terminate Debug Shift-F5

10/19/2011

Example of StackOverflowError

One cause of java.lang.StackOverflowError is a method calling itself without exit condition. The following Room class overrides toString method to produce more meaningful string representation, but is implemented wrong:

public class Room {
public static void main(String args[]) {
System.out.println("Created Room: " + new Room());
}

@Override public String toString() {
return "Room: " + this.toString();
//change to super.toString() to avoid StackOverflowError
}
}
Running Room class would fail with java.lang.StackOverflowError.

9/25/2011

Dependency Injection (DI) in Java EE

In a J2EE/JavaEE server, container has always been the backbone for creating and managing the lifecycle of resident components. It's not a surprise to see some forms of DI is already present at the beginning of J2EE. For example, javax.ejb.SessionBean, the legacy interface implemented by old-style EJB bean class, contains a setSessionContext(SessionContext ctx) method, which is called by ejb container to inject a SessionContext to the bean class. Similarly, javax.ejb.EntityBean has setEntityContext(EntityContext ctx), and javax.ejb.MessageDrivenBean has setMessageDrivenContext(MessageDrivenContext ctx) callbacks.

Another less obvious example is Servlet init methods, which are invoked by servlet container to initialize ServletContext and ServletConfig for servlet class:

//Declared in javax.servlet.Servlet interface and implemented in GenericServlet:
public void init(ServletConfig config)

//Convenience method implemented in GenericServlet
public void init();
This early forms of DI is typically defined as part of the lifecycle callbacks and strongly typed to a specific interface or abstract class.

DI capability was substantially expanded in Java EE 5 with the introduction of annotation- and xml-based DI. Both field- and setter-injections are supported, and xml deployment descriptors can be used to override or augment annotation meta data. Most common ones are:
@EJB private Calc calc;

@EJB(name="ejb/calc", beanName="DefaultCalc", beanInterface=com.my.ejb.Calc.class)

@Resource SessionContext ctx;

@Resource private String widgetName;

@Resource(mappedName="custom/widgetName")
private String widgetName;

@PersistenceContext private EntityManager em;

@PersistenceContext(unitName="advanced-pu", type=TRANSACTION)
private EntityManager em;
As you can see, the annotation type is specific to each resource type. The advantage of this approach is, you can easily customize how a specific type of dependency is constructed and initialized. This design makes sense as some resources need more initialization params than others. But the downside is lack of internal consistency across the board. For each new type of resources, we need to decide whether to reuse the existing annotations, add new attributes to existing annotations, or create a new one. This task is tackled in Java EE 6, with JSR 330 (AtInject spec) and JSR 299 (CDI spec), both are required component specs in Java EE 6 Platform.

Now in Java EE 6, all the above is doable with a single @java.inject.Inject. For injections with no attributes (or default attributes), we can easily replace @EJB/@Resource/@PersistenceContext with @Inject. Otherwise, a @javax.inject.Qualifier is needed to provider more information. Basically we are promoting annotation attributes to auxiliary annotations, replacing string-based qualifiers with typed qualifiers, and chaining them together to resolve the DI.

@EJB and @Resource are also enhanced in Java EE 6 to take an additional portable attribute lookup, mainly as a replacement for the nonportable mappedName attribute:
@EJB(lookup="java:global/myApp/myEJBModule/CalcBean")
private Calc calc;

@Resource(lookup="java:module/env/sleepSeconds")
private long sleepSeconds;

9/23/2011

JUnit Notes, Do's and Don'ts

A skeleton JUnit 4 test case:

import java.util.*;
import org.junit.*;
import static org.junit.Assert.*;

public class MyTest {
/** 4 optional lifecycle callback methods */
@Before public void before() {} //must be public void no-arg

@After public void after() {} //must be public void no-arg

@BeforeClass public static void beforeClass() {} //must be public static void no-arg

@AfterClass public static void afterClass() {} //must be public static void no-arg

@Test public void testAdd() {
int a = 1, b =2, expected = 3;
int actual = a + b;
assertEquals(expected, actual);
}
}
Do not name your test main class Test.java, which has the same short name as @Test annotation. Unless you explicitly use @org.junit.Test, @Test will resolve to your test class, instead of org.junit.Test annotation type.

Avoid using Java assert; use static assertXXX methods (statically imported) in org.junit.Assert class. Java assert is turned off by default, and can be enabled only at command line with "java -ea", or "java -enableassertions" options. So some tests using Java assert will always pass when running with the default configuration, regardless of the actual test conditions. For example, the following method passes unless -ea option is passed to java:
@Test public void testNumber() {
assert(1 < 0);
}

java -cp $tmp/junit-4.8.1.jar:. org.junit.runner.JUnitCore MyTest
JUnit version 4.8.1
Time: 0.009
OK (4 tests)
A secondary reason to favor org.junit.Assert.assertXXX over Java assert is, the former is a richer API with more descriptive output, whereas Java assert, in its common form, only fails with java.lang.AssertionError and stack trace with no description. Java assert can also provide details with a second param: assert false : reason

maven surefire plugin automatically enables Java assertions when running JUnit tests on JDK 1.4 and above. So when running with surefire, there is no problem of false positive test results. But still I would like my JUnit tests to work the same way across all testing environment. To disable Java assertion in mvn surefire:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.9</version>
<configuration>
<enableAssertions>false</enableAssertions>
</configuration>
</plugin>
</plugins>
</build>
If your tests passed in Eclipse or ant, but failed with mvn, chances are some java assertions failed the tests. Check target/surefire-reports/* for any AssertionError.

Avoid using assertTrue(list1.equals(list2)); use assertEquals(list1, list2) instead. assertTrue will only print out the useless message like "expecting true, but actual false", whereas assertEquals gives more helpful message like "expected:[null, 1, 2], but was:[1, 2]"

Do not add assertTrue(true) at the end of a @Test to signal the success result. It's just not needed. The test simply passes when nothing goes wrong.

@Test method can declare throws clause, e.g., throws NamingException. When an exception is thrown during test run, the test is failed. So use exception as a means to communicate test failures. Do not try-catch an exception, just to print it and manually fail() it. For example, do this:
@Test public void testNumber() {
int i = 10 / 0;
}
Do NOT do this:
@Test public void testNumber() {
try {
int i = 10 / 0;
} catch (ArithmeticException e) {
e.printStackTrace();
fail(); // or even worse, assert(false);
}
}
When running the try-catch version, ArithmeticException stack trace is printed, followed by another stack trace of java.lang.AssertionError from calling fail(), whereas in the shorter version, only the ArithmeticException is logged and hence much easier to trace.

How to run a single test? org.junit.runner.JUnitCore, the default command-line runner, takes a list of test classes as input, but doesn't support running a single test method. Other tools and IDE may support it by providing their own runner. With maven surefire 2.7.3+ and JUnit 4 tests, it is supported with a -D sysprop (link):
mvn test -Dtest=com.my.test.TestMain#test1
mvn test -Dtest=com.my.test.TestMain#*egative*
mvn org.apache.maven.plugins:maven-surefire-plugin:2.8:test -Dtest=com.my.test.TestMain#test1
I prefer to put the test name at the very end of the line so it's easier to go through the history and edit command.

To check the version of your JUnit:
$ java -cp $tmp/junit.jar junit.runner.Version
4.8.1
To skip or exclude a test, use @Ignore (org.junit.Ignore).

8/09/2011

Singleton, DCL, Holder Idiom, and When Singleton is not a Singleton

For eager-initialization singleton, the simplest way is to use an enum type with only 1 INSTANCE.  The limitation is your enum singleton class cannot extend from a super class.

public enum EnumSingleton implements Calc {
    INSTANCE;

    public int add(int a, int b) {
        return a + b;
    }
}
A traditional, non-enum implementation of eager-initialization singleton is:
public class EagerSingleton {
    private static EagerSingleton instance = new EagerSingleton();

    private EagerSingleton() {}

    public static EagerSingleton getInstance() { return instance; }
} 
A lazy-initialization singleton implemented with double-checked locking (DCL), which works in Java 5 onward:
package test.concurrent;
public final class MySingleton1 {
 private static volatile MySingleton1 instance;

 private MySingleton1() {}

 public static MySingleton1 getInstance() {
   MySingleton1 result = instance;
   if(result == null) {
     synchronized (MySingleton1.class) {
       result = instance;
       if(result == null) {
         instance = result = new MySingleton1();
       }
     }
   }
   return result;
 }
A lazy-initialization singleton implemented with holder idoim (applicable to lazy static field).  The limitation is the instance can only be created with default constructor, and there is no way to pass params from getInstance(...) to the constructor.  In that case, use DCL instead:
package test.concurrent;
public final class MySingleton2 {
 private MySingleton2() {}

 private static class Holder {
   private static final MySingleton2 instance = new MySingleton2();
 }

 public static MySingleton2 getInstance() {
   return Holder.instance;
 }
A main method that spawns multiple threads calling getInstance():
public static void main(String[] args) {
   int numOfThreads = Integer.parseInt(args[0]);
   for(int i = 0; i < numOfThreads; i++) {
     Thread t = new Thread(new Runnable() {
       public void run() {
         System.out.println(MySingleton1.getInstance());
       }
     });
     t.start();
   }
 }
When the same singleton class is loaded by different class loaders, each loader has its own class data. MySingleton1.class loaded by classloader1 is a distinct class from MySingleton1.class loaded by classloader2. Hence multiple instances will be created when calling MySingleton1.getInstance().

To test this scenario, write a servlet that calls MySingleton1.getInstance, and print it. Package the servlet class and MySingleton1 class in test.war, deploy it to appserver. Redeploy the same war in a different context root test2. Invoke the 2 webapp to see if the same instance is printed.

The servlet class:
package test;
import java.io.*;
import javax.servlet.*;
import javax.servlet.http.*;

@javax.servlet.annotation.WebServlet(urlPatterns = "/*")
public class TestServlet extends HttpServlet {
 protected void processRequest(HttpServletRequest request, HttpServletResponse response)
 throws ServletException, IOException {
   PrintWriter out = response.getWriter();
   out.println("singleton: " + MySingleton1.getInstance());
 }
 @Override
 protected void doGet(HttpServletRequest request, HttpServletResponse response)
 throws ServletException, IOException {
   processRequest(request, response);
 }
 @Override
 protected void doPost(HttpServletRequest request, HttpServletResponse response)
 throws ServletException, IOException {
   processRequest(request, response);
 }
}
The content of test.war:
WEB-INF/classes/test/MySingleton1$1.class
WEB-INF/classes/test/MySingleton1.class
WEB-INF/classes/test/TestServlet.class
To deploy the webapp to GlassFish:
$ asadmin deploy test.war
$ asadmin deploy --name test2 test.war
$ asadmin list-applications
To run the 2 webapp and notice 2 different instances of MySingleton class are instantiated within the same JVM, one for each webapp:
$ curl http://localhost:8080/test/
singleton: test.MySingleton1@114536a5

$ curl http://localhost:8080/test/
singleton: test.MySingleton1@114536a5

$ curl http://localhost:8080/test2/
singleton: test.MySingleton1@3997ebf6

$ curl http://localhost:8080/test2/
singleton: test.MySingleton1@3997ebf6

8/08/2011

Join threads with FutureTask and Callable

Here is another way of having the main thread wait for all child threads to complete, using java.util.concurrent.FutureTask and Callable.

package test.concurrent;  
import java.util.Vector;
import java.util.concurrent.*;

public class FutureTaskTest {
private Vector<String> threadNames = new Vector<String>();
public static void main(String[] args) {
FutureTaskTest test = new FutureTaskTest();
test.threadTest(Integer.parseInt(args[0]));
System.out.println(test.threadNames);
}

private void threadTest(int numOfThreads) {
Thread[] threads = new Thread[numOfThreads];
FutureTask<String>[] futureTasks = new FutureTask[numOfThreads];
for (int i = 0; i < threads.length; i++) {
futureTasks[i] = new FutureTask<String>(new MyCallable());
threads[i] = new Thread(futureTasks[i]);
threads[i].start();
}
for (FutureTask<String> f : futureTasks) {
try {
threadNames.add(f.get());
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
}
}

private static class MyCallable implements Callable<String> {
public String call() {
for (int i = 0; i < 1000000; i++) {
i = i + 0;
}
return Thread.currentThread().getName();
}
}
}
FutureTask is usually used along with ExecutorService, but it can also be passed to Thread constructor. The output is always in the order from thread-0 to thread-9, because when we iterate through futureTasks to get result, each get() call wait for a certain duration for the result to be published by child thread.
$ java test.concurrent.FutureTaskTest 10
[Thread-0, Thread-1, Thread-2, Thread-3, Thread-4, Thread-5, Thread-6, Thread-7, Thread-8, Thread-9]
The Vector field threadNames can also be replaced with ArrayList, since it is only accessed in the main thread and no synchronization is needed.

8/07/2011

When to join threads with CountDownLatch

In my previous post When to join threads, Thread.join() is used to wait for all child threads to complete. Now I will update it using CountDownLatch instead:

package test.concurrent;
import java.util.Vector;
import java.util.concurrent.CountDownLatch;

public class ThreadTest {
private Vector<String> threadNames = new Vector<String>();
public static void main(String[] args) {
ThreadTest test = new ThreadTest();
test.threadTest(Integer.parseInt(args[0]));
System.out.println(test.threadNames);
}

private void threadTest(int numOfThreads) {
final CountDownLatch latch = new CountDownLatch(numOfThreads);
Thread[] threads = new Thread[numOfThreads];
for (int i = 0; i < threads.length; i++) {
threads[i] = new ThreadTest.MyThread(latch);
threads[i].start();
}
try {
latch.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
}

private class MyThread extends Thread {
private final CountDownLatch latch;
public MyThread(CountDownLatch latch) {
this.latch = latch;
}

@Override public void run() {
for (int i = 0; i < 1000000; i++) {
i = i + 0;
}
threadNames.add(getName());
latch.countDown();
}
}
}
A CountDownLatch is created in main thread, passed to each child thread's constructor. Each child thread will count down by 1. After starting all child threads, the main thread then wait for the latch count to reach 0. To run it:
$ java test.concurrent.ThreadTest 10
[Thread-1, Thread-2, Thread-0, Thread-3, Thread-4, Thread-5, Thread-6, Thread-7, Thread-8, Thread-9]

My notes on java thread

(1) Thread.dumpStack(), not Thread.dumpStackTrace(), not Thread.currentThread().dumpStack().

(2) thread can sleep with or without acquiring a lock, inside or outside synchronized block. When sleeping without a lock, the CPU resource is re-allocated to one of the ready-to-run threads. When sleeping inside synchronized block with a monitor lock, the lock is not relinquished, and the CPU is re-allocated to one of the other runnable threads that are not waiting on this lock.

(3) wait-notify-flag pattern. wait is almost always inside a while(flag) loop. There can be multiple threads waiting on the same monitor, some is waiting for the flag to change value, and others are not. In other words, there could be homogeneous threads and/or heterogeneous waiters. Once the flag changes value and notifyAll() is invoked, all threads waiting on the same monitor will be awaken. All of them exit waiting state and enter ready-to-run state, but only one of them will get the lock and proceed. If the lucky one consumes (reset) the flag after obtaining the lock, the previous flag set in notifyAll() becomes obsolete. Therefore, other threads, after they get the monitor, will need to recheck the flag to decide to wait again or proceed.

Another reason wait is inside a loop is there can be spurious wakeup, which are not triggered by notify or notifyAll calls. wait can also be interrupted by other threads. So the wait-notify protocol is not 100% reliable, and that's why a flag is needed to aid the inter-thread communication.

When to use notify and when to use notifyAll? When all waiters are homogeneous, i.e., whoever gets the monitor doesn't matter, use notify. When waiters are heterogeneous, use notifyAll to ensure the correct threads are awaken. If waiter are heterogeneous and we call notify, a waiter that does not care about the flag may be picked to proceed, while those waiters interested in the flag will not get the signal.

The flag should be exclusively for this purpose, not used by other part of the application. All access to this flag is inside synchronized block, so no need to make it volatile or AtomicBoolean.

(4) make the lock object final if possible, for example:

private final Object lock = new Object();
Do not synchronize on string literals, which are merely reference to the same string object in the string pool, or global constants. It will unintentionally cause too many threads competing for the same lock.

(5) volatile can be used on fields of primitive type, primitive wrapper types, or any other Object types. A typical use is in singleton DCL pattern. If a mutable field is simply read or written by multiple threads, then making it volatile usually suffice. One step up in concurrency is to use java.util.concurrent.atomic.* classes like AtomicInteger, AtomicBoolean, etc. Atomic types are ideal for simple compound operations like read-update-assignment, or if-absent-set. Anything more complicated than that will need synchronized access.

Further readings:
Sun's Java Concurrency Tutorial

stackoverflow discussion on wait, notify, CountdownLatch, etc

(6), 3 ways to get a new thread:
* extend java.lang.Thread class and overriding its run() method;

* provide an impl of Runnable interface and use it to instantiate a new thread;

* provide an impl of Runnable interface and call ThreadFactory.newThread(runnable). Get the default ThreadFactory with Executors.defaultThreadFactory().

See their differences in another post: Extend Thread vs implement Runnable

Example of expensive computation

Sometimes I need to simulate some expensive computation, or long-running task, without using Thread.sleep or Object.wait. A real busy waiting. The following example of computing PI is adapted from this RMI tutorial:

package test;
import java.math.BigDecimal;

public class Pi implements java.io.Serializable {
private static final long serialVersionUID = 227L;
private static final BigDecimal FOUR = BigDecimal.valueOf(4);
private static final int roundingMode = BigDecimal.ROUND_HALF_EVEN;
public static void main(String[] args) {
BigDecimal result = computePi(Integer.parseInt(args[0]));
System.out.println("Pi is " + result);
}

/**
* Compute the value of pi to the specified number of digits after the
* decimal point. The value is computed using Machin's formula:
* pi/4 = 4*arctan(1/5) - arctan(1/239)
* and a power series expansion of arctan(x) to sufficient precision.
*/
public static BigDecimal computePi(int digits) {
int scale = digits + 5;
BigDecimal arctan1_5 = arctan(5, scale);
BigDecimal arctan1_239 = arctan(239, scale);
BigDecimal pi = arctan1_5.multiply(FOUR).subtract(arctan1_239).multiply(FOUR);
return pi.setScale(digits, BigDecimal.ROUND_HALF_UP);
}

/**
* Compute the value, in radians, of the arctangent of the inverse of the
* supplied integer to the specified number of digits after the decimal
* point. The value is computed using the power series expansion for the arc
* tangent:
* arctan(x) = x - (x^3)/3 + (x^5)/5 - (x^7)/7 + (x^9)/9 ...
*/
public static BigDecimal arctan(int inverseX, int scale) {
BigDecimal result, numer, term;
BigDecimal invX = BigDecimal.valueOf(inverseX);
BigDecimal invX2 = BigDecimal.valueOf(inverseX * inverseX);
numer = BigDecimal.ONE.divide(invX, scale, roundingMode);
result = numer;
int i = 1;
do {
numer = numer.divide(invX2, scale, roundingMode);
int denom = 2 * i + 1;
term = numer.divide(BigDecimal.valueOf(denom), scale, roundingMode);
if ((i % 2) != 0) {
result = result.subtract(term);
} else {
result = result.add(term);
}
i++;
} while (term.compareTo(BigDecimal.ZERO) != 0);
return result;
}
}
To run and time the task:
$ time java test.Pi 50000
Pi is 3.14159265358979323846264338327950288419716939937510 ...
9.263u 0.187s 0:09.22 102.3% 0+0k 0+6io 8pf+0w
It takes about 10 seconds to compute PI with 5000 digits after the decimal point.

My notes on serialization and de-serialization

I wrote the following test to check some aspects of java serialization and de-serialization, followed by some notes:

package test;
import java.io.*;
import java.util.logging.Logger;

public class SerializableTest implements java.io.Serializable {

private static final long serialVersionUID = 0L;
private static final String fileName = "Serializable.ser";
private static final Logger logger = Logger.getLogger("");
private static String var;
transient private String tran = "this is a transient instance field";
private int number;
private Thread th;

public static void main(String[] args) throws IOException, ClassNotFoundException {
if (args[0].startsWith("ser")) {
SerializableTest test = new SerializableTest();
test.number = 1;
var = "this is a static variable";
writeOut(test);
System.out.println("SerializableTest to be saved: " + test);
} else if (args[0].startsWith("deser")) {
System.out.println("SerializableTest deserialized: " + readIn());
}
}

private static Object readIn() throws IOException, ClassNotFoundException {
ObjectInputStream ois = new ObjectInputStream(new FileInputStream(new File(fileName)));
return ois.readObject();
}

private static void writeOut(java.io.Serializable obj) throws IOException {
ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(new File(fileName)));
oos.writeObject(obj);
oos.close();
}

@Override public String toString() {
return "SerializableTest: final static fileName=" + fileName + ", final static logger=" + logger
+ ", non-final static var=" + var + ", instance number=" + number
+ ", transient instance tran=" + tran + ", non-serializable instance field:=" + th;
}
}
To run the program to serialize an instance to current directory:
$ java test.SerializableTest ser

SerializableTest to be saved: SerializableTest:
final static fileName=Serializable.ser,
final static logger=java.util.logging.LogManager$RootLogger@43ef9157,
non-final static var=this is a static variable,
instance number=1,
transient instance tran=this is a transient instance field
The output dumps the in-memory state of the object, before writing it to disk. To show the file type of the data file containing the object:
$ file Serializable.ser
Serializable.ser: Java serialization data, version 5
To de-serialize, or re-constitute the object from the data file:
$ java test.SerializableTest deser

SerializableTest deserialized:
SerializableTest: final static fileName=Serializable.ser,
final static logger=java.util.logging.LogManager$RootLogger@252f0999,
non-final static var=null,
instance number=1,
transient instance tran=null
The output shows the object state reconstructed from the data file.

(1) transient and static fields are ignored in serialization. After de-serialization transient fields and non-final static fields will be null. Final static fields still have values since they are part of the class data.

(2) ObjectOutputStream.writeObject(obj) and ObjectInputStream.readObject() are used in serialization and de-serialization, not BufferedOutputStream/BufferedInputStream. There is no BufferedObjectOutputStream or BufferedObjectInputStream.

(3) During serialization, you need to handle IOException; during de-serialization, you need to handle IOException and ClassNotFoundException. So the transitive closure of the de-serializaed type must be in the classpath on the receiving side.

(4) Some non-serializable classes: Thread, Logger, LogManager and most classes in java.util.logging (Level and LogRecord are serializable, though). Throwable extends Serializable, so all Error and Exception types are serializable via inheritance.

(5) Uninitialized non-serializable, non-transient instance fields are tolerated. When adding "private Thread th;", no error in serializable. However, "private Thread th = new Thread();" will cause exception:
Exception in thread "main" java.io.NotSerializableException: java.lang.Thread
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1164)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1518)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1483)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1400)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1158)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:330)
at test.SerializableTest.writeOut(SerializableTest.java:33)
at test.SerializableTest.main(SerializableTest.java:19)
(6) It is possible to include static fields in serialization and de-serialization, by implementing writeObject and readObject methods. These 2 methods are not in java.io.Serialiable, which is just a marker interface, and not in java.lang.Object, either. They are just 2 magic methods with pre-defined signatures. The following is from Technical Article: Advanced Serialization
private void writeObject(ObjectOutputStream oos) 
throws IOException {
oos.defaultWriteObject();
// Write/save additional fields
oos.writeObject(new java.util.Date());
}

// assumes "static java.util.Date aDate;" declared
private void readObject(ObjectInputStream ois)
throws ClassNotFoundException, IOException {
ois.defaultReadObject();
// Read/initialize additional fields
aDate = (java.util.Date)ois.readObject();
}
For a definitive guide, refer to Official Java Object Serialization Specification version 6.0

(7) Serialization and de-serialization can be used for copying and cloning objects. It is slower than regular clone, but can produce a deep copy very easily. The following util method is adapted from Secrets of Java Serialization:
    public static Object clone(Object o) 
throws IOException, ClassNotFoundException {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(bos);
oos.writeObject(o);
oos.close();
ByteArrayInputStream bis = new ByteArrayInputStream(bos.toByteArray());
ObjectInputStream ois = new ObjectInputStream(bis);
Object copy = ois.readObject();
ois.close();
return copy;
}

(8) If I need to serialize a Serializable class Person, but one of its superclasses is not Serializable, can Person class still be serializaed and de-serialized? The answer is yes, provided that the non-serializable superclass has a no-arg constructor, which is invoked at de-serialization to initialize that superclass.

(9) How to use a custom class loader during de-serialization? Oftentimes the context class loader of the receiving end is not the right one. Then you will need to create a subclass of java.io.ObjectInputStream, overriding resolveClass method. This is one of the few occasions where you need your own i/o classes.

protected Class<?> resolveClass(ObjectStreamClass desc)
throws IOException, ClassNotFoundException

(10) Be careful when modifying a class implementing java.io.Serializable. If it does not contain a serialVersionUID field, its serialVersionUID will be automatically generated by the compiler. Different compilers, or different versions of the same compiler, will generate potentially different values. Usually the computation of serialVersionUID is based on not only fields, but also on other aspect of the class like implement clause, constructors, etc. So the best practice is to explicitly declare a serialVersionUID field to maintain backward compatibility. If you need to modify the serializable class substantially and expect it to be incompatible with previous versions, then you need to increment serialVersionUID to avoid mixing different versions.
private static final long serialVersionUID = 42L;
(11) The following is a symptom of a mismatched serialVersionUID:
java.io.IOException: Mismatched serialization UIDs :
The fix is to find out the previous serialVersionUID with $JAVA_HOME/bin/serialver tool, and use it as the value of serialVersionUID field in the new version of the class.

(12) 2 usages of serialization and de-serialization:
  • Temporal serialization: an object is serialized and some time later (could be years later) is restored and put to work.

  • Spatial serialization: an object is serialized and subsequently transferred to another memory space, where it is de-serialized and put to work. This is the more realistic use case than case 1. For example, product foo 1.0 works and exchanges data with product foo 2.0. Or server product A interoperates with server product B through some remoting protocol such as RMI/IIOP. A and B could share some common classes (e.g., javax.ejb.EJBException), and if some common classes do not explicitly declare serialVersionUID, product A and B may compile their own copy and end up having different serialVersionUID.

8/04/2011

Example of Java thread deadlock

A simple example showing a typical Java thread deadlock. 2 threads are spawned: thread-0 first locks lock1 and then lock2; thread-1 first locks lock2 and then lock1. Each thread sleeps a bit before acquiring the second lock, to increase the chance of deadlock.

package test.concurrent;
public class DeadLockTest {
private static long sleepMillis;
private final Object lock1 = new Object();
private final Object lock2 = new Object();

public static void main(String[] args) {
sleepMillis = Long.parseLong(args[0]);
DeadLockTest test = new DeadLockTest();
test.doTest();
}

private void doTest() {
Thread t1 = new Thread(new Runnable() {
public void run() {
lock12();
}
});
Thread t2 = new Thread(new Runnable() {
public void run() {
lock21();
}
});
t1.start();
t2.start();
}

private void lock12() {
synchronized (lock1) {
sleep();
synchronized (lock2) {
sleep();
}
}
}

private void lock21() {
synchronized (lock2) {
sleep();
synchronized (lock1) {
sleep();
}
}
}

private void sleep() {
try {
Thread.sleep(sleepMillis);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
To run the deadlock test with sleep time 1 millisecond:
java -cp . test.concurrent.DeadLockTest 1
The process seems to hang for ever. To get the thread dump, press Ctrl-\ (Ctrl-break on Windows). Thread dump contains the following deadlock diagnositc:
Found one Java-level deadlock:
=============================
"Thread-1":
waiting to lock monitor 1017fcfb0 (object 7f31050b0, a java.lang.Object),
which is held by "Thread-0"
"Thread-0":
waiting to lock monitor 1017fe360 (object 7f31050c0, a java.lang.Object),
which is held by "Thread-1"

Java stack information for the threads listed above:
===================================================
"Thread-1":
at test.concurrent.DeadLockTest.lock21(DeadLockTest.java:44)
- waiting to lock <7f31050b0> (a java.lang.Object)
- locked <7f31050c0> (a java.lang.Object)
at test.concurrent.DeadLockTest.access$1(DeadLockTest.java:40)
at test.concurrent.DeadLockTest$2.run(DeadLockTest.java:23)
at java.lang.Thread.run(Thread.java:680)
"Thread-0":
at test.concurrent.DeadLockTest.lock12(DeadLockTest.java:35)
- waiting to lock <7f31050c0> (a java.lang.Object)
- locked <7f31050b0> (a java.lang.Object)
at test.concurrent.DeadLockTest.access$0(DeadLockTest.java:31)
at test.concurrent.DeadLockTest$1.run(DeadLockTest.java:18)
at java.lang.Thread.run(Thread.java:680)

Found 1 deadlock.

8/02/2011

Simple Example of JSF and EJB 3.1

This is a very simple example of a JSF 2.0 managed bean invoking EJB 3.1 local singleton EJB. Everything is packaged into a test.war file, and deployed to GlassFish.

test.war consists of 4 entries:

WEB-INF/classes/test/HelloEJB.class
WEB-INF/classes/test/HelloJSFBean.class
WEB-INF/web.xml
index.xhtml
index.xhtml page:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"
xmlns:ui="http://java.sun.com/jsf/facelets"
xmlns:f="http://java.sun.com/jsf/core"
xmlns:h="http://java.sun.com/jsf/html">
<ui:composition>
<h:head>
<h1><h:outputText value="jsf-ejb example" /></h1>
</h:head>

<h:body>
<h:form id="form">
<h:outputLabel value="helloEJB returned message: "/>
<h:outputText value="#{HelloJSFBean.message}"/>
</h:form>
</h:body>
</ui:composition>
</html>
HelloJSFBean managed bean:
package test;
import javax.faces.bean.*;
import javax.ejb.*;

@ManagedBean(name="HelloJSFBean")
@RequestScoped
public class HelloJSFBean implements java.io.Serializable {
@EJB
private HelloEJB helloEJB;

public String getMessage() {
return helloEJB.hello();
}
}
HelloEJB singleton bean:
package test;
import javax.ejb.*;

@Singleton
public class HelloEJB {
public String hello() {
return "Hello from " + this;
}
}
web.xml:
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
version="3.0">
<servlet>
<servlet-name>Faces Servlet</servlet-name>
<servlet-class>javax.faces.webapp.FacesServlet</servlet-class>
<load-on-startup>1</load-on-startup>
</servlet>

<servlet-mapping>
<servlet-name>Faces Servlet</servlet-name>
<url-pattern>/faces/*</url-pattern>
</servlet-mapping>
</web-app>
To run the webapp:
curl http://localhost:8080/test/faces/index.xhtml

helloEJB returned message: Hello from test.HelloEJB@1326d2fe

8/01/2011

Runnable, Callable, FutureTask, ExecutorService and thread pool

Some notes on Java concurrency, starting with Runnable vs Callable:

(1), Runnable.run() does not return a value; Callable<T>.run() returns a value of type T. Callable<T> is a parameterized type whose type parameter indicates the return type of its run method. Runnable cannot be parameterized.

(2), Runnable.run() cannot throw any checked exception; Callable<T>.run() can throw checked and unchecked exceptions.

(3), so it seems Callable is more powerful than Runnable. But you cannot just replace all use of Runnable with Callable. For example, in new Thread(runnable). You cannot instantiate a new thread with Callable.

(4), Callable is introduced in Java SE 5, as part of java.util.concurrent library. Callable is mainly intended to use with ExecutorService, which is a higher level service than the direct manipulation of Thread. Both Runnable and Callable can be submitted to ExecutorService for (usually asynchronous) execution.

(5), java.util.concurrent.Executors (a util class similar to Collections, Arrays) has methods to upgrade a Runnable to Callable, but not the other way.

Callable<Object> callable(Runnable task)
<T> Callable<T> callable(Runnable task, T result)
The Callable converted from callable(Runnable) returns null. The Callable converted from callable(Runnable, T) returns T. However, this return value that was passed in by the client is not the real result of executing task. Its purpose is to tell the client that the return type will be Future<T>. In order to obtain the real result, the client still needs to arrange for a thread-safe shared data storage, e.g., a final or volatile field of Runnable impl. Therefore, the converted Callable is still a Runnalbe in its core.

(6) FutureTask is another artifact involving both Callable and Runnable. FutureTask class implements RunnableFuture interface, which in turn extends both Runnable and Future interfaces. So FutureTask is a Runnable by inheritance, and can be passed to new Thread(futureTask), or ExecutorService.submit(futureTask).

FutureTask has a Callable<V> by composition as FutureTask's constructor takes Callable<V> as a param. FutureTask also has another constructor that takes 2 params (Runnable, V), which are simply converted to Callable via Executors.callable(Runnable, V). In fact, the same pattern is employed in java.util.concurrent wherever a method is overloaded with (Callable) and (Runnable, V).

(7), Runnable and Callable run() method takes no param. Why? For one thing, there is no way to know which types are to be passed in. The best we can do is probably Object[] if we were to add params to it. More importantly, it would export state from the calling thread to the task thread, and entails synchronization between the 2 sides. I guess this is why the plain old Runnable.run() takes no param and returns void. In Java 5 to balance simplicity and usefulness, Callable was added to return a result.

(8), how can a task be performed without any params passed to run()? The typical answer is to pass any required data in when instantiating the Callable or Runnable impl class via its constructor. For example,
public class RunnableImpl implements Runnable {
private final List<String> names;

public RunnableImpl(List<String> names) {
this.names = names;
}

public void run() {...}
}
If RunnableImpl is a static inner class, its run() method can directly reference any static fields in the enclosing class. If RunnableImpl is an instance inner class, its run() method can directly reference any static and instance fields in the enclosing class. If RunnableImpl is an anonymous inner class, its run() method can directly reference any static and instance fields of the enclosing class, and any final local variables in the enclosing method.

(9), you can call Thread.setUncaughtExceptionHandler, or Thread.setDefaultUncaughtExceptionHandler to handle exceptions from Runnable.run(). When using ExecutionService where you have no direct control of threads, you can use a custom ThreadFactory. Executors methods newCachedThreadPool, newFixedThreadPool, newScheduledThreadPool, newSingleThreadExecutor, newSingleThreadScheduledExecutor all take a custom ThreadFactory.

(10), Worker thread in thread pools and regular thread are different. A regular thread dies after its run method terminates and cannot be restarted. A worker thread does not die after the completion of the assigned task; it goes back to the thread pool. The main run method of a work thread is an infinite loop: taking a task from the work queue, and invoke the task's run method to do the task.

(11), ThreadLocal variables should be cleared up by the client once the task is done. Otherwise the left-over reference in worker threads prevent the referenced objects from being garbage-collected. Over time it may cause OutOfMemoryError.

(12), The thread pool default work queue in ExecutionService framework is unbounded. It means once the corePoolSize is reached, all subsequent requests will go into this unbound BlockingQueue, until some busy worker threads complete its task. No new worker threads will be created beyond corePoolSize, and maximumPoolSize is ignored in this case. For more details, see bug 6756747: java.util.concurrent.ThreadPoolExecutor doesn't create new worker when it should.

7/29/2011

Example of JPA and Hibernate with JBoss and MySql

This post demonstrates how to write a simple web app using Java Persistence API (JPA), EJB 3 session bean and servlet, connecting to MySql database, and deployed to JBoss appserver 6.

1, JPA entity class, Greeting, is the same as in my previous example, also included here in its entirety:

package test;
import java.io.Serializable;
import javax.persistence.*;

@Entity
public class Greeting implements Serializable {
@Id @GeneratedValue private int id;
@Basic private String message;
@Basic private String language;

public Greeting() {}

public Greeting(String message, String language) {
this.message = message;
this.language = language;
}

public String toString() {
return "Greeting id=" + id + ", message=" + message +
", language=" + language;
}
}
2, a stateless session bean facade for CRUD operations with container-managed transaction and container-managed persistence context:
package test;
import javax.ejb.*;
import javax.persistence.*;

@Stateless
public class GreetingBean {
@PersistenceContext
private EntityManager em;

public void create(Greeting... gs) {
for(Greeting g : gs) {
em.persist(g);
}
}

public Greeting findByLang(String lang) {
return (Greeting) em.createQuery(
"select g from Greeting g where g.language = :language")
.setParameter("language", lang).getSingleResult();
}
}
3, TestServlet, which calls the stateless session bean:
package test;
import java.io.*;
import javax.servlet.*;
import javax.servlet.http.*;
import javax.ejb.EJB;

@javax.servlet.annotation.WebServlet(urlPatterns = "/*")
public class TestServlet extends HttpServlet {
@EJB
private GreetingBean bean;

protected void processRequest(HttpServletRequest request,
HttpServletResponse response)
throws ServletException, IOException {
PrintWriter out = response.getWriter();
Greeting g_en = new Greeting("hello world", "en");
Greeting g_es = new Greeting("hola, mundo", "es");
bean.create(g_en, g_es);
out.println("Created and persisted " + g_en + ", and " + g_es);

Greeting g = bean.findByLang("en");
out.println("Query returned: " + g);
}

@Override
protected void doGet(HttpServletRequest request,
HttpServletResponse response)
throws ServletException, IOException {
processRequest(request, response);
}

@Override
protected void doPost(HttpServletRequest request,
HttpServletResponse response)
throws ServletException, IOException {
processRequest(request, response);
}
}
4, persistence.xml with JTA transaction using a MySQL datasource:
<?xml version="1.0" encoding="UTF-8"?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd"
version="2.0">
<persistence-unit name="greeting-pu" transaction-type="JTA">
<!--
<jta-data-source>java:/DefaultDS</jta-data-source>
-->
<jta-data-source>java:/MySqlDS</jta-data-source>
<properties>
<!--
<property name="hibernate.dialect" value="org.hibernate.dialect.HSQLDialect"/>
-->
<property name="hibernate.dialect" value="org.hibernate.dialect.MySQLDialect"/>
<property name="hibernate.hbm2ddl.auto" value="create-drop"/>
</properties>
</persistence-unit>
</persistence>
5, create the MySql datasource in JBoss appserver:
cp $JBOSS_HOME/docs/examples/jca/mysql-ds.xml $JBOSS_HOME/server/default/deploy/
# change user-name and password elements in mysql-ds.xml
cp $HOME/downloads/mysql-connector-java-5.1.15-bin.jar $JBOSS_HOME/server/default/lib/
6, compile and package class files and persistence.xml into test.war:
javac -d ../ -cp "$JBOSS_HOME/client/*" *.java

content of test.war:
====================
WEB-INF/classes/META-INF/persistence.xml
WEB-INF/classes/test/Greeting.class
WEB-INF/classes/test/GreetingBean.class
WEB-INF/classes/test/TestServlet.class
7, start MySql database server, JBoss default server and deploy test.war:
cd /usr/local/mysql-5.1.32
sudo bin/mysqld_safe --user root

$JBOSS_HOME/bin/run.sh
8, deploy and run the test:
cp test.war $JBOSS_HOME/server/default/deploy
curl http://localhost:8080/test/
If you use the default HyperSonic embedded database in JBoss AS, instead of MySql database server, make the following adjustments:

(a), toggle the comments in persistence.xml;
(b), no need to copy mysql-ds.xml, or MySql driver jar;
(c), no need to start MySql database server.

7/05/2011

How to programmatically copy jar files

To copy a jar file programmatically in java involves the following steps:

1, create a JarOutputStream based on the destination jar file;

2, loop through all entries in the source jar file, and get the InputStream from each entry;

3, create a new jar entry with the same name as the source jar entry, and put the new entry to the JarOutputStream;

4, now we have InputStream, OutputStream, and the new jar entry in place, we are ready to transfer the bits, by reading bytes from InputStream into a buffer and writing buffer content to OutputStream.

5, close the InputStream, flush the JarOutputStream, and close the jar entry. This completes the copying of one jar entry.

6, after the loop ends, all entries are copied from source jar file to destination jar file. Finally, close the JarOutputStream.

package test;
import java.io.*;
import java.util.Enumeration;
import java.util.jar.*;

//java -cp . test.CopyZip xxx/lib/sac-1.3.jar /tmp
//
public class CopyZip {
public static void main(String[] args) throws Exception {
File sourceFileOrDir = new File(args[0]);
File destDir = new File(args[1]);
if (sourceFileOrDir.isFile()) {
copyJarFile(new JarFile(sourceFileOrDir), destDir);
} else if (sourceFileOrDir.isDirectory()) {
File[] files = sourceFileOrDir.listFiles(new FilenameFilter() {
public boolean accept(File dir, String name) {
return name.endsWith(".jar");
}
});
for (File f : files) {
copyJarFile(new JarFile(f), destDir);
}
}
}

public static void copyJarFile(JarFile jarFile, File destDir) throws IOException {
String fileName = jarFile.getName();
String fileNameLastPart = fileName.substring(fileName.lastIndexOf(File.separator));
File destFile = new File(destDir, fileNameLastPart);

JarOutputStream jos = new JarOutputStream(new FileOutputStream(destFile));
Enumeration<JarEntry> entries = jarFile.entries();

while (entries.hasMoreElements()) {
JarEntry entry = entries.nextElement();
InputStream is = jarFile.getInputStream(entry);

//jos.putNextEntry(entry);
//create a new entry to avoid ZipException: invalid entry compressed size
jos.putNextEntry(new JarEntry(entry.getName()));
byte[] buffer = new byte[4096];
int bytesRead = 0;
while ((bytesRead = is.read(buffer)) != -1) {
jos.write(buffer, 0, bytesRead);
}
is.close();
jos.flush();
jos.closeEntry();
}
jos.close();
}
}
A common error is:
Exception in thread "main" java.util.zip.ZipException: invalid entry compressed size (expected 1665 but got 1680 bytes)
at java.util.zip.ZipOutputStream.closeEntry(ZipOutputStream.java:206)
at test.CopyZip.copyJarFile(CopyZip.java:63)
at test.CopyZip.main(CopyZip.java:37)
It usually occurs when text file entries in the source jar file contain some non-ASCII characters such as ^I ^Z ^D ^C. Some common files are META-INF/COPYRIGHT.html, META-INF/LICENSE.txt, etc, probably because these files were created in a non-ASCII editor but saved as text files. Open them in vi or vim to see these offending characters.

To avoid this type of ZipException, always create a new JarEntry with the same name, and pass it to putNextEntry() method. Do not pass the existing jar entry from the source jar file to putNextEntry() method.

7/03/2011

Servlet init method vs PostConstruct method

There are 2 initialization methods in servlet:

(1), init(ServletConfig config) methods defined in javax.servlet.Servlet interface. Servlet init method has been there since the first version of Servlet. This method was later (around Servlet 2.3) overloaded in javax.servlet.GeneircServlet for convenience. According to the above javadoc, the servlet container calls the init method exactly once after instantiating the servlet.

(2), PostConstruct method, as defined in javax.annotation.PostConstruct, was introduced in Java EE 5 to all component types, including all web components (Servlet 2.5). Similarly, @PostConstruct (or post-construct, if declared in deployment descriptors) methods are to be invoked only once after the component class is instantiated and before it is put into service.

But these two servlet methods are not exactly the same.

The name of Servlet init method is set in the interface, and it must be public. PostConstruct method is more flexible: it can be private, package default, protected, or public, and can be named anything.

I could not find docs specifying the invocation order and relationship of the two methods. But it turns out the order is:

servlet class constructor --> PostConstruct --> init and init(ServletConfig)
So inside PostConstruct method at the second step, ServletConfig has not been initialized, and neither is ServletContext. Calling getServletConfig() in PostConstruct returns null. Calling getServletContext() results in IllegalStateException in most servers (GlassFish 3, Tomcat 7 and JBoss 6), but returns null in Resin 4.

Apparently there are overlappings and subtle differences between PostConstruct and servlet init methods. I guess ideally we could merge them into one. If there is no PostConstruct method in servlet class, init() will be considered as its PostConstruct method; if developers want to specify a PostConstruct method, it has to be the same as init() method, basically adding the redundant @PostConstruct to init(). This is roughly how ejbCreate and PostConstruct is handled in EJB 3.

What about web apps that already have both PostConstruct and init methods? In order to accommodate them, the next desirable approach is to allow both, but clearly specify their relationship, making clear getServletConfig() and getServletContext() both return valid values inside the both methods.

The same applies to PreDestroy and servlet destroy method, to a lesser extent.

6/20/2011

Jetty to GlassFish Interoperability: Remote EJB Invocation

Jetty-to-GlassFish EJB invocation is similar to Tomcat-to-GlassFish case. I tested with Jetty Hightide 7.4.2 and GlassFish 3.1.

Jetty configuration (in csh/tcsh syntax):

setenv JETTY_HOME $HOME/tools/jetty
setenv JETTY_PORT 7070 # avoid port conflict
mkdir $JETTY_HOME/lib/ext/glassfish
The following commands copy 32 GlassFish jars needed for this test to Jetty directory:
cd $GLASSFISH_HOME/modules
cp javax.ejb.jar ejb-container.jar deployment-common.jar dol.jar glassfish-corba-csiv2-idl.jar glassfish-corba-codegen.jar ssl-impl.jar security.jar ejb.security.jar management-api.jar gmbal-api-only.jar gmbal.jar glassfish-corba-asm.jar glassfish-corba-newtimer.jar glassfish-corba-orbgeneric.jar config-types.jar kernel.jar config.jar config-api.jar glassfish-corba-omgapi.jar glassfish-corba-orb.jar orb-connector.jar orb-enabler.jar orb-iiop.jar glassfish-api.jar auto-depends.jar hk2-core.jar internal-api.jar common-util.jar glassfish-corba-internal-api.jar glassfish-naming.jar bean-validator.jar $JETTY_HOME/lib/ext/glassfish
I had to slightly modify TestServlet to make it work in Jetty:

1, declare TestServlet with web.xml instead of using @javax.servlet.annotation.WebServlet, probably because servlet 3.0 support is not there yet in Jetty 7.4.2;

2, change @PostConstruct method to servlet init method. When using @PostConstruct to look up TestBean, it is not initialized and produced NullPointerException at request time. So I changed it to servlet init() method, which is a lifecycle method since early days of servlet. The complete TestServlet.java and web.xml:
package test;
import java.io.*;
import java.util.*;
import javax.servlet.*;
import javax.servlet.http.*;
import javax.naming.*;
import javax.annotation.*;

public class TestServlet extends HttpServlet {
private static final String glassfishJndiPropertiesPath = "/glassfish-jndi.properties";
private static final String testBeanJndi = "java:global/test-ejb/TestBean";
private TestIF testBean;

private Properties getGlassFishJndiProperties() {
Properties props = new Properties();
try {
props.load(getClass().getResourceAsStream(glassfishJndiPropertiesPath));
} catch (IOException e) {
System.out.println("Failed to load " + glassfishJndiPropertiesPath);
}
System.out.println("Got glassfish-jndi.properties: " + props);
return props;
}

public void init() {
try {
InitialContext ic = new InitialContext(getGlassFishJndiProperties());
testBean = (TestIF) ic.lookup(testBeanJndi);
System.out.println("Looked up " + testBeanJndi + ", got " + testBean);
} catch (NamingException e) {
throw new RuntimeException(e);
}
}

protected void processRequest(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
PrintWriter out = response.getWriter();
out.println("testBean.hello(): " + testBean.hello());
}

@Override
protected void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
processRequest(request, response);
}

@Override
protected void doPost(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
processRequest(request, response);
}
}

<?xml version="1.0" encoding="UTF-8"?>
<web-app version="2.5" xmlns="http://java.sun.com/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd">

<servlet>
<servlet-name>TestServlet</servlet-name>
<servlet-class>test.TestServlet</servlet-class>
</servlet>

<servlet-mapping>
<servlet-name>TestServlet</servlet-name>
<url-pattern>/*</url-pattern>
</servlet-mapping>
</web-app>
After packaging, test2.war (a sample app test.war is already included in jetty/webapps, so use a different name) contains the following entries:
WEB-INF/classes/glassfish-jndi.properties
WEB-INF/classes/test/TestIF.class
WEB-INF/classes/test/TestServlet.class
WEB-INF/web.xml
Other files, TestIF, TestBean, test-ejb.jar and glassfish-jndi.properties, are all the same as in Tomcat and Resin exercise.

6/16/2011

Resin to GlassFish Interoperability: Remote EJB Invocation

Resin-to-GlassFish EJB invocation is almost identical to Tomcat-to-GlassFish case. I tested with Resin-4.0.18 with GlassFish 3.1. One minor difference is that Resin has its own bean validation implementation, so GlassFish's bean-validator.jar need not be copied to Resin. The following is the list of 31 GlassFish jars needed for Resin client, in one single line for copying:

cd $GLASSFISH_HOME/modules
cp javax.ejb.jar ejb-container.jar deployment-common.jar dol.jar glassfish-corba-csiv2-idl.jar glassfish-corba-codegen.jar ssl-impl.jar security.jar ejb.security.jar management-api.jar gmbal-api-only.jar gmbal.jar glassfish-corba-asm.jar glassfish-corba-newtimer.jar glassfish-corba-orbgeneric.jar config-types.jar kernel.jar config.jar config-api.jar glassfish-corba-omgapi.jar glassfish-corba-orb.jar orb-connector.jar orb-enabler.jar orb-iiop.jar glassfish-api.jar auto-depends.jar hk2-core.jar internal-api.jar common-util.jar glassfish-corba-internal-api.jar glassfish-naming.jar $RESIN_HOME/lib
To change Resin http port number from the default 8080 to other number (e.g., 7070), edit resin/conf/resin.xml:
<http address="*" port="7070"/>
All the rest, including test app code, deployment & execution steps are exactly the same as in Tomcat exercise. This is the test command & output in resin:
curl http://localhost:7070/test/
testBean.hello(): From test.TestBean@47dccc2

6/11/2011

Tomcat to GlassFish Interoperability: Remote EJB Invocation

In order to make Tomcat(7.0.8)-to-GlassFish(v3) remote EJB invocation, one has to solve 2 challenges:

1, add relevent GlassFish client-side jar files to tomcat lib. There is an all-inclusive client jar (glassfish/lib/gf-client.jar), which references practically all GlassFish jars. But adding gf-client.jar to Tomcat lib would cause various conflict of system classes. So one still needs to figure out which GlassFish jars are needed, mostly by trial and error.

For a simple Tomcat-GlassFish test that includes a servlet (in Tomcat) invoking remote EJB 3 stateless bean (in GlassFish 3), the following 32 jars are the minimum set of jars to be included in Tomcat lib:

javax.ejb.jar
ejb-container.jar
deployment-common.jar
dol.jar
glassfish-corba-csiv2-idl.jar
glassfish-corba-codegen.jar
ssl-impl.jar
security.jar
ejb.security.jar
management-api.jar
gmbal-api-only.jar
gmbal.jar
glassfish-corba-asm.jar
glassfish-corba-newtimer.jar
glassfish-corba-orbgeneric.jar
bean-validator.jar
config-types.jar
kernel.jar
config.jar
config-api.jar
glassfish-corba-omgapi.jar
glassfish-corba-orb.jar
orb-connector.jar
orb-enabler.jar
orb-iiop.jar
glassfish-api.jar
auto-depends.jar
hk2-core.jar
internal-api.jar
common-util.jar
glassfish-corba-internal-api.jar
glassfish-naming.jar

2, add GlassFish jndi properties to application lookup code. The complete servlet class:

package test;

import java.io.*;
import java.util.*;
import javax.servlet.*;
import javax.servlet.http.*;
import javax.naming.*;
import javax.annotation.*;

@javax.servlet.annotation.WebServlet(urlPatterns = "/*")
public class TestServlet extends HttpServlet {
private static final String glassfishJndiPropertiesPath = "/glassfish-jndi.properties";
private static final String testBeanJndi = "test.TestIF";
private TestIF testBean;

private Properties getGlassFishJndiProperties() {
Properties props = new Properties();
try {
props.load(getClass().getResourceAsStream(glassfishJndiPropertiesPath));
} catch (IOException e) {
System.out.println("Failed to load " + glassfishJndiPropertiesPath);
}
System.out.println("Got glassfish-jndi.properties: " + props);
return props;
}

@PostConstruct
private void initTestBean() {
try {
InitialContext ic = new InitialContext(getGlassFishJndiProperties());
testBean = (TestIF) ic.lookup(testBeanJndi);
} catch (NamingException e) {
throw new RuntimeException(e);
}
}

protected void processRequest(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
PrintWriter out = response.getWriter();
out.println("testBean.hello(): " + testBean.hello());
}

@Override
protected void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
processRequest(request, response);
}

@Override
protected void doPost(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
processRequest(request, response);
}
}
The content of the test.war deployed in Tomcat:
WEB-INF/classes/glassfish-jndi.properties
WEB-INF/classes/test/TestIF.class
WEB-INF/classes/test/TestServlet.class
TestIF is the remote business interface of TestBean. WEB-INF/classes/glassfish-jndi.properties is packaged into the war file to provide GlassFish jndi bootstrap properties. Its content is basedon the one in GlassFish server:
java.naming.factory.initial=com.sun.enterprise.naming.impl.SerialInitContextFactory
java.naming.factory.url.pkgs=com.sun.enterprise.naming
# Required to add a javax.naming.spi.StateFactory for CosNaming that
# supports dynamic RMI-IIOP.
java.naming.factory.state=com.sun.corba.ee.impl.presentation.rmi.JNDIStateFactoryImpl

# uncomment the following lines if GlassFish is running on
# different host and non-default port number.
# org.omg.CORBA.ORBInitialHost=localhost
# org.omg.CORBA.ORBInitialPort=3700
test-ejb.jar deployed in GlassFish 3 just contains the bean class and its remote business interface:
test/TestBean.class
test/TestIF.class
TestIF.java:
package test;
import javax.ejb.*;

@Remote
public interface TestIF {
public String hello();
}
TestBean.java:
package test;
import javax.ejb.*;

@Stateless
@Remote(TestIF.class)
public class TestBean implements TestIF {
public String hello() {
return "From " + this;
}
}
After both test-ejb.jar and test.war are packaged, first deploy test-ejb.jar to GlassFish, and then deploy test.war to Tomcat:
cp test-ejb.jar $GLASSFISH_HOME/domains/domain1/autodeploy

# to verify it has been successfully deployed:
$GLASSFISH_HOME/bin/asadmin list-applications

cp test.war tomcat/webapps
Visit the test URL http://localhost:7070/test/ to see the result of remote EJB invocation. The default http port number in Tomcat and GlassFish are the same (8080). So if they are on the same host, need to change one of them. I changed Tomcat to use 7070 by editing tomcat/conf/server.xml.

When loading glassfish-jndi.properties, the current class loader, instead of ServletContext, is used to get the resource. The reason is in @PostConstruct method, ServletConfig has not been initialized, and so GenericServlet.getServletConfig().getServletContext() failed with NullPointerException.

The jndi name for looking up, test.TestIF, is the default GlassFish-specific global jndi name. The portable global jndi name, java:global/test-ejb/TestBean, works as well.

6/10/2011

javax.naming.Reference cannot be cast to in GlassFish

This error usually happens when the client tries to cast the lookup result to the expected type, such as EJB remote interface. This is what happens behind the scene. During the naming context lookup, a javax.naming.Reference is first acquired, which contains information how to create the actual object for the client. If the object creation step fails, then the half-baked javax.naming.Reference is returned to the client, hence the ClassCastException.

Some reasons why the naming manager failed to create object:

1, the object factory class responsible for creating the object is not available in the client classpath. Think of javax.naming.Reference as the raw material for creating the object. When there is no factory, no object is created, and the raw material is thrown back to the client.

In GlassFish 3, the object factory for remote EJB reference appears to be com.sun.ejb.containers.RemoteBusinessObjectFactory, packaged in ejb-container.jar. So ejb-container.jar and javax.ejb.jar should be in the client classpath by direct inclusion or by reference, along with a series of other jar files.

The following is the toString format of the returned javax.naming.Reference:

Reference Class Name: test.TestIF
Type: url
Content: ejb/TestBean__3_x_Internal_RemoteBusinessHome__
2, the object factory is in client classpath, but not loadable by the current thread context class loader.

The algorithm for creating object is in javax.naming.spi.NamingManager.

5/05/2011

VirtualBox shared folder for Ubuntu guest

After upgrading my Ubuntu guest OS, the configured Guest Addition and shared folders are gone. So I have to go through the exercise again. Keep this note for future reference:

1, Make sure dkms is up to date:

sudo apt-get install dkms

Reading package lists... Done
Building dependency tree
Reading state information... Done
dkms is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

2, install Guest Addition on Ubuntu:

Click VirtualBox menu Devices | Install Guest Additions..., nothing happened. Then run it manually instead:

sudo sh /media/VBOXADDITIONS_4.0.6_71344/VBoxLinuxAdditions.run

3, Add shared folders by clicking VirtualBox menu Devices | Shared Folders

4, Mount the shared folder to the guest Ubuntu and make it accessible to yourself.

The default mounting point is /media/sf_<the-name-entered-in-step-3>, and is accessible to group vboxsf only. To open it up for yourself, go to Ubuntu menu System | Administration | Users and Groups, then go to Manage Groups | vboxsf | Properties | Group Members, and add yourself to this group.

While there, also change the account type to Administrator. It's not a required step, but I like to have maximum permission.

Alternatively, you can also do it in command line, which should work in all Linux distributions:
su
/usr/sbin/usermod -a -G vboxsf my-user-name

4/20/2011

My mvn notes

To view dependency tree:

mvn dependency:tree
If you see the following errors when running dependency:tree, try upgrading the dependency plugin to version 2.8 or later.
[WARNING] Error injecting: org.apache.maven.shared.dependency.graph.internal.Maven3DependencyGraphBuilder
java.lang.NoClassDefFoundError: org/sonatype/aether/version/VersionConstraint

Caused by: java.lang.ClassNotFoundException: org.sonatype.aether.version.VersionConstraint
 at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:50)
 at org.codehaus.plexus.classworlds.realm.ClassRealm.unsynchronizedLoadClass(ClassRealm.java:259)
 at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:235)
 at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:227)
 ... 62 more

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:2.6:tree (default-cli) on project: 
Execution default-cli of goal org.apache.maven.plugins:maven-dependency-plugin:2.6:tree failed: 
A required class was missing while executing org.apache.maven.plugins:maven-dependency-plugin:2.6:tree: 
org/sonatype/aether/version/VersionConstraint

To skip running tests:
mvn install -DskipTests
mvn install -DskipTests=true  //to override skipTests property in pom
To skip compiling and running tests:
mvn install -Dmaven.test.skip=true
To run maven in offline mode, to avoid downloading:
mvn -o install
mvn --offline install
To force check for updates of snapshots and releases from remote repositories:
mvn -U install
mvn --update-snapshots install
To list all maven profiles, and see which is the active profile:
mvn help:all-profiles
To override maven surefire plugin forkMode:
mvn test -DforkMode=never
mvn test -DforkMode=always
To debug a non-forked test, just need to attach the remote debugger to mvn process itself. mvnDebug is a convenience script located in the same directory as mvn executable:
mvnDebug test
mvnDebug test -DforkMode=never
To debug a forked test, you will need to set the debug option to the forked VM, not the mvn VM. The way to do it is to set -Dmaven.surefire.debug flag to mvn VM (the parent VM):
mvn test -Dmaven.surefire.debug
mvn test -Dmaven.surefire.debug -DforkMode=always
mvn test -Dmaven.surefire.debug -Dtest=com.my.test.MyTest#test1
To avoid OutOfMemoryError from running maven, especially maven 3, set environment variable MAVEN_OPTS:
# in $HOME/.tcshrc, or $HOME/.cshrc, if using csh or tcsh:
setenv MAVEN_OPTS "-Xmx1024m -XX:MaxPermSize=256m"

# in $HOME/.profile or $HOME/.bashrc, if using ksh or bash:
export MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=256m"

# in Windows Control Panel, or set it in DOS command line (This is Windows. DO NOT quote values):
set MAVEN_OPTS=-Xmx1024m -XX:MaxPermSize=256m

To run individual unit test, use -Dtest system property. Copy reference on the test method in IDE to get the full test name (command-option-shift-c in Intellij on Mac):
mvn install -Dtest=test.UserTest#testNewUser
mvn test -Dtest=test.UserTest#testNewUser

To run individual integration test, use -Dit.test system property:
mvn install -Dit.test=test.UserTestIT#testLogin
mvn verify -Dit.test=test.UserTestIT#testLogin
mvn integration-test -Dit.test=test.UserTestIT#testLogin

To pass environment variables to tests executed with maven surefire plugin:
<build>
        <pluginManagement>
            <plugins>
                <plugin>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <configuration>
                        <environmentVariables>
                            <JAVA_HOME>${env.JAVA_HOME}</JAVA_HOME>
                        </environmentVariables>
                    </configuration>
                </plugin>
            </plugins>
        </pluginManagement>
    </build>

To pass system properties to integration tests executed with failsafe plugin:
    <build>
        <plugins>
            <plugin>
                <artifactId>maven-failsafe-plugin</artifactId>
                <version>2.11</version>
                <configuration>
                    <systemPropertyVariables>
                        <project.build.directory>${project.build.directory}</project.build.directory>
                        <project.artifactId>${project.artifactId}</project.artifactId>
                    </systemPropertyVariables>
                </configuration>
                <executions>
                    <execution>
                        <goals>
                            <goal>integration-test</goal>
                            <goal>verify</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

To run mvn with an alternate user setting file, e.g., settings-jboss.xml:
mvn -s $HOME/.m2/settings-jboss.xml
The content of settings-jboss.xml:
<settings>
<repositories>
       <repository>
         <id>jboss-public-repository-group</id>
         <name>JBoss Public Maven Repository Group</name>
         <url>https://repository.jboss.org/nexus/content/groups/public-jboss/</url>
         <layout>default</layout>
         <releases>
           <enabled>true</enabled>
           <updatePolicy>never</updatePolicy>
         </releases>
         <snapshots>
           <enabled>true</enabled>
           <updatePolicy>never</updatePolicy>
         </snapshots>
       </repository>
     </repositories>
</settings>
When including mvn command in a Windows batch file, use call command.
call mvn clean install
cd integration-tests
call mvn clean install


To push tags to the remote repository:
git push --tags
To delete a tag and push this deletiion to the remote repository:
git tag (to list all tags)
git tag -d MS1
git push --delete origin MS1
To search for mvn surefire test failures on the screen: Ctrl-F <<<
<<< is probably the quickest way to find it. Searching for words like Failure gives too many results.

To increment or bump up artifact version:

mvn versions:set -DnewVersion=9.9.0-SNAPSHOT -DgenerateBackupPoms=false
This is useful when releasing a new version of the application. Note that the system propertygenerateBackupPoms=false tells maven not to generate pom.xml backup files.