Embedding Jetty9 & Spring MVC

This post is re-done of one of my previous posts which was about embedding Jetty7. Now it's about new version - Jetty9 and also with support of Spring MVC. Just thought it would be a good idea to keep something like that as a reference. There is no much text below, this is because the source is clear enough and doesn't need much explanation. Though, feel free to raise questions in comments.

AtomicFieldUpdater vs. Atomic

Java 1.5 introduced new family of classes (Atomic*FieldUpdater) for atomic updates of object fields with properties similar to Atomic* set of classes and it seems like there is slight confusion about the purpose of these. And that confusion is understood, the reason for their existance is not very obvious. First of all they are no way faster than Atomics, if you look at source, you see that there are lots of access control checks. Then, they are not handy - developer has to write more code, understand new API, etc.

So why would you bother? There are two main use cases when Atomic*FieldUpdater can be considered an an option:

• There is a field which is mostly read and rarely changed. In that case, volatile field can be used for read access and Atomic*FieldUpdater for ocasional updates. Thought, that optimization is arguable, because there is a good chance that in latest JVMs Atomic*.get() is intrinsic and should not be slower than volatile.
• Atomics have much higher overhead on memory usage than primitives. In cases when memory is critical Atomic can be replaced with volatile primitive with Atomic*FieldUpdater.

References:
http://concurrency.markmail.org/message/ns4c5376otat2p54?q=FieldUpdater
http://concurrency.markmail.org/message/mpoy74yhuwgi52fa?q=FieldUpdater

File.setLastModified & File.lastModified

Have observed interesting behavior of File.lastModified file property on Linux. Basically, my problem was that I was incrementing the value of that property by 1 in one thread and monitoring the change in the other thread. And apparently no change in property's value happened, the other thread did not see increment. After some time trying to make it work, I realized that I have to increment it at least by a 1000 to make the change visible.

Wondering why that is happening, I have had a look at JDK source code and that's what I found:

JNIEXPORT jlong JNICALL
Java_java_io_UnixFileSystem_getLastModifiedTime(JNIEnv *env, jobject this,
                                                jobject file)
{
    jlong rv = 0;

    WITH_FIELD_PLATFORM_STRING(env, file, ids.path, path) {
        struct stat64 sb;
        if (stat64(path, &sb) == 0) {
            rv = 1000 * (jlong)sb.st_mtime;
        }
    } END_PLATFORM_STRING(env, path);
    return rv;
}

What happens is that on Linux File.lastModified has 1sec resolution and simply ignores milliseconds. I'm not an expert in Linux programming, so not sure is there any way get that time with millisecond resolution on Linux. Assume it should be possible because 'setLastModified' seems like is working as it is expected to work - sets modification time with millisecond resolution (you can find the source code in 'UnixFileSystem_md.c').

So, just a nice thing to remember: when you work with files on Linux, you may not see change in File.lastModified when it's value updated for less than 1000ms.

Building OpenJDK on Windows

Experimenting with some stuff, I found that it is often useful to have JDK source code available in hand to make some changes, play with it, etc. So I decided to download and compile that beast. Apparently, it took me some time to do that, although my initial thought was that it's should be as simple as running make command :). As you can guess, I found that it's not a trivial task and to simplify my life in future, it would be useful to keep some records of what I was doing.

Bug in Java Memory Model implementation

Just have came around amazing question on stackoverwflow:

http://stackoverflow.com/questions/10620680/why-volatile-in-java-5-doesnt-synchronize-cached-copies-of-variables-with-main

Basically the guy there is trying to use "piggybacking" to publish non-volatile variable and it doesn't work. "piggybacking" is a technique that uses data visibility guarantees of volatile variable or monitor to publish non-volatile data. For example such technique is used in ConcurrentHashmap#containsValue() and ConcurrentHashmap#containsKey(). The fact that is doesn't work in that case is a bug in Oracle's Java implementation. And that is rather scary - concurrency problems are very hard to indentify even on bug-free JVM and such bugs in Memory Model implementation making things much worse. Hopefully that's the only bug related to JMM and Oracle has good test coverage for such cases.

The good news is that this particular problem appears just on C1 (client Hotspot compiler) and not in all cases. It doesn't happen on C2 (server compiler, enabled with "-server" switch). Fortunately, the most of people are running java on server side and there are quite a few client application which are using advanced concurrency features.

For ones who want to understand that case better, please, follow the link, I've provided at the beginning of post. Also there is very useful post on "concurrency-interest", which also has a good explanation of what is going on there: http://cs.oswego.edu/pipermail/concurrency-interest/2012-May/009449.html

What is behind System.nanoTime()?

In java world there is a very good perception about System.nanoTime(). There is always some guys who says that it is fast, reliable and, whenever possible, should be used for timings instead of System.currentTimemillis(). In overall he is absolutely lying, it is not bad at all, but there are some drawback which developer should be aware about. Also, although they have a lot in common, these drawbacks are usually platform-specific.

Windows

Functionality is implemented using QueryPerformanceCounter API, which is known to have some issues. There is possibility that it can leap forward, some people are reporting that is can be extremely slow on multiprocessor machines, etc. I spent a some time on net trying to find how exactly QueryPerformanceCounter works and what is does. There is no clear conclusion on that topic but there are some posts which can give some brief idea how it works. I would say that the most useful, probably are that and that ones. Sure, one can find more if search a little bit, but info will be more or less the same.

So, it looks like implementation is using HPET, if it is available. If not, then it uses TSC with some kind of synchronization of the value among CPUs. Interestingly that QueryPerformanceCounter promise to return value which increases with constant frequency. It means that in case of using TSC and several CPUs it may have some difficulties not just with the fact that CPUs may have just different value of TSC, but also may have different frequency. Keeping all that in mind Microsoft recommends to use SetThreadAffinityMask to stuck thread which calls to QueryPerformanceCounter to single processor, which, obviously, is not happening in JVM.

Linux

Linux is very similar to Windows, apart from the fact that it is much more transparent (I managed to download sources :) ). The value is read from clock_gettime with CLOCK_MONOTONIC flag (for real man, source is available in vclock_gettime.c from Linux source). Which uses either TSC or HPET. The only difference with Windows is that Linux not even trying to sync values of TSC read from different CPUs, it just returns it as it is. It means that value can leap back and jump forward with dependency of CPU where it is read. Also, in contrast to Windows, Linux doesn't keep change frequency constant. On the other hand, it definitely should improve performance.

Solaris

Solaris is simple. I believe that via gethrtime it goes to more or less the same implementation of clock_gettime as linux does. The difference is that Solaris guarantees that counter will not leap back, which is possible on Linux, but it is possible that the same value will be returned back. That guarantee, as can be observed from source code, is implemented using CAS, which requires sync with the main memory and can be relatively expensive on multi-processor machines. The same as on Linux, change rate can vary.

Conclusion

The conclusion is king of cloudy. Developer has to be aware that function is not perfect, it can leap back or just forward. It may not change monotonically and change rate can vary with dependency on CPU clock speed. Also, it is not as fast as many may think. On my Windows 7 machine in a single threaded test it is just about 10% faster than System.currentTimeMillis(), on multi threaded test, where number of threads is the same as number of CPUs, it is just the same. And on IBM Z400 workstation with WinXP System.nanoTime() is always approximately 8 times slower.

So, in overall, all it gives is increase in resolution, which may be important for some cases. And as a final note, even when CPU frequency is not changing, do no think that you can map that value reliably to system clock, see details here (this example describes just Windows, but more or less the same stuff is applicable to all other OSes).

Appendix

Appendix contains implementations of the function for different OSes. Source code is from OpenJDK v.7.

Solaris

// gethrtime can move backwards if read from one cpu and then a different cpu
// getTimeNanos is guaranteed to not move backward on Solaris
inline hrtime_t getTimeNanos() {
  if (VM_Version::supports_cx8()) {
    const hrtime_t now = gethrtime();
    // Use atomic long load since 32-bit x86 uses 2 registers to keep long.
    const hrtime_t prev = Atomic::load((volatile jlong*)&max_hrtime);
    if (now <= prev)  return prev;   // same or retrograde time;
    const hrtime_t obsv = Atomic::cmpxchg(now, (volatile jlong*)&max_hrtime, prev);
    assert(obsv >= prev, "invariant");   // Monotonicity
    // If the CAS succeeded then we're done and return "now".
    // If the CAS failed and the observed value "obs" is >= now then
    // we should return "obs".  If the CAS failed and now > obs > prv then
    // some other thread raced this thread and installed a new value, in which case
    // we could either (a) retry the entire operation, (b) retry trying to install now
    // or (c) just return obs.  We use (c).   No loop is required although in some cases
    // we might discard a higher "now" value in deference to a slightly lower but freshly
    // installed obs value.   That's entirely benign -- it admits no new orderings compared
    // to (a) or (b) -- and greatly reduces coherence traffic.
    // We might also condition (c) on the magnitude of the delta between obs and now.
    // Avoiding excessive CAS operations to hot RW locations is critical.
    // See http://blogs.sun.com/dave/entry/cas_and_cache_trivia_invalidate
    return (prev == obsv) ? now : obsv ;
  } else {
    return oldgetTimeNanos();
  }
}

Linux

jlong os::javaTimeNanos() {
  if (Linux::supports_monotonic_clock()) {
    struct timespec tp;
    int status = Linux::clock_gettime(CLOCK_MONOTONIC, &tp);
    assert(status == 0, "gettime error");
    jlong result = jlong(tp.tv_sec) * (1000 * 1000 * 1000) + jlong(tp.tv_nsec);
    return result;
  } else {
    timeval time;
    int status = gettimeofday(&time, NULL);
    assert(status != -1, "linux error");
    jlong usecs = jlong(time.tv_sec) * (1000 * 1000) + jlong(time.tv_usec);
    return 1000 * usecs;
  }
}

Windows

jlong os::javaTimeNanos() {
  if (!has_performance_count) {
    return javaTimeMillis() * NANOS_PER_MILLISEC; // the best we can do.
  } else {
    LARGE_INTEGER current_count;
    QueryPerformanceCounter(¤t_count);
    double current = as_long(current_count);
    double freq = performance_frequency;
    jlong time = (jlong)((current/freq) * NANOS_PER_SEC);
    return time;
  }
}

References

Inside the Hotspot VM: Clocks, Timers and Scheduling Events
Beware of QueryPerformanceCounter()
Implement a Continuously Updating, High-Resolution Time Provider for Windows
Game Timing and Multicore Processors
High Precision Event Timer (Wikipedia)
Time Stamp Counter (Wikipedia)

The magic of conditional operator

Can you guess what is going to be the output of the following piece of code?

    Object obj = false ? new Long(1) : new Integer(1);
    System.out.println(obj.getClass());

The smart once can guess that it is going to be "class java.lang.Long", otherwise there won't be a question. Now can you answer why? To be honest, I was a surprised by such an outcome, but apparently, according to JLS that is correct behaviour. Here is quotation from "15.25 Conditional Operator ? :":

The type of a conditional expression is determined as follows:

• If the second and third operands have the same type (which may be the null type), then that is the type of the conditional expression.
• If one of the second and third operands is of type boolean and the type of the other is of type Boolean, then the type of the conditional expression is boolean.
• If one of the second and third operands is of the null type and the type of the other is a reference type, then the type of the conditional expression is that reference type.
• Otherwise, if the second and third operands have types that are convertible (§5.1.8) to numeric types, then there are several cases:
  • If one of the operands is of type byte or Byte and the other is of type short or Short, then the type of the conditional expression is short.
  • If one of the operands is of type T where T is byte, short, or char, and the other operand is a constant expression of type int whose value is representable in type T, then the type of the conditional expression is T.
  • If one of the operands is of type Byte and the other operand is a constant expression of type int whose value is representable in type byte, then the type of the conditional expression is byte.
  • If one of the operands is of type Short and the other operand is a constant expression of type int whose value is representable in type short, then the type of the conditional expression is short.
  • If one of the operands is of type; Character and the other operand is a constant expression of type int whose value is representable in type char, then the type of the conditional expression is char.
  • Otherwise, binary numeric promotion (§5.6.2) is applied to the operand types, and the type of the conditional expression is the promoted type of the second and third operands. Note that binary numeric promotion performs unboxing conversion (§5.1.8) and value set conversion (§5.1.13).
• Otherwise, the second and third operands are of types S1 and S2 respectively. Let T1 be the type that results from applying boxing conversion to S1, and let T2 be the type that results from applying boxing conversion to S2. The type of the conditional expression is the result of applying capture conversion (§5.1.10) to lub(T1, T2) (§15.12.2.7).

In the other words, it says that if second and third operands are convertible to primitive numeric types, then the result type is based on numeric promotion of converted values. Here how it looks after applying these rules:

Object obj = Long.valueOf(false ? (new Long(1L)).longValue() : (new Integer(1)).intValue());

My opinion, it all looks like a magic, and at the end, that's the price Java paid for autoboxing. Well world is not perfect and that's it's beauty, isn't it? :)

Java file flushing performance

There are many situations when it is required to ensure that data was written to the disk and write is also required to be fast. The most most common where it has to happen are databases, journalling, etc. Also, it is often required to update some random position in a file. I specifically what to place emphasis on random access here, as far as the rest will cover just cases where it is supported, i.e. I'm not going to mention OutputStream.flush() & related topics. Just haven't tried it, as far as that wasn't my case at the moment.

There are several way of flushing the data to disk in java. These options can be quite different in the way they implemented internally and in their performance. Here is the list of existing things you can do:

• FileChannel.force()
• 'rws' or 'rwd' mode of RandomAccessFile, which 'works much like the force(boolean) method of the FileChannel class' (from javadoc).
• MappedbyteBuffer.force()
• RandomAccessFile.getFD().sync()
• any close() method. Here I mean seek and close stream each time when access is required. Doing tests, I actually didn't seek, as far was updating data with zero offset.

Surprisingly (the only unsurprising exception is close()) all these methods gives very different performance and it varies almost randomly on different OSes and file systems. Worth noticing that hardware can also put it's correction on the performance of any of these methods. I have also a strong feeling that performance may vary even with minor change in OS or JVM version number. Here is the table with time it takes to flush 8bytes (keep in mind, that the real amount of flushed data depends on the size of caches and going to be much more that 8byes), just to give a flavour of how different is that:

	RandomAccessFile. getFD().sync()	RandomAccessFile, rwd mode	MappedbyteBuffer.force()	FileChannel.force()
Windows	0.2818ms	0.0125ms	0.007ms	0.139ms
Linux	0.5354ms	0.5144ms	0.4663ms	0.0093ms

Please, do not treat these numbers as any relevant result. They are here just to give an example how these things can vary.

So, what the conclusion? Conclusion is that if you would need to write high-performance application which does lots of IO, you really need to test different approached on different OSes, on different file systems and, preferably  on different JVMs. Do not expect something to be fast on Linux (Solaris, AIX, etc) production box, when it is fast on your Windows (Linux, etc) workstation and vice versa. As can be seen, the difference can be in orders of magnitude.

The most complete list of -XX options for Java JVM

There was a wonderful page with that list (http://www.md.pp.ru/~eu/jdk6options.html), but it seems it's gone now and not available any more. Luckily there is http://web.archive.org which allows to make time flowing backwards and recover things which have passed away long time back. So, I have just get that page out of grave and copy/pasted here to return that bit of information back to life. Also I'm constantly doing updates to the original list, so by now it should be longer than it was initially.

EhCache replication: RMI vs JGroups.

Recently, I was working on one product which required replicated caching. Caching provider was already decided - EhCache, and the remained was a question about transport. Which one is the best option? By the best option here I mean just the one which has better performance. The performance measurement was done just between two of available transports - JGroups and RMI, others were not considered, sorry.

jakarta regexp vs java.util.regex

Have never really been thinking about which library to use for regular expression, it always was java.util.regex by default. But I found that some people prefer to use Jakarta Regexp, and usually there is not clear reason for that choice. So, I decided to spent some time trying to find out what is better and probably write a couple of tests. After some time spent in the Internet looking for the examples of tests, it appeared that there was a good guy already who have done all that work. Here is the link to the page with results of his investigation:

http://tusker.org/regex/regex_benchmark.html

The conclusion is that Jakarta Regexp doesn't look good at all. Also you may notice that there are some other libraries who are doing regexps and some have very impressive performance and seems like worth trying.

UPDATE 09/2012: Thank's for comments from Cd MaN, there is newer update of that benchmark: http://hype-free.blogspot.ro/2008/12/big-java-regex-shoutout.html

JCaptcha, SecureRandom & performance

Personally, I'm not big fan of JCaptcha library and recently was lucky enough to find another problem there. The problem is related mostly to linux and it's implementation of java.security. SecureRandom, which is known to be slow and is locking on every call to it.

For some reason (I suspect that this was the reason) JCaptacha overuse java.security.SecureRandom when it generates background image using FunkyBackgroundGenerator. Number of calls to get next random float can easily reach something around 100 000 per one captcha image. It is basically called at least once for each pixel.

Lets run some quick tests to have a look how bad is that. I have wrote write simple captch engine:

public static class MyImageCaptchaEngine extends ListImageCaptchaEngine {
  protected void buildInitialFactories() {
      WordGenerator wgen = new RandomWordGenerator("ABCDEFGHIJKLMNOPQRSTUVWXYZ123456789");
      RandomRangeColorGenerator cgen = new RandomRangeColorGenerator(
         new int[] {0, 100},
         new int[] {0, 100},
         new int[] {0, 100});
      TextPaster textPaster = new RandomTextPaster(new Integer(7), new Integer(7), cgen, true);

      BackgroundGenerator backgroundGenerator = new FunkyBackgroundGenerator(new Integer(200), new Integer(100));

      Font[] fontsList = new Font[] {
         new Font("Arial", 0, 10),
         new Font("Tahoma", 0, 10),
         new Font("Verdana", 0, 10),
      };

      FontGenerator fontGenerator = new RandomFontGenerator(new Integer(20), new Integer(35), fontsList);

      WordToImage wordToImage = new ComposedWordToImage(fontGenerator, backgroundGenerator, textPaster);
         this.addFactory(new GimpyFactory(wgen, wordToImage));
  }
}

And the test itself:

long begin = System.currentTimeMillis();
for(int i = 0; i != 100; ++i) {
  engine.getNextCaptcha();
}
long end = System.currentTimeMillis();
System.out.println("Total time is [" + (end - begin) + "]");

now lets run it:

Total time is [10967]

Ok, so what we have now it 10967ms, which, I believe, is bad. It can be significantly improved. I'm not very big fan of high-quality random backgrounds, so I will replace SecureRandom class, used by parent of FunkyBackgroundGenerator with "pseudo" random Random class. Funs of high-quality random backgrounds can still use SecureRandom for seeding, through:

public static class MyFunkyBackgroundGenerator extends FunkyBackgroundGenerator {
  public MyFunkyBackgroundGenerator(Integer width, Integer height) {
     super(width, height);
     try {
         Field rndField = AbstractBackgroundGenerator.class.getDeclaredField("myRandom");
         rndField.setAccessible(true);
         rndField.set(this, new Random());
     }
     catch (Exception e) {
         e.printStackTrace();
     }
  }
}

I know, that is dirty hack, but as far "myRandom" is declared with default visibility, that's the shortest way to replace it for now . And what we have now is:

Total time is [1308]

Approximately 7 time quicker. Not that bad, specially for a sample case. In real world application improvement will be even more significant because some other processes may use '/dev/(u)random' or application itself can utilize SecureRandom for other purposes.

That's not it. There is another bottleneck, which is usage of Java2D for rendering captcha images. Java2D is well known for it's problems with multi-threading and the summary of these problems is that Java2D doesn't scale well. Some details can be found here. Possibly the way to fix that problem may be removing Java2D and using direct access to image via WritableRaster instead. However, it doesn't solve all problems, as far as Java2D is still used for drawing text.

ConcurrentHashMap revealed

Java 1.5 introduced some new cool concurrency stuff which is located in java.util.concurrent package. There are lots of good things there including new ConcurrentHashMap class which I'm going to talk about. That class is targeted to be used in concurrent environment and provides significant performance benefits over synchronized version of HashMap. Javadoc doesn't really provide lots of details on how that class works and why can be better than synchronized version of HashMap. I think that understanding of these details is crucial for using that class in a right way, so I've made some research to undercover implementation details and mechanisms used to implement that class.

Killing the private (and protected)

Recently on one forum I've found topic which released from memory my C++ days. That was topic about using "private" modifier on methods. The question there was about cases where to use it and why. Everybody knows when you are programming on C++ (as the most glaring example) one of "good practices" is "less client can do - better". Of course that's not in sense of functionality provided to client, but in mind of something which is not explicitly provided. In C++, I believe, the main reason for that is fragility of runtime, which can be easily killed if someone accidentally will do something wrong, which can also cause physical damage to person who is responsible for doing that (worth mentioning, in Java situation is slightly different, it's hard (but still possible, of course) to kill the application by "accidental coding"). And a consequence of such “good practice” is the basic rule – “make private as much as you can”.

Java bridge methods explained

Bridge methods in Java are synthetic methods, which are necessary to implement some of Java language features. The best known samples are covariant return type and a case in generics when erasure of base method's arguments differs from the actual method being invoked.

Exporting keys from keystore

Recently I had a similar feeling to one I had writing one of previous posts. It appeared that standard java tools do not have some basic functionality, which obviously (well, probably just for me :) ) should be there. Now I had to export key stored in keystore to share it with other department. It appeared, that keytool can't do that and you have to write tiny program by yourself. Not a big problem, really, it's even nice.

There are lots of posts in web describing how to solve that problem, and here is the best code example I found so far to solve that it.
And here is copy/paste of code snipped, just in case if original post will pass away.

File keystoreFile = new File("The filename of the keystore");
KeyStore ks = KeyStore.getInstance("JKS"); // or whatever type of keystore you have
char[] pw = "the keystore password".toCharArray();
InputStream in = new FileInputStream(keystoreFile);
ks.load(in, pw);
in.close();
for (Enumeration en = ks.aliases(); en.hasMoreElements();)
{
    String alias = en.nextElement();
    System.out.println("  Alias\t:" + alias);
    // If the key entry password is not the same a the keystore password then change this
    KeyStore.Entry entry = ks.getEntry(alias, new KeyStore.PasswordProtection(pw)); 
    if (entry instanceof KeyStore.SecretKeyEntry) {
        System.out.println("    SecretKey");
        KeyStore.SecretKeyEntry skEntry = (KeyStore.SecretKeyEntry) entry;
        SecretKey key = skEntry.getSecretKey();
        System.out.println("      alg\t: " + key.getAlgorithm());
    } else if (entry instanceof KeyStore.PrivateKeyEntry) {
        System.out.println("    PrivateKey");
        KeyStore.PrivateKeyEntry pkEntry = (KeyStore.PrivateKeyEntry) entry;
        PrivateKey key = pkEntry.getPrivateKey();
        System.out.println("      alg\t: " + key.getAlgorithm());
        java.security.cert.Certificate certificate = pkEntry.getCertificate();
        System.out.println("      Certificate type\t: " + certificate.getType());
        System.out.println("      Public key\t: " + certificate.getPublicKey().getAlgorithm());
    } else if (entry instanceof KeyStore.TrustedCertificateEntry) {
        System.out.println("    Certificate");
        KeyStore.TrustedCertificateEntry certEntry = (KeyStore.TrustedCertificateEntry) entry;
        java.security.cert.Certificate certificate = certEntry.getTrustedCertificate();
        System.out.println("      type\t: " + certificate.getType());
    }
}

If you need to send key to someone, it handy to make it base64 encoded:

byte[] keyData = key.getEncoded();
BASE64Encoder b64Encoder = new BASE64Encoder();
String b64 = b64Encoder.encode(keyData);
System.out.println("-----BEGIN KEY-----");
System.out.println(b64);
System.out.println("-----END KEY-----");

Compile recursively with javac

To my shame I have never used javac directly for compiling source files, always it was something like "ant". And today it was the first time on my memory when I had to do that :) It was a simple application, just an example for presentation and I didn't want to add anything additional there.
Thing which looks like a trivial task appeared to be not so trivial, because javac doesn't recursively compile files in directories, you have to specify each directory separately or create file with list of files for compilation, something like that:

find ./src -name "*.java" > sources_list.txt
javac -classpath "${CLASSPATH}" @sources_list.txt


On Windows first line should be replaced with: dir .\src\*.java /s /B > sources_list.txt

List of Open source trading software

For everybody who is interested in topic. List of links below should help to make initial evaluation of available open-source java trading software & related products and also provides some other interesting links. Note, that projects below are not ordered in any particular order.

Groups, forums, communities

Google Group, JavaTraders
http://groups.google.com/group/JavaTraders

Elite trader, the #1 community for active traders of Stocks, Futures, Options, and Currencies.
http://www.elitetrader.com/

Trading software

Marketcetera
Open source platform for strategy-driven trading, providing you with all the tools you need for strategy automation, integrated market data, multi-destination FIX routing, broker neutrality and more.
Looks like is the leader in that list - it's well supported, has lots of capabilities and is active project.
Latest version available at 23.12.2009: 1.5.0 (released 05.2009)
http://trac.marketcetera.org/
http://www.marketcetera.com/

EclipseTrade
EclipseTrader is an application focused to the building of an online stock trading system, featuring shares pricing watch, intraday and history charts with technical analysis indicators, level II/market depth view, news watching, and integrated trading. The standard Eclipse RCP plug-ins architecture allows third-party vendors to extend the functionality of the program to include custom indicators, views or access to subscription-based data feeds and order entry.
Latest version available at 23.12.2009: 0.30.0 (released 07.2009)
http://sourceforge.net/projects/eclipsetrader/
http://eclipsetrader.sourceforge.net/

JSystemTrader
JSystemTrader is a fully automated trading system (ATS) that can trade various types of market securities during the trading day without user monitoring. All aspects of trading, such as obtaining prices, analyzing price patterns, making trading decisions, placing orders, monitoring order executions, and controlling the risk are automated according to the user preferences. The central idea behind JSystemTrader is to completely remove the emotions from trading, so that the trading system can systematically and consistently follow a predefined set of rules.
Latest version available at 23.12.2009: 6.24 (released 09.2008)
http://groups.google.com/group/jsystemtrader

ActiveQuant
AQ is a framework or an API for automated trading, opportunity detection, financial engineering, research in finance, connecting to brokers, etc. - basically everything around trading, written in Java, using Spring. All is published under a usage friendly open source license.
Latest version available at 23.12.2009: ??? Failed to find any possibility to download it or get latest version number, all links to that information are broken.
http://www.activestocks.eu/?q=node/1
http://www.activestocks.eu/

AIOTrade
AIOTrade (former Humai Trader) is a free, open source (under the terms of BSD license) stock technical analysis platform with a pluggable architecture that is ideal for extensions such as indicators and charts. It's built on pure java.
Latest version available at 23.12.2009: 1.0.3a (released 02.2007)
http://sourceforge.net/projects/humaitrader
http://blogtrader.org/

JStock
JStock makes it easy to track your stock investment. It provides well organized stock market information, to help you decide your best investment strategy.
No automated trading support.
Latest version available at 29.12.2009: 1.0.5g (released 12.2009)
http://jstock.sourceforge.net
https://sourceforge.net/projects/jstock/

Merchant of Venice
Venice is a stock market trading programme that supports portfolio management, charting, technical analysis, paper trading and experimental methods like genetic programming. Venice runs in a graphical user interface with online help and has full documentation.
Latest version available at 23.12.2009: 0.7b (released 04.2006)
http://sourceforge.net/projects/mov
http://mov.sourceforge.net/

Market Analysis System
The Market Analysis System (MAS) is an open-source software application that provides tools for analysis of financial markets using technical analysis. MAS provides facilities for stock charting and futures charting, including price, volume, and a wide range of technical analysis indicators. MAS also allows automated processing of market data — applying technical analysis indicators with user-selected criteria to market data to automatically generate trading signals — and can be used as the main component of a sophisticated trading system.
Latest version available at 23.12.2009: 1.6.6 (released 07.2004)
http://sourceforge.net/projects/eiffel-mas
http://eiffel-mas.sourceforge.net/

Open Java Trading System
The Open Java Trading System (OJTS) is meant to be a common infrastructure to develop stock trading systems. The project's aim is to provide a self contained pure Java (platform independent) common infrastructure for developers of trading systems.
Latest version available at 23.12.2009: 0.13 (released 06.2005)
http://sourceforge.net/projects/ojts/
http://ojts.sourceforge.net/

Oropuro trading system
The software perform the technical analysis of stock or commodity for various markets, manage portfolio definitions and orders. It has the base characteristics of most populars technical analysis software.
The most of information about that project is in Italian language, so it's really hard to dive in it :(
Latest version available at 23.12.2009: 0.2.4 (released 11.2007)
http://sourceforge.net/projects/oropuro
http://www.oropuro.org

TrueTrade
TrueTrade is a framework for developing, testing and running automatic trading systems. It is intended to provide support for a wide range of orders, financial instruments and time scales. It provides tooling for backtesting the strategy against historical data, and a separate tool for running the strategies in live mode.
Latest version available at 23.12.2009: 0.5 (released 05.2007)
http://code.google.com/p/truetrade/
http://groups.google.com/group/TrueTrade-Gen
http://groups.google.com/group/TrueTrade-Dev

(j)robotrader
Robotrader is a simulation platform for automated stock exchange trading. It delivers statistics to analyse performance on historic data and allows comparison between trading strategies.
Latest version available at 23.12.2009: 0.2.7 (released 02.2006)
http://jrobotrader.atspace.com
http://sourceforge.net/projects/robotrader/

SFL Java Trading System Enviroment
At current moment project is inactive. Author suggests to use ActiveQuant instead.
http://sourceforge.net/projects/sfljtse
http://www.sflweb.org/index.php?blog=sfljtse

Related software

TA-Lib: Technical Analysis Library
TA-Lib is widely used by trading software developers requiring to perform technical analysis of financial market data.
* Includes 200 indicators such as ADX, MACD, RSI, Stochastic, Bollinger Bands etc...
* Candlestick pattern recognition
* Open-source API for C/C++, Java, Perl, Python and 100% Managed .NET
Latest version available at 23.12.2009: 0.4 (released 09.2007)
http://ta-lib.org/index.html

Tail - A java technical analysis lib
Technical Analysis studies forecasting future price trends with the objective of managea best moment to buy and sell shares. The Tail's target is to develop a Java Open-Source library that abstracts the basic components of Technical Analysis, supplying tools for creation, manipulation and evaluation of strategies to buy and sell.
Latest version available at 15.01.2010: 1.0 (released 12.2007)
http://tail.sourceforge.net/

JessX
JessX Project's main goal is to create a program allowing for the simulation of a financial market with realistic features (such as an order book and realistic orders). Researchers and teachers in Finance may find it helpful in their works.
Latest version available at 23.12.2009: 1.5 (released 05.2008)
http://jessx.ec-lille.fr/

QuickFIX/J
100% Java Open Source FIX (Financial Information eXchange protocol) Engine
Latest version available at 23.12.2009: 1.4 (released 02.2009)
http://www.quickfixj.org/

Auge
Auge is an easy-to-use and very simple financial portfolio management application. Auge will help you monitor and analyze your stock and mutual fund positions, providing powerful insight into your entire investment portfolio.
Latest version available at 23.12.2009: 0.2 (released 04.2007)
http://sourceforge.net/projects/auge
http://auge.sourceforge.net/

Matrex
Advanced spreadsheet.
Latest version available at 23.12.2009: 1.3.8 (released 10.2009)
http://sourceforge.net/projects/matrex/
http://matrex.sourceforge.net/

Data Visualizer
Data Visualizer displays text file stock market type data ("Date,Open,High,Low,Close,Volume,Adjusted Close Price") as Stock Charts, featuring a variation of Japanese "Candlesticks" chart elements.
Latest version available at 23.12.2009: 0.0.1 (released 03.2006)
http://sourceforge.net/projects/dataviews
http://dataviews.sourceforge.net/

Forex Optimizer
Absolutely new revolutionary trade platform, is intended both for beginners, and for the tempered traders of Forex. Beginners can study market Forex, using a simulator, not risking the capitals and not being connected to the Internet. For more skilled traders Forex Optimizer allows to create and optimize trade strategy, not having knowledge in programming to operate (to make trading operations) the real account of the broker. The platform can offer professionals greater functionality for application of the strategy and methods of trade in market Forex.
Latest version available at 08.12.2010: 2.7 (released ??)
http://www.gordago.com/opensource/forex-optimizer/

http://gordago.googlecode.com/svn/tags/2.7/

Hibernate + Spring in Standalone application

Just a quick reminder howto use Hibernate + Spring in standalone application.
First comes hibernate session factory. I prefer to use separate file for hibernate, Instead of configuring it Spring's applicationContext.xml:

<bean name="hibernateSessionFactory" class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean">
    <property name="configLocation" value="hibernate.cfg.xml"/>
    <!-- Those package will be scanned for classes with persistence annotations -->
    <property name="packagesToScan" value="net.test.domain"/>
    <!-- Annotated package. Contains package-level configuration. -->
    <property name="annotatedPackages" value="net.test.domain"/>
</bean>

Transaction manager:

<bean id="hibernateTransactionManager" class="org.springframework.orm.hibernate3.HibernateTransactionManager">
    <property name="sessionFactory" ref="hibernate.session.factory"/>
</bean>
<tx:annotation-driven transaction-manager="hibernateTransactionManager"/>

Next is hibernate session. Hibernate guys suggest to use HibernateDaoSupport class as base for your DAOs, but to be completely honest, I'm not really comfortable with it, because it adds really weird dependency on Spring in DAO classes. Instead, it looks more natural to use dependency injection, which is really Spring's approach. To archive that all you need to do is mark session definition as being scoped proxy, set scope to 'prototype' and define SessionFactoryUtils#getSession as factory method. In result, each call to "any_method" in hibernate session bean instance is converted to SessionFactoryUtils.getSession().any_method():

<bean name="hibernateSession" class="org.springframework.orm.hibernate3.SessionFactoryUtils" factory-method="getSession"
  scope="prototype">
    <constructor-arg index="0" ref="hibernateSessionFactory"/>
    <constructor-arg index="1" value="false"/>
    <aop:scoped-proxy/>
</bean>

And all together:

<?xml version="1.0" encoding="UTF-8"?>
<beans
xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:aop="http://www.springframework.org/schema/aop"
xmlns:tx="http://www.springframework.org/schema/tx"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-2.0.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-2.5.xsd">

    <bean name="hibernateSessionFactory" class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean">
        <property name="configLocation" value="hibernate.cfg.xml"/>
        <!-- Those package will be scanned for classes with persistence annotations ->
        <property name="packagesToScan" value="net.test.domain"/>
        <!-- Annotated package. Contains package-level configuration. -->
        <property name="annotatedPackages" value="net.test.domain"/>
    </bean>

    <bean id="hibernateTransactionManager" class="org.springframework.orm.hibernate3.HibernateTransactionManager">
        <property name="sessionFactory" ref="hibernateSessionFactory"/>
    </bean>
    <tx:annotation-driven transaction-manager="hibernateTransactionManager"/>
    <bean name="hibernateSession" class="org.springframework.orm.hibernate3.SessionFactoryUtils" factory-method="getSession"
  scope="prototype">
        <constructor-arg index="0" ref="hibernateSessionFactory"/>
        <constructor-arg index="1" value="false"/>
        <aop:scoped-proxy/>
    </bean>

    <bean name="someDao" scope="singleton" class="net.test.TestDAO">
        <property name="session" ref="hibernateSession"/>
    </bean>
</beans>

In DAO class there is no need to worry about session management, "tx:annotation-driven" will do all work for you. The only thing, which developer has to think about is appropriate usage of transaction annotations.

How to make IntelliJ Idea usable on linux

Which Idea on Linux there are two big problems - memory (which I failed to resolve :)) and overall performance. When I've just installed it, it wasn't usable on may box with four Xeon CPUs and four gigs of memory, and that fact surprised me a lot. After some time spent looking on Internet found several tips, which resolved most of my problems. First one is following java setting, which switches off storing images in pixelmaps:

-Dsun.java2d.pmoffscreen=false

Another thing is one of Linux file system settings, which disables update of accessed timestamp. I'm working with quite big projects, which have several thousand files and updating timestamp of each file access is unnecessary overhead. That feature is something which doesn't have lots of use and can be witched off, specially when you are running Linux as desktop. Just add following to your filesystem mount settings in fstab:

noatime,nodiratime

After that Idea became reasonably fast, but I must admit, on Win XP or Vista it still runs faster and uses less memory.

I'm running Fedora 10 with NVidia graphic drivers.

Udpate 2010.12: I've just installed Idea 10. It looks just awesome and it's much faster! Works brilliantly. Thanks to IntelliJ!