Tuesday, September 11, 2012

Building OpenJDK on Windows

Experimenting with some stuff, I found that it is often useful to have JDK source code available in hand to make some changes, play with it, etc. So I decided to download and compile that beast. Apparently, it took me some time to do that, although my initial thought was that it's should be as simple as running make command :). As you can guess, I found that it's not a trivial task and to simplify my life in future, it would be useful to keep some records of what I was doing.

Below are steps which I had to do to make it happen. I assume machine already has Visual Studio 2010 installed. I have a feeling that Express version should work just fine, but I haven't tried.

  1. Install cygwin. Ensure that you have installed all packages listed here, some of them are not installed by default. Just in case, here is the copy of that table, but it is recommended to verify with the master source:

    Binary NameCategoryPackageDescriptionInstalled by default
    ar.exeDevelbinutilsThe GNU assembler, linker and binary utilitiesNo
    make.exeDevelmakeThe GNU version of the 'make' utility built for CYGWIN.No
    m4.exeInterpretersm4GNU implementation of the traditional Unix macro processorNo
    cpio.exeUtilscpioA program to manage archives of filesNo
    gawk.exeInterpretersgawkPattern-directed scanning and processing languageYes
    file.exeUtilsfileDetermines file type using 'magic' numbersYes
    zip.exeArchivezipPackage and compress (archive) filesNo
    unzip.exeArchiveunzipExtract compressed files in a ZIP archiveNo
    free.exeSystemprocpsDisplay amount of free and used memory in the systemNo
    Do not forget to add cygwin's 'bin' folder into the PATH.

  2. Install Mercurial from here and add 'hg' to the PATH.

  3. Install Microsoft Windows SDK for Windows 7 and .NET Framework 4.

  4. Install DirectX SDK. JDK requires v9.0, but I couldn't find it easily. So I decided not to bother and installed the latest one. Seems like it's working just fine.

  5. Bootstrap JDK is required for the build. It just happened that I used JDK6, but suppose, any version >JDK6 will work with no probs.

  6. Download and install Ant. I used version 1.8.2. Add Ant to the PATH.

  7. Checkout sources. For a number of reasons it was the most complicated part. 'hg' is not particularly stable, so some things which are supposed to be done my scripts, had be done manually.

    So, to start run this in command line:

    hg clone --verbose --pull http://hg.openjdk.java.net/jdk7u/jdk7u <some_folder>\openjdk7'
    
    This should download root folder with some helper scripts.

    Then in cygwin go to just created 'openjdk7' folder and run 'get_source.sh'. 'get_source.sh' may fail or just hang (and that's exactly what happened to me). If it does, then you may try to to use '--pull' flag (pull protocol for metadata). I'm not absolutely sure why, but it helped me. Unfortunately, scripts are not written in very friendly manner and there is no way to pass any 'hg' arguments to sources retrieval script. So you need to go to 'make\scripts\hgforest.sh' and add '--pull' to every invocation of 'hg clone'.

    And if even after adding '--pull' it still fails, well... just give up and run these commands manually:

    hg clone --verbose --pull http://hg.openjdk.java.net/jdk7u/jdk7u/corba corba
    hg clone --verbose --pull http://hg.openjdk.java.net/jdk7u/jdk7u/hotspot hotspot
    hg clone --verbose --pull http://hg.openjdk.java.net/jdk7u/jdk7u/jaxp jaxp
    hg clone --verbose --pull http://hg.openjdk.java.net/jdk7u/jdk7u/jaxws jaxws
    hg clone --verbose --pull http://hg.openjdk.java.net/jdk7u/jdk7u/jdk jdk
    hg clone --verbose --pull http://hg.openjdk.java.net/jdk7u/jdk7u/langtools langtools
    
    Hopefully now you have sources and can contunue :)

  8. Build requires some external binaries and a version of 'make.exe', which works under windows. 'make' which comes with cygwin doesn't really work, because has some problems with drive letters in path names.

    Next is we need to compile a couple of files. One is fixed version of 'make.exe'. The other is FreeType library, which is only available to download as source.

    If you are not interested in compiling all that stuff and want just compile JDK with less hassle, I would recommend to download binaries from here (that's my Drive). Unpack 'make.exe' into 'openjdk7/bin'. Note, that 'make.exe' from the package is quite old and requires cygintl-3.dll, which is not provided with current cygwin. To fix it, just copy cygintl-8.dll -> cygintl-3.dll.
    Freetype lib and dll has to be put in folder referenced by 'ALT_FREETYPE_LIB_PATH' conf variable (see Step 13). Also, some Freetype headers are still required and are located by make via 'ALT_FREETYPE_HEADERS_PATH' variable (see Step 13). It means you will also need to download the source code.

    If you are not looking for a simple solution and want to compile these binaries yourself, then follow these instructions:

    1. Download make 3.82 from here and unpack it. Find 'config.h.W32' and uncomment line with 'HAVE_CYGWIN_SHELL' definition. Open make_msvc_net2003.sln solution in Visual Studio, select 'Release' configuration and make a build. In 'Release' folder, you will get 'make_msvc.net2003.exe', rename it to 'make.exe'.

    2. Now compile FreeType:

      1. Download source of FreeType v.2.4.7 from here.
      2. Unpack it somewhere and open '\builds\win32\vc2010\freetype.sln' in Visual Studio.
      3. Goto project properties (right click on project in project tree) and in 'Configuration Properties/General/Configuration type' select 'Dynamic Library (.ddl)' and rename output to 'freetype'.
      4. Update ftoption.h, add following two lines:
        #define FT_EXPORT(x) __declspec(dllexport) x
        #define FT_BASE(x) __declspec(dllexport) x
      5. Make a build and you will get dll & lib in 'objs\win32\vc2010'.
      6. Do not forget to assign appropriate values to 'ALT_FREETYPE_LIB_PATH' and 'ALT_FREETYPE_HEADERS_PATH' variables (see Step 13).

  9. I had some problems with javadoc generation, which was failing with OutOfMemory. In order to fix it, I had to change 'openjdk7\jdk\make\docs\Makefile'.
    This code:

    ifeq ($(ARCH_DATA_MODEL),64)
      MAX_VM_MEMORY = 1024
    else ifeq ($(ARCH),universal)
      MAX_VM_MEMORY = 1024
    else
      MAX_VM_MEMORY = 512
    endif
    
    has to be replaed with this:
    ifeq ($(ARCH_DATA_MODEL),64)
      MAX_VM_MEMORY = 1024
    else ifeq ($(ARCH),universal)
      MAX_VM_MEMORY = 1024
    else
      MAX_VM_MEMORY = 1024
    endif
    

  10. Copy 'msvcr100.dll' to drops:

    cp /cygdrive/c/Program\ Files\ \(x86\)/Microsoft\ Visual\ Studio\ 10.0/Common7/Packages/Debugger/X64/msvcr100.dll ./drops/
    

  11. Ensure that cygwin's 'find.exe' in the PATH before the Windows' one. The easiest way to do so is to copy it into 'openjdk7/bin', which is then set at the beginning of current PATH.

  12. Create a batch file similar to the following one. Do not forgot to update paths appopriately:

    ALT_BOOTDIR=C:/Stuff/java_libs/jdk1.6.0_25
    ANT_HOME=C:/Stuff/java_libs/apache-ant-1.8.2
    JAVA_HOME=
    CLASSPATH=
    PATH=C:/Stuff/openjdk7/bin;%PATH%
    ALLOW_DOWNLOADS=true
    ALT_MSVCRNN_DLL_PATH=C:/Stuff/java_libs/openjdk7/drops
    
    C:\WINDOWS\system32\cmd.exe /E:ON /V:ON /K "C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.cmd" /Release /xp /x86
    

  13. Run batch file. Now you have fully configured environment, which is ready for build. Run 'bash' and from the shell execute 'make':

    make ARCH_DATA_MODEL=32 ALT_OUTPUTDIR=C:/Users/Stas/Stuff/java_libs/openjdk7/output_32 ALT_FREETYPE_LIB_PATH=C:/Users/Stas/Stuff/java_libs/openjdk7/freetype-2.4.7/objs/win32/vc2010 ALT_FREETYPE_HEADERS_PATH=C:/Users/Stas/Stuff/java_libs/openjdk7/freetype-2.4.7/include ALT_BOOTDIR=C:/Users/Stas/Stuff/java_libs/jdk1.6.0_25 ALT_DROPS_DIR=c:/OpenJDK/ALT_DROPS_DIR ALT_DROPS_DIR=C:/Users/Stas/Stuff/java_libs/openjdk7/drops HOTSPOT_BUILD_JOBS=4 PARALLEL_COMPILE_JOBS=4 2>&1 | tee C:/Stuff/java_libs/openjdk7/output_32.log
    
    This will start build of 32bit JDK.

  14. Have a coffee, tea, or whatever you prefer to have and then after about an hour or so you should see something like this:

    #-- Build times ----------
    Target all_product_build
    Start 2012-09-01 23:08:55
    End   2012-09-01 23:55:48
    00:02:35 corba
    00:06:46 hotspot
    00:00:30 jaxp
    00:00:51 jaxws
    00:35:30 jdk
    00:00:37 langtools
    00:46:53 TOTAL
    -------------------------
    

References:
Yet another OpenJDK on Windows Build Instruction

6 comments:

fcassia said...

can´t it be built on WinXP?

Stas said...

I'm sure it can. Shoult be ok anywhere where you can run VS2010, Java6 & cygwin.

Martijn Verburg said...

Hi there, great post! We'd love to join forces with you on the Adopt OpenJDK programme - http://java.net/projects/adoptopenjdk/ let me know :-).

Stas said...

Thanks! Just tried to register on java.net (shame, but I think, I'm not yet there :) ). Was promissed to get an email with confirmation. Still waiting :) Will join programme as soon as I get it.

Bob DeMartino said...

which "make" utility did you wind up using?

Stas said...

hi Bob, you can use either 3.82, which you have to build youself or download one from here: https://docs.google.com/open?id=0B_Oq4PMUt3r6anNXQXF2eWdGZG8