Technology and fun

software development and computer gaming

Using JMX with SSL

Java platform includes JMX technology developed to simplify management of distributed enterprize systems. The idea is that components of enterprize application expose their management-related operations in declarative way, and enterprize container/framework discovers them and collects references in single registry called MBeanServer. Remote access to MBean server is provided by JMX connector to any software which should invoke management actions, such as generic GUI tools like JConsole or VisualVM, or Management consoles provided with enterprize application containers, or special-purpose tools.

JMX technology also available for Java applications which are not running in any container or framework. Java runtime library includes PlatformMBeanServer which is a singleton. However, registering of manageable classes should be done explicitly in your app, since there are no container. Remote access is provided by JMX connector which is configured via system properties usually specified via command-line.

JMX connector works on top of another Java technology called RMI (remote method invocation) which is configured by default to work over secure transport. Using secure transport by default is a strange choice because secure transport will not work itself by default, it requires some configuration. This configuration also could be done via system properties, but it will affect secure transport for whole process, not just JMX connectivity. So, it will not work for you if your Java process has several SSL connections each with specific settings.

So here we will take a closer look to what happens with built-in JMX when you switch it on in your application, and how you can control various aspects of it.

Java documentation includes “Java SE Monitoring and Management Guide” which explains various details of JMX technology. Chapter 2 is called “Monitoring and Management Using JMX Technology” and it describes system properties used to configure built-in JMX agent. There is also a very important part called “Example of Mimicking Out-of-the-Box Management” with a piece of code which will be our base for modifications.

Take a closer look at this code. First, an RMI registry is created, which will live in current Java process. Second, a PlatformMBeanServer is obtained. And, finally, and JMXConnectorServer is created, which is configured to use SslRMIClientSocketFactory and SslRMIServerSocketFactory. So, if we need to customize SSL params, those factories is the place to do that.

In SSL server confirms it’s identity by encoding some value supplied by client using server’s private key, and client checks that by decoding value using server’s public key. Those keys are part of SSL configuration for server and client. In Java those key are usually stored in files. File with private keys is called “KeyStore”, and file with public keys of trusted entities is called “TrustStore”, both files protected with passwords. These files are supplied to instance of SSLContext, which is used to create a SslRMIServerSocketFactory. If you will not supply any SSLContext to SslRMIServerSocketFactory, then a platform-default instance of SSLContext will be used. This instance is configured by system properties called “” and “”, and there are two more properties to supply passwords. There are no default value for keystore of default SSLContext, but there is a default value for truststore of default SSLContext. Default truststore is distributed with Java, and includes public keys of several certificate authorities.

If we will not specify any keystore and will start an application, then you’ll get errors on your server when client will try to connect: “ no cipher suites in common”. This obscure error means that server cannot find any private keys to prove it’s identity.

Since JMX Connector is an SSL server, then you’ll need to obtain a private key (or generate a self-signed key yourself). If you expose JMX to be used by generic GUI tools like JConsole or VisualVM, then you’ll need to generate a public key from your private key using keytool, then configure GUI tools to use truststore containing generated public key.

If you need to add secure JMX to application which already uses SSL, you have following options:
1. If default SSLContext is used by your application, you can add new private key to existing keystore. That’s easy to do, but you will not be able to control which private key will be used by which SSL connection. If this is ok to you, then you don’t need to change any SSL-related properties, which are already present in your application, since they are the only way to configure default SSLContext. This context is by default used by out-of-box JMX connector, so you can throw away all custom JMX code and use only command-line configs.
2. If your application doesn’t use default SSL context, then you can use default SSLContext for JMX only, without a risk to affect anything else. In this case you can (and should!) configure both SSL and JMX via system properties.
3. If you don’t want to use default SSLContext (either because it is already used by your application, or by other reasons), then you can create a separate SSLContext, then configure it with your custom keystore, then use this SSLContext to create a SslRMIServerSocketFactory.

Options 1 and 2 are quite simple and don’t require additional explanations. So we recommend using them if possible, especially option 2. There are quite a lot of command-line keys to specify, but this is a price of flexibility. Option 3 is mostly for those who prefer to have everything explicit, and remainder of article will continue after option 3.

So, after you have customized SslRMIServerSocketFactory, you will probably take a look at SslRMIClientSocketFactory, and you should be surprized to see that you can’t customize it in any way! We will get back to it later.

When you’ll need to troubleshoot your application, you should use a set of system properties to enable RMI logging, and also a set of system properties to enable SSL logging.

With logging enabled you’ll get to situation when everything works, so you can connect to your application via JConsole. However, you’ll have lots of error messages in your server log with following text: “ Received fatal alert: certificate_unknown”. This error will happen in case of using default SSLContext and in case of using dedicated SSLContext. What is going on?

When you start JMX connector, it will try to bind itself to JMX URL in RMI registry. This RMI registry is running locally in the same process, however there are no any shortcuts in RMI, and a network connection is used to communicate with registry. This connection is plain TCP, without any SSL. RMI registry receives not just a serialized remote interface, but also a serialized instance of RMIClientSocketFactory which will be used to make remote calls to this object. Each RMI client obtains a remote reference from RMI Registry together with corresponding RMIClientSocketFactory, and client should not care if underlying transport is plain TCP or SSL, because supplied RMIClientSocketFactory will be used to make connection from client to server. Unfortunatelly, this transport independence works only in theory, because in case of SSL it is a client’s job to configure SSLContext with truststore, because SSLContext is not serialized by SslRMIClientSocketFactory.

If server’s private key is signed with certificate authority which is present in client’s default truststore, then all will work out-of-box without any additional configuration. Otherwize, either a public key should be added to default truststore or special truststore should be created and specified via command-line key. That’s why you need to specify truststore when starting JMX GUI tools.

The same RMIClientSocketFactory is also used by Registry to return result of bind invocation. However, SslRMIClientSocketFactory always uses default SSLContext which in our case is not the same as SSLContext used by SslRMIServerSocketFactory. Server’s private key doesn’t have a public key counterpart in default truststore, that’s why you have this error in log.

If your server’s private key is signed with certificate authority which is present in default truststore, then you are lucky and you will not get this error at all. Another solution is to put server’s public key into truststore of default SSLContext. Yes, this sounds odd, but if you are starting JMX-over-RMI-over-SSL server, then you need to configure not only keystore, but also truststore! But that is not always possible.

If your RMI registry would be started as a separate process, you would have same errors in RMI registry, so truststore had to be configured for RMI registry. So, having RMI registry in the same process doesn’t save you from configuring truststore. Most funny, even if you are using default SSLContext by SslRMIServerSocketFactory, so private key is in keystore for default SSLContext, you still have to put public key into truststore for the same SSLContext! In other words, a process doesn’t trust itself. In my opinion, specifying only private key should be enough to allow loopback connections.

Good news are that in most cases you can switch off debug logging and ignore this error. JMX connector is succesfully bound in RMI registry, and error happens when regstry is trying to return result of this operation back to server. If server doesn’t care about result, then this error can be ignored.

Let’s check if we can solve this problem by providing a different RMIClientSocketFactory which will be serialized including server’s public key? Such solution will also allow us to avoid any configuration for GUI clients! Nice idea, but it will not work. You’ll get errors like: “ Failed to retrieve RMIServer stub: javax.naming.CommunicationException [Root exception is java.rmi.UnmarshalException: error unmarshalling return; nested exception is: java.lang.ClassNotFoundException: mycustomapp.SslRMIClientSocketFactory”. RMI serialization requires that client has classes to be serialized in it’s classpath. Configuring GUI JMX clients with server’s custom classes is more difficult then configuring truststore. Default SslRMIClientSocketFactory is a part of Java runtime library so it is present in classpath of every JMX GUI tool, that’s why there are no problems with RMI serialization, and that’s why you can’t customize it in any way.

It is very interesting to note that out-of-box JMX agent doesn’t suffer from such problem! You can specify only keystore and there will be no errors in debug logs. How do they do that? Take a look at class where system properties are handled. You’ll see that a special SingleEntryRegistry is created. So, there are no need for RMI call to bind object into such registry.

Practical essentials of Java web stack


Common prerequisites are listed here

  1. Maven is used as a build tool

Standalone Jetty 9 serving static files

  1. Download Jetty 9 distribution
  2. Generate Jetty base at some location, like this:
    C:\projects\my-jetty-instance>%JAVA_HOME%bin\java -jar c:\tools\jetty-distribution-9.4.3\start.jar --add-to-start=http,deploy,logging-jetty 
  3. In webapp dirs, create subdir with any name. Put your files there
  4. Start Jetty
  5. Your files will be available in URL

Plain servlet with standalone Jetty 9

  1. Download Jetty 9 distribution
  2. Generate Jetty base at some location, like this:
    C:\projects\my-jetty-instance>%JAVA_HOME%\bin\java -jar c:\tools\jetty-distribution-9.4.3\start.jar --add-to-start=http,deploy,logging-jetty
  3. From IDE or from command line generate a project using maven-archetype-webapp. It will generate a pom.xml and some project dirs, including almost empty web.xml.
  4. Maven, however, will not generate a source dir for your Java code. Do it yourself in /src/main/java.
  5. Also Maven will not add Servlet API jar to your project’s dependencies. Add something like this:
  6. Write your basic servlet class which will extend javax.servlet.http.HttpServletRequest. Note that to generated HTML should go to response’s body obtained via getOutputStream(). Use print() method.
  7. Extend web.xml by adding servlet descriptor and URI mappings:
  8. Use mvn package to compile and package your project into war file. Note that this file will be named as artefactId from pom.xml
  9. Put file to webapps dir of Jetty. Run Jetty.
  10. Test your app by opening URL (place artefactId of your app)

Q programming language tutorial

Basic interpreter actions

Installing: put following files to c:\tools\kdb: q.k, w32\q.exe

setx QHOME c:\tools\kdb

Then run q.exe.



q)exit 0

Switching between q and K languages:


Load script:

q)\l script.q

Start as server:

q -p 5001

Connect to server and assign a connection to variable hdl:

q)hdl: hopen `:localhost:5001

Communicate from client to server (synchronous message):

q)hdl "put command here"

Closing connection:

q)hclose hdl

Language basics


q)5 / this is comment. Before was an expression, immediatelly evaluated. Integer atom evaluates to itself
q)`symbol / symbol atom

Simple arithmetic:

q)3%4 / division


q)x:5 / define x
q)x / get value of variable
q)x+x / use arithmetic with variable

Simple functions:

q){x*x}[4] /square function, simple expression,x is an implicit parameter, applied to atom 4.
q){x*x} 4 /if applied to one parameter, square brackets can be omitted
q)f:{[a;b]b+a*a} /function with explicit parameters
q)f [4;5] /apply to parameters. This is full syntax which must be used if params more than 1
q)f /will display function definition
q)+ /built-in functions will be printed like this, it's not a symbol!
q)+[4;5] /can apply 2-arg functions in this way
q)g:{[a;b]c:a*a;c+b} /statements in function are separated by semicolon ;
q)g [4;5]
q)k:{7} /no-arg function, returns a constant
q)k[] /application
q)g[4][5] /partial application, or currying: first apply g to 4, then apply result to 5
q)p:{x*x} /just one-arg function
q)p 4
q)p@4 /@ is a two-arg 'apply' function which applies one-arg function to arg
q)@[p;4] /can to be called this way
q)`p 4 /you can address a function via symbol (only in application!)
q)u: `p /variable u holds symbol `p
q)u 4 /this will apply 4 to p, since u references p via symbol


q)x: 1 2 3 / define a list of items of same type using simplified syntax
1 2 3
q)b: (9 9 9) /more verbose syntax
9 9 9
q)v:(9;1;8;2;7;3;6;4;5) /most verbose syntax
9 1 8 2 7 3 6 4 5
q)n:() /empty list
q)m:(7) /attempt to create a single-item list will not work, brackets are ignored and atom is evaluated
q)enlist 7 /create a single-list element. Displayed starting with comma to distinguish from atoms
q)z: 7 6 5 4 3 2 1
q)z[0] /access by integer index. Result is atom. Syntax is like function application
q)z[5 4 1] /access by list of indexes. Result is list
2 3 6
q)z[x] /works this way too
6 5 4
q)z 0 /also works without brackets
q)z 5 4 1 /and this way too
2 3 6
q)z x /and this way too
6 5 4
q)z[] /empty indexing returns whole list
7 6 5 4 3 2 1
q)z[20] /indexing outside of list bounds returns empty value for list item type
q)z[4]:9 /assignment of list item, can't assign to non-existing index
7 6 5 4 9 2 1
7 6 5 4 3 2 1
q)z@1 / another way of indexing, since lists are like functions

q)"mystring" 0 1 /strings are just list of chars
q)y:4 4 4

q)5#z /take first 5 elements, # is a two-arg function, you can use spaces around it
7 6 5 4 3
q)-4#z /take last 2 elements
4 3 2 1
q)8#x /if list is shorter than left arg, then items will repeat
1 2 3 1 2 3 1 2
q)2 _ z /drop first 2 elements, _ is a two-arg function
5 4 3 2 1

q)x,y /joining lists, also works in joining list to atom
1 2 3 4 4 4
q)count 4 5 6 7 /function, returns length of list
q)count x
q)count y
q)#:[1 2 3 4 5] / in fact, function # is overloaded and one-arg is count. Use : to explicitly invoke one-arg
q)z ? 2 /search value in list from left, returns index of first found value
q)y ? 4 /will return 0, since it is an index of first item in list
q)z ? 12 /if value is not found, then returns index of element beyound last (it is equal to size of list)

q)x+5 / add an atom to each element in list
6 7 8
q)x+y /add corresponding elements of lists, works only on lists of same length
5 6 7
q)z = 4 /returns a binary value with each bit in 1 if element equals to atom
q)(x+1) = y /also works on lists of the same size, compares corresponding elements
q)where 2 3 0 1 /where works for lists of integers. It repeats value of index as many times as value of corresponding list item
0 0 1 1 1 3
q)&amp;:[2 3 0 1] /where is an alias for one-arg version of &amp;. Use : to explicitly invoke one-arg
0 0 1 1 1 3
q)where 0 0 1 0 1 1 0 /interesting fact: applied to list of booleans (0 or 1), where will return indicies of 1's
2 4 5
q)where z&gt;4 /so, where could be used as filter: returns list of indexes which match the condition
0 1 2
q)z where z&gt;4 /you can apply a list of indexes back to list to get values
7 6 5

q)mixed:("string";1;`mysymbol;1 2 3) /mixed list, must be enclosed in () and separated with ;
q)mixed[0] /indexing works in a same way
1 2 3
q)mixed[0 3;2] /semicolon allows to use indexing in depth, for example: take items 0 and 3, and from each take sub-item 2

q)p each x /apply function p to each value in list x and return a list of results
1 4 9

Functions and Lists:

q)count 4 5 6 7 /application syntax: count is one-arg function applied to single arg of list type
q)count[4 5 6 7] /the same, using explicit application syntax
q)f 4 5 /apply single argument of type list to two-arg function, it's a partial application
{[a;b]b+a*a}[4 5]
q)f[4 5] /the same
{[a;b]b+a*a}[4 5]
q)f[4 5;1 3] /explicit syntax is used to apply two args, each is a list, both have same size
17 28
q)p @ 4 /apply function to single arg of type atom
q)p @ 4 5 /apply function to single arg of type list
16 25
q)f @ 4 5 /since it's like f[4 5], it's a partial application
{[a;b]b+a*a}[4 5]
q)f . 4 5 /curry-style application: function . accepts a single arg of type list, and puts elements into args


q)monthdays: `Jan`Feb`Mar`Apr`May ! 31 28 31 30 31 / composed from two lists of same length using !
Jan| 31
Feb| 28
Mar| 31
Apr| 30
May| 31

q)monthdays[`Jan] / obtain value by key using function application syntax
q)monthdays[`Jan`Mar`May] /obtain list of values from lists of keys
31 31 31

q)key monthdays / function key returns list of keys
q)value monthdays / function value returns list of values
31 28 31 30 31
q)count monthdays
q)monthdays ? 31 / search value, returns key. Searches from left to right

q)monthdays[`Jan]:1  / assignment syntax allows to map existing key to new value
Jan| 1
Feb| 28
Mar| 31
Apr| 30
May| 31
q)monthdays[`Jan]:31 / change back
Jan| 31
Feb| 28
Mar| 31
Apr| 30
May| 31
q)monthdays[`Jun]:30 / it is possible to add new key-&gt;value to dictionary
Jan| 31
Feb| 28
Mar| 31
Apr| 30
May| 31
Jun| 30

q)monthdays - 1 / built-in functions for atoms are overloaded to work over dictionary values
Jan| 30
Feb| 27
Mar| 30
Apr| 29
May| 30

q)monthdays = 31 / compares each value with constant, results in dict of booleans
Jan| 1
Feb| 0
Mar| 1
Apr| 0
May| 1

q)where `A`B`C!3 4 2 / where works with dicts of integer values. It makes a list, repeating each key as many times as corresponding value
q)where `A`B`C`D! 0 1 0 1 / interesting fact: if applied to dict with boolean values, returns only keys which values are 1
q)where monthdays = 31 / so, where could be used as filter: return list of keys for which condition of value holds
q)monthdays where monthdays = 31 /we can apply those keys back to dictionary to obtain expected result
31 31 31

q)2#monthdays / take first 2 key-value pairs as dictionary (sub-dictionary)
Jan| 31
Feb| 28
q)`Jan`Feb`Mar#monthdays / take function with list of keys returns sub-dictionary for provided keys
Jan| 31
Feb| 28
Mar| 31
q)`Feb`Mar _ monthdays /drop function will accept a list of keys, and will return dictionary without these keys
Jan| 31
Apr| 30
May| 31
Jun| 30


q)dictX: `month`days!(`Jan`Feb`Mar`Apr`May;31 28 31 30 31) / this is called a column dictionary: dictionary which values are lists of same length
month| Jan Feb Mar Apr May
days | 31 28 31 30 31

q)tableX: flip dictX / this is a table. Table is a list of dictionaries with same keys for all list elements
month days
Jan 31
Feb 28
Mar 31
Apr 30
May 31

month days
Jan 31
Feb 28
Mar 31
Apr 30
May 31

q)tableX[1] / get table row by index, returns dictionary
month| `Feb
days | 28
q)tableX[1][`month] / you can first get row by index, then obtain values by column names
q)3#tableX / take function returns table containing first 3 rows
month days
Jan 31
Feb 28
Mar 31

q)select from tableX where month = `Mar
month days
Mar 31
q)select days from tableX where month = `Mar
q)select days from tableX
q)`tableX insert (`Jun;30) / adding rows to table, accepts untyped list, returns list of inserted row numbers. Table must be referenced by symbol
month days
Jan 31
Feb 28
Mar 31
Apr 30
May 31
Jun 30

q)meta tableX / shows meta info about table
c | t f a
-----| -----
month| s
days | j


Spring vs Dagger

Spring contains configurable factory classes called contexts. These contexts can read description (specified in XML, for example) about
creating objects and satisfying dependecies. Finally, your app can obtain instances of objects from context and start using

Dagger requires you to describe if class requires depdendency to be injected via @Inject annotation (right, that’s intruzive).
If dependency is concrete class, then it may be created in default way, and no more work required for it. Otherwise, if you
have to inject an instance of interface, or you need to create injected class in some specific way, then you use special
factory class and annotate it with @Module annotation and annotate each factory method with @Provides annotation. Finally, you
can generate a factory class for any classes which need injected dependency. This class will have factory methods identified
by return types, and they need all necessary @Modules to be supplied.


Spring Dagger
Classes which don’t have dependencies, but on which other classes depend Describe them in XML, init them via values If they require some specific way to create them, then you need a @Provides factory method in some @Module class. Otherwize they will be created automatically
Classes which have dependencies, and on which other classes depend Describe them in XML, inject them with references to other beans If they require some specific way to create them, then you need a @Provides factory method in some @Module class. Otherwize they will be created automatically. In both cases their dependencies will be satisfied
Classes which have dependencies, but which are not needed to be injected See previous These classes will be needed externally, so you need to generate factory methods for them
A way to make it work Load description, call refresh(), obtain objects which you need. All beans will be created and injected in each other in a way which you describe Invoke generated factory methods for objects which you need and supply specific @Module factories to customize the result

Essentials of Java web stack

Jetty 8

1. start.jar is a launcher. Use –help command-line key to list available command-line options.
2. Launcher can start server in the same process (by default), or, if –exec command-line arg is present, as a separate process (if, for example, you need different JVM settings)
3. Launcher is controlled via command-line keys, and by start.ini file from current dir. There is a special command-line key –ini allowing to either specify a different ini file, or to specify a configuration content which is expected from such file
4. Server is customizable. You can specify which modules will be loaded. These are called OPTIONS, and are read from OPTIONS section of start.ini (or specified in –ini command line arg). Example:

java -jar start.jar --ini OPTIONS=Server,websocket etc\jetty.xml

5. To get a list of available modules use –list-options command-line key. It will show which jar files are used by each module.
6. Information produced by –list-options is calculated based on start.config script file which is packed inside start.jar. You can take a look by using –list-config command line key to understand module dependencies. You can override this config file by using –config command line key to completelly change server’s behaviour, but this is rarely needed.
7. There are some modules which just aggregate several other modules. Very good aggregate module is Server, which includes HTTP server, servlet container and deployer
8. Modules may be explicitly configured by XML config scripts which are usually stored in etc/ dir. Application of such configs specified in start.ini
9. To configure Jetty logging, customize existing configuration script etc/jetty-logging.xml, and don’t forget to add it to start.ini
10. Web applications are deployed to webapps dir
11. If you’ve included “resources” module, then directory resources/ is added to classpath (by default start.config script). You can put config files for your application there
12. One of usages of resouces/ dir is for applications which use log4j. If you put file in resources/ dir, then log4j will discover that file and will configure itself based on that file

Jetty 9

1. start.jar is a launcher. Use –help command-line key to list available command-line options
2. Launcher can start server in the same process (by default), or, if –exec command-line arg is present, as a separate process (if, for example, you need different JVM settings)
2. Launcher is controlled via command-line keys, and by start.ini file from current dir. File start.ini is just a set of same keys as for command-line
3. Server is customizable. You can specify which modules will be loaded via –module key. Example:

java -jar start.jar --module=http,deploy

4. To get a list of available modules use –list-modules command-line key. It will show which jar files are used by each module, and will show module dependencies.
5. Information provided by –list-modules is calculated based on module descriptors stored in modules/ directory as .mod files.
6. Each module has a corresponding config in etc/ directory. To check which config corresponds to which module you can use output of –list-modules (property XML) or by looking into .mod file. Configs are implicitly executed if corresponding module is included.
7. Minimal set of modules for working servlet container with auto-deployment support is server,http,deploy.
8. Web applications are deployed to webapps dir (will work in deploy module is enabled)
9. Recommended way to start Jetty 9 is to create a separate base directory for each running instance, which will contain configs, webapps and logs for this instance. This is called jetty.base
10. When running Jetty 9 this instance base dir should be current dir, and path to start.jar should be fully specified:

java -jar path/to/jetty/installation/start.jar

11. You can manually place start.ini into instance base dir, and create sub-dirs log, webapps. But there is a better way: from empty instance base dir run start.jar via full path and use command-line key –add-to-start. Example:

java -jar path/to/jetty/installation/start.jar --add-to-start=http,server,deploy

This will create config files from templates, and all necessary dirs.
12. If you’ve included “resources” module, then directory resources/ is added to classpath. You can put config files for your application there

Difference between Jetty 9 and Jetty 8

1. In Jetty 9, a start.ini file is just a set of command-line keys, so you can live without it. In Jetty 8 value of OPTIONS could be specified only in ini file, so you could live without it if you have specified –ini command-line key had to override this
2. Instead of defining OPTIONS in single config script, each module has it’s own descriptor.

Servlets basics

1. Web applications are packed into .war files (similar to .jar files). Applications contain static files and dynamic logic.
2. Jetty deployer maps content of web app to URL path corresponding to name of .war file. For example, application example.war to URL http://host:80/example/ . It’s called servet URL context path.
2. All dynamic logic should be placed in WEB-INF/ dir inside a .war file. In this dir there should be a classes/ sub-dir for .class files, lib/ sub-dir for .jar files, and web.xml descriptor of web app
3. Descriptor file web.xml describes servlets (they have name and main class) and mappings from url patterns to servlets.
4. Mappings from URL patterns to servlets is done relative from web application URL context path. For example, if in web.xml servlet mainServlet is mapped to URL pattern /main for web application superapp.war,
then such servlet will handle HTTP requests for URL http://host:80/superapp/main
5. Jetty comes with several built-in servlets: DefaultServlet which serves static files, and JSP servlet which handles .jsp files. You can use them in your application.
6. By default, these servlets are already used in your application. This happens because for module “deploy” it’s config script “etc/jetty-deploy.xml” defines property “jetty.deploy.defaultsDescriptorPath” with value “etc/webdefault.xml”.
This webdefault.xml is pre-pended to your web.xml. To switch it off, override property in your config files, for example in start.ini
7. in etc/webdefault.xml servlet DefaultServlet is mapped to “/”
7. Use Maven with war packaging, maven-war-plugin at build stage, maven profiles to specify which content should be included in .war file

Spring MVC

1. Spring comes with servlet called org.springframework.web.servlet.DispatcherServlet. You can have different servlets of this class under several names
2. This servlet will create Spring application content based on XML file. It will search for Spring context descriptor XML file, depending on servlet name, with pattern -context.xml
3. Like with any Spring XML context, you can either specify beans, including controller beans, directly via tags, or via  and
3. To enable MVC annotations like @Controller etc use
4. If you have mapped DispatcherServlet to “/”, but still want static files to be served, use in your context XML following tag: . It will make static requests to be forwarded
from DispatcherServlet to container’s DefaultServlet (if it is included in your web.xml either explicitly or implicitly by Jetty, see above)
5. If your Spring context supports several profiles, then you can specify which Spring profiles are active using web.xml to set context param “”
6. Log examples to see how your RequestHandlerMappings work:

INFO RequestMappingHandlerMapping:534 - Mapped "{[/rest/query],methods=[GET]}" onto public org.springframework.http.ResponseEntity&amp;amp;amp;amp;amp;amp;lt;java.lang.String&amp;amp;amp;amp;amp;amp;gt;
INFO RequestMappingHandlerMapping:534 - Mapped "{[/proxy/**]}" onto public org.springframework.http.ResponseEntity&amp;amp;amp;amp;amp;amp;lt;java.lang.String&amp;amp;amp;amp;amp;amp;gt; com.GenericPurposeProxyController.processRequest(jav

For default servlet handler:

INFO SimpleUrlHandlerMapping:341 - Mapped URL path [/**] onto handler 'org.springframework.web.servlet.resource.DefaultServletHttpRequestHandler#0'


Building OpenJDK 8 for Windows using MSYS

This article will describe how to build OpenJDK 8 on Windows using MSYS. Since the building itself is performed by build scripts, we will focus on two things: installation of necessary libraries and compilers, and fixing build scripts, since they don’t work out of the box. As most of my articles, this one is written for my future self, because I’m sure I’ll get back to this task in future, and I don’t like solving problems I know I’ve solved before and I don’t remember how. Also this article is not a simple list of steps, I’ve tried to explain the reasons behind the actions. Readme file of OpenJDK says that “building the source code for the OpenJDK requires a certain degree of technical expertise”, so let’s make ourselves this expertise by learning while doing.

Getting the source.

The first step is to get the source. OpenJDK developers use Mercurial version control system as a code storage. You can open this URL: in browser to see a list of projects hosted by OpenJDK. The project you need is jdk8. If you click on jdk8, you’ll see a list of repositories which jdk8 consists of. The top one is called jdk8, which makes a full URL: You may wonder why there are two jdk8 directories in the URL? This remained from some old times when there were so called “gate” repositories to which changes were pushed for integration, and once those changes were verified, they were merged into read-only main repositories. So, jdk8/jdk8 is a read-only repository. Gate repositories approach was abandoned, but for OpenJDK 8 the path to read-only repository remains the same. If you are curious, you can read more about OpenJDK Mercurial repositories here.

So, let’s get ourselves this respository. You will need Mercurial tools for this. I like GUI tools, so I’ve downloaded SmartGit/Hg. It took me a while to figure out why there are no Mercurial option when you try to clone a remote repository. To make this work, you need to download and install official Mercurial command-line tools, and then go to settings of SmartGit and point it to hg.exe tool. This will make Mercurial to appear in a list of VCSes. Thus, GUI tools are not a full replacement for command-line tools, they just make life a little easier. If you don’t like GUIs, you can skip them and use command-line Mercurial tools, that’s quite easy. So go ahead and clone a repository to some local directory.

Structure of OpenJDK build

The top repository jdk8/jdk8 contains only build infrastructure, it doesn’t contain any real source code, which is sorted into several additional other repositories. So, from this point we can either download those repositories, or we can do that later, when we will prepare everything. Let’s take a second approach, and start with preparing for a build. Take a look at a repository we just cloned. There are two readme files: a short text file and a bigger HTML file. Both are worth reading. Also there are two directories: common and make, and three scripts. The script named will download all remaining sources using Mercurial command-line tools, and we will postpone this until later. Two remaining scripts are the core of build process.

C is not a Java, there are many aspects of the language which are not defined and compiler-specific, like, for example, size of int value. So C programmers achieve portability by having special cases for compiler-dependent things. This is usually done “on source level”: a compiler-specific information is moved to dedicated header file. So to port a C program to another compiler requires changing compiler-dependent info and recompilation. To simplify this task programs used scripts which probe the compiler they are running on and generate a compiler-dependent header files. By convention these scripts are called configure. And OpenJDK has this script. We need to run it at least once. After that we have to use make tool to build everything, becase we have a script for it, called Makefile. Such two-stage comfigure/make approach is standard in Unix world for open-source software.

Let’s take a look at configure file. It is a unix shell script which prepares a build. It is very small, all it does is executing another configure script, located in common/autoconf. This second configure does a little more, like parsing command-line parameters, of which you can read more in readme.html. The main job is done by big script called So, in order to run these scripts we need some Unix-like environment on Windows. There are two options: Cygwin and MSYS. Both environments are quite similar: each provides a shared library (dll) which implements some set of POSIX functions on Windows, and a set of Unix tools compiled as Windows executables, which rely on that dll. Cygwin is bigger, provides a larger set of POSIX calls and includes more Unix tools, so it’s like a complete unix-like environment. MSYS (which means “minimal system”) supports a smaller set of POSIX calls and provides a set of Unix tools just enough to be able to run typical configure scripts. I like everything minimal, so I prefer MSYS.

Installing MSYS and dealing with configure.

MSYS itself is not an independent project, it is a part of another project called MinGW (Minimalist Gnu for Windows), which is a quite interesting story worth telling. Most of the application programs written in C use standard library, and there are many reasons for that. On Unix systems it’s a convenient and portable way to do system calls. Standard library also includes lots of useful functions, like string manipulation. Since standard library relies on OS services, the OS kernel itself cannot use standard library. Windows provides it’s own set of services for applications, called Win32 API, but their compiler suite provides a standard library for compatibility and convenience. Some standard libraries are tied to specific compilers, but there are independent libraries: newlib, uClibc, dietlibc, mucl. When choosing a standard library one has to consider its features, performance, size, support of particular OS/CPU, and also the licence. For example, using library released with GPL requires you to release your program under GPL. The licence terms may be different depending on how you link against a library. There are two options: static linking (library will be included into executable) and dynamic linking. Licensing terms for dynamic linking are usually less restrictive then for static linking. However, if you choose dynamic linking you should somehow ensure that library is installed on computers where your program will run. So, knowing all this we can now get to MingGW. It is a version of GCC compiler which produces Windows executables dynamically linked with standard library supplied with Microsoft Visual C v 6.0 (msvcrt.dll). The license allows any code to dynamically link against it, and practically this library is present in all Windows systems (used by Microsoft’s own applications), so you don’t need to distribute it yourself. Thus MinGW produces executables which can be released under any license and distributed in a very simple way. Technically MinGW consists of a set of header files for standard library, an import library for mscvrt.dll and a version of GCC which produces Windows executables linked with import library. Later some additional libraries were ported to MinGW and now are provided as a part of it. Also MinGW was extended with include files and import libraries for Windows API, so now you can use it to write native Windows software. MinGW made it easier to port software written in C from Unix to Windows, but that was not enough. Thus MSYS was born, it is an environment for running configure scripts.

OK, back to building OpenJDK. Go to MinGW site and download installer. Run it. It will show a list of packages you can install. You don’t actually need MinGW compilers, since they are not used by OpenJDK built, but I advice you to install them. You’ll definitely need make and autoconf. Also you’ll need basic MSYS, and several specific MSYS packages: bsd cpio, mktemp, zip, unzip.

Now, as you have installed MSYS, you can start it’s shell (bash). You can use your windows paths in a special way, for example “C:\projects\openjdk” should be used as “/c/projects/openjdk”. You can try to run configure script right away. At the beginning this script will check availability of required tools, so if you forgot to install abovementioned cpio, mktemp, zip and unzip, then configure will complain (that’s how I learned that I need them). So here we will encounter a first problem with OpenJDK build environment which requires manual intervention. The script will fail finding cpio.

Learning autoconf

The script will fail finding cpio, since it is called bsdcpio. If you’ll try to track the problem (either by looking at source code or by reading log file) you’ll get to a script To fix our problem, we need to modify this script. However, editing it directly is a wrong way. This script is generated (hence the name) by a tool called autoconf from sources located in OpenJDK folder common/autoconf. So, let’s get there and edit the sources. The actual change should be made in file basics.m4. Replace cpio with bsdcpio.

To generate new you should execute But attempt to do it will fail, will complain that it can’t find autoconf. The reason is simple: autoconf was installed into MinGW location which is not available for MSYS by default. So, you should go to MSYS installation directory and find “etc” directory (on my machine it is located at c:\tools\mingw\msys\1.0\etc). Here you should create a file called fstab which will configure mounting of windows directories to msys filesystem. Take a look at fstab.sample to see how to do it, you may even copy it as fstab and edit it. Your task is to map root MinGW folder as /mingw. To apply changes in fstab you should restart MSYS bash. There is another file in etc called profile, which configures bash. By default this profile will add /mingw/bin into search path. So, if you did everything right, the result of “which autoconf” should be something like “/mingw/bin/autoconf”. Now you can get back and use to generate build script. Do it. Oops, another error.

This time autogen will complain that autoconf 2.69 or higher is required. However, MinGW includes version 2.68. When I encountered this error I’ve decided to try with 2.68, and believe me, it works perfectly fine. So, let’s hack OpenJDK build scripts and fix the required version. It is specified in file Again execute This time it should work. Ignore output about no custom hook found.

We just fixed our first configure-stage error, and there will be more. To simplify troubleshooting, you should take a look at file called config.log, which contains output produced by conifugure script. If this log is not verbose enough, you can start the configure with command-line argument –debug-configure. It will make the script to produce additional log called debug-configure.log which is very verbose.

Installing bootstrap JDK.

Large part of JDK is written in Java,including the compiler. So building JDK requires you to have some bootstrap JDK. I’ve never got any problems installing it. You can even install it into default directory, and at any path, even the one which includes spaces.

Having fun with Microsoft Windows 7 SDK.

MinGW provides a C and C++ compilers for Windows, but the only officially supported by OpenJDK is Microsoft Visual C++ compiler, and we are going to use it. Otherwise configure will complain that it cannot find Visual Studio and quit. If you own Visual Studio, that’s great, and you can skip this part. However, in this article I’ll describe how to use minimalist development tools. So, we will use Microsoft Windows 7 SDK, which includes command-line C and C++ compilers from Visual Studio 2010. And it is free! You should download it from official site of Microsoft. There are web installer and several ISO images: for 32-bit systems, for Itanium and for 64-bit systems (amd-64). During the installation you can select which components to install, and I suggest to keep default settings, which include all necessary libraries and the compiler. If you will encounter some problems during the installation, check installation logs for exact description of the failure. I’ve got an error saying that SDK can’t install redistributable runtime libraries. Even de-selecting these libraries in a list of installed components doesn’t help. This happens because you already have a more recent version of those libraries installed (I had version 10.0.40219, and SDK will install 10.0.30319). It’s a shame for Microsoft to keep such bugs in installer. The only workaround is to uninstall your current redistributable of Microsoft Visual C runtime libraries, then install Windows SDK, and then download and install latest version of runtime library.

Now let’s check if compilers are working. If you will skip this part, you may get nasty errors much later. So, go to “c:\Program files (x86)\Microsoft Visual Studio 10.0\VC\bin\amd64” and launch cvtres.exe. If it has started successfully, that’s good. But on some systems it fails with application error. In fact you can skip this error, since it will not manifest at configure stage, but you’ll get strange error messages later on make stage, so let’s fix it now. Careful investigation with Dependency Walker tool shows that cvtres.exe imports a bunch of functions from msvcr100_clr0400.dll, and this dll doesn’t have any exported functions. Actually a version of this library included in SDK is OK, but some update for Microsoft .Net framework overwrites it with no-export version. Nice. In order to fix this, you need to download a patch from Microsoft called Microsoft Visual C++ 2010 Service Pack 1 Compiler Update for the Windows SDK 7.1. It will fix dependency problem for cvtres.exe, it will use another version of runtime dll. Download the update, install it and check that cvtres.exe works.

No, that’s not all. The update we just applied broke another thing. Unbelievable. I’ve created an empty file called ammintrin.h just to get around this annoying thing.

Patching the build scripts

Having Windows SDK will let you get further with configure, but eventually it will fail. That happens because scripts for building OpenJDK 8 using MSYS have errors. These errors were fixed in scripts for OpenJDK 9. The link to fixes could be found in this mail thread. Initial letter from Volker Simonis contains change request, and in subsequent messages Eric Joelsson extended it. Here is a list of changes:

  1. Fix for bsdcpio in basics.m4, which we have already applied
  2. Change in basics_windows.m4, which fixes AC_DEFUN([BASIC_FIXUP_EXECUTABLE_MSYS] problem with configure cannot find set_env.cmd file of Windows SDK
  3. Two fixes in toolchain_windows.m4: one for architecture type, and another with quotes for grep
  4. Fixes in platform.m4 for correct environment value. It’s a supplementary fix for other fixes to work.
  5. Fixes in NativeCompilation.gmk and will help if you’ll have an error during make. Without those fixes you’ll have to clean everything and re-make again from scratch, which takes a lot of time

So we should manually apply those fixes for OpenJDK 8. There are also change in, but you don’t need to apply it. Instead, generate it via autogen.


OpenJDK requires FreeType library. You can build it yourself from sources, I’ve downloaded a pre-built version. However, this pre-build version was strange: it included import library freetype.lib with all functions prefixed with underscore (“_”). To fix this, I’ve created an import library manually from dll using lib tool included in Microsoft Visual C command-line compiler suite (lib.exe /def:freetype6.def). This will produce a file freetype6.lib, which you should rename to freetype.lib, overwriting existing file (I’ve made a backup copy of it called _freetype.lib). You also need to copy freetype6.dll from bin directory in to lib directory and rename it to freetype.dll. And, finally, you need to explicitly specify path to the location where you’ve installed FreeType. A corresponding command-line argument for configure script is called –with-freetype.

Completing the configure

If you’ve done everything right, the configure step will successfully finish. The result will be stored in build directory of OpenJDK. The main item here is specs.gmk. Now you should download modules with source code.


Launch make all. If make hangs edit specs.gmk and set JOBS=1. As a result you’ll get directory called j2sdk-image, that’s your JDK!

Types and programming languages

Typed program consists of expressions and types. Types are assigned to (or better say “declared for”) expressions by programmer. Programming language has means of declaring basic expressions and constructing complex expressions from parts. Each facility for constructing expressions has a corresponding way of constructing a type of compound expression. Sometimes an attempt to construct a type will produce an error, saying that types of subexpressions are incompatible this way. For example, a facility of function application will produce a type error if first argument is not a function. If a constructed type doesn’t match the type declared by programmer, it is another case of typing error. This is a one side of a types: they are abstract and they provide a way to check a structure of a program.
There is another side of types: they are sets of values. Of course, each value can belong to any number of sets. For example, 5 is a natural, integer and real. So, by looking at the value you cannot say it’s type, since it has many. However, values constitute a special kind of expressions: a trivial expression. It is called trivial because it is trivial to evaluate. So, the phrase “value ‘5’ has a type ‘Integer'” should mean that “I’m talking about trivial expression ‘5’ with type ‘Integer'”. This side of types helps us to predict the set of all possible results of our programs, and also helps runtime system to allocate enough space to store any possible value belonging to the set.


Had I ever designed any domain-specific languages? Sure, many times. Like many people, I did it accidentially. As soon as I notice a function which uses one of its arguments only to dispatch control flow, something like this:

process(action, data) {
 if (action == Open) {
 } else if (action == Close) {
 } else if (action == New) {

I know I’ve just got an interpreter. And action+data is a DSL. I don’t like dispatch code, because every branch means analysis complexity: you either need to keep a value of action in your head or track back to find out what is the value. I think that having an interpreter means that you are exposing a narrow generic interface. User of your API doesn’t have any clues on usage, and he cannot rely on compiler to check for errors. That’s why I consider interpreters as a code smell and avoid them. Good API is better than DSL.

The last difference between OpenJDK and Oracle JDK

Recently I’ve spent a lot of time investigating font rasterization (a great topic which deserves a separate post). Most applications use font engine which is built into graphics library or widget toolkit. Only few cross-platform applications which badly need to provide consistent text layout (Acrobat Reader, for example) are using their own font engines (like Adobe CoolType). Java platform is one of such applications, since it has its own graphics library. If you are curious take a look at this article comparing font engines, including one from Java platform. From publically available information I understood that OpenJDK uses FreeType library. I thought: “That’s great, I have JDK 1.7 installed so this library must be there, let’s take a look”. But I could not find any traces of freetype.dll in JDK. I was puzzled and tried to find some answers in sources of OpenJDK. Imagine my surprize then I’ve found that Oracle JDK still uses proprietary T2K font library (located in jre/bin/t2k.dll)! Both Oracle JDK and OpenJDK are built from the same sources, and linking to external libraries happens in runtime. There is a logic which checks if JDK is running in “OpenJDK” mode or “Oracle JDK” mode, and, depending on that, loads either FreeType or T2K (see I thought: “I was always curious about remaining differences between OpenJDK and Oracle JDK, and eventually I’ve found one!”. An interesting thing is how JDK determines in runtime if it is OpenJDK or Oracle JDK. It checks if there is a file for Lucida font in JRE. A long time ago Sun had lots of complains that “write once run anywhere” promise doesn’t actually hold, especially for look and feel. Different systems had different fonts, and sometimes even fonts with same names had different glyph sizes, leading to inconsistent text layout. To fix this Sun have licensed Lucida font for distribution with JRE and made this font default. This font is absent from OpenJDK distribution, and, as I said, this fact causes JDK to link in runtime with FreeType, and to link with T2K otherwise. I was surprized, since I’ve expected a global configuration flag, something like “isOpenJDK”. I thought: “OK, let’s take a look what other specific hacks other sub-systems use to distinguish between OpenJDK and Oracle JDK”. It turned out that there are no other parts where this difference matters. Font subsystem is a last one. So I was a lucky guy, getting directly into it.

More compact format for Java classes

I’ve noticed that compiled Java code takes more place then source code. For example, a simple equation solver written by me takes 15 kbytes in source and 30 kbytes in compiled form. Source code is archived to 3 kbytes, and compiled code  to 17 kbytes. Since I have written a library for reading and manipulating .class files, I’ve decided to find out why these files are so big.

For equation solver I have following results: contstant pool takes 61% of total space, and methods take 32%. So, majority of space is taken by constants. Maybe this could be optimized? There are several types of constants which contain references to other constants. These are MethodReference, ClassReference, StringReference, FieldReference, NameTypePair. Each reference takes two bytes. My program is composed of small classes, with typical constant pools consisting of 50-80 entries. So, for such classes it is possible to store references using only one byte. I’ve invented a new file format, named “slass”, which uses such technique for saving space, and implemented it in my library. Testing with abovementioned equation solver gives 7% improvement on size of constant pool, which makes a 4% improvement on size of .class file! No, it was not worth inventing.