The Java sandbox security model has been around since more than a decade, and with some enhancements over the years it remains pretty much the same since JDK 1.2., and is described succinctly by the following diagram:

Java Security Sandbox Model

In a nutshell, the security sandbox model has been introduced to provide a restricted environment for source code loaded from Java applets running on the customer’s environment (the browser). The untrusted code can access only the limited resources provided inside the sandbox, and is prevented access to vital system resources such as the file system unless explicitly allowed to do so.

In time, the Java sandbox security model expanded to pretty much any Java managed environment capable to load and execute custom application (such as OSGi, Java EE or even Java stored procedures running in Oracle RSBMS).

Every time the application triggers a permission check, either an instance of the java.lang.SecurityManager class installed in the JVM runtime or the static methods ot the java.security.AccessController utility need to be used. In fact the java.lang.SecurityManager delegates its operations to the java.security.AccessController. To describe the model briefly consider the following flow:

  • The Java application is being loaded by the JVM and, during classloading, for each class of the application a java.security.ProtectionDomain instance is set. The instance contains the code source (location of where the class was loaded from) and a set of Java permissions that apply to that class based on its location (as specified in the security.policy file);

  • During execution the checkPermission method of the installed security manager (or AccessController utility) is called to check if the calling code has the permission to perform the requested action identified by a particular Java permission class (instance of the java.security.Permission). The permission check is done by traversing the current call stack, and checking whether the protection domain of each class in the call stack has the requested permission (passed as a parameter to the checkPermission call). If there is even a single class that does have the permissions there is a security exception raised;

  • Sometimes there is a need to escalate privileges of a block of code, typically to allow temporary access to an operation that is forbidden to callers by default. That can be achieved by calling one of the doPrivileged methods.

In addition to the security sandbox model, the JDK provides a number of security utilities to developers such as:

  • JCA (Java Cryptography Architecture). Provides a number of utilities for performing cryptographic operations such as creating digital signatures and message digests, using symmetric/asymmetric block/stream cyphers, and other types of cryptographic services and algorithms. The JCA is provider-based, not bound to a particular set of algorithm implementations. There is a default implementation of the provider API provided by the JDK called SunJCA. Another widely used implementation is provided by the BouncyCastle library.

  • PKI (Public Key Infrastructure) utilities. These are utilities for working with digital certificates, certificate revocation lists (CRLs) or the OCSP (Online Certificate Status Protocol). The last two provide mechanisms for checking certificate revocation status as provided by the certificate authority. In that category we also find the operations used to deal with key and trust stores in different formats (such as JKS and PKCS12);

  • JSSE (Java Secure Socket Extension). This is JDK’s implementation of the TLS (formerly SSL) and, as of recently, also DTLS series of protocols. In fact the JSSE is the one with the most improvements as of latest versions of the JDK, as we shall see in the next sections;

  • Java GSS API (Java Generic Security Services). The GSS API can be considering an alternative to the JSSE API and it uses token-based communication for encryption of traffic;

  • Java SASL API (Java Simple Authentication and Security Layer). The framework provides a generic mechanism to establish the authentication channel between a client and a server.

All of these have been around for quite some time and enhanced throughout the JDK versions.

Moreover, developers still tend to choose a number of third-party security libraries and frameworks that typically provide extra security capabilities missing from the JDK.

For example the BouncyCastle library provides enhanced cryptography algorithms and utilities, and the JSch library provides SSH support for application (which is missing from the JDK).

JDK 9, 10 and 11 from a security perspective

When dealing with distributed systems, one of the first things that one can think of in terms of security is the necessity to establish secure communication channel between the distinct components of that system.

The de-facto standard for this purpose today is TLS, providing a number of improvements over its predecessor SSL:

  • SSL protocols have exposed vulnerabilities such as the notorious POODLE attack over SSL 3.0, which is also made possible in modern browsers due to automatic downgrade to SSL 3.0 (if not disabled) that is sometimes used for interoperability. Other vulnerabilities such as BEAST can even cross the boundary by exploiting not only SSL 3.0 but also TLS 1.0 (but not later versions);

  • TLS certificates are cryptographically stronger and hence more difficult to exploit than the SSL certificates;

Without diving into more details it is worth saying that the changes made in TLS break the interoperability with the earlier SSL protocol. Major enhancements have been introduced in the JSSE API in regard to the TLS support in the JDK. Let’s see how they fill in some of the gaps in the protocol’s capabilities:

  • The TLS protocol is working over TCP, meaning that it provides out of the box reliability of transfer thanks to retransmission of failed packets, error detection, flow and congestion control. But what about UDP protocols such as SIP (used by messaging applications) or DNS (used for name resolution)? DTLS comes to the rescue: the introduction of a datagram transport layer security protocol in the JDK enables applications to establish secure communication over an unreliable protocol such as UDP;

  • TLS is typically bound to a concrete application protocol: for HTTP we have HTTPS, for the FTP we have SFTP and so on. These require distinct ports for each distinct protocol running over TLS. Moreover, different versions of the same protocol have the same requirement. The Application Layer Protocol Negotiation (ALPN), enables client and server to negotiate the application protocol to use during the TLS handshake. Consider for example HTTPS/1 and HTTPS/2: a TLS client and server may negotiate which version of the HTTP protocol to use during the TLS handshake process;

  • When it comes to secure communication, we cannot ignore performance implications as well. Although latest hardware enhancements minimize the impact on applying TLS over an existing protocol, there are still some areas such as certificate revocation checking that imply increased network latency. OCSP stapling is an amelioration of the TLS protocol that allows for improved certificate revocation checks: eliminating the need for clients to contact the CA, it minimizes the number of requests to the certificate authority, improving both security and performance.

All of the above have been introduced in JDK 9 while in JDK 10 a default set of root certificate authorities have been provided to the trust store of the JDK for use by Java applications.

Moving further in time, the security highlight of JDK 11 is indisputably the implementation of major parts of the TLS 1.3 protocol in JSSE. Major benefits of TLS 1.3 are improved security and performance including the following enhancements introduced by the protocol (as highlighted by JEP322 upon which the implementation is based):

  • Enhanced protocol version negotiation using an extension field indicating the list of supported versions;

  • Improved full TLS handshake for both client and server sides (more compact than in TLS 1.3);

  • Improved session resumption using a PSK (Pre-Shared Key) extension;

  • The possibility to update cryptographic keys and corresponding initialization vectors (IV) after a Finish request is send in the TLS handshake;

  • Additional extensions and algorithms;

  • Two new cipher suites: TLS_AES_128_GCM_SHA256 and TLS_AES_256_GCM_SHA384;

  • RSASSA-PSS signature algorithms.

It is important to note that while TLS 1.3 is not backward compatible with previous versions of the protocol, the JSSE implementation provides a backward compatibility mode. In addition, both the synchronous (via the SSLSocket API) and asynchronous (via the SSLEngine API) modes of operation are updated to support TLS 1.3.

How to enable the TLS 1.3 protocol in JDK 11

Let’s build a simple demo of client-server pair that is using a custom protocol secured with the latest version of TLS. We can use JDK 11 and provide an implementation based on JSSE (already providing support for TLS1.3). The following example may be a good starting point:


// TLSv1.3 JSSE server
// -------------------------------

// setting the key store with the TLS server certificate
System.setProperty("javax.net.ssl.keyStore", "C:/sample.pfx");
System.setProperty("javax.net.ssl.keyStorePassword", "sample");

// getting a TLS socket factory for creating of server-side TLS sockets
SSLServerSocketFactory ssf = (SSLServerSocketFactory) SSLServerSocketFactory.getDefault();

// creating a server-side TLS socket on port 4444
SSLServerSocket ss = (SSLServerSocket) ssf.createServerSocket(4444);

// the essential part in enabling TLS 1.3 is setting the protocol version and supported cryptographic cypher, 
// typically one of new cypher combinations introduced by TLS 1.3
ss.setEnabledProtocols(new String[] {"TLSv1.3"});
ss.setEnabledCipherSuites(new String[] {"TLS_AES_128_GCM_SHA256"});

// following is standard blocking reading and then writing to the TLS server socket
while (true) {
  SSLSocket s = (SSLSocket) ss.accept();  
  BufferedReader in = new BufferedReader(new InputStreamReader(s.getInputStream()));
  String line = null;
  PrintStream out = new PrintStream(s.getOutputStream());
  while (((line = in.readLine()) != null)) {
    System.out.println(line);
    out.println("Hi, client");
  }
  ...
}


// TLSv1.3 JSSE client
// -------------------------------

// setting the trusted store with the set of certificate authorities trusted (by the TLS client)
System.setProperty("javax.net.ssl.trustStore", "C:/sample.pfx");
System.setProperty("javax.net.ssl.trustStorePassword", "sample");

// getting a TLS socket factory for creation of client-side TLS sockets
SSLSocketFactory ssf = (SSLSocketFactory) SSLSocketFactory.getDefault();

// creating a client-side TLS socket
String serverIP = "127.0.0.1";
if(args.length >= 1) {
    serverIP = args[0];
}
SSLSocket s = (SSLSocket) ssf.createSocket(serverIP, 4444);

// the essential part in enabling TLS 1.3 for the client
s.setEnabledProtocols(new String[] {"TLSv1.3"});
s.setEnabledCipherSuites(new String[] {"TLS_AES_128_GCM_SHA256"});

// standard writing and reading blocking from the TLS client socket
PrintWriter out = new PrintWriter(s.getOutputStream(), true);
out.println("Hi, server.");
BufferedReader in = new BufferedReader(new InputStreamReader(s.getInputStream()));
String x = in.readLine();
System.out.println(x);

As you can see, in order to enable TLS 1.3 in JDK 11, we simply need to: - specify the appropriate constant representing the TLS protocol version, on both the client and server SSL sockets - set the appropriate cypher suites that the TLS protocol expects, again on both the client and server SSL sockets

Enhancing the Security Sandbox Model

Now let’s assume that our TLS 1.3 server is implemented as a set of distinct Jigsaw modules (Jigsaw is an implementation of the Java Platform Module System specification that brings a new module system to the JDK ecosystem as of JDK 9). Moreover our server should provide a plug-in system that uses a distinct classloader to load different applications, providing integrations with third-party systems. This would imply that we need a form of access control for the applications managed by the server.

The Java security sandbox of JDK 11 comes to the rescue. It remains the same as of JDK 9, where Jigsaw modules bring subtle changes to the JDK so that the permission model can be applied to modules. This is achieved by simply introducing a new scheme (jrt) for referring to Jigsaw modules when specifying permissions in the security.policy file.

For example, if we enable the security manager in the JSSE server by passing the -Djava.security.manager JVM parameter we will get the following exception:

java.security.AccessControlException: access denied ("java.util.PropertyPermission" "javax.net.ssl.trustStore" "write")

This happens because we also need to put the proper permissions in a security.policy file (a default one is located under conf/security since JDK 9 but its use is discouraged) to run the JSSE server.

We may specify a dedicated policy file just for our server using the -Djava.security.policy parameter, pointing to the location of the custom security policy file.

If the codebase, the location from where we start the JSSE server, is the compilation directory (i.e. we run the JSSE server from the compiled Java class containing the snippet) we would end up with a security.policy file looking like this:

// including a few more required permissions
grant codeBase "file:/C:/project/target/" {
        permission java.util.PropertyPermission "javax.net.ssl.keyStore", "write";
        permission java.util.PropertyPermission "javax.net.ssl.keyStorePassword", "write";
        permission java.net.SocketPermission "localhost:4444", "listen,resolve";
};

If our JSSE server was packaged as a JDK module named “com.exoscale.jsse.server” we could have specified the above entry in the following format:

grant codeBase "jrt:/com.exoscale.jsse.server " {
        …
};

Deploying the sample application

You can try to deploy this simple example yourself. Assuming you have access to a provisioned Linux Ubuntu 18.04 LTS 64-bit instance, e.g. deployed using Exoscale CLI, you can ssh to the machine and first install Oracle JVM 11 using the following commands:

sudo apt-get update
sudo add-apt-repository ppa:linuxuprising/java
sudo apt-get install oracle-java11-installer

You then need to specify proper permissions in the security.policy file of the installed JDK. You can use the earlier grant clause by modifying the code source to point to the jsseserver.jar.

Finally you can then retrieve the bundled demo TLS server application from the repository associated to this article:

wget https://github.com/exoscale-labs/JDK11_security_article/raw/master/resources/jsseserver.jar –O jsseserver.jar
java -Djava.security.manager –jar jsseserver.jar

Once the server application is started you can establish the communication through it by creating a runnable JAR file out of the sample TLS client application (jsseclient.jar) and run it as follows:

java –jar jsseclient.jar 127.0.0.1

Here you need to provide the IP of the machine on which the TLS server application is started and make sure port 4444 is not firewalled.