How does blacklisting work in Java and how to take advantage of it (using whitelisting)

Florence Broderick    23 September, 2013
Oracle has introduced the notion of whitelisting in its latest version of Java 7 update 40. That is a great step ahead (taken too late) in security for this platform but… how does it work? how does it deal with older versions? And what is most important… how to block everything but the applets you want?

This is the first time in years that Java allows to whitelist applets. This has become a true necessity for security, because of quite aggressive kits exploiting everything related with Java and its self “natural born insecurity”. Before this version Java was the one blacklisting some applets, but it was managed by Oracle only, updated in each new version, not dynamic and very badly documented. But now, at last with Java 7u40 we have the chance to whitelist applets. It is not trivial though. But it is not trivial though.

What you will need is a ruleset.xml file, compile it and sign it. For signing it you may use a real certificate or a selfsigned certificate created by yourself but installed in your trust-store.

Step by step. Creating the ruleset.xml

This is a standard XML file with a simple syntax. It defines which applets to block or allow depending on the domain they come from or who signed them. It also defines which version of Java has to be used to run the applet. Wildcards and rules by default are accepted, doing it may be quite granular. Let’s create a file that allows only applets working with the ones hosted in, and deny other applets.

<ruleset version=”1.0+”>
  <id location=”http://*” />
  <action permission=”run” version=”SECURE-1.7″ />
<id />
  <action permission=”block”>
  <message>Bloqueado por las reglas del sistema</message>

Last “id” means this is the “by default” rule and matches everything not matched before. The “version” tag may be handy… or tricky. It allows you to specify that an applet will run only with a desired (older) version that will, by definition, have security problems. So if the computer is keeping older versions (6.x) and an applet uses it… be aware this rule doesn’t work for branch 6 (nothing will be blocked). So, if keeping this branch in the computer, this may all be useless.
Ruleset allows to execute, for example, only applets signed by a certificate and much more. Specifications are here.)

Step by step. Creating the jar and signing it

Download JDK and execute:

C:Archivos de programaJavajdk1.7.0_40bin>jar -cvf DeploymentRuleSet.jar rulset.xml

Then, sign it:

C:Archivos de programaJavajdk1.7.0_40bin>jarsigner -verbose -keystore keystore.jks -signedjar DeploymentRuleSet.jar DeploymentRuleSet.jar selfsigned

Where “keystore.jks” may be your actual key store and “selfsigned” the alias of your certificate. If you already have a valid certificate (signed by a CA), skip the following part. If not, create a self signed one with the command:

keytool -genkey -keyalg RSA -alias selfsigned -keystore keystore.jks -storepass 123456 -validity 360 -keysize 2048

Where “123456” is the password to unlock the keystore (do not confuse it with the password of the certificate itself).

It will ask some questions. It does not matter how you answer them.

Finally, export the certificate:

keytool -export -alias selfsigned selfsigned.crt -keystore keystore.jks

And import it as a trusted root. You may do it in Windows (installing it as a trusted root in certmgr.msc) or inside Java certificate store. This is the way to do it:

C:Archivos de programaJavajdk1.7.0_40bin>keytool -importcert -alias selfsign
ed -file self.crt -trustcacerts -keystore ….jre7libsecuritycacerts
Now  DeplymentRuleSet.jar is signed, copy it to its place (funny how Oracle still keeps some “Sun” names, four years later).

C:Archivos de programaJavajdk1.7.0_40bin>copy DeploymentRuleSet.jar c:WINDOWSSunJavaDeploymentDeploymentRuleSet.jar

Execute javacpl.cpl and check Java is aware of the rules.

Checking it all

So, if you check applets hosted inside the specified domain, they will run but any other will be blocked. This is a great and very expected security measure if you do not have to deal with older versions.

Do not forget:
  • This is useless if other Java security measurements are not deployed. For example rise up the security leverage to “high” in security options. So far, with this leverage we could be “protected” against self-signed java malware, but what about properly signed ones? This feature tries to cover that gap.
  • Very important: whitelisting an applet by any kind of rule makes other warning screens introduced in java 7u10 go away, like this one (it will not show up if whitelisted):
  • This is useless if Java is not updated and older versions deleted in the system.
Just to check, using a not trusted certificate for DeploymentRuleSet.jar will block applets with a different message.

It’s important to notice that Oracle has warned that they will blacklist certificates that sign DeploymentRuleSet.jar files that allow to execute everything.

Anyhow, Oracle had to keep backwards compatibility with 1.6 branch, and when they drop support for it, this is the best way they have found to help administrators with some native tool to control Java plugin madness. Not bad.

Sergio de los Santos


Leave a Reply

Your email address will not be published. Required fields are marked *