Building an all-in-one Jar in Gradle with the Kotlin DSL

Andy Balaam from Andy Balaam's Blog

To build a “fat” Jar of your Java or Kotlin project that contains all the dependencies within a single file, you can use the shadow Gradle plugin.

I found it hard to find clear documentation on how it works using the Gradle Kotlin DSL (with a build.gradle.kts instead of build.gradle) so here is how I did it:

$ cat build.gradle.kts 
import com.github.jengelman.gradle.plugins.shadow.tasks.ShadowJar

plugins {
    kotlin("jvm") version "1.3.41"
    id("com.github.johnrengelman.shadow") version "5.1.0"
}

repositories {
    mavenCentral()
}

dependencies {
    implementation(kotlin("stdlib"))
}

tasks.withType<ShadowJar>() {
    manifest {
        attributes["Main-Class"] = "HelloKt"
    }
}

$ cat src/main/kotlin/Hello.kt 
fun main() {
    println("Hello!")
}

$ gradle wrapper --gradle-version 5.5
BUILD SUCCESSFUL in 0s
1 actionable task: 1 executed

$ ./gradlew shadowJar
BUILD SUCCESSFUL in 1s
2 actionable tasks: 2 executed

$ java -jar build/libs/hello-all.jar 
Hello!

Creating a self-signed certificate for Apache and connecting to it from Java

Andy Balaam from Andy Balaam&#039;s Blog

Our mission: to create a self-signed certificate for an Apache web server that allows us to connect to it over HTTPS (SSL/TLS) from a Java program.

The tricky bit for me was generating a certificate that contains Subject Alternative Names for my server, which is needed to connect to it from Java.

We will use the openssl command.

Creating a self-signed certificate for Apache HTTPD

First create a config file cert.conf:

[ req ]
distinguished_name  = subject
x509_extensions     = x509_ext
prompt = no

[ subject ]
commonName = Example Company

[ x509_ext ]
subjectAltName = @alternate_names

[ alternate_names ]
DNS.1 = example.com

In the above, replace “example.com” with the name you will use for the host when you connect from Java. This is important, because Java requires the name in the certificate to match the name it is using to connect to the server. If you’re connecting to it as localhost, just put “localhost”. Note: do not include “https://” or any port or path after the hostname, so “example.com:8080/mypath” is wrong – it should be just “example.com”.

The alternate_names section above gives the “Subject Alternative Names” for this certificate. You can add more as “DNS.2”, “DNS.3”, etc.

Next, generate the server key and self-signed certificate:

openssl genrsa 2048 > server.key
chmod 400 server.key
openssl req -new -x509 -config cert.conf -nodes -sha256 -days 365 -key server.key -out server.crt

Now you have two new files: server.key and server.crt. These are the files that will be used by Apache HTTPD, so put them somewhere useful (e.g. inside /usr/local/apache2/conf/) and refer to them in the Apache config file using keys “SSLCertificateKeyFile” and “SSLCertificateFile” respectively. For more info see the SSL/TLS How-To.

Checking the certificate is being used

Start up your Apache and ensure you can connect to it over HTTPS using curl:

curl -v --insecure https://example.com:8080

Replace “https://example.com:8080” above with the full URL (this time, include “https://” and the port and path.

To examine the certificate that is being returned, run:

openssl s_client -showcerts -connect example.com:8080

Replace “example.com:8080” above with hostname and port (no “https:// this time!).

Connecting from Java

To be able to connect from Java, we need a Trust Store. We can create one in PKCS#12 format with:

openssl pkcs12 -export -passout pass:000000 -out trust.pkcs12 -inkey server.key -in server.crt

Note: Java 8 onwards is able to use .pkcs12 (PKCS#12) files for its trust store. The old .jks (Java Key Store) format is deprecated.

Now you have a file we can use as a trust store, follow my other article to connect from Java over HTTPS with a self-signed certificate.

Build with a different Java version (e.g. 11) using Docker

Andy Balaam from Andy Balaam&#039;s Blog

To spin up a temporary environment with a different Java version without touching your real environment, try this Docker command:

docker run -i -t --mount "type=bind,src=$PWD,dst=/code" openjdk:11-jdk bash

(Change “11-jdk” to the version you want as listed on the README.)

Then you can build the code inside the current directory something like this:

cd code
./gradlew test

Or similar for other build tools, although you may need to install them first.

Scheduling a task in Java within a CompletableFuture

Andy Balaam from Andy Balaam&#039;s Blog

When we want to do something later in our Java code, we often turn to the ScheduledExecutorService. This class has a method called schedule(), and we can pass it some code to be run later like this:

    ScheduledExecutorService executor =
        Executors.newScheduledThreadPool(4);
    executor.schedule(
        () -> {System.out.println("..later");},
        1,
        TimeUnit.SECONDS
    );
    System.out.println("do...");
    // (Don't forget to shut down the executor later...)

The above code prints “do…” and then one second later it prints “…later”.

We can even write code that does some work and returns a result in a similar way:

    // (Make the executor as above.)
    ScheduledFuture future = executor.schedule(
        () -> 10 + 25, 1, TimeUnit.SECONDS);
    System.out.println("answer=" + future.get())

The above code prints “answer=35”. When we call get() it blocks waiting for the scheduler to run the task and mark the ScheduledFuture as complete, and then returns the answer to the sum (10 + 25) when it is ready.

This is all very well, but you may note that the Future returned from schedule() is a ScheduledFuture, and a ScheduledFuture is not a CompletableFuture.

Why do you care? Well, you might care if you want to do something after the scheduled task is completed. Of course, you can call get(), and block, and then do something, but if you want to react asynchronously without blocking, this won’t work.

The normal way to run some code after a Future has completed is to call one of the “then*” or “when*” methods on the Future, but these methods are only available on CompletableFuture, not ScheduledFuture.

Never fear, we have figured this out for you. We present a small wrapper for schedule that transforms your ScheduledFuture into a CompletableFuture. Here’s how to use it:

    CompletableFuture future =
        ScheduledCompletable.schedule(
            executor,
            () -> 10 + 25,
            1,
            TimeUnit.SECONDS
         );
    future.thenAccept(
        answer -> {System.out.println(answer);}
    );
    System.out.println("Answer coming...")

The above code prints “Answer coming…” and then “35”, so we can see that we don’t block the main thread waiting for the answer to come back.

So far, we have scheduled a synchronous task to run in the background after a delay, and wrapped its result in a CompletableFuture to allow us to chain more tasks after it.

In fact, what we often want to do is schedule a delayed task that is itself asynchronous, and already returns a CompletableFuture. In this case it feels particularly natural to get the result back as a CompletableFuture, but with the current ScheduledExecutorService interface we can’t easily do it.

It’s OK, we’ve figured that out too:

    Supplier> asyncTask = () ->
        CompletableFuture.completedFuture(10 + 25);
    CompletableFuture future =
        ScheduledCompletable.scheduleAsync(
            executor, asyncTask, 1, TimeUnit.SECONDS);
    future.thenAccept(
        answer -> {System.out.println(answer);}
    );
    System.out.println("Answer coming...")

The above code prints “Answer coming…” and then “35”, so we can see that our existing asynchronous task was scheduled in the background, and we didn’t have to block the main thread waiting for it. Also, under the hood, we are not blocking the ScheduledExecutorService‘s thread (from its pool) while the async task is running – that task just runs in whatever thread it was assigned when it was created. (Note: in our example we don’t really run an async task at all, but just immediately return a completed Future, but this does work for real async tasks.)

I know you’re wondering how we achieved all this. First, here’s how we run a simple blocking task in the background and wrap it in a CompletableFuture:

    public static  CompletableFuture schedule(
        ScheduledExecutorService executor,
        Supplier command,
        long delay,
        TimeUnit unit
    ) {
        CompletableFuture completableFuture = new CompletableFuture<>();
        executor.schedule(
            (() -> {
                try {
                    return completableFuture.complete(command.get());
                } catch (Throwable t) {
                    return completableFuture.completeExceptionally(t);
                }
            }),
            delay,
            unit
        );
        return completableFuture;
    }

And here’s how we delay execution of an async task but still return its result in a CompletableFuture:

    public static  CompletableFuture scheduleAsync(
        ScheduledExecutorService executor,
        Supplier> command,
        long delay,
        TimeUnit unit
    ) {
        CompletableFuture completableFuture = new CompletableFuture<>();
        executor.schedule(
            (() -> {
                command.get().thenAccept(
                    t -> {completableFuture.complete(t);}
                )
                .exceptionally(
                    t -> {completableFuture.completeExceptionally(t);return null;}
                );
            }),
            delay,
            unit
        );
        return completableFuture;
    }

Note that this should all work to run methods like exceptionally(), thenAccept(), whenComplete() etc.

Feedback and improvements welcome!

Gradle: what is a task, and how can I make a task depend on another task?

Andy Balaam from Andy Balaam&#039;s Blog

In an insane world, Gradle sometimes seems like the sanest choice for building a Java or Kotlin project. But what on Earth does all the stuff inside build.gradle actually mean? And when does my code run? And how do you make a task? And how do you persuade a task to depend on another task?

Setting up

To use Gradle, get hold of any version of it for long enough to create a local gradlew file, and the use that.
$ mkdir gradle-experiments
$ cd gradle-experiments
$ sudo apt install gradle  # Briefly install the system version of gradle
...
$ gradle wrapper --gradle-version=5.2.1
$ sudo apt remove gradle   # Optional - uninstalls the system version
$ ./gradlew tasks
... If all is good, this should ...
... print a list of available tasks. ...
It is normal for gradlew and the whole gradle directory it creates to be checked into source control. This means everyone who fetches the code from source control will have a predictable Gradle version.

What is build.gradle?

build.gradle is a Groovy program that Gradle runs within a context that it has set up for you. That context means that you are actually calling methods of a Project object, and modifying its properties. The fact that Groovy lets you miss out a lot of punctuation makes that harder to see, but it's true. The first thing to get your head around is that Gradle actually runs your code immediately, so if your build.gradle looks like this (and only this):
println("Hello")
when you run Gradle your code runs:
$ ./gradlew -q 
Hello
... more guff ...
So that code runs even if you don't ask Gradle to run a task containing that code. It runs at "configuration time" - i.e. when Gradle is understanding your build.gradle file. Actually, "understanding" it means executing it. Remember when I said this code runs in the context of a Project? What that means is that if you have something like this in your build.gradle:
repositories {
    jcenter()
}
what it really means is something like this:
project.repositories(
    {
        it.jcenter()
    }
)
You are calling the repositories method on the project object. The argument to the repositories method is a Groovy closure, which is a blob of code that will get run later. I've used the magic it name above to demonstrate that jcenter is just a method being called on the object that is the context for the closure when it is run. When does it run? Let's find out:
println("before")
project.repositories( {
    println("within")
    jcenter()
})
println("after")
$ ./gradlew -q
before
within
after
... more guff ...
This surprised me - it means the closure you pass in to repositories is actually run immediately, as part of running repositories, before execution gets to the line after that call. As we'll see later, some closures you create do not run immediately like this one. Once you know that build.gradle is actually modifying a Project object, you have starting point for understanding the Gradle reference documentation.

How do you make a task

You probably shouldn't do it very often, but it was instructive for me to understand how to make my own custom task. Here's an example:
tasks.register("mytask") {
    doLast {
        println("running mytask")
    }
}
This creates a new task by calling the register method on the tasks property of the Project object. Register takes two arguments: a name for the task ("mytask" here), and a closure with some code in it to run when we decide we need this task. That closure gets run in a context that can't see the Project object, but instead can see a Task object which it is helping to make. That Task object has a doLast method that we call, and passing it a closure that will be run when the task is actually executed (not immediately). If we remove some of the syntactic sugar the above build.gradle looks like this:
tasks.register(
    "mytask",
    {
        it.doLast {
            println("running mytask")
        }
    }
)
Above we can see that register really does take two arguments as I said above - the first version uses a Groovy feature where if you miss out the last argument and write a closure immediately afterwards the closure is passed as the last argument. Confusing, eh? Again, notice that doLast is a method on the Task object that is implicitly available when the closure is run. So we have created a task that we can run:
 ./gradlew -q mytask
running mytask

How do you make a task depend on another task?

If I want to run my code formatting before my compile (for example) I sometimes need to modify a task to make it depend on another one. This can done for tasks you create or for pre-existing ones. Here's an example:
plugins {
    id "java"
}
tasks.register("mytask") {
    doLast {
        println("running mytask")
    }
}
compileJava {
    dependsOn tasks.named("mytask")
}
So, calling the plugins on the Project at the top with a closure that ran the id method on something modified the Project so that it had a new method called compileJava which we called at the bottom, passing it a closure to run. That closure ran in the context of a Task object (similar to when we created a task, but now allow us to modify a pre-existing one). We called the dependsOn method of the Task object, passing in another Task object which we had got by calling the named method on the tasks object. [Side note: the register method actually returns a Task object that we could have passed to dependsOn without looking it up again using named, but Groovy doesn't provide a very convenient way of holding on to that reference, so we did do it. The Kotlin example below shows that this is quite simple in Kotlin.]

How do I do all this in Kotlin?

Because one DSL that hides what's really going on wasn't enough for you, Gradle now provides a second DSL that hides what's going on in subtly different ways, which is a program written in Kotlin instead of Groovy. This is marginally better, because Kotlin doesn't let you do quite so many stupid tricks as Groovy does. Below are all our examples in Kotlin. You get started exactly the same way, by following "Setting up" above. Remember to name your build file build.gradle.kts.

Say hello in Gradle Kotlin

println("Hello")
This is identical to the Groovy version.)

Use jcenter repo in Gradle Kotlin

repositories {
    jcenter()
}
This is identical to the Groovy version, and with the same meaning: repositories is a method on the implicitly-available Project object. The "unsugared" version looks like this in Kotlin:
this.repositories(
    {
        this.jcenter()
    }
)
[Note that the word this is used to access the implicit context. The word it has a different meaning in Kotlin from in Groovy. In Groovy it means the implicit context, but in Kotlin it means the first argument. We didn't pass any arguments to jcenter when we called it, so we can't use it, but we were being run in a context, which we can refer to using this. Simple. huh?]

Execution order in Gradle Kotlin

We this build.gradle.kts:
println("before")
project.repositories( {
    println("within")
    jcenter()
})
println("after")
We see this behaviour:
$ ./gradlew -q
before
within
after
which is all identical to the Groovy version.

Making a new task in Gradle Kotlin

tasks.register("mytask") {
    doLast {
        println("running mytask")
    }
}
This is identical to the Groovy version, but slightly different when unsugared:
tasks.register(
    "mytask",
    {
        this.doLast(
            {
                println("running mytask")
            }
        )
    }
)
Notice that Kotlin lets you do the same trick as Groovy: providing an extra argument to a function that is a closure by writing it immediately after it looks like you've finished calling it. It's good for people who dislike closing brackets hanging around longer than they're welcome. As someone who likes Lisp, I'm OK with it, but what do I know?

One task depending on another in Gradle Kotlin

plugins {
    java
}
val mytask = tasks.register("mytask") {
    doLast {
        println("running mytask")
    }
}
tasks.compileJava {
    dependsOn(mytask)
}
This differs slightly from the Groovy version, even though the meaning is the same: we start off in the context of a Project object that we call methods on. The code to make one task depend on another gets hold of the Task object called compileJava from inside the tasks property of the Project, and calls it (because it's a callable object). We pass in a closure that runs in the context of this Task object, calling its dependsOn method, and passing in a reference to the mytask object, which is a Task and was created in the code above.

Corrections and clarifications welcome

The above is what I have worked out by experimentation and trying to read the Gradle documentation. Please add comments that clear up confusions and correct mistakes.

Performance of Java 2D drawing operations (part 3: image opacity)

Andy Balaam from Andy Balaam&#039;s Blog

Series: operations, images, opacity

Not because I was enjoying it, I seemed compelled to continue my quest to understand the performance of various Java 2D drawing operations. I’m hoping to make my game Rabbit Escape faster, especially on the Raspberry Pi, so you may see another post sometime actually trying this stuff out on a Pi.

But for now, here are the results of my investigation into how different patterns of opacity in images affects rendering performance.

You can find the code here: gitlab.com/andybalaam/java-2d-performance.

Results

  • Images with partially-opaque pixels are no slower than those with fully-opaque pixels
  • Large transparent areas in images are drawn quite quickly, but transparent pixels mixed with non-transparent are slow

Advice

  • Still avoid any transparency whenever possible
  • It’s relatively OK to use large transparent areas on images (e.g. a fixed-size animation where a character moves through the image)
  • Don’t bother restricting pixels to be either fully transparent or fully opaque – partially-opaque is fine

Opacity patterns in images

Non-transparent images drew at 76 FPS, and transparent ones dropped to 45 FPS.

I went further into investigating transparency by creating images that were:

  • All pixels 50% opacity (34 FPS)
  • Half pixels 0% opacity, half 100%, mixed up (34 FPS)
  • Double the size of the original image, but the extra area is fully transparent, and the original area is non-transparent (41 FPS)

I concluded that partial-opacity is not important to performance compared with full-opacity, but that large areas of transparency are relatively fast compared with images with complex patterns of transparency and opacity.

Numbers

Transparency and opacity

Test FPS
large nothing 90
large images20 largeimages 76
large images20 largeimages transparentimages 45
large images20 largeimages transparent50pcimages 34
large images20 largeimages transparent0pc100pcimages 34
large images20 largeimages transparentareaimages 41

Feedback please

Please do get back to me with tips about how to improve the performance of my experimental code.

Feel free to log issues, make merge requests or add comments to the blog post.

Performance of Java 2D drawing operations (part 2: image issues)

Andy Balaam from Andy Balaam&#039;s Blog

Series: operations, images

In my previous post I examined the performance of various drawing operations in Java 2D rendering. Here I look at some specifics around rendering images, with an eye to finding optimisations I can apply to my game Rabbit Escape.

You can find the code here: gitlab.com/andybalaam/java-2d-performance.

Results

  • Drawing images with transparent sections is very slow
  • Drawing one large image is slower than drawing many small images covering the same area(!)
  • Drawing images outside the screen is slower than not drawing them at all (but faster than drawing them onto a visible area)

Advice

  • Avoid transparent images where possible
  • Don’t bother pre-rendering your background tiles onto a single image
  • Don’t draw images that are off-screen

Images with transparency

All the images I used were PNG files with a transparency layer, but in most of my experiments there were no transparent pixels. When I used images with transparent pixels the frame rate was much slower, dropping from 78 to 46 FPS. So using images with transparent pixels causes a significant performance hit.

I’d be grateful if someone who knows more about it can recommend how to improve my program to reduce this impact – I suspect there may be tricks I can do around setComposite or setRenderingHint or enabling/encouraging hardware acceleration.

Composite images

I assumed that drawing a single image would be much faster than covering the same area of the screen by drawing lots of small images. In fact, the result was the opposite: drawing lots of small images was much faster than drawing a single image covering the same area.

The code for a single image is:

g2d.drawImage(
    singleLargeImage,
    10,
    10,
    null
)

and for the small images it is:

for (y in 0 until 40)
{
    for (x in 0 until 60)
    {
        g2d.drawImage(
            compositeImages[(y*20 + x) % compositeImages.size],
            10 + (20 * x),
            10 + (20 * y),
            null
        )
    }
}

The single large image was rendered at 74 FPS, whereas covering the same area using repeated copies of 100 images was rendered at 80 FPS. I ran this test several times because I found the result surprising, and it was consistent every time.

I have to assume some caching (possibly via accelerated graphics) of the small images is the explanation.

Drawing images off the side of the screen

Drawing images off the side of the screen was faster than drawing them in a visible area, but slower than not drawing them at all. I tested this by adding 10,000 to the x and y positions of the images being drawn (I also tested subtracting 10,000 with similar results). Not drawing any images ran at 93 FPS, drawing images on-screen at 80 FPS, and drawing them off-screen only 83 FPS, meaning drawing images off the side takes significant time.

Advice: check whether images are on-screen, and avoid drawing them if not.

Numbers

Transparency

Test FPS
large nothing 95
large images20 largeimages 78
large images20 largeimages transparentimages 46

Composite images

(Lots of small images covering an area, or a single larger image.)

Test FPS
large nothing 87
large largesingleimage 74
large compositeimage 80

Offscreen images

Test FPS
large nothing 93
large images20 largeimages 80
large images20 largeimages offscreenimages 83

Feedback please

Please do get back to me with tips about how to improve the performance of my experimental code.

Feel free to log issues, make merge requests or add comments to the blog post.