Posted on

Struggling with Rust to figure out the right types for a function signature

Andy Balaam from Andy Balaam's Blog

I am loving writing code in Rust. So many things about the language and its ecosystem feel so right*.

* For example: ownership of objects, expressive type system, compile to native, offline API docs, immutability, high quality libraries.

One of the things I like about it is that I don’t feel like I need to use an IDE, so I can happily code in Vim with no clever plugins.

One thing an IDE might give me would be an “extract function” refactoring. In most languages I am happy to do that manually, because I can let the compile errors guide me on what my function should look like.

However, in Rust I sometimes find it’s hard to find the right signature for a function I want to extract, and I am struggling to persuade the compiler to help me.

Here is an example from my new listsync project, in listsync-client-rust.rs:

use actix_web::{middleware, App, HttpServer};
use listsync_client_rust;
// ...
#[actix_rt::main]
async fn main() -> std::io::Result<()> {
//...
    HttpServer::new(|| {
        App::new()
            .wrap(listsync_client_rust::cookie_session::new_session())
            .wrap(middleware::Logger::default())
            .configure(listsync_client_rust::config)
    })
//...

I would like to extract the code highlighted above, the creation of an App, into a separate function, like this:

fn new_app() -> ??? {
    App::new()
        .wrap(listsync_client_rust::cookie_session::new_session())
        .wrap(middleware::Logger::default())
        .configure(listsync_client_rust::config)
}
//...
    HttpServer::new(|| {
        new_app()
    })

Simple, right? To find out what the return type of the function should be, I can just make a bad guess, and get the compiler to tell me what I did wrong. In this case, I will guess by changing the question marks above into i32, and run cargo test. I get quite a few errors, one of which is:

error[E0277]: the trait bound `i32: actix_service::IntoServiceFactory<_>` is not satisfied
  --> src/bin/listsync-client-rust.rs:27:5
   |
27 | /     HttpServer::new(|| {
28 | |         new_app()
29 | |     })
   | |______^ the trait `actix_service::IntoServiceFactory<_>` is not implemented for `i32`
   |
   = note: required by `actix_web::server::HttpServer`

So the first problem I see is that the error message I am seeing is about the later code, and there are no errors about my new function.

I obviously went a little too fast. Let’s change the HttpServer::new code back to how it was before, and only make a new function new_app. Now I get an error that should help me:

error[E0308]: mismatched types
  --> src/bin/listsync-client-rust.rs:12:5
   |
11 |   fn new_app() -> i32 {
   |                   --- expected `i32` because of return type
12 | /     App::new()
13 | |         .wrap(listsync_client_rust::cookie_session::new_session())
14 | |         .wrap(middleware::Logger::default())
15 | |         .configure(listsync_client_rust::config)
   | |________________________________________________^ expected i32, found struct `actix_web::app::App`
   |
   = note: expected type `i32`
              found type `actix_web::app::App<impl actix_service::ServiceFactory, actix_web::middleware::logger::StreamLog<actix_http::body::Body>>`

So the compiler has told us what type we are returning! Let’s copy that into the type signature of the function:

use actix_service::ServiceFactory;
use actix_http::body::Body;
// ...
fn new_app() -> App<impl ServiceFactory, middleware::logger::StreamLog<Body>> {
// ...

The first error I get from the compiler is a distraction:

error[E0432]: unresolved import `actix_service`
 --> src/bin/listsync-client-rust.rs:1:5
  |
1 | use actix_service::ServiceFactory;
  |     ^^^^^^^^^^^^^ use of undeclared type or module `actix_service`

I can fix it by adding actix-service = "1.0.5" to Cargo.toml. (I found the version by looking in Cargo.lock, since this dependency was already implicitly used – I just need to make it explicit if I am going to use it directly.)

Once I do that I get the next error:

error[E0603]: module `logger` is private
  --> src/bin/listsync-client-rust.rs:13:54
   |
13 | fn new_app() -> App<impl ServiceFactory, middleware::logger::StreamLog<Body>> {
   |                                                      ^^^^^^

This leaves me a bit stuck: I can’t use StreamLog because it’s in a private module.

More importantly, it makes the point that I don’t actually want to be as specific as I am being: I don’t care what the exact type parameters for App are – I just want to return an App of some kind and have the compiler fill in the blanks. Ideally, if I change the body of new_app later, for example to add another wrap call that changes the type of App we are returning, I’d like to leave the return type the same and have it just work.

With that in mind, I took at look at the type that HttpServer::new takes in. Here is HttpServer:

impl<F, I, S, B> HttpServer<F, I, S, B> where
    F: Fn() -> I + Send + Clone + 'static,
    I: IntoServiceFactory<S>,
    S: ServiceFactory<Config = AppConfig, Request = Request>,
    S::Error: Into<Error> + 'static,
    S::InitError: Debug,
    S::Response: Into<Response<B>> + 'static,
    <S::Service as Service>::Future: 'static,
    B: MessageBody + 'static,

and HttpServer::new looks like:

pub fn new(factory: F) -> Self

So it takes in a function which actually makes the App, and the type of that function is F, which is a Fn which returns a I + Send + Clone + 'static. From the declaration of HttpServer we can see that the type of I depends on S and B, which have quite complex types. Let’s paste the whole thing in:

use actix_http::{Error, Request, Response};
use actix_service::IntoServiceFactory;
use actix_web::body::MessageBody;
use actix_web::dev::{AppConfig, Service};
use core::fmt::Debug;
// ...
fn new_app<I, S, B>() -> I
where
    I: IntoServiceFactory<S> + Send + Clone + 'static,
    S: ServiceFactory<Config = AppConfig, Request = Request>,
    S::Error: Into<Error> + 'static,
    S::InitError: Debug,
    S::Response: Into<Response<B>> + 'static,
    <S::Service as Service>::Future: 'static,
    B: MessageBody + 'static,
{
    App::new()
        .wrap(listsync_client_rust::cookie_session::new_session())
        .wrap(middleware::Logger::default())
        .configure(listsync_client_rust::config)
}

Note that I had to modify I to include the extra requirements on the return type of F from the definition of HttpServer. (I think I did the right thing, but I’m not sure. If I just remove the + Send + Clone + 'static it seems to behave similarly.)

Now I get this error from the compiler:

error[E0308]: mismatched types
  --> src/bin/listsync-client-rust.rs:27:5
   |
17 |   fn new_app<I, S, B>() -> I
   |                            - expected `I` because of return type
...
27 | /     App::new()
28 | |         .wrap(listsync_client_rust::cookie_session::new_session())
29 | |         .wrap(middleware::Logger::default())
30 | |         .configure(listsync_client_rust::config)
   | |________________________________________________^ expected type parameter, found struct `actix_web::app::App`
   |
   = note: expected type `I`
              found type `actix_web::app::App<impl actix_service::ServiceFactory, actix_web::middleware::logger::StreamLog<actix_http::body::Body>>`
   = help: type parameters must be constrained to match other types
   = note: for more information, visit https://doc.rust-lang.org/book/ch10-02-traits.html#traits-as-parameters

The compiler really tries to help here, suggesting I read a chapter of the Rust Book, but even after reading it I could not figure out how to do what I am trying to do.

Can anyone help me?

Wouldn’t it be amazing if there were some way the compiler could give me easier-to-understand help to figure this out?

Posted on

Coding workshop example worksheets

Andy Balaam from Andy Balaam&#039;s Blog

This week we did a coding workshop exercise: 2 teams implemented the different sides of the SMPP protocol (without speaking to each other) and this morning we tried out connecting them together.

We successfully sent a message and received an acknowledgement!

It was a lot of fun and I we learned a surprising amount about SMPP (and quite how … interesting … the standard is).

In case they’re useful to anyone, here are the worksheets I made up: Team 1 ODT, Team 1 PDF, Team 2 ODT, Team 2 PDF.

Idea for a team who are less interested in SMPP (!) – try a similar exercise implementing FTP, which is a nice simple text-based protocol. I did this once and found it extremely rewarding.

Posted on

Building an all-in-one Jar in Gradle with the Kotlin DSL

Andy Balaam from Andy Balaam&#039;s Blog

To build a “fat” Jar of your Java or Kotlin project that contains all the dependencies within a single file, you can use the shadow Gradle plugin.

I found it hard to find clear documentation on how it works using the Gradle Kotlin DSL (with a build.gradle.kts instead of build.gradle) so here is how I did it:

$ cat build.gradle.kts 
import com.github.jengelman.gradle.plugins.shadow.tasks.ShadowJar

plugins {
    kotlin("jvm") version "1.3.41"
    id("com.github.johnrengelman.shadow") version "5.1.0"
}

repositories {
    mavenCentral()
}

dependencies {
    implementation(kotlin("stdlib"))
}

tasks.withType<ShadowJar>() {
    manifest {
        attributes["Main-Class"] = "HelloKt"
    }
}

$ cat src/main/kotlin/Hello.kt 
fun main() {
    println("Hello!")
}

$ gradle wrapper --gradle-version 5.5
BUILD SUCCESSFUL in 0s
1 actionable task: 1 executed

$ ./gradlew shadowJar
BUILD SUCCESSFUL in 1s
2 actionable tasks: 2 executed

$ java -jar build/libs/hello-all.jar 
Hello!
Posted on

Creating a self-signed certificate for Apache and connecting to it from Java

Andy Balaam from Andy Balaam&#039;s Blog

Our mission: to create a self-signed certificate for an Apache web server that allows us to connect to it over HTTPS (SSL/TLS) from a Java program.

The tricky bit for me was generating a certificate that contains Subject Alternative Names for my server, which is needed to connect to it from Java.

We will use the openssl command.

Creating a self-signed certificate for Apache HTTPD

First create a config file cert.conf:

[ req ]
distinguished_name  = subject
x509_extensions     = x509_ext
prompt = no

[ subject ]
commonName = Example Company

[ x509_ext ]
subjectAltName = @alternate_names

[ alternate_names ]
DNS.1 = example.com

In the above, replace “example.com” with the name you will use for the host when you connect from Java. This is important, because Java requires the name in the certificate to match the name it is using to connect to the server. If you’re connecting to it as localhost, just put “localhost”. Note: do not include “https://” or any port or path after the hostname, so “example.com:8080/mypath” is wrong – it should be just “example.com”.

The alternate_names section above gives the “Subject Alternative Names” for this certificate. You can add more as “DNS.2”, “DNS.3”, etc.

Next, generate the server key and self-signed certificate:

openssl genrsa 2048 > server.key
chmod 400 server.key
openssl req -new -x509 -config cert.conf -nodes -sha256 -days 365 -key server.key -out server.crt

Now you have two new files: server.key and server.crt. These are the files that will be used by Apache HTTPD, so put them somewhere useful (e.g. inside /usr/local/apache2/conf/) and refer to them in the Apache config file using keys “SSLCertificateKeyFile” and “SSLCertificateFile” respectively. For more info see the SSL/TLS How-To.

Checking the certificate is being used

Start up your Apache and ensure you can connect to it over HTTPS using curl:

curl -v --insecure https://example.com:8080

Replace “https://example.com:8080” above with the full URL (this time, include “https://” and the port and path.

To examine the certificate that is being returned, run:

openssl s_client -showcerts -connect example.com:8080

Replace “example.com:8080” above with hostname and port (no “https:// this time!).

Connecting from Java

To be able to connect from Java, we need a Trust Store. We can create one in PKCS#12 format with:

openssl pkcs12 -export -passout pass:000000 -out trust.pkcs12 -inkey server.key -in server.crt

Note: Java 8 onwards is able to use .pkcs12 (PKCS#12) files for its trust store. The old .jks (Java Key Store) format is deprecated.

Now you have a file we can use as a trust store, follow my other article to connect from Java over HTTPS with a self-signed certificate.

Posted on

Scheduling a task in Java within a CompletableFuture

Andy Balaam from Andy Balaam&#039;s Blog

When we want to do something later in our Java code, we often turn to the ScheduledExecutorService. This class has a method called schedule(), and we can pass it some code to be run later like this:

    ScheduledExecutorService executor =
        Executors.newScheduledThreadPool(4);
    executor.schedule(
        () -> {System.out.println("..later");},
        1,
        TimeUnit.SECONDS
    );
    System.out.println("do...");
    // (Don't forget to shut down the executor later...)

The above code prints “do…” and then one second later it prints “…later”.

We can even write code that does some work and returns a result in a similar way:

    // (Make the executor as above.)
    ScheduledFuture future = executor.schedule(
        () -> 10 + 25, 1, TimeUnit.SECONDS);
    System.out.println("answer=" + future.get())

The above code prints “answer=35”. When we call get() it blocks waiting for the scheduler to run the task and mark the ScheduledFuture as complete, and then returns the answer to the sum (10 + 25) when it is ready.

This is all very well, but you may note that the Future returned from schedule() is a ScheduledFuture, and a ScheduledFuture is not a CompletableFuture.

Why do you care? Well, you might care if you want to do something after the scheduled task is completed. Of course, you can call get(), and block, and then do something, but if you want to react asynchronously without blocking, this won’t work.

The normal way to run some code after a Future has completed is to call one of the “then*” or “when*” methods on the Future, but these methods are only available on CompletableFuture, not ScheduledFuture.

Never fear, we have figured this out for you. We present a small wrapper for schedule that transforms your ScheduledFuture into a CompletableFuture. Here’s how to use it:

    CompletableFuture future =
        ScheduledCompletable.schedule(
            executor,
            () -> 10 + 25,
            1,
            TimeUnit.SECONDS
         );
    future.thenAccept(
        answer -> {System.out.println(answer);}
    );
    System.out.println("Answer coming...")

The above code prints “Answer coming…” and then “35”, so we can see that we don’t block the main thread waiting for the answer to come back.

So far, we have scheduled a synchronous task to run in the background after a delay, and wrapped its result in a CompletableFuture to allow us to chain more tasks after it.

In fact, what we often want to do is schedule a delayed task that is itself asynchronous, and already returns a CompletableFuture. In this case it feels particularly natural to get the result back as a CompletableFuture, but with the current ScheduledExecutorService interface we can’t easily do it.

It’s OK, we’ve figured that out too:

    Supplier> asyncTask = () ->
        CompletableFuture.completedFuture(10 + 25);
    CompletableFuture future =
        ScheduledCompletable.scheduleAsync(
            executor, asyncTask, 1, TimeUnit.SECONDS);
    future.thenAccept(
        answer -> {System.out.println(answer);}
    );
    System.out.println("Answer coming...")

The above code prints “Answer coming…” and then “35”, so we can see that our existing asynchronous task was scheduled in the background, and we didn’t have to block the main thread waiting for it. Also, under the hood, we are not blocking the ScheduledExecutorService‘s thread (from its pool) while the async task is running – that task just runs in whatever thread it was assigned when it was created. (Note: in our example we don’t really run an async task at all, but just immediately return a completed Future, but this does work for real async tasks.)

I know you’re wondering how we achieved all this. First, here’s how we run a simple blocking task in the background and wrap it in a CompletableFuture:

    public static  CompletableFuture schedule(
        ScheduledExecutorService executor,
        Supplier command,
        long delay,
        TimeUnit unit
    ) {
        CompletableFuture completableFuture = new CompletableFuture<>();
        executor.schedule(
            (() -> {
                try {
                    return completableFuture.complete(command.get());
                } catch (Throwable t) {
                    return completableFuture.completeExceptionally(t);
                }
            }),
            delay,
            unit
        );
        return completableFuture;
    }

And here’s how we delay execution of an async task but still return its result in a CompletableFuture:

    public static  CompletableFuture scheduleAsync(
        ScheduledExecutorService executor,
        Supplier> command,
        long delay,
        TimeUnit unit
    ) {
        CompletableFuture completableFuture = new CompletableFuture<>();
        executor.schedule(
            (() -> {
                command.get().thenAccept(
                    t -> {completableFuture.complete(t);}
                )
                .exceptionally(
                    t -> {completableFuture.completeExceptionally(t);return null;}
                );
            }),
            delay,
            unit
        );
        return completableFuture;
    }

Note that this should all work to run methods like exceptionally(), thenAccept(), whenComplete() etc.

Feedback and improvements welcome!