Outreachy August 2022 update

Andy Balaam from Andy Balaam's Blog

I had the pleasure of being a mentor this summer for an Outreachy internship for the Matrix organisation. Outreachy provides internships to people subject to systemic bias and impacted by underrepresentation in the technical industry where they are living.

Outreachy is a fantastic organisation doing a brilliant job to try and make our sometimes terrible industry a little bit better.

Outreachy logo

Mentoring was great fun, mainly because it was such a pleasure working with my awesome intern Usman. There is lots of support available for interns and mentors through Outreachy’s Zulip chat (when will we persuade them to use Matrix? ;-) so you always have somewhere to turn if you have questions.

If you want to read more about the internship from Usman’s point of view, check out his blog posts:

  • Outreachy Blog – Introducing Myself
  • Wrap-up: Summary of my journey to being an outreachy intern at Element

    We talked every day on video calls, and really enjoyed working together. Some days we would just chat, sometimes I would give pointers for things to try in the code, or people to talk to. Some days we worked through some code together, and that was the most fun. Usman is incredibly enthusiastic and bright, so it was very satisfying making suggestions and seeing him put them into practice.

    Success!

    The work went very well, and Usman succeeded in creating a prototype that will help us design the Favourite Messages feature:

    Note: the feature isn’t ready to be fully release because it needs to be implemented on mobile platforms as well as changing where it stores its information: currently we use the browser’s local storage, but we plan to store things in Matrix, meaning it is automatically synchronised between devices.

    Things that went well

    • Meeting every day: we talked on a short video call every morning. This meant if we misunderstood each other it was quickly resolved, without lots of time being wasted.
    • Having a clear list of tasks: we kept a tracking issue on Github. This meant were clear what Usman was supposed to be doing now, and what was coming next.
    • Being flexible: we worked together to change the list of tasks every week or so. This meant we were being realistic about what could be achieved, and able to change in response to things we found out, or feedback from others.
    • Getting design input: we talked to Element’s designers several times during the project, showing them prototypes and early implementations. This meant we didn’t waste time implementing things that would need redesign later.
    • Support for me: I was able to work with Thib, who is our Outreachy Matrix community organiser, especially during the selection process. This meant I was not making decisions in isolation, and had support if anything tricky came up.
    • The Element Web community: Usman got loads of support from our community. Special thanks to Šimon, Olivier, Shay and t3chguy for your help!
    • Element the company: Element paid for this internship, and gave great support to Usman, integrating him into all our systems, inviting him to introductory meetings etc. He had every opportunity to see what working at Element is like, and to make an impression on everyone here. Element did a great job here.

    Things I would do differently

    • Managing the contribution period: before the project began, applicants are invited to contribute to the projects, allowing us to choose an intern based on those contributions. I felt slightly disorganised at this stage, and there was a lot of activity in issues and pull requests in the project from applicants. I think I should have warned our community and explained what was going to happen up-front, and maybe enlisted help from people willing to triage the contributions a little. Contributions varied in quality and understanding level, so having some volunteers who were primed to spend a little more time explaining and helping contributors get started would have prevented this impinging on the time of the team as a whole. Nevertheless, our team responded really well, and we got some useful contributions, and I hope the contributors had a good experience too.
    • Integrating Usman into the team: we chose a project that was independent from what other team members were doing, meaning he mostly interacted with others when he needed help. While it is sensible to make sure interns are decoupled from the main work (because it’s hard to predict how much progress they will make, and they are going stop after their internship), I do also wish we could have found a project that gave more opportunity to work with other people, not just “stealing” their time to help out, but actually working together on shared pieces of work. This is a tricky one to figure out, but food for thought.

    Conclusions

    The experience of being a mentor was really fun, and I would recommend it to anyone working on an open source project.

    I would emphasise, though, that you need to put aside enough time: the internship will not be successful if you don’t make time to work with your intern, get to know them, and introduce them to your community. Since interns may be new to the world of work, or shy about taking your time, as a mentor, you need to take responsibility for giving them enough support.

    Final note: as a mentor, you are NOT responsible for the work going well! Your responsibility is to help and support your intern, and give them everything they need to be successful (including feedback about things that are not working well), but it is up to the the intern themself to do the work, and how much work gets done is going to be the combination of a number of factors, including the intern’s experience and abilities. Don’t worry if you don’t get as far as you expected – after all, that happens in nearly all software projects…

Transcoding video files for playback in a browser

Andy Balaam from Andy Balaam's Blog

ffmpeg -i original.mkv -c:v libx264 -c:a aac -ac 2 -ab 384000 -ar 48000 new.mp4

(Short answer: use the above ffmpeg command line. Read on for how I did this in Tdarr.)

I recently discovered Jellyfin, which gives me a Netflix-like UI for viewing my own videos, and seems great.

The only problem I had was that some videos were in formats that can’t be played natively in a web browser. Jellyfin heroically tries to transcode them on the fly, but my server is very lightweight, and there’s no way it can do that.

So, I needed to transcode those videos to a more suitable format.

Tdarr allows transcoding large numbers of files, and with a little head-scratching I worked out how to get it running, but I still needed the right ffmpeg options to make the videos work well in Firefox, without needing transcoding of video, audio or the container.

Here are the Tdarr settings that I found worked well:

Output file container: .mp4
Encoder: ffmpeg
CLI arguments: -c:v libx264 -c:a aac -ac 2 -ab 384000 -ar 48000
Only transcode videos in these codecs: hevc

Explanation:

  • Output file container: .mp4 – wraps the video up in an MP4 container – surprisingly, Firefox doesn’t seem to support MKV.
  • -c:v libx264 – re-encodes the video as H.264. Firefox can’t do H.265, and H.264 is reasonably compatible with lots of browsers. If you don’t care about Safari or various Microsoft browsers, you might want to think about VP9 as it’s natively supported on Firefox, so should work on weird architectures etc.
  • -c:a aac -ac 2 -ab 384000 -ar 48000 – re-encodes the audio as AAC with the right bitrates etc. Jellyfin was still transcoding the audio when I just specified -c:a aac, and it took me a while to work out that you need those other options too.
  • Only transcode videos in these codecs: hevchevc means H.265 encoding, and the videos I had problems with were in that encoding, but you might have different problems. If in doubt, you can choose “Don’t transcode videos in these codecs:” and uncheck all the encodings, meaning all your videos will be re-encoded.

If you are not using Tdarr, here is the plain command line to use with ffmpeg:

ffmpeg -i original.mkv -c:v libx264 -c:a aac -ac 2 -ab 384000 -ar 48000 new.mp4

Building cross-platform Rust for Web, Android and iOS – a minimal example

Andy Balaam from Andy Balaam's Blog

One of the advantages of writing code in Rust is that it can be re-used in other places. Both iOS and Android allow using native libraries within your apps, and Rust compiles to native. Web pages can now use WebAssembly (WASM), and Rust can compile to WASM.

So, it should be easy, right?

Well, in practice it seems a little tricky, so I created a small example project to explain it to myself, so maybe it’s helpful to you too.

The full code is at gitlab.com/andybalaam/example-rust-bindings, but here is the general idea:

crates/example-rust-bindings - the real Rust code
bindings/ffi - uniffi code to build shared objects for Android and iOS
bindings/wasm - wasm_bingen code to build WASM for Web
examples/example-android - an Android app that generates a Kotlin wrapper, and runs the code in the shared object
examples/example-web - a web page that imports the WASM and runs it

Steps for WASM

Proof that I did this on Web - Firefox showing "This string is from Rust!"

Variation: if you modify the build script in package.json to call wasm-pack with --target node instead of --target web you can generate code suitable for using from a NodeJS module.

Steps for Android

Proof that I did this on Android: Android emulator showing a label "This string is from Rust!"

Steps for iOS

I am working on this and will fill it in later.

Improving my vimrc live on stream

Andy Balaam from Andy Balaam's Blog

I was becoming increasingly uncomfortable with how crufty my neovim config was getting, and especially how I didn’t understand parts of it, so I decided to wipe it clean and rebuild it from scratch.

I did it live on stream, to make it feel like a worthwhile activity:

Headline features of the new vimrc:

  • A new theme, using the base16 theme framework
  • A file browser (NERDTree)
  • A minimalist status line with vim-airline
  • Search with ripgrep
  • Rust language support with Coc

Note: after the stream I managed to resolve the remaining issues with highlight colours not showing by triggering re-applying them after the theme has been applied:

augroup tabs_in_make
    autocmd!
    autocmd ColorScheme * highlight MatchParen cterm=none ctermbg=none ctermfg=green
augroup END

You can find my current neovim config at gitlab.com/andybalaam/configs/-/tree/main/.config/nvim.

Comparison of Matrix events before and after “Extensible Events”

Andy Balaam from Andy Balaam's Blog

(Background: Matrix is the awesome open standard for messaging that I get to work on now that I work at Element.)

The Extensible Events (MSC1767) Matrix Spec Change proposal describes a new way of structuring events in matrix that makes it easy to send events that have multiple representations (e.g. something clever like an interactive map, and something simpler like an image of a map).

The main purpose of the change is to make it easy for clients that don’t support some amazing new feature to display something that is still useful.

Since there is an implementation of this change out in the wild (in Element), it seems reasonably likely that this change will be accepted into the Matrix spec.

I really like this change, but I find it hard to understand, so here is a simple example that I have found helpful to think it through.

An old event, and a new event

Here is an old-fashioned event, followed by a new, shiny, extensible version:

{
    "type": "m.room.message",
    "content": {
        "body": "This is the *old* way",
        "format": "org.matrix.custom.html",
        "formatted_body": "This is the <b>old</b> way",
        "msgtype": "m.text"
    },
    ... other properties not relevant to this, e.g. "sender" ...
}
{
    "type": "m.message",
    "content": {
        "m.message": [
            {"mimetype": "text/plain", "body": "This the *new* way"},
            {"mimetype": "text/html", "body": "This is the <b>new</b> way"}
        ],
    }
    ... other properties not relevant to this, e.g. "sender" ...
}

Notice that in the new extensible events, the property within content is the same as the message type (here: m.message).

The point is that as well as the primary event type (here, m.message) we can other representations of the same message, such as an image, location co-ordinates, or something completely different. The client will render the primary event type if it understands it (and is able to show it), but if not, it can look for other types that it does understand.

For example, in Polls when you send a new poll question, it could look like this:

{
    "type": "m.poll.start",
    "content": {
        "m.poll.start": {
            ... The actual poll question etc. ...
        },
        "m.message": [
            ... A text version of the question ...
        ]
    },
    ... other properties not relevant to this, e.g. "sender" ...
}

So clients that don’t know m.poll.start can still display the poll question (if they understand extensible events), instead of completely ignoring event types they don’t know about.

An abbreviated form of the new event

Of course, life is not quite as simple as that.

Because this is a lot of typing:

{
    "type": "m.message",
    "content": {
        "m.message": [
            {"mimetype": "text/plain", "body": "This the *new* way"},
            {"mimetype": "text/html", "body": "This is the <b>new</b> way"}
        ],
    }
    ... other properties not relevant to this, e.g. "sender" ...
}

We have an abbreviated form:

{
    "type": "m.message",
    "content": {
        "m.text": "This the *new* way",
        "m.html": "This is the <b>new</b> way"
    }
    ... other properties not relevant to this, e.g. "sender" ...
}

These two are exactly equivalent.

m.text is an abbreviation for an m.message containing an entry with "mimetype": "text/plain" and the relevant body. Similarly, m.html is an abbreviation for an m.message containing an entry with "mimetype": "text/html" and the relevant body. If you declare both, they effectively get squashed together into one m.message with both entries.

Those 2 are the only abbreviations listed, so they are special cases.

Backwards compatibility

Of course, life is way more complicated than that, so what we’re likely to see around if/when this gets widely adopted is some kind of mashed-together event like this:

{
    "type": "m.room.message",
    "content": {
        "msgtype": "m.text",
        "body": "Hello World",
        "format": "org.matrix.custom.html",
        "formatted_body": "<b>Hello</b> World",
        "m.text": "Hello World",
        "m.html": "<b>Hello</b> World"
    }
}

Note that the type here is m.room.message, where extensible events says it should be m.message. The idea is that an extensible-events-aware client will see "msgtype": "m.text" and know to look for m.message as the primary type. (This is further complicated here by the fact that there isn’t actually a m.message property – this is because m.text and m.html are abbreviated forms of it.)

Also, clients that want to display old events will need to preserve their code that parses the old event types in perpetuity.

Streaming to Twitch and PeerTube simultaneously using nginx on Oracle cloud

Andy Balaam from Andy Balaam&#039;s Blog

Simulcasting RTMP using NGINX

I want people to be able to watch my Matrix and Rust live coding streams using free software, so I’d like to simulcast to PeerTube as well as Twitch.

This is possible using NGINX and its RTMP module. It does involve building NGINX from source, but I actually found that reasonably easy to do.

Why Oracle cloud?

I would never recommend using Oracle for anything, but they do provide up to two virtual machines in their cloud for free, and the one I am using has been consistently available with very good connectivity, in a London data centre since I set it up several months ago.

So, we are making our lives more difficult by trying to do this on Oracle Linux, which is a derivative of RHEL.

Building NGINX and its RTMP module on Oracle Linux

I ran these commands on my Oracle cloud instance (running Oracle Linux):

sudo yum install git pcre-devel openssl-devel
mkdir nginx
cd nginx
wget http://nginx.org/download/nginx-1.21.4.tar.gz
git clone https://github.com/arut/nginx-rtmp-module.git
cd nginx-1.21.4
./configure --add-module=../nginx-rtmp-module/
make
sudo make install

After all this NGINX was installed to /usr/local/nginx/.

Creating the NGINX config file for RTMP simulcasting

Next I edited the NGINX config file by typing:

sudo nano /usr/local/nginx/conf/nginx.conf

And pasted in this config at the bottom of the file:

rtmp {
    server {
        listen 1935;
        chunk_size 4096;
        application live {
            live on;
            record off;
            push rtmp://live.twitch.tv/app/live_INSERT_TWITCH_STREAM_KEY;
            push rtmp://diode.zone:1935/live/INSERT_PEERTUBE_STREAM_KEY;
        }
    }
}

Notice that you will need to get your Twitch stream key from Twitch -> Creator Dashboard -> Settings -> Stream, then Copy next to the Primary Stream Key.

To get a PeerTube stream ID, you will need to go to your PeerTube page and click Publish, then Go Live, choose your channel and choose Go Live. Note that if you want the streams to record and be available later, you have to create a new stream key each time you start a stream, and change it in nginx.conf.

If you use a different PeerTube server (I use diode.zone) then you’ll need to change the server name in the config file above too.

Make sure your config file is saved with the right URLs in it.

Opening ports

To send RTMP traffic to my server, I needed to open the right port to the Oracle cloud instance. That involved creating an ingress rule, and adding a firewall rule.

Creating an ingress rule

In the web interface, I went to the menu in the top left, clicked Compute, then Instances.

I clicked on my instance’s name, then I clicked on the name of the subnet in the details (on the right).

I clicked on Default security list for…, then Add Ingress Rules.

I made an ingress rule with Source Type=CIDR, Source CIDR=0.0.0.0/0, IP Protocol=TCP, Source Port Range=(blank, meaning all), Destination Port Range=1935

Adding a firewall rule

Then I ssh’d into the machine and ran these commands to create a firewall rule allowing the traffic:

sudo firewall-cmd --zone=public --permanent --add-port=1935/tcp
sudo firewall-cmd --reload

Stop and Start NGINX

After creating the config file and opening the right port, I needed to start NGINX.

Every time I change the config file, I need to restart it.

If it’s already running, I stop it with:

sudo /usr/local/nginx/sbin/nginx -s stop

and then I start it up again with

sudo /usr/local/nginx/sbin/nginx

I can check whether it’s happy by looking at the log files, for example to see any errors:

less /usr/local/nginx/logs/error.log

Starting the stream

Now I go into OBS and go to File -> Settings -> Stream and choose the type as Custom, and the Server as rtmp://1.1.1.1/live. (But instead of 1.1.1.1 I put the public IP address of my instance, which I found by clicking the name of the instance in the Oracle cloud management console.)

New game: Tron – frantic multiplayer retro action

Andy Balaam from Andy Balaam&#039;s Blog

My newest game is out now on Smolpxl Games – Tron:

Pixellated lines fight each other to stay alive

Play at smolpxl.gitlab.io/tron.

It’s a frantic multiplayer retro pixellated thingy playable in your browser. Try to stay alive longer than everyone else!

This version allows many players (up to 16 if you can manage it), and is quite pure in its implementation.

There are bots to play against, and you can gather your friends around a keyboard to play together.

Part of the motivation for writing this game was to test my new smolpxl-remote remote-play system, but this is not enabled yet, so watch this space…

I love playing games with other people – preferably at least 3 other people. In theory you could have 8 players around a keyboard playing this – send me a picture if you try!

One feature I worked on in the Smolpxl library for this game: saving configuration to local storage (and asking permission to do so). I ended up with a very ugly hack to do this, so a bit more work is needed before I merge it into the library.

Preventing Virgin Media hijacking my DNS

Andy Balaam from Andy Balaam&#039;s Blog

Yesterday I learned that Virgin Media is inserting itself into some of my DNS requests. Much as I am not a fan of how powerful Cloudflare are, if they are telling the truth about their DNS, then it’s safe, so I followed their instructions on how to use their DNS and then removed the default DNS and hopefully my Internet will work now.

From the serverfault answer by lauc.exon.nod:

nmcli con mod "Wired connection 1" ipv4.dns "1.1.1.1 1.0.0.1"
nmcli con mod "Wired connection 1" ipv4.ignore-auto-dns yes
nmcli con down "Wired connection 1"
nmcli con up "Wired connection 1"

New Job at Element (Matrix)

Andy Balaam from Andy Balaam&#039;s Blog

I started a new job today at Element!

It has been a long-standing ambition of mine to work in Free and Open Source software, and I am very excited to work for a company that is the main developer of a really important project: the Matrix communication network.

I don’t know much about what I’ll be doing yet, but finding an open source company with a decent business model that is prepared to pay me is very exciting. The fact that they have offices that are close enough for me to go for is another huge bonus.

Wish me luck, and I’ll let you know what I’m working on when it becomes more clear.