My experience upgrading to Elm 0.19

Andy Balaam from Andy Balaam's Blog

Elm is unstable, so upgrading to the next version can be painful. Here’s what I needed to do to upgrade from 0.18 to 0.19.

  • Replace elm-package.json and tests/elm-package.json with elm.json – e06f5a1728
  • Switch to the new elm-testb964b7c7a
  • Re-arrange Main, and how we call it from JavaScript – 0c118c49f
  • Stop using eeue56/elm-all-dict (since it’s not ported to 0.19 and porting it looked hard due to a lack of Debug.crash) – fe100f256
  • Replace toString with String.fromX or Debug.toString – 9e78163d0a3
  • Stop “shadowing” names by making new variables with the same name as another in the scope – 9688a621de
  • Adapt to the changed Html.style function – b991ab4f
  • Stop using Debug.crash – f98a70ad1
  • Adapt to the changes in the Regex module – 856762a4
  • Stop using tuples with more than 3 parts – 472c0bb7

The lack of Debug.crash is really, really painful, especially for a library like eeue56/elm-all-dict that has lots of invariants that are hard or impossible to enforce via the type system. On the other hand, if Elm can give a hard guarantee that there will be no runtime errors, this seems pretty cool. The problem is that some code may well have to return the wrong answer silently, instead of crashing, which could be much worse than crashing in some use-cases.

I was annoyed by the lack of more-than-3-part tuples, but even as I did the work to change my code, I saw it get better, so it’s hard to argue with.

The hardest part to work out was how to run the tests. Fortunately the tests themselves needed almost no changes. I just needed to do this:

rm -r tests/elm-stuff
rm tests/elm-package.json
sudo npm install -g elm-test@0.19.0-beta8
elm-test install
elm-test

My next job is to check out the –optimize compiler flag, and the advice on making the code smaller and faster.

Bulk adding items to Wunderlist using wunderline on Ubuntu MATE

Andy Balaam from Andy Balaam's Blog

If you use Wunderlist and want to be able to bulk-add tasks from a text file, first install and set up wunderline.

Now, to be able to right-click a text file containing one task per line on Ubuntu MATE, create a file called “wunderlist-bulk-add” in ~/.config/caja/scripts/ and make it executable. Paste the code below into that file.

(Note: it’s possible that this would work in GNOME if you replaced “caja” with “nautius” in that path – let me know in the comments.)

#!/usr/bin/env bash

set -e
set -u
set -o pipefail

function nonblank()
{
    grep -v -e '^[[:space:]]*$' "$@"
}

for F in "$@"; do
{
    COUNT=$(nonblank "$F" | wc -l | awk '{print $1}')
    if [ "$COUNT" = "" ]; then
    {
        zenity --info --no-wrap --text="File $F does not exist"
    }
    else
    {
        set +e
        zenity --question --no-wrap \
            --text="Add $COUNT items from $F to Wunderlist?"
        ANSWER=$?
        set -e

        if [ "$ANSWER" = "0" ]; then
        {
            set +e
            nonblank "$F" | \
                wunderline add --stdin | \
                zenity --progress --text "Adding $COUNT items to Wunderlist"
            SUCCEEDED=$?
            set -e
            if [ "$SUCCEEDED" = "0" ]; then
            {
                zenity --info --no-wrap --text "Added $COUNT items."
            }
            else
            {
                zenity --error --no-wrap \
                    --text "Failed to add some or all of these items."
            }; fi
        }; fi
    }; fi
}; done

Writing a new Flarum extension on Ubuntu

Andy Balaam from Andy Balaam's Blog

In a previous post I described how to install Flarum locally on Ubuntu.

Here is how I set up my development environment on top of that setup so I was able to write a new Flarum extension and test it on my local machine.

Recap: I installed Apache and PHP and enabled mod_rewrite, installed MariaDB and made a database, installed Composer, and used Composer to install Flarum at /var/www/html/flarum.

More info: Flarum Contribution Guide, extension development guide, extension development quick start, workflow discussion.

I decided to call my extension “rabbitescape/flarum-ext-rabbitescape-leveleditor”, so that is the name you will see below.

Here’s what I did:

cd /var/www/html/flarum

Edit /var/www/html/flarum/composer.json (see the workbench explanation for examples of the full file).

At the end of the “require” section I added a line like this:

“rabbitescape/flarum-ext-rabbitescape-leveleditor”: “*@dev”

(Remembering to add a comma at the end of the previous line so it remained valid JSON.)

After the “require” section, I added this:

    "repositories": [
        {
            "type": "path",
            "url": "workbench/*/"
        }
    ],

Next, I made the place where my extension would live:

mkdir workbench
cd workbench
mkdir flarum-ext-rabbitescape-leveleditor
cd flarum-ext-rabbitescape-leveleditor

Now I created a file inside flarum-ext-rabbitescape-leveleditor called composer.json like this:

{
    "name": "rabbitescape/flarum-ext-rabbitescape-leveleditor",
    "description": "Allow viewing and editing Rabbit Escape levels in posts",
    "type": "flarum-extension",
    "keywords": ["rabbitescape"],
    "license": "MIT",
    "authors": [
        {
            "name": "Andy Balaam",
            "email": "andybalaam@artificialworlds.net"
        }
    ],
    "support": {
        "issues": "https://github.com/andybalaam/flarum-ext-rabbitescape-leveleditor/issues",
        "source": "https://github.com/andybalaam/flarum-ext-rabbitescape-leveleditor"
    },
    "require": {
        "flarum/core": "^0.1.0-beta.5"
    },
    "extra": {
        "flarum-extension": {
            "title": "Rabbit Escape Level Editor"
        }
    }
}

In the same directory I created a file bootstrap.php like this:

<?php

return function () {
    echo 'Hello, world!';
};

Then I told Composer to refresh the project like this:

cd ../..   # Back up to the main directory
composer update

Among the output I saw a line like this which convinced me it had worked:

  - Installing rabbitescape/flarum-ext-rabbitescape-leveleditor (dev-master): Symlinking from workbench/flarum-ext-rabbitescape-leveleditor

Now I went to http://localhost/flarum, signed in as admin, adminpassword (as I set up in the previous post), clicked my username in the top right and chose Administration, then clicked Extensions on the left.

I saw my extension in the list, and turned it on by clicking the check box below it. This made the whole web site disappear, and instead just the text “Hello world!” appeared. This showed my extension was being loaded.

Finally, I edited bootstrap.php and commented out the line starting with “echo”. I refreshed the page in my browser and saw the Flarum site reappear.

Now, my extension is installed and ready to be developed! See flarum.org/docs/extend/start for how to get started making it do cool stuff.

Ubuntu “compose” key for easy unicode character input

Andy Balaam from Andy Balaam&#039;s Blog

I found out on Mastodon recently that the Compose key exists, allowing you to enter special characters using easy-to-remember key sequences (e.g. “<compose>/=” gives you “≠”).

To do this on Ubuntu (and probably many other systems), hold SHIFT, then press AltGr, release them both, and type something like “/=”. You should see the ≠ symbol appear.

In Ubuntu MATE, you can customise what key is the compose key by opening “Keyboard”, clicking “Layouts”, then “Options…” and checking a box under “Position of compose key”. I checked “Right Alt”, meaning my “AltGr” key is now the compose key.

More info: Ubuntu ComposeKey docs.

Want to explore? See the list of the default sequences you can type.

Redirecting all requests to https and www using .htaccess in Apache

Andy Balaam from Andy Balaam&#039;s Blog

I want all requests to artificialworlds.net/rabbit-escape/levels/ to get redirected to use the https protocol, and to include “www.” at the beginning of the URL, and I found lots of Stack Overflow articles, but nothing that worked perfectly for me. Here is how I managed it.

I edited the .htaccess file in the directory where I want this to apply, and added this at the top:

<IfModule mod_rewrite.c>
  RewriteEngine On
  RewriteCond %{HTTP_HOST} ^artificialworlds\.net$ [NC]
  RewriteRule ^ https://www.artificialworlds.net%{REQUEST_URI} [R=301,L]

  RewriteCond %{HTTPS} off
  RewriteRule ^ https://www.artificialworlds.net%{REQUEST_URI} [R=301,L]
</IfModule>

Example of a systemd service file

Andy Balaam from Andy Balaam&#039;s Blog

Here is an almost-minimal example of a systemd service file, that I use to run the Mastodon bot of my generative art playground Graft.

I made a dedicated user just to run this service, and installed Graft into /home/graft/apps/graft under that username. Now, as root, I edited a file called /etc/systemd/service/graft.service and made it look like this:

[Service]
ExecStart=/home/graft/apps/graft/bot-mastodon
User=graft
Group=graft
[Install]
WantedBy=multi-user.target

Now I can start the graft service like any other service like this:

sudo systemctl start graft

and find out its status with:

sudo systemctl status graft

If I want it to run on startup I can do:

sudo systemctl enable graft

and it will. Easy!

If I want to look at its output, it’s:

sudo journalctl -u graft

As a reward for reading this far, here’s a little animation you can make with Graft:

Installing Flarum on Ubuntu 18.04

Andy Balaam from Andy Balaam&#039;s Blog

I am setting up a forum for sharing levels for my game Rabbit Escape, and I have decided to try and use Flarum, because it looks really usable and responsive, has features we need like liking posts and following authors, and I think it will be reasonably OK to write the custom features we want.

So, I want a dev environment on my local Ubuntu 18.04 machine, and the first step to that is a standard install.

Warning: at the time of writing the Flarum docs say it does not work with PHP 7.2, which is what is included with Ubuntu 18.04, so this may not work. (So far it looks OK for me.)

Here’s how I got it working (as far as the web installer stage, anyway):

sudo apt install \
    apache2 \
    libapache2-mod-php \
    mariadb-server \
    php-mysql \
    php-json \
    php-gd \
    php-tokenizer \
    php-mbstring \
    php-curl

php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"

# Get the neat line from https://getcomposer.org/download/
# Don't copy it exactly!
php -r "if (hash_file('SHA384', 'composer-setup.php') === '544e09ee996cdf60ece3804abc52599c22b1f40f4323403c44d44fdfdd586475ca9813a858088ffbc1f233e9b180f061') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;"

mkdir ~/bin
php composer-setup.php --install-dir=~/bin/ --filename=composer
rm composer-setup.php

cd /var/www/html
sudo mkdir flarum
sudo chown $(whoami) flarum

# Log out and in again here to get composer to be in your PATH
cd flarum
composer create-project flarum/flarum . --stability=beta

sudo chgrp -R www-data .
sudo chmod -R 775 .

sudo systemctl restart apache2

Go to http://localhost/flarum in your browser, and follow the instructions there to get set up.

If I get further, I will update this post, including on how to set up the MySQL database.

If you want to find and share levels for Rabbit Escape, check up on our progress setting up the forum at https://artificialworlds.net/rabbit-escape/levels.

Connecting to Slack from an IRC client using slirc

Andy Balaam from Andy Balaam&#039;s Blog

I tried to get back to an IRC interface to Slack using Matrix, and it had some problems. Thanks to Colin Watson’s comment on that post, I tried Daniel Beer’s slirc, and so far it seems to be working pretty well.

Here’s what I did:

Get a Slack legacy token which slirc will use to connect to Slack as you. Follow the instructions given at that link, and you should end up with a token that looks something like “abcd-123456768-ETC-ETC”. Keep a note of it.

Install the prerequisites for slirc, and download it:

sudo cpan AnyEvent AnyEvent::HTTP AnyEvent::Socket AnyEvent::WebSocket::Client URI::Encode Data::Dumper JSON
mkdir slirc
cd slirc
wget -q 'https://www.dlbeer.co.nz/articles/slirc/slirc-20180515.pl'
chmod +x slirc-20180515.pl

Create a file in the slirc directory you created above, called rc.conf, and make it look like this:

slack_token=abcd-123456768-ETC-ETC
password=somepassword
port=6667

Replace “abcd-123456768-ETC-ETC” with the Slack legacy token you noted down earlier.

Replace “somepassword” with something you’ve made up (not your Slack password) – this is what you will type as the password in your IRC client.

Run slirc and leave it running:

./slirc-20180515.pl rc.conf

(Make sure you are inside the slirc dir when you run that.)

Start your IRC client (e.g. HexChat) and add a server with address “localhost” and port 6667, with your slack username and the password you added in the rc.conf (which you wrote instead of “somepassword”).

This mostly works for me, except it has a tendency to open a load of ad-hoc chats as channels, so I have to close them all to get a usable list.

Using Matrix to connect to Slack from an IRC client on Ubuntu

Andy Balaam from Andy Balaam&#039;s Blog

I like using HexChat to talk to my colleagues, like one other guy. It is fast, and it pops up a new window when someone sends me a direct/private message.

Recently, Slack shut down their IRC gateway, forcing me to use their slow UI, and making me unable to be responsive to direct messages without spending my entire life checking said UI, or being disturbed by notifications.

So, I decided to set up a Matrix bridge, because how hard can it be?

My system is Ubuntu MATE 18.04.

What I did

Install Synapse

Install Synapse, a Matrix homeserver.

wget -qO - https://matrix.org/packages/debian/repo-key.asc | sudo apt-key add -
sudo apt-add-repository http://matrix.org/packages/debian/
sudo apt update
sudo apt install matrix-synapse

Edit /etc/matrix-synapse/homeserver.yaml to change the line containing “registration_shared_secret” to look something like:

registration_shared_secret: "some secret password you made up"

(Replacing “some secret password you made up” with something secret.)

Make a new user:

register_new_matrix_user -c /etc/matrix-synapse/homeserver.yaml http://localhost:8008

Enter a username and password, and take a note of them.

Install Riot

(Technically, this step is optional, but I definitely recommend it to check everything is working.)

Install Riot, a client that can talk to a homeserver.

curl https://riot.im/packages/debian/repo-key.asc | sudo apt-key add -
sudo apt-add-repository https://riot.im/packages/debian/
sudo apt update
sudo apt install riot-web

Start Riot from the menu or by typing riot-web in a console.

Choose to log in to a Custom Server (not create a new user), and enter the Home Server URL as http://localhost:8008, leave the Identity Server URL as it is, and enter the username and password you entered in the previous step.

Riot should be able to connect to the Synapse server, even though it won’t have any rooms in it yet.

If this doesn’t work, try running riot-web in a console, and check the Synapse log file at /var/log/matrix-synapse/homeserver.log

Install matrix-puppet-slack

(Note: after this step, the homeserver can only be for you now, because it is logged into Slack as you. If you want to make a new Matrix server for lots of people, I found that matrix-appservice-slack works, but this only connects channels, not direct messages.)

Get a Slack legacy token which matrix-puppet-slack will use to connect to Slack as you. Follow the instructions given at that link, and you should end up with a token that looks something like “abcd-123456768-ETC-ETC”. Keep a note of it.

Install matrix-puppet-slack which allows you to connect a homeserver to a Slack instance, pretending it is you:

cd
git clone https://github.com/matrix-hacks/matrix-puppet-slack.git
cd matrix-puppet-slack/
npm install
cp config.sample.json config.json

Edit config.json so it looks something like this:

{
  "slack": [{
    "team_name": "Name of my team",
    "user_access_token": "abcd-123456768-ETC-ETC"
  }],
  "registrationPath": "slack-registration.yaml",
  "port": 8090,
  "bridge": {
    "homeserverUrl":"http://localhost:8008",
    "domain": "localhost",
    "registration": "slack-registration.yaml"
  }
}

Now generate a config file that Synapse will use, called slack-registration.yaml, by typing this command (still in the matrix-puppet-slack directory):

node index.js -r -u "http://localhost:8090"

When asked, type in the username and password you entered in the first step (when you ran the register_new_matrix_user command).

This creates a config file telling Synapse that matrix-puppet-slack is running on the local machine on port 8090.

Now, run it:

node index.js

and edit Synapse’s config file /etc/matrix-synapse/homeserver.yaml to point at the new config file you just made, by editing the line mentioning app_service_config_files so that it looks like this:

app_service_config_files: [
    "/path/to/matrix-puppet-slack/slack-registration.yaml"
]

Make sure you replace “/path/to/matrix-puppet-slack” in the above with the directory where you put matrix-puppet-slack, which is /home/yourusername/matrix-puppet-slack if you followed the instructions exactly.

Restart Synapse, and when you go back into Riot, you should see your Slack channels appear gradually (as people talk in them):

sudo service matrix-synapse restart

If not, check the output from the node index.js command, and the Synapse log file at /var/log/matrix-synapse/homeserver.log.

Install matrix-ircd

To allow connecting to Matrix from an IRC client, install matrix-ircd:

cd
git clone https://github.com/matrix-org/matrix-ircd.git
cd matrix-ircd
cargo build
cargo run -- --url "http://localhost:8008"

(Note this works on Ubuntu 18.04, but operating systems with older versions of Rust may need to get the latest first – see the matrix-ircd page for details.)

Connect from an IRC client

Now use an IRC client to connect to matrix-ird. I recommend HexChat if you’re looking for one.

matrix-ircd runs on port 5999 by default, so you should be able to connect to it from your IRC client by setting the server to localhost/5999 and using the username and password from the first step (when you ran the register_new_matrix_user command). Use plain username/password authentication.

If it works, you should see your Slack channels appear in your IRC client. If not check the logs mentioned before, and the error messages and logs from your IRC client.

What works and doesn’t work

  • Messages typed by me and others into IRC, Riot and Slack appear in all the other places, including direct messages.
  • Synapse and Riot are installed fully, so Synapse will be running after a reboot, and Riot is available in the menus, but matrix-puppet-slack and matrix-ircd are just running in a console, and have to be manually started. It should be reasonably simple to make custom systemd files to start them automatically.
  • matrix-puppet-slack crashed once and I had to restart it, so this may not be very reliable.
  • The Slack channel names look terrible in my IRC client. I think matrix-ircd needs to find the pretty names (which are displayed correctly in Riot) and use them instead of the coded names used under the cover in Matrix.
  • My first ever direct message to someone does not work, even though I can choose them and attempt to send a message. As soon as they say something to me, they appear as a channel in Riot and IRC, and I can talk back and forth no problem from then on.
  • Direct messages look like normal channels in Matrix, which means I can’t use HexChat to pop up notifications for them, so this was all pointless.

Please leave comments below if you know how I can fix any of these problems.

Conclusions

  • Matrix seems really cool
  • It is basically possible to bridge IRC-Matrix-Slack as I wanted
  • But, some remaining bugs mean I have to keep using Slack’s UI for now :-(
  • Please comment telling me how to do this better