Pattern: Single CrUD Transaction

Paul Grenyer from Paul Grenyer

Software patterns have their roots in architecture. In 1978, Christopher Alexander published a book called ‘A Pattern Language: Towns, Buildings, Construction‘ (ISBN-13: 978-0195019193) about the patterns he’d discovered designing buildings. A pattern can be thought of as a tried and tested way of doing something which can be applied in different contexts.  Think about how the Observer or Visitor pattern is implemented across languages such as Java, Ruby and JavaScript, where the different language idioms dictate slightly different implementations of the same basic pattern.

Software Patterns became popular with the publishing of the Gang of Four book, “Design patterns: elements of reusable object-oriented software” (ISBN-13: 978-0201633610) in 1994. It contains a number of patterns, most of which every developer should know, even if it’s to know to avoid the likes Singleton. However, these aren’t the only patterns! Indeed, patterns are not created, they are discovered and documented. Whole conferences are dedicated to software patterns (http://www.europlop.net/), where delegates are encouraged to bring their pattern write-ups for appraisal by their peers and the experts.

In 2000 I joined the ACCU, a group for programmers who strive for better software. I was encouraged by another member to write for the group’s magazine, but I didn’t think I’d have anything to contribute that someone better hadn’t already thought of and written about. As I gained experience I found I had quite a lot to write about and to challenge.

In the same way you’d have thought that 23 years after the Gang of Four book most if not all of the software patterns had been discovered and documented. However, it appears not and I was very surprised to find that what I’m calling the “Single CrUD Transaction” pattern, although used by many, doesn’t appear to have been written up anywhere publically. I checked with industry experts and they weren’t aware of it being written-up either.

This is my first software pattern write up and where better to share it for the first time than Norfolk Developers Magazine?

Name

Single CrUD Transaction

Intent

To create, update and delete items in a datastore within a single transaction.

Problem

Sometimes it’s necessary to create, update and delete items in a datastore in a single transaction. Traditional web applications support create, update and delete in separate transactions and require the page to be reloaded between each action.

Modern web applications allow the items of a list to be created, updated and deleted in a browser without any interaction with the server or the underlying datastore. Therefore when the list is sent to the server side it must determine which items are new, which already exist and must be updated and which have been removed from the list and must be deleted.

One simple solution is to delete all of the items from the datastore and simply replace them with the list of line items passed from the browser to the server. There are at least two potential drawbacks with this approach:

  1. If the datastore (such as a relational database) uses unique, numerical ids to identify each item in the list, the size of the ids can become very big, very quickly.
  2. If the datastore (such as a relational database) has other data which references the ids of the items in the list, the items cannot be deleted without breaking the referential integrity.

Solution

The Single CrUD Transaction pattern gets around these drawbacks by performing three operations within a single transaction:

  1. Delete all of the list items from the datastore whose ids are not in the list passed from the browser to the server.
  2. Update each of the items in the datastore whose ids match ids in the list passed from the browser to the server.
  3. Create new items in the datastore for each item in the list passed from the browser to the server which do not yet have ids.
Each action is executed within a single transaction so that if any individual action fails the list is returned to its original state.

Applicability

Use the Single CrUD transaction pattern when:

  • Datastores cannot have new items added, existing items updated and/or items removed in separate transactions.
  • Creating new ids for each item in the list each time the datastore is modified is expensive or cumbersome.
  • Removing all the items of a list from a datastore and recreating the list in the datastore breaks referential integrity.

Advantages and Disadvantages

Advantages


  • Entire update happens within a single transaction.

Disadvantages


  • Three separate calls to the datastore within a single transaction.


Event: Burkhard Kloss on The Ethics of Software & Panel: Talking to the clouds

Paul Grenyer from Paul Grenyer


Event: Burkhard Kloss on The Ethics of Software & Panel: Talking to the clouds

When: 6 November 2017 @ 6.30pm

Where: Whitespace, 2nd Floor, St James Mill, Whitefriars, Norwich, NR3 1TN

RSVP: https://www.meetup.com/preview/Norfolk-Developers-NorDev/events/239616865

The Ethics of Software - some practical considerations
Burkhard Kloss
@georgebernhard

As Uncle Bob pointed out, software is everywhere, and without software, nothing works.

That gives us great power, and – as we all know – with great power comes great responsibility.

We have to make choices every day that affect others, sometimes in subtle and non-intuitive ways. To mention just a few:

  • What logs should we capture?
  • How does that change if we have to hand them over to the government?
  • Are our hiring practices fair? Are we sure about that?
  • Is there bias in our algorithms that unfairly disadvantages some groups of people?
  • Is the core function of our software ethical? How about if it’s deliberately misused?

I hope to raise a few of these questions, not to provide answers – I don’t have any – but to stimulate debate.

Burhard Kloss

I only came to England to walk the Pennine Way… 25 years later I still haven’t done it. I did, though, get round to starting an AI company (spectacularly unsuccessful), joining another startup long before it was cool, learning C++, and spending a lot of time on trading floors building systems for complex derivatives. Sometimes hands on, sometimes managing people. Somewhere along the way I realised you can do cool stuff quickly in Python, and I’ve never lost my fascination with making machines smarter.


Panel Discussion: Talking to the clouds

Conversational computing, the ability to talk to, an interact with a computer via voice, is becoming more and more prevalent. Most of us now have access to an intelligent assistant like Siri or Alexa, and how we interact with the devices is being defined. But are we going in the right direction. Should we be treating these devices as just "dumb computers", or should we speak to them as we do to other people?

Our panel of experts will discuss this topic with input from the audience as we look at one of the many areas where the question is not "can we?", but "should we?".

A review: nor(DEV):biz October 2017

Paul Grenyer from Paul Grenyer

The idea “Networking” strikes fear into the heart of many techies, but Norfolk Developers Business or nor(DEV):biz is different. The idea behind the monthly meetings over dinner at the Library Restaurant is to get tech companies in Norwich and Norfolk talking to each other and referring business between themselves and from external parties. It’s not just about tech companies though, we also invite people from academia (City College Norwich was represented tonight and the UEA attended the very first event), those running complementary business (such as accountants, lawyers, recruiters, etc), those looking to engage software companies and even those looking to be employed by them.

"It was relaxed and much like having a good dinner with a selection of your wittiest and most worldly wise friends !"
- Chris Sargisson, CEO Norfolk Chamber

Norwich has networking events coming out of its ears. nor(DEV):biz is different, not just because of the tech focus, but also because of the people who attend. Over the years Norfolk Developers has attracted the biggest personalities in the community (that’s you Dom Davis!) including many senior tech business owners. Yes, everyone has their one minute to speak to the group about who they are, what they do and what they’re looking for, but there’s no bell when your time is up and there’s humour, passion and interaction from the entire group. This isn’t just networking, this is building, bonding and rapport with people you may well work with in the future. It’s more than that, this is fun and raucous and entertaining. It’s a night out with friends rather than a pressure cooker for sales.

“Great event .Who would have thought that by having dinner with a bunch of techies I would learn that tomato ketchup is the best thing for smelly dog issues.. it just shows, never judge a book by its cover.” 
- Chris Marsh, AT&A BUSINESS INSURANCE BROKERS

At each nor(DEV):biz a member has the opportunity, not the obligation, to do a 15 minute spotlight. This is beyond their one minuter and the opportunity to give a more indepth overview of their business or something they are passionate about.

"Great evening arranged by Paul Grenyer and Dom Davis for the Norfolk Developers group. My highlight was Nikki and Tom Bool integrating the basics of Dog Training skills with leading a team in the workplace!"
- Anthony Pryke, Barclays 

For this, the fourth nor(DEV):biz, the spotlight was given by Nikki and Tom Bool. Nikki is a puppy trainer, while Tom runs a language services business, specialising in helping businesses market themselves and grow internationally. Nikki explained how to use positive reinforcement to encourage the right behavior in puppies, with some hilarious anecdotes. Tom went on to describe how similar techniques can be used to help foster the desired behavior in the people you work with. The spotlight fulfilled my favorite criteria by being both informative and entertaining.

"What a smashing group of people and thoroughly enjoyable puppy behaviours reflection on office management. "
 - Mike Peters, Evoke Systems

You know you’re onto a winner when you have to encourage people to leave and the conversation has moved from the table to a huddle by the doorway.  I’m already looking forward to the next nor(DEV):biz in November where we’re hoping to hear from Laura Flood and Anietie Ukpabio of City College, Norwich, about the young people they’re training to be software engineers.

If you’d like to attend nor(DEV):biz, please drop Paul an email on paul@norfolkdevelopers.com.


Norfolk Developers Magazine: AI

Paul Grenyer from Paul Grenyer

The first issue of the Norfolk Developers magazine (outside  of a conference) is out now and free to download!

This issue focuses on A.I., a topic we thought a good one to kick off with as everyone has an opinion about Artificial Intelligence, it affects our daily lives (see Dom Davis’ column about arguments with Alexa) and it gave us an excuse to use the awesome robot image on the front cover too.

It is because of people like you that we have  such a thriving tech community in Norwich and Norfolk, a community that has turned our Fine City into a Tech City. Without this passionate and dedicated community, there would be no reason for writers to contribute to this magazine, there would be no market for local companies to place adverts for, there would be no events to report from. Mainly, there would be no one to read it so thank you

How much will my software cost?

Paul Grenyer from Paul Grenyer


The question we get asked the second most when speaking to clients and potential clients is “how much will my bespoke software cost to build?” This is extremely difficult to answer without lots of detail and even then the complexities of software development, the complexity of client requirements and clients changing needs over the course of a project make an accurate estimate challenging.

For this reason, most software development companies shy away from including prices on their website. In fact we checked the websites of a number of our competitors and the closest we found was one who offers a range of fee options from fixed price to a daily rate and a couple who ask for your budget when contacting them for more information. As a client, until you get that first email response, phone call or face-to-face meeting you’re no closer to understanding how much your software will cost. Even then it may be some time before you are any the wiser.

We can’t help you understand how much your project will cost until we speak to you. What we can tell you is how much projects have cost our existing clients. We’ve broken the figures down into the types of services we provide, the minimum project cost, the maximum project cost, the average project costs and where in the range most of the projects sit:

* All values are approximate, exclude VAT and are correct as of August 2017

To start investigating how your business problem could be solved with a bespoke application, please contact us for a chat

My Fantasy Gig: Polish Death Metal

Paul Grenyer from Paul Grenyer



It’s no secret that I like death metal. Three of my favorite death metal bands are all from Poland. I’ve been lucky enough to see all of them at least twice individually, but never together. I’ve often wondered why they haven’t all toured together. I’ve never been to Poland either so I’d settle for seeing them all together in their home country.

Decapitated

Opening the show I’d have Decapitated a technical death metal band. Their style, as you would expect, is heavy and progressive. While currently the smaller and less well know of the three bands on this bill, Decapitated are growing in popularity and are poised to step into the shoes of metal titans such as Lamb of God.

After getting into Vader and Behemoth I was really excited to read about another Polish death metal band and I wasn’t disappointed, especially as I also have a soft spot for progressive metal. Often with metal bands who have been around a while, their back catalogue is noisy and unpalatable. Not the case with Decapitated. They’re tight, aggressive and heavy from the first album through to the more recent ones. I’ve seen them play three times now (once even in Norwich) and their live performance demonstrates their skill as musicians.

Behemoth

I’d have Behemoth second on the bill. By far the biggest of the three bands, Behemoth are one of the best metal bands around at the moment. Currently (summer 2017) they are touring the US with Lamb of God and Slayer. I’ve seen them play three times. Once to about 10 people at a club in Bradford and twice to thousands at Bloodstock.

Their singer is often in the press, in Poland and around the world. He famously burned a bible on stage in Poland and was promptly arrested. Later he was diagnosed with and beat cancer.

In the early days Behemoth’s style of ‘blackened death metal’ was heavily influenced by US death metal giants Morbid Angel, but much more palatable. That said they’ve improved on almost every album. Their 2013 album the Satanist is a masterpiece of modern metal. Probably their least heavy album to date, but still crushing.

Vader

Headlining I’d have Vader. I’d describe them as the godfathers of Polish death metal. While not as popular or well selling as Behemoth, they belong at the top. Vader play more traditional death metal, sometimes with trashy tinges. I really struggled to get into their back catalogue. I just wasn’t ready, but every album is superb.

I’ve seen them twice, both times in small clubs. Their sound wasn’t the best, but being a huge fan I put that down to the PA in the clubs. I am sure that atop such a fine bill, they would shine and show what they can really do.

Of course the final encore would comprise of all three bands playing a metal classic together.

A Review: Express in Action

Paul Grenyer from Paul Grenyer

Express in Action: Node applications with Express and its companion tools

By Evan Hahn
ISBN: 978-1617292422

This is another excellent JavaScript book from Manning. It contains a great introduction to Express.js and I wish I’d read it sooner as it explains a lot of things about Express.js and how to use it, as well as the tools surrounding it and Node.js, which I had previously worked out for myself. If you’re thinking of writing a web application, especially one in JavaScript, I recommend you read this book first.

The book is far from perfect. It could have been a lot shorter. There is a fair amount of repetition and the chatty style makes it overly verbose and irritating in many places.  The author tries to cover too much and goes beyond Express.js unnecessarily in a few places. However, given that, it’s still not a huge book and quite easy to read.

A review: JavaScript the Good Parts

Paul Grenyer from Paul Grenyer

By Douglas Crockford
ISBN: 978-0596517748

Every JavaScript developer with a pre-existing working knowledge of JavaScript should read this book. JavaScript is a powerful and varied language, but it was developed in a hurry and there’s plenty wrong with it. This book outlines the good bits of the language and highlights the bad bits and the bits you should just avoid. There’s also a fair amount about the author’s JSLint project in the appendices.

This book was written in 2008 and probably needs updating. It’s hard going in places and the diagrams did little to nothing to help my understanding. I’ve come away still wondering about new and constructors, but I know I just need to review them again when I need them and it’ll get clearer.  I’m still not sure which function declaration syntax is best, but I’m not sure it matters too much.


Getting to the route of the problem

Paul Grenyer from Paul Grenyer

In 2016, Venkat Subramaniam wrote an incredible book called ‘Test-Driving JavaScript Applications’ which, along with JavaScript tools such as Mocha, Istanbul, Prettier and Eslint, have made me fall in love with JavaScript and Node.js (well for UI development anyway). JavaScript isn’t a proper language, right? For a long time I argued not, because the tools weren’t available to develop software with unit tests, static analysis and code coverage. This has changed and now I’m starting to take JavaScript seriously, even beyond jazzing up a web based UI. I’m almost over the lack of static typing.

I’m currently using Express.js, a web framework for Node.js, a lot and Venkat includes a section on testing Express.js routes in his book. They’re a bit like controllers in the Modal View Controllers pattern:

router.get('/', function(req, res, next) {
task.all(function(err, tasks) {
res.send(tasks);
});
});

Venkat’s example test looks like this:

it('should register uri / for get', function(done) {
    // ...        

    var registeredCallback = router.get.firstCall.args[1];
    registeredCallback(req, res);
});

I’ve left out some mocking and other boilerplate for brevity and so that we can concentrate on the one bit I don’t like. Venkat describes the test in full detail in his book.  Take another look at this line:

    var registeredCallback = router.get.firstCall.args[1];

What it does is get the second argument for the first get route declared with the router. That’s what is returned by firstCall, the first declared route. So if there is more than one get route declared with the router and at some point you change the order in which they are declared or you declare another get route in-between, the test will break. It’s brittle.

In fact it’s worse. To get the second get route you’d use secondCall and so on. So although it’s probably a very large number, there are a finite number of get routes you can get from the router with this method. For me this rang alarm bells.
Google suggested this is the way that everyone is doing it. It appears to be the standard practice. It doesn’t sit at all well with me. I’d much rather be able to look up route in the router by its path. After a while printing all sorts of things to the console to find out the data structures, I was able to develop this:

var rh = {
    findGet: function(router, path) {
        for (var i = 0; i < router.get.args.length; i++)
            if (router.get.args[i][0] === path)
                return router.get.args[i];

        return null;
    },

   // ..
};

module.exports = {
    execGet: function(router, path, req, res) {
        var get = rh.findGet(router, path);
        if (get != null) get[1](req, res);
    },

    // ..
};

The findGet function takes a router and the path to test and returns all of the arguments declared for that route or null if it’s not found.  The execGet function uses those arguments to execute the route, meaning that the test now becomes:

it('should register uri / for get', function(done) {
        // ...

        execGet(router, '/', req, res);
    });

Which is not only far more expressive, but less brittle and less code per test. It means that the declaration order of the routes for the router no longer matters. Of course similar functions can be added to facilitate testing post, put and delete.

I wanted to write this up as I couldn’t find any other solution with Google. Hopefully it will encourage developers to write more tests for Express routes as they become easier and less brittle.


Node.js the Right Way: Practical Server-Side JavaScript That Scales

Paul Grenyer from Paul Grenyer

By Jim R. Wilson
ISBN-13: 978-1937785734

Node.js the Right Way is a fantastic little book. It’s a small book (but then it’s Pragmatic exPress) and it doesn’t go into anything in much detail, but then that’s what makes it fantastic. It gives a useful and practical overview of writing Node.js server side applications and explains many of the tools and JavaScript patterns which will be useful to Node.js programmers.

It starts off with examples of manipulating the local file system using Node.js. This struck me a little odd as the only thing I tend to use the local file system for is reading configuration files. If I need to write a file I tend to put it in Amazon S3. However, this is genius and looking at how to manipulate the filesystem gives some useful insights into Node.js programming.

The book then goes on to look at networking with sockets, something which is often neglected in a world where we expect everything to be RESTful. There’s then a tour through scalable messaging, including clustering, how to access databases and how to write web services, including JavaScript promises and generators!

The final chapter covers writing a web application with a single page front end and authentication. This is the only place the book falls down. Too much is covered in two short a chapter. It’s still quite useful though.

This is not a book for a novice JavaScript or even a novice Node.js developer, but for once a little knowledge is not a dangerous thing and Node.js the Right Way will help increase that knowledge. It even led me to believe JavaScript might actually be the future.