The Primacy of Testability

Austin Bingham from Good With Computers

The job of a software architect [1] is difficult, just like almost every role in software development. They have to keep track of many subtly interacting quality attributes, often on multiple projects, any one of which may be too big or evolving too quickly to meaningfully keep in mental cache. To make matters worse, architects don't have near the level of tool support - compilers, static analysis tools, auto-completion - available to developers. They are much more reliant on experience, awareness, intuition, and heuristics.

In light of this, it's interesting and useful to consider what tools are available to help architects. In particular, I want to look at the role of testability in the architect's job, and to try to show how it can serve as a meaningful proxy for other, perhaps more important qualities in a software system. Testability is a quality that can promote the health of other desirable qualities, and it can serve as an indicator of whether these requirements are being met. The metaphor I like to use is that testability is a kind of barometer for software architects. A barometer only really tells you the air-pressure, but you can often use this to determine if there's going to be rain. Testability only really tells you how amenable your code is to useful testing, but you can often use this to help determine if your system is modular, organizationally scalable, and so forth.

What is testability anyway?

To meaningfully discuss testability as a tool, we need to establish some definition of what it means. Like "software architect", there is no perfect answer. On some level all software is testable in that you can test it. By hook or by crook you can write some code that verifies the behavior of pretty much anything with a specification. So clearly just "being testable" isn't a sufficient definition.

At the same time, it's also pretty clear that it's simpler to test some software than other. It may be easy to test for a number of reasons. Perhaps it's easy to understand, so that you have a clear understanding of how to test it thoroughly and properly. Perhaps the chunk of code is easy to instantiate without requiring a whole bunch of scaffolding and support objects. This not only saves on keystrokes but it also has other big benefits: it isolates behavior, it may mean your tests are faster, and it generally means that your tests are easier to understand and thus maintain.

If you do a little poking around you'll find that people have hit upon certain code qualities that generally influence the testability of a piece of code. [2] But in the end we don't really have a "testability-o-meter" that we can point at a piece of code. There's no accepted way to assign a "testability rating" to software that tells you if code is more or less testable than other code, or even if it's "easy to test" or "hard to test". We can sometimes get these kinds of numbers for other qualities like modularity or complexity, and things like "scalability" also lend themselves to being measured, but testability isn't (yet) in that realm.

Instead, determining if something is testable is a decision that people need to make, and it's a decision that you can only make in an informed way if you understand code. And this is why my definition of "software architect" - from a practical standpoint - includes being able to understand code well at many levels. You have to be able to recognize when, say, dependency injection could replace local object construction to reduce coupling in a system. You need to be able to spot - or at least know to be on the lookout for - circular dependencies between modules. And in general you're going to need to be able to do this not only with code that you're writing but with code that you only see in reviews or maybe only see described in documents.

Why testability?

So I've just told you that testability is hard to measure or even to define. In fact, I've told you that to make heads or tails of it you need to be an experienced programmer. On its face, then, it sounds like the cure is worse than the disease: yes, you've got complexity in your projects to deal with, but now I want to you do something even harder to make those problems go away.

On some level that's true! Gauging testability isn't simple and it's not perfect, but by targetting testability we get a couple of important benefits because testability is special.

Testability represents your first customer, your first users: your tests! Tests are very often the first place your code is used outside of your head. This means that this is where you'll first spot difficult APIs or awkward relationships that slipped through your design.

Tests force us to use code, and they force us to consider it at many different zoom levels - from unit tests to functional tests to integration tests, we get to see it all. And tests can - and should - happen early and often in the development process. This is how you get maximum benefit from them.

If you're paying attention you'll notice that I just made a significant shift in terminology. I went from talking about "testability" to "tests", from "code that can be tested" to "code with tests". I guess it's arguable that you can have testable code without actually having tests, but that seems a bit academic to me. I've gone on and on about how difficult it is to measure testability, but one of most effective and practical ways to asses testability is to simply test your code!

So for my purposes, testable code is also tested code. I won't quibble able precisely how much testing is enough, or at what level it should be done; there are plenty of other people who are happy to tell you that. [3]

But if your tests add value to your software system, then I'd wager that they exercise your code enough to highlight a lot of the software qualities for which testability is a barometer.

Qualities correlated with testability

This article lays the foundation for the rest of this series in which we'll look at various software qualities that correlate with testability. Some of these qualities, such as modularity, are directly reflected in the testability of a system. Other qualities, like performance, are supported or enabled by testability but aren't directly related to it.

[1]This is an ill-defined term, to be sure, but I'm essentially talking about the person tasked with shepherding the so-called non-functional requirements...whether their job title is "software architect" or not.
[2]Wikipedia's got a nice, non-controversial list of things like "observability", "heterogeneity", and "understandability", and all of these things certainly would influence how easy or hard it is to test a piece of code.
[3]See for example TDD, BDD, or James Coplien's thoughts.

Series: The Primacy of Testability

Austin Bingham from Good With Computers

In this series we look at how software architects - or really anyone involved in creating software - can use testability to help manage other quality attributes. From modularity to performance to the SOLID principles, testability can act as a proxy and an enabler for many of the cross-cutting, interacting concerns that architects need to shepherd. Over several articles we'll explore testability's relationship to these qualities, and we'll see how paying attention to testability can help simplify the job of managing them.

How to write Boost.Python type converters

Austin Bingham from Good With Computers

Boost.Python [1] makes it possible to write C++ that "feels" like Python. The library is powerful and sometimes subtle. This is as compared with the Python C API, where the experience is very far removed from writing Python code.

Part of making C++ feel more like Python is allowing natural assignment of C++ objects to Python variables. For instance, assigning an standard library string to a Python object looks like this:

// Create a C++ string
std::string msg("Hello, Python");

// Assign it to a python object
boost::python::object py_msg = msg;

Likewise (though somewhat less naturally), it is also important to be able to extract C++ objects from Python objects. Boost.Python provides the extract [2] type for this:

boost::python::object obj = ... ;
std::string msg = boost::python::extract(obj);

To allow this kind of natural assignment, Boost.Python provides a system for registering converters between the languages. Unfortunately, the Boost.Python documentation does a pretty poor job of describing how to write them. A bit of searching on the internet will turn up a few links. [3]

While these are fine (and, in truth, are the basis for what I know about the conversion system), they are not as explicit as I would like.

So, in an effort to clarify the conversion system both for myself and (hopefully) others, I wrote this little primer. I'll step through a full example showing how to write converters for Qt's QString [4] class. In the end, you should have all the information you need to write and register your own converters.

Converting QString

A Boost.Python type converter consists of two major parts. The first part, which is generally the simpler of the two, converts a C++ type into a Python type. I'll refer to this as the to-python converter. The second part converts a Python object into a C++ type. I'll refer to this as the from-python converter.

In order to have your converters be used at runtime, the Boost.Python framework requires you to register them. The Boost.Python API provides separate methods for registering to-python and from-python converters. Because of this, you are free to provide conversion in only one direction for a type if you so choose.

Note that, for certain elements of what I'm about to describe, there is more than one way to do things. For example, in some cases where I choose to use static member functions, you could also use free functions. I won't point these out, but if you wear your C++ thinking-cap you should be able to see what is mandatory and what isn't.

To-python Converters

A to-python converter converts a C++ type to a Python object. From an API perspective, a to-python converter is used any time that you construct a boost::python::object [5] from another C++ type. For example:

// Construct object from an int
boost::python::object int_obj(42);

// Construct object from a string
boost::python::object str_obj = std::string("llama");

// Construct object from a user-defined type
Foo foo;
boost::python::object foo_obj(foo);

You implement a to-python converter using a struct with static member function named convert(), which takes the C++ object to be converted as its argument, and it returns a PyObject*. A to-python converter for QStrings looks like this:

/* to-python convert to QStrings */
struct QString_to_python_str
{
    static PyObject* convert(QString const& s)
    {
        return boost::python::incref(
            boost::python::object(
                s.toLatin1().constData()).ptr());
    }
};

The crux what this does is as follows:

  1. Extract the QString's underlying character data using toLatin1().constData()
  2. Construct a boost::python::object with the character data
  3. Retrieve the boost::python::object's PyObject* with ptr()
  4. Increment the reference count on the PyObject* and return that pointer.

That last step bears a little explanation. Suppose that you didn't increment the reference count on the returned pointer. As soon as the function returned, the boost::python::object in the function would destruct, thereby reducing the ref-count to zero. When the PyObject's reference count goes to zero, Python will consider the object dead and it may be garbage-collected, meaning you would return a deallocated object from convert().

Once you've written the to-python converter for a type, you need to register it with Boost.Python's runtime. You do this with the aptly-named to_python_converter [6] template:

// register the QString-to-python converter
boost::python::to_python_converter<
    QString,
    QString_to_python_str>()

The first template parameter is the C++ type for which you're registering a converter. The second is the converter struct. Notice that this registration process is done at runtime; you need to call the registration functions before you try to do any custom type converting.

From-python Converters

From-python converters are slightly more complex because, beyond simply providing a function to convert from Python to C++, they also have to provide a function that determines if a Python type can safely be converted to the requested C++ type. Likewise, they often require more knowledge of the Python C API.

From-python converters are used whenever Boost.Python's extract type is called. For example:

// get an int from a python object
int x = boost::python::extract(int_obj);

// get an STL string from a python object
std::string s = boost::python::extract(str_obj);

// get a user-defined type from a python object
Foo foo = boost::python::extract(foo_obj);

The recipe I use for creating from-python converters is similar to to-python converters: create a struct with some static methods and register those with the Boost.Python runtime system.

The first method you'll need to define is used to determine whether an arbitrary Python object is convertible to the type you want to extract. If the conversion is OK, this function should return the PyObject*; otherwise, it should return NULL. So, for QStrings you would write:

struct QString_from_python_str
{

    . . .

    // Determine if obj_ptr can be converted in a QString
    static void* convertible(PyObject* obj_ptr)
    {
        if (!PyString_Check(obj_ptr)) return 0;
        return obj_ptr;
    }

    . . .

};

This simply says that a PyObject* can be converted to a QString if it is a Python string.

The second method you'll need to write does the actual conversion. The primary trick in this method is that Boost.Python will provide you with a chunk of memory into which you must in-place construct your new C++ object. All of the funny "rvalue_from_python" stuff just has to do with Boost.Python's method for providing you with that memory chunk:

struct QString_from_python_str
{

    . . .

    // Convert obj_ptr into a QString
    static void construct(
        PyObject* obj_ptr,
        boost::python::converter::rvalue_from_python_stage1_data* data)
    {
        // Extract the character data from the python string
        const char* value = PyString_AsString(obj_ptr);

        // Verify that obj_ptr is a string (should be ensured by
        convertible())
        assert(value);

        // Grab pointer to memory into which to construct the new QString
        void* storage = (
            (boost::python::converter::rvalue_from_python_storage*)
            data)->storage.bytes;

        // in-place construct the new QString using the character data
        // extraced from the python object
        new (storage) QString(value);

        // Stash the memory chunk pointer for later use by boost.python
        data->convertible = storage;
    }

  . . .

};

The final step for from-python converters is, of course, to register the converter. To do this, you use boost::python::converter::registry::push_back(). [7] The first argument is a pointer to the function which tests for convertibility, the second is a pointer to the conversion function, and the third is a boost::python::type_id for the C++ type. In this case, we'll put the registration into the constructor for the struct we've been building up:

struct QString_from_python_str
{
    QString_from_python_str()
    {
        boost::python::converter::registry::push_back(
            &convertible,
            &construct,
            boost::python::type_id());
    }

    . . .

};

Now, if you simply construct a single QString_from_python_str object in your initialization code (just like you how you called to_python_converter() for the to-python registration), conversion from Python strings to QString will be enabled.

Taking a reference to the PyObject in convert()

One gotcha to be aware of in your construct() function is that the PyObject argument is a 'borrowed' reference. That is, its reference count has not already been incremented for you. [8] If you plan to keep a reference to that object, you must use Boost.Python's borrowed construct. For example:

class MyClass
{
public:
    MyClass(boost::python::object obj) : obj_ (obj) {}

private:
    boost::python::object obj_;
};

struct MyClass_from_python
{
    . . .

    static void construct(
        PyObject* obj_ptr,
        boost::python::converter::rvalue_from_python_stage1_data* data)
    {
        using namespace boost::python;

        void* storage = (
            (converter::rvalue_from_python_storage*)
                data)->storage.bytes;

        // Use borrowed to construct the object so that a reference
        // count will be properly handled.
        handle<> hndl(borrowed(obj_ptr));
        new (storage) MyClass(object(hndl));

        data->convertible = storage;
    }
};

Failing to use borrowed() in this situation will generally lead to memory corruption and/or garbage collection errors in the Python runtime.

There are a number of useful resources on the web for finding more information on Boost.Python objects, handles, and reference counting. [9]

When converters don't exist

Finally, a cautionary note. The Boost.Python type-conversion system works well, not only at the job of moving objects across the C++-python languages barrier, but at making code easier to read and understand. You must always keep in mind, though, this comes at the cost of very little compile-time checking.

That is, the boost::python::object copy-constructor is templatized and accepts any type without complaint. This means that your code will compile just fine even if you're constructing boost::python::object s from types that have no registered converter. At runtime these constructors will find that they have no converter for the requested type, and this will result in exceptions.

These exceptions [10] will tend to happen in unexpected places, and you could spend quite a bit of time trying to figure them out. I say all of this so that maybe, when you encounter strange exceptions when using Boost.Python, you'll remember to check that your converters are registered first. Hopefully it'll save you some time.

Resources

Boost.Python is fairly complex and can be difficult to understand all at once. Here are few more useful resources that might help you come up to speed on this useful technology:

  • This IPython notebook-based tutorial covers a lot of the major (and some of the more obscure) topics in Boost.Python.
  • The Boost.Python wiki contains a lot of collected Boost.Python knowledge.
  • And of course, the Boost.Python documentation itself is very useful.

Appendix: Full code for QString converter

struct QString_to_python_str
{
    static PyObject* convert(QString const& s)
    {
        return boost::python::incref(
            boost::python::object(
                s.toLatin1().constData()).ptr());
    }
};

struct QString_from_python_str
{
    QString_from_python_str()
    {
        boost::python::converter::registry::push_back(
            &convertible,
            &construct,
            boost::python::type_id());
    }

    // Determine if obj_ptr can be converted in a QString
    static void* convertible(PyObject* obj_ptr)
    {
        if (!PyString_Check(obj_ptr)) return 0;
        return obj_ptr;
    }

    // Convert obj_ptr into a QString
    static void construct(
        PyObject* obj_ptr,
        boost::python::converter::rvalue_from_python_stage1_data* data)
    {
        // Extract the character data from the python string
        const char* value = PyString_AsString(obj_ptr);

        // Verify that obj_ptr is a string (should be ensured by convertible())
        assert(value);

        // Grab pointer to memory into which to construct the new QString
        void* storage = (
            (boost::python::converter::rvalue_from_python_storage*)
            data)->storage.bytes;

        // in-place construct the new QString using the character data
        // extraced from the python object
        new (storage) QString(value);

        // Stash the memory chunk pointer for later use by boost.python
        data->convertible = storage;
    }
};

void initializeConverters()
{
    using namespace boost::python;

    // register the to-python converter
    to_python_converter<
        QString,
        QString_to_python_str>();

    // register the from-python converter
    QString_from_python_str();
}
[1]The Boost.Python homepage.
[2]boost::python::extract<> documentation.
[3]For example the Boost.Python FAQ.
[4]The Qt QString documentation.
[5]The boost::python::object documentation.
[6]The to_python_converter documentation.
[7]The boost::python::converter::registry documentation.
[8]Python reference counting details.
[9]For example, this discussion from the C++-sig discussion list, the Boost.Python documentation, and David Abrahams' guidelines. for handle<> on the Python wiki.))
[10]Boost.Python uniformly uses boost::python::error_already_set to communicate exceptions from Python to C++..

Speaking: Words in Code

Pete Goodliffe from Pete Goodliffe

The slides from my ACCU 2014 talk, Words in Code, are available on SlideShare, from http://www.slideshare.net/petegoodliffe/words-in-code.

Here's the synopsis:

As software developers we do not just write code. We write many, many words too.

We write documentation, comments, manuals, specifications, technical articles, wiki documentation, and more. Maybe even magazine articles and books.

This talk discusses some practicalities of writing well, both stylistically and practically. We'll talk about prose, but also about the right "geek" way of writing, the storage formats, toolchains, and the storage of our words.

We'll cover:
- writing style
- what's appropriate: what to write what not to write
- keeping track: "source control" for words
- toolchains: what toolsets to use to write and prepare output
- markup languages vs "wysiwyg" tools
- sharing your words with non-geeks

At the end of this talk, you'll have a good idea how to put together an example "document toolchain" taking source-controlled words in a humane markup style, and creating high-quality HTML, PDF (fully styled, print-ready) ePub and Kindle output, as well as Word-friendly versions.

Top four JavaZone 2013 talk – The Unreasonable Effectiveness of Dynamic Typing

Rob Smallshire from Good With Computers

I'm very happy to see that my talk on The Unreasonable Effectiveness of Dynamic Typing was rated fourth of all the talks in the show. Thanks to everyone who attended and voted.

This talk is perhaps deliberately provocative, but only with the intention of provoking critical thinking and empiricism around the tools we use. I'm genuinely curious as to why programs in dynamic languages are as reliable as they are, although I confess I don't yet have many of the answers.

New Book: Becoming a Better Programmer

Pete Goodliffe from Pete Goodliffe

After many years of gestation my latest book is available for purchase as an early-access pre-release.

Called Becoming a Better Programmer, it is a handbook for people who are about code.

This early access edition already contains 14 chapters, and there are many more coming. There is a free "sample" version available so you get a taster of what you'll be purchasing.

As a pre-release, it's available at an introductory price. The price will go steadily upwards as the book nears completion. Buy now to enjoy the best value! (That's the sales pitch - I suck at that kind of thing.)

Get it from gum.co/becomingbetter. Join the book discussion here: moot.it/becomingbetter.

It would genuinely love to hear any feedback, praise or criticism that will help improve the book. Suggestions for topics to cover are also of real interest.

My honest hope is that this book does just what it says on the cover: helps many developers improve their skills, to become more productive programmers.

Buy it now!

Mutable structs in C#

Frances Buontempo from BuontempoConsulting

We know what this does, right?

    struct Pricer
    {
        public double Price;
        public long Size;

        public void AddExecution(long lastSize, double lastPrice)
        {
            Price = (Price * Size + lastSize * lastPrice) / (Size + lastSize);
            Size += lastSize;
        }
    }

    class PriceData
    {
        public Pricer pricer;
    }

    class Program
    {
        static void Main(string[] args)
        {
            Pricer price = new Pricer{Price = 0.0, Size = 0};
            for (int i = 0; i < 5; ++i)
            {
                Console.WriteLine("{0} {1}", price.Price, price.Size);
                price.AddExecution(1, 2.5 * (i + 1));
            }

            for (int i = 0; i < 5; ++i)
            {
                Console.WriteLine("{0} {1}", price.Price, price.Size);
                price.Price= 2.5 * (i + 1);
            }

            PriceData priceData = new PriceData();
            priceData.pricer = price;
            for (int i = 0; i < 5; ++i)
            {
                Console.WriteLine("{0} {1}", price.Price, price.Size);
                priceData.pricer.AddExecution(1, 2.5 * (i + 1));
            }
       }
    }

Eric Lippert tells us about mutating *readonly* structs: http://blogs.msdn.com/b/ericlippert/archive/2008/05/14/mutating-readonly-structs.aspx but even non-readonly structs can get us in a mess.

Speaking: Words in Code (ACCU 2014)

Pete Goodliffe from Pete Goodliffe

I'll be speaking at this year's excellent ACCU Conference 2014.

This year my talk is: Words in Code, a technical (and not so technical) appraisal of how developers write. It's a practical distillation of my fourteen years as a magazine columnist, multiple book projects, and more.

Come and enjoy it on Thursday 10th April at 10am. The conference's earlybird booking deadline is February the 14th. ACCU is one of the highlights of my developer year - it's a truly excellent conference. If you've not considered going, check it out!

The full synopsis is available on the session page:
As software developers we do not just write code. We write many, many words too.
We write documentation, comments, manuals, specifications, technical articles, wiki documentation, and more. Maybe even magazine articles and books.
This talk discusses some practicalities of writing well, both stylistically and practically. We'll talk about prose, but also about the right "geek" way of writing, the storage formats, toolchains, and the storage of our words.
We'll cover:
  • writing style
  • what's appropriate: what to write what not to write
  • keeping track: "source control" for words
  • toolchains: what toolsets to use to write and prepare output
  • markup languages vs "wysiwyg" tools
  • sharing your words with non-geeks
At the end of this talk, you'll have a good idea how to put together an example "document toolchain" taking source-controlled words in a humane markup style, and creating high-quality HTML, PDF (fully styled, print-ready) ePub and Kindle output, as well as Word-friendly versions.

Pdb ftw

Frances Buontempo from BuontempoConsulting

pdb - python debugger

If a script throws an exception, try running it in the debugger
python -m pdb myscript.py
For example given the following (rubbish) python program
C:\Dev\src>cat bad.py
def naughty():
raise Exception()
naughty()

if we just run it the exception tries to escape ...
C:\Dev\src>python bad.py
Traceback (most recent call last):
File "bad.py", line 4, in <module>
naughty()
File "bad.py", line 2, in naughty
raise Exception()
Exception

So, we know what line the problem's on. We could change the script to see what's going on using
import pdb; pdb.set_trace()
or we could run it under a debugger.

Run script through debugger

Invoke the pdb module in python and send it your script, e.g.
python -m pdb bad.py
This gives a (Pdb) prompt to type instructions in to:
> c:\dev\src\bad.py(1)<module>()
-> def naughty():
(Pdb)
Type 'c' for continue - it will halt when it gets an exception, as follows
(Pdb) c
Traceback (most recent call last):
File "C:\Python27\lib\pdb.py", line 1314, in main
pdb._runscript(mainpyfile)
File "C:\Python27\lib\pdb.py", line 1233, in _runscript
self.run(statement)
File "C:\Python27\lib\bdb.py", line 387, in run
exec cmd in globals, locals
File "<string>", line 1, in <module>
File "bad.py", line 1, in <module>
def naughty():
File "bad.py", line 2, in naughty
raise Exception()
Exception
Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program
> c:\dev\src\bad.py(2)naughty()
-> raise Exception()
(Pdb)
At this point
 (Pdb) p <variable_name>
can be used to inspect variables, and
 (Pdb) a
shows any arguments to a function.

Main Pdb commands

Type 'q' for quit, 's' to step (maybe into another function) or 'n' to move on to the next line.
'w' shows where you are in the stack, 'u' moves up and 'd' moves down.
b(reak) [[filename:]lineno  sets a breakpoint.
There are more details in the manual: http://docs.python.org/2/library/pdb.html

Virtual functions in constructors and destructors

Frances Buontempo from BuontempoConsulting

Interview question.

What does this do?

class Base
{
public:
Base()
{
log();
}
virtual ~Base()
{
log();
}

//virtual void log() = 0;//note this compiles but doesn't link
virtual void log()
{
std::cout << "Base\n";
}
};

class Derived : public Base
{
public:
Derived()
{
log();
}
~Derived()
{
log();
}
virtual void log()
{
std::cout << "Derived\n";
}
};

int main()
{
Derived d;
}

I half remembered this http://www.artima.com/cppsource/nevercall.html so wasn't sure.