Optional streaming

Phil Nash from level of indirection

Catch has a number of macros that allow values of arbitrary types to be streamed into an ostringstream. The canonical example is the INFO macro:

INFO( "There were " << bottles.size() << " green bottles, hanging on the wall" );

This macro builds up a string that will be passed to the next assertion to be included as an annotation. Note that, unlike with a naked ostringstream there is no leading <<. This makes it clean and uncluttered when you just want to log a single value (such as a string), for example:

INFO( "Weirdness" );
The obvious way to do this is for the macro to provide the leading << prior to its argument. Conceptually something like this:
#define INFO( log ) { \
	std::ostringstream oss; \
	oss << log; \
	useTheString( oss.str() ); 
}

This all works quite nicely. But there are a few other macros that use this idiom, too: WARN, SUCCEED and FAIL.

The last two are of interest because the logging behaviour is more of a secondary concern. The primary behaviour is to appear like a passing or failing assertion, respectively, without the need to actually assert on anything. SUCCEED can be useful if you otherwise have no assertions in a test and you don't want to see warnings about it. FAIL is useful if the situation that leads to the failure is not captured in an expression for some reason. It can also be useful to force a test to fail, perhaps as a placeholder. These are useful macros to have available, but they are not often needed in practice. So when they are it's nice to be able to annotate their useage inline - hence the streamed argument.

This is all well and good. But I've found there are still enough cases where I don't want to annotate that having to pass an empty string or make something up is a little annoying. I also use a similar idiom in other projects where it would be nice to be able to make the stream completely optional.

This is not as easy as it sounds, though. The first, and most obvious, issue is that this requires support for variadic macros. Catch has made use of variadic macros, where available, for some time now. In theory they are available to any C++11 compiler. In practice most, if not all, compilers that support any reasonable chunk of C++11 support variadic macros - and most supported them as an extension even before that. That's certainly true of Visual C++, GCC and Clang.

The technically more interesting problem, though, is dealing with that initial <<. Remember the first << is being supplied inside the macro. It will still be there even if the caller does not supply an argument to the macro. If we wrote FAIL the same way we presented INFO earlier (but with variadic macros) it might look something like this:

#define FAIL( ... ) { \
	std::ostringstream oss; \
	oss << __VA_ARGS__; \
	notifyFail( oss.str() ); \
}
... which, with no argument provided, would expand to...
{ 
	std::ostringstream oss; 
	oss << ; 
	notifyFail( oss.str() ); 
}

Do you see the problem? With nothing following the << this will not compile.

So do we need a different operator? What properties would we need? It seems we'd need an operator that comes in two forms: a binary operator that allows us to capture an argument, and a unary operator that allows us to omit the argument. Furthermore the binary form must not require its argument to be enclosed in any sort of brackets. Finally it must have higher precedence than << so we can switch over to normal stream insertion at that point.

That's a long list. Does such an operator exist? Fortunately there's not just one but two such operators to choose from! + and -. The only slight hitch is that the unary form is right-to-left associative, whereas the binary form is left-to-right. So how can we work these in?

Let's pick one of the operators. I've gone with +, but I don't think there is any advantage either way. Because unary + is right-to-left associative it needs to prefix something. So we can't use it at the start of our streaming expression. We can, however, use it at the end. Then we'll need an object to apply it to. The object doesn't actually need to do anything else. I've gone with this implementation of StreamEndStop in Catch:

struct StreamEndStop {
    std::string operator+() {
        return std::string();
    }
};
With this definition the expression, +StreamEndStop() now yields an empty string - which is idempotent with a stringstream. Which means we can write:
{
	std::ostringstream oss; 
	oss << +StreamEndStop();
	notifyFail( oss.str() ); 
}
And oss.str() evaluates to an empty string. Perfect. But what about when we do stream something? Well that would expand to:
{
	std::ostringstream oss; 
	oss << something +StreamEndStop();
	notifyFail( oss.str() ); 
}
... where something could be a string or variable or literal of any type. So we need some way for the expression:
something +StreamEndStop()
to yield the value of something. That's where the binary form of operator+ comes in:
template<typename T>
T const& operator + ( T const& value, StreamEndStop& ) {
	return value;
}
Now, whether we supply nothing, a single value or multiple values joined by <<s we'll end up with a stringstream containing what we expect. The relevant bit of code in Catch actually looks like this:
Catch::ExpressionResultBuilder( messageType ) \
	<< __VA_ARGS__ \
	+::Catch::StreamEndStop()
which yields an ExpressionResultBuilder that gets passed on elsewhere. This is all protected by CATCH_CONFIG_VARIADIC_MACROS. Otherwise it falls back to:
Catch::ExpressionResultBuilder( messageType ) << log
So a lot of work to save a few explicit empty strings, but sometimes it's the little things.

Exit code crimes

Frances Buontempo from BuontempoConsulting

I've seen a few exit code crimes recently, so I thought I'd list them.

1. Catch something specific

if __name__ == "__main__":
    try:
        run(None)
        sys.exit(0)
    except ValueError:
        sys.exit(1)

So, what does this do if another type of error is thrown? Why is it catching a specific type of error? The rest of the code is logging info - why don't we log the error? How does this make troubleshooting easy, or even possible?

2. Throw something specific

The pattern above is of course fine when the run function works like this:

def run(config):
    try:
    catch:
        raise ValueError

3. Tell no-one

Make the last line in you script

exit(0)

regardless of whatever just happened. There was a chance the script could return the exit code of whatever it called, so typing the extra line is first, effort, second achieves nothing, and finally means no-one will notice if something went wrong. Or at least not at the time. Perhaps whoever did that thought a shell script won't exit without the keyword exit at the end.



There are others - I'll add them as I find them.




Writing: The Ethical Programmer

Pete Goodliffe from Pete Goodliffe

The latest C Vu magazine from ACCU is out now. It contains my latest Becoming a Better Programer column. This month it's called The Ethical Programmer; the first instalment of a two-part series on ethics and the modern programmer. Gripping stuff.

This month, I look at our attitudes, at legal issues, and discuss software licenses.

To make the world a better place, you can enjoy a picture of some old rope, and a chicken. I also throw in some bad puns.

Capturing lvalue references in C++11 lambdas

Pete Barber from C#, C++, Windows &amp; other ramblings

Recently the question "what is the type of an lvalue reference when captured by reference in a C++11 lambda?" was asked. It turns out that it's a reference to whatever the original reference was too. This is just like taking a reference to an existing reference, e.g.

int foo = 7;
int& rfoo = foo;
int& rfoo1 = rfoo;
int& rfoo2 = rfoo1;

All references refer to foo rather than rfoo2->rfoo1->rfoo->foo meaning the following code

std::cout << "foo:" << foo << ", rfoo:" << rfoo 
          << ", rfoo1:" << rfoo1 << ", rfoo2:" << rfoo2 
          << '\n';
++foo;

std::cout << "foo:" << foo << ", rfoo:" << rfoo 
          << ", rfoo1:" << rfoo1 << ", rfoo2:" << rfoo2 
          << '\n';

std::cout << "&foo:" << &foo << ", &rfoo:" << &rfoo 
          << ", &rfoo1:" << &rfoo1 << ", &rfoo2:" << &rfoo2 
          << '\n';

Which gives:

foo:7, rfoo:7, rfoo1:7, rfoo2:7
foo:8, rfoo:8, rfoo1:8, rfoo2:8
&foo:00D3FB0C, &rfoo:00D3FB0C, &rfoo1:00D3FB0C, &rfoo2:00D3FB0C

I.e. all the references are aliases for the original foo hence the same value is displayed including when the original is modified and that the address of each variable is the same, that of foo.

There is nothing surprising here it's just basic C++ but it's along time since I've thought about it which is why with lambdas, l-value, r-value and universal references I sometimes I do a double take on what was once obvious.

The same happens with lambda capture but it's a slightly more interesting story. Take the following example:

int foo = 99;
int& rfoo = foo;
int& rfoo1 = foo;

std::cout << "foo:" << foo << ", rfoo:" << rfoo 
          << ", rfoo1:" << rfoo1 
          << '\n';

std::cout << "&foo:" << &foo << ", &rfoo:" << &rfoo 
          << ", &rfoo1:" << &rfoo1 
          << '\n';

auto l = [foo, rfoo, &rfoo1]()
{
    std::cout << "foo:" << foo << '\n';
    std::cout << "rfoo:" << rfoo << '\n';
    std::cout << "rfoo1:" << rfoo1 << '\n';

    std::cout << "&foo:" << &foo << ", &rfoo:" 
              << &rfoo << ", &rfoo1:" << &rfoo1 
              << '\n';
};

foo = 100;

l();

Which gives:

foo:99, rfoo:99, rfoo1:99
&foo:00D3FB0C, &rfoo:00D3FB0C, &rfoo1:00D3FB0C
foo:99
rfoo:99
rfoo1:100
&foo:00D3FAE0, &rfoo:00D3FAE4, &rfoo1:00D3FB0C

To begin with it behaves as per the first example in that foo, rfoo and rfoo1 all give the same value as rfoo and rfoo1 are effectively aliases for foo as shown when displaying their addresses; they're all the same.

However, when these same variables are captured it's a different story: The capture of foo is of no surprise as this is by-value so displays the captured value of 99 despite the original foo being changed to 100 prior to the lambda being invoked. Its address is that of a new variable; a member of the lambda.

It starts to get interesting with the capture of rfoo. When the lambda is invoked this too displays 99, the original captured value. Also, its address is not that of the original foo. It seems that the reference itself has not been captured but rather what it refers too, in this case an int with the value of 99. It appears to have been magically dereferenced as part of the capture.

This is the correct behaviour and when thought about becomes somewhat obvious. It's just like assigning a variable from a reference, e.g.

int foo = 7;
int& rfoo = foo;
int bar = rfoo;

bar doesn't become an int& and  rfoo is magically dereferenced except in this scenario there is nothing magical at all, it's as expected. If int were replaced with auto, e.g.

auto bar = rfoo;

then it would be expected that bar is an int as auto strips of CV and reference qualifiers.

Finally, there is rfoo1. This too is odd as it is attempting to take a reference to a reference. As seen in the first example this is perfectly fine. The end effect is that there can't be a reference to reference and so on and all are aliases of the original variable.

This is pretty much what's happening here. It's irrelevant that the target of the capture is a reference. In the end the capture by reference is capture by reference of the underlying variable, i.e. what rfoo1 refers too, in this case foo not rfoo1 itself. This is demonstrated twofold by rfoo1 within the lambda displaying the updated value of foo and also that the address of rfoo1 within the lambda is that of foo outside it.

This is as per the standard section 5.1.2 Lambda expression sub-note 14:

An entity is captured by copy if it is implicitly captured and the capture-default is = or if it is explicitly
captured with a capture that does not include an &. For each entity captured by copy, an unnamed nonstatic
data member is declared in the closure type. The declaration order of these members is unspecified.
The type of such a data member is the type of the corresponding captured entity if the entity is not a
reference to an object, or the referenced type otherwise. [ Note: If the captured entity is a reference to a
function, the corresponding data member is also a reference to a function. —end note ]

The sentence in bold states that for a reference captured by value then the type of the captured value is the type referred to, i.e. the reference aspect as been removed the crucial part being "or the referenced type otherwise". (NOTE: I haven't experimented with references to functions).

Finally, a vivid example showing that a reference captured by value involves a dereference.

class Bar
{
private:
int mValue;

public:
Bar(const Bar&) : mValue(9999)
{
}

public:
Bar(const int value) : mValue(value) {}
int GetValue() const { return mValue; }
void SetValue(const int value) { mValue = value; }
};

Bar bar(1);
Bar& rbar = bar;
Bar& rbar1 = bar;

std::cout << "&bar:" << &bar << ", &rbar:" << &rbar<< ", &rbar1:" << &rbar1 << '\n';

auto l2 = [bar, rbar, &rbar1]()
{
std::cout << "bar:" << bar.GetValue() << '\n';
std::cout << "rbar:" << rbar.GetValue() << '\n';
std::cout << "rbar1:" << rbar1.GetValue() << '\n';

std::cout << "&bar:" << &bar << ", &rbar:" << &rbar<< ", &rbar1:" << &rbar1 << '\n';
};

bar.SetValue(2);

l2();

The class bar provides a crude copy-constructor that sets the stored value to 9999. The following output is similar to that in the previous example in that the addresses of bar and rbar in the lambda differ from that of bar showing they're copies whilst rbar1 is the same. Secondly, the value of mValue stored within Bar is shown as 9999 for the first two captured variables meaning they were copy-constructed.

&bar:00D3FB0C, &rbar:00D3FB0C, &rbar1:00D3FB0C
bar:9999
rbar:9999
rbar1:2
&bar:00D3FAE0, &rbar:00D3FAE4, &rbar1:00D3FB0C

Making the copy-construct private (by commenting out the seemingly unnecessary 'public:') prevents compilation.

1>------ Build started: Project: References, Configuration: Debug Win32 ------
1>  main.cpp
1>c:\users\pete\desktop\references\references\main.cpp(85): error C2248: 'Bar::Bar' : cannot access private member declared in class 'Bar'
1>          c:\users\pete\desktop\references\references\main.cpp(59) : see declaration of 'Bar::Bar'
1>          c:\users\pete\desktop\references\references\main.cpp(54) : see declaration of 'Bar'
1>          c:\users\pete\desktop\references\references\main.cpp(59) : see declaration of 'Bar::Bar'
1>          c:\users\pete\desktop\references\references\main.cpp(54) : see declaration of 'Bar'

Writing this post has clarified the situation for me, I hope it helps you as well.

The sample code is available here.

Effective C++11/14

Frances Buontempo from BuontempoConsulting

Work sent me on Scott Meyers' latest course at Developer Focus. He gave us these guidelines:
  1. Prefer auto to explicit type declarations - remember that auto + {expr} => std::initializer_list
  2. Prefer nullptr to 0 and Null
  3. Prefer scoped enums to unscoped enums
  4. Distinguish () and {} when creating objects - the latter disallows narrowing, that might be good
  5. Declare functions noexcept whenever possible - esp. swap move
  6. Make const member functions threadsafe - make them bitwise const or internally synchronised
  7. Distinguish "universal references" from r-value references - "universal references" is his phrase, just note if you see type&&& type might not be an r-value
  8. Avoid overloading on && - typically r-value ref versus l-value ref is ok, but just r-value ref might be trouble cos it's greedy
  9. Pass and return r-value refs via std::move, universal refs by std::forward - and allow RVO to happens as before in C++98/03
  10. Understand reference collapsing See SO
  11. Assume that move operations are not present, not cheap and not used
  12. Make std::thread unjoinable on all paths - even if there's an exception
  13. Use std::launch::async with std::async if asynchronicity is essential - but is it really essential?
  14. Be aware of varying thread handle destructive behaviour
  15. Create tasks not threads
  16. Consider void functions for one-shot event communication
  17. Pass parameterless functions to std::asyncatd::thread}} and {{{std::call_once - the arguments are unconditionally copied, as with std::bind. Use a lambda instead
  18. Use native handles to transcend the C++11/14 API - if you need to configure your thread, but don't use a thread, so you won't need too
  19. Prefer lambdas to std::bind - inlining is possible
  20. Beware default captures in member functions - [=] captures the this pointer, and so member variables via this->variable, which could dangle and are "by ref" i.e. will match [&]. C++14 will add stuff to help
  21. Use std::make_shared and std::make_unique whenever possible
  22. Keep abreast of standardisation

Hello concurrent world

Frances Buontempo from BuontempoConsulting


From Anthony William's "C++ concurrency in action - practical multithreading", section 1.4.1 gives a simple "Hello world" program using C++11's thread.

#include <iostream>
#include <thread>

void hello()
{
std::cout <<"Hello concurrent world!\n";
}


int main()
{
std::thread t(hello);
t.join();
}


After Matthew Wilson re-starting his series in Overload, "Quality Matters #7 Exceptions: the story so far" http://accu.org/var/uploads/journals/Overload114.pdf page 10ff, I had a nagging feeling I should put some exception handling round this.

First question, what happens if we make the hello throw an exception? For example, what would this do?

std::thread t_trouble( []{ throw std::exception("Oops");} );

It calls abort. The thread function mustn't let exceptions escape. Also, main should probably catch some exceptions; for example, maybe there aren't enough resources to start the thread yet.

#include <iostream>
#include <thread>

void hello()
{
try
{
std::cout <<"Hello concurrent world!\n";
}
catch(const std::exception& e)
{
//erm... what to do with it?
}
}


int main()
{
try
{
std::thread t(hello);//can I pass parameters? Nico says I can to async (page 964)
t.join(); //Nico says we can do a t.detach and when main exits it will get stopped
}
catch(const std::system_error& e) //pulled in by thread I presume
{
if(e.code() == std::errc::resource_unavailable_try_again)
{
std::cout << "Try again\n";
}
}
catch(const std::exception& e)
{
std::cout << e.what() << '\n';
}
}

Right, so now we are ignoring any exceptions that get thrown.
What should I do with any exceptions I get in a function that's sent to a thread? I could use std::exception_ptr, and std::rethrow_exception when a client tries to get the result. It might be better if I read all of Anthony's book (esp Chapter 8) and use std::packaged_task instead.

accu-general (http://accu.org/index.php/mailinglists) helpfully told me to read all the chapters in the book concurrently.

Writing: Bug Hunting

Pete Goodliffe from Pete Goodliffe

The latest  C Vu magazine from ACCU is out now. It contains my latest Becoming a Better Programer column. This month it's called Bug Hunting and, as you might guess, is about the art of debugging code.

This was inspired by conversations with Greg Law in the lead up to the 2013 ACCU conference.