Replication: not always worth the effort

Derek Jones from The Shape of Code

Replication is the means by which mistakes get corrected in science. A researcher does an experiment and gets a particular result, but unknown to them one or more unmeasured factors (or just chance) had a significant impact. Another researcher does the same experiment and fails to get the same results, and eventually many experiments later people have figured out what is going on and what the actual answer is.

In practice replication has become a low status activity, journals want to publish papers containing new results, not papers backing up or refuting the results of previously published papers. The dearth of replication has led to questions being raised about large swathes of published results. Most journals only published papers that contain positive results, i.e., something was shown to some level of statistical significance; only publishing positive results produces publication bias (there have been calls for journals that publishes negative results).

Sometimes, repeating an experiment does not seem worth the effort. One such example is: An Explicit Strategy to Scaffold Novice Program Tracing. It looks like the authors ran a proper experiment and did everything they are supposed to do; but, I think the reason that got a positive result was luck.

The experiment involved 24 subjects and these were randomly assigned to one of two groups. Looking at the results (figures 4 and 5), it appears that two of the subjects had much lower ability that the other subjects (the authors did discuss the performance of these two subjects). Both of these subjects were assigned to the control group (there is a 25% chance of this happening, but nobody knew what the situation was until the experiment was run), pulling down the average of the control, making the other (strategy) group appear to show an improvement (i.e., the teaching strategy improved student performance).

Had one, or both, low performers been assigned to the other (strategy) group, no experimental effect would have shown up in the results, significantly reducing the probability that the paper would have been accepted for publication.

Why did the authors submit the paper for publication? Well, academic performance is based on papers published (quality of journal they appear in, number of citations, etc), a positive result is reason enough to submit for publication. The researchers did what they have been incentivized to do.

I hope the authors of the paper continue with their experiments. Life is full of chance effects and the only way to get a solid result is to keep on trying.

ACCU Conference 2018

Products, the Universe and Everything from Products, the Universe and Everything

We had an absolute blast at this year's ACCU Conference and if you were there we imagine you did too.

For us the highlight had to be the launch of #include <C++>, a new global, inclusive, and diverse community for developers interested in C++.

@include_cpp logo
The #include <C++> community logo.

This is an initiative that has been brewing for a while, and one we are very happy to be a part of. Above all else #include <C++> is designed to be a safe place for developers irrespective of their background, ethnicity, gender identity or sexuality. The group runs a Discord server which is moderated to ensure that it remains a safe space and which you are welcome to join.

On the technical front, one unexpected highlight of the conference was Benjamin Missel's wonderful short talk on writing a C compiler for a BBC Micro, during which he demonstrated SSHing into a BBC Model B through the serial port!



Most conference sessions were recorded so even if you weren't there you can still watch them.

See you at ACCU 2019!

ACCU Conference 2018

Products, the Universe and Everything from Products, the Universe and Everything

We had an absolute blast at this year's ACCU Conference, and if you were there we imagine you did too.

For us the highlight had to be the launch of #include <C++>, a new global, inclusive, and diverse community for developers interested in C++.

#include C++ logo

This is an initiative that has been brewing for a while, and we're very happy to be a part of. Above all else #include <C++> is designed to be a safe place for developers irrespective of their background, ethnicity, gender identity or sexuality. The group runs a Discord server which is moderated to ensure that it remains a safe space and which you are welcome to join.


On the technical front, one unexpected highlight of the conference was Benjamin Missel's wonderful short talk on writing a C compiler for a BBC Micro, during which he demonstrated SSHing into a BBC Model B through the serial port!

Most conference sessions were recorded so even if you weren't there you can still watch them.

See you at ACCU 2019!

ACCU Conference 2018

Products, the Universe and Everything from Products, the Universe and Everything

We had an absolute blast at this year's ACCU Conference, and if you were there we imagine you did too.

For us the highlight had to be the launch of #include <C++>, a new global, inclusive, and diverse community for developers interested in C++.

#include C++ logo

This is an initiative that has been brewing for a while, and we're very happy to be a part of. Above all else #include <C++> is designed to be a safe place for developers irrespective of their background, ethnicity, gender identity or sexuality. The group runs a Discord server which is moderated to ensure that it remains a safe space and which you are welcome to join.


On the technical front, one unexpected highlight of the conference was Benjamin Missel's wonderful short talk on writing a C compiler for a BBC Micro, during which he demonstrated SSHing into a BBC Model B through the serial port!

Most conference sessions were recorded so even if you weren't there you can still watch them.

See you at ACCU 2019!

ACCU Conference 2018

Products, the Universe and Everything from Products, the Universe and Everything

We had an absolute blast at this year's ACCU Conference, and if you were there we imagine you did too. For us the highlight had to be the launch of #include <C++>, a new global, inclusive, and diverse community for developers interested in C++.
#include C++ logo
This is an initiative that has been brewing for a while, and we're very happy to be a part of. Above all else #include <C++> is designed to be a safe place for developers irrespective of their background, ethnicity, gender identity or sexuality. The group runs a Discord server which is moderated to ensure that it remains a safe space and which you are welcome to join.
On the technical front, one unexpected highlight of the conference was Benjamin Missel's wonderful short talk on writing a C compiler for a BBC Micro, during which he demonstrated SSHing into a BBC Model B through the serial port! Most conference sessions were recorded so even if you weren't there you can still watch them. See you at ACCU 2019!

ResOrg 2.0.7.27 has been released

Products, the Universe and Everything from Products, the Universe and Everything

ResOrg 2.0.7.27 has just been released. This is a maintenance update for ResOrg 2.0, and is compatible with all ResOrg 2.0 licence keys.

The following changes are included:

  • Fixed a bug in the Symbols Display which could cause some "OK" symbols to be incorrectly shown in the "Problem Symbols Only" view.

  • Corrected the upper range limit for control symbols from 28671 (0x6FFF) to 57343 (0xDFFF).

  • ResOrg binaries are now dual signed with both SHA1 and SHA256.

  • Added support for Visual Studio 2017.

  • Corrected the File Save Dialog filters used by the ResOrgApp "File | Export" command.

  • The ResOrgApp "File | Export", "File | Save", "File | Save As" and "File | Properties" commands (which apply only to symbol file views) are now disabled when the active view is a report.

  • Fixed a crash in the Symbol File Properties Dialog.

  • Fixed a typo on the Symbol File "Next Values" page.

  • Various minor improvements to the installer.

Download ResOrg 2.0.7.27

ResOrg 2.0.7.27 has been released

Products, the Universe and Everything from Products, the Universe and Everything

ResOrg 2.0.7.27 has just been released. This is a maintenance update for ResOrg 2.0, and is compatible with all ResOrg 2.0 licence keys.

The following changes are included:

  • Fixed a bug in the Symbols Display which could cause some "OK" symbols to be incorrectly shown in the "Problem Symbols Only" view.
  • Corrected the upper range limit for control symbols from 28671 (0x6FFF) to 57343 (0xDFFF).
  • ResOrg binaries are now dual signed with both SHA1 and SHA256.
  • Added support for Visual Studio 2017.
  • Corrected the File Save Dialog filters used by the ResOrgApp "File | Export" command.
  • The ResOrgApp "File | Export", "File | Save", "File | Save As" and "File | Properties" commands (which apply only to symbol file views) are now disabled when the active view is a report.
  • Fixed a crash in the Symbol File Properties Dialog.
  • Fixed a typo on the Symbol File "Next Values" page.
  • Various minor improvements to the installer.

Download ResOrg 2.0.7.27

ResOrg 2.0.7.27 has been released

Products, the Universe and Everything from Products, the Universe and Everything

ResOrg 2.0.7.27 has just been released. This is a maintenance update for ResOrg 2.0, and is compatible with all ResOrg 2.0 licence keys.

The following changes are included:

  • Fixed a bug in the Symbols Display which could cause some "OK" symbols to be incorrectly shown in the "Problem Symbols Only" view.
  • Corrected the upper range limit for control symbols from 28671 (0x6FFF) to 57343 (0xDFFF).
  • ResOrg binaries are now dual signed with both SHA1 and SHA256.
  • Added support for Visual Studio 2017.
  • Corrected the File Save Dialog filters used by the ResOrgApp "File | Export" command.
  • The ResOrgApp "File | Export", "File | Save", "File | Save As" and "File | Properties" commands (which apply only to symbol file views) are now disabled when the active view is a report.
  • Fixed a crash in the Symbol File Properties Dialog.
  • Fixed a typo on the Symbol File "Next Values" page.
  • Various minor improvements to the installer.

Download ResOrg 2.0.7.27

ResOrg 2.0.7.27 has been released

Products, the Universe and Everything from Products, the Universe and Everything

ResOrg 2.0.7.27 has just been released. This is a maintenance update for ResOrg 2.0, and is compatible with all ResOrg 2.0 licence keys. The following changes are included:
  • Fixed a bug in the Symbols Display which could cause some "OK" symbols to be incorrectly shown in the "Problem Symbols Only" view.
  • Corrected the upper range limit for control symbols from 28671 (0x6FFF) to 57343 (0xDFFF).
  • ResOrg binaries are now dual signed with both SHA1 and SHA256.
  • Added support for Visual Studio 2017.
  • Corrected the File Save Dialog filters used by the ResOrgApp "File | Export" command.
  • The ResOrgApp "File | Export", "File | Save", "File | Save As" and "File | Properties" commands (which apply only to symbol file views) are now disabled when the active view is a report.
  • Fixed a crash in the Symbol File Properties Dialog.
  • Fixed a typo on the Symbol File "Next Values" page.
  • Various minor improvements to the installer.
Download ResOrg 2.0.7.27

The Product Owner is dead, long live the Product Owner!

Allan Kelly from Allan Kelly Associates

3ProductOwners-2018-04-26-17-33.jpg

For years I have been using this picture to describe the Product Owner role. For years I have been saying:

“The title Product Owner is really an alias. Nobody should have Product Owner on their business cards. Product Owner is a Scrum defined role which is usually filled by a Product Manager or Business Analyst, sometimes it is filled by a Domain Expert (also known as a Subject Matter Expert, SME) and sometimes by someone else.”

Easy right?

In telling us about the Product Owner Scrum tells us what one of these people will be doing within the Scrum setting. Scrum doesn’t tell us how the Product Owner knows what they need to know to make those decisions – that comes by virtue of the fact that underneath they are really a Product Manager, BA or expert in the field.

In the early descriptions of Scrum there was a tangible feel that the Product Owner really had the authority to make decisions – they were the OWNER. I still hope that is true but more often than not these days the person playing Product Owner is more likely to be a proxy for one or more real customers.

I go on to say:

“In a software company, like Microsoft or Adobe, Product Managers normally fill the role of Product Owner. The defining feature of the Product Manager role is that their customers are not in the building. The first task facing a new Product Manager is to work out who their customers are – or should be – and then get out to meet them. By definition customers are external.”

“Conversely in a corporate setting, like HSBC, Lufthansa, Proctor and Gamble, a Product Owner is probably a Business Analyst. There job is to analyse some aspect of the business and make it better. By definition their customers are in the building.”

With me so far?

Next I point out that having set up this nice model these roles are increasingly confused because software product companies increasingly sell their software as a service. And corporate customer interact with their customers online, which means customer contact is now through the computer.

Consider the airline industry: twenty years ago the only people who interacted with airline systems from United, BA, Lufthansa, etc. were airline employees. If you wanted to book a flight you went to a travel agent and a nice lady used a green screen to tell you what was available.

Today, whether you book with Lufthansa, SouthWest or Norwegian may well come down to which has the best online booking system.

Business Analyst need to be able to think like Product Managers and Product Managers need to be able to think like Business Analysts.

I regularly see online posts proclaiming “Product Managers are not Product Owners” or “Business Analysts are not Product Owners.” I’ve joined in with this, my alias argument says “they might be but there is so much more to those roles.”

It makes me sad to see the Product Manager role reduced to a Product Owner: the Product Owner role as defined by Scrum is a mere shadow of what a good Product Manager should be.

But the world has moved on, things have changed.

The world has decided that Product Owner is the role, the person who deals with the demand side, the person decides what is needed and what is to be built.

I think its time to change my model. The collision between the world of Business Analysts and Product Managers is now complete. The result is an even bigger mess and a new role has appeared: “Digital Business Analyst” – the illegitimate love child of Business Analysis and Product Management.

The Product Owner is now a superset of Product Manager and Business Analyst.

ProductOwnerSkills-2018-04-26-17-33-1.jpg

Product Owners today may well need the skills of business analysis. They are even more likely to need the skills of Product Management. And they are frequently expected to know about the domain.

Today’s Product Owner may well have a Subject Matter Expert background, in which case they quickly need to learn about Product Ownership, Product Management and Business Analysis.

Or they may have a Business Analysis background and need to absorb Product Management skills. Conversely, Product Owners may come from a Product Management background and may quickly need to learn some Business Analysis. In either case they will learn about the domain but they may want to bring in a Subject Matter Expert too.

To make things harder, exactly which skills they need, and which skills are most important is going to vary from team to team and role to role.

The post The Product Owner is dead, long live the Product Owner! appeared first on Allan Kelly Associates.