The story should deliver business value: it should be meaningful to some customer, user, stakeholder. In some way the story should make their lives better.
The story should be small enough to be delivered soon: some people say “within 2 days” but I’d generous, after all I used to be a C++ programmer, I’m happy as long as the story can be delivered within 2-weeks, i.e. the standard size of a sprint.
Now these two rules are in conflict, the need for value – and preferably more value! – pushes stories to be bigger while the second rule demands they are small. That is just the way things are, there is no magic solution, that is the tension we must manage.
Those two rules also help us differentiate between stories and epics – and tasks if you are using them:
Epics honour rule #1, epics are very valuable but they are not small, by definition they are large this epics are unlikely to be delivered soon
Tasks honour rule #2, they are small, very small, say a day of work. But they do not deliver value to stakeholders – or if they do it is not a big deal
Tasks are the things you do to build stories. And stories are the things you do to deliver epics. If you find you can complete a story without doing one of the planned tasks then great, and similarly not all stories need to be completed for an epic to be considered done.
In an ideal world you would not need tasks, every story would be small enough to stand alone. Nor would you need epics because stories would justify themselves. We can work towards that world but until then most teams of my experience use two of these three levels – stories and tasks or epics and stories. A few even use all three levels.
Using more than three is an administration problem. There is always a fourth level above these, the project or product that is the reason they exist in the first place. But really, three levels is more than enough to model just about anything: really small, small, and damn big.
And every story is a potential epic until proven guilty.
The story should deliver business value: it should be meaningful to some customer, user, stakeholder. In some way the story should make their lives better.
The story should be small enough to be delivered soon: some people say “within 2 days” but I’d generous, after all I used to be a C++ programmer, I’m happy as long as the story can be delivered within 2-weeks, i.e. the standard size of a sprint.
Now these two rules are in conflict, the need for value – and preferably more value! – pushes stories to be bigger while the second rule demands they are small. That is just the way things are, there is no magic solution, that is the tension we must manage.
Those two rules also help us differentiate between stories and epics – and tasks if you are using them:
Epics honour rule #1, epics are very valuable but they are not small, by definition they are large this epics are unlikely to be delivered soon
Tasks honour rule #2, they are small, very small, say a day of work. But they do not deliver value to stakeholders – or if they do it is not a big deal
Tasks are the things you do to build stories. And stories are the things you do to deliver epics. If you find you can complete a story without doing one of the planned tasks then great, and similarly not all stories need to be completed for an epic to be considered done.
In an ideal world you would not need tasks, every story would be small enough to stand alone. Nor would you need epics because stories would justify themselves. We can work towards that world but until then most teams of my experience use two of these three levels – stories and tasks or epics and stories. A few even use all three levels.
Using more than three is an administration problem. There is always a fourth level above these, the project or product that is the reason they exist in the first place. But really, three levels is more than enough to model just about anything: really small, small, and damn big.
And every story is a potential epic until proven guilty.
For years people have been comparing software construction with building construction. Think about how we talk about “architecture” or foundations, or the cost of change and so on. As I’ve said before, building software is not like building a house. Now it occurs to me that a better metaphor is the ongoing ownership of the building.
Every building requires “maintenance” and over time buildings change – indeed buildings learn. While an Englishman’s home is his castle those of us, even the English, who are lucky enough to own a house don’t have a free hand in the changes we make to our houses
Specifically I’m thinking about the Product Owner. Being a Product Owner is less about deciding what you want your new house to look like, or how the building should be constructed, its not even about deciding how many rooms the house should have. The role of the Product Owner is to ensure the house continues to be liveable, preferably the house is getting nicer to live in, and the house is coping with the requests made on it.
I own a house – a nice one in West London. As the owner I am responsible for the house. I do little jobs myself – like painting the fences. More significantly I have to think about what I want to do with the house: do we want to do a loft conversion? What would that entail and when might I be able to afford that?
I am the Product Owner of my own house. I have to decide on what is to be done, what can wait and what trade-offs I can accept.
When I bought the house the big thing to change was the kitchen and backroom. There was little point in any other works until those rooms were smashed to bits and rebuilt. I had to think though what was needed by my family, what was possible and what the result might be like. I received quotes from several builders – each of whom had their own ideas about what I wanted. I hired an architect for advice. I looked at what neighbours had done. And I had a hard think about how much money I could spend.
An Englishman’s home is his castle – I am the lord of my house and I can decide what I want, except…
My wife and children have a say in what happens to the house. Actually my wife has a pretty big say, while the children have less say there needs are pretty high on my list of priorities.
My local council and even the government have a say because they place certain constraints on what I can do – planning permission, rules and building codes. The insurance company and mortgage bank set some constraints and expectations too.
My neighbours might not own my house but they are stakeholders: I can’t upset them (too much) and they impose some constraints. (In my first flat/apartment the neighbours were a bigger issue because we shared a roof, a garden and the walls.)
So while I may be lord of my own house I am not a completely a free agent. And the same is true with Product Owners.
The secret with Product Owners is: they are Owners. They are more than managers – managers are just hired help. But neither do POs have a free hand, they don’t have unlimited power, the are not dictators, they are not completely free to do what they want and order people around.
Like me, Product Owners have limited resources available: how much money, how many helpers, access to customers and more. I have to balance my desire for a large loft conversion (with shower, balcony and everything else) with the money I can afford to spend on it. That involves trade-off and compromises. I could go into debt – increase my mortgage – but that comes with costs.
Product owners have responsibilities: to customers and users, to the those who fund the work (like the mortgage bank), to team members and peers to name a few. Some decisions they can make on their own, but on other decisions they can only lead a conversation and guide it towards a conclusion.
What the homeowner metaphor misses entirely is the commercial aspect: my house exists for me to live in. I don’t expect to make money out of it. The house next door to mine is owned by a commercial landlord who rents it out: the landlord is actively trying to make money out of that house.
Most Product Owners are trying to further some other agenda: commercial (generating money), or public sector (furthering Government policies), or third sector (e.g. a charity). In other words: Product Owners are seeking to add value for their organization. This adds an additional dimension because the PO has to justify their decisions to a higher authority.
Advertisement: at the time of writing there are still a few tickets available for my online User Stories Masterclass beginning this Wednesday, 90 minutes each week for 4 weeks.
“Gee, I really wish I could be that type of Product Owner.”
His comment made me smile. He nicely summarised much of the argument in Art of PO. The book makes a case for an expansive product owner – one with product management skills and business analysis skills; a product owner who looks to improve value over the short and long run, and for product owners with more customer empathy and marketing skills than code empathy and technical skills.
Many of the Product Owners I meet aren’t really owners of the product. Rather they are “Backlog Administrators” and as such the industry is creating another problem for itself.
Over the years the product owner role has been diluted, so many product owners are not really owners of their products. Instead their role is limited and constricted. They are judged on how many features they get the team deliver, whether those features are delivered by some date or whether they have met near term goals of doing the things customers – or internal users – are complaining about.
In other words the whole team is a feature factory: requests go in and success is measured by how many of those requests reach production.
Sure that is one way to run a team, and in some places that might be the “right” way to do it (a team dedicated to addressing production/customer issues perhaps.)
Unfortunately agile is prone to this failing because of the sprint-sprint-sprint nature of work. The things in front of you are obviously more valuable than the things that are not. The people shouting at you today obviously represent greater value than those who are sitting quietly asking nicely. And both groups can mask bigger insights and opportunities.
Hang on you say: Is this the same Allan who has argued against long term planning? And against analysis phases? The Allan who always argues for action this day?
Well, yes I am that Allan. And I agree that I regularly argue that teams should get started on coding and limit planning and analysis.
But that doesn’t mean I’m against these things, it only means I’m conscious of the diminishing returns of planning; and I know that what is technically possible frames not only the solution but the problem – because often we can’t conceive of the problem until we see how a solution might change things.
Teams need to watch out for the “bigger” questions. Teams need to take some time to thing long term. Time needs to be spent away from the hurly-burly of sprint-sprint-sprint to imagine a different world. Dis-economies of scale may rule but there still needs to be consideration of larger things, e.g. jobs to be done over user stories.
The responsibility rests with the Product Owner.
They own the product the way I own my house: I have to pay the mortgage and I have to change blow light bulbs but I also need to think: how long will the roof last? Will we build an extension? When will we rebuild the patio? And where am I going to put a car charging point when that day comes?
I don’t take those decisions in isolation, I don’t spend lots of time on them and I don’t let them get in the way of work today. But spending a little time thinking about them, and I may well leader on the discussion. Taking a little time to think through out how things might fit together (don’t do the roof until after the extension is built) has benefits.
And so many Product Owners aren’t doing that. Worse still their organizations don’t expect them to. Maybe they see an Architects doing that, or a Product Manager – or maybe nobody does.
The thing is: the Product Owner is the OWNER.
Managers and architects are hired and fired as needed. The buck stops with owners.
Many organizations have got this the wrong way round. The Product Owner role is diluted and individual Product Owners emasculated.
Advertisement: at the time of writing there are still a few tickets available for my online User Stories Masterclass beginning this Wednesday, 90 minutes each week for 4 weeks.
Last month I ran an experiment: I delivered my one day workshop on User Stories, Requirements and Backlogs as a series of four 90 minute workshops and offered them for free. I had a great response with over 20 people signing up immediately and during the last month we all learned something – I had great feedback.
I’m now reworking that workshop further into two series: User Stories Masterclass and Value Driven Planning. Both of these will run as four 90 minute workshops online over successive weeks.
Places are limited so don’t hold off booking. (If you miss out let me know and I’ll try to schedule another soon.)
Workshops 1 and 2 will focus on User Story format and what makes a good story. Workshop 3 will consider some of the processes that fit around User Stories (definitions of done and ready, acceptance criteria, etc.) and, importantly non-functional requirements. The final workshop will look at splitting stories.
For everyone else there are two price points. “Self pay” for those of you who are paying out of your own pocket. And a more expensive “Company paying” ticket for those with a generous employer – who can reclaim or avoid UK VAT.
To sweeten the extra price I’m including a one-hour one-on-one consultation for those who pay the higher price.
More about the value workshop soon – and why the split? Well, a) it turns out I have more than enough material, and b) people had some great questions and I want to allow time for conversation.
Consider this a gift, its also an experiment. Numbers are limited so if you would like to join please e-mail me today – if it goes well I’ll repeat, although I might ask for money next time.
I’m going to tun an online workshop entitled: Stories and Value.
Participation is limited to a 16 and its going to be first come first served – blog/newsletter readers are getting the first chance to sign-up.
This is based on my existing “Requirements, Backlogs and User Stories” workshop which has itself mutated into a discussion of stories and value. The workshop will run as a series of 90 minute sessions, one a week for four weeks online.
I want the workshop sessions to remain interactive, I’m sure I will use some slides at some point but I want to keep it interactive. So I’m going to limit participation to 12.
The draft schedule is:
Workshop 1: How value influences our thinking
Workshop 2: Good and Bad User Stories
Workshop 3: Estimating story value
Workshop 4: Time value profiles and closing discussion
I plan on using exercises in throughout and I think I know how to run them online. And I want discussion! – I may even set a little homework between sessions.
But in all honesty, it’s an experiment. So, I’m not planning on charging for this – it is Free!
If you find it valuable you can make a payment – like those “pay what you like” restaurants. That will itself be feedback.
I’m thinking Wednesday, 3pm UK time so those in mainline Europe could join too (sorry US and Asia, maybe next time); on a Zoom conference. Start next week, April 1 ? – once I know who’s in we might debate this between ourselves.
My thinking is still developing on this so let me know if you have any ideas to contribute. (And if you can’t join but want to let me know, feedback is valuable too! Likewise, if you are tempted but want to see something different please tell me and I’ll see what I can do.)
Regular expressions are widely used, but until recently they were rarely studied empirically (i.e., just theory research).
This week I discovered two groups studying regular expression usage in source code. The VTLeeLab has various papers analysing 500K distinct regular expressions, from programs written in eight languages and StackOverflow; Carl Chapman and Peipei Wang have been looking at testing of regular expressions, and also ran an interesting experiment (I will write about this when I have decoded the data).
Regular expressions are interesting, in that their use is likely to be purely driven by an application requirement; the use of an integer literals may be driven by internal housekeeping requirements. The number of times the same regular expression appears in source code provides an insight (I claim) into the number of times different programs are having to solve the same application problem.
The data made available by the VTLeeLab group provides lots of information about each distinct regular expression, but not a count of occurrences in source. My email request for count data received a reply from James Davis within the hour
The plot below (code+data; crates.io has not been included because the number of regexs extracted is much smaller than the other repos) shows the number of unique patterns (y-axis) against the number of identical occurrences of each unique pattern (x-axis), e.g., far left shows number of distinct patterns that occurred once, then the number of distinct patterns that each occur twice, etc; colors show the repositories (language) from which the source was obtained (to extract the regexs), and lines are fitted regression models of the form: , where: is driven by the total amount of source processed and the frequency of occurrence of regexs in source, and is the rate at which duplicates occur.
So most patterns occur once, and a few patterns occur lots of times (there is a long tail off to the unplotted right).
The following table shows values of for the various repositories (languages):
The lower (i.e., closer to zero) the value of , the more often the same regex will appear.
The values are in the region of -2.5, with two exceptions; why might StackOverflow and npm be different? I can imagine lots of duplicates on StackOverflow, but npm (I’m not really familiar with this package ecosystem).
I am pleased to see such good regression fits, and close power law exponents (I would have been happy with an exponential fit, or any other equation; I am interested in a consistent pattern across languages, not the pattern itself).
Some of the code is likely to be cloned, i.e., cut-and-pasted from a function in another package/program. Copy rates as high as 70% have been found. In this case, I don’t think cloned code matters. If a particular regex is needed, what difference does it make whether the code was cloned or written from scratch?
If the same regex appears in source because of the same application requirement, the number of reuses should be correlated across languages (unless different languages are being used to solve different kinds of problems). The plot below shows the correlation between number of occurrences of distinct regexs, for each pair of languages (or rather repos for particular languages; top left is StackOverflow).
Why is there a mix of strong and weakly correlated pairs? Is it because similar application problems tend to be solved using different languages? Or perhaps there are different habits for cut-and-pasted source for developers using different repositories (which will cause some patterns to occur more often, but not others, and have an impact on correlation but not the regression fit).
Today I’m giving away my crown jewels. Once you have read this post I can’t run my favourite exercise with you. There is a bit of experiential learning I can’t give you. So if you’ve rather have the experience stop reading and go and book yourself on my May workshop.
I’m describing an exercise that models what happens in “the real world(tm).” Plenty of the people who have done the exercise recognise it was a real life problem. The insights are many, and some a little disturbing.
Dozens of teams and the answers are always the same. I live in dread that someone will guess and ruin the exercise but it never happens. Now I’m telling the world that might change.
On screen I put a story something like:
As a widget maker I want an online store to sell my widgets so that I can make money
I separate the room into teams. Each team represents a technology supplier – an agency, an outsourcer, whatever. I want each team to competitively bid on a piece of work. Each team gets to think about the problem and estimate the work. At the end I want each team to be ready to name their price, how long it will take and how many people they need. They may add any more details they like, e.g. staging, design, technology, etc. (and most do).
The teams on the right get a story which says:
As an international widget maker I want to sell direct to customers so that I can cut out distributors. I anticipate $10million turnover within 3 years and have budgeted $1.2m for this project.
15 minutes later the teams on the right read out their bids. They always want a million give or take. They want months, if not years. They want teams of half a dozen or more engineers, testers, UXD, analysts and project managers. They may propose an ongoing maintenance contract too.
Very disconcerting for these teams is that while they are answering and taking questions the other teams, those on the left, often burst out laughing – literally – when they hear these proposals.
What neither side knows is that they have different stories. The teams on the left get a story saying:
As an artisan widget maker I want to sell my widgets to customers so that I can give up my day job. When I make $100,000 a year in sales I can live my dream. My accountant tells me a WordPress website will cost $5,000.
These teams want a week or two, an engineer or two and perhaps $10,000 tops. In some cases they say “We can do it this afternoon, we’ll set up Etsy.” Even if they don’t want to use Esty or eBay they probably propose an OpenSource solution.
So what do you think?
True, it is a semi-artificial set-up but I would argue that these situations happen all the time in “the real world” work environment. However they are usually well disguised and hard to see. Maybe now you will spot them.
That aside there are many potential lessons this exercise illustrates, almost everyone is worth a discussion in its own right. To keep things brief I’ll just highlight some of them:
Anchoring (cognitive bias): individuals are anchored to those numbers, bigger number lead them to frame their answers as bigger numbers.
Assumptions: people jump to assumptions, people automatically fill in the blanks when they lack information and the information they fill in flows from the numbers mentioned. Few questions get asked.
Different solutions: the key lesson for me, confronted with similar problems, people (especially engineers) are capable of formulating very different solutions. Those solutions have different time and cost implications.
Problem bounding: presenting the same problem with different bounding constraints results in massively different solutions.
Effort estimates, and therefore cost estimates, flow from value: whether through anchoring assumptions or solution designs the estimates (time and money) flow from the value available NOT the other way around.
Prior experience often goes out the window. In one run a low-end teams told me: “We did this last week. A digital consultant showed us how to install WordPress and Magento for online retail in the morning and in the afternoon we did it ourselves.” While this team came up with a low cost proposal their colleagues who were given the $1m story forgot everything they learned last week.
People don’t ask questions: I answer questions while teams are creating their answers but people rarely challenge what is asked for. Maybe its because I’m usually in some position of authority as a consultant or workshop trainer and my word should be followed.
Occasionally a team with the million dollar story say “We could do this with eBay/WordPress/Shopify.” I keep a poker face and let them be. Inevitably left alone for long enough they talk themselves into a much more complex and expensive bid.
In fact, the longer I give teams the higher the estimates go. I heard a team in Australia say three times “Those estimates look low, lets double them.” And they did. (Again, planning has diminishing returns.)
So far nobody has offered two solutions: you could offer up a Shopify solution and a custom build solution but nobody has.
While we are going through the exercise the minimal viable product idea often gets mentioned – usually by the teams on the right. So recently I introduced a third story. This read the same as the international widget maker but has an extra paragraph underneath:
MacAllan consulting has advised the company to take an iterative and agile approach to this work using a minimally viable product model.
How do you think teams respond?
Think for a minute… your answer is?
It makes no difference.
Or rather, so far I’ve not had any of the million dollar teams propose anything close to the $5,000 solution. In one case a team with the MVP story actually came in more expensive – and longer – than the million dollar team without the MVP clause.
My learning here: talking MVP makes no difference. If you want an MVP you have to impose constraints. (Hence try an MVT.)
People continue to fill in the blanks after the charade is exposed. I’ve heard software architects argue forcefully they these are different problems because of the money involved and therefore require different architecture. They clearly feel cheated and want to justify the proposal they have made. I suspect they feel I’ve made them look silly and want to undo that impression, I’m sorry if I’ve made anyone feel silly.
I wonder how often that happens in the work place? How many of us would back down in real life? I’d like to think I would but I would probably first try and justify my position.
The architects have a point, in many ways the stories are functionally the same but the differences lie in the non-functional requirements: load, throughput, security and so on. But all of that is inferred by people from the price tag without question.
It makes me said that teams ask so few questions. People easily see themselves as a tailor not as a consultant (my Tailor or Image consultant post.)
Then there are the questions about the bidding process and companies bidding on the work.
Imagine you are the buyer: one supplier bids really low, the others were much higher but inline with your expectations. Would you trust the low bid? Have they blow their credibility?
And as a bidder: if you know the client plans to spend $1,000,000 why bid lower? The engineers cost estimates and designs aren’t relevant. Ideally you bid just below the competition. You are the lowest price with all the credibility and maximum revenue.
For that matter, should you be bidding on this at all?
If you work for a small e-commerce provider in rural Cornwall you may never know about, let alone, bid on a million dollar piece of work from an American multi-national. And if you did, would anyone take you seriously?
Suppose you got your big break: you walk in and offer a quick, low cost solution based on something like Shopify. Would they take you seriously? Would they want to listen?
Do corporations increase their own costs simply by being?
Conversely, if you work for a big consultancy and bid on million dollar deals every week will you be interested in bidding on a $5,000 piece of work? Of course not!
But that also means that if a corporation approaches you for a million dollar online shop, even if you could deliver it in a week’s time running on a third party platform do you have any incentive?
I don’t have answers to these questions. Indeed, there aren’t any right answers. All answers are valid, just some answers are “better” in some places than others.
Ultimately the lesson I take away from this is: we craft solutions within constraints.
More specifically: Engineers engineer within constraints, that is what engineers do.
That reinforces my belief that one needs to really understand benefit (value) and how that changes with time. From there we can work back to a solution.
At the end of January I’m running my 1-day Requirements, User Stories and Backlogs workshop in London with Learning Connexions. I get great feedback from people who attend the course, perhaps because it is mostly exercised based.
If your interested check out the Learning Connexions page – its just one day and won’t break the bank! Hope to see you there.
I feel compelled to write this blog because I keep coming across the wrong type of Product Owner. I feel bad about writing this blog because a) I’ve made these points before in other forums so I’m repeating myself, and b) at the end of the day you, your team, and your organization, is free to define and use any title you like for any role you like, you are free to define any given role as you like.
So let me set out my model of a Product Owner and then at least there is a model to compare any other definition with.
Our old friend the Triangle of Constraints can help here – also know as “The Iron Triangle” and pictured above (I like to call it the FRT triangle). Now notice the version I use is slightly different from the more common model:
Rather than “cost” I label one side of the triangle “People”. I could label it resources but in software development resources are overwhelmingly people and the knowledge they bring. People deserve respect, calling them “resources” makes them sound like paperclips.
For software development costs are function of how many people you have and how long you have them for: costs = people x time. OK, there are some other “resources” to add to costs, e.g. buying laptops, renting time in the cloud, and so on but these are often themselves a function of the number of people you have. Such costs are a small increment on top of the wage bill.
Now the number of people you have is fixed in the short term, or to be more accurate: it is upward fixed. People can get ill or resign at anytime but adding people takes time. So in the short run one can consider that dimension fixed.
Time is also fixed. There is usually a business deadline, or rather a business benefit which is time elastic so you have a date to aim for. And on agile teams there are sprint deadlines (two-weeks, two-weeks, two weeks). So a large part time is fixed.
The final side of the triangle is labelled features or functionality, but might be labelled “requirements”, “the what” or “what are we building” – I like to think of it as the demand side.
With me so far? – so far that should be uncontroversial.
Now the traditional Project Manager role, and to a lesser degree the newer Delivery Manager role, tend to regard the third side – the what side – as fixed. There is a thing to be delivered. It is a known thing. It has been decided on and the manager’s job is to get it delivered.
To this end Project Managers are trained to regard the “thing to be built” as a given, preferably fixed, thing. Their training centres on the other sides: cost and time. They are trained both in rationing these commodities and allocating them in an efficient way. When things go wrong these managers ask for more time (which means more money because the same people need paying) or more people (which both costs more and makes things worse because of Brook’s Law).
So to summarise: traditional Project Managers focus on “when” and the input variables: people/resources and money.
Can you guess what I’m going to say next?
Product Owners – plus Product Managers and Business Analysts – focus on the “what”. What do we need to build next? What has the most benefit? What should we be building for the future?
For Product Owners the time and people are fixed. (This is most obvious in an agile environment but is actually true everywhere sooner or later.)
The thing being built is negotiable, the desired outcome may be achieved by different routes, different technologies and different solutions – the different time and cost will be a consideration but outcome is the primary focus.
In other words: Product Owners are all about the what.
In order to operate in the what-space product owners need authority and legitimacy to flex what they are building. When they don’t have that they are reduced to backlog administrators simply ordering the backlog and feeding it to technical teams. That turns the role into a type of Project or Delivery Manager.
So if you need to tell a real Product Owner from all the other misinterpretations of the role ask:
Does the product owner focus on what?
Can the product owner discuss different solutions and approaches to achieve an outcome?
Is the PO flexible about the backlog? (as opposed to slavishly trying to deliver it all)
Real product owners can answer Yes to all three.
(Notice I’m deliberately being careful in what I say about “Delivery Managers.” This role is still emerging and as such its wrong to generalise about it too much. In so much as a Delivery Manager brings management skills, communication and organization to an effort it can be a positive role. When a Delivery Manager is relabelling of the Project Manager role it can be damaging.)
Now that said, the fact that some organizations choose to define the “Product Owner” role as a role closer to “Project Manager” or “Delivery Manager” rather than a role closer to “Product Manager”, “Business Analyst” or (heaven forbid) business owner causes a lot of confusion.
Perhaps I’m wrong here, perhaps the “Product Owner” is a type of “delivery manager” but I think the majority of writers, thinkers and practitioners agree with me.
Even if you disagree with me I hope we can agree on one thing: because there are different interpretations and implementations of the role there is room for confusion; and that confusion makes it harder to fill the role and harder to be seen as a successful Product Owner.
Like this post? – Like to receive these posts by e-mail?
, where: 
is driven by the total amount of source processed and the frequency of occurrence of regexs in source, and 
is the rate at which duplicates occur.
