▲ 85 points
back
39 comments
My happiness at work drastically improved when I started giving 99th-percentile estimates instead of medians.
It turned out management usually didn't want to know what something was most likely to cost. They wanted to know what they could reliably plan for and promise to clients. They treated my median estimates as deadlines and I ended up working a lot of extra hours.
With 99% estimates, management stayed happy and I seldom had to work late. Usually I had extra time. Sometimes I gave that time back to management, but for some managers that had a downside because they would push back more on my estimates.
So instead, I used the extra time to reduce technical debt, automate routine tasks, or improve my skills. That reduced the risks of future projects and improved my productivity. In the end, I ended up delivering more this way.
I recently came across this:
"Bye Bye Velocity! Kanban metrics as an alternative to story points & velocity.":
Velocity is an unavoidable metric, you can count lines added and deleted, tickets moved to done, commits added, it doesn't matter these are all proxies for velocity.
In my PSM1 course this reaction was mentioned as being pretty standard when teams are judged by whether they make their estimations a reality.
I have seen this happen in my team.
I am off course happy that it works for you, but it hints that there might be underlying issues to me.
Did you not get time for technical debt before you switched to 99% estimates?
I think 99th-percent estimate is a great point to make. You said some managers pushed back on estimates if you gave that time back. Did you tell them they were 99-percent estimates?
Generally I told them my estimate was what I could commit to, and allowed that it was possible I'd finish in less time.
Look at this peace from Jeff Sutherland: https://qr.ae/pv4DI3. There is a study out there that shows, that teams working with estimates are less productive than teams working with no estimates. The key is: "Best teams have small stories and do no tasking." What I'll try next: only estimate with t-shirt sizes S and L. And everytime it's an L trying to slice it smaller.
So, work that doesn't need the long-term planning coordination that estimates allow, is likely to go to more skilled teams? Routine project work goes to the rank-and-file while exceptions go to the tiger team?
And that possible interpretation is assuming that "slowest" and "team performance" are even measuring the right thing and not accounting artifacts. (Does completing the same work in the same time get counted differently when it's tracked as two 3-hour tasks vs three 2-hour tasks vs 12 "no estimate" tasks?)
Split tasks so they don't take more than a morning or a day.
If you want to know how much a big project will take, you count the tasks, multiply by the half days and there you go.
You can also get a good sense of your throughput when each story is roughly the same.
If a task takes longer or shorter, doesn't matter, it'll bring the throughput of the week down, and next time you're doing this sort of thing you'll likely split it. Just keep counting tasks.
With more sophisticated estimation you'll still get it wrong, so the point is to make the process of planning faster and simpler, since you won't get it right with a bunch of investment anyway.
That is sophisticated estimation. So in order to say a project will take a month, first you'd have to plan it in such detail to break into like 40 half day tasks? That's more detailed than most waterfall shops.
In practice I've reduced the time spent in planning sessions and gotten more reliable throughput quarter over quarter like this, but feel free to do whatever works for your team.
Is this similar to the "No Estimates" talk by Alan Holub[1]?
Correlation, not causation.
My personal experience is that unreliable or unpredictable teams end up with mandatory estimates to ease the burden of managing them.
Overperforming teams will organize themselves and management may choose not to interfere to avoid jinxing the winning combo they have.
You should look into the noestimates approach which will give you more of a skeleton around that insight.
If this study was able to clearly define a way to measure work, why don't we just use the system they came up with for estimating?
I think the point was there are two real sizes. Things you know the size of, and are small. And things that you don't know the size of, but are probably huge.
He starts with this misleading premise, which sends everything in the wrong direction:
> The software developer defence is that every software development task is doing something new, but this isn’t really true, most software development is actually fairly routine, and the problem isn’t specific to software development.
I’ve worked on teams where most tasks were routine and they did a lot of accurate estimation. These were the worst teams in terms of value delivered to the organization. Lots of headcount, cranking out lots of diffs, closing lots of Jira tickets, very little innovation. Management loves this kind of team because the output seems measurable.
The best teams focused on the velocity of the organization as a whole. Instead of making other teams file tickets to have us do routine tasks, we build a platform so they can do the tasks themselves. We look unproductive sitting on our thumbs after the platform is built, but around us everyone else is getting things done much faster.
In this working style, every task really is novel, because once we solve it, it’s solved more-or-less permanently for everyone. And the estimates suck, but things work better overall.
My discomfort with task estimation stems from the fact that the description of the task is usually not detailed enough, either because the domain experts overlooked something or forgot to tell me, or because there are simply too many potentially relevant circumstances that have not yet been nailed down.
Think of a rather innocent question like this: How much time does it take to drive the 500 km from A to B? -- How well could you answer that question for a particular case, even if you had already driven from A to B a hundred times? There is just so much that could be different this time. Still, some estimates can be made. And as the article also pointed out: Sometimes it matters a lot that the guess is right, or you need a big margin of safety (catching a plane in B), sometimes it doesn't matter at all (going to your cabin in B).
Typically, the accuracy of an estimate cannot exceed a certain level, even when all relevant known factors are included: How many hours of sunshine may we get during our trip to Scotland in week 30 of 2023?
As for estimating tasks in software development, my approach is similar, but not identical to the one described in the article: I give a worst-case and a best-case estimation, insisting that I do not promise anything because there might be unknown circumstances that are even worse than in the worst-case scenario. What is also very helpful is to divide a larger task into smaller subtasks and give an estimation interval for each of them.
Typically the worst-case estimation is about twice as long as the best-case estimation. This seems to work quite well: Almost all of the time it turns out that the actual time spent was indeed somwhere in this interval. After a while, my clients get used to this kind of interval estimates and no longer ask for a single number.
My criticism of the article is that the question "Why do you want to know?" becomes less important (but it does not disappear completely) when management is given an interval instead of a single number. It is up to them to decide whether they can live with the worst-case outcome directly or have to adjust for it elsewhere or scrap the whole project because it might not be worth it. I estimate the risks, but it is not my job to take them.
Similar/same approach here.. I break the job down into individual tasks, estimate min + max for each of those, then multiply subtotals by (usually) 1.5 for "unknown unknowns". Initially the "max" total is almost always 3-4x the "min" total, which is a sign I need to go back and get requirements nailed down better. I ask questions and tweak numbers until "max" is <2x "min" and at that point I consider the estimate finished.
Whoever needs the estimate can then make a decision based on those best and worst case scenarios. I do these numerous times a year for bits of work which are usually from a few weeks to a few months long, and they end up being accurate to the degree intended.
I consider task estimation as part of Sprint Planning an interesting domain, as multiple people in a team with potentially vastly different experience levels and skills are involved.
MacIver points out "if you’re doing sprint planning, and you absolutely have to do it through estimating individual tasks", give three point estimates a try. Here is the Wiki entry for that: https://en.wikipedia.org/wiki/Three-point_estimation
I guess explicitly recording the best-case and worst-case estimates might be interesting.
I don't think a PERT distribution accurately describes software estimates due to the upper tail being potentially infinite.
See figure 1-3 here for my mental model: https://www.microsoftpressstore.com/articles/article.aspx?p=...
(Pretty sure if this distribution has a name but I can't remember it)
One issue that could arise is that delivery managers could take the worst case estimations as an absolute truth and would think that no task could go over the worst case scenarios. I'm sure there would be a few tasks that would go over and would break planning.
But it could work if people are aware that even worst case estimates are not absolute truths and could go over the limit some times.
This talks about time estimation, I think this might be an unusual opinion, but I actually wish we did that. It'd make more sense to me than estimating 'complexity', and then pretending it's time anyway.
I agree with this. It irritates me hugely when we estimate tasks by complexity and then use our estimations to determine which tasks we can accomplish in a sprint.
Exactly - and it also ends up seeping back into the estimation as a result, 'a junior/recent hire/someone unfamiliar might pick this up, so let's make it [the higher number discussed] to account for that'. If it was actually complexity, it's just as complex whoever works on it. How hard they find it and how long they take might vary, sure, (and that's fine, and sometimes it'd be me taking longer than someone else) but not the work's complexity.
As someone who has done this professionally across multiple sectors (built environment, software, public-policy) for all types of projects and outputs both of the most concrete and most abstract, I usually cringe at a lot of "estimation" articles and discussion. This however I found to be a solid intro because it focuses on the most important thing, the answer is irrelevant or incorrect unless both parties truly know and understand "why" they need to know.
The secret to estimating for scheduling is you only really need to answer this for the critical path. The hard part is defining and uncovering what the critical path actually is, and rarely is that ever achieved in projects or programs with the necessary accuracy or reflective of reality. This is why the critical-path is often seen as a myth, and perhaps it is, but like most myths it has an important message. There will be a subset of the project that requires a priority of focus to ensure success (in time, cost, or value delivered)
Assuming a narrow range of effort but a desire to be complete as soon as possible (because money costs money), luckily, most of the time you don't need to define the critical path exactly, to do so would be nigh impossible anyway, you only need to define it to a comfortable level of abstraction which is as free from external dependencies as possible, because, as humans we are flexible; sometimes the details change and we can accommodate that on an individual basis. The sticking point is the complexity of involving many people, or skills, or processes, or settings, or conditions. At some points this will work for you, and at others against you, something else will vy for criticality and you must then understand it.
If your software already had a working architecture, everything is just plug and play. So estimation is fine. It'll eventually be deliverable.
Secondly, admit that all engineering code is tech debt, it'll pay back in future, but don't worry about it. Just be aware you should pay the debt regularly.
Gonna drop my pet conspiracy theory here and say that a Chinese person had to have come up with the asinine Fibonacci system everyone uses for estimation.
Nobody ever uses anything above 5. If you do, you end up story splitting/slicing it until you have a bunch of <5’s. You’re left with an estimation system of 1, 2, 3, and 5. Four would be really fucking convenient to use but we can’t because of the system. Who superstitiously hates the number four more than anyone? The Chinese.
I’m joking about the Chinese thing, of course, but the Fibonacci system really is stupid for estimation.
If you truly embrace the fibonnaci system, the estimation system is 0, 1, 1, 2, 3, 5, ..., as defined in the standard https://oeis.org/A000045 .
So if someone says they estimate "1", they should clarify if they mean the former one or the latter one. The former one is less likely to grow into a "2" or a "3" than the latter one, but it does risk growing into the latter "1".
I've worked on 21s before.
It's my understanding that there are multiple theoretic reasons for using Fibonacci numbers:
* The bigger they get, the further apart they are. That is helpful because estimating big things is less accurate than estimating small things.
* If the estimate gets to big, the next step is to split the task. Humans tend to err towards splitting things 50/50 (? I don't have an actual source for that) so it's generally more helpful to split a task into a slightly smaller and a slightly bigger subtask. Fibonacci numbers make that very easy, and recursively possible.
It’s a cute shorthand. Like T-shirt sizing: small, medium, large.
The point is that sizing is relative. That 2 point story kinda looks like it might be twice the work of that 1 point story.
If your story is an 8 (or xxl) it’s likely to be underestimated due to hidden complexity. That’s why you should try to split it up.
Although, it’s not like I haven’t had a bunch of “small” stories that ended up being triple-x large.
Is there actually evidence that splitting up a big task reliably reduces the time it takes to complete?
I've definitely seen cases where significant time has been spent trying to break up a story to avoid 8+ point monsters, but the result was a bunch of tasks so interdependent that any attempt to work on one inevitably ended up being held up by the need to complete others (or worse, two people end up doing the same thing). I'm pretty sure the orginal task could have been more efficiently handled by a single dev all in one go, and other unrelated tasks could have been taken on by other devs.
Great question. The only rigorous work on estimation I'm familiar with is Steve McConnell's "Software Estimation". It's been a while since I read it, but here's a paragraph from "estimating by analogy", kinda what we're talking about:
> The basic approach that Mike is using in this example is estimation by analogy, which is the simple idea that you can create accurate estimates for a new project by comparing the new project to a similar past project.
> I’ve had several hundred estimators create estimates for the Triad project. Using the approach implied in the example, their estimates have ranged from 30 to 144 staff months, with an average of 53 staff months. The standard deviation of their esti- mates is 24, or 46% of the average answer. That is not very good! A little bit of structure on the process helps a lot.
and then later on story points:
> On a numeric scale, the ratios between the numbers on the scale suggest that the underlying quantities being measured bear a proportionate relationship. If your story points scale is a Fibonacci sequence, a scale of 1, 2, 3, 5, 8, 13 suggests that a story of 5 points will take 5/3 as much effort as a story of 3 points. It suggests that a story of 13 points will take more than 4 times as much effort as a story of 3 points.
> These relationships turn out to be a double-edged sword. If the necessary care is taken to ensure that stories classified as 13 points really are about 4 times as much effort as stories classified as 3 points, that’s great. That means you can compute an average effort per story point (as described earlier), multiply the total number of story points by the average, and get a meaningful result (also as described earlier). Accomplishing this level of accuracy requires that great discipline be exercised in assigning story points to stories. It also requires checking actual project data to ensure that the ratios that are estimated are the ratios actually found in practice.
I've strayed from your question.. no I'm not aware of any evidence that this is actually worth doing. To me, the value is in trying to trying to decompose tasks and think about the consituent parts. McConnell expressly calls this out as being a double edged sword if you cannot do it properly though.
I've used it with my team for a couple of years and I don't really dislike it as long as people don't take it very seriously. It is mainly a tool to reason about what needs to be done. It's interesting when someone gives a 2 and someone gives an 8. Why 2? Why 8? Discussing about a thing makes it clearer what may be complications and things to be aware of.
Honestly I don't care about the number themselves and probably three levels are enough and may well be orange, apple, banana.
< As a result, you sadly can’t be quite as precise in your estimates as we were in our coin tossing model. The coin tossing model is the best case scenario for how accurate our estimates can be, and most real world estimates will be even harder than that. >
An estimate for building a CRUD app is a lot closer to "how long will it take you to walk a mile?" than it is to "pick from this true-random distribution".
What's the typical hour estimate range for building an MVP crud app? Professionally I have been working on developing custom functionality for WP sites and these projects rarely exceed 100h. I have been thinking about switching to web app development as I enjoy modern web technologies much more (especially TypeScript), can't work with PHP much longer. (I have not built a web app yet, but I have been working on an Electron desktop app for myself, which is a little more than a CRUD app and after 2,000h spent I estimate the total effort will take about 4,000 - 5,000h by the way). My guess would be 200h for something ultra simple and something around 500 - 1000 h for an average app with multi-user support?
Only if you’ve done it a couple of times before… which is the point about uncertainty. Estimates should become wider when you’re less certain of how that mile looks like. Is it a paved road or a mountain trail? How much work is it to figure this out?
Because it’s tiny. Tiny projects are uninteresting to estimate - you should be able to estimate them reasonably accurately.
How long will it take to build the new realtime risk engine? By the way, the team working on the FPGA deep pipeline calculators don’t think they’ll have a prototype for you before Christmas. How many devs do you need so that we’re we on track for March go-live so we avoid the $3mm/day fine for being late?
A bit meandering.. In defense of the author, they say at the beginning, it's an exploratory post without necessarily heading towards a conclusion