[Cycle times] Improving throughput calculation (based on Github label change)

[peterflynn]

Original Subject: Throughput calculation appears to be wrong

Copied from PR comment:

I think the cycle time data shouldn't actually factor into the throughput calculation at all. For example:

• If we finish 5 cards in a week in serial, each card has a 1-day cycle time -> avgCycleTime === 1
• If we finish 5 cards in a week in parallel, each card has a 5-day cycle time -> avgCycleTime === 5

The throughput should be the same in both cases: 5 cards per week. But the equation in getCategorySummary() will instead give very different answers for these two cases...

[peterflynn]

NJ & I came up with this suggestion for how to calculate it:

1. Find the earliest start date of any card completed in the sprint
2. Find the latest end date of any card completed in the sprint
3. Calculate the number of business days in this entire range
4. Divide total number of cards completed in the sprint by number of business days, then * 5 to convert to business weeks

[redmunds]

Re-assigned to me.

[redmunds]

@peterflynn I brought this up before, so I'm glad we're finally getting around to discussing it. The algorithm suggested here eliminates the overlap between cards within a release, but there could still be date ranges for cards that overlap across releases.

For example, a card for current release that was pushed from release to release could be 4 or 5 months old (e.g. "PR: #6409 Switch language / syntax mode of current document" card in current release) and really skew throughput for release.

Using most recent action date for the "end" for this release is reasonable. What we want for the "start" date is the "end" date of the previous release, right? This will eliminate overlap of dates across releases. We already prompt for "Ship It!" column for current release, so we can add a prompt for "Ship It?" column for previous release as well. How does that sound?

cc @njx @pthiess @dangoor

[dangoor]

@pthiess and I spoke about throughput a couple of days ago. My understanding of throughput is that it has nothing to do with releases whatsoever.

Throughput is "number of cards per X amount of time" where we could choose to make X a week or a month. So, to calculate throughput you just need to look at how many cards landed in the done column over a given period. We'd then average that over time.

We have the extra variable of two different card sizes, but otherwise it's the same idea.

[redmunds]

@dangoor

My understanding of throughput is that it has nothing to do with releases whatsoever.

Correct, but we just happen to organize our cards around releases, so it's convenient to use that info while we're calculating cycle times.

We could make a separate report where input is start and end time, and then search through all "Ship It!" columns for Cards that were complete during that time range. This would be addition work, but will be (somewhat?) more accurate.

[pthiess]

@redmunds

Correct, but we just happen to organize our cards around releases, so it's convenient to use that info while we're calculating cycle times.

I believe we shouldn't intermingle releases and throughput calculation. As @dangoor mentioned the two are independent. If the throughput calculation is based on releases we should fix that. The release should just be a random point in time when we review the outcome.
The idea behind Kanban as I understand it is that the focus is on throughput and not on time boxing (as opposed to sprints in SCRUM). It is not so important at which date something is landing, it is more important that it is landing as soon as possible after it has been prioritized and started. This is my understanding of optimizing the Kanban flow.
I'm inclined to put the release 43 milestone on this since we need accurate data.