The importance of being idle

Years ago, when I first moved into management, I seemed to suddenly have a lot of time on my hands: It looked as though the business had decided to give me more responsibilities, and yet needed me less at the same time. Sure enough, I had my projects and my meetings, but sometimes entire days would go by when the company simply placed no demands on me. I could have, figuratively speaking, fallen off the face of the Earth without anyone noticing (at least, I suppose, until I started missing deadlines).

At first, I found this very disconcerting. In my previous positions, one was expected to “put in a full day’s work” every day, after all. Eight-hour days were the norm, and late nights weren’t unusual. I didn’t want to discuss this with anyone, because I was afraid my superiors would suddenly realize I was redundant, and I felt like a bit of a fraud for having so many spare cycles.

Eventually, I started noticing two patterns. First, nobody was complaining about my performance; clearly, however small my perceived workload, the people above me thought I was pulling my weight. More importantly, the fact that I wasn’t always busy meant that I could easily jump on any new or unexpected problem with a great deal of speed. Since, as a manager, my job consists of making other people productive, this made it possible for me to positively impact a lot of ongoing work.

It wasn’t until years later that I finally realized what was going on, and the explanation to why having so little to do made me more productive came from a remarkable concept in queueing theory called Little’s Law, which states that the pace at which work moves through a queue is directly proportional to the number of new jobs that arrive into it, and inversely proportional to the time any single job spends inside it.

Intuitively, this makes a lot of sense: The more jobs are waiting in the queue, the more can be processed, and the longer it takes to process one job, the fewer we can process for each unit of time.

Less intuitively—at least to me—Little’s Law establishes a relationship between the wait time at the front of the queue and the queue’s utilization:

$$ t_w = p_b / p_i $$

Where $$t_w$$ is the wait time, $$p_b$$ is the percentage of time the resource is busy, and $$p_i$$ is the percentage of time the queue is idle.

Nothing that surprising here: If the queue is completely empty, it can accept new work right away, and the wait time is zero. Conversely, if it’s always busy, it will never be able to accept any new work.

Since the “idle time” is essentially the same as “all time minus busy time,” however, we can rewrite this equation as follows:

$$ t_w = p_b / (1 - p_b) $$

This becomes interesting when we plot it on a chart:

As you can see, the relationship between busy time and wait time is not linear; at 50% utilization, the wait time is one unit, but, as we approach full utilization, it asymptotically tends to infinity. In other words, it takes an increasingly massive amount of effort for a resource to be able to pivot from one unit of work to another, and its agility suffers as a consequence.

Applied to a team, this concept tends to trip up a inexperienced managers who schedule their teams to full utilization based on the work they are aware of. When, inevitably, the businesses asks them to deal with something off-menu, which they must handle right away without stopping the current workload, they find themselves stuck between two irreconcilable demands, and their performance suffers despite their best efforts.

Of course, idle time is not lazy time: It can be invested in non-critical tasks that still bring benefit to the business and can be interrupted without consequence. For a person, this could mean planning for the long term, learning something new, or—why not—writing a blog post. A team could take advantage of idle time to work on speculative research projects, perform some maintenance tasks, and so on.

If idle team is necessary, though, how much idle time is the right amount? After all, a resource that is always idle is, without a doubt, wasted (not to mention bored and, most likely, looking for a new job).

The answer is beyond the scope of this blog post, but it’s not that hard to intuit: One must be constantly be aware of the amount of work that is being scheduled into a system, and ensure that the rate of arrival for new work never exceeds capacity minus an amount of slack sufficient to deal with the unexpected. That’s easier said than done, perhaps, but also necessary to long-term success.

Did you find this interesting? You can comment on Hacker News, and follow me on Twitter.