I read a lot of Twitter every day, and my meta cognition kicks in and starts noticing why I’m going to Twitter.
When I play the Witcher, I feel this obsessive need to finish every single side quest and question mark on the map.
When I’m at a party, I feel the urge to talk to every single person.
At home on a Saturday night, even if my body desperately needs rest, I still tell myself I have to go out somehow (pre-Corona).
Underlying all of this is an anxiety that I am not optimizing to get the most out of life. Whatever is in front of me, I have to get the most joy out of it. In attempting to do so I invariably squeeze all the joy out of it.
Similarly, it can be counterproductive to optimize in our careers as well. We might be tempted to make every day or every week count. Given measures of productivity like lines-of-code or CLs/PRs sent, we think we can see our progress stall or speed up. But this is an illusion that can end up leaving us exhausted. Work in a large organization is rarely linear.
Linear optimization is a fairly straightforward problem that we intuitively understand. Progress in technology, whether on a grand societal level or an individual learning level, is decidedly nonlinear.
There are two reasons for this. One is based off of human psychology and neuroscience. The other is based on organizational theory.
Default Mode Network
Your brain consists of many layers. The conscious part that you think you use to solve problems at work is but a small sliver of your overall computational prowess. In truth, there are countless layers of processing underneath, churning away at the very same problem. The conscious you is merely the one that takes credit for solving the problem. There is a large chunk of processing that happens in the Default Mode Network - the one that is turned on when we are daydreaming, wandering and otherwise goofing off. The evidence is mounting that this daydreaming network is as useful for our problem solving as our conscious selves.
However, since this network is out of our control and functions on its own time schedule, we cannot squeeze its output into a linear schedule. One must allow for nonlinear processes like daydreaming and mind-wandering to happen on their own. The famous mathematician Poincare once wrote:
I then began to study arithmetical questions without any great apparent result, and without suspecting that they could have the least connexion with my previous researches. Disgusted at my want of success, I went away to spend a few days at the seaside, and thought of entirely different things. One day, as I was walking on the cliff, the idea came to me, again with the same characteristics of conciseness, suddenness, and immediate certainty, that arithmetical transformations of indefinite ternary quadratic forms are identical with those of non-Euclidian geometry. — Henri Poincaré Science and Method (1908), trans. Francis Maitland (1914), 53-4.
Problem-solvers of any age knew of the power of mind-wandering. It is the fount of incredible, supralinear jumps in insight. Harnessing those involves letting go of the greedy algorithm linear optimization mindset that focuses on lines-of-code per week or quarter.
Organizational Crush
It takes only eight minutes for a photon to get from the surface of the sun to the earth. However, it can take nearly 100,000 years for that same photon to make it from the core of the sun, where it was generated, to the surface.
This is what I like to call the organizational crush. Organizations are incredibly dense, like the sun. They need to be, with lots of particles bumping into each other to fuse and create really powerful things. A lower density organization simply wouldn’t be able to reach the critical mass necessary for a self-sustaining nuclear reaction. But unleashing that creativity requires jumping many hoops.
Projects do not progress linearly within large or small organizations. Large organizations attempt to replicate the challenges of the outside world in a streamlined fashion internally so that your ultimate launch is successful. A good example here is l10n or i18n or accessibility. These teams will help you to build your product so that you don’t encounter issues once the product has been launched. Either way, the world will throw up roadblocks to the progress of your project - and so your lines-of-code and other measures of productivity will dip at inopportune times.
What does one do about this?
Rather than attempt to optimize the output of each time unit, attempt to build momentum. Every day is an opportunity to deepen understanding of the overall codebase and this is about doing random walks, which are inherently chaotic and return unexpected results. Instead of optimizing for output every day, optimize for serendipity.