Confusion

I am working through a task at work of reproducing a dataset using a long SQL script consisting of separate chunks of data manipulation and manual edits (to the data table, not the code). As often happens, documentation is less than perfect and I am left to figure out much of what I am doing in my own. This naturally leads to some confusion - what should I be looking for, how do I find an issue to correct, how should I correct the issue when I find it etc.

In grad school, as someone with an unusual background, I also spent a lot of time confused, unsure about how this concept related to that, the interpretation of this coefficient or that derivative.

This feeling of being intellectually unmoored is uncomfortable. It makes my insides itch and triggers the instinct to get up and walk away - go do something else, have a snack (hungry or not), scroll through Twitter, check my email - anything to get away from this sensation of groping in the dark for something whose shape I only have the vaguest impression of.

There are benefits to this confusion. One, it’s an indicator that there is something here to figure out, something I don’t understand that’s going to take some work to figure out. If I can use confusion to my advantage, then I will come out the other side more fully understanding whatever it is I am grappling with. This is great. I will be smarter! I will be more competent! I will feel confident!

The problem is that the confusion sucks. Unless it’s a task that is inherently interesting or that currently has a grip on my curiosity, stumbling about in the dark can be quite uncomfortable. I would like to get better at sitting with this confusion and traversing is contours when I’m sitting in its midst.

Other questions:

  • What are the limits to the benefits of confusion? How do we know when to stop groping around on our own and call out for a hand?
  • How to find the optimal path through the swamp of confusion? When might the fastest way out not be the best way or (i.e. when is it better to muck about in the mud for a while?
  • How to become more comfortable down in the swamp (of confusion)? This is the motivation for this reflection, really.