The Ugly Truth (of Anxiety and Data Science)

Anxiety is a bitch. Every mental health problem is. Working as a Data Scientist seems to really make the anxiety tough to handle.

I am working on a research project exploring different aspects of argument mining, so far its been a large stream of negative results. I’ve been fairly successful at re-framing it as “learning”, but after the 100th failed attempt, my chest starts to tighten, and I can physically feel my anxiety level growing. So when the results on the screen show that I failed improve on the baseline, I get out of my chair and curl up on the rug on the floor. It is shaggy and comfortable. I do a few yoga style stretches which starts to activate my parasympathetic system again. Back at the office, I would hide on the floor in the nursing room, so rug is a bit of an upgrade.

At this moment, I start to question how I have not been fired yet and whether I am actually going to succeed in my career. It’s imposter syndrome, of course, I’ve talked about it many times with friends, mentors and therapists. The real problem right now is how exhausted the anxiety spells make me feel. The idea of sitting back down fills me with dread, it’s too much pressure. Everyone agrees that breaks are good, but often I feel like I can not focus for more than couple of hours a day . What inevitably follows is wave guilt and shame, because it seems like everyone I know is working 10–12 hour days all the damn time…the thought further drives the anxiety and exhaustion. A self-perpetuating cycle.

Tying what feels like my entire self-worth to a research project is a common (and toxic) pattern for academic work, which makes me grateful that I decided to not pursue a PhD.

But the anxiety and productivity shame follow me through every project. Earlier this year, I am developing a prototype for an internal tool for document summarization. It is a hard problem for Machine Learning, and I was skeptical that the results will be useful, some internal A/B testing confirmed this theory. The negative result was not my fault, but a product of the technology we had and the data. Still I found myself guilty and anxious: Did I make the right choice by only building a simple system? Maybe if I was actually smart and productive, I would have built a better system by now. Data does not care about anxiety. It just is. But as Data Scientists, we constantly have to decide if it is worth building a more complex model to handle a dataset or if we are better off investing in a new project. The ambiguity does not bode well for my anxiety, because making the wrong call feels too dangerous. When anxiety takes over, all mistakes feel life-ending…in practice this leaves me trying to take some deep breaths on the floor, then crying about how little I got done…again.

There is no happy thesis to this story, but over time I have discovered a few things that help:

• Good friends and family — support systems are so important.
• Actually spending time on my interests outside of work. It is easier to handle failure when your self-worth relies on multiple sources. (Rock Climbing has the added benefit of being exercise)
• Just repetition…after the 20th time you have not been fired for what felt like an unproductive few weeks, the brain starts to acquiesce to the data. (Sometimes)
• Talking about it — I tried to present an honest “dump” of my brain here. From my CBT experiences, I know that a lot of it is grounded in cognitive distortions…but it is powerful to acknowledge that and start honest conversations.