Chapter 21: Dissertation.

2011-06-11.

The day of my dissertation. A strange feeling of calmness invaded me —in the morning, just before facing the elements. I had always thought that it was going to be a pivotal moment in my life... yet, having reached the moment, it didn't seem all that transcendental.

Don't get me wrong, I was reasonably ecstatic! If successful, it meant a change for the better, the culmination of six years of alienating work —of six years of a short-sighted limbo where aimless promises had been constantly sprayed over me, often by me.

It was the same day were I could understand what I was —and where my future was lying. I stopped being myopic about my essence: the hard work scattered over those six years, the exasperatingly intermittent tour-de-force had finally made sense.

At least for a while. I should have known already that those transitory states of mental clarity, boldness, and strength are simply illusions.

That was the reasoning for ecstasy. But I lied: No. I wasn't ecstatic. I was —and would remain— unmoved. A crucial time bringing nothing but indifference. As deep inside, I knew I had kept looking away from the bonfire.

Maybe indifference was a defensive mechanism. If so, it was quite effective: it soothed my nerves. Those can betray you in the defense.

In the slice of absolute calm, my problem had been to realize the pointlessness of it all. I wanted to get out of my body and astrally travel to where I would not care about those things I had cared too much about. Some spiritual realm of peace and love. And a lake. I could hear the flames —the wood crackles and hisses turning into bird's chirps.

At the same time, at the back of my mind, I feared a different kind of chirping. Her criticism. Even hundreds of miles away. Looping thoughts were insinuating the hellish nightmare of failing the dissertation. Fortunately, that slice was to be subdued. My lethargic state kept that cause for anxiety at bay. Still, I kept alternating between anxiety and apathy. So fast, that I felt I was in both states at the same time. Maybe it was multitasking, but it felt like multiprocessing.

The dissertation went well. There was no improvisation. I felt like an automaton: I had gone through all of it so many times that I could almost do it without any conscious intervention.

None of the questions was surprising.

The examiners really like my suggested methods and ideas around applying genetic algorithms to improve recurrent neural networks —to automatically find proper architectures, among other things. I believed that, as parallel processing power would keep growing, artificial neural networks were going to become much bigger, and more relevant in more diverse fields —not just computer vision.

The main idea, within the area of deep supervised learning, comprised some clever techniques regarding automatic optimization of a network. After training the network and assessing its mean error with the validation dataset, an algorithm would determine the mutations with a higher chance of being better. Sort of a guided evolution.

The ‘mutations’ changed the parameter set, constants and topology of the network. Activation function types per layer, et cetera. Another network was used —training it on the fly— to try to steer mutations in the right direction. These changes include what would be increasingly known as —and ultimately, the proper and accepted term, at least a few years later— hyperparameters.

Another step for those changes not to be done in a random involved keeping track of a tree with the lineages of those configurations and using yet another small network to guess which branch was most worthy of processing a mutation —all while keeping the genetic algorithm from indirectly overfitting on validation data.

There were a few novel ideas in those methods, for the time. Although, in essence, I was mixing concepts of genetic programming and deep learning, and over-engineering the whole thing. Some subsystems didn't work as expected.

The use of GPUs for machine learning was starting to be popular, but my implementation was purely CPU-based. It was a chaotic implementation, part calling GotoBLAS, part including its own vector calculus functions. Not that it mattered, as clearly the same principles could be easily cleaned up and ported elsewhere. Still, it was a functional implementation for the paper.

It went well, it ended well. The deliberation was quick and so was the verdict: I passed. No corrections required.

They appeared to be proud of me. No one who mattered was proud of me. But at last, I had finished. I only wished that I had been emotionally awake to enjoy those times of liberation.