DevDiary

In the last post, I wrote about how to setup an eGPU on Ubuntu to get started with TensorFlow. I shortly mentioned that a eGPU is definitely worth it for Machine Learning, but I did not tell any numbers. This article tries to catch up on that. There are a gazillion benchmarks already out there about GPU gaming performance. A handful of them also include eGPU benchmarks, like this one for example. One important thing that that these eGPU benchmarks show is that the use of an external display is from benefit, because otherwise the Thunderbolt 3 port becomes a bigger bottleneck. This is due to the fact that each rendered frame has be sent back to the internal display, which consumes valuable bandwidth of the 40 Gbps Thunderbolt 3 connection. However, there is no alternative for using an eGPU for Machine Learning, because the computed gradients have to be sent back to the CPU. Talking about Machine Learning, there are a few articles our official benchmarks about TensorFlow performanc...

I remember when I read about eGPUs for the first time. The symbiosis of having a light weight laptop at university or on the go, but still having a desktop like power horse when having some spare-time at home sounded like a dream. But everything faded into obscurity because I almost lost full interest into gaming the last years. But this changed, since I'm spending a lot of time in deep learning since about two years. And it's well known that taking advantage of a GPU boosts training time by a huge margin. That's why I tried to get access to a high-performance graphics card in order to be able to train non-trivial networks and so some more serious research. At first, I had a look at some offers in the cloud. I did not try out a GPU-enabled instance on AWS, because the use a billing based on a hourly rate. This means that you have to pay for a full hour, even when you just run a simple example for 1 minute. Next, I checkout out the 300$ free credit on Google compute engi...

While checking out some tools for automated hyperparameter optimization, I came across a quite popular library called Hyperopt . It provides an implementation for Random Search and Tree-of-Parzen-Estimators (TPE). Unfortunately, most examples out there us a dummy function to replace the model, but I could not find any example that uses TensorFlow. That's why I wanted to provide a basic simple Hyperopt example with TensorFlow. This example can be found on my Github . Do you have any experiences with other libraries for hyperparameter optimization? I would be happy if you share your experiences? For instance, I have read that a  Sacred  extension called Labwatch  also allows to define a search space for algorithmic hyperparameter optimization, but comes with different algorithms.

There are various ways to install TensorFlow. For instance, you can install it using a Docker image or Python's package manager pip. But since the version 1.0 release of TensorFlow, you probably might have faced the following warnings each time you run a TensorFlow session: 2017-05-29 11:50:22.977500: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations. 2017-05-29 11:50:22.977513: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations. 2017-05-29 11:50:22.977517: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations. 2017-05-29 11:50:22.977519: W tenso...

In the course of the development of my Master's Thesis "Deep Learning Approaches to Predict Future Frames in Videos" at TUM, I realized that the high flexibility of TensorFlow has its price: boilerpate code. Many things that are needed in almost every neural network training or evaluation script have to be implemented over and over again. To that end, I started to implement a high-level API for Google's machine intelligence library, called TensorLight . TensorLight comes with four guiding principles: Simplicity:  Straight-forward to use for anybody who has already worked with TensorFlow. Especially, no further learning is required regarding how to define a model's graph definition. Compactness:  Reduce boilerplate code, while keeping the transparency and flexibility of TensorFlow. Standardization:  Provide a standard way in respect to the implementation of models and datasets in order to save time. Further, it automates the whole training and validatio...

I finally finished my Master's Thesis in the Computer Vision chair at TUM. In the course of this thesis, I analyzed existing deep learning approaches to predict future frames in videos. Based on these findings and other modern deep learning practices, such as batch normalization, scheduled sampling to improve recurrent network training or ConvLSTMs, we were able to reach or event outperform state-of-the-art performance in future frame generation. So far, many people asked me about the practical application of frame prediction. Unfortunately, it won't tell us the end of any cliff-hanger movie such as Inception, but the main purpose of such a system is not to generate a perfect forecast of the long-term continuation of any movie clip. This completely impossible in my opinion, since there is not always a wrong or right in many situations. A neural network cannot be able to predict every decision made by all objects inside the scene. Furthermore, the pose of the camera or the ...