Find your model's optimal hyperparameters with Hyperopt

While checking out some tools for automated hyperparameter optimization, I came across a quite popular library called Hyperopt. It provides an implementation for Random Search and Tree-of-Parzen-Estimators (TPE). Unfortunately, most examples out there us a dummy function to replace the model, but I could not find any example that uses TensorFlow. That's why I wanted to provide a basic simple Hyperopt example with TensorFlow. This example can be found on my Github.

Do you have any experiences with other libraries for hyperparameter optimization? I would be happy if you share your experiences? For instance, I have read that a Sacred extension called Labwatch also allows to define a search space for algorithmic hyperparameter optimization, but comes with different algorithms.

Comments

Post a Comment

Popular posts from this blog

Benchmarking Tensorflow Performance on eGPU

Image
In the last post, I wrote about how to setup an eGPU on Ubuntu to get started with TensorFlow. I shortly mentioned that a eGPU is definitely worth it for Machine Learning, but I did not tell any numbers. This article tries to catch up on that. There are a gazillion benchmarks already out there about GPU gaming performance. A handful of them also include eGPU benchmarks, like this one for example. One important thing that that these eGPU benchmarks show is that the use of an external display is from benefit, because otherwise the Thunderbolt 3 port becomes a bigger bottleneck. This is due to the fact that each rendered frame has be sent back to the internal display, which consumes valuable bandwidth of the 40 Gbps Thunderbolt 3 connection. However, there is no alternative for using an eGPU for Machine Learning, because the computed gradients have to be sent back to the CPU. Talking about Machine Learning, there are a few articles our official benchmarks about TensorFlow performanc...
Read more

Setup Dell C1660W printer in Ubuntu

To be honest, this post is more or less a self-memo for myself. Nevertheless, this might be still helpful for others. While drivers for almost every hardware components or peripheral devices are automatically detected and installed in Windows, this is often not the case in Ubuntu. For instance, my Dell C1660W printer was not working in a plug & play fashion on Ubuntu 17.10. In order to setup this printer, I had to do the following steps: 1. Add a device in Settings > Devices > Printers 2. Download and install the Xerox Phaser 6000 drivers 3. Open the Printer Details of the printer added earlier 4. The the Xerox Phaser 6000B v1.0 driver using Select form Database... The printer is now ready to use. However, the driver does not fully work as expected. As an example, after every printing job, I have to restart the printer. Otherwise, the printer is simply rejecting the job, or it prints the job with some minutes delay outta nowhere.
Read more

How to setup an eGPU on Ubuntu for TensorFlow

Image
I remember when I read about eGPUs for the first time. The symbiosis of having a light weight laptop at university or on the go, but still having a desktop like power horse when having some spare-time at home sounded like a dream. But everything faded into obscurity because I almost lost full interest into gaming the last years. But this changed, since I'm spending a lot of time in deep learning since about two years. And it's well known that taking advantage of a GPU boosts training time by a huge margin. That's why I tried to get access to a high-performance graphics card in order to be able to train non-trivial networks and so some more serious research. At first, I had a look at some offers in the cloud. I did not try out a GPU-enabled instance on AWS, because the use a billing based on a hourly rate. This means that you have to pay for a full hour, even when you just run a simple example for 1 minute. Next, I checkout out the 300$ free credit on Google compute engi...
Read more