Why machine learning algorithms are hard to tune (and the fix)
Why machine learning algorithms are hard to tune (and the fix)
“In machine learning, linear combinations of losses are all over the place. In fact, they are commonly used as the standard approach, despite that they are a perilous area full of dicey pitfalls. Especially regarding how these linear combinations make your algorithm hard to tune.
Therefore, in this post we hope to lay out the following arguments:
A lot of problems in machine learning should be treated as multi-objective problems, while they currently are not.
This lack of multi-objective treatment leads to difficulties in tuning the hyper-parameters for these machine learning algorithms.
It is nigh on impossible to detect when these problems are occurring, making it tricky to work around them.
There are methods to solve this which might be slightly involved, but do not require more than a few lines of code…”
Source: engraved.ghost.io/why-machine-learning-algorithms-are-hard-to-tune/