This post contains a summary and reviews of my research content.
Concept of duality
In general, when solving the optimization problem a maximization problem can be associated to a given minimization problem so that both problems have the same optimal values. This principle is designed in a way that, under appropriate assumptions, a minimal element of a subset of a partially ordered linear space is also a maximal element of an associated set. We will explain the duality as much as possible using the pictures instead of the complex formulas.
For example, suppose that set P and D are defined as shown in the figure below. Here, set D is a dual set and set P is a primal set. D and P are generated by the cone defined in the partially ordered linear space, but the description is skipped. The general concept of duality is that the minimization problem for set P can be regarded as a maximization problem for set D that duality holds.
This is all that is most easily explained. In the end, when we solve an optimization problem, we choose whether to solve the problem as a minimization problem or as a maximization problem.
Learning algorithms generally have a large number of learning parameters, so optimization of min & max methods will obviously have different optimal values. Therefore, we can solve the optimization problem more easily by applying duality to the problem to be solved.
Application
A representative example of using duality for machine learning problem solving is the Wasserstein GAN with Kantorovich-Rubenstein Duality.
It defines cost using duality in metric space, and shows its interpretation.
What we need to keep in mind is that the function of infimum has been changed to supremum.
More specifically, the expression for cost is modified by the difference value for the function to which the lipschitz constraint is applied.
This allows us to define the value of cost in the metric space more intuitively.
That's it. This is my first post and I will post more interesting topics in the future.
Reference
- (2018). Arxiv.org. Retrieved 24 May 2018, from https://arxiv.org/pdf/1805.09575.pdf
