Beyond Adversarial Training: A Robust Counterpart Approach to HSVM

Wait 5 sec.

Table of LinksAbstract and 1. IntroductionRelated WorksConvex Relaxation Techniques for Hyperbolic SVMs3.1 Preliminaries3.2 Original Formulation of the HSVM3.3 Semidefinite Formulation3.4 Moment-Sum-of-Squares RelaxationExperiments4.1 Synthetic Dataset4.2 Real DatasetDiscussions, Acknowledgements, and References\A. ProofsB. Solution Extraction in Relaxed FormulationC. On Moment Sum-of-Squares Relaxation HierarchyD. Platt Scaling [31]E. Detailed Experimental ResultsF. Robust Hyperbolic Support Vector MachineF Robust Hyperbolic Support Vector MachineIn this section, we propose the robust version of hyperbolic support vector machine without implemention. This is different from the practice of adversarial training that searches for adversarial samples on the fly used in the machine learning community, such as Weber et al. [7]. Rather, we predefine an uncertainty structure for data features and attempt to write down the corresponding optimization formulation, which we call the robust counterpart, as described in [42, 43].\ \ \Then, by adding the uncertainty set to the constraints, we have\ \where the last step is a rewriting into the robust counterpart (RC). We present the 𝑙∞ norm bounded robust HSVM as follows,\ \Note that since 𝑦𝑖 ∈ {−1, 1}, we may drop the 𝑦𝑖 term in the norm and subsequently write down the SDP relaxation to this non-convex QCQP problem and solve it efficiently with\ \For the implementation in MOSEK, we linearize the 𝑙1 norm term by introducing extra auxiliary variables, which we do not show here. The moment relaxation can be implemented likewise, since this is constraint-wise uncertainty and we preserve the same sparsity pattern so that the same sparse moment relaxation applies.\ \:::infoAuthors:(1) Sheng Yang, John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA (shengyang@g.harvard.edu);(2) Peihan Liu, John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA (peihanliu@fas.harvard.edu);(3) Cengiz Pehlevan, John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA, Center for Brain Science, Harvard University, Cambridge, MA, and Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University, Cambridge, MA (cpehlevan@seas.harvard.edu).::::::infoThis paper is available on arxiv under CC by-SA 4.0 Deed (Attribution-Sharealike 4.0 International) license.:::\