Learning Small Decision Trees With Few Outliers: A Parameterized Perspective

Decision trees are a fundamental tool in machine learning for representing, classifying, and generalizing data. It is desirable to construct “small” decision trees, by minimizing either the size (s) or the depth (d) of the decision tree (DT). Recently, the parameterized complexity of DECISION TREE LEARNING has attracted a lot of attention. We consider a generalization of DECISION TREE LEARNING where given a classification instance E and an integer t, the task is to find a “small” DT that disagrees with E in at most t examples. We consider two problems: DTSO and DTDO, where the goal is to construct a DT minimizing s and d, respectively. We first establish that both DTSO and DTDO are W[1]-hard when parameterized by s+δmax and d+δmax, respectively, where δmax is the maximum number of features in which two differently labeled examples can differ. We complement this result by showing that these problems become FPT if we include the parameter t. We also consider the kernelization complexity of these problems and establish several positive and negative results for both DTSO and DTDO.