Recursive Teaching Dimension, Learning Complexity, and Maximum Classes

Abstract

Abstract. This paper is concerned with the combinatorial structure of concept classes that can be learned from a small number of examples. We show that the recently introduced notion of recursive teaching dimension (RTD, reflecting the complexity of teaching a concept class) is a relevant parameter in this context. Comparing the RTD to self-directed learning, we establish new lower bounds on the query complexity for a variety of query learning models and thus connect teaching to query learning. For many general cases, the RTD is upper-bounded by the VC-dimension, e.g., classes of VC-dimension 1, (nested differences of) intersection-closed classes, "standard" boolean function classes, and finite maximum classes. The RTD thus is the first model to connect teaching to the VC-dimension. The combinatorial structure defined by the RTD has a remarkable resemblance to the structure exploited by sample compression schemes and hence connects teaching to sample compression. Sequences of teaching sets defining the RTD coincide with unlabeled compression schemes both (i) resulting from Rubinstein and Rubinstein's corner-peeling and (ii) resulting from Kuzmin and Warmuth's Tail Matching algorithm.

Sauer’s Bound for a Notion of Teaching Complexity

“…For example, maximum classes form one of the few general cases of concept classes known to have labeled and unlabeled sample compression schemes of the size of their VC-dimension [3,5]. Moreover, the recursive teaching dimension (RTD, a complexity parameter of the recently introduced recursive teaching model [13]) of any maximum class equals its VC-dimension [2].…”

“…Recent work [2] indicates connections between the VC-dimension and the RTD; besides maximum classes, several other types of concept classes are shown to have an RTD upper-bounded by their VC-dimension. An open question is whether or not the RTD has an upper bound linear in the VC-dimension.…”

“…The following definitions are based on [2,13]. A teaching plan for a concept class C is a sequence P = ((c 1 , S 1 ), .…”

“…We will also use the notation RTD * (C) = max X ⊆X RTD(C| X ). The RTD has the following properties [2,13]: -RTD is monotonic, i.e, RTD(C ) ≤ RTD(C) whenever C ⊆ C.…”

Order Compression Schemes

“…The precise relationship between these two combinatorial parameters is hence of great interest to the learning theory community, as witnessed by a continuing series of publications on the topic, see, e.g., [2,4,5,8,11,12,14]. Floyd and Warmuth [8] conjectured that any concept class C of VC-dimension d It is even open whether or not the sample compression number is linear in the VC-dimension.…”