Nowadays, with the fast development of backbone networks, performance monitoring and troubleshooting on IP-over-optical networks have become increasingly important. However, a real-time monitoring scheme, which is programmable to reveal the end-to-end multilayer information of an arbitrary flow in an IP-over-optical network, is still absent. Also, for such a monitoring scheme, how to balance the tradeoff between the accuracy and the overhead has not been studied yet. To address these challenges, we design a P4-based flexible multilayer in-band network telemetry (ML-INT) system to visualize an IPover-optical network in real time. Specifically, the flexible ML-INT scheme only selects a small portion of packets in an IP flow to encode INT headers, while each INT header only contains a part of the statistics of all the electrical/optical network elements (NEs) on the flow's routing path. We design the packet processing pipelines for the scheme, program P4-based hardware programmable data-plane (PDP) switches to implement the pipelines, develop the optical performance monitors that can cooperate with the PDP switches to facilitate ML-INT, and implement a high-performance data analyzer that can extract, parse, and analyze the INT data carried by the high-speed IP flows [i.e., with an arrival rate up to 2 million packets per second (Mp/s)]. The whole flexible ML-INT system is experimentally demonstrated in a small-scale but real IP-over-optical network testbed. The experimental results verify that our proposal only introduces very small overhead and can make the IP-over-optical network more visible in real time for performance monitoring and troubleshooting. INDEX TERMS IP-over-optical network, multilayer in-band network telemetry (ML-INT), programmable data-plane (PDP). I. INTRODUCTION Recently, with the development of emerging network services, both the traffic and infrastructure of backbone networks are undergoing dramatic changes [1]. Specifically, the traffic has been not only growing exponentially in volume but also becoming increasingly bursty and dynamic [2], [3], and this has applied heavy pressure on the rigid IPover-optical infrastructure of backbone networks [4]-[6]. Therefore, network operators first need a much more flexible network architecture that can build dynamic lightpaths adaptively in the optical layer for time-variant traffic flows in The associate editor coordinating the review of this manuscript and approving it for publication was Wei Quan.