The Internet of Things (IoT) is revolutionizing the industry. Powered by pervasive embedded devices, the Industrial IoT (IIoT) provides a unique solution for retrieving and analyzing data near the source in real-time. Many emerging techniques, such as Tiny Machine Learning (TinyML) and Complex Event Processing (CEP), are actively being developed to support decision making at the edge, shifting the paradigm from centralized processing to distributed computing. However, distributed computing presents management challenges, as IoT devices are diverse and constrained, and their number is growing exponentially. The situation is even more challenging when various on-device applications (so-called artifacts) are deployed across decentralized IoT networks. Questions to be addressed include how to discover an appropriate function, whether that function can be executed on a certain device, and how to orchestrate a cross-platform service. To tackle these challenges, we propose an approach for the scalable management of on-device applications among distributed IoT devices. By leveraging the W3C Web of Things (WoT), the capabilities of each IoT device, or more precisely, its interaction patterns, can be semantically expressed in a Thing Description (TD). In addition, we introduce semantic modeling of on-device applications to supplement an TD with additional information regarding applications on the device. Specifically, we demonstrate two examples of semantic modeling: neural networks (NN) and CEP rules. The ontologies are evaluated by answering a set of competency questions. By hosting the enriched semantic knowledge of the entire IoT system in a Knowledge Graph (KG), we can discover and interoperate edge devices and artifacts across the decentralized network. This can reduce fragmentation and increase the reusability of IoT components. We demonstrate the feasibility of our concept on an industrial workstation consisting of a conveyor belt and several IoT devices. Finally, the requirements for constructing an IoT semantic management system are discussed.