Incorporating Human Domain Knowledge in 3-D LiDAR-Based Semantic Segmentation

Mei, Jilin; Zhao, Huijing

doi:10.1109/tiv.2019.2955851

Cited by 22 publications

(11 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It attempts to build digital sensing systems in the cloud that integrate both high-fidelity sensor models and the surrounding environment. In addition to the physical environment, social environments and human behaviors also have a significant impact on how well sensing systems operate [20]. Descriptive sensing is the pioneering work that takes into account of social environment with human factors and can be used to generate more realistic virtual data compared with simple digital twins' sensor models.…”

Section: A Descriptive Sensing Systemsmentioning

confidence: 99%

Parallel Sensing in Metaverses: Virtual-Real Interactive Smart Systems for “6S” Sensing

Shen¹,

Liu²,

Tian

et al. 2022

IEEE/CAA J. Autom. Sinica

View full text Add to dashboard Cite

In the construction of Metaverses, sensors that are referred to as the "bridge of information transmission", play a key role. The functionality and efficiency of today's sensors, which operate in a manner similar to physical sensing, are frequently constrained by their hardware and software. In this research, we proposed the Parallel Sensing framework, which includes background, concept, basic methods and typical application of parallel sensing. In our formulation, sensors are redefined as the integration of real physical sensors and virtual software-defined sensors based on parallel intelligence, in order to boost the performance of the sensors. Each sensor will have a parallel counterpart in the virtual world within the framework of parallel sensing. Digital sensors serve as the brain of sensors and maintain the same properties as physical sensors. Parallel sensing allows physical sensors to operate in discrete time periods to conserve energy, while cloud-based descriptive, predictive, and prescriptive sensors operate continuously to offer compensation data and serve as guardians. To better illustrate parallel sensing concept, we show some example applications of parallel sensing such as parallel vision, parallel point cloud and parallel light fields, both of which are designed by construct virtual sensors to extend small real data to virtual big data and then boost the performance of perception models. Experimental results demonstrate the effective of parallel sensing framework. The interaction between the real and virtual worlds enables sensors to operate actively, allowing them to intelligently adapt to various scenarios and ultimately attain the goal of "Cognitive, Parallel, Crypto, Federated, Social and Ecologic" 6S sensing.

show abstract

Section: A Descriptive Sensing Systemsmentioning

confidence: 99%

Parallel Sensing in Metaverses: Virtual-Real Interactive Smart Systems for “6S” Sensing

Shen¹,

Liu²,

Tian

et al. 2022

IEEE/CAA J. Autom. Sinica

View full text Add to dashboard Cite

show abstract

“…The knowledge-driven models are derived from domain knowledge, including crop phenology and growth models, which can be used to simulate crop growth and development. With the development of AI, IoT and big data, studies on integrating these two types of modeling approaches have been conducted to take advantage of both knowledge and data-driven models to reduce the demand for the mass of data and improve the training efficiency, leading to so-called knowledge-and data-driven models, such as those used in [21] to predict crop yield and in [36] to incorporate human domain knowledge into the neural network model in semantic segmentation.…”

Section: Foundation Model For Agriversementioning

confidence: 99%

DeCASA in AgriVerse: Parallel Agriculture for Smart Villages in Metaverses

Wang

Kang

Sun

et al. 2022

IEEE/CAA J. Autom. Sinica

View full text Add to dashboard Cite

The demand for food is tremendously increasing with the growth of the world population, which necessitates the development of sustainable agriculture under the impact of various factors, such as climate change. To fulfill this challenge, we are developing Metaverses for agriculture, referred to as AgriVerse, under our Decentralized Complex Adaptive Systems in Agriculture (DeCASA) project, which is a digital world of smart villages created alongside the development of Decentralized Sciences (DeSci) and Decentralized Autonomous Organizations (DAO) for Cyber-Physical-Social Systems (CPSSs). Additionally, we provide the architectures, operating modes and major applications of DeCASA in Agri-Verse. For achieving sustainable agriculture, a foundation model based on ACP theory and federated intelligence is envisaged. Finally, we discuss the challenges and opportunities.

show abstract

“…Some works utilize scene-level or sub-cloud-level weak labels [44,54]. There are also several approaches using rule-based heuristics or handcrafted features to help annotation [37,51,20].…”

Section: Label-efficient 3d Semantic Segmentationmentioning

confidence: 99%

LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds

Liu¹,

Yin²,

Qi³

et al. 2022

Preprint

View full text Add to dashboard Cite

Semantic segmentation of LiDAR point clouds is an important task in autonomous driving. However, training deep models via conventional supervised methods requires large datasets which are costly to label. It is critical to have label-efficient segmentation approaches to scale up the model to new operational domains or to improve performance on rare cases. While most prior works focus on indoor scenes, we are one of the first to propose a label-efficient semantic segmentation pipeline for outdoor scenes with LiDAR point clouds. Our method co-designs an efficient labeling process with semi/weakly supervised learning and is applicable to nearly any 3D semantic segmentation backbones. Specifically, we leverage geometry patterns in outdoor scenes to have a heuristic pre-segmentation to reduce the manual labeling and jointly design the learning targets with the labeling process. In the learning step, we leverage prototype learning to get more descriptive point embeddings and use multi-scan distillation to exploit richer semantics from temporally aggregated point clouds to boost the performance of single-scan models. Evaluated on the SemanticKITTI and the nuScenes datasets, we show that our proposed method outperforms existing label-efficient methods. With extremely limited human annotations (e.g., 0.1% point labels), our proposed method is even highly competitive compared to the fully supervised counterpart with 100% labels.

show abstract

Incorporating Human Domain Knowledge in 3-D LiDAR-Based Semantic Segmentation

Cited by 22 publications

References 29 publications

Parallel Sensing in Metaverses: Virtual-Real Interactive Smart Systems for “6S” Sensing

Parallel Sensing in Metaverses: Virtual-Real Interactive Smart Systems for “6S” Sensing

DeCASA in AgriVerse: Parallel Agriculture for Smart Villages in Metaverses

LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds

Contact Info

Product

Resources

About