JoinABLe: Learning Bottom-up Assembly of Parametric CAD Joints

Willis, Karl D. D.; Jayaraman, Pradeep Kumar; Chu, Hang; Tian, Yunsheng; Li, Yifei; Grandi, Daniele; Sanghi, Aditya; Tran, Linh; Lambourne, Joseph G.; Solar-Lezama, Armando; Matusik, Wojciech

doi:10.1109/cvpr52688.2022.01539

Cited by 28 publications

(9 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…AutoMate [12] was created for analyzing CAD assemblies, but focuses on modeling the individual mating relationship between mechanical parts without considering target shapes. Fusion 360 Gallery [19,5] is similar to AutoMate, but not constrained to pairwise assembly. Objects are decomposed to assembly parts that match the manufacturing scenario.…”

Section: Related Workmentioning

confidence: 99%

“…Shape assembly has been studied from various perspectives. A line of research focuses on learning to predict the poses of parts that assemble the target shape [3,4,5,20]. These works use shapes from synthetic datasets such as PartNet [9] and do not consider the motion planning of moving and mating the part.…”

Section: Related Workmentioning

confidence: 99%

“…While there is a growing interest in building autonomous models that tackle the shape assembly task in the vision [3,4,5] and robotics communities [6,7,8], the information in human-designed manuals is largely overlooked. We identify two reasons: first, most works leverage shapes from synthetic datasets such as PartNet [9] as the assembly targets, which do not have corresponding humandesigned manuals; second, directly leveraging the information from visual manuals is challenging: these manuals are delivered in plain unstructured images, guiding humans via visual elements including arrows and the 2D projections of 3D shapes.…”

Section: Introductionmentioning

confidence: 99%

“…In summary, we introduce IKEA-Manual, a dataset for understanding shape assembly from 3D shapes and human-designed visual manuals. It contains 102 3D objects of IKEA furniture paired with visual Annotation Type PartNet [9] Pix3D [11] Fusion 360 Gallery [5] AutoMate [12] IKEA Furniture [6] IKEA OS [13] assembly manuals. We provide dense annotations on the 3D models and assembly manuals, as well as correspondences between 3D assembly parts and their projections in the visual manuals.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

IKEA-Manual: Seeing Shape Assembly Step by Step

Wang¹,

Zhang²,

Mao³

et al. 2023

Preprint

View full text Add to dashboard Cite

Human-designed visual manuals are crucial components in shape assembly activities. They provide step-by-step guidance on how we should move and connect different parts in a convenient and physically-realizable way. While there has been an ongoing effort in building agents that perform assembly tasks, the information in human-design manuals has been largely overlooked. We identify that this is due to 1) a lack of realistic 3D assembly objects that have paired manuals and 2) the difficulty of extracting structured information from purely image-based manuals. Motivated by this observation, we present IKEA-Manual, a dataset consisting of 102 IKEA objects paired with assembly manuals. We provide fine-grained annotations on the IKEA objects and assembly manuals, including decomposed assembly parts, assembly plans, manual segmentation, and 2D-3D correspondence between 3D parts and visual manuals. We illustrate the broad application of our dataset on four tasks related to shape assembly: assembly plan generation, part segmentation, pose estimation and 3D part assembly.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

IKEA-Manual: Seeing Shape Assembly Step by Step

Wang¹,

Zhang²,

Mao³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…160 3D models with high visual complexity are selected from the Thingi10K [38] and Fusion 360 gallery [40] datasets to test the generalization performances. 14 complicated 3D models and the corresponding support structure volume yielded by four approaches are illustrated in Table II, whereas the rests are shown in the supplementary material.…”

Section: Part Orientation Performance When σ Equals To 90 •mentioning

confidence: 99%

Learn to Rotate: Part Orientation for Reducing Support Volume via Generalizable Reinforcement Learning

Shi

Qin

et al. 2023

IEEE Trans. Ind. Inf.

View full text Add to dashboard Cite

In design for additive manufacturing, an essential task is to determine the optimal build orientation of a part according to one or multiple factors. Heuristic search is used by the most part orientation methods to select the optimal orientation from a large solution space. Search algorithms occasionally converge towards the local optimum and waste considerable time on trial and error. The above issues could be addressed if there was an intelligent agent that knew the optimal search/rotation path for a given 3D model. A straightforward method to construct such an agent is reinforcement learning (RL). By adopting this idea, the time-consuming online searches in existing part orientation methods will be moved to the offline learning stage, potentially improving part orientation performance. This is a challenging research problem because the goal is to build an agent capable of rotating arbitrary 3D models, whereas RL agents frequently struggle to generalize in new scenarios. Therefore, this paper suggests a generalizable reinforcement learning (GRL) framework to train the agent, and a GPU-accelerated GRL benchmark to support the training, testing, and comparison of part orientation approaches. Experimental results demonstrate that the proposed part orientation method on average outperforms others in terms of effectiveness and efficiency. It is proved to have the potential to solve the local minima problems raised in the existing approaches, to swiftly discover the global (sub-)optimal solution (i.e. on average 2.62x to 229.00x faster than the random search algorithm), and to generalize beyond the environment in which it was trained.

show abstract