SESR: Self-Ensembling Sim-to-Real Instance Segmentation for Auto-Store Bin Picking

On Improving Boundary Quality of Instance Segmentation in Cluttered and Chaotic Scenarios

“…The second is on adapting to unseen objects with least data requirement. Some methods use sim-to-real technique [21], [22], [4], [23], [6], [5], [24] while others [9], [25] harness the strength of knowledge distillation, trained in a weaklysupervised manner. Meanwhile, various datasets are released for facilitating studies on instance segmentation for autostore.…”

“…We implemented our framework with PyTorch [59] and trained it for 50 epochs with the AdamW [60] optimizer using an initial learning rate of 10 −4 . For AutoStore dataset [9] with 1,000 images, the training takes about 17h and the inference runs at around 11.0 Fps on a single GeForce GTX 1080 Ti GPU. We use Swin-T (Tiny) as our base architecture of Swin Transformer and initialize the length of the object queries K as 200, i.e., the maximum predicted instance number is 200.…”

“…I. We compare our method with the general Instance Segmentation (IS) network Mask R-CNN [1], the Unseen Object Instance Segmentation (UOIS) network UOIS-2D [6], and the recent Self-Ensembling (SEIS) network SESR [9]. Note that SESR has full access to the testing images but without annotations.…”

“…To improve the segmentation accuracy on unseen SKUs, the Self-Ensembling Instance Segmentation approach (SEIS, Fig. 1 top-right) [8], [9] is proposed. By acquiring quite a number of images prepared for each unseen SKU, teacherstudent pseudo labeling can be exploited to enable the pretrained model to fit the unseen objects.…”

“…1. Results on unseen SKUs produced by (i) the general Instance Segmentation (IS) method Mask R-CNN [1], (ii) the Unseen Object Instance Segmentation (UOIS) method UOIS-2D [6], (iii) the Self-Ensembling (SEIS) method SESR [9], and (iv) SKU-Patch (ours). Clearly, SKU-Patch can generate more precise masks by leveraging only a few easily-obtainable SKU patches.…”