Recently, pirates are infesting on the sea and they have been hijacking the several vessels for example Samho Dream and Samho Jewelry of Korea. One of the items to reduce the risk is to adopt the invader detection system. If the pirates break in to the ship, the detection system can monitor the pirates and then call the security alarm. The crew can gain time to hide to the safe room and the report can be automatically sent to the control room to cope with the situation. For the invader detection, an unmanned observation system was proposed using the image detection algorithm that extracts the invader image from the recording image. To detect the motion area, the difference value was calculated between the current image and the prior image of the invader, and the 'AND' operator was used in calculated image and edge line. The image noise was reduced based on the morphology operation and then the image was transformed into morphological information. Finally, a neural network model was applied to recognize the invader. In the experimental results, it was confirmed that the proposed approach can improve the performance of the recognition in the invader monitoring system.