For naval operations in a coastal environment, detection of boats is not sufficient. When doing surveillance near a supposedly friendly coast, or self protection in a harbor, it is important to find the one object that means harm, among many others that do not. For this, it is necessary to obtain information on the many observed targets, which in this scenario are typically small vessels. Determining the exact type of ship is not enough to declare it a threat. However, in the whole process from (multi-sensor) detection to the decision to act, classification of a ship into a more general class is already of great help, when this information is combined with other data to assist an operator.We investigated several aspects of the use of electro-optical systems. As for classification, this paper concentrates on discriminating classes of small vessels with different electro-optical systems (visual and infrared) as part of the larger process involving an operator. It addresses both selection of features (based on shape and texture) and ways of using these in a system to assess threats. Results are presented on data recorded in coastal and harbor environments for several small targets.