In online transactions of textile products, fabric hand was thought to be inaccessible to consumers. Recently, much effort has been made to study the feasibility of providing consumers with a real sense of fabric through a virtual experience. The current paper proposes to extract fabric hand information from the perspective of visual perception. Two sensory experiments are conducted according to the standardized sensory evaluation procedures on a set of representative textile fabrics by two trained panels. The first experiment is aimed to measure how much fabric hand can be perceived through fabrics’ visual displays. On the basis of the positive results obtained, the second experiment is carried out to further investigate the interactive mechanism between samples’ visual features and their tactile properties. A novel algorithm based on rough set theory and fuzzy set theory is proposed in order to quantitatively measure relations between different sensory information.