The recent explosion of advancements in natural language processing (NLP) are encouraging in the industrial sector for leveraging the volumes of unstructured, technical data that currently sit unused. However, results from direct application of many NLP pipelines to technical text often fail to address the business needs of industrial companies. One requirement for satisfactory performance is an effective representation of the unstructured text in a form which containstheinformationrequiredforanapplicationtask. We know of no standard methodology for evaluating word representations for technical text tailored to industry needs. In this paper, we propose guidance and methods for evaluating the performance of word representations for industrial use-cases.