One of the most exciting new capabilities in Smart Manufacturing (SM) and Cyber-Physical Production Systems (CPPS) is the provisioning of manufacturing services as unbundled "apps or services", which could be significantly more flexible and less expensive to use than the current generation of monolithic manufacturing applications. However, bundling and integrating heterogeneous services in the form of such apps or composite services is not a trivial job. There is a need for service vendors, cloud vendors, manufacturers, and other stakeholders to work collaboratively to simplify the effort to "mix-and-match" and compose the apps or services. In this regard, a workshop was organized by the National Institute of Standards and Technology (NIST) and the Open Applications Group Inc. (OAGi), with the purpose to identifythrough parallel sessionstechnology and standard needs for improving interoperability and composability between services. The workshop was organized into five working session. This paper documents evidences gathered during the "Smart Manufacturing Systems Characterization" (SMSC) session, which aims at establishing a roadmap for a unified framework for assessing a manufacturer's capability, maturity and readiness level to implement Smart Manufacturing. To that end, the technology maturity, information connectivity maturity, process maturity, organizational maturity, and personnel capability and maturity, have been identified as critical aspects for Smart Manufacturing adoptions. The workshop session culminated at providing a coherent model and method for assisting manufacturing companies in their journey to smart manufacturing realizations. This paper shows three different maturity models and tools that, thanks to their complementarity, enable one to reflect on the different perspectives required by SMSC. These models and tools are usable together for assessing a manufacturing company's ability to initiate the digital transformation of its processes towards Smart Manufacturing. Therefore, based on their comparison, the ultimate purpose of the research is to come up with a set of coherent guidelines for assessing a manufacturing system and its management practices for identifying improvement opportunities and for recommending SM technologies and standards for adoption by manufacturers.