A deterministic design philosophy for systems, structures, and components of nuclear facilities, accepted by all stakeholders, has been adopted to address safety concerns. However, over-conservatism makes the design of nuclear facilities, especially research reactors, expensive; therefore, the industry needs to reevaluate its level of over-conservatism to obtain an optimized design. This paper proposes a probabilistic design and optimization approach for instrumentation & control (I&C) systems of research reactors based on sensitivity, availability criteria, and cost. Compared to commercial nuclear power plants, research reactors are sensitive to the cost competitiveness of I&C systems, so optimization of I&C system design is needed. As a case study, we formulated four architecture configurations for a reactor protection system and performed the reliability feature analysis, assessed cost, and computed reliability index. In addition to International Electrotechnical Commission (IEC) and American Society of Mechanical Engineers (ASME) criteria, we introduce the reliability index, a novel parameter describing the correlation between cost and reliability used to determine the point at which an I&C architecture has been optimized. To observe the significance of this study, we compared our results with a benchmark architecture. Our novel probabilistic design and optimization approach produced two optimized I&C architectures with design cost reductions of 30-40% compared to the benchmark architecture, while maintaining safety and availability.Index Terms-I&C Architecture, probabilistic design approach, reliability index, research reactors, sensitivity analysis, sensitivity.