Quantifying the Predictive Abilities of Speculative Fiction: A Feasibility Study

Abstract

In this article, we explore the capability of speculative fiction to predict future realized technologies. We review a large set of speculative technologies introduced in speculative fiction to determine if the technologies were subsequently realized. Additionally, we explore the time between the speculated introduction and actual realization. Our dataset for analysis is built from the ‘Technovelgy’ database of speculative technologies. A realization assessment methodology is created that includes detailed rubrics to rate and quantify the predictability or realizability of speculative technologies. Three independent raters perform realization assessments for each entry. An inter-rater agreement analysis is carried out to validate the rating method. Based on the dataset of 3095 speculated technologies, 45% are labeled as ‘realized’ by at least one rater. A moderate overall agreement with a Fleiss’ Kappa of 0.57 is reached by all raters. The average time to realization of realized technologies is approximately 45 years with a standard deviation of approximately 34 years. We observe patterns in the realization of speculative technologies and analyze the underlying reasons preventing the technologies from realization. We conclude that speculative fiction predicts future technologies to such a degree that the introduction of speculative technology can be used as an input to designer decision-making.

How good is ChatGPT? An exploratory study on ChatGPT's performance in engineering design tasks and subjective decision-making

“…How good is ChatGPT at knowledge discovery of a technical nature from non-technical content such as mining speculative technologies from a given text as compared to humans? To answer these questions, we designed two experiments based on a previous study completed in 2021 (Xu et al 2021). Researchers have explored the potential of leveraging science fiction narratives to enhance designers' ideation processes (Sterling 2005;Bleecker 2009;Dunne and Raby 2013;Callaghan 2015;Kotecha et al 2021).…”

“…We then employed five human raters to determine if the technologies were realized so that each technology was reviewed by three different human raters. Further details about this study can be found in a previous paper (Xu et al 2021).…”

“…In the current study, we asked ChatGPT to complete the same task as what our human team members performed in (Xu et al 2021) and compared the outcomes against each other. The study was conducted under the assumption that ChatGPT had similar accessible resources as our human raters in early 2021, given that current GPT models were all trained on data up to September 2021 (OpenAI 2022).…”

“…Task: The task was to determine whether the given technology was realized or not and provide a confidence level when making the decision. Details of the assessment can be found in our previous work (Xu et al 2021). ChatGPT was provided with the same instruction for the technology realization task and the same information from the Technovelgy datasets as human raters.…”

“…The manual process of technology realization evaluation was labour intensive. To assess ChatGPT's performance relative to humans, we assigned ChatGPT the same task that our human team members completed in that science-fiction related study (Xu et al 2021). Through this exploratory study, we aim to answer the following research questions:…”