Who Is in the Crowd? Characterizing the Capabilities of Prize Competition Competitors

Szajnfarber, Zoe; Zhang, Lihui; Mukherjee, Shatavisa; Crusan, Jason; Hennig, Anthony; Vrolijk, Ademir

doi:10.1109/tem.2020.2991370

Cited by 14 publications

(10 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To that end, we identified nine comparable alternative robotic arm designs from a previously run robotics design competition. The selection included all designs from the challenge that were deems to be sufficiently detailed for DSM representation 49,50 . The nine designs were by independent solvers in the “crowd” to address an identical set of requirements.…”

Section: Methodsmentioning

confidence: 99%

“…The selection included all designs from the challenge that were deems to be sufficiently detailed for DSM representation. 49,50 The nine designs were by independent solvers in the "crowd" to address an identical set of requirements. This ensured both comparability in scope and purpose but also variance in design approach, as the crowd designers chose a wide variety of approaches and design strategies as seen in the CAD renderings in Figure 4.…”

Section: Stage 1: Set Of Design Alternatives (Basis Of Replication)mentioning

confidence: 99%

See 1 more Smart Citation

The impact of system representation choices on architecting insights

Hennig

Szajnfarber

2023

Systems Engineering

Self Cite

View full text Add to dashboard Cite

Systems engineers regularly rely on analysis of early design artifacts like system architecture representations to predict system performance, lifecycle costs, and development schedules, and to support design decision‐making. Recent recognition of challenges in this type of measurement has led to a heightened focus on developing better metrics. Less attention has been paid to the system representations upon which all subsequent analysis is performed. With this study, we demonstrate that choices about how to represent the system can explain variation in measurement, even holding metrics constant. This is important because most of these representation choices remain unarticulated in current practice. To do this, we conduct a controlled experiment where we experimentally manipulated the Design Structure Matrix (DSM) architecture representation of nine crowdsourced robotic arm designs and compared the value and relative ranks of their modularity and complexity. We found statistically significant changes in both value and rank, attributable to differences in choices in the system representation. The direction and magnitude of these changes also differed across modularity and complexity. In addition, some underlying designs seemed to be more robust to representation changes. This suggests an interaction between representation, design, and lifecycle properties. These results emphasize the importance of developing standard guidelines for how to represent system architectures and better documenting their use.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Stage 1: Set Of Design Alternatives (Basis Of Replication)mentioning

confidence: 99%

The impact of system representation choices on architecting insights

Hennig

Szajnfarber

2023

Systems Engineering

Self Cite

View full text Add to dashboard Cite

show abstract

“…This said, forward-looking, "big" ideas for solving current challenges in predictive analytics commonly focus on promoting bigger and more inclusive collaborative science. Prize competitions (e.g., Ansari X-Prize, NASA Solve) have been successful in bringing diverse individuals together to imagine and develop unique solutions, particularly those individuals who perceive the competition to be at the boundary of their expertise (Szajnfarber et al 2022). Additional ideas include hosting hackathons to develop, test, and identify the best predictor tools and software.…”

Section: Predictive Analytics Developmentmentioning

confidence: 99%

Current Challenges and Future of Agricultural Genomes to Phenomes in the U.S.

Tuggle¹,

Clarke²,

Murdoch³

et al. 2023

Preprint

View full text Add to dashboard Cite

Dramatic improvements in measuring genetic variation across agriculturally relevant populations (genomics) must be matched by improvements in identifying and measuring relevant trait variation in such populations across many environments (phenomics). Identifying the most critical opportunities and challenges in the area of genome to phenome (G2P) research is the focus of this paper. Previously (Tuggle et al. 2022), we laid out how Agricultural Genome to Phenome Initiative (AG2PI) will coordinate activities with US federal government agencies, including USDA, expand public–private partnerships, and engage with external stakeholders to achieve a shared vision of future the AG2PI. Acting on this latter step, AG2PI organized the “Thinking Big: Visualizing the Future of AG2PI” two-day workshop held September 9-10, 2022, in Ames, Iowa, co-hosted with the United State Department of Agriculture’s National Institute of Food and Agriculture (USDA NIFA). During the meeting, attendees were asked to use their experience and curiosity to review the current status of agricultural genome-to-phenome (AG2P) work and envision the future of the AG2P field. To tackle some of these big challenges, we designed this workshop to include diverse voices with the intention of hearing the different perspectives important to solving these challenges. The topic summaries composing this paper are distilled from two 1.5-hour small group discussions that included 6-8 participants, each representing a mix of crop researchers and livestock researchers, data scientists, social scientists, engineers, and other diverse stakeholders. Subsequent to the conference, additional writing, idea exchanges, and e-mails were shared among participants and the output of those conversations is summarized in this report. Challenges and solutions were identified across multiple topics at the workshop and are reported here. We end our discussion with a vision for the future of agricultural progress, identifying two areas of innovation needed: 1) innovate in genetic improvement methods development and evaluation and 2) innovate in agricultural research processes to solve societal problems. To address these needs, we then provide six specific goals that we recommend be implemented immediately in support of advancing AG2P research.

show abstract

“…Is golf representative of complex systems design?Although we made the case in Section 3.1 for why an abstract simulation inspired by the game of golf can provide insight into complex system architecture, we revisit that assertion here, analysing whether the features of how the simulation was formulated reflect key attributes of a more representative engineering system, and consequently whether specific results are likely to generalise (Szajnfarber et al 2020). We chose a robotic arm as the reference system for this comparison because it a reasonably complex interdisciplinary system and also one that we have explored extensively in the context of decomposing for open innovation through a recent open innovation field experiment (Szajnfarber et al 2020). In the golf model, there are four key features of system representation: problem architecture(s), solver types and associated capabilities, innovative solving and coordination costs.…”

Section: Problem Architecturementioning

confidence: 99%

“…In the context of a robotic arm, there are many more design parameters in play. Here, solving means designing a robotic arm that meets all the requirements and minimises mass (per the challenge rules; Szajnfarber et al 2020). Looking across the system, engineers can find many approaches to doing so.…”

Section: Problem Architecturementioning

confidence: 99%

Towards a solver-aware systems architecting framework: leveraging experts, specialists and the crowd to design innovative complex systems

2022

Self Cite

View full text Add to dashboard Cite

This article proposes the solver-aware system architecting framework for leveraging the combined strengths of experts, crowds and specialists to design innovative complex systems. Although system architecting theory has extensively explored the relationship between alternative architecture forms and performance under operational uncertainty, limited attention has been paid to differences due to who generates the solutions. The recent rise in alternative solving methods, from gig workers to crowdsourcing to novel contracting structures emphasises the need for deeper consideration of the link between architecting and solver-capability in the context of complex system innovation. We investigate these interactions through an abstract problem-solving simulation, representing alternative decompositions and solver archetypes of varying expertise, engaged through contractual structures that match their solving type. We find that the preferred architecture changes depending on which combinations of solvers are assigned. In addition, the best hybrid decomposition-solver combinations simultaneously improve performance and cost, while reducing expert reliance. To operationalise this new solver-aware framework, we induce two heuristics for decomposition-assignment pairs and demonstrate the scale of their value in the simulation. We also apply these two heuristics to reason about an example of a robotic manipulator design problem to demonstrate their relevance in realistic complex system settings.

show abstract

Who Is in the Crowd? Characterizing the Capabilities of Prize Competition Competitors

Cited by 14 publications

References 46 publications

The impact of system representation choices on architecting insights

The impact of system representation choices on architecting insights

Current Challenges and Future of Agricultural Genomes to Phenomes in the U.S.

Towards a solver-aware systems architecting framework: leveraging experts, specialists and the crowd to design innovative complex systems

Contact Info

Product

Resources

About