Putting Humans in the Natural Language Processing Loop: A Survey

Wang, Zijie; Choi, Deokjai; Xu, Shenyu; Yang, Diyi

doi:10.48550/arxiv.2103.04044

Cited by 9 publications

(5 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(b) If AI, then Natural Language Processing (NLP) Natural Language Processing (NLP), a discipline to explore how humans and computers can communicate using human language instead of computer codes, is commonly agreed to be the candidate for HCI design (Alkatheiri, 2022;Heuer and Buschek, 2021;Wang et al, 2021). NLP exploration in architecture can be traced back to the 1960s.…”

Section: The Potential Of Large Language Models (Llms)mentioning

confidence: 99%

Exploring Large Language Model as a Design Partner through Verbal and Non-verbal Conversation in Architectural Design Process

Cheung,

Dall'Asta,

Di Marco

2024

Blucher Design Proceedings

View full text Add to dashboard Cite

Section: The Potential Of Large Language Models (Llms)mentioning

confidence: 99%

Exploring Large Language Model as a Design Partner through Verbal and Non-verbal Conversation in Architectural Design Process

Cheung,

Dall'Asta,

Di Marco

2024

Blucher Design Proceedings

View full text Add to dashboard Cite

“…7.3.2 Human-in-the-loop and RLHF. [321] Human-in-the-loop is meant to improve the outcome of an event or process via user input in the system loop. Humans can intervene at many steps in a QA system from task definition or data creation, to final answer assessment.…”

Section: Improvement Loop and Knowledge Capitalizationmentioning

confidence: 99%

Complex QA and language models hybrid architectures, Survey

Daull¹,

Bellot²,

Bruno³

et al. 2023

Preprint

View full text Add to dashboard Cite

This paper provides a survey of the state of the art of hybrid language models architectures and strategies for "complex" question-answering (QA, CQA, CPS). Very large language models are good at leveraging public data on standard problems but once you want to tackle more specific complex questions or problems you may need specific architecture, knowledge, skills, tasks, methods, sensitive data, performance, human approval and versatile feedback... This survey extends findings from the robust community edited research papers BIG, BLOOM and HELM which open source, benchmark and analyze limits and challenges of large language models in terms of tasks complexity and strict evaluation on accuracy (e.g. fairness, robustness, toxicity, ...). It identifies the key elements used with Large Language Models (LLM) to solve complex questions or problems. Recent projects like ChatGPT and GALACTICA have allowed non-specialists to grasp the great potential as well as the equally strong limitations of language models in complex QA. Hybridizing these models with different components could allow to overcome these different limits and go much further. We discuss some challenges associated with complex QA, including domain adaptation, decomposition and efficient multi-step QA, long form QA, non-factoid QA, safety and multi-sensitivity data protection, multimodal search, hallucinations, QA explainability and truthfulness, time dimension. Therefore we review current solutions and promising strategies, using elements such as hybrid LLM architectures, human-in-the-loop reinforcement learning, prompting adaptation, neuro-symbolic and structured knowledge grounding, program synthesis, and others. We analyze existing solutions and provide an overview of the current research and trends in the area of complex QA.

show abstract

“…In recent years, large language models have improved significantly in various NLP areas, especially in generative tasks. A lot of new concepts were introduced, starting from attention mechanism [1], transformers [25] to multitask, learning from instructions [28] and human feedback [29]. The last becomes extremely popular in the generative context including machine translation.…”

Section: Related Workmentioning

confidence: 99%

Translate your gibberish: black-box adversarial attack on machine translation systems

Chertkov¹,

Tsymboi²,

Pautov³

et al. 2023

Preprint

View full text Add to dashboard Cite

Neural networks are deployed widely in natural language processing tasks on the industrial scale, and perhaps the most often they are used as compounds of automatic machine translation systems. In this work, we present a simple approach to fool state-of-the-art machine translation tools in the task of translation from Russian to English and vice versa. Using a novel black-box gradient-free tensor-based optimizer, we show that many online translation tools, such as Google, DeepL, and Yandex, may both produce wrong or offensive translations for nonsensical adversarial input queries and refuse to translate seemingly benign input phrases. This vulnerability may interfere with understanding a new language and simply worsen the user's experience while using machine translation systems, and, hence, additional improvements of these tools are required to establish better translation.

show abstract

Putting Humans in the Natural Language Processing Loop: A Survey

Cited by 9 publications

References 23 publications

Exploring Large Language Model as a Design Partner through Verbal and Non-verbal Conversation in Architectural Design Process

Exploring Large Language Model as a Design Partner through Verbal and Non-verbal Conversation in Architectural Design Process

Complex QA and language models hybrid architectures, Survey

Translate your gibberish: black-box adversarial attack on machine translation systems

Contact Info

Product

Resources

About