Quantum Attention Based Language Model for Answer Selection

Zhao, Qin; Hou, Chenguang; Xu, Ruifeng

doi:10.1007/978-3-030-96033-9_4

Cited by 2 publications

(4 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then they implemented Grover's search algorithm to find correct answer. Zhao et al (2022) [34] found a balance between a model's performance and interpretability while proposed a quantum attention-based language model. Density matrix was used in quantum attention mechanism.…”

Section: Question Classificationmentioning

confidence: 99%

Implications of Deep Circuits in Improving Quality of Quantum Question Answering

Katyayan¹,

Joshi²

2023

Studies in Computational Intelligence

View full text Add to dashboard Cite

Question Answering (QA) has proved to be an arduous challenge in the area of natural language processing (NLP) and artificial intelligence (AI). Many attempts have been made to develop complete solutions for QA as well as improving significant submodules of the QA systems to improve overall performance through the course of time. Questions are the most important piece of QA, because knowing the question is equivalent to knowing what counts as an answer [1]. In this work, we have attempted to understand questions in a better way by using Quantum Machine Learning (QML). The properties of Quantum Computing (QC) have enabled classically intractable data processing. So, in this paper, we have performed question classification on questions from two classes of SelQA (Selection-based Question Answering) dataset using quantum-based classifier algorithms-quantum support vector machine (QSVM) and variational quantum classifier (VQC) from Qiskit (Quantum Information Science toolKIT) for Python. We perform classification with both classifiers in almost similar environments and study the effects of circuit depths while comparing the results of both classifiers. We also use these classification results with our own rule-based QA system and observe significant performance improvement. Hence, this experiment has helped in improving the quality of QA in general.

show abstract

Section: Question Classificationmentioning

confidence: 99%

Implications of Deep Circuits in Improving Quality of Quantum Question Answering

Katyayan¹,

Joshi²

2023

Studies in Computational Intelligence

View full text Add to dashboard Cite

show abstract

“…Ref. [26] understood the quantum attention mechanism as a density matrix by which more powerful sentence representations can be constructed. Unfortunately, the above two approaches only involve certain physical concepts in quantum mechanics without providing specific quantum circuits.…”

Section: Introductionmentioning

confidence: 99%

“…In contrast to Ref. [25,26,40], QSAN is potentially fully deployed and realized on quantum devices with fewer measurements and a beneficial byproduct called QBSASM. Whereas, the essential motivation for proposing this QSAN is to explore whether young quantum computers can have quantum characteristic attention and can depict the distribution of outputs in a quantum language, not to replace SAM or to beat all the schemes in the Ref.…”

Section: Introductionmentioning

confidence: 99%

“…Whereas, the essential motivation for proposing this QSAN is to explore whether young quantum computers can have quantum characteristic attention and can depict the distribution of outputs in a quantum language, not to replace SAM or to beat all the schemes in the Ref. [25,26,40]. Quantum characteristic here could be understood as probability, linearity and reversibility, while the quantum language refers to specialized terms in quantum mechanics, such as density matrix, Hamiltonian operators, etc.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

QSAN: A Near-term Achievable Quantum Self-Attention Network

Zhao¹,

Shi²,

Zhang³

2022

Preprint

View full text Add to dashboard Cite

A Quantum Self-Attention Network (QSAN) that can be achieved on near-term quantum devices is investigated. First, the theoretical basis of QSAN, a linearized and reversible Quantum Self-Attention Mechanism (QSAM) including Quantum Logic Similarity (QLS) and Quantum Bit Self-Attention Score Matrix (QBSASM), is explored to solve the storage problem of Self-Attention Mechanism (SAM) due to quadratic complexity. More importantly, QLS uses logical operations instead of inner product operations to enable QSAN to be fully deployed on quantum computers and meanwhile saves quantum bits by avoiding numerical operations, and QBSASM is a by-product generated with the evolution of QSAN, reflecting the output attention distribution in the form of a density matrix. Then, the framework and the quantum circuit of QSAN are designed with 9 execution steps and 5 special functional sub-modules, which can acquire QBSASM effectively in the intermediate process, as well as compressing the number of measurements. In addition, a quantum coordinate prototype is proposed to describe the mathematical connection between the control and output bits in order to realize programming and model optimization conveniently. Finally, a miniaturized experiment is implemented and it demonstrates that QSAN can be trained faster in the presence of quantum natural gradient descent method, as well as produce quantum characteristic attention distribution QBSASM. QSAN has great potential to be embedded in classical or quantum machine learning frameworks to lay the foundation for quantum enhanced Natural Language Processing (NLP).

show abstract

Quantum Attention Based Language Model for Answer Selection

Cited by 2 publications

References 27 publications

Implications of Deep Circuits in Improving Quality of Quantum Question Answering

Implications of Deep Circuits in Improving Quality of Quantum Question Answering

QSAN: A Near-term Achievable Quantum Self-Attention Network

Contact Info

Product

Resources

About