A Simple Survey of Pre-trained Language Models

Zhu, Zhenyi

doi:10.20944/preprints202208.0238.v1

Cited by 4 publications

(3 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Currently, many pre‐training tasks can extract a fixed‐dimensional speaker embedding as a representation of speaker identity from the enormous amount of information in speech, such as the large language model [18]. In other words, the speaker encoder can generate a vector from the target audio that can meaningfully represent the speaker's identity.…”

Section: Introductionmentioning

confidence: 99%

Chinese personalised text‐to‐speech synthesis for robot human–machine interaction

Pang,

Teng,

et al. 2023

IET Cyber-Syst and Robotics

View full text Add to dashboard Cite

Speech interaction is an important means of robot interaction. With the rapid development of deep learning, end‐to‐end speech synthesis methods based on this technique have gradually become mainstream. Chinese deep learning‐based speech synthesis techniques suffer from problems such as unstable synthesised speech, poor naturalness and poor personalised speech synthesis, which do not satisfy some practical application scenarios. Hence, an F‐MelGAN model is adopted to improve the performance of Chinese speech synthesis. A post‐processing network is used to refine the Mel‐spectrum predicted by the decoder and alleviate the Mel‐spectrum distortion phenomenon. A phoneme‐level and sentence‐level combined module is proposed to model the personalised style of speakers. A combination of an acoustic conditioning network, speaker encoder network GCNet and feedback‐constrained training is proposed to solve the problem of poor personalised speech synthesis and achieve personalised speech customisation in Chinese. Experimental results show that the whole model can generate high‐quality speech with high speaker similarity for both speakers that appear in the training process and speakers that never appear in the training process.

show abstract

Section: Introductionmentioning

confidence: 99%

Chinese personalised text‐to‐speech synthesis for robot human–machine interaction

Pang,

Teng,

et al. 2023

IET Cyber-Syst and Robotics

View full text Add to dashboard Cite

show abstract

“…It has also been reported that ChatGPT could have passed a practice US bar exam with a headline score of 70% (35/50) 4 . If such generalpurpose models are further enhanced by incorporating legal domain knowledge [24], they will be even more proficient at legal tasks.…”

Section: Presentmentioning

confidence: 99%

Making a Computational Attorney

Dell¹,

Schilder²,

Conrad³

et al. 2023

Preprint

View full text Add to dashboard Cite

This "blue sky idea" paper outlines the opportunities and challenges in data mining and machine learning involving making a computational attorney -an intelligent software agent capable of helping human lawyers with a wide range of complex high-level legal tasks such as drafting legal briefs for the prosecution or defense in court. In particular, we discuss what a ChatGPT-like Large Legal Language Model (L 3 M) can and cannot do today, which will inspire researchers with promising short-term and long-term research objectives.

show abstract

“…This results in a more compact architecture, reducing computational requirements. • Compression (Zhu et al, 2023c) and Quantization (Tao et al, 2022): reduce the size of the model or reduce the precision of model parameters, using fewer bits to represent them. This reduces memory usage and computational complexity.…”

Section: Efficiency and Costmentioning

confidence: 99%

A Survey of Reasoning with Foundation Models

Sun,

Zheng,

Xie

et al. 2023

Preprint

View full text Add to dashboard Cite

https://github.com/reasoning-survey/Awesome-Reasoning-Foundation-Models Reasoning, a crucial ability for complex problem-solving, plays a pivotal role in various real-world settings such as negotiation, medical diagnosis, and criminal investigation. It serves as a fundamental methodology in the field of Artificial General Intelligence (AGI). With the ongoing development of foundation models, there is a growing interest in exploring their abilities in reasoning tasks. In this paper, we introduce seminal foundation models proposed or adaptable for reasoning, highlighting the latest advancements in various reasoning tasks, methods, and benchmarks. We then delve into the potential future directions behind the emergence of reasoning abilities within foundation models. We also discuss the relevance of multimodal learning, autonomous agents, and super alignment in the context of reasoning. By discussing these future research directions, we hope to inspire researchers in their exploration of this field, stimulate further advancements in reasoning with foundation models, and contribute to the development of AGI.

show abstract

A Simple Survey of Pre-trained Language Models

Cited by 4 publications

References 49 publications

Chinese personalised text‐to‐speech synthesis for robot human–machine interaction

Chinese personalised text‐to‐speech synthesis for robot human–machine interaction

Making a Computational Attorney

A Survey of Reasoning with Foundation Models

Contact Info

Product

Resources

About