Benchmarking Spreadsheet Systems

Rahman, Sajjadur; Mack, Kelly; Bendre, Mangesh; Zhang, Ruilin; Karahalios, Karrie; Parameswaran, Aditya

doi:10.1145/3318464.3389782

Cited by 7 publications

(4 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Spreadsheets remain an important data management and processing tool for end-users (Rahman et al 2020) and estimates of the number of spreadsheet users are in the billions (Morgan Stanley). However, despite a large user base, spreadsheet environments still do not have access to nearly the same range of productivity tools available for general programming environments.…”

Section: Introductionmentioning

confidence: 99%

FLAME: A Small Language Model for Spreadsheet Formulas

Joshi,

Ebenezer,

Cambronero Sanchez

et al. 2024

AAAI

View full text Add to dashboard Cite

Spreadsheets are a vital tool for end-user data management. Using large language models for formula authoring assistance in these environments can be difficult, as these models are expensive to train and challenging to deploy due to their size (up to billions of parameters). We present FLAME, a transformer-based model trained exclusively on Excel formulas that leverages domain insights to achieve competitive performance while being substantially smaller (60M parameters) and training on two orders of magnitude less data. We curate a training dataset using sketch deduplication, introduce an Excel-specific formula tokenizer, and use domain-specific versions of masked span prediction and noisy auto-encoding as pre-training objectives. We evaluate FLAME on formula repair, formula completion, and similarity-based formula retrieval. FLAME can outperform much larger models, such as the Davinci (175B) and Cushman (12B) variants of Codex and CodeT5 (220M), in 10 of 14 evaluation settings for the repair and completion tasks. For formula retrieval, FLAME outperforms CodeT5, CodeBERT, and GraphCodeBERT.

show abstract

Section: Introductionmentioning

confidence: 99%

FLAME: A Small Language Model for Spreadsheet Formulas

Joshi,

Ebenezer,

Cambronero Sanchez

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…Spreadsheets are widely used for data exploration and analysis due to their intuitive layout [1,2]. While modern spreadsheet systems provide some analysis tools, such as PivotTables and aggregation formulas, they do not support more advanced tasks, such as iterative analyses and model building.…”

Section: Introductionmentioning

confidence: 99%

SheetReader: Efficient Specialized Spreadsheet Parsing

Gavriilidis

Henze

Zacharatou

et al. 2023

Information Systems

View full text Add to dashboard Cite

“…Many of these users are comfortable doing analyses using spreadsheets. However, due to their limited scalability and expressivity [5,13], existing spreadsheet applications aren't adequate for accessing and analyzing data residing CDWs.…”

Section: Introductionmentioning

confidence: 99%