Evaluating Web Scraping Performance Using XPath, CSS Selector, Regular Expression, and HTML DOM With Multiprocessing Technical Applications

Darmawan, Irfan; Maulana, M. R.; Gunawan, Rohmat; Widiyasono, Nur

doi:10.30630/joiv.6.4.1525

Cited by 3 publications

(2 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…As regex foram utilizadas para auxiliar na seleção de elementos nas páginas, uma vez que estudos [4,11] demonstram o menor consumo de memória com uso dessa técnica dentre as mais populares de web scraping: HTML DOM, XPath, seletores CSS e regex.…”

Section: Percurso Metodológicounclassified

PySol: Uma Proposta Python para Automação de Busca na SBC OpenLib

Chagas Souza,

Barros de Sales,

G. Q. Palmeira

2024

Anais Do XV Computer on the Beach - COTB'24

View full text Add to dashboard Cite

RESUMOConducting a systematic review is complex, but it can be simplifiedusing computational resources. Utilizing multiple research sourcesis essential to cover most studies relevant to the investigated topic.In this context, SBC OpenLib (SOL), the open digital library ofthe Brazilian Society of Computing (SBC), is an important bibliographicsource for systematic reviews in Computing-related fields,offering access to all academic and scientific content produced bythe SBC. However, SOL’s limitation in its automatic search featureis the lack of flexibility in exporting results, a crucial criterionfor a database to be included in a systematic review’s search strategy.Addressing this issue, this paper introduces pySol, a Pythonbasedtool designed to automate searches and export results fromthe SOL database. PySol was developed using web scraping techniquesto extract automatic search results from the SOL database. Itsdevelopment followed Test-Driven Development (TDD) principles,resulting in over 280 unit tests and a code coverage of 93%. Theseindicators highlight the tool’s reliability for systematic searches.The tool intends to support the study identification stage in systematicreviews, significantly reducing the time and effort neededto search in the SOL database. The automation of this process notonly eases the execution of systematic searches in this databasebut also enables the export of results in BibTeX format, facilitatingintegration with reference managers.

show abstract

Section: Percurso Metodológicounclassified

PySol: Uma Proposta Python para Automação de Busca na SBC OpenLib

Chagas Souza,

Barros de Sales,

G. Q. Palmeira

2024

Anais Do XV Computer on the Beach - COTB'24

View full text Add to dashboard Cite

show abstract

“…This approach would ensure accessibility across various devices, including mobile platforms. The suitability of a web application is further reinforced by the effectiveness of webpage data representation (DOM, HTML, and CSS) in crafting responsive and interactive surveys that dynamically adapt their content to present educational information upon completion [20]. Figure 3.…”

Section: React App Implementationmentioning

confidence: 99%

A Study on Exploring the Level of Awareness of Privacy Concerns and Risks

Nguyen,

Yeates,

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

With the recent increase in phishing attacks and other kinds of malicious activities, increasing the awareness of security and privacy awareness is arguably one of the best proven ways of preventing these kinds of threats. The main challenge in security and privacy awareness is the end user’s awareness of aspects of privacy and security they give up when using the Internet. Thus, this study focuses on identifying and discussing the growing threats of Internet usage and the lack of privacy and security knowledge of the everyday person. This study presents the results of a survey conducted to determine discrepancies between what rights users think they sign away when they agree to terms of service versus what rights they actually give away. It is hypothesized that people are genuinely unaware of what rights they are giving up, especially since they are signing the terms of service without reading the fine print. In this study, the terms of service were presented to respondents, and they answered questions on what they thought they were giving up, but once they answered the questions, they were explicitly told whether they truly knew what rights they signed away. The experimental results of this study examine how much knowledge the everyday person lacks with respect to the privacy policies they sign. All of this is ultimately used to examine possible flaws in the system. The experimental results illustrate the results of the survey. Based on the total surveys completed, the average score was 62%. This means that out of 10 clauses described in a given terms of service document or privacy policy, people are truly unaware of at least 4 of these statements.

show abstract