Work

Degelijke datasets en methoden voor webbeveiligingsonderzoek

Year: 2023
Type: dissertation
Abstract: Sound research practices are the foundation of valid, reliable, and trustworthy research results. In the context of web security research, the importance of measurements that yield the empirical real-world data for analyzing and improving security on the web emphasizes the need to use sound data sets and methods that allow these measurements to be accurate, comprehensive, representative, and transparent. In this PhD project, we discuss three case studies in web security that have affinity with the discipline of meta-research, which critically evaluates these research practices and proposes new methods to improve and refine the way in which research is conducted. The three presented case studies contribute to critically analyzing current web security data sets or systems that are commonly held to be reliable, while we also propose improved methods as well as discuss data set considerations. Our first case study covers our work analyzing and improving rankings of the most popular websites or domain names on the Internet. These rankings form an important data source for many web security, privacy, and Internet measurement studies. We show how previously commonly used rankings hold potentially undesirable properties that endanger the soundness, validity, and reproducibility of research. We also propose the novel Tranco ranking that improves upon these properties. The remaining two case studies cover large-scale automated decision-making systems. These are seen as essential tools for processing security-related decisions at scale, and are commonly deployed to handle critical security tasks. However, even error rates that are low on a relative scale can translate into a high error count in absolute terms, which can cause significant harm. In our second case study, we develop a hybrid approach to resolve collisions between benign and malicious domains generated and used by the Avalanche botnet. As erroneous law enforcement decisions would result in unjustified website takedowns and the risk of the botnet reemerging respectively, we involve a human investigator for those domains where an automated model is least certain. This approach reduces the errors that result from blind trust in the automated decision-making system. In our third case study, we audit Facebook's enforcement of its self-developed policy on political ads. We find that even simple rules for detecting violating ads are not implemented, while many benign ads are falsely taken down, suggesting Facebook's enforcement is imprecise. Our audit reveals the limitations of large-scale automated decision-making systems and questions their appropriateness for systems with important societal impact.
Source: Lirias
Language: English
Open Access status: green
OpenAlex ID: W7046294980
Cites: 293
Cited by:
Related to: 10