Distribution
PDF

Content removal bias in web scraped data: A solution applied to real estate ads

Loading...
Thumbnail Image

Format

PDF

Authors

Journal ISSN

Publisher

Dataset

Dataset
Content removal bias in web scraped data: A solution applied to real estate ads
(De Gruyter, 2022-12-31) Marconi, Gabriele
propose a solution to content removal bias in statistics from web scraped data. Content removal bias occurs when data is removed from the web before a scraper is able to collect it. The solution I propose is based on inverse probability weights, derived from the parameters of a survival function with complex forms of data censoring. I apply this solution to the calculation of the proportion of newly built dwellings with web scraped data on Luxembourg, and I run a counterfactual experiment and a Montecarlo simulation to confirm the findings. The results show that the extent of content removal bias is relatively small if the scraping occurs frequently compared with the online permanence of the data; and that it grows larger with less frequent scraping.

Description

Keywords

Files

Original bundle
Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
10.1515_openec-2022-0119.pdf
Size:
495.65 KB
Format:
Adobe Portable Document Format
Description:
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1.71 KB
Format:
Item-specific license agreed upon to submission
Description: