Year: 2023
Type: article
Abstract: We investigate the effectiveness of five weighting and matching techniques, including propensity score matching (PSM), in improving covariate balance and reducing bias when estimating treatment effects in finite-sample situations through Monte Carlo simulations. King and Nielsen (2019. “Why Propensity Scores Should Not Be Used for Matching.” Political Analysis 27 (4): 435–54) argue that pruning observations based on PSM with 1-to-1 greedy matching can worsen rather than improve covariate balance and can increase the bias in the estimates of treatment effects. In our simulations, we observed this phenomenon not only in PSM with 1-to-1 greedy matching but also in other covariate balancing techniques that King and Nielsen (2019. “Why Propensity Scores Should Not Be Used for Matching.” Political Analysis 27 (4): 435–54) recommend as better matching methods, i.e. Mahalanobis distance matching (MDM) and coarsened exact matching (CEM). Regardless of the weighting/matching techniques and the data generation processes in this study, our findings indicate that matching and weighting under extreme caliper or cut-point settings does not improve covariate balance. In addition, once a substantially improved covariate balance is achieved in a given sample, the estimated bias tends to worsen slightly as the covariate balance continues to improve. Moreover, our simulation results suggest that OLS with proper covariates reduces selection bias as well as other weighting and matching methods. The results suggest that when analyzing observational data, it is important to avoid looking for a one-size-fits-all estimator and to identify the appropriate nonexperimental estimator carefully for the sample by thoroughly investigating the available data’s characteristics.
Source: Statistics Politics and Policy
Author: Hideki Fukui
Institution: Ehime University
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $2,163
Awards: A study of the effectiveness of non-experimental estimators of policy effects and their application to Evidence Based Policy Making (EBPM), Assessment of Optimal Configurations of Air Networks and Hub Locations in Asia from an Environmental Perspective, What encourages volunteer participation? Causal inference based on field experiments
OpenAlex ID: W4385234307