At the forefront of social science research, novel techniques are being developed that allow researchers to make robust inferences from complex data. These tools and methods rest on proof of their performance, which in turn relies on using the right kind of data to test them. These tests are hard to conduct well because real social science data is so complex: parametric tests may not comport well with actual social science applications. Conversely, benchmarking on well-known studies leaves researchers unable to determine true performance since the population parameters are unknown. In this paper we introduce a new solution using synthetic data: a strategy in which the underlying relationships between variables in real-world data are learned through deep, generative neural networks, and from which an arbitrary number of entirely new but realistic observations can be generated. We demonstrate our method by synthesising realistic looking, but entirely novel, survey data. We discuss how this synthetic data can be used to benchmark statistical designs and methods, and contribute new open-source software for researchers to use.

Despite its consequences, voters keep electing corrupt politicians. One common explanation is that voters simply lack information on whether candidates are corrupt, yet studies that deliberately provide such information find electoral accountability is weak. Can we root out corrupt politicians? We take a novel approach: first, we employ machine learning to identify widely available political/personal characteristics that are predictive of corrupt practices in Colombia. We then design an experiment that varies the provision of these predictors to study if voters discriminate corrupt from non-corrupt politicians. Results indicate that the presence of candidate images, the presence of a large donor, and information on candidates' political experience decreases the likelihood of choosing a corrupt politician. Moreover, voters behave differently according to their understanding of what corruption is and their attitudes towards it. Compared to established findings, using a novel approach, we show it is possible for voters to root-out corrupt politicians.

This pre-registration report set out an analysis plan for the public goods game experiment embedded within the second wave of COVID-19 vaccine preference and Opinion Survey (or CANDOUR II study). The study involves a sample of 1200 residents from each of 18 countries interviewed via an anonymous online survey.

Forced-choice conjoint designs have become a staple method in the experimentalist's toolkit. Conjoint experiments are useful for estimating multiple marginal effects simultaneously, but the forced-choice outcome is neither always consistent with the types of choices individuals make in real political contexts, nor is it statistically efficient. In this paper, we show how political scientists can, instead, ask subjects to \textit{rank} conjoint profiles. We demonstrate that this ranked-choice conjoint is statistically more efficient and, with minimal new assumptions, does not affect the broader validity of the conjoint design. We validate the utility of this design in a survey experiment, randomising subjects into forced-choice and ranked-choice versions of two conjoint experiments: one choosing between candidates, and another choosing between budget proposals. Alongside this paper, we provide \texttt{R} implementations of the methods needed to handle ranked-choice conjoints.

Early in the pandemic, individuals in numerous countries experienced quite different rates of COVID-19 infections and deaths dependent on where they lived. This within-country variation offers an opportunity to study how the intensity of a catastrophic shock to systems affects individuals’ economic preferences – a topic without consensus in the literature. In April 2020, we conducted an online survey with approximately 1500 subjects in China, 800 in Chile, and 800 in Italy. Our sampling strategy deliberately sampled subjects with exposure to different levels of local COVID-19 infections. We find that respondents condition their behavior and economic preferences on this intensity – levels of COVID-19 preventive behavior are correlated with the intensity of community infections; exposure to intense infection rates correlates, positively, with risk aversion and patience; and, negatively, with other-regarding preferences. Using machine-learning to estimate individual-level effects, we find notable effect heterogeneity with respect to education levels. Finally, using multilevel regression and poststratification (MRP) we demonstrate province-level estimates of economic preferences for 107 Italian provinces.

The recent Covid-19 pandemic offers a rare opportunity to understand how citizens attribute responsibility for governments’ responses to unanticipated negative--and in this case, systemic--exogenous shocks. Classical accounts of responsibility are complicated when crises are pervasive, involve multiple valence dimensions, and where individuals can make relative assessments of performance. We fielded a conjoint experiment in 16 countries with 22,147 respondents. In this experiment, subjects made re-election decisions regarding 178,184 randomly assigned incumbent profiles. We find that incumbents' performance along both health and economic dimensions drive these hypothetical reelection decisions. Using machine learning techniques, we find only muted heterogeneity in the magnitude and distribution of these treatment effects. This result suggests that these widely reported performance signals have consistent political effects across countries. In a complementary analysis, we also find that subjects' intentions to vote for incumbent governments are positively correlated with subjective and relative evaluations of the government's pandemic performance, along both health and economic dimensions. These results provide consistent evidence that evaluations of pandemic performance matter politically.

Active learning can improve the efficiency of training prediction models by identifying the most informative new labels to acquire. However, non-response to label requests can impact active learning's effectiveness in real-world contexts. We conceptualise this degradation by considering the type of non-response present in the data, demonstrating that biased non-response is particularly detrimental to model performance. We argue that biased non-response is likely in contexts where the labelling process, by nature, relies on user interactions. To mitigate the impact of biased non-response, we propose a cost-based correction to the sampling strategy--the Upper Confidence Bound of the Expected Utility (UCB-EU)--that can, plausibly, be applied to any active learning algorithm. Through experiments, we demonstrate that our method successfully reduces the harm from labelling non-response in many settings. However, we also characterise settings where the non-response bias in the annotations remains detrimental under UCB-EU for specific sampling methods and data generating processes. Finally, we evaluate our method on a real-world dataset from an e-commerce platform. We show that UCB-EU yields substantial performance improvements to conversion models that are trained on clicked impressions. Most generally, this research serves to both better conceptualise the interplay between types of non-response and model improvements via active learning, and to provide a practical, easy-to-implement correction that mitigates model degradation.

Conjoint experiments are fast becoming one of the dominant experimental methods within the social sciences. Despite several scholars advancing novel ways to model heterogeneity within this type of design, the relationship between these new quantities and the conjoint design is underdeveloped. In this note, we clarify how conjoint heterogeneity can be construed as a set of nested, causal parameters that correspond to the levels of the conjoint design. We then use this framework to propose a new estimation strategy that allows researchers to evaluate treatment effect heterogeneity and which exhibits good statistical properties. Replicating two conjoint experiments, we first demonstrate our theoretical argument, and then show how this method helps uncover interesting heterogeneity. To accompany this paper, we provide new a R package, cjbart, that allows researchers to model heterogeneity in their experimental conjoint data.

This paper introduces software packages for efficiently imputing missing data using deep learning methods in Python (MIDASpy) and R (rMIDAS). The packages implement a recently developed approach to multiple imputation known as MIDAS, which involves introducing additional missing values into the dataset, attempting to reconstruct these values with a type of unsupervised neural network known as a denoising autoencoder, and using the resulting model to draw imputations of originally missing data. These steps are executed by a fast and flexible algorithm that expands both the quantity and the range of data that can be analyzed with multiple imputation. To help users optimize the algorithm for their particular application, MIDASpy and rMIDAS offer a host of user-friendly tools for calibrating and validating the imputation model. We provide a detailed guide to these functionalities and demonstrate their usage on a large real dataset.

In recent American elections political candidates have actively emphasized features of their fundraising profiles when campaigning. Yet, surprisingly, we know comparatively little about how financial information affects vote choice specifically, whether effects differ across types of election, and how robust any effects are to other relevant political signals. Using a series of conjoint experiment designs, I compare the effects of campaigns’ financial profiles on vote choice across direct democratic and representative elections, randomizing subjects’ exposure to additional political cues. I find that while the financial profile of candidates can affect vote choice, these effects are drowned out by non-financial signals. In ballot initiative races, the explicit policy focus of the election appears to swamp any effect of financial information. This paper is the first to explore the comparative effects of financial disclosure across election type, contributing to our understanding of how different heuristics interact across electoral contexts.

How does the public want a COVID-19 vaccine to be allocated? We conducted a conjoint experiment asking 15,536 adults in 13 countries to evaluate 248,576 profiles of potential vaccine recipients who varied randomly on five attributes. Our sample includes diverse countries from all continents. The results suggest that in addition to giving priority to health workers and to those at high risk, the public favors giving priority to a broad range of key workers and to those with lower income. These preferences are similar across respondents of different education levels, incomes, and political ideologies, as well as across most surveyed countries. The public favored COVID-19 vaccines being allocated solely via government programs but were highly polarized in some developed countries on whether taking a vaccine should be mandatory. There is a consensus among the public on many aspects of COVID-19 vaccination, which needs to be taken into account when developing and communicating rollout strategies.

An established finding on ballot design is that top positions on the ballot improve the electoral performance of parties or candidates because voters respond behaviorally to salient information. This article presents evidence on an additional unexplored mechanism: campaigns, that can act before voters, can also adjust their behavior when allocated a top position on the ballot. We use a constituency-level lottery of ballot positions in Colombia to establish, first, that a ballot-order effect exists: campaigns randomly placed at the top earn more votes and seat shares. Second, we show that campaigns react to being placed on top of the ballot: they raise and spend more money on their campaign, and spending itself is correlated with higher vote shares. Our results provide the first evidence for a new mechanism of ballot-order effects examined in many previous studies.

Principled methods for analyzing missing values, based chiefly on multiple imputation, have become increasingly popular yet can struggle to handle the kinds of large and complex data that are also becoming common. We propose an accurate, fast, and scalable approach to multiple imputation, which we call MIDAS (Multiple Imputation with Denoising Autoencoders). MIDAS employs a class of unsupervised neural networks known as denoising autoencoders, which are designed to reduce dimensionality by corrupting and attempting to reconstruct a subset of data. We repurpose denoising autoencoders for multiple imputation by treating missing values as an additional portion of corrupted data and drawing imputations from a model trained to minimize the reconstruction error on the originally observed portion. Systematic tests on simulated as well as real social science data, together with an applied example involving a large-scale electoral survey, illustrate MIDAS’s accuracy and efficiency across a range of settings. We provide open-source software for implementing MIDAS.

Firms in the USA rely on highly skilled immigrants, particularly in the science and engineering sectors. Yet, the recent politics of immigration marks a substantial change to US immigration policy. We implement a conjoint experiment that isolates the causal effect of nativist, anti-immigrant, pronouncements on where skilled potential-migrants choose to immigrate to. While these policies have a significantly negative effect on the destination choices of Chilean and UK student subjects, they have little effect on the choices of Indian and Chinese student subjects. These results are confirmed through an unobtrusive test of subjects’ general immigration destination preferences. Moreover, there is some evidence that the negative effect of these nativist policies are particularly salient for those who self-identify with the Left.

Experiments should be designed to facilitate the detection of experimental measurement error. To this end, we advocate the implementation of identical experimental protocols employing diverse experimental modes. We suggest iterative nonparametric estimation techniques for assessing the magnitude of heterogeneous treatment effects across these modes. And we propose two diagnostic strategies—measurement metrics embedded in experiments, and measurement experiments—that help assess whether any observed heterogeneity reflects experimental measurement error. To illustrate our argument, first we conduct and analyze results from four identical interactive experiments: in the lab; online with subjects from the CESS lab subject pool; online with an online subject pool; and online with MTurk workers. Second, we implement a measurement experiment in India with CESS Online subjects and MTurk workers.

Objective Immigration is a highly salient political issue. We examine the migration preferences of potential emigrants from the United Kingdom to determine whether the migration calculus is primarily economic or political. ,,Revise and Resubmit ,,Revise and Resubmit ,,Revise and Resubmit,Journal of Politics ,,Revise and Resubmit,Journal of Politics Methods A conjoint survey experiment was conducted with U.K. subjects drawn from the CESS, Nuffield College, Oxford University, student subject pool to identify causal drivers of emigration preferences. Results Logit estimation of emigration preferences indicates that economics and politics matters. Anti-immigrant rhetoric, “Trumpian policies,” and the United States deter high-skilled U.K. potential emigrants; economic growth, education, and social benefits attract them. Politics and social benefits are more important for those on the political left, while economics and education weigh more heavily for those on the right. Conclusion What will attract the highly skilled migrants from a post-Brexit United Kingdom? Economics matters of course but for many of these potential emigrants politics is important—they are particularly sensitive to anti-immigrant rhetoric.

Policy referendums around the world succeed regularly and on important policy areas. But why do these policies pass by direct democracy and not through the legislature? While previous work has explored mechanisms that help explain policy incongruence, less work has considered how this impacts policymaking in systems where citizens have alternative venues to pass legislation. I test two novel theories – exploring institutional and behavioral factors respectively – using a combination of district-level voting data, campaign finance information, and a survey of state legislators to understand why policymaking occurs via ballot initiative and not the legislature. I find successful initiatives tend not to be fully captured by the partisan dimension and are supported by more ideologically extreme donors than successful legislative candidates in the same cycles. Taken together, the evidence suggests that initiatives succeed when policies have not taken root in the mainstream policy networks that regulate conventional policymaking.

When researchers suspect that error terms are correlated by group in observational research the standard correction is to cluster the standard errors. But what about in experimental contexts where treatment is randomised? Despite their ubiquity in analyses with group-constant variables, the rationale for using clustered standard errors in experimental contexts remains underdeveloped. In this paper I present an intuitive and applied explanation of when clustering is appropriate, building on recent contributions in the statistics and econometrics literatures. I demonstrate why randomisation does not lead to identical variance estimates across estimation strategies, and conduct a review of experimental studies published between 2017 and 2019 to show that these differences can be considerable. Finally, I provide practical guidance for when and why to cluster standard errors for common experimental designs.