Experimental Laboratory

Experimental Research Project

Project Title: Exploring Motives of Environmental Actions: An Experimental Study of Taiwan National Public Service Training Program
Investigators: Don-Yun CHEN (NCCU), Evan BERMAN (VUW), XiaoHu WANG (CityU)
Duration: 2015-2017
Funder: National Science (NSC) Foundation, Taiwan (US$ 100,000)
Abstract: This project uses experimental design to examine how interventional measurement influences environmental consciousness in Taiwan National Public Service Training program. Intervention consists of several layers of stimuli that replicate real-life events in a public service setting. Protest and posttest observations will be obtained for environmental consciousness measured by New Ecological Paradigms Scale (NEP), and how it is used in decision making. Longitudinal data will be made to observe intermediate and long term impact.
 


Project Title: Performance Information Use: Experiments on Performance Dimensions, Communication and Data Sources in Education and Solid Waste Recycling
Investigators: Richard M. WALKER (PI, CityU)
Duration: 2015-16
Funder: Public Policy Research Funding Scheme, HKSARG (HK$597,264)
Abstract: The proposed public policy research project will address several important questions concerning performance information use (PIU) and highlight the role that experimental research methods can play in enhancing the quality of public policy-making in Hong Kong. It will:
 
  1. Contribute to PIU theory and policy debate in public organisations by examining PIU by two important stakeholder groups (users and managers) in two topical policy areas (education and solid waste).
  2. Bring to the attention of key stakeholders in the public policy-making arena the role that experimental methods can play in enriching the policy process in Hong Kong and provide practical guidance on the conduct of those methods to improve the evidence available to support the public policy-making process.
Performance information (PI) is widely available in Hong Kong. Economic theory argues that such information is key to public service users in the absence of market mechanisms. From a political theory point of view, PI brings the government closer to the public by increasing transparency and accountability by permitting scrutiny of and engagement with public service providers. However, research findings suggest that stakeholders’ PI preferences are often unclear and that PI is not always successfully communicated. The proposed study will undertake interlinked experimental interventions in the two aforementioned policy areas  to expand the knowledge base of two key stakeholder groups – users and managers – concerning the:
 
  1. longstanding but highly pertinent trade-offs between the performance dimensions of efficiency and equity;
  2. presentation and communication of absolute and relative PI data; and
  3. presentation and communication of PI from perceptual and archival sources.
Experimental methods place public policy issues under the microscope, examining relationships in detail while controlling for confounding effects, thereby facilitating the isolation of cause-and-effect relationships. Although such methods can advance knowledge in important ways, they have not received widespread attention in Hong Kong across various public policy fields. Thus, in addition to contributing to the debate surrounding PIU, the proposed project will raise awareness of the role of experimental methods and provide guidance and resource tools to policy-makers
 


Project Title: Public Management Evidence Lab (PuMEL)
Investigators: Richard M. WALKER (PI, CityU)
Duration: 2014-2016
Funder: Interdisciplinary Opportunity Fund, City University of Hong Kong (HK$1,200,000)
Abstract: The Public Management Evidence Lab (PuMEL), the forerunner to LaMP, was established by an internal grant from the Interdisciplinary Opportunity Fund run by the offices of the Provost and Vice-President (Research and Technology). It put in place resources to establish the experimental lab and initiate research.
 


Project Title: Special issue of Public Management Review, ‘Experiments, replication and knowledge in public management research’
Investigators: Richard M. WALKER (PI, CityU)
Duration: 2014-2016
Funder: N/A
Abstract: Public management research draws on the full methodological repertoire of the social sciences in the search for valid and reliable evidence, but has favoured observational techniques because they deliver practically relevant research findings. The repertoire has grown to embrace the use of experimental methods and the number public management studies adopting experimental methods (including field, laboratory and survey types) have grown substantially over recent years (Margetts 2011). Recent studies implementing experimental methods have focused on key and important public management topics including citizen-government relations (James 2011; Walker et al 2013; Van Ryzin 2013; Grimmelikhuijsen and Meijer 2014;), equity (Jakobsen and Calmar Andreson 2013), rules (Kaufman and Feeney 2014) and decision-making (Avellaneda 2011).

Experiments hold attraction because they offer strong internal validity—the researcher is in control of the variables under investigation. However, like many methods they suffer from limited external validity. This special issue will tackle this important social science topic in relation to public management research methodologies and discuss a range of conceptual, methodological and empirical issues surrounding external validity and the replication and extension of studies that implement experimental research methods.
Replication of experimental method studies is important because it asserts the external validity of knowledge and its ability to be generalized. Replication is discussed and undertaken, sometimes controversially, in other social science disciplines (Freese 2007; Nosek and Lakens 2014). In public management the growth in studies employing experimental methods has not been matched by replication as very few have been replicated to date. Limited replications may arise because there is a tendency to publish positive results—rather than accept the null hypothesis—and novel ideas are more valued by journal editors than the incremental accumulation of knowledge that comes through replication. A possible outcome is that false positives are likely to be reported because of the desire to reject the null hypothesis, coupled with the discretion researchers often have in research practice and reporting.

This special issue of Public Management Review requests manuscripts on topics including:
 
  • Conceptual, methodological, theoretical and ethical questions of research methodology and replication in social science and public management research,
  • Experiences and best practices in replication studies from other disciplines for public management research, and
  • Replications and extensions of prior studies that use laboratory, field, survey and vignette methods examining public management questions. Extensions can include methodological enhancements, theoretical refinements and improvements to substantive knowledge about important topics.


Project Title: Discovery and Innovation through Experimental Methods
Investigators: Richard M. WALKER (PI, CityU), Samuel M. Y. HO (CityU), Chan Su JUNG (CityU), Graeme LANG (CityU), Wanxin LI (CityU), Martin PAINTER (CityU), Julia TAO (CityU)
Duration: 2013-2015
Funder: Idea Incubator Scheme, City University of Hong Kong (HK$497,500)
Abstract: Public policy is often made on an intuitive basis, however interventions in fields such as health care show how randomized controlled trial experiments can provide invaluable evidence to produce rigorous and better outcomes. The ‘Experiments in Public Policy’ (EPP) Idea Incubator will involve CityU students in the generation of valid and robust policy evidence for Hong Kong that has applicability to CityU, government and civil society groups. Students from all stages of the learning career will be able to benefit from involvement in the whole research process, from design through to policy recommendation and – for some – to academic publication.
 


Project Title: Motivating Waste Reduction and Recycling in the University Workplace: An Experimental Evaluation of Negatively-framed Collective Incentives and Group-level Feedback
Investigators: Richard M. WALKER (PI, CityU), Ivan Po-Chiu, LEE (Co-I, CityU)
Duration: 2016-2017
Funder: Campus Sustainability Fund, City University of Hong Kong (HK$202,560)
Abstract: Research suggests that behavioral interventions, such as providing incentives and feedback, could promote waste reduction and recycling in the residential sector. However, substantially less attempt has been made to investigate whether the positive results can be extended to organizational settings. Based on a cluster-randomized field experiment with a 2 x 2 factorial design, this project will examine the effectiveness of two group-based interventions—negatively-framed collective incentives and group-level feedback—in motivating waste reduction and recycling in the university workplace. Surveys will be administered before and after interventions to identify the role of individual behavioral determinants (e.g., environmental awareness) in influencing the treatment effects. Academic departments as well as administrative and academic support units will be involved in the study. The research findings will demonstrate ways waste reduction and other environmental improvements at CityU can be facilitated. It will better inform the implementation of waste charging policy in organizational settings i.e. CityU.


Project Title: Organizational Design and Public Management: Extending the Experimental Methods Agenda
Investigators: Prof. Richard WALKER (POL)
Duration: Jul, 2016 - Nov, 2016
Funder: RGC General Research Fund (GRF)
Abstract: Public management research draws on the full methodological repertoire of the social sciences in the search for valid and reliable evidence, but has favoured observational techniques because they deliver practically relevant research findings. The repertoire has grown to embrace the use of experimental methods and the number public management studies adopting experimental methods (including field, laboratory and survey types) have grown substantially over recent years (Margetts 2011). Recent studies implementing experimental methods have focused on key and important public management topics including citizen-government relations (James 2011; Walker et al 2013; Van Ryzin 2013; Grimmelikhuijsen and Meijer 2014;), equity (Jakobsen and Calmar Andreson 2013), rules (Kaufman and Feeney 2014) and decision-making (Avellaneda 2011).<br /> <br /> Experiments hold attraction because they offer strong internal validity&mdash;the researcher is in control of the variables under investigation. However, like many methods they suffer from limited external validity. This special issue will tackle this important social science topic in relation to public management research methodologies and discuss a range of conceptual, methodological and empirical issues surrounding external validity and the replication and extension of studies that implement experimental research methods.<br /> Replication of experimental method studies is important because it asserts the external validity of knowledge and its ability to be generalized. Replication is discussed and undertaken, sometimes controversially, in other social science disciplines (Freese 2007; Nosek and Lakens 2014). In public management the growth in studies employing experimental methods has not been matched by replication as very few have been replicated to date. Limited replications may arise because there is a tendency to publish positive results&mdash;rather than accept the null hypothesis&mdash;and novel ideas are more valued by journal editors than the incremental accumulation of knowledge that comes through replication. A possible outcome is that false positives are likely to be reported because of the desire to reject the null hypothesis, coupled with the discretion researchers often have in research practice and reporting.<br /> <br /> This special issue of <em>Public Management Review</em> requests manuscripts on topics including:<br /> &nbsp; <ul> <li>Conceptual, methodological, theoretical and ethical questions of research methodology and replication in social science and public management research,</li> <li>Experiences and best practices in replication studies from other disciplines for public management research, and</li> <li>Replications and extensions of prior studies that use laboratory, field, survey and vignette methods examining public management questions. Extensions can include methodological enhancements, theoretical refinements and improvements to substantive knowledge about important topics.</li> </ul>