Quantitative Empirical Research in Management Accounting: A Proposed Typology and Implications for Internal versus External Validity
DOI:
https://doi.org/10.17524/repec.v16i3.3155Keywords:
Quantitative empirical methods, Management accounting, Control group, Sample representativeness, Validity typesAbstract
Purpose: The purpose of this study is twofold. First, we propose a typology of quantitative empirical research in management accounting based on two design features: presence of control group and sample representativeness. Second, we discuss implications of the methods for trade-offs between internal and external validity.
Method: Based on previous methodological studies we develop a typology with eight quantitative empirical methods.
Results: Based on the two design features, we propose eight quantitative empirical methods for management accounting studies: (1) laboratory experiment, (2) crowdsourcing experiment, (3) field experiment, (4) natural experiment, (5) single entity survey, (6) proprietary archival study, (7) large-scale survey and (8) pre-structured archival study. In addition, we critically compare the trade-offs and discuss the implications of these methods for internal and external validity.
Contributions: The contribution is twofold. First, the proposed typology can help junior management accounting researchers increase the familiarity with the available empirical methods, some of which are still incipient in Brazil. Second, this study states that the choice of an empirical method typically implies benefits in terms of a validity type (e.g. internal validity) at the expense of other validity type (e.g., external validity). Claims of causality and results generalizability depend on which validity type is prioritized and remedies adopted to increase the overall validity of a study’s results.
References
Aguiar, A. B. (2017). Pesquisa Experimental Em Contabilidade: Propósito, Desenho E Execução. Advances in Scientific and Applied Accounting, 10(2), 224–244. https://doi.org/10.14392/asaa.2017100206
Aguiar, A. B. (2018). O pequeno mundo da pesquisa em contabilidade gerencial no Brasil: discussão sobre desenhos alternativos de pesquisa. Revista de Contabilidade e Organizações, 12, e151933. https://doi.org/10.11606/issn.1982-6486.rco.2018.151933
Aguinis, H., & Bradley, K. J. (2014). Best Practice Recommendations for Designing and Implementing Experimental Vignette Methodology Studies. Organizational Research Methods, 17(4), 351–371. https://doi.org/10.1177/1094428114547952
Aguinis, H., & Ramani, R. S. (2021). MTurk Research : Review and Recommendations. Journal of Management, 47(4), 823–837. https://doi.org/10.1177/0149206320969787
Asay, H. S., Guggenmos, R. D., Kadous, K., Koonce, L., & Libby, R. (2021). Theory Testing and Process Evidence in Accounting Experiments. The Accounting Review.
Bedford, D. S., Spekle, R. F., & Widener, S. K. (2022). Accounting , Organizations and Society Budgeting and employee stress in times of crisis : Evidence from the Covid-19 pandemic. Accounting, Organizations and Society, (xxxx). https://doi.org/10.1016/j.aos.2022.101346
Bentley, J. W. (2021). Improving the Statistical Power and Reliability of Research Using Amazon Mechanical Turk. Accounting Horizons, 35(4), 45–62. https://doi.org/10.2308/HORIZONS-18-052
Bloomfield, R., Nelson, M. W., & Soltes, E. (2016). Gathering Data for Archival, Field, Survey, and Experimental Accounting Research. Journal of Accountig Research, 54(2), 341–395. https://doi.org/10.1111/1475-679X.12104
Bol, J. C., Braga de Aguiar, A., & Lill, J. B. (2020). Peer-Level Calibration of Performance Evaluation Ratings : Are There Winners or Losers ?
Brüggen, A., Grabner, I., & Sedatole, K. L. (2021). The Folly of Forecasting: The Effects of a Disaggregated Demand Forecasting System on Forecast Error, Forecast Positive Bias, and Inventory Levels. The Accounting Review, 96(2), 127–152. https://doi.org/10.2308/tar-2018-0559
Carpenter, J. P., Harrison, G. W., & List, J. A. (2005), Field experiments in economics: An introduction. In: Harrison, G. W., Carpenter, J., & List, J. A. (Eds.) Field Experiments in Economics (Research in Experimental Economics, Vol. 10), Emerald Group Publishing Limited, Bingley, pp. 1-15. https://doi.org/10.1016/S0193-2306(04)10001-X
Chmielewski, M., & Kucker, S. C. (2020). An MTurk Crisis ? Shifts in Data Quality and the Impact on Study Results. Social Psychological and Personality Science, 11(4), 464–473. https://doi.org/10.1177/1948550619875149
Cronin, M., Erkens, D. H., Schloetzer, J. D., & Tinsley, C. H. (2021). How controlling failure perceptions affects performance: Evidence from a field experiment. Accounting Review, 96(2), 205–230. https://doi.org/10.2308/TAR-2018-0146
Das, R., Jain, K. K., & Mishra, S. K. (2016). Archival Research: A Neglected Method in Organization Studies. Benchmarking: An International Journal. https://doi.org/https://doi.org/10.1108/BIJ-08-2016-0123
Dyckman, T. R., & Zeff, S. A. (2014). Some methodological deficiencies in empirical research articles in accounting. Accounting Horizons, 28(3), 695-712. https://doi.org/10.2308/acch-50818
Flammer, C., & Kacperczyk, A. (2016). The impact of stakeholder orientation on innovation: Evidence from a natural experiment. Management Science, 62(7), 1982–2001. https://doi.org/10.1287/mnsc.2015.2229
Floyd, E., & List, J. A. (2016). Using Field Experiments in Accounting and Finance. Journal of Accounting Research, 54(2), 437–475. https://doi.org/10.1111/1475-679X.12113
Forker, E., Grabner, I., & Sedatole, K. (2020). Does learning by disaggregating accelerate learning by doing? The effect of forecast disaggregation on the rate of improvement in demand forecast accuracy.
Frezatti, F., Aguiar, A. B., Wanderley, C. A., & Malagueño, R. (2015). A pesquisa em contabilidade gerencial no Brasil: desenvolvimento, dificuldades e oportunidades. Revista Universo Contábil, 11(1), 47-68. http://dx.doi.org/10.4270/ruc.2015147-68
Gassen, J. (2014). Causal inference in empirical archival financial accounting research. Accounting, Organizations and Society, 39(7), 535–544. https://doi.org/10.1016/j.aos.2013.10.004
Haesebrouck, K. (2021). The Effects of Information Acquisition Effort, Psychological Ownership, and Reporting Context on Opportunistic Managerial Reporting*. Contemporary Accounting Research, 38(4), 3085–3112. https://doi.org/10.1111/1911-3846.12712
Harrison, G. W. (2005), Field experiments and control. In: Harrison, G. W., Carpenter, J., & List, J. A. (Eds.) Field Experiments in Economics (Research in Experimental Economics, Vol. 10), Emerald Group Publishing Limited, Bingley, pp. 17-50. https://doi.org/10.1016/S0193-2306(04)10002-1
Heckman, J. J. (1979). Sample Selection Bias as a Specification Error. Econometrica, 47(1), 153–161.
Hesford, J. W., Lee, S. H., Van der Stede, W. A., & Young, S. M. (2006). Management Accounting: A Bibliographic Study. Handbooks of Management Accounting Research, 1, 3–26. https://doi.org/10.1016/S1751-3243(06)01001-7
Hiebl, M. R. W., & Richter, J. F. (2018). Response Rates in Management Accounting Survey Research. Journal of Management Accounting Research, 30(2), 59–79. https://doi.org/10.2308/jmar-52073
Ikäheimo, S., Kallunki, J.-P., University, S. M., & Schiehll, E. (2018). Do White-Collar Employee Incentives Improve Firm. Journal of Management Accounting Research, 30(3), 95–115. https://doi.org/10.2308/jmar-51902
Kinney, W. R. (2019). The Kinney Three Paragraphs (and More) for Accounting Ph.D. Students. Accounting Horizons, 33(4), 1–14. https://doi.org/10.2308/acch-52451
Laviers, L., Sandvik, J., & Xu, D. (2021). CEO Pay Ratio Voluntary Disclosures and Investor Reactions.
Lonati, S., Quiroga, B. F., Zehnder, C., & Antonakis, J. (2018). On doing relevant and rigorous experiments: Review and recommendations. Journal of Operations Management, 64(April), 19–40. https://doi.org/10.1016/j.jom.2018.10.003
Lourenço, S. M. (2019). Field Experiments in Managerial Accounting Research. Foundations and Trends® in Accounting, 14(1), 1–72. https://doi.org/10.1561/1400000059
Luft, J., & Shields, M. (2002). Zimmerman’s Contentious Conjectures: Describing the Present and Prescribing the Future of Empirical Management Accounting Research. European Accounting Review, 11(4), 795–803. https://doi.org/10.1080/0963818022000047091
Luft, J., & Shields, M. D. (2014). Subjectivity in developing and validating causal explanations in positivist accounting research. Accounting, Organizations and Society, 39(7), 550-558. https://doi.org/10.1016/j.aos.2013.09.001
Mcvay, S. E. (2011). Discussion of Do Control Effectiveness Disclosures Require SOX 404(b) Internal Control Audits? A Natural Experiment with Small U.S. Public Companies. Journal of Accounting Research, 49(2), 449–456. https://doi.org/10.1111/j.1475-679X.2011.00403.x
Moers, F. (2006). Doing Archival Research in Management Accounting. Handbooks of Management Accounting Research, 1, 399–413. https://doi.org/10.1016/S1751-3243(06)01016-9
Mortensen, T., Fisher, R., & Wines, G. (2012). Students as surrogates for practicing accountants: Further evidence. Accounting Forum, 36(4), 251–265. https://doi.org/10.1016/j.accfor.2012.06.003
Mucci, D. M., Frezatti, F., & Bido, D. de S. (2021). Enabling design characteristics and budget usefulness. RAUSP Management Journal, 56, 38–54. https://doi.org/10.1108/RAUSP-04-2019-0058
Murphy, P. R., Wynes, M., Hahn, T.-A., & Devine, P. G. (2019). Why are People Honest? Internal and External Motivations to Report Honestly. Contemporary Accounting Research, 53(9), 1689–1699. https://doi.org/10.1017/CBO9781107415324.004
Nascimento, A. R., Junqueira, E., & Martins, G. A. (2010). Pesquisa Acadêmica em Contabilidade Gerencial no Brasil: Análise e Reflexões sobre Teorias, Metododologias e Paradigmas. Revista de Administração Contemporânea, 14(6), 1113–1133.
Oehlert, G. W. (2003). A First Course in Design and Analysis of Experiments. In The American Statistician (Vol. 57). https://doi.org/10.1198/tas.2003.s210
Peer, E., Brandimarte, L., Samat, S., & Acquisti, A. (2017). Beyond the Turk : Alternative platforms for crowdsourcing behavioral research. Journal of Experimental Social Psychology, 70, 153–163. https://doi.org/10.1016/j.jesp.2017.01.006
Roe, B. E., & Just, D. R. (2009). Internal and external validity in economics research: Tradeoffs between experiments, field experiments, natural experiments, and field data. American Journal of Agricultural Economics, 91(5), 1266-1271. https://doi.org/10.1111/j.l467-8276.2009.01295.
Saunders, M. N. K., Lewis, P., & Thornhill, A. (2019). Research Methods for Business Students. In Pearson (8th ed., Vol. 3). https://doi.org/10.1108/qmr.2000.3.4.215.2
Shadish, W. R. ., Cook, T., & Campbell, D. (2002). Experimental and quasi-experimental designs for generalized causal inference. In Houghton Mifflin Company. https://doi.org/10.1016/j.evalprogplan.2004.01.006
Smith, M. (2022). Research Methods in Accounting (6th ed.). SAGE.
Spekle, R. F., & Widener, S. K. (2018). Challenging Issues in Survey Research: Discussion and Suggestions. Journal of Management Accounting Research, 30(2), 3–21. https://doi.org/10.2308/jmar-51860
Sprinkle, G. B., & Williamson, M. G. (2006). Experimental research in managerial accounting. Handbooks of Management Accounting Research, 1, 415-444. https://doi.org/10.1016/S1751-3243(06)01017-0
Swieringa, R. J., & Weick, K. E. (1982). An Assessment of Laboratory Experiments in Accounting. Journal of Accounting Research, 20, 56–101.
Trochim, W. M., Donnelly, J. P., & Arora, K. (2016). Research Methods - The essential knowledge base (2nd ed.). Cengage Learning.
Trottier, K., & Gordon, I. M. (2018). Students as surrogates for managers: Evidence from a replicated experiment. Canadian Journal of Administrative Sciences, 35(1), 146–161. https://doi.org/10.1002/cjas.1377
Van der Stede, W. A., Young, S. M., & Chen, C. X. (2006). Doing Management Accounting Survey Research. Handbooks of Management Accounting Research, 1, 445–478. https://doi.org/10.1016/S1751-3243(06)01018-2
Van der Stede, W. A. (2014). A manipulationist view of causality in cross-sectional survey research. Accounting, Organizations and Society, 39(7), 567-574. https://doi.org/10.1016/j.aos.2013.12.001
Wouters, M., & Wilderom, C. (2008). Developing performance-measurement systems as enabling formalization: A longitudinal field study of a logistics department. Accounting, Organizations and Society, 33(4–5), 488–516. https://doi.org/10.1016/j.aos.2007.05.002
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Journal of Education and Research in Accounting (REPeC)
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who publish with this journal agree to the following terms:
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 3.0 Unported License, which allows the sharing of the work and recognition of authorship and its initial publication in this journal. This license allows others to distribute, remix, adapt, or create derivative works, even for commercial purposes, provided credit is given for the original creation.
b)There is no financial compensation to the authors in any capacity, for articles published in RePEc.c) The articles published in RePEc are the sole responsibility of the authors.
