Publications with critique on NCA

This section refers to publications that critisize NCA, followed by a short comment by the NCA development team.  

Sorjonen, K., Wikström Alex, J., & Melin, B. (2017). Necessity as a Function of Skewness. Frontiers in psychology8, 2192.

Reply: This article shows by simulation that two unrelated variables can produce an empty space in the upper left corner. This is a valid observation. However, this situation can be detected by NCA’s significance test (Dul, van der Laan, & Kuik, 2020); see also Dul, van der Laan, Kuik, & Karwowski, 2019) , which tests the null hypothesis that the two variables are unrelated. Therefore, NCA’s significance test is also called a “randomness test”. 

Sorjonen, K., & Melin, B. (2019). Predicting the significance of necessity. Frontiers in psychology10, 283.

Reply: Dul, J., van der Laan, E., Kuik, R, & Karwowski, M. (2019). Necessary Condition Analysis: Type I error, power, and over-interpretation of test results. A reply to a comment on NCA, Frontiers in Psychology, 10, 1493.

This article shows by simulation the probability that NCA’s significance test results in p < 0.05 when X and Y are related. This corresponds to the definition of power of a test. The simulation shows that NCA has high power. Sorjonen and Melin (2019) criticize NCA’s significance test because of its inability to specify which alternative hypothesis of relatedness between X and Y resulted in p < 0.05. However, as shown in a reply by Dul,  van der Laan, Kuik and Karwowski (2019) (see also the long version by Dul, van der Laan, Kuik and Karwowski, 2019) , no null hypothesis test can test a specific alternative hypothesis, and expecting this from a null hypothesis test is a common misconception. NCA’s significance test is a null-hypothesis test (“randomness test”), not a test of an alternative hypothesis.

Thiem, A. (in press). The Logic and Methodology of “Necessary but Not Sufficient Causality” A Comment on Necessary Condition Analysis (NCA). Sociological Methods & Research. (Published online July 2018).

Reply: Dul, J., Vis, B., & Goertz, G. (in press). Necessary Condition Analysis (NCA) Does Exactly What It Should Do When Applied Properly: A Reply to a Comment on NCA. Sociological Methods & Research. (Published online October 2018).

Thiem criticized NCA as a valid method for identifying necessary conditions. He argues that QCA is better equipped to do so. In a reply to this article Dul, Vis and Goertz (in press) show that there are two critical flaws in this article because it is based on wrong assumptions about what NCA aims to do and because it applies NCA incorrectly.