Separation-Resistant and Bias-Reduced Logistic Regression: STATISTICA Macro

Abstract

Logistic regression is one of the most popular techniques used to describe the relationship between a binary dependent variable and a set of independent variables. However, the application of logistic regression to small data sets is often hindered by the complete or quasicomplete separation. Under the separation scenario, results obtained via maximum likelihood should not be trusted, since at least one parameter estimate diverges to infinity. Firth's approach to logistic regression is a theoretically sound procedure, which is guaranteed to arrive at finite estimates even in a separation case. Firth's procedure was also proved to significantly reduce the small sample bias of maximum likelihood estimates. The main goal of the paper is to introduce the STATISTICA macro, which performs Firth-type logistic regression.