On the quality of qualitative studies in psychological and social research

By Walter H. Schneider
Fathers for Life, 2006 08 05

The terms psychological research and social research are often used interchangeably. Furthermore, we frequently hear or read that the results of psychological or social research are based on qualitative studies.

Qualitative studies are generally based on non-randomly-selected samples, such as opinion polls.  It can be said that, from the perspective of the field of statistics and probabilities (a.k.a. statistical mathematics), the prominent characteristic of qualitative studies is that they have no redeeming qualities.  Still, provided that sample selection is made by rigorously controlled random sampling methods, the results of a qualitative study do permit to draw valid conclusions pertaining, or to make projections, to the general population from which a study sample was selected.

However, study samples that are analyzed in psychological or social research are almost invariably chosen selectively and not randomly, such as that a request for respondents and participants in a survey is made by some biased, non-random process, for example through an e-mail alert or through an Internet opinion poll.  Such sample selection methods will not result in randomly selected, representative study samples.  The resulting study samples are self-selective samples.  A self-selective study sample can only be used to establish a hypothesis.  A hypothesis established on the basis of the characteristics of a self-selected sample can be valid for the sample population, but it cannot be used to draw valid conclusions about the general population from which the self-selected or skewed sample was chosen.

On account of the wide audience of the website of Fathers for Life, social or psychological researchers often make inquiries on whether we would like to participate or assist in sample selection.  For example, recently we received a telephone call from a social researcher in Texas who is working on her doctoral dissertation and is trying to find survey respondents for a study that is to become the "scientific" foundation of her dissertation.  She became elated when I offered to post her request for respondents to the anorexia-nervosa pages of Fathers for Life, provided that she would provide a more detailed outline of what she intended to achieve with her study.  She then wrote:

I am a doctoral student at [omitted] University in [omitted], TX and am currently recruiting fathers of women in recovery from, or who are fully recovered from anorexia nervosa. If you fit this criteria and are interested in participating, please contact me at [omitted]@hotmail.com or (210) [telephone number omitted]. Your participation is totally voluntary, confidential and you can withdraw at any time. The study requires a one on one, confidential interview regarding your role in your daughters' [sic] recovery, any obstacles you've encountered in helping your daughter recover and your general experience of how your daughters' [sic] recovery has changed your relationship with her. Interviews may be conducted in person, online or through telephone and you can choose a male or female interviewer. If you have any further questions, please contact me at the above email address or phone number. Thanks! [omitted]

I responded to the woman:

I will post your request to the anorexia-nervosa pages at Fathers for Life, but I have some reservations regarding the intentions of your study.

It seems to me that your study will not permit to draw valid conclusions from the study sample you wish to recruit to the full population of fathers of women in recovery from anorexia nervosa or of women recovered from anorexia nervosa.

Your selection method is a classic example of how not to select a random sample.

Your sample will be a self-selected sample from a selected population sector, namely such fathers who are willing to participate in an interview and who are active on the Internet.

No valid projections can be made from the results of a statistical analysis of a non-randomly selected study sample.

I will post your request for recruits, along with the reservations I expressed here.  Do you still wish to have me post your notice?

A good number of days went by since then; and we did not hear from her again.  That appears to indicate that she is quite well aware of the limitations of qualitative studies using self-selected study samples.  Even though she would be able to recruit far more respondents with far less effort than she had dreamed was possible, once the limitations of her approach to sample selection were to be pointed out along with her request for respondents, she lost interest.

In psychology qualitative research has come to be defined as research whose findings are not arrived at by statistical or other quantitative procedures.

History

Qualitative research approaches began to gain recognition in the 1970s. The very phrase 'qualitative research' was until then marginalized as a discipline of anthropology or sociology, and terms like ethnography, fieldwork, participant observation and Chicago school (sociology) approach were used instead. During the 1970s and 1980s qualitative research began to be used in other disciplines, and became a dominant - or at least significant - type of research in the fields of women's studies, disability studies, education studies, social work studies, information studies, management studies, nursing service studies, human service studies and others. In the late 1980s and 1990s after a spate of criticisms from the quantitative side, new methods of qualitative research have been designed, to address the problems with reliability and imprecise modes of data analysis.[Taylor, 1998, quoted in Wikipedia]

Social or psychological research is dominated by feminist ideology and by feminist ideologists.  It matters little whether the feminist ideologists (male and female) are leaders, activists or merely followers.  What matters is that the field of social research is dominated by "researchers" who deprecate the value of logical "linear" thinking that they hold to be a deplorable characteristic of the dreaded and hated patriarchy. 

In contrast, quantitative research based on logically correct projections from the characteristics of study samples to the populations from which the study samples were randomly and objectively selected does permit to draw valid conclusions about the populations from which the samples were taken.

In the social sciences, qualitative research is a broad term that describes research that focuses on how individuals and groups view and understand the world and construct meaning out of their experiences. Qualitative research methods are sometimes used together with quantitative research methods to gain deeper understanding of the causes of social phenomena, or to help generate questions for further research. Unlike quantitative methods, qualitative research methods place little importance on developing statistically valid samples, or on searching for statistical support for hypotheses. [Wikipedia]

In other words, the basis of the belief of researchers who engage in qualitative research is that a correct view of reality can be constructed from beliefs, from perceptions and opinions, and that perception trumps reality and the truth.

Research findings in the hard sciences (e. g.: mathematics, physics, chemistry) are based on mathematical proof, often provided through extrapolations and projections provided by the mathematics of statistics (quantitative research) and never on qualitative research, as correct mathematical validation of a hypothesis based only on qualitative research is logically impossible.

"Social sciences" are regarded by many to be pseudosciences, rightfully so when study reports claimed to be the results of valid research studies are based on nothing more than selective and self-selected study samples.  That does not mean that the social sciences are not influential.  After all, uncorroborated hypotheses, even if they were demonstrably nothing more than superstitions (such as the belief that the Earth is flat and the centre of the universe), dominated the course of human social and cultural evolution for many centuries and kept humanity for far too long in the Dark Age.  It does mean that the field of social sciences provides fertile ground in which ignorant or false prophets can thrive and profit.

In consequence, the main goal of true scientific endeavors — the age of enlightenment — has lately become not only ever more elusive but is even being actively avoided and deplored, while the proponents of enlightenment and the truth are being hounded and have their careers destroyed.  The revival of the Dark Age is very much a reality now.

Further reading:

The New Dark Age

The Frankfurt School and 'Political Correctness'

By Michael Minnicino
Schiller Institute, Winter, 1992 issue of FIDELIO Magazine.

....The optimistic belief that one could compose music like Beethoven, paint like Rembrandt, study the universe like Plato and Nicolaus of Cusa, and change world society without violence, was alive in the 1890's—admittedly, it was weak, and under siege, but it was hardly dead. Yet, within twenty short years, these Classical traditions of human civilization had been all but swept away, and the West had committed itself to a series of wars of inconceivable carnage.

What started about a hundred years ago, was what might be called a counter-Renaissance.....it was as though a long checklist had been drawn up, with all of the wonderful achievements of the Renaissance itemized—each to be reversed. As part of this "New Age" movement, as it was then called, the concept of the human soul was undermined by the most vociferous intellectual campaign in history; art was forcibly separated from science, and science itself was made the object of deep suspicion. Art was made ugly because, it was said, life had become ugly.

For the Frankfurt School conspirators, the worst crime was the belief that each individual was gifted with sovereign reason, which could enable him to determine what is right and wrong for the whole society; thus, to tell people that you have a reasonable idea to which they should conform, is authoritarian, paternalistic extremism.

By these standards, the judges of Socrates and Jesus were correct in condemning these two individuals (as, for example, I.F. Stone asserts in one case in his "Trial of Socrates.") It is the measure of our own cultural collapse, that this definition of authoritarianism is acceptable to most citizens, and is freely used by political operations like the Anti-Defamation League and the Cult Awareness Network to "demonize" their political enemies....

Full Story (22 pages)

________________
Posted 2006 08 05