Threats to Construct Validity

• Inadequate Explication of Constructs
  • Be sure to provide a careful definition of what you are studying.
  • If you are studying "violence against women" define what you mean
• Do not over-generalize from your data
  • If you are studying battered women, but label your construct as "violence against women," you lack construct validity
• Be careful about generalizing from specific settings
  • For example, if crime statistics come from a city that happens to have an unusually large Hmong population, be careful about generalizing to other settings. Also be careful about making claims about "Asian Americans" based on this data
• Do not identify the wrong construct
  • If levels of women's victimization match levels of men's victimization by socioeconomic status, then crime statistics may show more about risk based on SES than gender
• Construct confounding
  • Rarely are indicators pure representations of a concept
  • If women in a particular ethnic group or social class are over-represented as victims, then victimization statistics confound ethnic identity or social class identification with the construct "violence against women"
• Mono-method of gathering data
  • If all your data is from telephone polls, then any group not having a phone or using technology to block unknown numbers will be excluded from the sample
  • This becomes important if the excluded group tend to have common characteristics, like being of a particular age group or socioeconomic status
• Self-report bias
  • If the data collected is based on self-reported information, be wary
  • People will bias their self-representations if they think they could derive benefit or harm from coming across a particular way. This includes the desire to be seen as respectable or cooperative in the eyes of the researcher
  • Be especially wary of cross-cultural and political differences. For example, in a society where female victims face share for their victimizations, many women will not report crimes committed against them
  • Self-report bias is not only a problem for data collected from individuals, but data collected from institutions as well
• Lack of consistency in measurement
  • If data is derived from more than once source, but is not always collected consistently, it is a threat to construct validity
  • International crime statistics gathered by various governments may not define a particular category of crime the same way
  • Some countries report convictions, some report crimes based on victimization. These are different

SUMMARY

Always pay attention to how data was collected and compiled.

Many psychological or behavioral constructs (e.g. self-esteem, intelligence, marital happiness, etc.) are measured using psychological tests. Reviews of the validity of these measurements are available in The Mental Measurements Yearbooks [REF BF 431.3 M4].

Next