What are the different types of validity that must be considered when conducting a survey?

The different types of validity that are important to survey research include construct validity, convergent validity, content validity, representation validity, face validity, criterion validity, concurrent validity, predictive validity, statistical conclusion validity, internal validity, external validity, and ecological validity. Descriptions of each are provided below.

  • Construct validity refers to the extent to which a survey measures what it says it measures. For example, to what extent is an IQ questionnaire actually measuring “intelligence”?
  • Convergent validity refers to the degree to which a measure is correlated with other measures that it is theoretically predicted to correlate with.
  • Content validity evidence involves the degree to which the content of the survey matches a content domain associated with the construct. For example, a survey of the ability to add two numbers should include a range of combinations of digits. A survey with only one-digit numbers, or only even numbers, would not have good coverage of the content domain. Content related evidence typically involves subject matter experts (SME’s) evaluating survey items against the survey specifications.
  • Representation validity, also known as translation validity, is about the extent to which an abstract theoretical construct can be turned into a specific practical survey.
  • Face validity is very closely related to content validity. Face validity is an estimate of whether a survey appears to measure a certain criterion; it does not guarantee that the survey actually measures phenomena in that domain. Measures may have high validity, but when the survey does not appear to be measuring what it is, it has low face validity. Indeed, when a survey is subject to faking (malingering), low face validity might make the survey more valid. Considering one may get more honest answers with lower face validity, it is sometimes important to make it appear as though there is low face validity whilst administering the measures.
  • Criterion validity evidence involves the correlation between the survey and a criterion variable (or variables) taken as representative of the construct. In other words, it compares the survey with other measures or outcomes (the criteria) already held to be valid. For example, employee selection surveys are often validated against measures of job performance (the criterion), and IQ surveys are often validated against measures of academic performance (the criterion).
  • Concurrent validity refers to the degree to which the operationalization correlates with other measures of the same construct that are measured at the same time. When the measure is compared to another measure of the same type, they will be related (or correlated). For example, surveys are administered to current employees and then correlated with their scores on performance reviews.
  • Predictive validity refers to the degree to which the operationalization can predict (or correlate with) other measures of the same construct that are measured at some time in the future. Again, with the selection survey example, this would mean that the surveys are administered to applicants, all applicants are hired, their performance is reviewed at a later time, and then their scores on the two measures are correlated. Without a valid design, valid conclusions cannot be drawn.
  • Statistical conclusion validity is the degree to which conclusions about the relationship among variables based on the data are correct or ‘reasonable’. Statistical conclusion validity involves ensuring the use of adequate sampling procedures, appropriate statistical surveys, and reliable measurement procedures. As this type of validity is concerned solely with the relationship that is found among variables, the relationship may be solely a correlation.
  • Internal validity is an inductive estimate of the degree to which conclusions about causal relationships can be made (e.g. cause and effect), based on the measures used, the research setting, and the whole research design. Good experimental techniques, in which the effect of an independent variable on a dependent variable is studied under highly controlled conditions, usually allow for higher degrees of internal validity than, for example, single-case designs.
  • External validity concerns the extent to which the (internally valid) results of a study can be held to be true for other cases, for example to different people, places or times. In other words, it is about whether findings can be validly generalized. If the same research study was conducted in those other cases, would it get the same results?
  • Ecological validity is the extent to which research results can be applied to real life situations outside of research settings. This issue is closely related to external validity but covers the question of to what degree experimental findings mirror what can be observed in the real world (ecology = the science of interaction between organism and its environment).

Related Posts