According to Hoc , pp. Importantly, Hoc notes that this distinction is made from the perspective of the researcher. For example, according to Sonkusare et al. Furthermore, researchers have long recognized that artificiality arises when the experimental methods employed by researchers interfere with the naturality of the psychological phenomena one aims to study.
Consequently, there is always an inherent trade-off between the degree of artificiality imposed by the experimental conditions and the naturality of the phenomena under scientific investigation Brunswik, ; Barker, ; Banaji and Crowder, ; Kingstone et al. Interestingly, discussions on the naturality—artificiality of experimental methods have a long pedigree in psychological science.
By the end of the 19th century, Thorndike and Mills already argued fiercely about what methodology should be favored to study the behavior of cats.
Mills , p. In response to Mills, Thorndike , p. He simply names them unnatural. Figure 2. A hungry cat is placed in a box which can be opened if the cat pushes a latch. Thorndike discovered that after several trials, the time it takes the cat to escape from the box decreases. Experiments with puzzle boxes remain popular today to study the cognitive capacities of animals, for example, see Richter et al.
Copyright statement — Image created and owned by author IH and is based on E. We think that this historical discussion between Thorndike and Mills is illuminating, because it characterizes the heart of the discussion on ecological validity nowadays.
As Dunlosky et al. Indeed, our survey of the literature also shows that the historical discussion between Thorndike and Mills is by no means a discussion of the past. We must often ask for the specifics, because such remarks typically remain undefined by those who toss them around.
For example, Sonkusare et al. In the field of social attention, researchers have often used schematic images, photographs and videos of people and social scenes as stimuli to study the cognitive, behavioral, and physiological processes of face perception, gaze following and joint attention Langton et al.
Or do they mean that the former is more complex than the latter from the perspective of the researcher who wants to understand the cognitive, behavioral, and physiological processes of face perception and social attention? Experimental psychologists use Gabor patches as visual stimuli because they offer a high degree of experimental control over various stimulus parameters e.
Gabor patches can described with mathematical precision i. Figure 3. Are Gabor patches simple or complex compared to a picture of zebras? A A Gabor patch. B A photograph of zebras. The uniquely striped patterns of the zebra makes them most familiar to humans, whereas the question why zebras have such beautiful stripes remains the topic of much discussion among biologists, see e.
Photograph of zebras was made by Ajay Lalu and has been made publicly available by the owner for non-profit purposes via Pixabay. Our point here is that simplicity—complexity is in the eye of the beholder. Who is to say what is more simple or complex? Physicists, computer scientists, information theorists, and evolutionary biologists have developed various definitions and measures of complexity e.
But what definitions and measures of complexity are used by psychologists and cognitive scientists? According to Gell-Mann , p. They depend on the coarse graining level of detail of the description of the entity, on the previous knowledge and understanding of the world that is assumed, on the language employed, on the coding method used for conversion from that language into a string of bits, and on the particular idealized computer chosen as a standard.
In short, the concept of ecological validity has transformed significantly over the past several decades since it was introduced by Brunswik It has lost most of its former theoretical and methodological cohesion for a history, see Hammond, , and the definitions and requirements of ecological validity used by researchers nowadays are seldom made explicit.
A closer look at the literature suggests that many researchers seem to assume that everyone understands what is meant by this term, while in fact the concept of ecological validity is seldom defined. In our opinion, the contemporary framework of ecological validity eventually falls short of providing researchers with a tractable research program.
Researchers seem to primarily base their judgments of ecological validity upon their own particular theoretical assumptions and considerations about the so-called artificiality—naturality and simplicity—complexity of experimental situations, typically in the absence of a more formal set of criteria.
He commented that:. Thus, in order to talk sensibly about whether some results can also be generalized to particular situations beyond the experimental conditions in which those results were obtained, researchers first need to specify the range and distributions of the variables and conditions to which their results are supposed to be applicable.
Ultimately, the real challenge for researchers is to develop a theory of how specific environmental contexts are related to various forms of cognitive and behavioral functioning.
Researchers in psychological science are typically concerned with the nature of the organism, yet, the nature of the environment and its relation to cognitive and behavioral functioning has received considerably less attention from a theoretical point of view Barker, ; Heft, Interestingly, there have been several scholars who have dedicated themselves to precisely this question, and whose theories of cognition and behavior included a clear perspective on the nature of the environment.
According to Tolman and Brunswik , the nature of the environment, as it appears to the organism, is full of uncertainties. However, since the environment is inherently probabilistic and only partly predictable, the organism continually needs to adjust its assumptions about the state of the environment based on the available information sources.
Important to behavior settings is that they are the product of the collective actions of a group of individuals. Their geographical location can be specified e. According to Gibson, there exists a strong mutuality and reciprocity between the organism and its environment. For example, a sufficiently firm and smooth surface may be walk-on-able, run-on-able, or dance-on-able, whereas a rough surface cluttered with obstacles does not afford such actions Heft, In short, affordances are properties of an organism-environment system.
They are perceiver-relative functional qualities of an object, event or place in the environment and they are dependent on the particular features of the environment and their relationships with the functional capabilities of a particular individual for more in-depth discussions, see e. In order to describe and specify the environment and its relation to cognitive and behavioral functioning, we may draw on these scholars to guide us in a more specific direction.
While we do not specifically recommend any of these perspectives, we think they are illuminating because these scholars motivate us to ask questions such as: What is the specific functional context of the cognitive and behavioral processes one is interested in?
What do we know or assume to know about the range and distribution of these variables and conditions? And how can these variables and conditions be represented in experimental designs to study specific patterns of cognitive and behavioral functioning?
In order to answer some these questions, several researchers have emphasized the importance of first observing how people behave in everyday situations prior to experimentation. For example, Kingstone et al. To exemplify how some of the ideas outlined above may be useful to researchers, we will apply these ideas to a research topic of our interest: social attention. The field of social attention, as briefly discussed previously, is primarily focused on how attention is influenced by socially relevant objects, events, and situations, most notably, interactions with other social agents.
One example of this is given by Macdonald and Tatler In this study, Macdonald and Tatler investigated how social roles given to participants influenced their social gaze behavior during a collaborative task: baking a cake together. Macdonald and Tatler showed that, regardless of whether social roles were assigned or not, participants did not gaze at their cake-baking partners very often while carrying out the task.
Instead, statements about generalizability need to be considered within a more constrained and carefully defined context cf. Brunswik, ; Simons et al. By doing so, we may begin to understand the context-generic and context-specific aspects of attentional processes, allowing for a more sophisticated theory of social attention.
There are also good examples where researchers have been more explicit about the specific characteristics of social situations that they are interested in. Researchers in the field of social attention have, for example, tried to unravel the different functions of gaze behavior.
One important function of gaze behavior is to acquire visual information from the world, however, within a social context, gaze may also signal important information to others which may be used to initiate and facilitate social interaction see e. In a series of experiments, researchers have systematically varied whether, and the degree to which social interaction between two people was possible, and measured how gaze was modulated as a function of the social context Laidlaw et al.
In other studies, researchers have been explicit about the task-demands and social contexts that elicit specific patterns of gaze behavior, for example, in the context of face-to-face interactions and conversational exchanges Ho et al.
We think that, if researchers would try to be more explicit in their descriptions of task-demands and social contexts in relation to gaze, this may prove to be a solid basis for a more sophisticated theory of social attention, yet such work remains challenging for a recent review, see Hessels, in press.
The concept of ecological validity itself is seldom defined and interpretations differ among researchers. This will be a more constructive way to uncover the context-specific and context-generic principles of cognition and behavior. The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Aanstoos, C. Experimental psychology and the challenge of real life. Adami, C. What is complexity? Bioessays 24, — PubMed Abstract Google Scholar. Adolph, K. Alderman, N. Ecological validity of a simplified version of the multiple errands shopping test.
Anderson, C. Research in the psychological laboratory: truth or triviality? Araujo, D. Ecological validity, representative design, and correspondence between experimental task constraints and behavioral setting: comment on Rogers. Kadar, and Costall Ashcraft, M. Cognition , 5th Edn. Google Scholar. Aspland, H. Observational measures of parent-child interaction: an introductory review.
Child Adolesc. Mental Health 8, — Banaji, M. The bankruptcy of everyday memory. Barker, R. Berkowitz, L. External validity is more than skin deep: Some answers to criticisms of laboratory experiments. Birmingham, E. Burack, J. Enns, and N. Fox Oxford University Press , — Blanco-Elorrieta, E. Ecological validity in bilingualism research and the bilingual advantage.
Trends Cogn. Bronfenbrenner, U. Toward an experimental ecology of human development. Brunswik, E. Organismic achievement and environmental probability. Remarks on functionalism in perception.
The Conceptual Framework of Psychology. Chicago: University of Chicago Press. Representative design and probabilistic theory in a functional psychology. Perception and the Representative Design of Psychological Experiments. Berkeley: University of California Press.
Campbell, D. Factors relevant to the validity of experiments in social settings. Caro, T. Concordance on zebra stripes: a comment on Larison et al. Open Sci. Caruana, N. Simulating social interactions for the experimental investigation of joint attention. Cohen, G. Memory in the Real World. Abingdon: Psychology Press. Danziger, K. Cambridge: Cambridge University Press. Davids, K. Ecological validity in understanding sport performance: some problems of definition. Quest 40, — Dhami, M.
The role of representative design in an ecological approach to cognition. Dunlosky, J. Hacker, J. Dunlosky, and A. Graesser Abingdon: Routledge , — Eaton, W. Interracial imitation at a summer camp. Edmonds, B. Bonner Dordrecht: Kluwer. Foulsham, T. The where, what and when of gaze allocation in the lab and the natural environment. Fredericksen, R. How big is a Gabor patch, and why should we care?
JOSA A 14, 1— Freeth, M. What affects social attention? Social presence, eye contact and autistic traits. PLoS One 8:e Frischen, A. Gaze cueing of attention: visual attention, social cognition, and individual differences. Gell-Mann, M. Complexity 1, 16— Gibson, J.
The Perception of the Visual World. Cambridge: Houghton Mifflin Company. On the relation between hallucination and perception. Leonardo 3, — Original date of publication Gillis, J. Gobel, M. The dual function of social gaze. Cognition , — Goodwin, C. A History of Modern Psychology , 5 Edn. Greenwald, A. Within-subjects designs: to use or not to use? Gregory, N. Does social presence or the potential for interaction reduce social gaze in online social scenarios?
Hammond, K. Ecological Validity: Then and Now. Hatfield, G. Psychology, philosophy, and cognitive science: reflections on the history and philosophy of experimental psychology. Mind Lang. Heft, H. Hove: Psychology Press. An ecological approach to psychology. Heinrichs, R. Current and emergent applications of neuropsychological assessment: problems of validity and utility. Hessels, R. How does gaze to faces support face-to-face interaction? A review and perspective.
Gaze allocation in face-to-face communication is affected primarily by task structure and social context, not stimulus-driven factors. But results may not be generalizable to other populations. Thus external validity is poor for studies with sociodemographic restrictions; studies that exclude severely ill and suicidal patients, or patients with personality disorders, substance use disorders, and medical comorbidities; studies that disallow concurrent treatments; and so on.
External validity is also limited in short-term studies of patients who need to be treated for months to years. External validity, like internal validity, is based on judgment and is not a computed statistic. Ecological validity examines whether the results of a study can be generalized to real-life settings.
External validity asks whether the findings of a study can be generalized to patients with characteristics that are different from those in the study, or patients who are treated in a different way, or patients who are followed up for longer durations.
In contrast, ecological validity specifically examines whether the findings of a study can be generalized to naturalistic situations, such as clinical practice in everyday life. Ecological validity is, therefore, a subtype of external validity. The ecological validity of an instrument can be computed as a correlation between ratings obtained with that instrument and an appropriate measure in naturalistic practice or in everyday life.
The ecological validity of a study is a judgment and is not a computed statistic. Ecological validity was originally invoked in the context of laboratory studies that required to be generalized to real-life situations. In fact, these cognitive and psychomotor tests, especially when based on computerized tasks, have no parallel in everyday life. How much less ecological validity, then, would research in animal models of different neuropsychiatric states have for patients in clinical practice?
This explains why drugs that work in animal models often fail in humans. On a parting note, a good understanding of the concepts of internal, external, and ecological validity is necessary to properly design and conduct studies and to evaluate the merits and applications of published research.
National Center for Biotechnology Information , U. Indian J Psychol Med. Chittaranjan Andrade. Author information Copyright and License information Disclaimer. Address for correspondence: Dr. E-mail: moc.
This is an open access journal, and articles are distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4. This article has been cited by other articles in PMC. Abstract Reliability and validity describe desirable psychometric characteristics of research instruments. Financial support and sponsorship Nil. Conflicts of interest There are no conflicts of interest.
Am J Psychiatry. Streiner DL. Statistics commentary series: Commentary reliability. J Clin Psychopharmacol. Statistics commentary series: Commentary no validity. Assessing the quality of randomized controlled trials.
0コメント