Glossary

The criterion that shows how low a p-value should be before the sample result is considered unlikely enough to reject the null hypothesis (Usually set to .05).

Another term for reversal design.

A brief summary of the study's research question, methods, results and conclusions.

A statistical test used when there are more than two groups or condition means to be compared.

When a participants name and other personally identifiable information is not collected at all.

Stands for the APA’s Ethical Principles of Psychologists and Code of Conduct. It was first published in 1953 and includes about 150 specific ethical standards that psychologists and their students are expected to follow.

An application of the principles of experimental analysis of behavior that plays an important role in contemporary research on developmental disabilities, education, organizational behavior, and health, among many other applied areas.

Research conducted primarily to address some practical problem.

A persons right to make their own choices and take their own actions free from coercion.

Research conducted primarily for the sake of achieving a more detailed and accurate understanding of human behavior, without necessarily trying to address any particular practical problem.

Measures in which some other aspect of participants’ behavior is observed and recorded.

A set of federal guidelines written in 1978 as a response to the abuses of the Tuskegee study that recognize three important principles in research with humans: justice, respect for persons, and beneficience, and that formed the basis for federal regulations applied to research.

Underscores the importance of maximizing the benefits of research while minimizing harms to participants and society.

An experiment in which each participant is tested in only one condition.

All of the independent variables are manipulated between subjects.

All the conditions occur once in the sequence before any of them is repeated.

An effect of being tested in one condition on participants’ behavior in later conditions.

A variable that represents a characteristic of an individual, such as chosen major, and is typically measured by assigning each individual's response to one of several categories (e.g., Psychology, English, Nursing, Engineering, etc.).

Is the middle of a distribution—the point around which the scores in the distribution tend to cluster. (Another term for central tendency is average.)

The diagnosis and treatment of psychological disorders and related problems.

Questionnaire items that ask a question and provide a limited set of response options for participants to choose from.

A part of structured observation whereby the observers use a clearly defined set of guidelines to "code" behaviors—assigning specific behaviors they are observing to a category—and count the number of times or the duration that the behavior occurs.

The most widely used measure of effect size for differences between group or condition means, which is the difference between the two means divided by the standard deviation.

A method in which an equal number of participants complete each possible order of conditions. 

Describes the behaviors and internal processes that make up a psychological construct, along with how it relates to other variables.

A form of criterion validity, where the criterion is measured at the same time (concurrently) as the construct.

The different levels of the independent variable to which participants are assigned.

A helper who pretended to be a real participant in a study.

A range of values that is computed in such a way that some percentage of the time (usually 95%) the population parameter will lie within that range.

An agreement not to disclose participants’ personal information without their consent or some appropriate legal authorization.

Tendency to focus on cases that confirm our intuitive beliefs and to disregard cases that disconfirm our beliefs.

An extraneous variable that varies systematically with the independent variable, and thus confuses the effect of the independent variable with the effect of the extraneous one.

A specific type of extraneous variable that systematically varies along with the variables under investigation and therefore provides an alternative explanation for the results.

The process of obtaining informed consent by having the participants read and sign the form.

One of the "big four" validities, whereby the research question is clearly operationalized by the study's methods.

Psychological variables that represent an individual's mental state or experience, often not directly observable, such as personality traits, emotional states, attitudes, and abilities.

A family of systematic approaches to measurement using qualitative methods to analyze complex archival data.

The extent to which a measure reflects all aspects of the construct of interest.

Unintended influences on respondents’ answers because they are not related to the content of the item but to the context in which the item appears.

Holding extraneous variables constant in order to separate the effect of the independent variable from the effect of the extraneous variables.

The condition in which participants do not receive the treatment.

A common method of non-probability sampling in which the sample consists of individuals who happen to be easily available and willing to participate (such as introductory psychology students).

A form of criterion validity whereby new measures are correlated with existing established measures of the same construct.

When psychologists use multiple operational definitions of the same construct—either within a study or across studies.

Describes the strength and direction of the relationship between two variables (often measured by Pearson's r).

Shows the correlation coefficient between pairs of variables in the study.

Varying the order of the conditions in which participants are tested, to help solve the problem of order effects in within-subjects experiments.

A variable that theoretically should be correlated with the construct being measured (plural: criteria).

The extent to which people’s scores on a measure are correlated with other variables (known as criteria) that one would expect them to be correlated with.

The absolute value that a test statistic (e.g., F, t, etc.) must exceed to be considered statistically significant.

A statistic that measures internal consistency among items in a measure.

Means the independent variable has an effect at both levels but the effects are in opposite directions.

This is the process of informing research participants as soon as possible of the purpose of the study, revealing any deception, and correcting any other misconceptions they might have as a result of participating.

Misinforming participants about the purpose of a study, using confederates, using phony equipment like Milgram’s shock generator, and presenting participants with false feedback about their performance (e.g., telling them they did poorly on a test when they actually did well).

An ethics code that was created by the World Medical Council in 1964.

Subtle cues that reveal to participants how the researcher expects them to respond in the experiment.

The variable the experimenter measures (it is the presumed effect).

Used to compare two means for the same sample tested at two different times or under two different conditions (sometimes called the paired-samples t-test).

Refers to a set of techniques for summarizing and displaying data.

A method to reduce pairs of scores (e.g., pre- and post-test) to a single score by calculating the difference between them.

The extent to which scores on a measure of a construct are not correlated with measures of other, conceptually distinct, constructs and thus discriminate between them.

An academic degree earned through intensive study of a particular discipline and the completion of a set of research studies that contribute new knowledge to the academic literature.

A process in which the reviewers of a research article do not know the identity of the researcher(s) and vice versa.

A method to reduce experimenter bias, where neither the participant nor the experimenter is knowledgeable about the condition to which the participant is assigned.

Books that are collections of chapters written by different authors on different aspects of the same topic, and overseen by one or more editors.

Describes the strength of a statistical relationship.

These are questions about the way the world actually is and, therefore, can be answered by systematically observing it.

An article that presents the results of one or more new studies.

Research reports that describe one or more new empirical studies conducted by the authors.

A treatment that that has been shown through systematic observation to lead to better outcomes when compared to no-treatment or placebo control groups.

The branch of philosophy that is concerned with morality—what it means to behave morally and how people can achieve that goal.

Research on the effectiveness of normal educational activities, the use of standard psychological measures and surveys of a nonsensitive nature that are administered in a way that maintains confidentiality, and research using existing data from public sources.

Research reviewed by the IRB that is not anonymous and/or may involve potentially stigmatizing information, or invasive or uncomfortable procedures, but exposes participants to risks that are no greater than minimal risk (risks encountered by healthy people in daily life or during routine physical or psychological examinations).

A type of study designed specifically to answer the question of whether there is a causal relationship between two variables.

A subfield of psychology (behaviorism) that focuses exclusively on the effects of rewards, punishments, and other external factors on behavior.

Refers to the degree to which we can generalize the findings to other circumstances or settings, like the real-world environment.

Any variable other than the dependent and independent variable.

The extent to which a measurement method appears, on superficial examination, to measure the construct of interest.

A statistical method to detect differences in the means between conditions when there are two or more independent variables in a factorial design. It allows the detection of main effects and interaction effects.

Experiments that include more than one independent variable in which each level of one independent variable is combined with each level of the others to produce all possible combinations.

A scientific claim that must be expressed in such a way that there are observations that would—if they were made—count as evidence against the claim

An effect where participants perform a task worse in later conditions because they become tired or bored.

How likely is the research question going to be successfully answered depending on the amount of time, money, equipment and materials, technical knowledge and skill, and access to research participants there will be.

A set of laws based on the Belmont Report that apply to research conducted, supported, or regulated by the federal government.

A type of field study where an independent variable is manipulated in a natural setting and extraneous variables are controlled as much as possible.

A study that is conducted in a "real world" environment outside the laboratory.

Graphical depictions of data, such as pie charts, bar graphs, or scatterplots used to clearly and efficiently report a number of results.

The problem of research results not being published that fail to find a statistically significant result. As a consequence, the published literature fails to contain a full representation of the positive and negative findings about a research question.

Manuscripts that are prepared by the author in their final form and submitted for publication.

Intuitive beliefs about people’s behavior, thoughts, and feelings.

A display of each value of a variable and the number of participants with that value.

Research that poses greater than minimal risk to participants and must be reviewed by the full board of IRB members.

Hypothesizing After the Results are Known: A practice where researchers analyze data without an a priori hypothesis, claiming afterward that a statistically significant result had been originally predicted.

In the case of undisguised naturalistic observation, it is a type of reactivity when people know they are being observed and studied, they may act differently than they normally would.

Mental shortcuts in forming and maintaining our beliefs.

Guidelines in the APA Publication Manual for the clear expression of ideas, including writing that is formal, straightforward, and avoids biased language.

A graphical display of a frequency distribution.

Events outside of the pretest-posttest research design that might have influenced many or all of the participants between the pretest and the posttest.

A specific prediction about a new phenomenon that should be observed if a particular theory is accurate.

A cyclical process of theory development, starting with an observed phenomenon, then developing or using a theory to make a specific prediction of what should happen if that theory is correct, testing that prediction, refining the theory in light of the findings, and using that refined theory to develop new hypotheses, and so on.

The variable the experimenter manipulates.

Used to compare the means of two separate samples (M1 and M2).

A research method that allows researchers to draw conclusions or infer about a population based on data from a sample.

This means that researchers obtain and document people’s agreement to participate in a study after having informed them of everything that might reasonably be expected to affect their decision.

A committee that is responsible for reviewing research protocols for potential ethical problems.

A potential threat to internal validity when the basic characteristics of the measuring instrument change over the course of the study.

The extent to which different observers are consistent in their judgments.

How interesting the question is to people generally or the scientific community. Three things need to be considered: Is the answer in doubt, fills a gap in research literature, and has important practical implications.

The consistency of people’s responses across the items on a multiple-item measure.

Refers to the degree to which we can confidently infer a causal relationship between variables.

A set of measurements taken at intervals over a period of time that is "interrupted" by a treatment.

Involves taking a set of measurements at intervals over a period of time both before and after an intervention of interest in two or more nonequivalent groups.

A measurement that involves assigning scores using numerical scales in which intervals have the same interpretation throughout.

A qualitative research method to collect lengthy and detailed information from participants using structured, semi-structured, or unstructured sets of open-ended questions.

The importance of conducting research in a way that distributes risks and benefits fairly across different groups at the societal level.

A study that is conducted in the laboratory environment.

Four categories, or scales, of measurement (i.e., nominal, ordinal, interval, and ratio) that specify the types of information that a set of scores can have, and the types of statistical procedures that can be used with the scores.

Relationships between two variables whereby the points on a scatterplot fall close to a single straight line.

The effect of one independent variable on the dependent variable—averaging across the levels of any other independent variable(s).

Changing the level, or condition, of the independent variable systematically so that different groups of participants are exposed to different levels of that variable, or the same group of participants is exposed to different levels at different times.

Verifying the experimental manipulation worked by using a different measure of the construct the researcher is trying to manipulate.

An experiment design in which the participants in the various conditions are matched on the dependent variable or on some extraneous variable(s) prior the manipulation of the independent variable.

The average of a distribution of scores (symbolized M) where the sum of the scores are divided by the number of scores.

Is the assignment of scores to individuals so that the scores represent some characteristic of the individuals.

The midpoint of a distribution of scores in the sense that half the scores in the distribution are less than it and half are greater than it.

A review article that provides a statistical summary of all of the previous results.

A design which manipulates one independent variable between subjects and another within subjects.

The most frequently occurring score in a distribution.

A coherent written presentation of a topic much like an extended review article written by a single author or a small group of authors.

When the participants and the situation studied are similar to those that the researchers want to generalize to and participants encounter every day.

The condition in which participants receive no treatment whatsoever.

A measurement used for categorical variables and involves assigning scores that are category labels.

An independent variable that is measured but is non-manipulated.

Relationships between two variables in which the points on a scatterplot do not fall close to a single straight line, but often fall along a curved line.

A set of 10 ethical principles for research written in 1947 in conjunction with the Nuremberg trials of Nazi physicians accused of war crimes against prisoners in concentration camps.

Used to compare a sample mean (M) with a hypothetical population mean (μ0) that provides some interesting standard of comparison.

Where we reject the null hypothesis only if the t score for the sample is extreme in one direction that we specify before collecting the data.

A practice in which researchers openly share their research materials with other researchers in hopes of Increasing the transparency and openness of the scientific enterprise.

A definition of the variable in terms of precisely how it is to be measured.

The specification of exactly how the research question will be studied in the experiment design.

An effect that occurs when participants' responses in the various conditions are affected by the order of conditions to which they were exposed.

A measurement that involves assigning scores so that they represent the rank order of the individuals.

The variable that is being predicted by a predictor variable in a regression equation.

The probability of obtaining the sample result or a more extreme result if the null hypothesis were true.

When researchers make various decisions in the research process to increase their chance of a statistically significant result (and type I error) by arbitrarily removing outliers, selectively choosing to report dependent variables, only presenting significant results, etc. until their results yield a desirable p value.

A method of controlling extraneous variables by measuring them and including them in the statistical analysis.

This is the percentage of responses in the treatment condition that are more extreme than the most extreme response in a relevant control condition.

For any given score, the percentage of scores in the distribution that are lower than that score.

Measures that involve recording any of a wide variety of physiological processes, including heart rate and blood pressure, galvanic skin response, hormone levels, and electrical activity and blood flow in the brain.

A simulated treatment that lacks any active ingredient or element that is hypothesized to make the treatment effective, but is otherwise identical to the treatment.

Condition in which the participants receive a placebo rather than the treatment.

An effect that is due to the placebo rather than the treatment.

A large group of people about whom researchers in psychology are usually interested in drawing conclusions, and from whom the sample is drawn.

An unplanned (not hypothesized) test of which pairs of group mean scores are different from which others.

Another way to present research at a conference by using a large size board which demonstrates and summarizes the researchers study.

A one- to two-hour session that takes place in a large room at an professional conference site where dozens of research posters are presented.

Participants in one group are exposed to a treatment, a nonequivalent group is not exposed to the treatment, and then the two groups are compared.

An effect where participants perform a task better in later conditions because they have had a chance to practice it.

A way to minimize risks in a study and to identify and eliminate participants who are at high risk.

A form of validity whereby the criterion is measured at some point in the future (after the construct has been measured), to determine that the construct "predicts" the criterion.

A variable in a regression equation that is hypothesized to be related to ("predicts") the value of an outcome or criterion variable.

A persons right to decide what information about them is shared with others.

Occurs when the researcher can specify the probability that each member of the population will be selected for the sample.

A conference that ranges from small- to large-scale events where researchers in psychology share their research with each other through presentations.

Are periodicals that publish original research articles.

Is used to select a sample in which the proportion of respondents in each of various subgroups matches the proportion in the population.

A detailed description of the research—that is reviewed by an independent committee.

Refers to activities and beliefs that are claimed to be scientific by their proponents—and may appear to be scientific at first glance—but are not.

Where the same mental process is used in both the laboratory and in the real world.

A subfield of psychology concerned with the theories and techniques of psychological measurement.

A comprehensive electronic database covering thousands of professional journals and scholarly books going back more than 100 years—that for most purposes its content is synonymous with the research literature in psychology.

A quantity, such as height, that is typically measured by assigning a number to each individual.

A form of non-probability sampling in which subgroups in the sample are recruited to be proportional to those subgroups in the population.

Means using a random process to decide which participants are tested in which conditions.

A method in which the order of the conditions is randomly determined for each participant.

An experiment that researches the effectiveness of psychotherapies and medical treatments.

A measure of dispersion that measures the distance between the highest and lowest scores in a distribution.

A measurement that involves assigning scores in such a way that there is a true zero point that represents the complete absence of the quantity.

An in text citation to the work in which that idea originally appeared and a full reference to that work in the reference list.

A statistical technique that allows researchers to predict the value of one variable given another.

Refers to the statistical fact that an individual who scores extremely high or extremely low on a variable on one occasion will tend to score less extremely on the next occasion.

A decision made by researchers using null hypothesis testing which occurs when the sample relationship would be extremely unlikely.

Refers to the consistency of a measure.

All the published research in that field.

One of the Belmont report principles that emphasizes the need for participants to exercise autonomy and protection for those with reduced autonomy, often through informed consent.

When one or both variables have a limited range in the sample relative to the population, making the value of the correlation coefficient misleading.

A decision made by researchers in null hypothesis testing which occurs when the sample relationship would not be extremely unlikely.

The most basic single-subject research design in which the researcher measures the dependent variable in three phases: Baseline, before a treatment is introduced (A); after the treatment is introduced (B); and then a return to baseline after removing the treatment (A). It is often called an ABA design.

Articles that summarize previously published research on a topic and usually present new ways to organize or explain the results.

A smaller portion of the population the researcher would like to study.

Occurs when a sample is selected in such a way that it is not representative of the entire population and therefore produces inaccurate results.

A list of all the members of the population from which to select the respondents.

A graph that presents correlations between two quantitative variables, one on the x-axis and one on the y-axis. Scores are plotted at the intersection of the values on each axis.

Books written by researchers and practitioners mainly for use by other researchers and practitioners.

The systematic study of the structure and behaviour of the physical and natural world through observation and experiment.

The scientific method is a process of systematically collecting and evaluating evidence to test ideas and answer questions.

Measures in which participants report on their own thoughts, feelings, and actions.

Are a way of breaking down the interaction to figure out precisely what is going on.

A statistical procedure which uses the value of one variable to predict another. Sometimes called "linear regression."

When an experiment has one independent variable that is manipulated to produce more than two conditions.

An experiment design involving a single independent variable with two conditions.

A type of quantitative research that involves studying in detail the behavior of each of a small number of participants.

Pausing to consider alternatives and to search for evidence—especially systematically collected empirical evidence—when there is enough at stake to justify doing so.

Referred to as treatments that have substantial effects on important behaviors and that can be implemented reliably in the real-world contexts in which they occur.

When participants respond in ways that they think are socially acceptable.

A score that is derived by splitting the items into two sets and examining the relationship between the two sets of scores in order to assess the internal consistency of a measure.

Means there is an effect of one independent variable at one level of the other independent variable and there is either a weak effect or no effect of that independent variable at the other level of the other independent variable.

Correlations that are a result not of the two variables being measured, but rather because of a third, unmeasured, variable that affects both of the measured variables.

Is the average distance between the scores and the mean in a distribution.

Concerns the proper statistical treatment of data and the soundness of the researchers’ statistical conclusions.

An effect that is unlikely due to random chance and therefore likely represents a real effect in the population.

In this design the treatment is removed from the first group when it is added to the second group.

Empiricism refers to learning based on observation, and scientists learn about the natural world systematically, by carefully planning, making, recording, and analyzing observations of it.

A statistic (e.g., F, t, etc.) that is computed to compare against what is expected in the null hypothesis, and thus helps find the p value.

When researchers measure a construct that they assume to be consistent across time, then the scores they obtain should also be consistent across time.

The ability to test the hypothesis using the methods of science and the possibility to gather evidence that will disconfirm the hypothesis if it is indeed false.

A threat to internal validity that occurs when the measurement of the dependent variable during the pretest affects participants' responses at posttest.

A review article that is devoted primarily to presenting a new theory.

A qualitative research method that involves an interpretation of the data in terms of the themes a researcher has identified.

A coherent explanation or interpretation of one or more phenomena.

Accepting that there are many things that we simply do not know.

Any intervention meant to change people’s behavior for the better.

The condition in which participants receive the treatment.

Where we reject the null hypothesis if the test statistic for the sample is extreme in either direction (+/-).

A false positive in which the researcher concludes that their results are statistically significant when in reality there is no real effect in the population and the results are due to chance. In other words, rejecting the null hypothesis when it is true.

A missed opportunity in which the researcher concludes that their results are not statistically significant when in reality there is a real effect in the population and they just missed detecting it. In other words, retaining the null hypothesis when it is false.

The extent to which the scores from a measure represent the variable they are intended to.

The extent to which the scores vary around their central tendency in a distribution.

A quantity or quality that varies across people or situations.

A measurement of the average distance of scores from the mean.

This means plotting individual participants’ data, looking carefully at those plots, and making judgments about whether and to what extent the independent variable had an effect on the dependent variable.

Condition in which participants are told that they will receive the treatment but must wait until the participants in the treatment condition have already received it.

An experiment in which each participant is tested under all conditions.

Is the difference between that individual’s score and the mean of the distribution, divided by the standard deviation of the distribution. It represents the number of standard deviations the score is from the mean.