α (alpha)

The criterion that shows how low a p-value should be before the sample result is considered unlikely enough to reject the null hypothesis (Usually set to .05).

ABA design

Another term for reversal design.


A brief summary of the study's research question, methods, results and conclusions.

Analysis of variance (ANOVA)

A statistical test used when there are more than two groups or condition means to be compared.


When a participants name and other personally identifiable information is not collected at all.

APA Ethics Code

Stands for the APA’s Ethical Principles of Psychologists and Code of Conduct. It was first published in 1953 and includes about 150 specific ethical standards that psychologists and their students are expected to follow.

Applied behavior analysis

An application of the principles of experimental analysis of behavior that plays an important role in contemporary research on developmental disabilities, education, organizational behavior, and health, among many other applied areas.

Applied research

Research conducted primarily to address some practical problem.


A persons right to make their own choices and take their own actions free from coercion.

Basic research

Research conducted primarily for the sake of achieving a more detailed and accurate understanding of human behavior, without necessarily trying to address any particular practical problem.

Behavioral measures

Measures in which some other aspect of participants’ behavior is observed and recorded.

Belmont Report

A set of federal guidelines written in 1978 as a response to the abuses of the Tuskegee study that recognize three important principles in research with humans: justice, respect for persons, and beneficience, and that formed the basis for federal regulations applied to research.


Underscores the importance of maximizing the benefits of research while minimizing harms to participants and society.

Between-subjects experiment

An experiment in which each participant is tested in only one condition.

Between-subjects factorial design

All of the independent variables are manipulated between subjects.

Block randomization

All the conditions occur once in the sequence before any of them is repeated.

Carryover effect

An effect of being tested in one condition on participants’ behavior in later conditions.

Categorical variable

A variable that represents a characteristic of an individual, such as chosen major, and is typically measured by assigning each individual's response to one of several categories (e.g., Psychology, English, Nursing, Engineering, etc.).

Central tendency

Is the middle of a distribution—the point around which the scores in the distribution tend to cluster. (Another term for central tendency is average.)

Clinical practice of psychology

The diagnosis and treatment of psychological disorders and related problems.

Closed-ended items

Questionnaire items that ask a question and provide a limited set of response options for participants to choose from.


A part of structured observation whereby the observers use a clearly defined set of guidelines to "code" behaviors—assigning specific behaviors they are observing to a category—and count the number of times or the duration that the behavior occurs.

Cohen’s d

The most widely used measure of effect size for differences between group or condition means, which is the difference between the two means divided by the standard deviation.

Complete counterbalancing

A method in which an equal number of participants complete each possible order of conditions. 

Conceptual definition

Describes the behaviors and internal processes that make up a psychological construct, along with how it relates to other variables.

Concurrent validity

A form of criterion validity, where the criterion is measured at the same time (concurrently) as the construct.


The different levels of the independent variable to which participants are assigned.


A helper who pretended to be a real participant in a study.

Confidence intervals

A range of values that is computed in such a way that some percentage of the time (usually 95%) the population parameter will lie within that range.


An agreement not to disclose participants’ personal information without their consent or some appropriate legal authorization.

Confirmation bias

Tendency to focus on cases that confirm our intuitive beliefs and to disregard cases that disconfirm our beliefs.

Confounding variable

An extraneous variable that varies systematically with the independent variable, and thus confuses the effect of the independent variable with the effect of the extraneous one.


A specific type of extraneous variable that systematically varies along with the variables under investigation and therefore provides an alternative explanation for the results.

Consent form

The process of obtaining informed consent by having the participants read and sign the form.

Construct validity

One of the "big four" validities, whereby the research question is clearly operationalized by the study's methods.


Psychological variables that represent an individual's mental state or experience, often not directly observable, such as personality traits, emotional states, attitudes, and abilities.

Content analysis

A family of systematic approaches to measurement using qualitative methods to analyze complex archival data.

Content validity

The extent to which a measure reflects all aspects of the construct of interest.

Context effect (or contrast effect)

Unintended influences on respondents’ answers because they are not related to the content of the item but to the context in which the item appears.


Holding extraneous variables constant in order to separate the effect of the independent variable from the effect of the extraneous variables.

Control condition

The condition in which participants do not receive the treatment.

Convenience sampling

A common method of non-probability sampling in which the sample consists of individuals who happen to be easily available and willing to participate (such as introductory psychology students).

Convergent validity

A form of criterion validity whereby new measures are correlated with existing established measures of the same construct.

Converging operations

When psychologists use multiple operational definitions of the same construct—either within a study or across studies.

Correlation coefficient

Describes the strength and direction of the relationship between two variables (often measured by Pearson's r).

Correlation matrix

Shows the correlation coefficient between pairs of variables in the study.


Varying the order of the conditions in which participants are tested, to help solve the problem of order effects in within-subjects experiments.


A variable that theoretically should be correlated with the construct being measured (plural: criteria).

Criterion validity

The extent to which people’s scores on a measure are correlated with other variables (known as criteria) that one would expect them to be correlated with.

Critical value

The absolute value that a test statistic (e.g., F, t, etc.) must exceed to be considered statistically significant.

Cronbach’s α

A statistic that measures internal consistency among items in a measure.

Cross-over interaction

Means the independent variable has an effect at both levels but the effects are in opposite directions.


This is the process of informing research participants as soon as possible of the purpose of the study, revealing any deception, and correcting any other misconceptions they might have as a result of participating.


Misinforming participants about the purpose of a study, using confederates, using phony equipment like Milgram’s shock generator, and presenting participants with false feedback about their performance (e.g., telling them they did poorly on a test when they actually did well).

Declaration of Helsinki

An ethics code that was created by the World Medical Council in 1964.

Demand characteristics

Subtle cues that reveal to participants how the researcher expects them to respond in the experiment.

Dependent variable

The variable the experimenter measures (it is the presumed effect).

Dependent-samples t-test

Used to compare two means for the same sample tested at two different times or under two different conditions (sometimes called the paired-samples t-test).

Descriptive statistics

Refers to a set of techniques for summarizing and displaying data.

Difference score

A method to reduce pairs of scores (e.g., pre- and post-test) to a single score by calculating the difference between them.

Discriminant validity

The extent to which scores on a measure of a construct are not correlated with measures of other, conceptually distinct, constructs and thus discriminate between them.

Doctor of philosophy [Ph.D.]

An academic degree earned through intensive study of a particular discipline and the completion of a set of research studies that contribute new knowledge to the academic literature.

Double-blind peer review

A process in which the reviewers of a research article do not know the identity of the researcher(s) and vice versa.

Double-blind study

A method to reduce experimenter bias, where neither the participant nor the experimenter is knowledgeable about the condition to which the participant is assigned.

Edited volumes

Books that are collections of chapters written by different authors on different aspects of the same topic, and overseen by one or more editors.

Effect size

Describes the strength of a statistical relationship.

Empirical questions

These are questions about the way the world actually is and, therefore, can be answered by systematically observing it.

Empirical research report

An article that presents the results of one or more new studies.

Empirical research reports

Research reports that describe one or more new empirical studies conducted by the authors.

Empirically supported treatments

A treatment that that has been shown through systematic observation to lead to better outcomes when compared to no-treatment or placebo control groups.


The branch of philosophy that is concerned with morality—what it means to behave morally and how people can achieve that goal.

Exempt research

Research on the effectiveness of normal educational activities, the use of standard psychological measures and surveys of a nonsensitive nature that are administered in a way that maintains confidentiality, and research using existing data from public sources.

Expedited research

Research reviewed by the IRB that is not anonymous and/or may involve potentially stigmatizing information, or invasive or uncomfortable procedures, but exposes participants to risks that are no greater than minimal risk (risks encountered by healthy people in daily life or during routine physical or psychological examinations).


A type of study designed specifically to answer the question of whether there is a causal relationship between two variables.

Experimental analysis of behavior

A subfield of psychology (behaviorism) that focuses exclusively on the effects of rewards, punishments, and other external factors on behavior.

External validity

Refers to the degree to which we can generalize the findings to other circumstances or settings, like the real-world environment.

Extraneous variables

Any variable other than the dependent and independent variable.

Face validity

The extent to which a measurement method appears, on superficial examination, to measure the construct of interest.

Factorial ANOVA

A statistical method to detect differences in the means between conditions when there are two or more independent variables in a factorial design. It allows the detection of main effects and interaction effects.

Factorial designs

Experiments that include more than one independent variable in which each level of one independent variable is combined with each level of the others to produce all possible combinations.


A scientific claim that must be expressed in such a way that there are observations that would—if they were made—count as evidence against the claim

Fatigue effect

An effect where participants perform a task worse in later conditions because they become tired or bored.


How likely is the research question going to be successfully answered depending on the amount of time, money, equipment and materials, technical knowledge and skill, and access to research participants there will be.

Federal Policy for the Protection of Human Subjects

A set of laws based on the Belmont Report that apply to research conducted, supported, or regulated by the federal government.

Field experiment

A type of field study where an independent variable is manipulated in a natural setting and extraneous variables are controlled as much as possible.

Field study

A study that is conducted in a "real world" environment outside the laboratory.


Graphical depictions of data, such as pie charts, bar graphs, or scatterplots used to clearly and efficiently report a number of results.

File drawer problem

The problem of research results not being published that fail to find a statistically significant result. As a consequence, the published literature fails to contain a full representation of the positive and negative findings about a research question.

Final manuscripts

Manuscripts that are prepared by the author in their final form and submitted for publication.

Folk psychology

Intuitive beliefs about people’s behavior, thoughts, and feelings.

Frequency table

A display of each value of a variable and the number of participants with that value.

Greater than minimal risk research

Research that poses greater than minimal risk to participants and must be reviewed by the full board of IRB members.


Hypothesizing After the Results are Known: A practice where researchers analyze data without an a priori hypothesis, claiming afterward that a statistically significant result had been originally predicted.

Hawthorne effect

In the case of undisguised naturalistic observation, it is a type of reactivity when people know they are being observed and studied, they may act differently than they normally would.


Mental shortcuts in forming and maintaining our beliefs.

High-level style

Guidelines in the APA Publication Manual for the clear expression of ideas, including writing that is formal, straightforward, and avoids biased language.


A graphical display of a frequency distribution.


Events outside of the pretest-posttest research design that might have influenced many or all of the participants between the pretest and the posttest.


A specific prediction about a new phenomenon that should be observed if a particular theory is accurate.

Hypothetico-deductive method

A cyclical process of theory development, starting with an observed phenomenon, then developing or using a theory to make a specific prediction of what should happen if that theory is correct, testing that prediction, refining the theory in light of the findings, and using that refined theory to develop new hypotheses, and so on.

Independent variable

The variable the experimenter manipulates.

Independent-samples t-test

Used to compare the means of two separate samples (M1 and M2).

Inferential statistics

A research method that allows researchers to draw conclusions or infer about a population based on data from a sample.

Informed consent

This means that researchers obtain and document people’s agreement to participate in a study after having informed them of everything that might reasonably be expected to affect their decision.

Institutional review board (IRB)

A committee that is responsible for reviewing research protocols for potential ethical problems.


A potential threat to internal validity when the basic characteristics of the measuring instrument change over the course of the study.

Inter-rater reliability

The extent to which different observers are consistent in their judgments.


How interesting the question is to people generally or the scientific community. Three things need to be considered: Is the answer in doubt, fills a gap in research literature, and has important practical implications.

Internal consistency

The consistency of people’s responses across the items on a multiple-item measure.

Internal validity

Refers to the degree to which we can confidently infer a causal relationship between variables.

Interrupted time-series design

A set of measurements taken at intervals over a period of time that is "interrupted" by a treatment.

Interrupted time-series design with nonequivalent group

Involves taking a set of measurements at intervals over a period of time both before and after an intervention of interest in two or more nonequivalent groups.

Interval level

A measurement that involves assigning scores using numerical scales in which intervals have the same interpretation throughout.


A qualitative research method to collect lengthy and detailed information from participants using structured, semi-structured, or unstructured sets of open-ended questions.


The importance of conducting research in a way that distributes risks and benefits fairly across different groups at the societal level.

Laboratory study

A study that is conducted in the laboratory environment.

Levels of measurement

Four categories, or scales, of measurement (i.e., nominal, ordinal, interval, and ratio) that specify the types of information that a set of scores can have, and the types of statistical procedures that can be used with the scores.

Linear relationships

Relationships between two variables whereby the points on a scatterplot fall close to a single straight line.

Main effect

The effect of one independent variable on the dependent variable—averaging across the levels of any other independent variable(s).


Changing the level, or condition, of the independent variable systematically so that different groups of participants are exposed to different levels of that variable, or the same group of participants is exposed to different levels at different times.

Manipulation check

Verifying the experimental manipulation worked by using a different measure of the construct the researcher is trying to manipulate.

Matched-groups design

An experiment design in which the participants in the various conditions are matched on the dependent variable or on some extraneous variable(s) prior the manipulation of the independent variable.


The average of a distribution of scores (symbolized M) where the sum of the scores are divided by the number of scores.


Is the assignment of scores to individuals so that the scores represent some characteristic of the individuals.


The midpoint of a distribution of scores in the sense that half the scores in the distribution are less than it and half are greater than it.


A review article that provides a statistical summary of all of the previous results.

Mixed factorial design

A design which manipulates one independent variable between subjects and another within subjects.


The most frequently occurring score in a distribution.


A coherent written presentation of a topic much like an extended review article written by a single author or a small group of authors.

Mundane realism

When the participants and the situation studied are similar to those that the researchers want to generalize to and participants encounter every day.

No-treatment control condition

The condition in which participants receive no treatment whatsoever.

Nominal level

A measurement used for categorical variables and involves assigning scores that are category labels.

Non-manipulated independent variable

An independent variable that is measured but is non-manipulated.

Nonlinear relationships

Relationships between two variables in which the points on a scatterplot do not fall close to a single straight line, but often fall along a curved line.

Nuremberg Code

A set of 10 ethical principles for research written in 1947 in conjunction with the Nuremberg trials of Nazi physicians accused of war crimes against prisoners in concentration camps.

One-sample t-test

Used to compare a sample mean (M) with a hypothetical population mean (μ0) that provides some interesting standard of comparison.

One-tailed test

Where we reject the null hypothesis only if the t score for the sample is extreme in one direction that we specify before collecting the data.

Open science practices

A practice in which researchers openly share their research materials with other researchers in hopes of Increasing the transparency and openness of the scientific enterprise.

Operational definition

A definition of the variable in terms of precisely how it is to be measured.


The specification of exactly how the research question will be studied in the experiment design.

Order effect

An effect that occurs when participants' responses in the various conditions are affected by the order of conditions to which they were exposed.

Ordinal level

A measurement that involves assigning scores so that they represent the rank order of the individuals.

Outcome variable or Criterion variable

The variable that is being predicted by a predictor variable in a regression equation.

p value

The probability of obtaining the sample result or a more extreme result if the null hypothesis were true.


When researchers make various decisions in the research process to increase their chance of a statistically significant result (and type I error) by arbitrarily removing outliers, selectively choosing to report dependent variables, only presenting significant results, etc. until their results yield a desirable p value.

Partial correlation

A method of controlling extraneous variables by measuring them and including them in the statistical analysis.

Percentage of non-overlapping data

This is the percentage of responses in the treatment condition that are more extreme than the most extreme response in a relevant control condition.

Percentile rank

For any given score, the percentage of scores in the distribution that are lower than that score.

Physiological measures

Measures that involve recording any of a wide variety of physiological processes, including heart rate and blood pressure, galvanic skin response, hormone levels, and electrical activity and blood flow in the brain.


A simulated treatment that lacks any active ingredient or element that is hypothesized to make the treatment effective, but is otherwise identical to the treatment.

Placebo control condition

Condition in which the participants receive a placebo rather than the treatment.

Placebo effect

An effect that is due to the placebo rather than the treatment.


A large group of people about whom researchers in psychology are usually interested in drawing conclusions, and from whom the sample is drawn.

Post hoc comparisons

An unplanned (not hypothesized) test of which pairs of group mean scores are different from which others.


Another way to present research at a conference by using a large size board which demonstrates and summarizes the researchers study.

Poster session

A one- to two-hour session that takes place in a large room at an professional conference site where dozens of research posters are presented.

Posttest only nonequivalent groups design

Participants in one group are exposed to a treatment, a nonequivalent group is not exposed to the treatment, and then the two groups are compared.

Practice effect

An effect where participants perform a task better in later conditions because they have had a chance to practice it.


A way to minimize risks in a study and to identify and eliminate participants who are at high risk.

Predictive validity

A form of validity whereby the criterion is measured at some point in the future (after the construct has been measured), to determine that the construct "predicts" the criterion.

Predictor variable

A variable in a regression equation that is hypothesized to be related to ("predicts") the value of an outcome or criterion variable.


A persons right to decide what information about them is shared with others.

Probability sampling

Occurs when the researcher can specify the probability that each member of the population will be selected for the sample.

Professional conferences

A conference that ranges from small- to large-scale events where researchers in psychology share their research with each other through presentations.

Professional journals

Are periodicals that publish original research articles.

Proportionate stratified random sampling

Is used to select a sample in which the proportion of respondents in each of various subgroups matches the proportion in the population.


A detailed description of the research—that is reviewed by an independent committee.


Refers to activities and beliefs that are claimed to be scientific by their proponents—and may appear to be scientific at first glance—but are not.

Psychological realism

Where the same mental process is used in both the laboratory and in the real world.


A subfield of psychology concerned with the theories and techniques of psychological measurement.


A comprehensive electronic database covering thousands of professional journals and scholarly books going back more than 100 years—that for most purposes its content is synonymous with the research literature in psychology.

Quantitative variable

A quantity, such as height, that is typically measured by assigning a number to each individual.

Quota sampling

A form of non-probability sampling in which subgroups in the sample are recruited to be proportional to those subgroups in the population.

Random assignment

Means using a random process to decide which participants are tested in which conditions.

Random counterbalancing 

A method in which the order of the conditions is randomly determined for each participant.

Randomized clinical trial

An experiment that researches the effectiveness of psychotherapies and medical treatments.


A measure of dispersion that measures the distance between the highest and lowest scores in a distribution.

Ratio level

A measurement that involves assigning scores in such a way that there is a true zero point that represents the complete absence of the quantity.

Reference citation

An in text citation to the work in which that idea originally appeared and a full reference to that work in the reference list.


A statistical technique that allows researchers to predict the value of one variable given another.

Regression to the mean

Refers to the statistical fact that an individual who scores extremely high or extremely low on a variable on one occasion will tend to score less extremely on the next occasion.

Reject the null hypothesis

A decision made by researchers using null hypothesis testing which occurs when the sample relationship would be extremely unlikely.


Refers to the consistency of a measure.

Research literature

All the published research in that field.

Respect for persons

One of the Belmont report principles that emphasizes the need for participants to exercise autonomy and protection for those with reduced autonomy, often through informed consent.

Restriction of Range

When one or both variables have a limited range in the sample relative to the population, making the value of the correlation coefficient misleading.

Retain the null hypothesis

A decision made by researchers in null hypothesis testing which occurs when the sample relationship would not be extremely unlikely.

Reversal design

The most basic single-subject research design in which the researcher measures the dependent variable in three phases: Baseline, before a treatment is introduced (A); after the treatment is introduced (B); and then a return to baseline after removing the treatment (A). It is often called an ABA design.

Review articles

Articles that summarize previously published research on a topic and usually present new ways to organize or explain the results.


A smaller portion of the population the researcher would like to study.

Sampling bias

Occurs when a sample is selected in such a way that it is not representative of the entire population and therefore produces inaccurate results.

Sampling frame

A list of all the members of the population from which to select the respondents.


A graph that presents correlations between two quantitative variables, one on the x-axis and one on the y-axis. Scores are plotted at the intersection of the values on each axis.

Scholarly books

Books written by researchers and practitioners mainly for use by other researchers and practitioners.


The systematic study of the structure and behaviour of the physical and natural world through observation and experiment.

Scientific Method

The scientific method is a process of systematically collecting and evaluating evidence to test ideas and answer questions.

Self-report measures

Measures in which participants report on their own thoughts, feelings, and actions.

Simple effects

Are a way of breaking down the interaction to figure out precisely what is going on.

Simple regression

A statistical procedure which uses the value of one variable to predict another. Sometimes called "linear regression."

Single factor multi level design

When an experiment has one independent variable that is manipulated to produce more than two conditions.

Single factor two-level design

An experiment design involving a single independent variable with two conditions.

Single-subject research

A type of quantitative research that involves studying in detail the behavior of each of a small number of participants.


Pausing to consider alternatives and to search for evidence—especially systematically collected empirical evidence—when there is enough at stake to justify doing so.

Social validity

Referred to as treatments that have substantial effects on important behaviors and that can be implemented reliably in the real-world contexts in which they occur.

Socially desirable responding

When participants respond in ways that they think are socially acceptable.

Split-half correlation

A score that is derived by splitting the items into two sets and examining the relationship between the two sets of scores in order to assess the internal consistency of a measure.

Spreading interactions

Means there is an effect of one independent variable at one level of the other independent variable and there is either a weak effect or no effect of that independent variable at the other level of the other independent variable.

Spurious correlations

Correlations that are a result not of the two variables being measured, but rather because of a third, unmeasured, variable that affects both of the measured variables.

Standard deviation

Is the average distance between the scores and the mean in a distribution.

Statistical validity

Concerns the proper statistical treatment of data and the soundness of the researchers’ statistical conclusions.

Statistically significant

An effect that is unlikely due to random chance and therefore likely represents a real effect in the population.

Switching replication with treatment removal design

In this design the treatment is removed from the first group when it is added to the second group.

Systematic empiricism

Empiricism refers to learning based on observation, and scientists learn about the natural world systematically, by carefully planning, making, recording, and analyzing observations of it.

Test statistic

A statistic (e.g., F, t, etc.) that is computed to compare against what is expected in the null hypothesis, and thus helps find the p value.

Test-retest reliability

When researchers measure a construct that they assume to be consistent across time, then the scores they obtain should also be consistent across time.

Testable and falsifiable

The ability to test the hypothesis using the methods of science and the possibility to gather evidence that will disconfirm the hypothesis if it is indeed false.


A threat to internal validity that occurs when the measurement of the dependent variable during the pretest affects participants' responses at posttest.

Theoretical article

A review article that is devoted primarily to presenting a new theory.

Theoretical narrative

A qualitative research method that involves an interpretation of the data in terms of the themes a researcher has identified.


A coherent explanation or interpretation of one or more phenomena.

Tolerance for uncertainty

Accepting that there are many things that we simply do not know.


Any intervention meant to change people’s behavior for the better.

Treatment condition

The condition in which participants receive the treatment.

Two-tailed test

Where we reject the null hypothesis if the test statistic for the sample is extreme in either direction (+/-).

Type I error

A false positive in which the researcher concludes that their results are statistically significant when in reality there is no real effect in the population and the results are due to chance. In other words, rejecting the null hypothesis when it is true.

Type II error

A missed opportunity in which the researcher concludes that their results are not statistically significant when in reality there is a real effect in the population and they just missed detecting it. In other words, retaining the null hypothesis when it is false.


The extent to which the scores from a measure represent the variable they are intended to.


The extent to which the scores vary around their central tendency in a distribution.


A quantity or quality that varies across people or situations.


A measurement of the average distance of scores from the mean.

Visual inspection

This means plotting individual participants’ data, looking carefully at those plots, and making judgments about whether and to what extent the independent variable had an effect on the dependent variable.

Wait-list control condition

Condition in which participants are told that they will receive the treatment but must wait until the participants in the treatment condition have already received it.

Within-subjects experiment

An experiment in which each participant is tested under all conditions.

Z score

Is the difference between that individual’s score and the mean of the distribution, divided by the standard deviation of the distribution. It represents the number of standard deviations the score is from the mean.


Icon for the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License

Research Methods in Psychology by Rajiv S. Jhangiani, I-Chant A. Chiang, Carrie Cuttler, & Dana C. Leighton is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License, except where otherwise noted.

Share This Book