29 Evaluating Quality
Evaluating your sources is critical to the process of academic research. One common tool is called CRAAP. Yes, you read that correctly! The CRAAP test is an evaluation tool that helps you determine a source’s quality and provides you with a way to analyze your sources and determine if they are appropriate for your research. You will come across various versions of CRAAP, but for the purposes of this chapter, the acronym stands for:
C: Currency
R: Relevance
A: Authority
A: Accuracy
P: Purpose/point of view
Figure 3.7: Dissecting CRAAP
Time to dissect CRAAP a bit more!
The CRAAP test uses a series of questions that address specific evaluation criteria like the authority and purpose of the source. This test should be used for all of your sources. It is not intended to make you exclude any sources, but to help you analyze how to use them to support your own arguments. You will find that some sources are of such low quality that it wouldn’t be helpful to your argument if you used them. See Table 3.2. for an overview of the CRAAP test.
Table 3.2: Overview of the CRAAP test
| Criteria | Relation to nursing |
|---|---|
|
C = Currency: The timeliness of the information. When was the information published or posted? Based on this date, will the material accurately reflect your topic? Has the information been revised or updated? Does your topic require current information, or will older sources work as well? |
What is considered current and how current does a resource have to be? This is a complex question. Some say a current text was written within five years, and some say within ten years. If currency is not noted in your assignment guidelines, ask your instructor. Currency will vary based on the topic. There are evolutions in knowledge that inform thinking and practices based on the most current research. For example, if you are exploring cannabis use in Canada, the currency of this information is important because so much has changed in the last few years. For this topic, you would probably search for texts produced within the last one or two years to ensure currency. Sometimes, you’ll also need to consider seminal literature. This type of literature refers to a pivotal study or theoretical article that is foundational to a discipline. How do you know if a text is seminal? One clue is when you see the same article cited in the literature over and over again. Frequency of the citation usually indicates that an article has had a major influence. For example, Dr. Heather Laschinger’s nursing work on empowerment from the 1990s and early 2000s has been cited thousands of times. Although this work is older, it is considered seminal in this topic. To find out how many times an article has been cited, search the article title in Google Scholar – this website will indicate how many times the article has been cited. See Figure 3.8 as an example. Determining whether literature is seminal takes experience and becoming familiar with a topic. Early in your career, you may need to consult your instructors to help you determine whether literature is seminal. |
|
R = Relevance: The importance of the information for your needs. Does the information relate to your topic or answer your question? Who is the intended audience? Is the information at an appropriate level (i.e., not too elementary or advanced for your needs or audience)? Have you looked at a variety of sources before determining the ones that you will use?
|
You should consider various factors when considering the relevance of your sources. Does the source align with the topic that you are studying? For example, if you are studying parents’ grieving experiences after their child dies, your sources need to focus on this topic. A source that documents nurses’ grieving experiences after a client dies is not relevant, as the relationships are quite different. Also, if you are exploring the nurse’s role in medical assistance in dying, for example, it is important to consider legislation in the related country and province/territory/state. For example, the nurse’s role in Ontario is different than other locations. You will review sources from a wide variety of locations and written for a wide variety of audiences. You need to consider whether your reader will believe that the sources you have chosen are relevant. |
|
A = Authority: The source of the information. Is the author qualified to write on the topic? Do you trust the author? If applicable, what are the author’s credentials or organizational affiliations? Does the URL reveal anything about the author or source? For example: .com .edu .gov .org .net Is the journal or textbook publisher reputable or well known? |
Often, an author with one or more degrees is accepted as being an authority. For example, does the author have a Master of Nursing and/or a PhD in Nursing or a related field? Does the author have experience and/or expertise in the area that they are writing about? Have they published multiple articles in this area? Are they from a reputable university, college, hospital, or healthcare organization? Try Googling the author to find out where they work and their publication record. Depending on your topic, you might also draw upon sources in which the expert is the client or the family. This shift has recently become more common, particularly in Canada, where patients are encouraged to collaborate in health research – this is sometimes required when applying for research funding. There has also been a shift toward incorporating narratives, so patients, families, and students make important contributions to various bodies of literature. It is also important to examine whether the publisher and/or journal is reputable, particularly in this era of open access publishing, where access to many journals is free – but the quality of these journals is sometimes questionable. Open access can be of great benefit to students who don’t have access to an institutional library database. However, the risk is that many predatory journals are emerging – they may be focused on making money. As a result, the quality of articles published in these journals may be lacking. You need to make sure that the article that you are reading is from a reputable journal. How do you figure this out? Start by Googling the journal name. Find its homepage and check whether the editor and editorial board members are from reputable institutions and whether the journal has a peer-review process (peer review is discussed later in this chapter). You can also check out websites such as: Beall’s List of Predatory Journals and Publishers. Your university librarians can also help you figure out whether the article is from a reputable journal. |
|
A= Accuracy: The reliability, truthfulness, and correctness of the content. Where does the information come from? Is the information supported by evidence? Has the information been reviewed or refereed? Can you verify any of the information in another source or from personal knowledge? Does the language or tone seem unbiased and free of emotion? Are there spelling, grammar, or typographical errors? |
You must evaluate and ensure that your researched sources are both reliable and truthful. Again, assess whether the article underwent peer review. Usually, the “About” or “Journal Info” section of the homepage will specify whether the journal has a peer-review process. Unfortunately, some predatory journals and publishers are getting smart and falsely claiming to use a peer-review process. Check out the editorial board members. Are they real people from real institutions? Check out the authors’ citations if the article refers to claims or facts that seem questionable. Also, if the article presents the results of a study, you will need to evaluate the research methods. You will develop the skills to evaluate research methods in a research course as part of your nursing program. |
|
P = Purpose/Point of view: The reason the information exists. What is the purpose of the information? Is it to inform, teach, sell, entertain, or persuade? Do the authors/sponsors make their intentions or purpose clear? Do they identify the overall goal, research aim, or paper’s objectives? Is the information fact, opinion, or propaganda? Does the point of view appear objective and impartial? Are there political, ideological, cultural, religious, institutional, or personal biases? For example, are they trying push their own agenda? Do they have a bias? Are they trying to convince you to buy something? |
These are all important questions to consider when you are evaluating the sources that you plan to use when writing a paper. Your source’s purpose should be clear and informative. You should avoid incorporating sources that have any kind of underlying bias. Avoid journal sites with click-bait . Many of you will be familiar with this from social media. Usually, these types of sites are trying sell something and/or push an agenda related to a particular topic. |
Figure 3.8: Search on how many times an article has been cited
Checklist Source Evaluation
Here is a checklist that you can use to evaluate your sources:
- Have you applied the CRAAP test to your sources?
- Is the type of source appropriate for your purpose? Is it a high-quality source or one that needs to be looked at more critically?
- Can you establish that the author is credible and the publication is reputable?
- Does the author support ideas with specific facts and details that are carefully documented? Is the source of the author’s information clear? (When you use secondary sources, look for sources that are not too removed from primary research.)
- Does the source include any factual errors or instances of faulty logic?
- Does the author leave out any information that you would expect to see in a discussion of this topic?
- Do the author’s conclusions logically follow from the evidence that is presented? Can you see how the author got from one point to another?
- Is the writing clear and organized, and is it free from errors, clichés, and empty buzzwords? Is the tone objective, balanced, and reasonable? (Be on the lookout for extreme, emotionally charged language.)
- Are there any obvious biases or agendas? Based on what you know about the author, are there likely to be any hidden agendas?
- Are graphics informative, useful, and easy to understand? Are websites organized, easy to navigate, and free of clutter like flashing ads and unnecessary sound effects?
- Is the source contradicted by information found in other sources? (If so, it is possible that your sources are presenting similar information but taking different perspectives, which requires you to think carefully about which sources you find more convincing and why. Be suspicious, however, of any source that presents facts that you cannot confirm elsewhere.)
Activities: Check Your Understanding
Attribution statement
The activities were created by the authors.
Unless otherwise noted, all other content was remixed with our own original content, and adapted from:
Write Here, Right Now by Dr. Paul Chafe, Aaron Tucker with chapters from Dr. Kari Maaren, Dr. Martha Adante, Val Lem, Trina Grover and Kelly Dermody, under a Creative Commons Attribution 4.0 International License. Download this book for free at: https://pressbooks.library.ryerson.ca/writehere/
The checklist course evaluation was adapted from:
Writing for Success 1st Canadian Edition by Tara Horkoff is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License, except where otherwise noted. Download for free at: https://opentextbc.ca/writingforsuccess/
Evaluation tool that helps determine a source’s quality.
Source in which the purpose is to attract your attention and prompt you to click on links to web pages that might be selling something or promoting fake news.
