Evaluating an open textbook is similar to evaluating any textbook. To begin, let’s look at the qualities that make a good textbook, whether it’s open or not.

What are the qualities of a good textbook?

You have probably reviewed many textbooks in your career and have your own set of evaluation guidelines. Before we begin looking at the criteria that various open textbook projects are using to evaluate the textbooks in their collections, let’s review some of the qualities and structures that make up a textbook.

Pedagogical aids for textbooks

When it comes to textbooks, there is no doubt that the content of the book is important. You want to be sure that the content in the textbook aligns with the learning outcomes of your course and that the content is written at the right level for your learners.

However, a good textbook is more than a collection of facts. Textbooks also contain a number pedagogical aids, which are features or attributes of the book that help a learner understand the content being presented.

There are a number of common pedagogical aids in textbooks, including:

• Chapter objectives
• Chapter learning outcomes
• Chapter outlines
• Chapter summaries or reviews
• Checklists
• Headings and subheadings
• Bold and italicized text
• Tables of contents
• Tables of figures
• Indexes
• Focus questions and practice questions
• Case studies, vignettes, and examples of best practices
• Glossary and key terms
• Demonstrations and simulations
• Maps and timelines
• Illustrations, including photos, charts, diagrams, and figures
• Multimedia
• Pronunciation guides

Usefulness of pedagogical aids in textbooks

Of these pedagogical aids, which are the ones that students actually use and find useful? Weiten, Guadagno & Beck (1996) surveyed 134 students asking them how familiar they were with the different pedagogical aids in their textbooks.^[1] They also asked the students the probability they would use each pedagogical aid and their perceived value of each aid. With their research, Weiten, Guadagno, and Beck showed that the top three pedagogical aids that students used in textbooks were boldfaced technical terms, chapter and section summaries, and glossaries.

In his 2003 research Pedagogical Aids and Student Performance, Gurung surveyed more than 200 undergraduate students and asked them to rate the usefulness of 10 pedagogical aids and instructional techniques. ^[2]Gurung’s research wasn’t specific to textbook aids, but did include a number of textbook-specific aids, like outlines, chapter summaries and reviews, boldfaced and italicized terms, and key terms and practice questions found in textbooks. When you eliminate the non-textbook–related aids from Gurung’s research, we find that the top textbook aids used by students were boldfaced terms, italicized terms, and practice questions. Chapter summaries and reviews was a very close fourth.

In terms of helpfulness, students rated boldfaced (92%) and italicized (81%) terms as the most useful pedagogical aids, followed by practice test questions (77%) and chapter summaries and reviews (73%) as being moderately to extremely helpful.

So, as we can see, it is often the simplest aids, like bold and italicized terms and chapter summaries and outlines, that are most often used by students.

Textbook structure

Another consideration when reviewing a textbook is to take a close look at the way the content is structured. A good textbook should follow some basic rules of structure.

Open textbook–specific considerations

So far, we have been looking at textbooks in general and not specifically open textbooks. For the most part, the criteria you would use to judge the effectiveness of an open textbook would be the same as any textbook. However, there are some unique criteria that you might want to consider for open textbooks that may not apply to a closed textbook. Some of the additional items you may want to consider when evaluating an open textbook are:

• Can I modify or edit the book to meet my needs, and if so, does the licence allow me edit or modify the book?
• Is the open textbook available in a technical format that I can edit, if I want to modify it?
• Is there supplementary material available for the textbook? For some faculty, supplementary resources are important components of a textbook adoption. While some open textbooks do have supplementary materials associated with them, many do not. This may or may not be an issue for you.
• Is there a community to support the open textbook? Often, these communities can help provide support for you in your adoption. If you have a problem or concern, or are just looking to connect with others who are using the same resource, a well-developed community can help provide support for your adoption. For example, Stitz-Zeager’s open pre-calculus textbook website has a community with discussion forums and blogs that can help provide some support for your adoption. The Writing Commons is another example of an open textbook with a community associated with it.

Open textbook review criteria

Increasingly, open textbook projects are placing a greater emphasis on having peer-reviewed materials in their collection to help faculty with adoption and address concerns some have about the quality of open textbooks. A number of open textbook projects have created criteria for evaluating and reviewing open textbooks and OER collections.

• BCcampus open textbooks review rubric [Word file]
• Faculty checklist for evaluating course materials
• Open education resource repository rubric

Long Descriptions

5 rules of textbook development long description: An infographic of the 5 rules of textbook development, which are as follows:

1. Rule of Frameworks
   • Maintain a consistent structure. The text can best aid understanding by making this framework visible early on. The framework acts as a mental roadmap that allows learners to navigate within and through the subject domain.
2. Rule of Meaningful Names
   • Create and use consistent titles and terminologies. These names are critical to the ability to recall or retrieve the things we know and remember. Use terminology that is common in your discipline.
3. Rule of Manageable Numbers
   • Limit the amount of new information introduced at one time. Most of us are limited in our ability to absorb new material. As we become familiar with part of a subject domain, this number expands. For new material, four to six new elements is a reasonable limit.
4. Rule of Hierarchy
   • New knowledge builds on learned knowledge. The student needs to understand the foundational knowledge before being introduced to a new concept. When new concepts are introduced, they should be explicitly connected to the foundational material. When introducing new material, only refer to foundational material if it is relevant to the new material.
5. Rule of Repetition
   • Repeat important concepts. There is a pattern of repetition that aids in promoting the elements of a subject from short-term to long-term memory. Frameworks and important hierarchies are repeated as many as five or six times. Frequently used elements are repeated three or four times. Elements of lesser utility may not be repeated at all.

[Return to 5 rules of textbook development]

Attributions

5 rules of textbook development

This work is used under a CC BY licence.

This work is based upon Wikibooks:Textbook considerations by various authors and released under a CC BY-SA 2.0 Canadian Licence.

Outer surface of the human brain is in the public domain.

Brain Anatomy by OpenClips is in the public domain.