The Making of Meaningful & Measurable Outcomes: Gathering Credible Evidence of Student Learning

Round Table Discussion at the Global Conference on Education & Research

University of South Florida, Sarasota, FL

Vicki Caruana, Ph.D.

Chris Caruana, M.Ed.

May 23, 2019

Learning outcomes and objectives are the first step in backwards design. As such they hold the primary spot in the course development process. Faculty engaged in course or program development face the prospect of a poorly designed course and less than meaningful learning experiences for students if they are ill-equipped to design meaningful and measurable outcomes. In addition, when outcomes and objectives are mandated by an institution and those outcomes are not meaningful and measurable, faculty can address this lack by designing module objectives that are meaningful and measurable. The process of developing learning outcomes that actively engage learners in a way that clearly describe what learners will learn and be able to do includes design principles to guide course authors.

The following phases of outcomes and course design are connected by the theoretical frameworks of Understanding by Design (Wiggins, 1998), Universal Design for Learning (CAST, 1998), and Quality Matters (2003) and create a nexus of effective practices. In addition, we must ask and answer these questions:

  1. Who is responsible for creating meaningful and measurable outcomes?
  2. When are outcomes designed?
  3. Who has a stake in the design of meaningful and measurable outcomes?
  4. What are the barriers to effective design of outcomes?
  5. Where does the responsibility and accountability for outcomes assessment lie?

The use of Backwards Design (1) provides for more relevant and meaningful learning experiences; (2) ensures that the required course outcomes are met; and (3) prepares learners to perform successfully on their final assessment. One starts with the end—the desired results (goals or standards)—and then “derives the curriculum from the evidence of learning (performances) called for by the standard and the teaching needed to equip students to perform” (Wiggins & McTighe, 2000).

Pre-Design Phase

Conduct a Needs Assessment: Are there institutional, disciplinary, or professional knowledge and skills that must be addressed in outcomes that are developed or that must be adopted without revision in a program or course? Are there specific student/learner needs that should be addressed during course development? Are there course delivery needs that must be addressed during course development? The answers to these guiding questions form your needs assessment. In some institutions, program and/or course outcomes may be mandated and faculty are not able to revise them. In this case, you will learn how to develop meaningful and measurable learning experiences that will ultimately meet those mandated course outcomes.

Design Phase

The Design Phase for making meaningful and measurable outcomes includes the following four (4) steps. A brief overview of these steps will be provided:

Design 5-8 Course Outcomes. Sample course outcomes from different disciplines will be provided with an opportunity to discuss how to revise existing outcomes.

Design Module Objectives. How to align module level objectives with course level outcomes will be discussed with samples from various disciplines provided.

Design Assessments that Meet Outcomes and Objectives. A brief overview of this process will be provided along with a template for a Course Outcome Alignment Table.

Design Meaningful Learning Experiences, Resources, and Activities. A brief introduction to how to create meaningful learning experiences through scaffolded student learning will be provided along with a sample Learning Activities Alignment Table.

The focus of this session is the Design Phase*. In order to provide the most practical application of this topic, the micro focus of designing module/unit-level learning objectives will be provided.

*Using Bloom’s Revised Taxonomy to Construct Objectives

Bloom’s Revised Taxonomy outlines six levels of learning. Moving up the list of levels allows for deeper learning to occur, however, it is important to include objectives that meet multiple levels across the taxonomy. Each level of learning can be described with verbs when developing an objective. For example, to write an objective aimed at the Remember level, use a verb like list or define.

Example:

The learner will list the 50 states and their capitals.

Level of LearningObjective Action
(Lowest) Remember: Recall facts and knowledge from long-term memoryList, Recognize, Recall, Identify, Define, Describe, State
Understand: Assign meaning to given informationSummarize, Classify, Clarify, Predict, Interpret, Infer, Compare, Explain, Discuss, Distinguish
Apply: Use learned information in a given situationRespond, Provide, Carry out, Use, Execute, Implement, Apply, Demonstrate, Prepare, Conduct, Utilize, Show
Analyze: Break down the material and determine how the pieces relate to one another or the larger picture of the informationSelect, Differentiate, Integrate, Deconstruct, Organize, Attribute, Analyze, Develop, Design, Illustrate, Defend
Evaluate: Judge the results of something based on the information learnedCheck, Determine, Judge, Reflect, Critique, Conclude, Compare, Interpret, Justify, Test, Verify
Create: Put learned information together to generate new structures/patterns(Highest)Generate, Assemble, Design, Create, Plan, Produce, Integrate, Revise, Edit, Transform, Organize, Reorganize

Characteristics of Well Designed Learning Objectives:

  1. Objectives should identify a learning outcome.
  2. Objectives should be consistent with course goals. When objectives and goals are not consistent, two avenues of approach are available: change or eliminate the objective, or change the course goal.
  3. Objectives should be precise. It is sometimes difficult to strike a balance between too much and too little precision in an objective. There is a fine line between choosing objectives that reflect an important and meaningful outcome of instruction, objectives that trivialize information into isolated facts, and objectives that are extremely vague. Remember, the purpose of an objective is to give all learners the same understanding of the desired instructional outcome.

Post-Design Phase

The Post-Design Phase includes activities that support faculty’s work in outcomes development and offers insight into how to use data about the extent to which learners have met the outcomes to improve the overall learning experiences in both courses and the program as a whole.

Create a Course Map. A brief overview and sample of a Course Map will be provided.

Propose Curriculum Design or Revision to Relevant Governing Bodies. How to propose new or revised courses with meaningful outcomes along with a rationale will be discussed.

Program and Course Evaluation. How to use your assessment data to determine to what extent learners have mastered course outcomes will be discussed.

To find out more about using meaningful and measurable outcomes in your course development, read these articles from Faculty Focus:

Course Outcome or ObjectiveModule/Unit-Level Objectives
PSY 100: SLO.1Students will demonstrate knowledge of the major concepts, models, and issues in psychology.Describe and discuss how the subject of aggression is perceived through the biological perspective, the cross-cultural perspective, and the psychodynamic perspective.Using a provided case study, identify which of the seven psychological perspectives is applied by the psychologist.Compare and contrast the seven psychological perspectives for origin, theory and clinical application.

How a Course Map Puts You on Track for Better Learning Outcomes

Scaffolding Student Learning: Tips for Getting Started

References:

Beld, J.M. (2010). Engaging departments in assessing student learning: Overcoming common obstacles. Retrieved from

https://www.aacu.org/publications-research/periodicals/engaging-departments-assessing-student-learning-overcoming-common

CAST (n.d.) CAST: CAST timeline. Retrieved from http://www.cast.org/about/timeline.html#.XEuHrVxKg2x

Center for Teaching and Learning. (n.d.) Revised Bloom’s taxonomy. Iowa State University. Retrieved from http://www.celt.iastate.edu/teaching/effective-teaching-practices/revised-blooms-taxonomy/

Meyers, N. M., & Nulty, D. D. (2009). How to use (five) curriculum design principles to align authentic learning environments, assessment, students’ approaches to thinking and learning outcomes. Assessment & Evaluation in Higher Education, 34(5), 565-577.

National Institute for Learning Outcomes Assessment. (2011, Spring). What new faculty need to know about assessment. Retrieved from http://www.learningoutcomesassessment.org/documents/ABfaculty.pdf

Osters, S. (2018, May 7-8). Writing measurable learning outcomes. 3rd Annual Texas A&M Transportation Technology Conference. Retrieved from

https://www.gavilan.edu/research/spd/Writing-Measurable-Learning-Outcomes.pdf

Pappas, C. (2015, August 31). Writing learning objectives for elearning: What elearning professionals should know. Retrieved January 25, 2019 from https://elearningindustry.com/writing-learning-objectives-for-elearning-what-elearning-professionals-should-know

Quality Matters. (n.d.) Quality matters: About. Retrieved from https://www.qualitymatters.org/about

Smith, T. (2012, July 2). Writing measurable learning objectives. Retrieved January 25, 2019 from https://teachonline.asu.edu/2012/07/writing-measurable-learning-objectives/

Towns, M.H. (2010). Developing learning objectives and assessment plans at a variety of institutions: Examples and case studies. Journal of Chemical Education, 87(1), 91-96. DOI: 10.1021/ed8000039

Trustees of Boston University (n.d.) How to write clear learning objectives. Retrieved from https://www.bu.edu/cme/forms/RSS_forms/tips_for_writing_objectives.pdf

Wiggins, G., & McTighe, J. (1998). Understanding by design. Alexandria, VA: Association for Supervision and Curriculum Development.



Please follow and like us:

Leave a Reply

Your email address will not be published. Required fields are marked *