OTD 8220: Measurement of Human Function and Learning

Course Description

Measurement is an essential feature for quantifying and understanding change in human performance. In particular, this course will focus on changes in performance that occur in clinical environments and in research programs. This course provides an introduction to the basic principles of measuring human function including selecting, implementing, and evaluating assessment tools. Topics include: levels of measurement; types of assessments (such as self-reported, clinician-observed performance, instruments), purposes of assessment (e.g., screening, diagnosis, effectiveness, decision-making); basic psychometric concepts such as reliability and validity, and introduction to modern psychometric methods. In addition, the course will look at best practices in development of patient reported outcomes assessments based on FDA guidelines.

Course Purpose

The purpose of the course is to increase students’ skill and comfort in applying fundamental psychometric concepts to reviewing and selecting assessment tools in the major areas of occupational therapy practice i.e., clinical practice, education, or research. The course is also designed to help students increase their confidence with crosscutting skills such as searching for, reviewing, and appraising scientific literature, and in developing professional-level writing and presentation skills.

Course Objectives

By participating in this course, the learner will be able to:

Define and appropriately use the terms reliability, validity, precision, and responsiveness
Search, identify, review, and summarize psychometric studies of assessments
Identify and describe the psychometric factors and practical considerations in selecting assessment tools for research, teaching or clinical practice
Appraise the strengths and limitations of different assessment approaches (self-report, observational, instrumented) for setting goals and evaluating outcomes in research, teaching, or practice
Demonstrate professional-level writing and presentation skills by completing two Research Measurement Database summaries.
Articulate/summarize the process for developing and testing a patient-reported outcome assessment

Course Schedule

Weekly Objectives

Week 1: Introduction:

Review the course syllabus and expectations
Appraise accuracy and depth of information provided by several online assessment databases.
Compare and contrast strengths and limitations of several assessment databases.

Week 2: Levels of Measurement:

Examine the 4 levels of measurement
Review selected assessments and determine the level of measurement.
Discuss the relationship between the level of measurement of the selected assessments and their clinical utility.

Week 3: Assessment Quality-Reliability:

Identify and explain reliability in terms of true score theory, types of reliability and the sources of measurement error addressed by each type, and how it is established.
Debate issues affecting reliability including cultural relevance/bias and translation using clinical examples.

Week 4: Assignment 1 Presentations

Week 5: Assessment Quality-Validity:

Identify and explain validity in terms of purpose, types of validity, and how it is established.
Discuss the relationship between validity and reliability in terms of specific OT assessments.

Week 6: Assessment Quality-Sensitivity, Precision and Measuring Change:

Identify and explain the need for precision in measurement, kinds of precision (including targeting, minimally clinically important differences, minimal detectable change, and standardized response means)
Discuss the level of precision needed for different clinical, educational and research situations.

Week 7: Interpreting Assessment Results:

Identify meaningful change has occurred based on assessment results.
Determine the meaning of normative values for clinical decision making
Interpret assessment results for clinical practice or education

Week 8: Exam

Week 9: Why We Measure:

Describe and appraise the different reasons for using assessment tools including screening diagnosis, clinical decision making, educational and health outcomes measurement.

Week 10: How We Measure:

Compare and contrast the uses, strengths, and limitations of assessment types, including patient reported outcomes (PROs) including the use of proxies, clinician-observed measures, and instrumented assessment approaches.
Appraise and debate the merits of using assessments developed by other disciplines.

Week 11: Standardization in Assessment:

Explain the pros and cons of using standardized assessments in clinical practice.
Describe the circumstances under which adaptations can be made to assessments and the consequences for meaningful interpretation of data.
Demonstrate understanding of the concept of proximity between assessment and intervention/educational effect.

Week 12: Using Assessment Tools for Enhanced Clinical Decision-making:

Identify the main principles for interpreting a run chart
Appraise different visual formats for tracking outcomes
Describe how to use assessment results for goal setting
Discuss how to use assessment results to support accreditation, reimbursement, and audit reviews.

Week 13: Instrument development:

Outline the main steps in the development of a patient-reported outcome
Describe instrument design and development from the perspective of the FDA guidelines for endpoint development.
Discuss the pros and cons of using nonstandardized home grown assessments.

Week 14: Measurement Using Rasch Analysis:

State the main assumptions of the Rasch model
Describe how hierarchical, linear assessment can improve goal setting and treatment planning
Appraise assessments developed using Rasch Analysis including AMPS, AM-PAC, and Fugl-Meyer

Week 15: Assignment 3 Presentations

Synthesize information learned over the course to review and evaluate a clinical assessment.
Search, identify, appraise, and present an assessment for the Rehabilitation Measures Database

Course Assessments