Lecture Notes: Graduate Course in Assessment

Ever wanted to take a graduate-level course in assessment and psychometrics? Well, here’s your chance to access that level of learning, but without the price tag of tuition or having to travel thousands of miles.

The Course

ASC’s cofounder and Chief Product Officer, Dr. Nathan Thompson, was invited to serve as an instructor for several years in the Masters in Medical Education program offered by the University of Cincinnati and the Cincinnati Children’s Hospital Medical Center.  The program has a very specific and admirable goal: training the next generation of medical educators.  Most of the students were professors or deans at prestigious medical schools, extremely accomplished in their field, but wanted to learn more about Education.  This particular course was dedicated to the role of assessment and psychometrics in medical education, since the topic is not only vital to student learning, but also to accreditation of the medical school.

The content of the course was tailored to this specific focus.  Rather than generally surveying issues in psychometrics, the course focused on what was most relevant in graduate medical education.  Entire weeks were devoted to performance testing and simulations, while topics such as linking and equating were downplayed; these are hugely important in K12 and Certification assessment, but not medical education.

Nevertheless, there is a huge amount of content in this course and the textbook that is relevant to general practice of psychometrics.  For this reason, we’ve made the lecture notes available for free.  This closely aligns with ASC’s mission: Improving Assessment via Technology and Psychometrics.  Our fundamental goal is to improve the quality of assessment around the world, and a core part of that goal is educating more practitioners in the psychometrics that goes into developing a high quality assessment.

Sounds good?  Fill out this form and we’ll send you a download link.

The Lecture Notes

Because this was an online course, lecture notes were written out rather than drawn on a whiteboard during an in-person class.  All the content not covered by the book, as well as some from the book, is in the lecture notes.  The course was 10 weeks long, so there are 10 PDF files of notes.  Some are short (1-2 pages) but some are lengthy.  Here are the topics:

WeekTopicsBook chapters
1Intro; validity1,2
2Reliability, Standard error, generalizability theory3,4
3Classical test theory (statistics)5
4Standard setting6
5Written tests and MCQs7
6Observational assessment8
7Performance tests9
8Simulations10
9Item response theory11
10Advanced topics: adaptive testing, linking, DIF12

The Textbook

The textbook used for this course was Downing and Yudkowsky (2009).  You can preview much of it at Google Books here.  You don’t need a copy of the book to gain value from reading the notes, but it will definitely help.  However, if you are interested in a more general treatment of psychometrics, you’d probably prefer books like Shultz and Whitney (2005) Measurement Theory in Action (basic level) or Embretson and Riese (2000) Item Response Theory For Psychologists (advanced).