Assessment vs. Benchmarking
Once your graduate faculty have identified your primary learning outcome(s) to target for assessment, you can begin developing your assessment. To do so, it’s important to understand the difference between assessment and benchmarking. While both assessment and benchmarking are highly valuable activities for graduate learning outcomes, it is important to distinguish them for purposes of focusing graduate learning outcomes assessment (GLOA) activities.
GLOA, in general, assesses student learning outcomes, particularly outcomes of higher-level learning beyond course work. GLOA focuses on individual student outcomes and aggregates individual student data to evaluate program effectiveness (by way of contrast, undergraduate learning outcomes assessment tends to focus on course outcomes and aggregates course data).
Doctoral GLOA has an even more specific focus. Doctoral programs train students to contribute original knowledge to the field and prepare for professional application of their skills. Doctoral programs, and to some extent many master’s programs, prepare students not only to acquire knowledge, but also to add to the total knowledge of their disciplines. Doctoral GLOA reflects this focus.
Many doctoral programs, moreover, focus on aspects of professionalism in addition to research, such as the ability to teach, to function in an organization, to lead, to collaborate, to create, to publish, to perform, to secure funding (or show the potential to do so), and so on. Programs define their goals for their students, and those goals will vary by program and discipline.
As a result, assessment and benchmarking share characteristics and are generally used in tandem: both are aimed at optimizing the quality of graduate programs and the success of their students. They also differ in significant ways, however, and it is important that graduate programs appreciate and apply their different characteristics appropriately.
Benchmarking tallies and tracks key, agreed-upon, markers of accomplishment, usually to help students progress through a program and/or to demonstrate the program’s success. For example, a program might monitor how many students pass their qualifying exams within three years of entering the program, how many papers each student publishes, and/or the timely completion of coursework for each student. A program will set out particular expectations and how many students meet them and when they meet them. This is a crucial process for ensuring that students meet program expectations and that they don’t get stuck at certain stages. Benchmarking, then, is a way of tracking the progress of each student, and, in aggregate, of demonstrating that a program has succeeded (or perhaps failed) in advancing its students appropriately.
Assessment, however, gathers and analyzes information on individual students that can be used to mentor students and that can be used to improve the program itself. In contrast to benchmarking, an assessment process does not track abstract markers of success (e.g., did the student pass the exam), but practices a finer grain of inquiry to identify patterns in what students are learning and producing and to use that information to determine what aspects of a program might be working well, what aspects might need more attention, and what the focus of that attention might be.
The publication of articles, an important doctoral achievement, fits both benchmarking and assessment. A program might set the benchmark that students should publish a paper within a year of passing their qualifying exams. For benchmarking purposes, the program needs only to track the number of students who do so. A program also might believe, however, that published articles in its field require characteristic X in order to have longevity. For assessment purposes, the program can identify the presence or absence of X in a wide sample of publications by its students, track the frequency of X, determine if the frequency of X meets the program’s expectations, and, if not, take programmatic steps to train individual students to develop X and to increase the frequency with which the program’s students do so.
Examples of Benchmarking Questions
- What percentage of students pass the qualifying exam on the first attempt?
- What percentage of students publish an article or articles in the top journals in the field?
- What percentage of students complete dissertation defenses without problems?
- What percentage of students complete the degree in the nominal time-to-degree in the field?
- What percentage of students land postdocs or tenure-track positions in the year following the degree?
Examples of Outcomes Assessment Questions: Dissertation Defense
- How original is the research?
- How well is the research presented in the dissertation? Verbally? Graphically?
- How persuasively does the student present the importance of the research?
- Does the dissertation reflect adequate knowledge of disciplinary context?
- How well does the student respond to oral questions on his or her research? To oral questions on related issues in the discipline?
- How adroitly does the student think on their feet?