Assessment vs. Benchmarking
While both assessment and benchmarking are highly valuable activities for GOA, it is important to distinguish them for purposes of focusing Graduate Outcome Assessment (GOA) activities.
GOA, in general, assesses student learning outcomes, particularly outcomes of higher level learning beyond course work. GOA focuses on individual student outcomes, and aggregates individual student data to evaluate program effectiveness (by way of contrast, undergraduate LOA tends to focus on course outcomes and aggregates course data).
Doctoral GOA has an even more specific focus. Doctoral programs train students to contribute original knowledge to the field and to prepare for professional application of their skills. Doctoral programs, and to some extent many masters programs, prepare students not only to acquire knowledge, but also to add to the total knowledge of their disciplines. Doctoral GOA reflects this.
Many doctoral programs, moreover, focus on aspects of professionalism in addition to research, such as the ability to teach, to function in an organization, to lead, to collaborate, to create, to publish, to perform, to secure funding (or show the potential to do so), and so on. Programs define their goals for their students, and those goals will vary by program and discipline.
As a result, assessment and benchmarking share characteristics and are generally used in tandem: both are aimed at optimizing the quality of graduate programs and the success of their students. They also differ in significant ways, however, and it is important that graduate programs appreciate and apply their different characteristics appropriately.
Benchmarking tallies and tracks key, agreed-upon, markers of accomplishment, usually to help students progress through a program and/or to demonstrate the program’s success. For example, a program might monitor how many students pass their qualifying exams within three years of entering the program, how maåny papers each student publishes, and/or the timely completion of coursework for each student. A program will set out particular expectations and how many students meet them and when they meet them. This is a crucial process for ensuring that students meet program expectations and that they don’t get stuck at certain stages. Benchmarking, then, is a way of tracking the progress of each student, and, in aggregate, of demonstrating that a program has succeeded (or perhaps failed) in advancing its students appropriately.
Assessment, however, gathers and analyzes information on individual students that can be used to mentor students and that can be used to improve the program itself. In contrast to benchmarking, an assessment process does not track abstract markers of success (e.g., did the student pass the exam) but practices a finer grain of inquiry to identify patterns in what students are learning and producing and to use that information to determine what aspects of a program might be working well, what aspects might need more attention, and what the focus of that attention might be.
The publication of articles, to take an important doctoral achievement, fits both benchmarking and assessment. A program might set the benchmark that students should publish a paper within a year of passing their qualifying exams. For benchmarking purposes, the program need only track the number of students who do so. A program also might believe, however, that published articles in its field require characteristic X in order to have longevity. For assessment purposes, the program can identify the presence or absence of X in a wide sample of publications by its students, track the frequency of X, determine if the frequency of X meets the program’s expectations, and, if not, take programmatic steps to train individual students to develop X and to increase the frequency with which the program’s students do so.
Examples of Benchmarking Questions
- What percentage of students pass the qualifying exam on the first attempt?
- What percentage publish an article or articles in the top journals in the field?
- What percentage complete dissertation defenses without obvious problems?
- What percentage complete the degree in the nominal time-to-degree in the field?
- What percentage land post-docs or tenure/tenure track positions in the year following the degree?
Examples of Outcomes Assessment Questions: Dissertation Defense
- How original is the research?
- How well is the research presented in the dissertation? Verbally? Graphically?
- How persuasively does the student present the importance of the research?
- Does the dissertation reflect adequate knowledge of disciplinary context?
- How well does the student respond to oral questions on his or her research? To oral questions on related issues in the discipline?
- How adroitly does the student think on his/her feet?