Students are important, too!

I continue to be surprised when I visit colleges and universities who are investing hundreds if not thousands of hours of faculty time in learning outcomes data collection but do not provide the students with feedback on their performance. That is, they have mapped courses to outcomes, they have (in some cases) developed well written analytic scoring rubrics for the outcomes, faculty are scoring student performance in multiple classes, and results are being aggregated and reported as evidence of student learning for the program/institution to satisfy accreditors and/or state agencies.  When I ask faculty if students are provided with the rubrics before the scoring so they understand how they are going to be evaluated, I am generally told, "No." I then ask if the students are provided with the scoring results along with the rubric so they can see what they need to do to improve their performance, I am again told, "No." It seems that, in our anxiety over collecting data for accreditation and/or program review,  students are not even an afterthought. We know that student performance is cumulative over time and what they learn in one class, they use, practice, and develop in other classes. This is especially true for the core curriculum competencies as well as the discipline outcomes. Without being intentional, there is no reason to believe that learning will be enhanced in this data collection process.

For example, several years ago I was at a comprehensive university and was speaking before a large group of faculty about student outcomes assessment and their accreditation process. While talking about core competencies, one of the faculty members from the School of Business stood up and said, "Our students cannot write! They are terrible writers and I am embarrassed to think that they will represent our institution in the workplace." Then one of the faculty members from the English Department stood up and said, "Well, I can tell you one thing! They could write when they left our courses!" I then asked the Business faculty member if, when students turned in these terrible papers, did he return the papers to the students and ask them to rewrite them? "No." Did you mark their grade down for bad writing? "No." Do you give them any feedback at all on their writing? "No." When this happens, what is the message that we send our students when we don't give them feedback on their writing? "Writing doesn't matter!"

Why should we expect student performance to improve if we don't give them feedback on their performance on program outcomes over multiple courses? Providing students with well defined rubrics written to describe levels of performance can provide students with information they need to not only understand what their current performance level is related to a given program outcome, but also what they need to do to improve their performance.  The outcomes at the end of the program cannot be put at the feet of one department or course.  We have a collective responsibility for student learning when scoring program or institutional level outcomes.  Let's bring students back into the process.

Should the Accreditors Determine Your Learning Outcomes?

In the November 18, 2011 Inside Higher Education electronic newsletter, there was an article that highlighted what is happening at the Western Association of Schools and Colleges (WASC) a regional accrediting association.  Higher education professionals need to read it to be aware of the next wave in the accreditation and assessment of student outcomes wars.  The full article can be found at:

http://www.insidehighered.com/news/2011/11/18/western-accreditor-pushes-boundaries-quality-assurance#ixzz1e40Gb7lu.

The underlying premise is that, in the name of quality assurance and it's responsibility to the public to assure the credibility of the institutions it accredits, policies have been adopted where the accreditation letter and the accrediting team's report will be posted for the public to see.  The article states:

"One major prong of the package (proposal package) is that the accreditation process needs to become more transparent, and with the commission's approval this month, WASC will now be the first of the regional accrediting agencies to make public on its own website all of its "action letters" (in which the commission announces whether it has reaccredited an institution or taken some punitive action instead) and the reports of its accrediting teams on which the commission based its action. The norm for accrediting agencies to date has been to release a list of institutions that were either approved or sanctioned in some ways, and lists of the relevant provision numbers, but little to no additional detail."

Two other proposals were not adopted (yet, anyway):

"One would require institutions not only to define a "stated level of proficiency" for five skill areas for graduates (written and oral communication, quantitative skills, critical thinking, and information literacy) but to compare themselves to other institutions on at least two of those areas. The other was a suggestion that all institutions might be required to map their expectations for degree recipients to the Degree Qualifications Profile proffered by the Lumina Foundation for Education." 

If you haven't seen the Lumina Foundation Degree Qualifications Profile, it is a must read:

http://www.luminafoundation.org/publications/The_Degree_Qualifications_Profile.pdf

Outcomes assessment is coming under greater scrutiny than ever...will we be ready?

Baby or Bathwater?

I am working with a number of great institutions/programs on the development of meaningful assessment processes.  Of course, I love what I do so it is easy to let it consume me as I know that ultimately, a successful process will lead to more successful students.  However, I am struck about how easy it is to get so focused on the process that we forget the real purpose of all this activity--thus the title of this entry.  Yes, the process is designed to determine how well the program/institution is doing in producing student outcomes.  However, is that all there is?

If we believe that student learning is cumulative over time and that what students learn in one course they will use, practice, or further develop in another course, then it becomes clear that collecting data in a "summative" course that documents student learning is sufficient to know how well the program/institution is doing in delivering education.  However, we need to remember that this is about STUDENTS and what they know or can do.   Research tells us that students do best when they know the performance that is expected of them and when they get feedback on their performance.  Individual students should be provided information on what is expected of them (i.e., what will be on the test) and given feedback on their performance in formative contexts (courses/experiences before the summative assessment).  For example, do they get the scoring rubric that will be used to evaluate their performance or do they get written comments as to how to improve their performance?  Does their performance "count" in the final grading process?  If they are being asked to write an essay about the importance of diversity in the work setting, do they also get feedback on their writing as well as the "content" of the essay?  If so, is it considered a part of the grade for the assignment?  Is writing important in all contexts or just writing classes?  What is the message that is being sent to students? 

Providing students with feedback on their performance throughout the curriculum will improve the student outcomes by the end of the program.  It is important that we don't leave students out of this process.  Too many times we only get bathwater.

Procrastinate and Perish

As an experienced assessment professional, I often get calls to help institutions and/or programs with their student outcomes assessment plans as they are preparing for an accreditation visit.  Without exception (well, almost), my phone rings a year before the report is due.  Unfortunately this is an indication that this process is treated as an event instead of a process. 

A well-developed assessment plan is a continuous process.  This does not mean that you assess every outcome continuously.   What it does mean is that you have a systematic plan that enables you to smooth out of the workload over time.  There are well-defined cycles of data collection with defined timelines and areas of responsibility.  For example, if you have six learning outcomes and you have a three-year cycle, you would assess two outcomes every year.  This does not mean that there is no activity related to the outcomes for four more years.  To see an example of what a systematic cycle might look like, see an example table below:



Notice that data are collected in '10-11, evaluated in '11-12 and action taken with implementation of recommendations taken in '12-13 and a second cycle of data collection in '13-14.  The cycle is: outcomes defined into measurable statements (performance indicators), curriculum mapped, decision made about where to collect the data, data collected, evaluation of results and processes, recommendations for improvements, improvements designed and implemented, data collection. 

When last minute processes are developed, it becomes chaotic and puts an undue burden on faculty.  Also, it generally results in massive data collection which is difficult, if not impossible, to interpret.  Plan wisely!