Pages

July 23, 2011

Using Sampling for Assessment of Programs

One of the most important questions around program assessment is the one of data collection: how many, from whom, how often.  It is critical to remember that the process of assessing the program is about the program, NOT individual students.  We are trying to answer the question:  "How is the program doing in promoting student learning."  Whether we are talking about general education curriculum or a specific program, the issues are the same.

If the number of students in any program are sufficient, sampling is acceptable.  Good sampling techniques should be used and the sample being assessed should mirror the population of students on those characteristics which are identified as critical to justify generalizing the findings.  For example, if there are seven sections of a particular course where students are given opportunities to demonstrate their learning it may not be necessary to assess every section.  If the number of students in the population are not sufficient to justify sampling then you would be need to assess the performance of all students in the cohort.

Refer to my post  of May 22, 2010, "Data Do Not Always Information Make." for a discussion of how many data need to be collected.  In my next post, I will address the issue of "how often" to collect data.

7 comments:

Rebeca Leal said...

Hi Gloria!, we are planning to have some members of faculty get trained in rubrics. Can you recommend us where to take this training? And also, do you know of one or two universities that have “Assessment Centers” that we can visit to learn what they are doing?

Rebeca Leal said...

Hi Gloria!, we are planning to have some members of faculty get trained in rubrics. Can you recommend us where to take this training? And also, do you know of one or two universities that have “Assessment Centers” that we can visit to learn what they are doing?

Gloria Rogers said...

Rebecca, Most well-developed faculty development workshops on classroom and/or program assessment should include sections on rubric development. I would suggest you contact your regional accreditation agency. Most of them host annual or special meetings which include training on assessment. If not they would be a good source of recommending workshops in your region.

"Assessment Centers" come under different banners and with different names. Some are housed within Institutional Research offices, others are in centers for teaching and learning. Some are headed by assessment/evaluation professionals and others by faculty members whose academic disciplines are in other areas. It would be important to be sure that the institution visited was similar in terms of focus and resources to be sure that you get a good idea of what can be done to improve your own efforts. To begin with, I would look at JMU (James Madison University www.jmu.edu/assessment/). They have a very robust assessment center which also has as its mission to support multiple graduate students and professional programs in the field. Because of their pominence in this area, they would also probably be aware of other strong assessment centers which are led by an assessment professional.

I hope this has been helpful.

Rebeca Leal said...

Thank you Gloria!

Rebeca Leal said...

Thank you Gloria!

Unknown said...

Thanks Gloria.

there is a school of thought back in my country that we need to measure each students program outcomes (POs) and if they do not achieve these POs, they cannot graduate.

What is your view on this?

Gloria Rogers said...

Faris, An institution needs to be clear why they are doing the PO assessment. If it is truly a process that is designed to assess how well the program is doing in attaining it's intended outcomes, then sampling is appropriate (if the numbers of students in a cohort are large enough). The practice of not allowing students to complete the program if they cannot demonstrate the outcomes is generally done in the context of courses--if they cannot pass the course, they cannot complete the program. If an institutions adopts the practice of, "if they (students) do not achieve these POs, they cannot graduate" then the faculty, collectively, must take the responsibility to provide students with adequate opportunities to learn, practice, develop and get feedback on their performance on the outcomes. Curriculum mapping is very critical in such a high stakes assessment. The institution must be clear as to where students are getting opportunities these opportunities and that there is both formative and summative assessments and students are being provided feedback throughout the curriculum to maximize the likelihood of their success.

The biggest challenge in a process like this is to get faculty to expand their focus from "my courses" (the ones that they teach) to OUR curriculum.

 
Site Meter