Learning and Teaching > Program Assessment > Process to achieve outcomes
MSE Response 2/2/05 – Genalo
Organization – how are departments organized for continuous assessment/improvement of learning?
The answer to ii below includes continuous improvement aimed at defining and meeting objectives. Continuous improvement is also facilitated by the maintenance of a course portfolio in which the faculty member writes a self-reflective memo on the course and outlines future plans in light of the feedback. The faculty member also collects representative samples of student work (direct assessment). This portfolio is maintained for each course and a self-reflective memo is written for each course offering (responsibility – each faculty member teaching an undergraduate course, monitored by the ABET coordinator and the Assistant Chair). Faculty retreats focusing on curriculum are held periodically (approximately every two years) and one-hour faculty meetings on various aspects of the curriculum are held each semester (responsibility – all). The retreats give the opportunity for the faculty to consider the curriculum as a whole. The students in each class fill out an “outcomes addendum” at the time they do course and instructor evaluations. These results are summarized and reported to the curriculum committee and the appropriate teaching faculty in case of needed changes (responsibility – Assistant Chair). The graduating seniors are interviewed and surveyed about several items including their opinion on whether or not program objectives have been met in their own case as the “output” of the program. These results are tabulated and reported to the curriculum committee (responsibility – Assistant Chair). Annual meetings of the undergraduate students are held to solicit their opinions on anything departmentally-related. Many of these items are program or course related and are fed back to the entire faculty for consideration. OPAL results are analyzed for co-op/internship students and reported to the faculty (responsibility – co-op coordinator). Our design sequence of courses includes significant industry interaction and faculty mentors for each project. Both the industry sponsor and the faculty mentors are surveyed.
Objectives – What processes do you use to develop and review objectives (course, program)?
Program objectives were established by the faculty with input from the Industrial Advisory Council and students. We adopted the ABET A-K objectives verbatim, added in four outcomes from professional materials society outcomes, and added three locally developed outcomes that demonstrate the uniqueness of an ISU/Mat E degree program. These objectives are reviewed periodically by the same constituents, but to be honest not “religiously.” (Responsibility – curriculum committee)
The following is a flowchart used for course-level outcomes related to the A-R (A-K + locally added outcomes) for the Mat E program.
Assessment Measures – direct vs. indirect, qualitative vs. quantitative, what groups/constituencies, what rubrics
As stated previously all of these methods are used. Direct assessment of student work is, of course, done in each course. It is also included in the course portfolio maintained by the teaching faculty. Indirect measures include surveys of students in each course and at the time of graduation. OPAL measures, collected from students and employers, are used with experiential learning. Surveys of alumni have proved spotty and inconclusive. Surveys of faculty mentors and industry sponsors of design projects are used. Qualitative measures include the student forums held each year, the interviews at graduation, the faculty retreats and meetings, and the IAC sessions. Survey data is quantified in all cases.
We have rubrics for written and oral reports, as well as teamwork. Faculty are free to use these rubrics and many do, but some choose their own.
Continuous Improvement – What changes have been made in your program as result of your assessment feedback process?
Many changes have been made to the mapping of course outcomes to program outcomes after looking at student surveys and faculty input solicited at the end of each course. The content of several courses has changed slightly due to input from students (course evaluations, exit interviews, forums) about content overlap among courses. Mat E 213, formerly the second in a sequence of 5 design courses (ENGR 170, Mat E 213, 313, 413, 414) has been dropped from the 05-07 catalog due to significant overlap with ENGR 170. The four-course ceramics, polymers, and electronics specialties have been altered due to faculty and student input. Some faculty have been reassigned to other courses due to student and faculty input on specific courses.
v. Documentation and Dissemination – This would be primarily the web record of our objectives, processes, and results.
Objectives are disseminated to students on the department’s web site, in the catalog, hung up on department bulletin boards, and included in various course syllabi. They are disseminated to the IAC at meetings. The processes used are on the department web site. The results, meaning changes made due to continuous improvement assessments, are documented in reports to the faculty, the students, the IAC, to the college when appropriate (e.g. drop Mat E 213), and (every six years) to ABET.
vi. College role – What could the college do to help with portfolios, out-year assessments, data analysis and tools, etc.?
OPAL assessments are rich and can be used for much more than co-op assessments. Alumni and employer surveys could be done on a uniform basis so that the same employer doesn’t receive multiple requests. Studies of the latest and greatest methods for improvement assessment could be done and communicated to the departments.
