Materials Science and Engineering

B. Accreditation Summary

3. Program Outcomes and Assessment

In this section, program outcomes and assessment processes and results are discussed including:

Program Outcomes and their Relationship to Criterion 3
Program Outcomes Relationship to Program Objectives
Process to Assure Graduates have Achieved Outcomes
Qualitative and Quantitative Data & Documentation of Changes
Continuous Improvement Process
Process and Procedures for Acceptance of Transfers and Validating courses from Outside ISU
List of Materials Available at the Time of Review

Statement of Program Outcomes and their Relationship to Criterion 3.
Program outcomes for materials engineers are divided into three categories. Outcomes a-k are applicable to any type of engineer. Outcomes l-o apply to any materials engineer and outcomes p-r are those which are specific to materials engineers graduating with a Materials Engineering Degree from Iowa State University. A more complete description of the interpretation of these outcomes appears in the Departmental vision statement "The MSE Roadmap." Many of these outcomes are multidimensional and it is sometimes possible to achieve part, without achieving all of a particular outcome. In the assigning of particular experience as appropriate to assessing achievement of an outcome, we have taken the broad interpretation of "contributing towards" achieving the outcome rather than "proves achievement of" an outcome.

Graduates in Materials Science and engineering demonstrate the following related to general engineering practice:

an ability to apply knowledge of mathematics, science, and engineering
an ability to design and conduct experiments, as well as to analyze and interpret data
an ability to design a system, component, or process to meet desired needs
an ability to function on multi-disciplinary teams
an ability to identify, formulate, and solve engineering problems
an understanding of professional and ethical responsibility
an ability to communicate effectively
the broad education necessary to understand the impact of engineering solutions in a global/societal context
a recognition of the need for and an ability to engage in life-long learning
a knowledge of contemporary issues
an ability to use the techniques, skills, and modern engineering tools necessary for engineering practice
Additional Materials-Specific Outcomes; Graduates of this program must have demonstrated:
an ability to apply advanced science (such as chemistry and physics) and engineering principles to materials systems
an integrated understanding of the scientific and engineering principles underlying the four major elements of the field (structure, properties, processing, & performance)
an ability to apply and integrate knowledge from each of the above four elements of the field to solve materials selection and design problems
an ability to utilize experimental, statistical and computational methods consistent with the goals of the program.
Additional Materials-Specific Outcomes specified for graduates of the Materials Engineering Program at Iowa State University. Students will:
demonstrate mastery of creative, independent, problem solving skills, under time and resource constraints, in a broad range of materials-related applications critical to the success of the final product.
have gained experience in materials engineering practice through co-ops or internships in industry, national laboratories, or other funded research work.
demonstrate hands-on skills with a broad range of modern materials processing and characterization equipment and methods, with special in-depth concentration in two student-selected areas from among ceramic, electronic, metallic, and polymeric materials.

Relating Program Outcomes to Program Objectives
It is critical to demonstrate that achieving program outcomes results in meeting program objectives. Table B3.1 shows this relationship and should be read, for example "In order to practice materials engineering in a broad range of …, a student must demonstrate an ability to apply math, science, and engineering…." . A case could be argued for situations where many of the program outcomes contribute to each program objective, however, only those outcomes most critical to the achievement of a particular objective are listed. Program Objectives are repeated below for reference.

practice materials engineering in a broad range of industries including materials production, semiconductors, medical/environmental, consumer products, and transportation products.
are capable of responding to environmental, social, political, ethical and economic constraints and will improve the quality of life in Iowa and the world
are capable of working independently and in teams and are proficient in written, oral and graphical communication
engage in lifelong learning in response to the rapidly expanding knowledge base and changing environment of our world
engage in advanced study in materials and related or complementary fields

TABLE B3.1, TABLE B3.2 and TABLE B3.3

Process to Assure Graduates have Achieved Outcomes
The process used to assure graduates have achieved program outcomes was designed after carefully examining Table B3.3. Clearly, almost all outcomes rely heavily on curricular content (as designated by a ‘C’). Therefore a significant fraction of assessment efforts are an evaluation of the efficacy of the curriculum.

Assessment Methods
Various assessment methods are used to ensure that program outcomes are met prior to graduation including:

Curricular

Performance in individual courses
Course evaluations
Student Transcripts (grades)
Design experiences (final reports and presentations, evaluations from industry sponsors)
Peer Evaluation
Supporting Programs & Activities
Co-op/Internship Evaluations
International Experience evaluations
Honors programs of study
Participation in Professional Societies and Student organizations
Participation in Academic/National Laboratory Research Summative Measures
Student Feedback Session
Senior Surveys
Senior Exit Interviews
Standardized exams (discussed in evaluation of meeting objectives –section B1)

Not all methods are used (or are appropriate) for every desired outcome, but in most cases at least three are used to document achievement of each outcome. Selected assessment methods are discussed below.

Curricular Contribution to the Achievement of Program Outcomes
Table B3.4 shows the desired outcomes are mapped against the courses required for a materials engineering degree to delineate which courses contribute to each of the outcomes. This mapping was done by 5 faculty committees: one for the core curriculum and each of the four specialization areas. For each course within the department, the syllabus outlines the course objectives and outcomes as well as how they relate to the departmental objectives and outcomes. Faculty felt that at least two designators for levels of involvement were necessary to accurately reflect a course’s contribution to a particular outcome. A significant contribution is indicated by a large circle and designates that a student successfully completing this course will have made significant strides towards achieving this outcome. A small diamond indicates a moderate level of contribution to an outcome by a particular course. The "significant" contribution also indicates that outcome is measured, the "moderate" contribution indicates that outcome is not directly measured in that class. In the first year’s evaluation, all outcomes were measured to determine whether the student would perceive the same differing levels of involvement as the faculty. Course evaluations at the end of the semester are administered using the standard University form, but the department has a course evaluation addendum which addresses specifically the extent to which the student believes each of these outcomes has been met (both forms will be available at the time of the visit). The faculty teaching the course also completes the same form before reviewing student replies.

TABLE B3.4

English Proficiency Requirement:
The Department of Materials Science and Engineering requires a grade of C or better in Engl 104 and 105 and certification from the departmental curriculum committee.

Certification by the curriculum committee is achieved through earning a "pass" on the communication grade in 4 of 8 selected communications-intensive courses (Mat E 211, Mat E 214, Mat E 321, 331, 341, 352, Mat E 316, Mat E 414). The communication grade component of these courses is kept separately from the overall grade. (i.e. it is possible to PASS the course and FAIL the communications component).

Should a student not be certified by the end of the junior year, he/she has the option of;

taking an advanced English course from the following list: 314, 302, 306 and earning a C or better, or
getting certified by earning a passing grade in the communications portion of 414 if he/she already has 3 communication credits in previous courses. (Note graduation will be delayed if this credit is not earned.)

Supporting Programs and Activities Contribution to the Achievement of Program Outcomes

Co-op/Internship Evaluations: The accurate evaluation of students in the professional workplace can be an extremely valuable tool to assess the effectiveness of our program. Evaluations are administered to both the supervisor and the student. These evaluations have very recently been restructured to better reflect the competencies in which we are interested. This instrument is likely to be adopted across the curriculum for the evaluation of student competencies. (A copy of this Assessment Guide will be available at the time of the visit.)

Participation in Professional Societies and Student organizations: Most students participate in the field-specific professional societies, many in other student organizations and some in a leadership capacity. Although participation doesn’t validate achievement of a particular outcome, it does suggest that the student has the opportunity to develop many "soft skills" which are more difficult to address in the classroom.

Participation in Research: A significant fraction of students (~40%) work in a research lab (either academic or national lab) during their undergraduate career. This experience affords opportunities to apply analytic skills, gain an appreciation for lifelong learning, and hone their laboratory skills and facility with modern engineering equipment. In most cases these students are required to communicate their results and achievements in written/and or oral form. An evaluation tool is currently being developed to capture competencies gained during this experience.

Summative Measures
Standardized exams (Discussed in Program Objectives section)
Senior Surveys (data presented below)
Senior Exit Interview
Student Feedback Session(data presented below)

Other opportunities that contribute to the meeting of these outcomes are described in Appendix II including

University Opportunities
Intramural Athletics, Extracurricular activities, Campus Organizations:

College Opportunities
Learning Communities, College Student Organizations:
National Conventions and Student Conferences, Honor Societies

Qualitative and Quantitative Data & Documentation of Changes
Course Evaluations
The standard university course evaluation form is used with a supplement added by the MSE department. The survey is divided into two areas "the course" and "the instructor." Departmental averages for "the instructor" are traditionally the highest in the college – a 4.2/5.0. When asked if the stated goals of the course were attained, 85% of the students said "yes". Eliminating the design sequence, the score is 94%. This information allows us to conclude that the students perceive that both the quality of the instruction and course content is very high. Both the university forms and the departmental supplement will be available at the time of the visit.

Outcomes Ratings for Fall 1999/ Spring 2000 (Supplemental Course Evaluation Forms)
Colored cells represent outcomes which were rated by students as below 3 on a scale of 1(course does not contribute to this outcome) to 5 (course contributes significantly to this outcome. Yellow cells represent an average student rating of an outcome below 3, blue cells represent a faculty rating below 3 and green cells represent BOTH student averages and faculty rating below 3. Each specialization committee examines the data for courses in that area and develops an action plan to correct the deficiency or restate contribution to outcomes.

General Observations and Data Analysis
o The new vertically integrated design course was rated very low by the students and comprises over 45% of the student-only low ratings. (Actions taken will be discussed later)
Without the design contribution: (Design is discussed below)
o faculty were slightly more critical of their courses than the students. (29:23)
o faculty and students agreed in 88% of the evaluations.
o only students marked they had not satisfactorily achieved b, faculty cited no deficiencies.
o the most low marks were on d (functioning in multidisciplinary teams)
o there was large standard deviation in the ratings of outcome d indicating there may have been interpretation difficulties.

After each specialization committee reviewed the results of assessment of achievement of objectives, changes in the programs have been made. In many cases it was found that where outcomes were not met in a particular course, it was because the assignment of that outcome to that course was inappropriate. In this case, the faculty subcommittee convenes and reviews which outcomes should be addressed in that course.

TABLE B3.5

For example, it was concluded that outcomes b, c, d and g do not belong in the Mat E 315 Kinetics class. The revised outcomes grid will reflect these changes. The revised outcomes/course grid is then reviewed by the curriculum committee to determine the impact of the changes on the curriculum as a whole. A complete description of course changes will be available at the time of review. In general, we believe that while courses may contribute to a number of outcomes, it is desirable to focus assessment on relatively few. In effect many courses "claimed" too many outcomes. This will be remedied when the curriculum committee reconvenes and reviews the course evaluations.

It was also concluded that in some cases (particularly where the data show a large standard deviation), students had difficulty interpreting the outcomes statement with respect to their classroom experiences. This problem can be minimized by helping the students relate program outcomes to specific course learning outcomes. This will be done in future syllabi. It is also anticipated that future generations of students will be more capable of evaluating their performance when they and the faculty become more comfortable with the terminology and philosophy of outcomes-based assessment practices.

Areas for improvement:
Vertically integrated design course: This novel vertically integrated design sequence was first administered in Fall 1999. This course involves seniors, juniors and sophomores working on design teams. During the course, a graduate student collected data on group performance, group functioning and individual performance for all students. Additionally, the students completed both the standard course evaluation and the MSE course evaluation at the end of the class. There were significant discrepancies in the faculty’s perception of success of the course and the students. The results were thoroughly studied by the faculty involved, the curriculum chair and by a graduate student and an undergraduate student. In summary, it appears that a significant part of the problem was in communication of the course goals and explanation of the pedagogical approaches. (The course focussed on very open-ended design problems which the students interpreted as lack of organization and structure). Other student objections were traced to the logistical difficulties associated with a team-taught course A full report of the design class, with suggested changes will be available at the time of the visit. Several concrete changes have been made in the structure and implementation strategies for these classes.

Summative assessment
Senior Surveys
Senior surveys and interviews have been implemented since 1994, although the form of the survey changed in 1998 to better reflect stated objectives and outcomes. With the program changes in the 1999 catalog, it is reasonable that only the most recent surveys will be discussed. Nineteen senior exit surveys and interviews were administered in F1999/S2000 (representing the entire graduating population). Students were asked to rate their achievement of each of the program outcomes a-r. It should be recognized that most of these students began at ISU before the department had published our program objectives or outcomes, so this cohort of students were less familiar with the terminology than future students will be. Students were asked to rate themselves between 1 (outcome not achieved) to 10 (achieved at a bachelor’s level). The results are shown below. The average of 19 responses is shown in the first row and the standard deviation in the second.

a b c d e f g h i j k l m n o p q r

9.4 9.4 8.8 9.6 9.4 8.1 9.3 8.2 9.3 8.1 9.2 9.3 8.9 8.6 8.7 8.8 8.6 7.9

0.6 0.7 1 1 .06 2.6 0.7 1.7 1.1 1.2 0.7 0.7 1 1.3 1.1 1.1 3.1 1.8

Overall, the students rated themselves 8.6/10, a very good average. The lowest ratings are italicized and the highest are in bold. Despite the fact that the materials engineering curriculum has the highest number of laboratory credits in the engineering college, the students felt that the they had only achieved to a level of 7.9/10 in (q) demonstrating hands-on skills with a broad range of modern characterization equipment and methods with a special in-depth concentration in two areas among ceramic, metallic, electronic and polymeric materials. Note that the standard deviation is fairly large in this outcome, too. It is thought that because all of these students graduated with either a metallurgical engineering degree or a ceramic engineering degree, students could have focussed on the part of this outcome that speaks to the concentration in two areas, which they didn’t have. Other lower ratings are in the areas of contemporary issues and professional and ethical issues. The new vertically integrated design sequence should aid significantly in improving these areas. Students also rated themselves low in the area of "broad education necessary to understand engineering solutions in a global and societal context". Reflecting on the outcomes grid, within the materials curriculum, the design course is responsible for this outcome. General education electives are also expected to help in this area. The curriculum committee is carefully considering the design course content to insure this outcome is adequately addressed. It has been suggested that students may hesitate to rate themselves a "10" regardless of what they are being asked. To account for this we will redesign the instrument to go from 1-12 with 10 still representing "achieved at a Bachelor’s level, and 12 representing a "Master’s" level.

Student Feedback Session
A program feedback session is held toward the end of the spring semester. This session is facilitated by senior students (no faculty are present). The facilitator has a list of questions/topics related to all program elements (the curriculum, advising, co-op/internships, etc.) to address during this session. A recorder (also a senior leader) records comments and delivers them to faculty. This information is brought to the curriculum committee to discuss and suggest changes for continuous improvement. Feedback 2000 resulted in the following concerns:

Scheduling difficulties for people returning from co-ops/internships
Faculty should take care to give more practical examples
Lab work exceeds the number of credits awarded
Report formats should be uniform throughout the department

The curriculum committee is considering these concerns and will act on them immediately. The committee has already begun a detailed analysis of scheduling options for people involved in experiential education and has developed a set of rubrics for report writing which is being reviewed by faculty.

Continuous Improvement Process
For curricular evaluations, the results of the students' perceptions are compared to those of the faculty, and discrepancies are carefully examined. The flow sheet for the evaluation and continuous improvement process of this program element is shown in Figure B3.1. This process occurs on a yearly basis.

FIGURE B3.1

Continuous improvement is also facilitated by the maintenance of a course portfolio in which the faculty write a self-reflective memo on the course and outline future plans in light of the feedback. This process occurs on a per course offering basis (usually yearly).

Faculty retreats focussing on curriculum are held periodically (approximately every two years). This gives the opportunity for the faculty to consider the curriculum as a whole. The next retreat is scheduled for August 16, 2000.

Materials Available at the Time of Review

Course Portfolios
Course information

Relationship to program objectives, Relationship to industry practices (if appropriate), assignments, learning activities list, grading standards.

Pedagogy

Description of teaching practices, philosophy and goals

Evidence of Student Learning

Assessment practices and tools, results, Samples of student work

Future Plans

Self-reflective memo, changes to be made, professional growth related to course (papers, presentations)

Student Transcripts
Core and Specialization Chair Report
(Detailing 1999/2000 Evaluation of "specializations", enrollment, course structure, actions taken to respond to assessments)
Senior Design Project Reports
Sample Co-op/Internship Evaluations
Copy of all survey instruments

Process for Acceptance of Transfers and Validating courses from Outside ISU
Students often transfer into Materials Engineering after one of two years at a community college or other 4 year institution. Also, frequently students often take single college courses during high school, summer or during co-op or internships. The University has in place a system for evaluation of transfer credits from these institutions. The process is described in detail in Appendix II in the sections "Admission of Students by Transfer from Other Institutions" and "Transfer Credit Practices". In addition to the University process for transfer credit and course evaluation, the department evaluates materials courses by examining the syllabus, textbook and often the results of a student interview. The faculty memberwho has most recently taught the course for which transfer credit is being sought evaluates the course and makes a recommendation to the curriculum committee for approval. In the case of a transfer from one or two years at another institution, the department uses an internal degree audit to track all changes and substitutions in the standard program. An example of this degree audit will be available at the time of the visit.