Introduction and Invitation

Constructing resources for assessment and instruction related to the eleven student outcomes contained in Criterion 3 of the ABET Engineering Criteria requires contributions across the entire engineering community. If you have one or more resources (for example, helpful papers, survey forms, assessment materials, instructional materials) for assessment and/or instructional related to outcome i click here. Please indicate whether and how you would like your contribution to be acknowledged. Thanks for contributing the growing understanding of how we might help engineering students develop knowledge and skills that they will draw upon throughout their careers.

Learning Objectives

The first step in selecting assessment and instructional approaches for a learning outcome is to formulate learning objectives that support the outcome. Learning objectives describe expectations associated with the outcome in terms of expected and observable performances. Several researchers have already constructed learning objectives and these may provide worthwhile starting points for others.

A team of researchers (Larry Shuman, Mary E. Besterfield-Sacre, Harvey Wolfe, Cynthia J. Atman, Jack McGourty, Ronald L. Miller, Barbara M. Olds, and Gloria M. Rogers) working a NSF-supported project, Engineering Education: Assessment Methodologies and Curricula Innovation, used Bloom's Taxonomy to develop and organize a set of learning objectives for outcome 3i (recognition of the need for, and an ability to engage in life-long learning) [1]. They developed learning objectives for all six levels of learning in Bloom's taxonomy for seven outcome elements:

• Demonstrates reading, writing, listening and speaking skills
• Demonstrate an awareness of what they need to learn
• Following a learning plan
• Identifying, retrieving, and organizing information
• Understand and remember new information
• Demonstrate critical thinking skills
• Demonstrate ability to reflect on own understanding

Felder and Brent offer the following three learning objectives for outcome 3i (recognize the need for life-long learning and be able to engage in it) [2]: The student will be able to:

• Find relevant sources of information about a specified topic in the library and on the World Wide Web (or perform a full literature search)
• Identify his or her learning style and describe its strengths and weaknesses. Develop strategies for overcoming the weaknesses.
• Participate effectively in a team project and assess the strengths and weaknesses of the individual team members (including himself or herself) and the team as a unit

Mourtos [3] breaks outcome 3i into two outcome elements: (a) a recognition of the need for lifelong learning and (b) the ability to engage in lifelong learning. Mourtos places the first element "recognition of the need ..." in the affective domain of Bloom's taxonomy and offers five learning objectives, one for each level of the affective domain.

• Willingness to learn new material on their own
• Reflecting on their learning process
• Participation in professional societies activities
• Reading engineering articles / books outside of class
• Attending extracurricular training or planning to attend graduate school

He places the second outcome element in the more familiar cogntive domain and offers nine learning objectives:

• Observe engineering artifacts carefully and critically, to reach an understanding of the reasons behind their design
• Access information effectively and efficiently from a variety of sources
• Read critically and assess the quality of information available (ex. question the validity of information, including that from textbooks or teachers)
• Categorize and classify informationAnalyze new content by breaking it down, asking key questions, comparing and contrasting, recognizing patterns, and interpreting information
• Synthesize new concepts by making connections, transferring prior knowledge, and generalizing
• Model by estimating, simplifying, making assumptions and approximations
• Visualize (ex. create pictures in their mind that help them see what the words in a book describe)
• Reason by predicting, inferring, using inductions, questioning assumptions, using lateral thinking, and inquiring

Todd [4], as part of a module on lifelong learning, offers the following learning objectives

• Explain the importance of lifelong learning in an engineering or computer science career
• Describe a process for learning new material
• Given a situation, identify what you need to learn
• Find appropriate resources in library and on the web
• List sources for continuing education opportunities
• Assess academic and professional development
• Given an assignment, show that they can learn material on their own

Assessment Approaches

In a report from the National Research Council, Knowing What Students Know: The Science and Design of Educational Assessment [5], assessment, once expectations have been constructed, rests on three pillars: cognition, observation, and interpretation.

Theories of Cognition

Under construction (15 December 2004)

Assessment of lifelong learning assumes, either explicitly or implicitly, a theory of how performance with respect to lifelong learning develops. One set of applicable theories is referred to as models of intellectual development. Models of intellectual development include the Perry Model of Intellectual Development [6,7]. Richard Felder wrote an accessible article [8] to help engineering faculty members translate behaviors they observe in their students to several different stages in Perry's model. Perry's model was based on over 400 interviews of male students at Harvard. Belenky et al [9] constructed a five-stage model based on interviews with women students. Baxter Magolda [10,11,12,13] and King and Kitchener [14,15] have developed stage models that might be viewed either as alternatives to Perry's model or extensions based on additional data and reflection. A review of the first book on the Reflective Judgment Model by King and Kitchener provides an accessible summary of their book. Wolcott and Lynch have constructed instructional and assessment materials based on the Reflective Judgment Model.

Another set of theories, referred to a learning styles, attempts to systematically describe differences in the ways people learn. There are many different approaches to learning styles.

Theories of Observation

Under construction

Theories of Interpretation

Under construction

Potential Resources

Self-Directed Learning Readiness Survey

The Learning Preference Assessment (LPA) is a new, self-scoring format of the Self-Directed Learning Readiness Scale (SDLRS) [16.17]. The SDLRS was developed by Lucy Guglielmino in 1977 and most of the research has been undertaken on the 58-item version of this instrument. Lucy and Paul Guglielmino collaborated to publish it in a self-scoring format in 1991. with the LPA being designed for self-scoring by Paul Guglielmino in 1991. The instrument has 58 five Likert scale (almost always true, usually true, sometimes true, usually not true, almost never true) items, with 41 of the items positively phrased and 17 negatively phrased. The instrument measures the attitudes, values and abilities of learners relating to their readiness to engage in self-directed learning at the time of response. This readiness is assessed as a total score which is then converted into bands of 'high', 'above average', 'average', 'below average' and 'low ' readiness. The SDLRS has been used at Penn State in a longintudinal study of the growth of self-directed learning during the undergraduate engineering program [18,19,20,21,22,23,24].

Learning and Study Skills Inventory

The Learning and Study Skills Inventory (LASSI) is an instrument intended to assess awareness and use of learning and study strategies. It has 80 multiple-choice items and return scores in ten scales: Anxiety, Attitude, Concentration, Information Processing, Motivation, Selecting Main Ideas, Self-Testing, Study Aids, Test Strategies and Time Management. It was used at Texas A&M University to assess growth in learning and study strategies of engineering majors from the first year to the junior year [25]

Study Process Questionnaire

The Study Process Questionnarie (SPQ) has been used in several forms to assess approaches to learning. The original instrument was intended to assess three different approaches to learning: shallow, deep, and achieving [26, 27]. A revised, shorter version was developed to assess only shallow and deep approaches to learning [28].

GAMES

The GAMES instrument was developed by Svinicki to assess the extent to which students employed five elements essential to self-regulated learning in the study habits: goal-oriented study, active study, meaningful and memorable study, explain to understand, and self-monitor successes and errors.

Need for Cognition Scale

Cacioppo & Petty, 1982; Cacioppo, Petty, & Kao, 1984; Cacioppo, Petty, Feinstein, & Jarvic, 1996; Evans, Kirby, & Fabrigar, 2003

Strategic Flexibility Questionnaire

Cantwell & Moore, 1996; Cantwell & Moore, 1998; Archer, Cantwell, & Bourke, 1999; Evans, Kirby, & Fabrigar, 2003

Reasoning about Current Issues Test

King & Kitchener, 2004

Motivated Strategies for Learning Questionnaire

Pintrich P., Smith D., Garcia T., and McKeachie W. (1991). A Manual for the Use of the Motivated Strategies for Learning Questionnaire. Technical Report 91-B-004. The Regents of The University of Michigan.

Instructional Approaches

Motivation

Motivation for lifelong learning results from the convergence of many different forces.

• "Accreditation mandates have brought to the forefront the need to be 'life-long learners'in the ever-changing and evolving engineering profession, coupled with the fast changing technologies and the need to accommodate a global society" [28].
• "In his 1991 President's Message, then Society of Manufacturing Engineering (SME) President, James F. Barcus Jr. commissioned a special committee for Life-long Learning and Career Development. About the work of this committee, he said, 'the committee believes life-long learning is emerging as the most important competitive consideration.' He went on to say '&the need for work-life quality that ensures maximum productivity takes on new meaning-and so does learning. In fact, learning how to learn may become our #1 priority'" [29].
• "A new class within the workforce has been identified as 'knowledge workers'&the key knowledge workers are engineers&Engineers must continually learn in order to stay abreast of the technologies that impact their jobs" [30].
• "Learning how to learn, and learning how to effectively use continuing education are two of the most important skills that an undergraduate engineering student can develop. The student will be an extremely valuable asset to any future employer and will remain employable throughout his/her career" [31].
• "The half-life of an engineer's technical skills is 2.5-7.5 years, depending on your discipline. This means that the vast majority of the technology that will exist in the latter part of a 40-year career has not yet been developed&During an engineer's career, he/she will develop some of this new technology. New tools and techniques will be used in daily work. Employers expect engineers to either learn this new information on their own or to find someone who can teach it to them" [31]. Other information on the rate of growth of scientific and engineering knowledge can be found in [32].
• "Finally it should be acknowledged that the greatest motivation for learning is learning itself. If a student can make the transition from extrinsic rewards (recognition, grades, etc.) to intrinsic rewards, then the basis for lifelong learning will have been established. In engineering, there is a joy of learning that is associated with knowing and predicting how the world works. Students need to have opportunities to experience this" [33]
  

Potential Resources

Under construction (20 April 2005)

References for Further Information

1. Learning Outcomes/Attributes, ABET iA recognition of the need for, and an ability to engage in life-long learning, accessed 23 November 2004
2. Felder, R.M., and Brent, R. (2003). Designing and Teaching Courses to Satisfy the ABET Engineering Criteria. Journal of Engineering Education, 92:1, 7-25.

   Abstract: Since the new ABET accreditation system was first introduced to American engineering education in the middle 1990s as Engineering Criteria 2000, most discussion in the literature has focused on how to assess Outcomes 3a3k and relatively little has concerned how to equip students with the skills and attitudes specified in those outcomes. This paper seeks to fill this gap. Its goals are to (1) overview the accreditation process and clarify the confusing array of terms associated with it (objectives, outcomes, outcome indicators, etc.); (2) provide guidance on the formulation of course learning objectives and assessment methods that address Outcomes 3a3k; (3) identify and describe instructional techniques that should effectively prepare students to achieve those outcomes by the time they graduate; and (4) propose a strategy for integrating programlevel and course-level activities when designing an instructional program to meet the requirements of the ABET engineering criteria.

3. Mourtos, N.J., (2003). Defining, Teaching, and Assessing Lifelong Learning Skills, Proceedings, Frontiers in Education Conference, accessed 23 November 2004

   Abstract: Lifelong learning skills have always been important in any education and work setting. However, ABET EC 2000 recently put a new focus on these skills in engineering education. Outcome 3i states the expectation that engineering graduates must have a recognition of the need for, and an ability to engage in lifelong learning. The paper first defines a set of attributes / skills, which are necessary for students to develop as lifelong learners. It is postulated that the recognition of the need requires skills in the affective domain, while the ability to engage requires skills in the cognitive domain. Next, the paper offers course design elements, which help students develop lifelong learning skills. Finally, the paper presents a method for assessing these skills. Assessment of data from a variety of engineering courses at San Jose State University are presented and analyzed. This assessment method can be used for any of the eleven outcomes in ABET EC 2000, criterion 3.

4. Todd, B., Engineering Module on Lifelong Learning, accessed 24 November 2004
5. National Research Council. (2001). Knowing What Students Know: The Science and Design of Educational Assessment. Committee on the Foundations of Assessment, James W. Pellegrino, Naomi Chudowsky, and Robert Glaser, editors, Board on Testing and Assessment, Center for Education, National Research Council.
6. Perry, W.G. (1970). Forms of Intellectual and Ethical Development in the College Years, New York: Holt, Rinehart and Winston, Inc.
7. Perry, William G., Jr. (1981). Cognitive and Ethical Growth: The Making of Meaning. In Arthur W. Chickering and Associates, The Modern American College (San Francisco: Jossey-Bass): 76-116.
   
8. Felder, R.M., (1997). Meet Your Students 7. Dave, Martha, and Roberto. Chemical Engineering Education, 31(2), 106-107.
9. Belenky, M. F., Clinchy, B.M., Goldberger, N.R., and Tarule, J.M. (1986). Women's Ways of Knowing: The Development of Self, Voice, and Mind. New York: Basic Books.
10. Baxter Magolda, M., & Porterfield, W. (1985). A new approach to assess intellectual development on the Perry scheme. Journal of College Student Personnel, 26 (4), 343-351.
11. Baxter Magolda, M. B. (1992). Knowing and reasoning in college: gender-related patterns in students' intellectual development. San Francisco: Jossey-Bass.
12. Baxter Magolda, M.B. (1999). Creating contexts for learning and self-authorship: Constructive-developmental pedagogy. Nashville, TN: Vanderbilt University Press.
13. Baxter Magolda, M.B. (2001). Making their own way: Narratives for transforming higher education to promote self-development. Sterling, VA: Stylus Publishing, LLC.
14. King, P. M. & Kitchener, K. S. (1994). Developing reflective judgment: understanding and promoting intellectual growth and critical thinking in adolescents and adults. San Francisco: Jossey-Bass.
15. King, P. M. and Kitchener, K. S. (2001). The reflective judgment model: Twenty years of research on epistemic cognition. In B. Hofer and P. Pintrich (Eds.), Personal epistemology: The psychology of beliefs about knowledge and knowing. Mahwah, NJ: Lawrence Erlbaum Associates.
16. Guglielmino, L. M. (1989). Development of an adult basic education form of the Self-Directed Learning Readiness Scale. In the H. B. Long and Associates, Self-directed learning: Emerging theory and practice. Norman OK: Oklahoma Research Center for Continuing Professional and Higher Education, University of Oklahoma.
17. Guglielmino, L.M. (1997). Reliability and validity of the Self-Directed Learning Readiness Scale and the Learning Preference Assessment. In H. B. Long & Associates, Expanding horizons in self-directed learning (pp. 209-222). Norman, OK: Public Managers Center, College of Education, University of Oklahoma.

18. Marra, M.R., Camplese, K. Z., and Litzinger, T. A. (1999). Lifelong Learning: A Preliminary Look at the Literature in View of EC 2000. Proceedings, Frontiers in Education Conference

    Abstract: ABET EC2000 and the "ABET 11" outcomes have initiated tremendous activity as engineering schools prepare for accreditation under the new criteria. While the new process and outcomes present many challenges to engineering faculty and administrators, the outcome on lifelong learning represents perhaps the greatest challenge; it states that graduates must demonstrate "a recognition of the need for, and an ability to engage in life-long learning". This outcome raises many questions including What constitutes life-long learning? How will we demonstrate that our graduates recognize the need for and have the ability to lifelong learn? And last but not least, how will we measure this attribute in our graduates? This paper summarizes the results of a preliminary literature review of lifelong learning as it pertains to engineering education and discusses plans for assessing lifelong learning of Penn State students, along with some data gathered in a survey of recent graduates.

19. Palmer, B., Marra, R.M., Wise, J.C., and Litzinger, T.A. (2000). A Longitudinal Study of Intellectual Development of Engineering Students: What Really Counts in our Curriculum? Proceedings, Frontiers in Education Conference.

    Abstract: In the early 1990s several national reports called for reform in engineering education and suggested that the current preparation of engineering students fell short of the skills and competencies that would be required of practicing engineers. Many engineering colleges across the country sought to address these problems with curricular reforms that incorporated more hands-on design work into the engineering curricula. The task of assessing the effectiveness of these design-infused curricula presents a critical challenge for engineering educators. At Penn State, we developed a longitudinal assessment program to evaluate the qualitative changes in students thinking as they progressed through the engineering curriculum. This paper presents a summary of the results of the first longitudinal component of this assessment.

20. Litzinger, T.A., Wise, J., Lee, S., Simpson, T., Joshi, S. (2001) Assessing Readiness for Lifelong Learning. Proceedings, ASEE Annual Conference and Exposition.

    Abstract: In general, lifelong learning can occur in two modes: formal and informal. Formal (or directed) modes include university courses or corporate training, whereas the informal modes, which occur naturally as part of learning to accomplish work tasks, are self-directed. The work presented in this paper focuses on assessment related to students ability to engage in self-directed learning and some early attempts at course enhancement to allow students to develop their abilities to engage in self-directed learning. The Self-directed Learning Readiness Scale (SDLRS) is used to assess of readiness for self-directed learning. In a preliminary study, this instrument was administered to approximately 60 senior engineering students to investigate the extent to which it correlated with academic performance as indicated by grade-point average. In a second study, the SDLRS is being taken by randomly selected first-year, sophomore, junior, and senior engineering students to determine how the readiness for engaging in self-directed learning changes during their engineering studies. Finally, two new, problem-based learning courses were implemented to enhance students learning as well as their readiness for self-directed learning. The students were given the SLDRS as a pre-test and post-test to determine whether the new courses enhanced their readiness for self-directed learning. These two new courses are briefly described and the results of the assessment are presented.

21. Wise, J., Lee, S., Litzinger, T.A., Marra, R.M., and Palmer, B. (2001) Measuring Cognitive Growth in Engineering Undergraduates: A Longitudinal Study. Proceedings, ASEE Annual Conference and Exposition.

    Abstract: This paper builds on previously reported findings1,2 by describing the completion of a four-year longitudinal investigation into the cognitive development of engineering undergraduates as measured using the Perry Scheme of Intellectual Development.3 Fifty-four students were randomly selected during their first year and invited to participate in three hour-long interview sessions. During the interview, each student reflected on his or her view of knowledge, education, and learning. The interviews were transcribed and sent to a rater experienced in assigning positions relative to the Perry Scheme based on student responses to these types of questions. While it was hoped that students would progress from simple dualistic views (position 1 / 2) through complex dualism (position 3) and relativism (4 / 5) to commitment in relativism (position 6+), most students in this sample did not make it beyond position four. This paper will review the findings with an eye towards curricular activities that may or may not encourage this type of growth.

22. Litzinger, T.A., Bjorklund, S., Wise, J.C. (2002). From Intellectual Development to Expertise. Proceedings, ASEE Annual Conference and Exposition.
23. Litzinger, T.A., Wise, J., Lee, S., and (2003). Assessing Readiness for Self-directed Learning, Proceedings, ASEE Annual Conference and Exposition.
24. Litzinger, T.A., Lee, S., and Wise, J. (2004). Engineering Students Readiness for Self-directed Learning. Proceedings, ASEE Annual Conference and Exposition.

    Abstract: The study summarized in this paper extends the previous work of the authors that attempted to determine whether capstone engineering courses have an effect on readiness for self-directed learning. The previous study suffered from a poor participation rate and several other potential problems. A new experimental design eliminated these problems. Pre-test and post-test data were collected in two sections of a capstone course in Mechanical Engineering. Results show no statistically significant change in the average pre-test and post-test scores; however, a fraction of the students were found to experience significant increases and decreases. A regression analysis was conducted in an attempt to understand the effect of the characteristics of the students such as gender and grade point average as well as project and section; however, no statistically significant correlation between the change in SDLRS score and any of these factors were found. Interviews with instructors were also conducted and suggested that the decreases in the scores for one project were likely due to the nature of the interactions of the project mentor with the students. Implications of the results of this study for curricular design are discussed.

25. Fowler, D., Maxwell, D., Froyd, J.E. (2003). Learning Strategy Growth Not What Expected After Two Years through Engineering Curriculum. Proceedings of the ASEE Annual Conference and Exposition.

    Abstract: As the pace of technological development continues to increase, consensus has emerged that undergraduate science, technology, engineering and mathematics (STEM) curricula cannot contain all of the topics that engineering professionals will require, even during the first ten years of their careers. Therefore, the need for students to increase their capability for lifelong learning is receiving greater attention. It is anticipated that development of this capability occurs during the undergraduate curricula. However, preliminary data from both first-year and junior engineering majors may indicate that development of these competencies may not be as large as desired. Data was obtained using the Learning and Study Skills Inventory (LASSI), an instrument whose reliability has been demonstrated during the past fifteen years. The LASSI is a ten-scale, eighty-item assessment of students awareness about and use of learning and study strategies related to skill, will and self-regulation components of strategic learning. Students at Texas A&M University in both a first-year engineering course and a junior level civil engineering course took the LASSI at the beginning of the academic year. Improvements would normally be expected after two years in a challenging engineering curriculum. However, data on several different scales appears to indicate that improvements are smaller than might be expected.

26. Biggs, J.B. (1987). Student approaches to learning and studying. Camberwell, Victoria: Australian Council for Educational Research.
27. Biggs, J.B. (1987). The Study Process Questionnaire (SPQ): Manual. Hawthorn, Victoria: Australian Council for Educational research.
28. Biggs, J.B., Kember, D., & Leung, D.Y.P. (2001). The revised two-factor StudyProcess Questionnaire: R-SPQ-2F. British Journal of Educational Psychology, 71, 133-149.
29. Barcus, J. Jr. (1991). Lifelong Learning: The Lifeblood of Manufacturing. Manufacturing Engineering, Society of Manufacturing Engineering, 4.
30. Wells, D.L., and Langenfeld, G.P. (1999) Creating an Environment for Lifelong Learning. Proceedings, ASEE Annual Conference and Exposition, accessed 29 March 2005.
31. Todd, B. (2001) Instructor's Guide, Lifelong Learning Module, accessed 29 March 2005.
32. Wright, B.T. (1999). Knowledge Management. Presentation at meeting of Industry-University-Government Roundtable on Enhancing Engineering Education, May 24, 1999, Iowa State University, Ames, Iowa.
33. Parkinson, A. (1999). Developing the Attribute of Lifelong Learning. Proceedings, Frontiers in Education Conference, accessed 29 March 2005.
    

Resources

Summary Report: SUCCEED Coalition Course Design Workshop, Richard M. Felder, Rebecca Brent, offered 2 March 2001, accessed 24 November 2004

A half-day workshop on designing and re-designing courses to address Engineering Criteria 2000 was held on March 2, 2001, at McKimmon Center, North Carolina State University. Richard Felder and Rebecca Brent, co-directors of the SUCCEED Coalition Faculty
Development program, presented the workshop. The following questions were discussed in the workshop:

• What learning outcomes are specified in Criterion 3 of ABET Engineering Criteria 2000?
• How should course goals and instructional objectives be formulated to address departmental requirements and EC2000 outcomes?
• Which EC2000 outcomes are addressed by traditional instructional methods?
• What instructional methods can be used to address the other outcomes?
• Which EC2000 outcomes are addressed by traditional assessment methods?
• What assessment methods can be used to address the other outcomes?
• How can individual course objectives and outcomes be assembled to assure that all EC2000 outcomes in Criterion 3 are being addressed satisfactorily by an instructional program.

Felder, R., and Brent, R. (2005). Understanding Student Differences. Journal of Engineering Education, 94(1), 57-72, accessed 31 March 2005.

Abstract: Students have different levels of motivation, different attitudes about teaching and learning, and different responses to specific classroom environments and instructional practices. The more thoroughly instructors understand the differences, the better chance they have of meeting the diverse learning needs of all of their students. Three categories of diversity that have been shown to have important implications for teaching and learning are differences in students learning styles (characteristic ways of taking in and processing information), approaches to learning (surface, deep, and strategic), and intellectual development levels (attitudes about the nature of knowledge and how it should be acquired and evaluated). This article reviews models that have been developed for each of these categories, outlines their pedagogical implications, and suggests areas for further study.

Felder, R. M., and Brent, R. (2004). The Intellectual Development of Science and Engineering Students. Part 1: Models and Challenges. Journal of Engineering Education, 93(4), 269-277.

Abstract: As college students experience the challenges of their classes and extracurricular activities, most undergo a developmental progression in which they gradually relinquish their belief in the certainty of knowledge and the omniscience of authorities and take increasing responsibility for their own learning. At a high developmental level (which few reach before graduation), they recognize that all knowledge is contextual, gather and interpret evidence to support their judgments from a wide range of sources, and willingly reconsider those judgments in the light of new evidence. This paper reviews several models of intellectual development, discusses their applicability to science and engineering education, and defines the difficulties that confront instructors seeking to promote the development of their students. A subsequent paper formulates an instructional model for promoting development that addresses those difficulties.

Felder, R. M., and Brent, R. (2004). The Intellectual Development of Science and Engineering Students. Part 2: Teaching to Promote Growth. Journal of Engineering Education, 93(4), 279-291.

Abstract: As college students experience the challenges of their classes and extracurricular activities, they undergo a developmental progression in which they gradually relinquish their belief in the certainty of knowledge and the omniscience of authorities and take increasing responsib ility for their own learning. At the highest developmental level normally seen in college students (which few attain before graduation), they display attitudes and thinking patterns resembling those of expert scientists and engineers, including habitually and skillfully gathering and analyzing evidence to support their judgments. This paper proposes an instructional model designed to provide a suitable balance of challenge and support to advance students to that level or close to it. The model components are (1) variety and choice of learning tasks; (2) explicit communication and explanation of expectations; (3) modeling, practice, and constructive feedback on high-level tasks; (4) a student-centered instructional environment; and (5) respect for students at all levels of development.

Jalkio, J. A. (2002). Using Self-Evaluation and Student Generated Portfolios for Assessment of Student Learning and Course Effectiveness, Proceedings, ASEE Annual Conference and Exposition, accessed 22 April 2005

Abstract: One advantage of having clearly articulated learning objectives for courses is that students can focus on these objectives to unify course material. Unfortunately, students often ignore the stated course objectives and focus their attention on the specific work required to earn good grades from the instructor. This paper presents a technique for shifting the student focus from the external validation of course grades to a self-evaluation of accomplishment of course learning objectives. Preliminary results from a classroom trial of this technique will be
presented.

This approach aims to tie the course grade directly to the students self-assessment. At the beginning of the semester, students are given a detailed list of course learning objectives and a grading rubric that relates letter grades to demonstrated levels of accomplishment of these objectives. During the course of the semester assignments are collected and graded to provide formative feedback to the students. At midterm and at the end of the semester, students are required to give the instructor a portfolio of work demonstrating accomplishment of the learning objectives and a summary evaluation specifying the letter grade earned and how the attached portfolio supports their self-assessment. The portfolio primarily includes graded examinations and reports, but can also include other material selected by the student. Summative feedback for the course is based directly on the student self-assessment.

In addition to focusing the students attention on course learning objectives, this approach has benefits for program assessment. The portfolio submitted by the student is documentation of successful accomplishment of course objectives and the student self-assessment provides useful information to the instructor on the efficacy of instructional methods and the adequacy of graded work in providing feedback to the student.

Jalkio, J. A., and Greene, C. S. (2004). Evaluation of the Accuracy and Effectiveness of Portfolio Based Student Self-Assessment. Proceedings, ASEE Annual Conference and Exposition, accessed 22 April 2005

Abstract: One advantage of having clearly articulated learning objectives for courses is that students can focus on these objectives to help them unify course material. Unfortunately, students often ignore the stated course objectives and focus their attention on the specific work required to earn good grades from the instructor. Although there should be alignment between these specific grading opportunities and the course objectives, the connections are frequently lost on the students. The authors have previously presented a technique for shifting the student focus from the external validation of course grades to a self-assessment of accomplishment of course
learning objectives. The current paper documents the effectiveness of the method based on data collected in twelve classes over three academic years by two professors and discusses enhancements that have been implemented.

The approach aims to tie the course grade directly to the students self-assessment. At the beginning of the semester, students are given a detailed list of course learning objectives and a grading rubric that relates letter grades to demonstrated levels of accomplishment of these objectives rather than to percentage of points earned. During the course of the semester assignments are collected and graded as usual to provide formative feedback to the students. Twice each semester students are required to give the instructor a portfolio of work demonstrating accomplishment of the learning objectives and a summary evaluation specifying the letter grade earned and, most importantly, how the attached portfolio supports their selfassessment.

This paper will examine the correlation of student self-assessment with traditional grading and evaluate its effectiveness in altering student focus from obtaining good grades to achieving course objectives. The use of these self-assessment reports and portfolios for course and program assessment as part of an ABET review will also be discussed.

Falchikov, N. and Boud, D. (1989). Student Self-Assessment in Higher Education: A Meta-Analysis. Review of Educational Research, 59(4), 395-430.

Abstract: Quantitative self-assessment studies that compared self- and teacher marks were subjected to a meta-analysis. Predictions stemming from the results of an earlier critical review of the literature (Boud & Falchikov, 1989) were tested, and salient variables were identified. Factors that seemed to be important with regard to the closeness of correspondence between self- and teacher marks were found to include the following: the quality of design of the study (with better designed studies having closer correspondence between student and teacher than poorly designed ones); the level of the course of which the assessment was a part (with student in advanced courses appearing to be more accurate assessors than those in introductory courses); and the broad area of study (with studies with the area of science appearing to produce more accurate self-assessment generally than did those from other areas of study). Results of the analysis are discussed and differences signaled by the results of the three common metrics examined. The distinction between relative and absolute judgment of performance is drawn. It is recommended that researchers give attention to both good design and to adequate reporting of self-assessment studies.

Evans, C. J., Kirby, J. R., and Fabrigar, L. R. (2003). Approaches to learning, need for cognition, and strategic flexibility among university students. British Journal of Educational Psychology, 73, 507-528.

Background. Considerable research has described students deep and surface approaches to learning. Other research has described individuals self-regulated learning and need for cognition. There is a need for research examining the relationships among these constructs.

Aims. This study explored relationships among approaches to learning (deep, surface), need for cognition, and three types of control of learning (adaptive, inflexible, irresolute). Theory suggested similarities among the deep approach, need for cognition, and adaptive control (aspects of self-regulated learning); and among surface, inflexible, and irresolute control (aspects of an ineffective approach to learning). One-factor and two-factor models were proposed.

Sample. Participants were 226 Canadian military college students.

Method. Participants completed the following questionnaires: the Study Process Questionnaire (Biggs, 1978), the Need for Cognition Scale (Cacioppo & Petty, 1982), and the Strategic Flexibility Questionnaire (Cantwell & Moore, 1996).

Results. Confirmatory factor analysis supported the identification of the six scale factors. Second order confirmatory factor analysis indicated three factors representing constructs underlying these factors.
Conclusions. Neither the one- nor two-factor models accounted adequately for the data. Self-regulated learning was defined by measures of the deep approach to learning, need for cognition, and adaptive control of learning. The second factor divided into one factor consisting of irresolute control, the surface approach, and negative need for cognition; and another consisting of inflexible and negative adaptive control. Substantial relationships among scales support the need for further theory development.

Cacioppo, J. T., & Petty, R. E. (1982). The need for cognition. Journal of Personality and Social Psychology, 42, 116131.

Cacioppo, J. T., Petty, R. E., & Kao, C. F. (1984). The efficient assessment of need for cognition. Journal of Personality Assessment, 43, 306307.

Cacioppo, J. T., Petty, R. E., Feinstein, J. A., and Jarvic, W. B. G. (1996). Dispositional Differences in Cognitive Motivation: The Life and Times of Individuals Varying in Need for Cognition. Psychological Bulletin, 119(3), 197-253

Abstract: Need for cognition in contemporary literature refers to an individual's tendency to engage in and enjoy effortful cognitive endeavors. Individual differences in need for cognition have been the focus of investigation in over 100 empirical studies. This literature is reviewed, covering the theory and history of this variable, measures of interindividual variations in it, and empirical relationships between it and personality variables, as well as individuals' tendencies to seek and engage in effortful cognitive activity and enjoy cognitively effortful circumstances. The article concludes with discussions of an elaborated theory of the variable, including antecedent conditions; interindividual variations in it related to the manner information is acquired or processed to guide perceptions, judgments, and behavior; and the relationship between it and the 5-factor model of personality structure.

Cantwell, R. H., and Moore, P. J. (1996). The Development of Measures of Individual Differences in Self-Regulatory Control and Their Relationship to Academic Performance. Contemporary Educational Psychology, 21(4), 500-517

Abstract: Two studies are reported describing the development and validation of the Strategic Flexibility Questionnaire (SFQ): a self-report instrument aimed at eliciting students beliefs about the need for, and conditional nature of, self-regulatory control over learning. In Study 1, 281 first-year university education students completed a 40-item pilot questionnaire. Factor analysis of responses revealed a 21-item instrument indicating three types of control beliefs: adaptive executive control, inflexible executive control, and irresolute executive control. In Study 2, the predictive validity of these conceptions was tested against the academic performance of 105 third-year university education students. Results indicated that students reporting adaptive executive control beliefs were more successful academically, while those students reporting inflexible or irresolute control beliefs were significantly less successful academically.

Cantwell, R. H., & Moore, P. J. (1998). Relationships among control beliefs, approaches to learning and academic performance of final-year nurses. The Alberta Journal of Educational Research, 44, 98102.

Archer, J. Cantwell, R., & Bourke, S. (1999). Coping at university: An examination of the achievement, motivation, self-regulation and method of entry. Higher Education Research & Development, 18(1), 31-54.

Abstract: Undergraduate university students (n=132) completed a questionnaire containing measures of psychological variables, verbal ability, an evaluation of their course of study, and demographic characteristics. We also had access to their academic results. We examined the relationships among these variables, especially the psychological variables, and compared them with those posited by recent theoretical work that makes connections among motivation, self-regulation, and self-efficacy. We then looked at which variables predicted academic achievement. The sample contained two sub-samples, mature-age students who had entered university via an enabling program; and younger students who entered university on the basis of high school results. With universities under severed financial pressure, university-run enabling programs must demonstrate their effectiveness in terms of students' successful progress in undergraduate degrees if they are to continue. We compared these two groups of students on the measures noted above.

King, P. M., and Kitchener, K. S. (2004). Reflective Judgment: Theory and Research on the Development of Epistemic Assumptions Through Adulthood. Educational Psychologist, 39(1), 5-18

Abstract: The reflective judgment model (RJM) describes the development of complex reasoning in late adolescents and adults, and how the epistemological assumptions people hold are related to the way they make judgments about controversial (ill-structured) issues. This article describes the theoretical assumptions that have guided the development of the RJM in the last 25 years, showing how these ideas influenced the development of assessment protocols and led to the selection of research strategies for theory validation purposes. Strategies discussed here include a series of longitudinal studies to validate the proposed developmental sequence, cross-sectional studies examining age/educational level differences, and studies of domain specificity. Suggestions for assessing and promoting reflective thinking based on these findings are also offered here.

Reflective Judgment Website

The Reflective Judgment Website contains information and references for the Reflective Jugment Model, the Reflective Judgment Interview, and the Reasoning about Current Issues instrument.