Archive for the ‘Assessment’ Category

Today’s DCAL workshop began with the image of a map, and a route marked from Hanover to Saginaw. The map inspired quite a bit of conversation before we settled in to the workshop proper. “Why that particular route to Saginaw?” one faculty member wondered. “There’ll be quite a traffic jam crossing into Canada at Niagara Falls,” another added. “Why not go through Cleveland?” I queried. Finally one faculty member asked, “Why go to Saginaw at all?”

Of course, the map had been offered as an analogy. The questions it raised might be applied to our teaching as well. When designing a course, we, too, are planning a journey from point A to point B. Our students may well ask, “Why that particular route?” or “Why take this journey at all?” The point of this workshop was to practice composing learning outcomes, so that these important questions might be answered.

Facilitators Tom Luxon and Prue Merton began the workshop by asking faculty to consider, in pairs, two course descriptions: one that focused entirely on course content, another that focused entirely on what students would be doing in the course. The two examples demonstrated powerfully how important it is to keep students at the heart of course design.

Once we’d been appropriately student-centered, we turned our attention to composing course objectives—a task that is always more challenging than it seems. The aim of our challenge was not simply to compose the objectives, but to assess them in light of Bloom’s Taxonomy of Higher Thinking. I enjoyed the opportunity to reconsider Bloom, who has been an important figure in my professional thinking. I found myself preoccupied by the fact that Bloom’s taxonomy is often interpreted hierarchically—i.e., one must know in order to understand; must understand in order to apply; must apply in order to analyze; and so on. I wondered, along with my colleagues, if learning indeed happens this way, or if it is in fact a messier process. I’m leaning towards the latter.

While we didn’t reach any conclusions, we raised some first-rate questions. Another excellent conversation at DCAL. In the end, we didn’t make it all the way to Saginaw. But we had the feeling that the journey was well begun.

Read Full Post »

How do we design an effective oral interview for assessing foreign language competence? The American Council on the Teaching of Foreign Languages (ACTFL) offers an extensive set of guidelines in its Oral Proficiency Interview Tester Training Manual. There is also the Common European Framework of Reference (CEFR) to offer some guidance as well.

In the DCAL workshop on February 20, 2011, a diverse community of 18 faculty members representing all of Dartmouth’s foreign language departments gathered to hear a presentation by Prof. Elizabeth Polli, Spanish Language Program Director, and Mario Ruiz Legido, Director of the Instituto Cervantes-Boston University, on these two sets of guidelines.

Mario Ruiz Legido began by discussing the Instituto Cervantes approach to Oral Assessment for the Spanish International Diploma (DELE), based on the CEFR. The CEFR was developed as a common base to guide and inform textbooks, curricula, exams, certification systems and other programs. The CEFR offers both coherence and transparency to foreign language programs throughout European countries. At its core, the CEFR is used to evaluates learners along a continuum of six steps, from basic users to proficient users. A series of “can do…” statements helps to define the expectations of the six steps.

The oral exam is a component of the global DELE exam, and takes between 10 and 20 minutes depending on the exam level undertaken. The candidate is examined by two examiners, one who is doing a holistic analysis and the other who is doing an analytics assessment. The goal is to confirm what the candidate CAN DO for his/her level. The process may include a personal monologue (2 min), a topical dialogue (2-3 min), a conversation with interviewer (3-4 min) and a role-play (2-3 min) at lower levels and additional types of tasks as levels increase. The exam is scored from 0-3, with 2-3 as passing scores.

In comparison, the ACTFL guidelines are quite different. The OPI is the only nationally recognized instrument for measuring communicative competence in the U.S. It does not measure what the student has learned in the classroom, but does attempt to measure what a student can do in a particular language. The OPI is generally a 20-30 min 1-on-1 recorded session. It looks at the individual’s ability to use language in real life situations. It measures ability against the specific tasks, not against other learners. It is a series of evolving questions throughout the interview, rather than a set of specific tasks or questions. It relies on having a very highly skilled interviewer, and the training process for interviewers alone can be 3-4 days long.

While knowledge of both the OPI and CEFR is certainly beneficial for classroom instructors, the goal, though, is not to teach to the exam. The challenge for instructors is to consider the ways in which they can design classroom activities that will encourage the kind of proficiency that is measured by these assessment tools.

Read Full Post »

Michael Chaney (English) and Leslie Butler (History) led a discussion packed with advice and information for anyone looking down the road toward getting tenure at Dartmouth or any other research university that values teaching. Michael had fifteen specific points of advice, but he began by reviewing the language in Dartmouth’s Faculty Handbook from the section on “promotion and tenure”:

Specific evidence of outstanding performance in scholarship and teaching is essential…. It is difficult to define outstanding teaching in specific terms. Comparative judgment by current and former students and by faculty colleagues is a necessary part of weighing the candidate’s performance against the standards of the College. Consideration will be given primarily to classroom instruction, but work with individual students and a creative role in course and program development will be fully recognized. (Faculty Handbook 35-36 )

In a brief analysis, Michael made a few observations about teaching and the tenure process as described in the Handbook. It appears that

  • Scholarship and teaching are the two most important components to a tenure decision
  • The handbook declines to define “outstanding teaching in specific terms”
  • The teaching activities that count for tenure are primarily those one can observe in the classroom

These observations give rise to familiar questions, like

  • Which is more important in a tenure decision, scholarship or teaching?
  • Why do we tend to assume these are competing rather than integrated activities?
  • If outstanding teaching is “difficult to define,” then against what standards to students and colleagues make the judgments that determine who gets tenure?
  • Why is classroom performance rather than other evidence like course design, assignments and assessments the focus of evaluation?

These questions prompted energetic discussion amongst the dozen or so participants, most of whom this year were graduate students. (I wondered why so few assistant professors showed up for such an important discussion.)

Some of the helpful recommendations Leslie and Mike advanced include the following:

  • Identify and use the resources that support your teaching: DCAL, Educational technologists, librarians and colleagues.
  • Share your ideas with colleagues and borrow from them.
  • Limit the time you devote to preparing for each class; stop worrying so much about what you are going to do or say in class and concentrate on what you plan to ask students to do.
  • Listen to your students, even their course evaluations; ask for feedback in the middle of term, not just at the end.
  • Tell students frequently why you do what you do, assign what you assign and evaluate it the way you do; this will help them write more intelligent and useful course evaluations.
  • Manage your teaching schedule to allow maximum time for scholarship; repeat courses; avoid trying to teach two new courses at once.
  • Teach outside your area of expertise when you can; it helps you reflect on how you learn.
  • Teach students how you do your scholarship; focus on methods and skills more than content. (That does not mean ignore content!)
  • Make sure you know what your department and division expects from you regarding teaching; how many courses, how often, what levels; these vary widely across the university.
  • Figure out your strengths and play to them. Do more not with less, but with what you already do.
  • Explain to your students that strong criticism of their work is meant as a sign of respect and an invitation to engage with you; otherwise they might take it the wrong way; all praise all the time leads to vacuous evaluations, however positive.
  • No one will be promoted and tenured without a strong record of recognized scholarship, but many department committees consider a candidate’s teaching evidence first and it can color how they look at the scholarship.
  • Like winter in northern New England (do winter or it will do you), embrace the high expectations for teaching at Dartmouth and the satisfaction you get can energize your scholarship.

Read Full Post »


yeah, it's long -- I didn't have time to make it shorter

Feral Librarian

Research libraries & higher education. Sometimes music, sports, & other stuff.