|
|
|
|
|
|
|
|
IN THIS presentation I briefly discuss both the circumstances that led one particular university to use proficiency as the organized principle for the evaluation of students satisfying its language requirement and the procedures that have been established to carry out the policy.
Let me begin with a short sketch of the setting. The University of Pennsylvania, an Ivy League institution, consists of four undergraduate schools and a number of graduate and professional schools clustered together on a single campus in Philadelphia, with the original School of Arts and Sciences as its core. Traces of its Quaker origins are still discernible, I believe, in the way decisions are generally reached by consensus.
My narrative really starts in 1974. Penn had retained its language requirement through the years when many other institutions had either lowered or dropped their. Language teaching and learning at that time were thoroughly traditional—that is the approach was predominantly grammar-translation, and the language requirement was based on time as expressed in semesters of study. Then Vice Provost Humphrey Tonkin (now president of the University of Hartford) and I conceived the idea of establishing a new kind of committee. Constructed in a particular way, the Language Committee would consist not of senior professor serving as representatives of the language and literature departments but, rather, of the people in the trenches, the coordinators of the various language programs and other faculty members concerned with language teaching and research. In its current form the Language Advisory Committee consists of the departmental language coordinators (five of whom are ACTFL proficiency trainers); faculty members from the linguistics department, the Graduate School of Education, the English Program for foreign Students, and the Lauder Institute; and administrators from the Advising Office, the Office of Academic Computing, and the Penn Language Center 1 —some twenty people in all, each directly involved in the operation of one or more language programs or in research in language acquisition. this committee both advises the dean of arts and sciences and supervises the activities I describe here.
Two other relevant aspects of the University of Pennsylvania in the 1970s need to be identified. First, in 1972 all the language-teaching departments and the department of linguistic moved into a single building; as present chair of the Language Committee, I cannot overestimate the value that such proximity has had on the development of common goals in language teaching and learning at our institution. Second, I responded in 1975 to a suggestion from a colleague in the Graduate School of Education that we add to our new committee a recent graduate, Barbara Freed. I did so, and nothing has been the same since. In a variety of situations—from an initial and radical experiment, a one-day workshop for language teachers (which I will describe in more detail), to Freed's appointment as vice dean for language instruction and to the introduction of proficiency as an organizing principle in our language programs—her combination of expertise and diplomacy was essential at whatever success our language programs have had (see Freed). Moreover, the measures taken received the full budgetary support of three successive deans, Robert Dyson (now director of the University Museum), Vartan Gregorian (now president of Brown University), and Joel Conarroe (now president of the Guggenheim Foundation). I make the point here not only to acknowledge the vision of these scholar-administrators but to serve notice that the program I am about to describe is labor-intensive and therefore cannot avoid having an impact on the budget.
In 1978 the Language Committee found itself confronting a problem. Entering students could be exempt from the language requirement—at that time, the first four semesters of language study—by scoring 650 on the College Entrance Examination Board tests in foreign languages. For students who could not achieve such a score and therefore had to take language courses, placement into a particular course level was determined by their score on the exam. As the result of a clearly provocative experiment in comparison (itself funded by the dean of arts and sciences), we discovered what many had long feared: a large percentage of those taking courses at Penn and thereby satisfying the language requirement could not achieve 650 on the CEEB tests. It was precisely while we were contemplating the need for a common measurement (dare I say “yardstick”?), in 1979, that Freed and other members of our French faculty were invited to the first college teacher's workshop involving the Foreign Service Institute and the Educational Testing Service. After ACTFL workshops and a great deal of other organizational work, in 1984 the French department adopted proficiency as a principle of evaluation in its courses. German, Arabic, and Russian departments soon followed; they have since been joined by Italian, Hindi, Spanish, and Chinese, with Hebrew, Japanese, and other less commonly taught languages now involved in various experiments and projects aimed in that direction.
It was clear from the outset that any system we set in place would have to separate the issue of proficiency rating from the use of grading as an evaluation of course performance. The method that was decided on has the following features (I cite from the actual documentation that students in Arabic receive): 2
At three equidistant stages in the sequence of language-requirement courses, you will take a proficiency examination (the categories of which are laid out in full in table 1); note that there is no proficiency examination in the first semester of study. The exam will not be given as either a midterm or a final. It will not be a test of your knowledge of the “textbook” for the course (if there is one) but rather of your ability to use the language in the native-speaking environment [see appendix]. The Minimal Functional Proficiency Standards listed in table 1 are those required at the level for completion of the language requirement in Arabic (or for exemption from it).
There are, of course, many issues for discussion and debate regarding the basic principle(s) involved in this system—proficiency—and in the way that we applied it. At this juncture, I will stress only a few points. First, the skill ranges listed in the tables privilege reading and place the oral-aural skills in the middle; the writing skill demands the lowest range of levels. Second, the difference between testing procedures that are achievement-based and those that are proficiency-based is clearly identified to the students, and, as noted in the information given to students, the two categories are kept procedurally and chronological separate. Third, students who accrue a score of 40 and are thus declared “proficient” under this system are entitled to receive a grade for the course in which they are enrolled, on the basis of the usual criteria (classroom participation, quizzes, written assignments, etc.). 3 Fourth, students can take individual skill segments again in order to accrue sufficient points to obtain a high enough score on the proficiency profile. Fifth, this procedure has establishment minimum proficiency levels and tests the ability of students to meet them. In answer to what is probably an inevitable question, first-time failure rate for our students hovers around the two percent mark. As an important but separate point, I note that the increased rigor of this evaluation process not only rendered the often-preferred student option for a pass-fail grade basically irrelevant but also led us to discover almost immediately the percentage of learning-disabled and, especially, dyslexic students admitted to our university. After a national conference on that topic, in fact, we adopted a diagnostic testing procedure (with both linguistic and psychological components), and an exemption policy.
In retrospect, initiating this system may have been the easiest part. It will surprise none of you that we have expended considerable effort since 1984 on the training of teachers, the search for appropriate materials (with particular focus on the large number of less commonly taught languages offered at the university), and the designing of test instruments in skills other than speaking, for which the ACTFL oral proficiency interview is used.
The original one-day teacher-training workshop of the 1970s has expanded into three separate workshops held annually: a five-day workshop in August for all new language teachers—faculty members and teaching assistants; an October three-day oral proficiency interview workshop (run by the language coordinators on the faculty who are ACTFL trainers, and required of all faculty members and teaching assistants involved in proficiency-based courses); and (new in 1993) a two-day materials preparation workshop in January that will (at least initially) concentrate on the less commonly taught languages, including those of Eastern Europe, Central Asia, and Africa, and deal with such issues as curricular planning, materials selection and preparation, classroom strategies for proficiency-based teaching and learning, and assessment. These workshops are all conducted by members of the Penn faculty and are budgeted by the School of Arts and Sciences.
Because of Penn's prominence in proficiency-based teaching, faculty members have been busy in the preparation of proficiency-based materials and tests, especially in the less commonly taught languages. I myself have chaired the committee that prepared the ACTFL Arabic guidelines, have joint-written a proficiency-based Arabic syllabus, and am now involved in an ACTFL validation project for Arabic, Chinese, and Japanese. Mu colleague Vijay Gambhir has been similarly active on Hindi projects. Eugene Liu, coordinator and ACTFL trainer in Chinese, has been preparing materials for content-based courses, particularly in business. Our work has also resulted in the development of significant amounts of computer software, using both the IBM and Macintosh platforms (most recently including videodisk technology for listening comprehension).
As is often the case, most of our initial attention has been directed at the lower levels of language competence. From one point of view, such a concentration of effort has worked to our advantage. The very focus of the content in proficiency-based courses has enabled us to cater to the needs of a new and broader undergraduate clientele: both the nursing school and the Wharton School of Business have recently introduced language requirements into their curricula. However, beyond the lower levels of the proficiency scale, the use of the native-speaker construct as a yardstick challenges all of us—perhaps especially those of us involved in the teaching of the less commonly taught languages—to consider the upper levels of competence and the teaching and learning strategies by which learners can approach the top of the rating scale. The distressing results of John Carroll's 1967 study on frequently taught languages have often been cited as one of the starting points of the proficiency endeavor; from my perspective as a teacher of a less commonly taught language (Arabic), the attention devoted to the upper levels of competence remains relatively slight. The program at the Lauder Institute—combining an MBA degree in the Wharton School and an MA in international studies, with an entry requirement of Advanced proficiency in a foreign language of choice and an exit requirement of Superior—serves as an excellent testing ground for research into language teaching and learning at the higher levels on the proficiency scale. As these experiments proceed and as Penn's schools and the university as a whole, perhaps universities in general, strive to internationalize their curricula—a current administrators' buzz phrase of variable and uncertain significance and implication—there is clearly a need for research on the relative efficacy of university language programs and study abroad in developing and maintaining language proficiency at the higher levels.
I have tried to illustrate the ways in which one university has dealt with the numerous issues connected with language instruction for American undergraduates, along with some of the procedures and policies that have emerged as corollaries of the system. Each institution, of course, will have particular college and departmental structures that require differing administrative mechanisms to monitor language programs and, where appropriate, to foster change. I hope that this description of one such system may have proved if not enlightening then at least stimulating.
The author is Professor of Arabic at the University of Pennsylvania. This article is based on a paper presented at ADFL Seminar West, 4–6 June1992, in Berkeley, California.
1 The Penn Language Center was initially established in the School of Arts and Sciences in 1989 to offer “nonfocus” languages, including Armenian, Yiddish, Korean, and Swahili—those that, while not required for any degree program, were still much in demand. This center offers a number of such languages (recently adding courses in the languages of Africa, Central Asia, and Eastern Europe to its listings) and provides instruction with specific content emphasis (e.g., medicine or business) and alternative means of evaluation (e.g., non-credit courses). The center has a director and advisory committee of its own and works in close collaboration with the Language Advisory Committee; the courses themselves are offered through the language departments and, when no appropriate language department exists, through the department of linguistics. While the courses often reflect a good deal of current research on language teaching and learning, the Penn Language Center is a course-offering institution within the School of Arts and Sciences and not a research and advocacy agency like some language centers that have recently been established at a number of academic institutions.
2 The proficiency levels in Arabic for satisfying the language requirement at the University of Pennsylvania are identical to those for French (within the same time frame). This any corollaries for academic language programs—particularly courses at the lower levels—that have been and may be deduced from the data contained in the Interagency Language Roundtable (ILR) listing of languages by categories (with Arabic joining Chinese, Japanese, and Korean in category 4, those that require the longest time to achieve an ILR level 3 [full professional, ACTFL-ETS Superior] competence) may need to be viewed with extreme caution. While presumably no one would challenge the notion that the diglossia of Arabic poses significant issues of language teaching and learning at ILR levels 2 and 3 (ACTFL-ETS Advanced and Superior), at which the replication of native-speaker behaviors becomes a requirement, the data from the Penn language programs indicate that the pattern of acquisition from level 0 to 1 and 1+ (ACTFL-ETS Novice to Intermediate-Mid and-High) does not differ in any substantial way between French and Arabic. In view of the strong disincentives that the category 4 listing suggests for college students who may wish to learn these non-Western languages, I believe that academic programs should emphasize that the ILR categories are not especially relevant to their programs and purposes.
3 The process described here focuses on the language requirement and the courses within which it is satisfied (normally the fourth semester of study, although some students manage to fulfill the requirement in the third semester). Evaluation in courses for earlier semesters of study is based on achievement criteria (although an oral proficiency interview is also part of the assessment mechanism).
I am sure that all of you have been used to taking a variety of tests in school for a very long time. In most cases (and particularly in older types of language tests) there is no possibility of choice or range: the answer is either right or wrong . The tests are also heavily based on specific learning materials found in something usually called a “textbook.” You may have realized by now that the Arabic program at Penn has carefully avoided referring to our books as “textbooks,” quite simply because they are not designed to be the only source of information. We call them a syllabus , by which we imply that they are certainly a guide for teaching and learning but that your instructor is free to adapt them and use them as she or he sees fit (within certain constraints established by the Arabic program itself, of course).
You will continue to be given the more traditional types of exams (vocabulary lists, quizzes on changing verb tense, etc.) as part of the evaluation process in your class. Such tests are called achievement tests, because they measure whether you have fully learned discrete bodies of information found in discrete segments of a published syllabus. However, all teachers in the elementary and intermediate Arabic courses will also use proficiency -based exam procedures—they will tests your ability to use the language outside the context of the particular syllabus we happen to have written and to apply the language for meaningful purposes. Designing those tests is not an easy task, and I will immediately state that we are in the process of experimenting with new types of tests.
Since using the language to communicate with a native speaker implies that you will be presented with unfamiliar topics, words, and so on, you should get used to encountering in tests topics and vocabulary with which you are not familiar. Do not be worried about this (“Oh, but we haven't had that word,” etc.). We are obviously not going to mark you down for not knowing a word you haven't seen before. What we are trying to discover is whether, on the basis of what you do know, you can manage to get the gist of something that you don't know. After all, won't such an ability be useful when you are in the Arab world at any stage of the process of learning the language?
Thus all lesson-tests will continue to measure what you know and will be announced as such ( achievement tests ). When proficiency tests are to be given, they will also be clearly designated as such. In them your teachers will be testing your ability to use the language to the limit at whatever your current level. To be fair to all students, we will score the answers on a range.
Carroll, John B. “Foreign Language Proficiency Levels Attained by Language Majors near Graduation from College.” Foreign Language Annals 1–2 (1967): 131–51.
Freed, Barbara. “Preliminary Impressions of the Effects of a Proficiency-Based Language Requirement.” Foreign Language Annals 20 (1987): 139–46.
|
Skill and method of testing |
Points | ||
|---|---|---|---|
| 5 | 10 | 15 | |
| Oral interview | Int.-Low a | Int.-Mid. | Int.-High |
| Listening comprehension | Int.-Low | Int.-Mid. | Int.-High |
| Reading comprehension | Int.-Mid. | Int.-High | Adv. |
| Writing | Nov.-High | Int.-Low | Int.-Mid. |
| a Int.=Intermediate; Nov.=Novice; Adv.=Advanced. | |||
| 1 | 2 | 3 | |
|---|---|---|---|
| Oral interview | Int.-High=15 | Int.-Low=5 | Int.Mid.=10 |
| Listening comprehension | Int.-Mid.=10 | Int.-Low=5 | Int.-Low=5 |
| Reading comprehension | Int.-High=10 | Int.-High=10 | Adv.=15 |
| Writing | Int.-Low=10 | Int.-Low=10 | Nov.-High=5 |
| Pass 45 | Fail 30 | Fail 35 |
© 1993 by the Association of Departments of Foreign Languages. All Rights Reserved.
|
|---|
|
|
|
