|
|
|
|
|
|
|
|
A FOUR-SEMESTER foreign language exit requirement in one foreign language was recently reaffirmed for the College of Arts and Sciences (CAS) at the University of Arizona, and a universitywide foreign language requirement is now under consideration. The CAS requirement states that a student must “demonstrate proficiency at the fourth semester level,” and we are now faced with the problem of defining this level concretely in terms of specific competencies. 1
Our efforts to establish a competency-based requirement are motivated by the call for educational accountability in general, by “national need” (as expressed, for instance, in the Education for Economic Security Act), by common sense, and by a number of recent and not so recent clarion calls that requirements at all levels be based on “standards of competency rather than time spent in the classroom (“Global Competence”). The President's Commission on Foreign Language and International Studies, appointed in 1978, called for the establishment of “language proficiency achievement goals for the end of each year of study at all levels, with special attention to speaking proficiency” (23). And Lambert makes the most recent case for developing and using “a common metric,” equivalent across all languages for “measuring in an objective, consistent fashion the degree of proficiency a person–student or adult–has in a foreign language” (13).
Arizona is, of course, not alone in its efforts to define and measure proficiency. According to Draper et al., as well as Cummins, an increasing number of states (e.g., California, Colorado, Connecticut, Florida, Georgia, Massachusetts, Minnesota, Oregon, Texas, Washington) are calling for admission, exit, or certification requirements to be expressed in proficiency terms. While there has been some progress toward that goal with the adaptation of the ILR Oral Proficiency Scale by ACTFL-ETS and the development of the ACTFL Proficiency Guidelines, the profession is far from having available valid, reliable, and pragmatic competency measures that are acceptable for largescale academic use at the lower levels of instruction.
Bachman and Clark maintain that most currently available tests are inadequate as proficiency measures because “they are based on a model of language proficiency that does not include the full range of abilities required for communicative language use, and they are based on norm-referenced principles of test development that only permit interpretation of ability levels relative to the performance of specific groups of language users” (21). The authors also point to the complexity of the task of developing valid, reliable, authentic, and practical language tests. Yet, in spite of these and other reservations (see Bachman and Savignon; Kramsch; Jarvis), if we wait until all theoretical objections to proficiency testing have been satisfied, we will never get off the ground.
The University of Arizona, like many other institutions, is confronted with the problems of defining the notion of proficiency, identifying the level of competence our students should reach at various stages of formal classroom instruction, and deciding how those levels can be measured.
The German department of the University of Arizona already states in the catalog the minimum proficiency requirements for its majors (advanced on the ACTFL-ETS Oral Proficiency Interview and sehr gut on the Zertifikat Deutsch als Fremdsprache ). These levels of expectation, however, were based on considerations of how well our majors should speak, read, write, and so forth if they want to teach or use the language for other professional purposes, rather than on considerations of curricular or learner constraints (i.e., intelligence and motivation, time allowed, and quality and quantity of instruction) that determine their actual ability to communicate in the language. While we might be justified in setting instructional goals for majors based on future job requirements, we cannot avoid a careful analysis of instructional and learning constraints when we seek to determine competence levels expected of students fulfilling requirements in four semesters or two hundred hours of instruction, without having any experience abroad. 2
What are we going to expect of the large majority of students that take a language course mainly for the requirement? We could limit our expectations for non-majors to oral proficiency, specifying that students must be able to
create with the language by combining and recombining learned elements, though primarily in a reactive mode; initiate, minimally sustain, and close in a simple way basic communicative tasks; and ask and answer questions.
These are the intermediate-level speaker abilities as described in “ACTFL Proficiency Guidelines” (16). Studies conducted by Magnan and by Freed indicate that we can realistically expect students to master such skills after about four semesters of classroom instruction.
If oral language proficiency is indeed the major goal of college foreign language instruction, we know how to reach that goal most efficiently and effectively: take motivated students, put them into small classes in well-taught intensive or immersion programs—programs that include consciousness-raising on grammatical patterns as well as plenty of opportunities for communicative interaction—for about a year, and then send them abroad for about another year. But for most students at the University of Arizona this approach is neither feasible nor desirable. Thus, left with the mandate to define and test proficiency at the fourth-semester level, we drew up a plan for action that consists of the following steps:
Task 1. Review the literature on proficiency. Several faculty members in various departments are either trained ACTFL-ETS oral proficiency testers or have at least attended workshops or familiarization sessions on the topic and are reasonably well informed, at least about the ACTFL proficiency projects. We did, however, search the professional literature for accounts of other institutions or departments that had already started moving toward proficiency-based requirements. Our search revealed that the University of Pennsylvania and the University of Minnesota are among the few—if not the only—institutions making a systematic effort to develop proficiency-based requirements in all skills (see Freed; Foreign Language Proficiency Standards ). As reported by Jimenez and Murphy, other efforts are under way by the Spanish department at Stanford University, which has added a fifteen-minute oral interview to its placement test, requiring students to score at the ACTFL-ETS intermediate level or higher to fulfill the three-quarter foreign language requirement. Spanish majors at the University of Florida have to score at the advanced level on the OPI as part of their graduation requirement. The French department at the University of Illinois requires performance in the intermediate range after one year of instruction. James describes efforts at the German department at Hunter College that aim at an intermediate high after a four-semester requirement. And Fischer states that the German section of the department of foreign languages at Portland State University translates performance on an oral proficiency interview into a grade for first-year students, expecting an intermediate-mid for an A after one semester and an intermediate high for an A after one year.
The predominant concern of the published literature is with oral proficiency, and experience shows that for most students four semesters of university language instruction yield a performance in the intermediate range on the ACTFL-ETS scale. In a study conducted in Texas by Hiple and Manley, a sizable percentage of seniors majoring in a foreign language reached only intermediate high.
Task 2. Redefine the foreign language requirement in competence terms, using the ACTFL Proficiency Guidelines and departmental goals and objectives statements as points of departure. Our foreign language requirement is a humanities requirement. There is strong feeling among faculty members that required competencies should not be limited to oral proficiency but that students should be able to demonstrate specific abilities in all language skills, as well as some knowledge of the grammatical system of the target language or some knowledge of the target-language culture or both. After all, our requirement courses do not just fulfill a service function but have the more or less hidden agenda of attracting at least some of these requirement students, possibly as majors, into upper-level language, literature, and culture courses.
Given the differences in aptitude, ability, and motivation of students who study a language purely for a requirement, it would be unrealistic to expect all students to perform at the same level in all skill and knowledge areas unless expectations were stated at inordinately low levels. Students should therefore have a choice of what competencies they wish to have tested. (Thus, a student might be required to take a minimum of only four parts of the proposed six-part competence test, e.g., the listening and reading comprehension parts, communicative speaking, and cultural understanding.) Prospective majors may, however, be required to take the grammatical structures part of a test in addition to the parts testing skills for admission into the major program.
Task 3. Select possible item types (testing techniques) and procedures. Here I should note the major difficulties encountered in finding testing procedures that meet Bachman and Clark's criteria of authenticity and practicality. Authenticity refers to the similarity between a testing task and actual language use, and practicality refers to considerations of time, cost, facilities, ease and objectivity of scoring, and so on that will permit the test(s) to be administered to large numbers of students. Thus, while cloze tests have proved to be good measures of overall language proficiency, they fall short on the authenticity criterion. (How often in real life are we faced by the writing task of filling in the blanks?) Furthermore, a recent (as yet unpublished) study conducted by the department of German indicates that students generally dislike this testing technique. Dictations and summary protocols, individual interviews, guided letter writing, and so on would meet the criterion of authenticity but fall short in the area of practicality—particularly in programs with large enrollments—either requiring too much time for administration or being too subjective or time-consuming to evaluate.
Task 4. Determine implications, if any, for teaching and TA training. Most elementary modern foreign language courses at the University of Arizona already focus on communicative language use, and TA orientation programs and required methods courses exist in most departments. Internal course changes are more easily implemented in lower-division language courses taught by TAs than in upper-division courses taught by senior faculty members.
I would like to stress the importance of task 4 for departments moving toward proficiency-based instruction. The requirement that students demonstrate what they can do with the language in real-life or simulated communicative situations has profound implications for teacher and learner behavior, use of instructional time, materials, teaching techniques, testing, and so forth. The traditional textbook—with its grammatical sequencing; graded texts; sentence-based, discrete-point exercises; and early emphasis on structural accuracy—will require hefty supplementation in the form of contextualized, personalized role-play activities. These activities encourage spontaneous, creative language use; frequent opportunities for small-group communicative interaction; extensive—if not exclusive—target language use in the classroom; early and frequent work with authentic texts to augment language input; function and content-oriented exercises (rather than exclusively grammar-oriented exercises); and testing procedures in all skills that replicate, as much as possible, language functions in the real world.
Task 5. Create item banks for all skills and knowledge areas to be tested, making heavy use of computers for testing reading comprehension, discrete-point grammatical knowledge, and cultural awareness. Such an item bank will eventually permit easy development of new or alternative test forms.
Task 6. Pretest items in all levels of language instruction. This step is necessary to determine realistic expectations of competence levels for all parts of the test. Obviously, regardless of how much we would like our students to be “advanced” speakers, writers, and readers, we cannot require that level of competence unless at least seventy-five percent of them reach it through classroom instruction.
Task 7. Conduct item analysis and other statistical analyses on objective items and conduct interrater reliability studies on “subjective” testing procedures. These are important and necessary steps in establishing and protecting the quality of our program.
Task 8. Repeat pretesting with revised tests as a final quality check.
Task 9. Establish minimum competence levels on various skills and knowledge tests or on the overall composite score necessary to fulfill the language requirement.
Task 10. Establish remedial courses or policies for students who do not meet competency standards after four semesters of study or the equivalent.
Task 11. Implement a competency-based requirement with continuing development of item banks and continuing item analyses and interrater reliability checks for each administration.
Task 12. Develop an assessment guide for high schools and community colleges. Since the University of Arizona has a large percentage of transfer students and a considerable number of students who enter the university with a background in foreign language study, articulation is, of course, a major concern. For effective articulation, regular conferences need to be conducted with high school and community college departments to explain the competency-based requirement.
Task 13. Implement competency-based placement testing for students with prior language study. Entrance scores for various courses can be based at or above the average scores collected on various parts and in various courses during pretesting.
Early in the spring of 1987, there came an exciting call for a national effort in the area of proficiency testing in lower-level, postsecondary language instruction. An appeal by the Oregon State System of Higher Education to the Arizona Board of Regents resulted in my attendance at a one-day, intensive discussion between the Educational Testing Service (ETS) and representatives of twenty-one interested institutions or university systems nationwide. 3 The group called for a national effort in proficiency measurement, assisted by Em, with the following considerations in mind:
The benefits of potential access to testing instruments meeting the above criteria are obvious:
In conclusion, given the complex task involved in developing proficiency or competency-based requirements, the advantages of a national effort are obvious and exciting. Without it individual institutions may waste tremendous time, energy, and resources in duplicating a process and procedure that can be much more efficiently accomplished through a joint nationwide effort.
The author is Professor of German and Foreign Language Acquisition and Head of the Department of German at the University of Arizona. This article is based on a paper presented at the ADFL Seminar East, 4–6 June, 1987, in Charlottesville, Virginia.
1 While the terms proficiency-based and competency-based are used interchangeably in this paper, the current proficiency movement limits the meaning of proficiency to real-life communicative ability in the four language skills, that is, to what an individual can do with the language rather than what he or she knows about the language. Since the exit goals and measurements for the CAS foreign language requirement will most likely be defined in somewhat broader terms, including demonstration of knowledge about the target language and culture, competency-based requirement is the more appropriate term for our efforts at the University of Arizona.
2 For a more detailed discussion of the effect of curricular and learner constraints on the development of language proficiency in a classroom setting, see Schulz.
3 Participating institutions were the University of Colorado at Boulder, University of Delaware, UCLA, University of Arkansas, University of Oregon, Illinois State University, Southern Illinois University at Edwardsville, University of Utah, University of Houston, University of Arizona, Arizona State University, Northern Arizona State University, University of Minnesota, University of Oklahoma, City University of New York, University of Hawaii at Manoa, and the Florida State Board of Community Colleges.
University of Houston, University of Arizona, Arizona State University, Northern Arizona State University, University of Minnesota, University of Oklahoma, City University of New York, University of Hawaii at Manoa, and the Florida State Board of Community Colleges.
American Council on the Teaching of Foreign Languages. “ACTFL Proficiency Guidelines 1986.” Defining and Developing Proficiency: Guidelines, Implementations and Concepts. Ed. Heidi Byrnes and Michael Canale. Lincolnwood: Natl. Textbook, 1987. 15–24.
Bachman, Lyle F, and John L. D. Clark. “The Measurement of Foreign/Second Language Proficiency.” Annals of the American Academy of Political and Social Science 490 (1987): 20–33.
Bachman, Lyle, and Sandra Savignon. “The Evaluation of Communicative Language Proficiency: A Critique of the ACTFL Oral Interview.” Modern Language Journal 70 (1986): 380–90.
Cummins, Patricia W. “School-College Articulation and Proficiency Standards: A Status Report.” ADFL Bulletin 19.1(1987): 8–15. [Show Article]
Draper, James B., et al. “State Initiatives and Activities in Foreign Languages and International Studies.” Modern Language Journal 70 (1986): 236–41.
Fischer, William B. “Not Just Lip Service: Systematic Oral Testing in a First-Year College German Program.” Die Unterrichtspraxis 17 (1984): 225–39.
Foreign Language Proficiency Standards for Entrance and Graduation: College of Liberal Arts. Minneapolis: U of Minnesota, 1986.
Freed, Barbara F “Issues in Establishing and Maintaining a Language Proficiency Requirement.” Indiana University Foreign Language Proficiency Symposium. Bloomington, 4–6 March 1987.
———“Preliminary Impressions of the Effects of a Proficiency-Based Language Requirement.” Foreign Language Annals 20 (1987): 139–46.
“Global Competence: Education for the Future: Position Paper of the Joint National Committee for Languages.” ACTFL Public Awareness Network Newsletter 3 May 1987: 6.
Hiple, David V., and Joan H. Manley. “Testing How Well Foreign Language Teachers Speak: A State Mandate.” Foreign Language Annals 20 (1987): 147–53.
James, Dorothy. “Toward Realistic Objectives in Foreign Language Teaching.” ADFL Bulletin 16.2 (1985): 9–12. [Show Article]
Jarvis, Gilbert A. “Proficiency Testing: A Matter of False Hopes?” ADFL Bulletin 18.1 (1986): 20–21. [Show Article]
Jimenez, Reynaldo, and Carol J. Murphy. “Proficiency in Action.” Teaching for Proficiency: The Organizing Principle. Ed. Theodore V. Higgs. Lincolnwood: Natl. Textbook, 1984. 201–17.
Kramsch, Claire. “From Language Proficiency to Interactional Competence.” Modern Language Journal 70 (1986): 366–72.
Lambert, Richard D. “The Improvement of Foreign Language Competency in the United States.” Annals of the American Academy of Political and Social Science 490 (1987): 9–19.
Magnan, Sally Sieloff. “Assessing Speaking Proficiency in the Undergraduate Curriculum: Data from French.” Foreign Language Annals 19 (1987): 429–39.
President's Commission on Foreign Language and International Studies. Strength through Wisdom: A Critique of U.S. Capability. Modern Language Journal 64 (1980): 9–57.
Schulz, Renate A. “From Achievement to Proficiency through Classroom Instruction: Some Caveats.” Hispania 69 (1986): 187–93.
© 1988 by the Association of Departments of Foreign Languages. All Rights Reserved.
|
|---|
|
|
|
