|
|
|
|
|
|
|
|
IN HIS survey of the current state of the foreign language curriculum, Dale L. Lange remarks that the challenges we leveled in our 1985 and 1988 papers against the guidelines for the Oral Proficiency Interview (OPI) “‘rattled’ the ‘proficiency movement’” (86) and that our criticism remains largely unanswered. Indeed, we wondered about the lack of a response from representatives of the oral-proficiency movement and expressed our concern in our 1988 study. As it turns out, at about the same time that Lange's paper appeared, the ADFL Bulletin published L. Kirk Hagen's thoughtful rebuttal to our criticisms of the OPI.
We are pleased to see counters like those Hagen presents, and we welcome his paper as an opportunity to sharpen our arguments and to eliminate any misconceptions we might have engendered. Hagen's comments broadly concern logic—ours and the OPI's—and the relation between linguistic knowledge and proficiency. We address his comments in that order.
One of Hagen's objections to our work is that we have improperly characterized the logic of the OPI and the ACTFL guidelines, mostly because of our errors regarding logic in general. In “Oral Proficiency Testing,” we claim that the ACTFL levels are entailments—more particularly, analytic, unempirical, arbitrary statements related less to facts than to definitions. We go on to make the stronger claim that the levels, as mere definitions, are useless and that it is thus premature to institutionalize them. Hagen criticizes our views of entailment, arbitrariness, and our supposed use of circularity.
First, let us consider entailment. Hagen is correct in claiming that our definition of entailment as simply true or false implication is impoverished. In the original manuscript of our 1985 paper, however, the discussion of logic was much more explicit, more in line with Hagen's comments. For the sake of potential readers, and at the recommendation of external reviewers, we toned down the formalism. No doubt, we oversimplified. Be that as it may, any shortcomings and flaws in our argumentation must, ultimately, lie on our doorstep and on no one else's.
Still, even though Hagen dislikes the connection between implication and entailment, there is no other way to view the matter. As a logical inference, entailment has the form of an implication, because the antecedent-consequent relation is a necessarily true one or a reversed necessarily false one (i.e., if the consequent is false, the antecedent must have been false; see e.g., Kempson 48–49). While Hagen's excursus into elementary logic is illuminating, it strikes us as beside the point. We in fact agree with everything he has to say about implication but note further that what he says does not change the OPI issues in the least.
Nonetheless, on rereading our 1985 paper, we see that we were probably responsible for Hagen's consternation regarding entailment. Line 8 of the first column on page 341 reads “since it could entail ‘jump’ or ‘leap,’” but we had intended to write “since it could be entailed by ‘jump’ or ‘leap,’” because that reflects our argument regarding the asymmetry of antecedents and consequents in entailments. This slip in wording is a critical error on our part, and if it is the cause of Hagen's objection, we apologize for the oversight.
The preceding discussion aside, the issues do not change a bit with a sharper definition of entailment. A particular level necessarily implies its criteria; alternatively, using the test of reversed falsity, we can say that if the criteria are false, then the level must be false. Our argument has always been that the logic of the OPI guidelines is logic, not fact. ACTFL makes up the world. If you are a Novice, then you are “thus and so” in reading, speaking, listening, and writing, because the category “Novice” entails its criteria; if you do not meet these criteria (i.e., if they are false), then you must not be a Novice. This reasoning looks like entailment to us.
Remarkably, even though Hagen criticizes our explanation of entailment, he never denies that the ACTFL levels are entailments. We can only conclude that he agrees with us but would like a few clarifications. It is particularly interesting to note that he relies on the OPI guidelines themselves to illustrate entailment: “If my French 101 students are Novices then they are not fluent” (47). This sentence is an entailment only in the world construed by the guidelines. Borrowing phraseology from our own paper, we can say that if students are rocket ships, rather than Novices, they are not fluent either, assuming that we choose to define the world in accordance with that statement. Hagen's example of entailment is, we believe, symptomatic of what we refer to in our 1988 paper as “the tail wagging the dog” (182), a problem we return to shortly.
Another of Hagen's objections is our use of the term arbitrary. He correctly quotes note 14 from our 1985 paper, where we claim that analytic statements can be arbitrary. A paragraph later, however, he implies that we have said that such statements are inherently arbitrary (47). Even more remarkably, nowhere does he explain what he means by “arbitrary.” In the first place, we never say that analytic statements are arbitrary. We chose the words can be because some analytic statements turn out to be empirically true: we just do not think that the analytic statements of the ACTFL guidelines are in that category.
Furthermore, we use the word arbitrary because we think that it nicely covers the characteristics of the analytic statements of the ACTFL guidelines: (1) true by fiat rather than by fact; (2) determined a priori, not through induction from empirically tested instances; (3) decided on the basis of what policymakers think a second language ought to be, rather than of how a second-language learner actually works; (4) motivated by ideology, policy, and the textbook market more than by fact. In short, the guidelines are groundless, made-up—that is, arbitrary.
When the original guidelines of the Foreign Service Institute assert, for example, that the ability to hire domestic help in another language is an indication of advanced proficiency, we think that the criterion is arbitrary. When information questions form the backbone of the analytic statements that motivate the OPI, even though no evidence from discourse analysis or pragmatics suggests that any single speech act forms the backbone of ordinary discourse, we think that the OPI is arbitrary. For this reason the guidelines seem suspect, and this is what we spent dozens of pages explaining outside note 14. Nonetheless, we would be happy to change the word arbitrary to a more appropriate term, such as uninformative or hegemonic. In our view, the ACTFL guidelines, because they are analytic statements, are simply ideas about how to proceed. A more flattering label might be heuristics. Certainly, as we state explicitly in our 1988 paper, the ACTFL guidelines are not principles.
Hagen's final objection to our logic is that we have misunderstood circularity and its relation to definitions; our conclusion “that the definitions of levels in the guidelines are circular” is way off base (47). Reading this, we feel like the narrator in T. S. Eliot's “The Dry Salvages”: we had the experience but missed the meaning. In the only place we discuss logical circularity, we claim that the pyramid structures underlying the ACTFL guidelines are circular, since one is used to justify the other (“Oral Proficiency Testing” 339). And that process is circular, even (and especially) according to Hagen's discussion of circularity. In two other places (“Proficiency” 182, 185), we describe the levels and definitions as circular, but we are not speaking of logic at these points. On the contrary, we think that these levels are wonderful definitions; that is why we say the same things Hagen does about symmetry and mutual substitutability (“Oral Proficiency Testing” 345 n22). 1
The ACTFL levels are definitions, lovely symmetrical definitions, but that is all they are. Because they are, as we have said in numerous places, analytically derived, they seem dubious to people like us, who think that knowledge involves responsibility to the facts. Language teachers, researchers, and administrators should not presume to call the ACTFL levels facts, and they should certainly not institutionalize the levels unless they are prepared to embrace authoritarianism, hegemony, and arbitrariness in the process.
In addition to criticizing our use of logic, Hagen objects to our caution not to institutionalize the OPI until the field learns more from linguistics about the components of linguistic competence. He wrongly accuses us of failing to present the theory of proficiency, which we supposedly intended to outline in “Proficiency.” In fact, as the statement Hagen quotes from our paper attests, we did not declare any intention to present a theory of proficiency: “Later in the same paper they state their intentions ‘to outline a picture of what a serious theory of [oral proficiency] should address’” (Hagen 49). An outline of what a theory ought to address is not the same thing as a theory. Lange recognizes this obvious difference in his remark that in 1988 we attempted to provide the beginnings of a different theory of proficiency rather than a full-blown theory (86).
At another point, Hagen faults us for not showing how our perspective on proficiency, which we began to develop in our 1988 essay, would “allow researchers to test objectively and empirically” the kind of claims we made about the construct. He further criticizes our failure to show that the claims are “relevant to the business of language testing” (50). Not only does he appear to have missed our point about the relation between language tests and real-world language use, but his commentary provides additional evidence for our tail-wagging-the-dog argument. To repeat this argument briefly, we believe that the OPI has been reified to the point that it now forces its developers to see the real world through the filter of the guidelines. This, in our view, reverses the way proper scientific research ought to proceed. Our argument all along has aimed at extricating proficiency from the OPI in order to arrive at an understanding of what proficiency means. If it turns out that proficiency cannot be “tested” according to the doxa of the testing profession, so be it. But we think that it is necessary to decide this issue before proceeding.
Finally, Hagen says that since we cannot measure the precise contribution each level of linguistic structure (morphology, syntax, phonology, lexicon, discourse, etc.) makes to communicative effectiveness, we cannot know that the OPI's emphasis on grammatical competence over sociolinguistic competence is “doomed to failure” (49). To be honest, we do not quite follow him here. To think, as researchers cited by Hagen have, that such measurements are possible is once again to buy into the quantitative paradigm. This, we agree, would be a fruitless, if not a ludicrous, endeavor.
It is not at all clear to us what the OPI stresses. If it does stress grammatical competence, as Hagen suggests, its view of grammatical knowledge is much less adequate than the definition advanced by even the least abstract linguistic theory. To assess grammatical competence, one must determine whether learners know not only what the possible grammatical sentences of a particular language are but also what the constraints on the impossible sentences of that language are. The OPI, as far as we can tell, does not explore the limits of competence through possible wrong answers, because it is directed toward correct performance only. If the guidelines are to be truly oriented toward grammatical competence, they need to test the degree to which a learner can make native linguistic judgments; in other words, the tester must investigate the limits of competence (a very old point about linguistic knowledge, made in the earliest Chomskyan literature).
The point is that even if we know that the OPI stresses grammatical competence (narrowly defined), the view of grammatical knowledge that informs the test is so impoverished that it does not tell us anything about grammatical knowledge. The same statement could be made about communicative competence, which is much more abstract than what ACTFL seems to have decided it is: speakers know implicatures, for example, but the test has nothing on this ability. We can only conclude that the guidelines have little, if any, basis in linguistic knowledge.
Some current empirical research suggests that the grammatical and communicative competencies of a person using a second language may necessarily be different from (i.e., shallower than) those of a native speaker (e.g., Coppieters's work showing speakers' divergent competencies despite equivalent proficiency levels). If so, one who has acquired a second language cannot have the depth of grammatical and communicative knowledge about that language that a native has. Ironically, then, the OPI, if it is a shallow test, may have stumbled on part of the truth rather than detected it on a principled basis. We think that it is better to make predictions than to wait for lucky breaks. 2
Proponents of the OPI would do well to pay close attention to the negative lessons derived from research in intelligence testing. From its inception, IQ research has been “strewn with the wreckage of misguided attempts at reification” of mathematical abstractions into a general intelligence factor with physical roots somewhere in the brain (Gould 268). But because of continued empirical research and theorizing—two activities that Hagen strangely contends must be subverted in favor of educational policy—psychologists have come to realize that the locus of intelligence is
neither wholly within the individual nor wholly within the environment, but rather within the interaction between the two.… Thus it may be difficult to understand intelligence fully without first considering the interaction of the person with one or more environments and recognizing the possibility that a person may be differentially intelligent in different environments, depending upon the demands of these various environments. (Sternberg 42)
Substitute proficiency and proficient for intelligence and intelligent , and you will get precisely the kind of framework for studying proficiency that we began to construct in our 1988 paper. It is critical to move outside the vacuum of the academic laboratory (i.e., the OPI) and into the everyday world of real people doing real things with other people through their language system(s).
A cogent example of the kind of research we have in mind is provided by Richard Schmidt's 1983 case study of Wes, a native speaker of Japanese attempting to carry on “the business of living” in the English-speaking milieu of Hawaii. Wes's English is woefully ungrammatical; yet his friends and acquaintances evaluate his English favorably and maintain that he is a master at “initiating, maintaining, and regulating relationships.” Moreover, several sociolinguists interviewed by Schmidt concur with this assessment, “sometimes proclaiming him a superior language learner who just doesn't care about grammatical do-dads, most of which are eliminated in normal speech anyway.” Only grammar teachers, and probably OPI testers, consider Wes “a disaster, possibly beyond rescue,” perhaps even a Terminal 2 (168).
Hagen's privileging of educational policy and his seemingly cavalier attitude toward linguistic research (50) buy into the very idea we are trying to dismiss: policy before truth. Decisions uninformed by facts are authoritarian. We agree that policy plays a role in testing, but only in the purposes of the tests, not in the content. The policy question here is whether the OPI works to fulfill the purpose of a reasonable proficiency test, to place persons according to how they can develop in a second language.
The authors are, respectively, Professor of Linguistics at Cornell University, and Professor of Linguistics at the University of Delaware, Newark.
1 We do say that circularity is analytic, since anything defined in terms of itself can be verified without empirical test (“Oral Proficiency Testing” 339). But we do not say that analyticity is circular. This would be false, since not all analytic statements are circular. Perhaps the reason Hagen believes that we call analyticity circular is that we stress analyticity so much throughout our 1985 paper. For the record (again), we do not believe that the ACTFL levels are circular in the logical sense; we believe that the justification for the pyramids is circular and that the levels are analytic rather than empirically based. We are grateful to Hagen for forcing these clarifications, even though the OPI's troubles are unaffected by the refinements in our phraseology.
2 As for principled work, Dandonoli and Henning have recently tested the content of OPI-ACTFL guidelines. We have no space to discuss their study, but their investigation is promising, though mixed in its results. For example, the findings for listening skills are inconsistent, the results do not fully hold for French and English, and tests on other languages may produce different results. We also have some concern about the experimental design and statistics. Nonetheless, the work is useful, in spite of its psychometric leanings, because it honestly examines both the successes and the failures of the OPI.
Coppieters, René. “Competence Differences between Native and Non-native Speakers.” Language 63 (1987): 544–73.
Dandonoli, Patricia, and Grant Henning. “An Investigation of the Construct Validity of the ACTFL Proficiency Guidelines and Oral Interview Procedure.” Foreign Language Annals 23.1 (1990): 11–22.
Gould, Stephen Jay. The Mismeasure of Man. New York: Norton, 1981.
Hagen, L. Kirk. “Logic, Linguistics, and Proficiency Testing.” ADFL Bulletin 21.2 (1990): 46–51. [Show Article]
Kempson, Ruth. Presupposition and the Delimitation of Semantics. Cambridge: Cambridge UP, 1975.
Lange, Dale L. “Sketching the Crisis and Exploring Different Perspectives in Foreign Language Curriculum.” New Perspectives and New Directions on Foreign Language Education. Ed. Diane W. Birkbichler. Lincolnwood: National Textbook, 1990. 77–109.
Lantolf, James P., and William Frawley. “Oral Proficiency Testing: A Critical Analysis.” Modern Language Journal 64 (1985): 337–45.
———. “Proficiency: Understanding the Construct.” Studies in Second Language Acquisition 10 (1988): 181–95.
Schmidt, Richard W. “Interaction, Acculturation, and the Acquisition of Communicative Competence: A Case Study of an Adult.” Sociolinguistics and Language Acquisition. Ed. Nessa Wolfson and Elliot Jud. Rowley: Newbury, 1983. 137–74.
Sternberg, Robert J. Metaphors of Mind: Conceptions of the Nature of Intelligence. Cambridge: Cambridge UP, 1990.
© 1992 by the Association of Departments of Foreign Languages. All Rights Reserved.
|
|---|
|
|
|
