Holistic Rating and Syntactic Maturity in Assessing ESL Students' Written Composition

来源 :外语学法教法研究 | 被引量 : 0次 | 上传用户:dknight123lin
下载到本地 , 更方便阅读
声明 : 本文档内容版权归属内容提供方 , 如果您对本文有版权争议 , 可与客服联系进行内容授权或下架
论文部分内容阅读
  摘要:本研究旨在考察总体等级评分(holistic rating)和句法成熟程度 (syntactic maturity)在评估学生作文之间的关系。 本研究通过计算句构成熟的五个指标(TNW, MTUL, MEFTUL, %EFTU and SR)来研究学生作文的句构成熟度。研究数据表明,根据总体等级评分标准所评估的作文分数同 MTUL 和MEFTUL这两个指标具有有意义的相关性,但是同TNW和%EFTU却存在无有意义的相关。根据这些发现,本研究提出了一些关于作文和课程设置方面的建议以供英语教师参考。
  Abstract:
   The study examines the relationship between holistic and objective measures in assessing ESL students' written composition. Five objective measures (TNW, MTUL, MEFTUL, %EFTU and SR) are calculated to measure the syntactic maturity of students' compositions. The data of the study indicate that significant correlations are found between holistically assigned scores and MTUL and MEFTUL. The holistic rating, however, is not significantly related to TNW, %EFTU and SR in the present study. According to these findings, implications are drawn for ESL composition and curriculum.
  Hunt (1965) and O'Donnel(1970) developed objective measures of syntactic maturity, which was defined as the ability to manipulate the syntax of the language can be measured by objective measures such as mean T-unit length (MTUL), mean error-free T-unit length (MEFTUL), percent of error-free T-units(%EFTU), subordination ratio(SR), average clauses length, average sentence length and coordination ratio and the rest. In terms of Hunt, T-unit is “minimal terminal syntactic maturity” i.e., any main clause with all of its modifiers (Nystrand, 1982). Many researchers favor the use of syntactic maturity, because it is easy to count, provides more objective quantifiable measures of significant structural characteristics(Kerek; 1979; Stewart, 1978; Kameen, 1983; Tomlinson & Straehley, 1978). On the other hand, other researchers points out the problems of syntactic maturity including the unreliability of T-unit(Witte, 1983; Reed, 1983; Sharma, 1979; Witte and Sodowsky, 1978; Chew, 1978) Thus, more studies need to be done concerning the objective measures of syntactic maturity. The present study attempts to provide more empirical data in the study of written compositions by examining the relationship between holistic and objective measures in assessing the written proficiency of college-level ESL students.
  Two key research questions are examined here: The first one is how holistic ratings as defined by high-rated students and low-rated students relates to syntactic maturity as measured by the total number of words(TNW), mean T-unit length (MTUL), mean error-free T-unit length(MEFTUL) and percent of error-free T-units(%EFTU) and subordination ratio(SR)? The second research question is how the holistically-assigned scores (N=20) relates to five objective measures of syntactic maturity? In the present study, four indices plus total number of words(TNW) were used to measure syntactic maturity of the compositions written by college-level ESL students.
  1.Method
  The analysis of the study involved the identification and tallying of six measures (TNW, MEDRUL,MTUL, MEFTUL, %EFUL, SR) used to characterize the students' written performance and functioning as dependant variables of this study, that is, the variables that were investigated in the study. The holistic rating of the students' composition (ten best compositions and ten worst compositions) was used as the independent variable of this study. The study attempted to investigate the relationship between these two kinds of variables to see whether they have any correlation with each other.
  1.1 Sample
  The subjects for this study were drawn from students enrolled in a college-level ESL composition course at an American university. These subjects came from different countries 20 students' compositions were examined in this research, which were 10 high-rated ones and 10 low-rated ones out of a total of 40 compositions. The essay prompt required them to describe three underlying rules shaping student behavior in American classrooms. They needed to show how American students behaved based on those rules by providing specific examples. In this case, the subject matter was controlled in the study.
  1.2 Independent Variables
  As noted earlier, the independent variables consisted of total number of words(TNW), mean T-unit length (MTUL), mean error-free T-unit length (MEFTUL) and percent of error-free T-units (%EFTU) and subordination ratio(SR). The TNW and MTUL were intended to measure the quantity of written performance. I hypothesized that the high-rated students would write more words and produce longer T-units than low-rated ones.
  In addition, the two variables, MEFTUL and %EFTU were evaluated in the study to measure the quality of written performance. Concerning these two variables, the hypothesis was that high-rated compositions might contain more error-free T-units than low-rated ones. Correspondingly, the percentage of error-free T-units of high-rated compositions would be higher than low-rated ones.
  As far as the subordination ratio was concerned, this variable was used as an additional measure of syntactic maturity of students'written performance, which was the number of subordinate clauses for every main clause or clauses per T-unit. I noted that some students' written proficiency was very low, even though they used a lot of subordinate clauses in their compositions according to my previous teaching experience and my study of students' compositions in the research. Moreover, the subordinate clauses they used in their compositions were not appropriate and full of mistakes. Therefore, I had my hypothesis that there was no significant difference between holistic rating and the number of subordinate clauses students used in their compositions.
  1.3 Dependant Variable
  The dependant variable of this study was the holistic rating of the instructor. The scores of 10 high-rated students ranged from 94 to 100 while those of 10 low-rated students varied from 73 to 81. Since these 20 compositions were graded by the same instructor both holistically and analytically, the intra-rater reliability was obtained in this study.
  1.4 Data Analysis
  Three procedures were carried out in analyzing statistical data related to holistic judgments of written performance and objective measures of syntactic maturity. First, the objective measures of TNW, MTUL, MEFTUL, %EFTU and SR were marked and tallied. Means of each objective measures were computed separately concerning high-rated compositions and low-rated ones. Next, bar graphs were drawn to vividly demonstrate the difference of TNW, MTUL, MEFTU, %EFTU and SR between high-rated students and low rated students in terms of mean and the percentage difference of each variable between high and low rated students. Finally, a Pearson product-moment was carried out to show how holistically assigned score correlated with each variable of objective measures.
  2. Results
  The first key research question in the study concerns whether there is any difference between high-rated students and low-rated students in terms of TNW,MTUL, MEFTUL,%EFTU and SR respectively. Thus, the analyses were illustrated in the following sections to reveal these relationships separately.
  Table I and II provided detailed information of the raw scores. TNW, MTUL, MEFTUL, %EFTU and SR of ten high-rated students and ten low-rated students respectively, both of which showed five objective measures of each student separately.
  Next, as revealed in Table III, there was a positive correlation between high-rated and low-rated students concerning five dependant variables (r=0.9999, p<0.0001), which was both high and significant. This finding indicated a high positive relationship between high-rated and low-rated students regarding five objective measures of syntactic maturity. That is to say, compared with low-rated students, high-rated students as a whole tended to produce more words, longer T-units, more error-free T-units and use more subordinate clauses. In addition, the Pearson correlation coefficient and probability value revealed that five variables inter-correlated with each other significantly. Besides, Table III also showed the percentage difference of each variable between high and low-rated students, which was computed by subtracting the mean of each variable of low-rated students from that of high-rated students and then dividing the number by the mean of the low-rated students. As noted in Table II, the biggest difference was found in MTUL(%44) followed by MEFTUL (36%), TNW(18%), SR(15%)and %EFTU (6%).
  
  Finally, concerning the second research question, Table IV revealed the results of correlations computed between the holistic ratings of all 20 students as a whole and five dependant variables. As noted in the Table, five objective measures indicated a positive correlation with the holistic rating of students' written performance; i.e., as students scored higher, the five objective measures tended to increase. The positive correlation confirmed that students with higher written proficiency had a tendency to produce more words, longer T-units and more error-free T-units and would like to incorporate more subordinate clauses. However, Table IV also demonstrated that only two variables, MTUL and MEFTUL correlated significantly with the holistic measure. In other words, there were significant positive relationships between the students' scores and their ability to perform on the written exam in terms of MTUL and MEFTUL. Interestingly, the percentage difference of each variable between high and low-rated students as mentioned in Table III and Bar Graph II also showed that high and low-rated students could be better differentiated as far as MTUL and MEFTUL were concerned, because their percentages were obviously bigger than those of other variables.
   Table IV
  Pearson Correlation for Written Performance Criteria and Holistic scores (N=20)
   3. Discussion
  The purpose of this study was to examine how objective measures of syntactic maturity related to the holistically-graded scores assigned to the compositions written by college-level ESL students. As noted earlier, three hypotheses were made on the relationship between five dependent variables and the holistic assessment of students' written work. Accordingly, the findings of the study were summarized in the following discussion in relation to three hypotheses followed by the implications of the study.
  3.1 Hypothesis 1
  The first hypothesis was concerned with two dependant variables, TNW and MTUL, which indicated the quantity of written performance. According to this hypothesis, the researcher assumed that students with high written proficiency would produce more words and longer T-units than those with low written proficiency. The data revealed a significant difference between MTUL and the scores, but not between TNW and the scores. Meanwhile, the highest percentage was discovered in MTUL among five variables in Table III. Specifically, ‘good’ students can be differentiated from ‘poor’ students with MTUL, but not with TNW. ‘Good’ students usually write longer T-units and but not necessarily produce more words. This finding correlates with that of other researchers. Stewart (1978) investigated the written syntactic maturity of students from grade ten through sixth year university and pointed out that words per T-unit appeared to be the best of the three measures of syntactic maturity (MTUL, Clause length and SR).
  3.2 Hypothesis 2
  The second hypothesis was about the quality of written performance as measured by MEFTUL and %EFTU. The research hypothesized that a significant difference could be found between these two variables and the holistic grading. The data revealed that MEFTUL correlated significantly with the scores while no significant correlation was found between %EFTU and the holistic scoring. Therefore, like MTUL, MEFTUL could serve as a good index to differentiate “good” students from “poor” students. The finding of the study was consistent with Sharma (1979) given that MEFTUL was found to be significantly correlated with the holistic rating. Nevertheless, as revealed in Pearson correlation for written performance criteria and holistic rating, MEFTUL didn't correlate with holistic rating as high as MTUL did(see Table IV), which was also indicated as far as the probability value is concerned. So it would be safe to say that MEFTUL was the second most reliable index of objective measures in differentiating students' written performance.
  3.3 Hypothesis 3
  The third hypothesis related to the use of subordinate clauses in students' composition, by which the researcher assumed that no significant difference existed between the variable of subordinate ratio and the holistic assessment based on the researcher's previous teaching experience and the observation on the compositions involved in the study. The finding of the study confirmed this hypothesis and revealed that there was no significant difference between subordination ratio and holistic rating. In this sense, SR is not a reliable index in determining written performance. This finding was in line with Chew(1978), who found that there was no direct correlation between a paper's quality and the ratio of clauses per T-unit. On the whole, the findings of the study correlate with many studies in that T-unit length is proved to be the most reliable index of syntactic maturity to differentiate between “good” and “poor” students with regard to written performance (Hunt,1965; O'Hare,1973; Tomlinson & Staehley,1978; Stewart,1978; Kameen,1983).
  4. Conclusions
   To conclude, the study revealed that significant correlations existed between holistic assessment and objective measures of syntactic maturity in terms of Mean T-unit length and Mean Error-Free T-unit length. Meanwhile, the study indicated no significant correlation between the use of subordinate clauses and students' holistically-assigned scores. Based on these findings, the researcher reinforced that sentence-combining exercises could be applied to “poor” students to increase their syntactic complexity. However, it is recommended that the technique of embedding subordinate clauses into a main clause may not benefit “poor” students. Therefore, this kind of practice should be avoided in ESL student composition class. In addition, the study indicated that counting objective measures of syntactic maturity could be used in ESL composition and curriculum.
  References
  Chew, C.(1978). A study to determine the relationship between indexes of syntactic maturity and a quality grade in a written examination. Retrieved November 7.
  Hunt, K.W. (1965a). Grammatical structures written at three grades levels. Research Report N.3, Urbana, Illinois: NCTE.
  Hunt, K.W. (1965b). Sentence structures used by superior students in grades four and twelve and superior adults. Cooperative Research Project, No.5-0312 ED 010 047.Tallahassee, Florida: Florida State University.
  Kameen, P. (1983). Syntactic skill and ESL writing quality. In A.Freedman, I. Pringle & J.Yalden. (Eds.) Learning to write: first language/second language (pp.139-158). London: Longman.(162-170)
  Kerek, A. (979). Measuring and evaluating syntactic maturity: some questions of methodology. Retrieved November 7, 2002 from ERIC database. ED 173783.
  O'Donnell, R.C; Griffin, W.J. and Norris, R.C. (1967). Syntax of kindergarten and elementary school children: a transformational analysis, NCTE Research Report No.8.Urbana, Illinois: National Council of Teachers of English.
  O'Hare, F. (1971). Sentence combining: improving student writing without formal grammar instruction. (NCTE Research Report No.150) Urbana, IL: National Council of Teachers of English.
  Reed, W.M.(1983). Daly and Miller's writing apprehension test and Hunt's T-unit analysis: two measurement precautions in writing research. Retrieved November 7, 2002 from ERIC database. ED 248539.
  Sharma, A.(1979).Syntactic maturity: assessing writing proficiency in a second language. Retrieved November 7, 2002 from ERIC database ED185105. Paper presented at the international conference on frontiers in language proficiency and dominance testing (3rd, Carhondate.IL, September 26-28, 1979).
  Stewart, N. (1978). Syntactic maturity from high school to university: a first look. Research in the Teaching of English 12.1, 37-46.
  Tomlinson, B & Straehley, M(1978). Developing and Measuring Mature Syntax. Retrieved November 7, 2002 from ERIC database. ED158244.
  Witte, S & Sodowsky, R (1978). Syntactic maturity in the writing of college freshman. Retrieved November 7, 2002 from ERIC database. ED163460.
  Witte, S.(1983). The reliability of mean T-unit length: some questions for research in written composition. In A. Freedman, I. Pringle, & J.Yalden. (Eds.) Learning to Write: First language/second language (PP. 172-177). London: Longman.
其他文献
摘要:认知语言学以经验主义认知观为理论基础,认为语言的形成和人类认知经验之间有密切的联系。从认知角度来研究隐喻在英语构词及词义拓展过程中所发挥的作用,词语的成因和发展轨迹可以帮助我们了解词语理据。隐喻思维不仅使人们能够用词根和词缀来组合构词,还能使词语引申出其它意义。通过分析构词及词义拓展中的隐喻机制及理据性,可以加深对词语的理解和记忆,会给词汇教学带来很多有益的启示。  关键词:隐喻;构词法;词
期刊
一、发展听、说技能必须以巩固、扩大学生的基础知识为前提  初中英语新大纲指出,英语教学“要努力使基础知识转化为言语技能”。反过来,要发展以听说为先的言语技能,就必须巩固、扩大学生的基础知识,以不断产生新的“转化”。这是因为我国学生不具备自然习得目标语的语言环境,“转化”的主渠道只能来自不断巩固,扩大的基础知识。  系统的语言基础知识体现了语言结构形式的变化规律。随着学生初中学习的不断深化,仅靠随机
期刊
传统的英语教育模式多采用“语法——翻译法”。其主要课堂教学活动包括:对整篇课文大意的译述,把课文逐句从外语译成母语,对课文中语法规则作演绎式的讲解以及直接阅读课文以加深对内容的理解等。从总体来看,逐句翻译和语法讲解时“语法——翻译法”课堂教学的中心活动,它占去课堂活动的大部分时间。这种教学方法的采用,使得课堂气氛较为沉闷,英语基础差的学生学习的热情不够,积极性不高。  在英语课堂教学中,如果教学形
期刊
摘要:考生必须认真掌握听力考试中的应试技巧,方能取胜听力、取胜英语、取胜高考。  关键词:听力 英语 高中     自从高考英语加试听力以来,听力试题做得成功与否直接关系到考生的英语成绩。  一、听力的具体特点  1. 语体、语音特点  作为语言运用能力的重要组成部分,高考英语听力以考查口头语言的理解为主要目的,要求考生能够听懂日常生活中的简短对话或独白。在语体方面,听力试题属于口语,它的结构不像
期刊
[摘要]我们的教育对象学生及其家长,对英语的重视程度还不够,势单力薄的英语教师群体要想提高自己的教学质量,需要做大量艰难的工作。我认为最应该抓的就是课堂教学,课堂教学是我们英语教学的主战场。   [关键词]小学英语 课堂 备课 动静结合 氛围    就目前现状来看,小学英语并没有与语文、数学放在同等重要的位置,尤其是我们的教育对象学生及其家长,对英语的重视程度还不够。势单力薄的英语教师群体要想提高
期刊
摘要:中等职业学校英语教师如何加强英语课堂教学方法创新,造就出真正具有初步运用英语进行真实交际的中等职业技术人才,在课堂教学方法的创新方面,有很大潜力可供挖掘。本文就英语课堂教学中需高度重视的环节进行探讨,在教学方法上加强创新研究,一些课堂教学方法的创新已经在实践中得到了很好的验证。实践已经证明,在教学实践中进行积极有效地灵活运用,能够取得更好的教学效果。  关键词:中职英语 课堂教学 创新研究 
期刊
摘要:本文主要讲的是通过高三英语课堂上的一些灵感发挥,激情调动,使学生的学习积极性大增,学习兴趣变浓,老师的创造性引领着同学们主动参与、善于思考、勤于动手的习惯养成,阐述了本人在繁忙的高三教学中,如何创造性地化解同学们的紧张情绪,轻松上阵,提高效率的自主性品质培养,和如何形成良好融洽的师生关系,在宽松、和谐的教学环境获取所学知识。  关键词:提神兴趣 姓名巧编 问题巧设 培养品质    提到高三年
期刊
摘要:本文对信息技术与英语教学的整合进行了理论和实践上的分析和探讨。从理论上对信息技术与英语教学整合做出分析,并结合本人实践,谈谈如何实践信息技术与英语课程的整合。最后提出了信息技术与英语教学整合后,对英语教师提出的新要求。  关键词:信息技术 英语教学 信息技术与英语教学的整合    社会生活的信息化和经济的全球化,使英语的重要性日益突出。英语作为最重要的信息载体之一,已成为人类生活各个领域中使
期刊
摘要:提高初中学生的阅读理解能力是初中英语教学的目标之一,阅读的过程是对书面信息进行认知构建的心理语言过程,是理解和吸收语言信息的重要途径之一。阅读能力的发展与听、说、写等技能紧密联系。通过阅读能得到必要的语言输入,扩大词汇量,增强语感,提高语言水平,为口语能力的发展打下基础;写作时才能言语地道,内容丰富。  关键词:初中英语 阅读兴趣 训练阅读 教学方法    阅读材料为阅读者提供了丰富语言背景
期刊
摘 要:21世纪,国际、国内激烈竞争对人才提出更高的要求,也对中学英语提出了挑战。为适应时代需要,要把素质教育渗透到教学中去,以培养出高素质、跨世纪的人才。  关键词:中学 英语教学 素质教育    历史跨入21世纪,国际、国内竞争将更加激烈,这种竞争对人才提出了更高的要求, 也对中学英语教学提出了挑战。面对这种形势,中学英语教学必须适应现代化人才培养要求的地位和作用,以培养出高素质、跨世纪的复合
期刊