论文部分内容阅读
【Abstract】In nature， writing is a linguistic， social-cultural and cognitive activity. Through assessment， students’ practical language competence could be reflected， what can also be reflected are the teacher’s success or teaching deficiency. This study takes assignments from two undergraduate classes， the contents and scores of the students’ writings are analyzed by reliability and validity， with the aim of obtaining a feedback which would be helpful to future teaching and learning.
【Key words】reliability； validity； writing
1. Introduction
Effective language test measures how many teaching goals have been achieved， how much knowledge students have learned in order to get positive washback effects of the instruction implemented. Writing is regarded as important aspect of language output. Writing test is valued as one of the most scientific and effective ways to evaluate learners’ language proficiency， and thus becomes an important and necessary component of many large-scale English tests.
2. Students’ writing test
Diverse types of language test exist and they differ from one another due to the test method and purpose. They can be classified into different perspectives， but the following four types are most common：proficiency test， diagnostic test， placement test and achievement test.
This paper is based on achievement tests taken by two English Major classes. In the test， students are asked to write compositions based on short stories from the textbook， and the composition should contain two parts：a short review and a literature criticism. The length of the composition is not strictly limited and the number of students is around 40.
2.1 Achievement test
Achievement test is generally divided into two types：final achievement test and progress achievement test. As the terms indicate， “final achievement test is conducted at the end of a study course”；while progress achievement test “measure the progress that students are making”. Undoubtedly， the content of these tests must be relevant to the given courses.
Different from proficiency tests， achievement tests have something to do with language courses， and the purpose of them are to assess “how successful an individual student， groups of students， or the courses themselves have been in achieving the objectives”.
2.2 Writing test
Writing test is used when the teacher wants to know whether the students have mastered what has been taught in class according to certain teaching objectives. An achievement test is related directly to the lessons， or even a curriculum. Therefore， a writing test should be narrowly focused on particular teaching and learning goals and curriculum， and hence it is an achievement test. Teachers tailor writing tasks by the goals of the course and the backgrounds and needs of the students， and build in both authenticity and interactivity into the writing test tasks. So， the writing assessment should be as closely related as possible to writing instruction and teaching curriculum so as to provide information for teachers to evaluate their students’ progress and abilities in writing.
2.3 Content-based writing
This kind of writing can be regarded as an achievement test， only that students make their compositions after class. Test takers（students） are required to write on the basis of given material.
3. Reliability and Validity of the writing content
Content is the most important element in the whole process， for if anyone cheats and gets high score， the reliability would be very low. So， we have to analyze it first to make sure the reliability is high， if not， the following analysis would be in vail， because the foundation of the analysis is based on honest writing.
3.1 The validity of traditional scoring
It is the fact that the whole writing process is finished after class， both teacher and the scorers can’t monitor the students and could never know if any of the students finish their assignments by copying others’. But， according to the nature of the writing， the first part of which is brief introduction of the given short story， some students would use words or sentences from it， and if this kind of act has caused some resemblances in some writings， it would not be counted as copying， although they use them without quoting marks.
3.2 The validity of computer-/internet-assisted scoring
There are at least three ways of computer-/ internet-assisted scoring：（1） hand in the assignment onto one particular website， in which only scorers have the access to the scoring end；（2） students e-mail the assignments to a particular address respectively；（3） students hand in assignments to the monitor in digital form， then the monitor sends them together to scorers’ e-mail address. All the three ways above have one advantage：it can avoid bad handwritings， so as to increase scorer validity and student reliability. But， at the same time， it will decrease student validity. So， it leaves to the teacher to balance.
In internet-assisted scoring， scorers sign in the teacher’s account and give scores respectively and manually. Some advantages are obvious：if any student copies others， the system would point it out by warning the assignment is dubious， so as to get a higher scorer validity. However， the disadvantage is also obvious：the writings are based on reading of given short story， so the contents are restricted and tend to be similar to one another. In this condition， although students do not copy others， the system would still view some of them as copied， for the reason that the computer system only count the same words used， but not regard the content as a whole. Another disadvantage would also occur when the total student number of the two classes is only around 40. All of the students make the composition on the computer， and the scorers give scores with the assistance of computer， all these together would take longer time than traditional form of handing the assignments in. So， take the small number of students into account， computer-/ internet-assisted scoring has low validity. 4. The relevance between handwriting and score
Handwriting do have influence in the scoring process， and sometimes scorers won’t examine the content of the composition and just give a very low score directly after seeing a very bad handwriting. So， handwriting is also a determinant of the final score.
4.1 Students’ handwriting and score
As students make their compositions under a pressure-free condition， they can bring into play their true language competence. However， they may show little care on their handwriting， spelling， grammar control， etc.， all these would lead to mistakes and score loss. Scorers cannot identify students’ words and would take scores off， this would be one of the most important reasons for students’ losing score.
4.2 Scorer Reliability and score
A phenomenon exists that the handwriting of any student is too bad to read. In fact， during the process of scoring， we can see that in some of the compositions with bad handwritings， the words are correct and their grammars have no obvious mistakes. So， conclusions could not be drawn by intuition that a student who has bad handwriting would also have bad grammar control. In this point， in the process of scoring， we have to make sure that the scorers do have patience and checked every piece of composition in detail.
4.3 Summary
Ill characters caused by bad handwriting can’t be judged as occurred by chance or by high frequency. In this case scorer reliability would be affected by their patience and prudence. The more patient and prudent the scorers are in the process of scoring， the higher reliability the scores have.
5. The relevance between writing content and score
Although the handwriting is very good， it can’t be counted as a good piece of composition， the content must be good. A cohesive and coherent content is what makes a high score. So， students should focus on the content before the handwriting. Of course， if a composition has both good content and good handwriting， it would be perfect.
5.1 The length of the content and score
The content of the composition is restricted， normally be a literature criticism on a short story in the textbook. There is no limited number of words， but two parts are to be included：brief review of the story and personal view. Each time students hand in their assignments on the same story， so that it is available to compare the scores between the students.
However， students’ writings differ in number of words. Some write a little more than 100， and some write as long as 300 to 400. If they make mistakes in the same rate， the student who write fewer words would score higher than the one who write a lot more. Also， the score is given by taking off 1 point from 1mistake each， that doesn’t take cohesiveness and cohesion of the content into account. So， in the perspective of writing ability， the final scores have low reliability.
【Key words】reliability； validity； writing
1. Introduction
Effective language test measures how many teaching goals have been achieved， how much knowledge students have learned in order to get positive washback effects of the instruction implemented. Writing is regarded as important aspect of language output. Writing test is valued as one of the most scientific and effective ways to evaluate learners’ language proficiency， and thus becomes an important and necessary component of many large-scale English tests.
2. Students’ writing test
Diverse types of language test exist and they differ from one another due to the test method and purpose. They can be classified into different perspectives， but the following four types are most common：proficiency test， diagnostic test， placement test and achievement test.
This paper is based on achievement tests taken by two English Major classes. In the test， students are asked to write compositions based on short stories from the textbook， and the composition should contain two parts：a short review and a literature criticism. The length of the composition is not strictly limited and the number of students is around 40.
2.1 Achievement test
Achievement test is generally divided into two types：final achievement test and progress achievement test. As the terms indicate， “final achievement test is conducted at the end of a study course”；while progress achievement test “measure the progress that students are making”. Undoubtedly， the content of these tests must be relevant to the given courses.
Different from proficiency tests， achievement tests have something to do with language courses， and the purpose of them are to assess “how successful an individual student， groups of students， or the courses themselves have been in achieving the objectives”.
2.2 Writing test
Writing test is used when the teacher wants to know whether the students have mastered what has been taught in class according to certain teaching objectives. An achievement test is related directly to the lessons， or even a curriculum. Therefore， a writing test should be narrowly focused on particular teaching and learning goals and curriculum， and hence it is an achievement test. Teachers tailor writing tasks by the goals of the course and the backgrounds and needs of the students， and build in both authenticity and interactivity into the writing test tasks. So， the writing assessment should be as closely related as possible to writing instruction and teaching curriculum so as to provide information for teachers to evaluate their students’ progress and abilities in writing.
2.3 Content-based writing
This kind of writing can be regarded as an achievement test， only that students make their compositions after class. Test takers（students） are required to write on the basis of given material.
3. Reliability and Validity of the writing content
Content is the most important element in the whole process， for if anyone cheats and gets high score， the reliability would be very low. So， we have to analyze it first to make sure the reliability is high， if not， the following analysis would be in vail， because the foundation of the analysis is based on honest writing.
3.1 The validity of traditional scoring
It is the fact that the whole writing process is finished after class， both teacher and the scorers can’t monitor the students and could never know if any of the students finish their assignments by copying others’. But， according to the nature of the writing， the first part of which is brief introduction of the given short story， some students would use words or sentences from it， and if this kind of act has caused some resemblances in some writings， it would not be counted as copying， although they use them without quoting marks.
3.2 The validity of computer-/internet-assisted scoring
There are at least three ways of computer-/ internet-assisted scoring：（1） hand in the assignment onto one particular website， in which only scorers have the access to the scoring end；（2） students e-mail the assignments to a particular address respectively；（3） students hand in assignments to the monitor in digital form， then the monitor sends them together to scorers’ e-mail address. All the three ways above have one advantage：it can avoid bad handwritings， so as to increase scorer validity and student reliability. But， at the same time， it will decrease student validity. So， it leaves to the teacher to balance.
In internet-assisted scoring， scorers sign in the teacher’s account and give scores respectively and manually. Some advantages are obvious：if any student copies others， the system would point it out by warning the assignment is dubious， so as to get a higher scorer validity. However， the disadvantage is also obvious：the writings are based on reading of given short story， so the contents are restricted and tend to be similar to one another. In this condition， although students do not copy others， the system would still view some of them as copied， for the reason that the computer system only count the same words used， but not regard the content as a whole. Another disadvantage would also occur when the total student number of the two classes is only around 40. All of the students make the composition on the computer， and the scorers give scores with the assistance of computer， all these together would take longer time than traditional form of handing the assignments in. So， take the small number of students into account， computer-/ internet-assisted scoring has low validity. 4. The relevance between handwriting and score
Handwriting do have influence in the scoring process， and sometimes scorers won’t examine the content of the composition and just give a very low score directly after seeing a very bad handwriting. So， handwriting is also a determinant of the final score.
4.1 Students’ handwriting and score
As students make their compositions under a pressure-free condition， they can bring into play their true language competence. However， they may show little care on their handwriting， spelling， grammar control， etc.， all these would lead to mistakes and score loss. Scorers cannot identify students’ words and would take scores off， this would be one of the most important reasons for students’ losing score.
4.2 Scorer Reliability and score
A phenomenon exists that the handwriting of any student is too bad to read. In fact， during the process of scoring， we can see that in some of the compositions with bad handwritings， the words are correct and their grammars have no obvious mistakes. So， conclusions could not be drawn by intuition that a student who has bad handwriting would also have bad grammar control. In this point， in the process of scoring， we have to make sure that the scorers do have patience and checked every piece of composition in detail.
4.3 Summary
Ill characters caused by bad handwriting can’t be judged as occurred by chance or by high frequency. In this case scorer reliability would be affected by their patience and prudence. The more patient and prudent the scorers are in the process of scoring， the higher reliability the scores have.
5. The relevance between writing content and score
Although the handwriting is very good， it can’t be counted as a good piece of composition， the content must be good. A cohesive and coherent content is what makes a high score. So， students should focus on the content before the handwriting. Of course， if a composition has both good content and good handwriting， it would be perfect.
5.1 The length of the content and score
The content of the composition is restricted， normally be a literature criticism on a short story in the textbook. There is no limited number of words， but two parts are to be included：brief review of the story and personal view. Each time students hand in their assignments on the same story， so that it is available to compare the scores between the students.
However， students’ writings differ in number of words. Some write a little more than 100， and some write as long as 300 to 400. If they make mistakes in the same rate， the student who write fewer words would score higher than the one who write a lot more. Also， the score is given by taking off 1 point from 1mistake each， that doesn’t take cohesiveness and cohesion of the content into account. So， in the perspective of writing ability， the final scores have low reliability.