Full metadata record
| DC Field | Value | Language |
|---|---|---|
| dc.contributor | Department of English | en_US |
| dc.creator | Liu, Weiwei | - |
| dc.identifier.uri | https://theses.lib.polyu.edu.hk/handle/200/6676 | - |
| dc.language | English | en_US |
| dc.publisher | Hong Kong Polytechnic University | - |
| dc.rights | All rights reserved | en_US |
| dc.title | Gender and occupational knowledge : the effects of rater variability on the assessment of ESP oral performance | en_US |
| dcterms.abstract | The overarching aim of the research is to find out empirically the effects of rater variability on the assessment of ESP oral performance, and provide empirical evidence for the possible interaction between individual rater variables. Specifically, this research intends to explore the effects of rater variability on the assessment of ESP oral performance on eight types of speaking tasks, with a focus on the variability in two rater variables -occupational knowledge and gender. Besides, it intends to examine whether or not there is an interaction between raters' level of occupational knowledge and gender in the assessment of ESP oral performance on each of these types of speaking tasks. It also sets out to investigate whether the impacts of rater variability on the assessment of ESP oral performance are consistent across various types of oral task. The research was motivated by (1) Erdosy's (2004) view that rater variability in performance assessment is inherent as "raters are individual persons with personal characters" (p. 1), (2) Gerrig and Zimbardo's (2009) view that most human behaviours vary with the specific context in which they take place. and (3) the highlighted importance of background knowledge in Douglas' (2000) construct of specific purpose language ability. Based on Bynate's (1987) taxonomy of speaking tasks, the research consists of eight parallel studies, each exploring the issue of rater variability in the assessment of ESP oral performance on one specific type of speaking task. The data for this research were collected from 360 rater participants. Based on the finance or accounting qualifications they held, the 360 participants were categorized into three occupational knowledge groups: good mastery group (raters with an accredited senior qualification), basic understanding group (raters with an accredited junior qualification), and no understanding group (raters with no accredited qualification). Based on raters' gender, the three groups of raters were subsequently divided into six subgroups. Raters assessed 30 Chinese students' ESP oral performance on eight types of speaking tasks. Both holistic and analytical scales (pronunciation, fluency, task fulfillment, grammar and vocabulary) were employed in the research. To determine whether there were statistically significant differences among different groups of raters in the assessment of ESP oral performance, ratings of different rater groups were subsequently analyzed by repeated measure two-way ANOVA with two between-subject factors (raters' gender and level occupational knowledge) and a single within-subject factor at 30 levels (the 30 speech samples). Interviews were also held with some of the participants to provide further information that would help explain and substantiate the quantitative results. | en_US |
| dcterms.abstract | The research showed that both raters' gender and their level of occupational knowledge would affect the assessment of ESP oral performance, depending on the specific testing context. The quantitative analyses of ratings of ESP oral performance revealed that in the assessment of ESP oral performance on each type of oral tasks, significant effects of the two rater variables would be identified only when particular rating scales were used. Meanwhile, the quantitative results also showed that the rating scales that would result in significant effects of the two rater variables on the assessment of ESP oral performance varied with the task type. These findings indicate that the effects of raters' gender and their level of occupational knowledge on the assessment of ESP oral performance are context-specific. The specific testing context can be defined by two factors: the type of oral task that is used to elicit the ESP oral performance and the rating scale that is used to assess the performance. The interaction between raters' gender and level of occupational knowledge in the assessment of ESP oral performance was also found to be context-specific, depending on both the rating scale and the task type. Both the effect size and the effect direction of the interaction were found to vary with the specific testing context. The findings of the present research, based on which an improved model of assessment of ESP oral performance was proposed, may contribute to a better understanding of the impacts of rater variability on the assessment of ESP oral performance. The research may also contribute to the design and improvement of rater training programs and the interpretation and proper use of scores of speaking assessment. | en_US |
| dcterms.extent | xxiii, 860 p. : ill. ; 30 cm. | en_US |
| dcterms.isPartOf | PolyU Electronic Theses | en_US |
| dcterms.issued | 2011 | en_US |
| dcterms.educationalLevel | All Doctorate | en_US |
| dcterms.educationalLevel | Ph.D. | en_US |
| dcterms.LCSH | English language -- Spoken English -- Examinations. | en_US |
| dcterms.LCSH | Language and languages -- Ability testing. | en_US |
| dcterms.LCSH | Hong Kong Polytechnic University -- Dissertations | en_US |
| dcterms.accessRights | open access | en_US |
Files in This Item:
| File | Description | Size | Format | |
|---|---|---|---|---|
| b25072249.pdf | For All Users | 15 MB | Adobe PDF | View/Open |
Copyright Undertaking
As a bona fide Library user, I declare that:
- I will abide by the rules and legal ordinances governing copyright regarding the use of the Database.
- I will use the Database for the purpose of my research or private study only and not for circulation or further reproduction or any other purpose.
- I agree to indemnify and hold the University harmless from and against any loss, damage, cost, liability or expenses arising from copyright infringement or unauthorized usage.
By downloading any item(s) listed above, you acknowledge that you have read and understood the copyright undertaking as stated above, and agree to be bound by all of its terms.
Please use this identifier to cite or link to this item:
https://theses.lib.polyu.edu.hk/handle/200/6676