literature review. This chapter lists related literature and specific knowledge which concerns multiple-choice on the exam results quality in readi

MINISTRY OF INDUSTRY AND TRADE INDUSTRIAL UNIVERSITY OF HO CHI MINH CITY FACULTY OF FOREIGN LANGUAGES ------ GRADUATIONTHESIS STUDENT NAME: ID NUMBER: CLASS CODE: COHORT: SUPERVISOR NAME: HCMC, 05/2023 1 MINISTRY OF INDUSTRY AND TRADE INDUSTRIAL UNIVERSITY OF HO CHI MINH CITY FACULTY OF FOREIGN LANGUAGES ------ Student name : ……………………………………………. ID Number : ……………………………………………. Class Code : ……………………………………………. Cohort : ……………………………………………. Supervisor Name : …………………………………………… Student’s declaration: I declare that this graduation report is entirely my own work and does not involve in plagiarism or collusion. It also has not been accepted as part of a submission to another purpose else where. I accept heavy penalty for any cheating or plagiarism. Date submitted : ……………………… Signed : ……………………… Word count : ................................. words ABSTRACT 2 This study aimed at investigating the quality of 40-item multiple-choice taken from Reading class. It was carried out in order to get clearer picture and provide feedbacks on how the Critical Reading test for the new curriculum should be developed and what factors should be taken into account when developing the test. The data were taken from 24 fourth semester students in English Education Department who took the reading test. The data were analyzed by conducting Classical Test Theory analysis with the assistance of jMetrik software. In general, the result shows that the test was consistent but it was easy and only 40% of total items are eligible to be used. Along the findings and discussion, some feedbacks are provided for future development for the reading test. Keywords: multiple-choice exam, reading test. CHAPTER I: INTRODUCTION 3 1.1.Rationale The quality of multiple-choice questions (MCQs) as well as the student's solve behavior in MCQs are educational concerns. MCQs cover wide educational content and can be immediately and accurately scored. However, many studies have found some flawed items in this exam type, thereby possibly resulting in misleading insights into students’ performance and affecting important decisions. This research sought to determine the characteristics of MCQs and factors that may affect the quality of MCQs by using item response theory (IRT) to evaluate data. For this, four samples of different sizes from US and China in secondary and higher education were chosen. Item difficulty and discrimination were determined using item response theory statistical item analysis models. Results were as follows. First, only a few guessing behaviors are included in MCQ exams because all data fit the two parameter logistic model better than the three-parameter logistic model. Second, the quality of MCQs depended more on the degree of training of examiners and less on middle or higher education levels. Lastly, MCQs must be evaluated to ensure that high-quality items can be used as bases of inference in middle and higher education. This research was based on exam data from Vietnamese students to obtain a general conclusion. MCQs could compensate for the heavy work pressure. Moreover, examinations in many countries such as China and the US may provide different information. This research used item response theory (IRT) and selected four reading exams in Vietnam to evaluate the quality of MCQs. First, this research briefly reviewed the methods of assessing MCQ quality. 1.2.Research Goals  Evaluate the effectiveness of MCQs in English reading tests?  Find out students' reactions and attitudes in taking the test? 1.3. Research Questions 4 Throughout the process of teaching and learning, there are undoubtedly some difficulties and disadvantages. The research is guided by the following questions:  What are the characteristics of the multiple-choice test in the Reading test?  What variables impact a student's ability to perform well on multiple-choice examinations of English-majored?  What are some strategies students should use when answering multiple- choice questions?  How can teachers use multiple-choice examinations to measure their students' reading ability? 1.4. Scope of the study This research only surveys teaching methods of teachers. The study focuses on seeking some difficulties met during teaching English about multiple-choice on the exam results quality in Reading tests in class and the reasons why teachers meet those ones. 1.5. Research structure This research is divided into five chapters: Chapter 1: Introduction of the study, which includes the problem statement as well as reasons for carrying out this research. Chapter 2 – Literature Review. This chapter lists related literature and specific knowledge which concerns multiple-choice on the exam results quality in Reading tests Chapter 3 – Methodology. This chapter presents the methodology and design of the research: how data were collected, the instruments used to collect the data and how they were analyzed. Chapter 4 – Data analysis and findings. This chapter deals with the findings in detail and analyze as well as discuss of the research findings. 5 Chapter 5 – Implications and recommendations. This chapter concludes the study with a summary of all the main points presented in the study and some teaching implications drawn out from the outcome. CHAPTER II: LITERATURE REVIEW 2.1. An overview of multiple choice 6 2.1.1. The definition of multiple-choice questions Multiple choice questions are fundamental survey questions which provides respondents with multiple answer options (Mike, 2017). Primarily, multiple choice questions can have single select or multi select answer options. These are the most fundamental questions of a survey or questionnaire where the respondents are expected to select one or more than one option from the multiple answer options (Jorge et al., 2018). According to Brassil (2019), multiple choice exams is a method to test knowledge and skills through assessment by choosing true or false questions or choosing answers A, B, C, D. There are various types of survey questions that a survey creator can ask to evoke necessary responses from the person undertaking the survey (Product, 2005). Out of these variations, the close ended questions are the most used in surveys. 2.1.2. Classification of multiple-choice questions According to Kim (2020), there are broadly two types-single select and multiple select:  Single select multiple choice questions  Multi-select multiple choice questions  Dropdown menu multiple choice questions  Star rating multiple choice questions  Text slider multiple choice question  Numeric slider multiple choice question  Thumbs Up/Down multiple choice question  Matrix table multiple choice question  Rank order multiple choice question  Image/picture based multiple choice question 2.1.3. Characteristics of multiple-choice questions 7 According to Elgadal (2021), keep the following factors in mind while designing Multiple Choice Questions:  Have a fair idea of the concepts that you would want to cover in the question and avoid composing questions that have hazy attributes.  Form questions that evaluate respondents at conscious and subconscious levels like some factual questions, situational questions or analysis based questions. A multiple choice question comprises of a stem, the correct answer and the distractors (Elgadal, 2021):  A stem, that’s the question i.e. a problem or an incomplete statement - Make sure that you create a crisp, grammatically error-free and simple stem which has relevant information.  The correct answer - This should be relevant to the stem and shouldn't consist of too many qualifiers like "always" and "some”. Use phrases as options when the stem is an incomplete statement.  The other incorrect responses which are called ‘distractors’ - Ideally, create 4 distractors and should be in line with the correct answer. These distractors should usually be common misconceptions that your target audience may have. 2.2. Characteristics of multiple-choice in reading test 2.2.1. Features of reading test Language assessment is an integral part of any language programme. It provides the information about people's language ability that carries influential consequences for the test takers and determines their academic and professional future. Consequently, such highstakes assessment must be effective for the test takers and the various stakeholders who use the test outcomes. Testing literature is replete with evidence that language tests are generally of poor quality and do not measure accurately what they are supposed to measure. Several research studies have proved that test tasks 8 influence the performance of the test takers, resulting in testing experts to focus attention towards enhancing the quality of test tasks. Bachman and Palmer (1996) proposed a framework for task characteristics which includes five set of characteristics: setting, rubric, input, expected response, and relationship between input and response. Using this framework, the current study investigated the characteristics of reading test tasks designed for summative assessment at undergraduate language courses which are compulsory components of study programmes across all disciplines and with reading being a significant part of the syllabus and also of the summative assessment. Thirty exam papers within the domain of English for general purposes were collected from different public and private sector universities for the analysis of reading tasks. There are various reasons that lead to poor performance on reading test; they are inaccurate word reading, lack of reading fluency, lack of interest in reading, weak vocabulary or limited background knowledge, and varying socioeconomical background of test takers. Similarly, test tasks affect the reading performance of the test takers. Carroll (1993) defines tasks as an activity which allows a person to engage in an appropriate setting to achieve specific objectives. Salmani-Nodoushan (2003) explored the effects of text familiarity, task types and language proficiency on the reading comprehension and concluded that all three variables are influential factors that affect the reading performance of students. Aghajani, Motahari and Qahraman, (2013) conducted a similar study to explore the relationship between text familiarity and task type with the reading performance and found that both the variables significantly affect the reading comprehension. Students who were familiar with the content of the test performed better than their counterparts. Similarly, reading comprehension and performance differed on the basis of different task types. Alderson, Clapham and Wall (2010) have drawn attention to the 'method effect' explaining that the test technique will strongly affect students' performance on the test; therefore, test developers should consider what will be the effect of test task. In this respect, Education 9