Computer-Assisted Formative Assessment
-A Study on the Use of “E-learning at Home” in A Chinese Middle School
Educational assessment is an integral part of instruction and a significant mean of monitoring and realising teaching goals, which helps ameliorate teaching and learning.
The introduction of formative assessment not only assists teachers to get effective feedback to improve their teaching, but also motivates students to be active learner to take their initiative voluntarily. Utilising computer in education assessment has been a popular trend in recent years, and it is believed that this application provides formative assessment with more scientific and modern evaluation tools and methods. In China, most schools have realised the significance of doing formative assessment; however, some of them are still sceptical about the effectiveness of computer-assisted formative assessment. This study carefully investigates the use of an e-assessment tool in one Chinese middle school to help this school to make improvements on its usage. Moreover, the conclusions can give suggestions to those schools which are still doubting whether to apply e-assessment tools in their teaching and learning.
1 Introduction 簡介
Based on certain educational values or goals, educational assessment makes use of feasible scientific means, through systemic collection and analysis of information to determine the value of educational activities, processes and results to provide basis for improving the quality of education and educational decision-making (Andersson & Palm, 2017). Educational assessment plays an important role in guiding, diagnosing and promoting the timely discovery of the problems in the process of teachers’ teaching and students’ learning, which greatly contributes to the improvement of teaching level as well as the improvement of students' learning ability and achievement (Meusen-Beekman, Brinke & Boshuizen, 2016). #p#分頁標題#e#
China is a country with a long history, in terms of educational assessment, China has taken thousands of years of summative evaluation means, which is generally carried out to understand the final results of teaching activities when teaching activities come to an end (Zou & Zhang, 2013). The purpose of summative assessment is to test whether students’ studies have finally met the requirements of the teaching objectives. The implementation of summative evaluation leads to the fact that more emphasis has now been put on examination scores, and a greater number of people take test scores as the most important criteria for teachers’ and students’ success (Senye-Mir, Arumí-Prat, Pla-Campas & Ramírez, 2016).
To solve these problems, formative assessment began to be introduced into the compulsory education stage by the Chinese education sector in the 1990s (Zou and Zhang, 2013). Formative assessment is conducted towards students’ learning results and teachers’ teaching effect to guide the teaching process to correctly and effectively progress (Hwang & Chang, 2011). It is not utilised to select a few excellent students, and test scores are not regarded as the only criteria for doing assessment, instead, its purpose is to find out the potential of each student to better promote students’ learning, and to provide feedbacks to teachers to improve teaching activities (Andersson & Palm, 2017).
The current formative assessment has been vigorously advocated at all levels of education in China and in the teaching of various disciplines. In 2001, a new curriculum system was adopted and implemented in China, and from then on, Chinese teachers and students have improved their understandings on educational assessment. Ten years later, the English Curriculum Standards for Compulsory Education (2011 Edition) was published and put into practice, which pointed out that formative assessment has a significant role in improving student achievement and motivation, improving teachers’ teaching activities, and it appealed both teachers and students to attach great importance to conducting formative assessment (Ministry of Education of the People's Republic of China [MEPRC], 2011).
Flexibility and diversity of assessment methods is one necessary requirement of conducting educational assessment. The development of technology makes it possible for teachers to implement formative assessment with the help of internet, which brings great conveniences and changes to the original teacher-student relationship, communication and information transfer model. A critical understanding of the current situation and the drawbacks of computer-assisted formative assessment in China can help further promote formative assessment to play its positive roles in the Internet Age, so as to improve China's education in the assessment means, methods and theoretical deficiencies.#p#分頁標題#e#
1.2 Subject of the Study
In China, the most commonly-used formative assessment method is doing mid-term exam, which takes place in the middle of a semester. The results of mid-term exam can reflect students’ current learning situation, and then provide feedbacks to improve teaching and learning in the next half semester. However, students who take the mid-term exam still care a lot about the test score, exploring limitations and promoting teaching and learning seem to be less important for most of teachers and students. Even though some students strive to get high marks, they may have a lack of self-learning ability, poor creativity, and so on.
To make the implementation of formative assessment a more effective way to swift teachers’ and students’ attention from test results to test feedback, more diverse assessing methods should be introduced. In 2008, a Chinese middle school began to explore innovate assessing methods, and it designed a website called “E-learning at Home” to combine internet with English education. The website consists of three parts, self-assessment, blog and teacher evaluation. Unlike doing mid-term exam, this online assessment takes place throughout the whole semester. After the learning of one chapter, students are asked to go to the website and finish three steps.
In the first section, students will be given 50 multiple-choice questions or true/false questions. They can review the answers after they finish all the questions and the explanations of all the questions are available. The blog is the second evaluation. The teacher gives a topic after each chapter, and students are required to think independently and make their response. After answering, students can view others’ answers and comment on their peers. The third part is teacher evaluation, which is done anonymously. In this part, students are given the chance to make some comments or give suggestions on their teachers. Also, they can express their feelings like what they would like to be included in the class. According to the school, compared with those traditional methods, doing online formative assessment is welcomed by most students.
1.3 Topics of the Study
This research aims at investigating the current situation of computer-assisted formative assessment in Chinese middle schools and figuring out the shortcomings to recommend on the future development of computer-assisted formative assessment in Chinese middle schools. Based on this, the following research objectives are formed.
Research topic 1: to present the current status of using computer and network for formative assessment in Chinese middle schools;
Research topic 2: to take “E-learning at Home” as an example to discuss the effects brought by using computer to implement formative assessment in Chinese middle schools;#p#分頁標題#e#
Research topic 3: to critically analyse the existed problems in the process of using “E-learning at Home”, and to give suggestions on making improvements in the future.
2 Literature Review 文獻綜述
2.1 Understandings of Formative Assessment
Formative assessment is the evaluation on students’ learning process, which aims at confirming the potential of students to improve and develop students’ learning (Andersson & Palm, 2017). The main task of formative assessment is to evaluate the performance of students' daily learning process, achievements, emotions, attitudes, strategies and other aspects reflected, so as to motivate students and help them effectively control themselves in the learning process, so that students can get a sense of accomplishment, enhance self-confidence and cultivate the spirit of cooperation (Meusen-Beekman, Brinke & Boshuizen, 2016). Formative assessment is not simply from the needs of evaluators, but also from the needs of those who are being evaluated. It pays attention to students’ learning process and experience in learning, and it emphasises on the interaction between people, stressing on the interaction of a variety of factors in assessment as well as teacher-student exchanges (Senye-Mir, Arumí-Prat, Pla-Campas & Ramírez, 2016). In formative assessment, a teacher's duty is to determine tasks, to collect information, to discuss with students, to play a guiding role in the discussion and to evaluate with students.
Summative evaluation is evaluation carries out to understand the final results of teaching activities when teaching activities finish for a period of time (Andersson & Palm, 2017). All the examinations at the end of a semester or at the end of a year belong to this evaluation. The purpose is to test whether a student's studies have finally met the requirements of the teaching objectives. Summative evaluation emphasizes the results, in order to make a comprehensive assessment for those evaluated, to distinguish the levels of students, and to assess the effectiveness of the entire teaching activities (Sheard & Chambers, 2014).
From the above concepts, it can sum up the main differences between formative assessment and summative evaluation (Zou & Zhang, 2013; Meusen-Beekman, Brinke & Boshuizen, 2016; Senye-Mir, Arumí-Prat, Pla-Campas & Ramírez, 2016; Andersson & Palm, 2017). Firstly, the purpose of formative assessment is mainly to improve students’ learning process and adjust teaching program to promote students’ progress and development. The purpose of summative assessment is to determine the final learning outcomes for the screening and selection of students. Secondly, from the point of view of evaluation, formative assessment mainly evaluates students’ learning process, and the summative evaluation mainly evaluates students' grades. Thirdly, considering from the subjects of evaluation, formative assessment is mainly carried out by teachers and students, and summative evaluation is main completed by teachers. Fourthly, judging from the content of evaluation, the main evaluation content of formative assessment includes knowledge, skills, emotional attitudes, learning strategies, cultural awareness and so on. While summative evaluation mainly evaluates knowledge and skills of students’. Fifthly, in terms of evaluation means, the means of formative assessment can include daily observation, job assessment, questionnaires, self-assessment / mutual assessment, interviews, quizzes, activity records, etc., and summative evaluation means are mainly tests. Sixthly, the implementation of formative assessment is after the end of teaching of a unit or a project, and the frequency is higher. The implementation of summative evaluation is mainly after the end of a semester, it is generally 1-2 times per semester. Finally, the results of formative assessment include a description of whether it meets the requirements of the target, so as to point out the shortcomings and to make recommendations, and summative evaluation mainly records a student's test scores.#p#分頁標題#e#
2.2 Understandings of Computer-Assisted Assessment (CAA)
With the progress and popularization of computer technology, in the late 20th century, information technology has been widely used in educational evaluation, according to a British national survey conducted in 1995 and 1999, computer-assisted assessment (CAA) was applied in almost all disciplines (Xu & Liu, 2005). With the research and application of CAA, some new development methods and the development of formative assessment theory emerge, computer-assisted formative assessment has become the most important content of current educational evaluation theory and practice.
Computer-assisted formative assessment (CAFA) is the use of computers to carry out formative assessment. It includes assessments conducted by both online and offline ways (Sheard & Chambers, 2014). Computer-assisted formative assessment is expected to be increasingly important in students’ learning. Computer-assisted formative assessment has been proved to be effective in study in acquiring positive student feedback to improve students’ performance (Maier, Wolf & Randler, 2016).
Formative assessment theory holds that human learning and development is a dynamic process, positive feedback in the process is more effective than result control, because the information obtained from the process is more comprehensive and authentic (Sheard & Chambers, 2014). Through assessment on if students have achieved periodic goals, it can confirm the development achievements of students in time to enhance students' confidence and improve their interest in learning. Assessment on students' learning process requires certain process information (Gikandi, Morrow & Davis, 2011). Traditional paper-based folders have many problems and difficulties in storage, maintenance, and retrieval. Information technology applications such as e-works and electronic archives provide technical solutions for these problems (Gikandi, Morrow & Davis, 2011).
Electronic works are documents or procedures that students use to express their innovative thinking and problem-solving results by modern information technology. Through assessment on students' electronic works can help students to understand their strength and lack in learning to promote students to establish their own learning goals (Wilson et al., 2011; Gikandi, Morrow & Davis, 2011). Learning archives collect and preserve some information in learners’ learning process, which is a very effective means of demonstrating and evaluating learners' efforts and achievements in learning (Wilson et al., 2011; Gikandi, Morrow & Davis, 2011).
Due to the support of computer technology, formative assessment can have more advantages, such as rapid feedback, more flexible forms of feedback, multimedia feedback content, free assessment time and place to reduce the pressure of part-time students’. There is great convenience for communication between students, between students and teachers, and it is very beneficial to promote mutual trust and cooperation among them (Maier, Wolf and Randler, 2016; Gikandi, Morrow & Davis, 2011).#p#分頁標題#e#
China's research on computer-assisted formative assessment (CAFA) is consistent with that of foreign research. They also explain the importance of the use of computer in formative assessment in terms of communication convenience, information transfer, storage and data analysis. In addition, China's related research is also concerned about the importance of CAFA for adult education, continuing education and vocational training. Sang (2004) points out that the lack of effective teacher-student interaction and time pressure are two important factors that affect the quality of distance learning, and CAFA will help to solve these two types of problems.
2.3 Current Situation of Computer-Assisted Formative Assessment (CAFA) In China
Computer-assisted formative assessment (CAFA) outperforms traditional evaluation methods on the efficiency and flexibility of evaluation since timely feedbacks are available, and students are able to do the assessment whenever necessary.)
More emphasis has now been put on CAFA by schools and teachers in China, especially in the field of English teaching in secondary schools, CAFA has been applied to a certain degree. Some scholars have also published some articles to elaborate on the advantages of CAFA, but in general, CAFA is currently not widely used in China, the main reasons are showed as follows.
With the development of computer technology, measurement and evaluation theory, the role of computers in the formative assessment of foreign education is becoming more and more prominent. The related research is also very rich and mature. However, China's CAFA research is not optimistic, Xing (2006) believed that China's research on test theory and assessment theory is very weak, there are not many scholars who study examinations and assessment problems, and there is very rare CAFA works and articles. Due to the lack of teachers with skills in terms of education measurement and assessment, some schools have deleted the course of "education measurement and assessment. There is quite a lack of funds and personnel for research on CAFA.
CAFA is an interdisciplinary subject, which involves knowledge of related fields such as education, statistics and computer. However, there is not a systematic teaching material which can combine computer knowledge, formative assessment knowledge, as well as teaching practice knowledge to introduce CAFA (Wang, 1998). This is very unfavourable for the training of CAFA-related personnel.
Compared with foreign countries, there is still very low frequency and small range in the use of CAFA by Chinese schools and teachers, some teachers and students even have some misunderstanding on CAFA that there is too much trouble in the use of CAFA and the efficiency is not high. This is closely related to the use of traditional summative assessment and the over-reliance on measurement and evaluation results by schools and teachers in China (Huang, Liu & Li, 2002).#p#分頁標題#e#
China's CAFA development and application is still in the relatively low stage, most of the main functions of CAFA is only recording learning results, publishing information, and it fails to resolve many practical problems in education by applying CAFA, such as how to use CAFA to test learners' advanced cognitive ability, how to use CAFA to get more favourable evaluation information, how to strengthen the security of network examination and how to improve the operability of formative assessment through computer aids (Huang, Liu & Li, 2002). CAFA is not just a technical issue, but a problem of a rational application of technology, deficiencies in CAFA features and application also lead to students’ and teachers’ lack of enthusiasm and motivation in the use of CAFA.
In short, CAFA has made initial development in China, but due to the lack of CAFA research and personal training in terms of CAFA, teachers and students have deviation on the concept of CAFA, and in China, CAFA itself has deficiencies in the application and function, resulting in that CAFA has not been widely used in China.
3 Discussion 討論
3.1 Validity and Reliability
Reliability refers to whether the test results of formative assessment are consistent, stable and reliable, namely, whether the results of assessment can reflect the actual study of the subjects (Tridane et al., 2015). In the process of using formative assessment, the following factors will usually have a negative impact on reliability, including whether the assessment conclusions are based on previous biases or personal impressions, whether it is based on irrelevant characteristics, for example, how they are dressed, the achievements of their siblings and so on to evaluate students’ abilities, whether it is based on just one observation or piece of information to make judgments, whether it assumes that when a student behaves in a certain way in a certain situation, and he will be behave the same way in other situations (Gikandi, Morrow and Davis, 2011; Lin and Lai, 2013; Timmers, Walraven and Veldkamp, 2015).
In this practice, application of “E-learning at Home” in formative assessment played a positive role in improving the reliability, which was mainly reflected in the following areas. First, the use of the technical advantages of “E-learning at Home” helped the students’ performance and academic records to be preserved in a timely, accurate and long-term, open and comprehensive way, so that teachers, students, parents and other relevant people could have an access to this information to ensure that in the assessment on students by teachers, teachers could assess students fairly to avoid the influences of previous biases towards students and or irrelevant characteristics. Second, “E-learning at Home” could record students’ performance and achievements in a comprehensive and long-term manner, and it could be more intuitive to show the progress or regression of students to avoid the influence of a student's performance of a certain time on the teacher’s overall assessment on the students. Finally, the openness of “E-learning at Home” allowed more teachers, students, parents to participate in the process of learning and education, so that students' academic performance and performance could recorded more accurately, and teachers’ assessment became more fair.#p#分頁標題#e#
Validity means to what extent the measured results reflect the content observed, if the measurement results and the content observed are more consistent, the validity will be higher; conversely, the validity will be lower (Timmers, Walraven and Veldkamp, 2015; Lin and Lai, 2013). The validity of formative assessment is to determine the extent to which the results of the assessment can effectively reflect a student's performance, so as to improve the student's academic performance and learning ability, and maximize the development of the student’s learning potential (Gikandi, Morrow and Davis, 2011; Lin and Lai, 2013). The validity of the assessment in learning evaluation usually includes process validity, subject validity of assessment, content validity, and so on (Gikandi, Morrow and Davis, 2011; Timmers, Walraven and Veldkamp, 2015). The practice of using “E-learning at Home” in formative assessment this time has a positive effect on the improvement of validity, which is mainly reflected in the following aspects.
Validity of assessment process is concerned with the process in which a student assessed continues to grow and improve under the established evaluation objectives and the sum of the student's performance throughout the educational activities (Gikandi, Morrow and Davis, 2011; Lin and Lai, 2013). In this research, “E-learning at Home” used electronic documents, electronic portfolio, database and other forms to conveniently store all information and data of students to help teachers, students and parents to have a accurate understanding, statistics and analysis of all aspects of students in learning process, as well as to understand and the progress and overall performance of the students in the whole process of learning, therefore CAFA has a positive effect on improving the process validity of formative assessment.
Formative assessment requires teachers, students and parents to participate together, but based on different values, goals, different roles have different views on teaching, which may make the results of assessment and interpretation of the results vague and uncertaint, and it may affect the fairness and impartiality of formative assessment activities, thereby affecting the effectiveness of assessment results (Zou and Zhang, 2013). Validity of assessment subject is concerned with the coordination of the contradictions between the subjects and to what extent the assessment criteria are consistent, thus ensuring the fairness, impartiality and effectiveness of the assessment results (Gikandi, Morrow and Davis, 2011; Timmers, Walraven and Veldkamp, 2015). In this study, “E-learning at Home” helped to improve the efficiency of communication between students, teachers and parents, so teachers could have a timely detection of what problems that student have, through the network to feed back to students and parents in a timely manner, students and parents could also be convenient to consult with teachers and wait for feedback from teachers through the network. Making use of the network to achieve more efficient and convenient communication is important to coordinate the contradictions between different participants and ensure the fairness and effectiveness of assessment results.#p#分頁標題#e#
Content validity refers to the degree of matching of the indicators and contents of test or assessment with the learning content and objectives that need to be examined (Gikandi, Morrow and Davis, 2011; Lin and Lai, 2013). The goal of formative assessment is to evaluate a student's learning process, give the student feedback and guidance in a timely manner to help the student to develop its own learning ability (Wilson, 2011). In this study, the use of the network helped to achieve the above objectives. First of all, advantages of "E-learning at Home” better assessed the learning process of students in terms of information storage, data statistics, analysis aspects, so that the students could understand their own shortcomings and progress. Then, convenience of “E-learning at Home” in the communication greatly facilitated the communication between teachers and students to facilitate students to accept the guidance. Finally, the network itself contains a lot of learning resources, the teachers guided the students to find learning resources through the computer-assisted application, so as to achieve the purpose of improving the students’ self-learning ability.
3.2 Equity and Inclusion
In this research, “E-learning at Home” brought more equity and inclusion to formative assessment. Thus the assessment results could not only improve the students’ performance, but also improve the students’ enthusiasm to learn, which was specifically reflected in the following areas. The use of “E-learning at Home” allowed students who could not personally participate in assessment to receive teachers’ educational assessment in a timely manner, so that they got a fair learning opportunity to improve their learning enthusiasm. In addition, the use of "E-learning at Home allowed the teachers to have more time and energy to assess and send feedback to each student, each student could have a fair access to teachers’ evaluation and guidance. Moreover, “E-learning at Home” is a open platform, teachers, students, parents could exchange, share information through the platform, therefore, any contradictions arising from the formative assessment between the participants could be solved by timely communication and interpretation, therefore, formative assessment by "E-learning at Home” became more inclusive, and formative assessment participants were more likely to understand and support each other.
3.3 The Effects of the Practice
Compared to traditional formative assessment, using "E-learning at Home” in formative assessment in the Chinese middle school in this study brought the following benefits, first, the use of "E-learning at Home” during the formative assessment process, the frequency of communication between teachers and students was improved; in the classroom, under pressure or shy, the students sometimes dare not to communicate with teachers, and through "E-learning at Home”, the frequency of communication between teachers and students has been improved more substantially, so that teachers could more clearly understand the levels of the students to give a more correct assessment. Second, compared with traditional formative assessment, the use of "E-learning at Home” in formative assessment allowed the quality of communication between the students and teachers to be improved; in the classroom, the content of communication between teacher and student is often general due to time limitation, and the use of "E-learning at Home” in communication helped teachers and students to fully express their views, which made it easier for students to understand the views of teachers to make improvements, on the other hand, it also encouraged students to actively discuss with teachers, so as to improve their enthusiasm for learning. Third, traditional formative assessment is mainly aimed at assessment on students’ performance in the classroom and school, and "E-learning at Home” broke through the geographical constraints, so that students could also communicate with teachers, upload their homework after school through the network, so the use of "E-learning at Home” students could assess students’ performance in school and after school to have a more complete evaluation of students’ learning process, which could better supervise students to pay attention to improve their learning performance and improve their learning autonomy after school. Finally, one of the important purposes of formative assessment is to promote and improve students' autonomous learning ability, and the use of "E-learning at Home” for formative assessment could better achieve this purpose. There is a lot of learning resources on the network, through the use of "E-learning at Home”, it facilitates students to more active to make use of the network to collect the learning resources they need, so that it not only makes their improve their academic achievements, but also improves their ability to collect information and self-learning ability.#p#分頁標題#e#
3.4 The Limitations of the Practice
The use of "E-learning at Home” for the formative assessment this time has the following deficiencies which need to be improved in future. First, in this research, the formative assessment by using "E-learning at Home” was mostly teachers’ assessment on the learning situation of students, and there was no mutual assessment between students, while mutual assessment is an important way to improve students’ learning ability, thus in future, it should also pay attention to how to use the "E-learning at Home” for students to evaluate each other. Second, considering from learning environment, the use of "E-learning at Home” in formative assessment in most cases this time was one-to-one assessment carried out by teacher on students, and "E-learning at Home” was less used in collective class and students have less opportunity to evaluate, such learning environment largely differs from real environment, which may lead to that the assessment results of students do not match with students’ performance in the actual environment, therefore, the future use of "E-learning at Home” for formative assessment should be used more in collective class to assess students’ performance in collective class. Finally, information transmission and communication of using "E-learning at Home” in the formative assessment process in this research were in the form of text, in the future, it can make more use of pictures, audio, video and other forms for information transmission, so that the assessment will be more intuitive, efficient and fun to improve students’ participation in formative assessment and their interests of using "E-learning at Home”, which requires" E-learning at Home” to improve the design of the website to provide video, audio, and picture delivery functions, teachers and students should also learn how to use these functions.
4 Conclusion 結論
4.1 Findings of Present Paper
CAFA has achieved initial development in China, but due to the lack of adequate research and personnel training for CAFA, teachers and students have bias for CAFA (Hwang and Chang, 2011), coupled with the application and functional deficiencies of CAFA itself in China, resulting in that CAFA in China has not yet been widely applied (Zou and Zhang, 2013).
Compared with traditional formative assessment, the use of "E-learning at Home” in formative assessment in Chinese middle schools brought the following benefits, first, during the process of using "E-learning at Home” in formative assessment, the frequency of communication between teachers and students has been improved; second, the use of "E-learning at Home” in formative assessment improved the quality of communication between students and teachers. Third, the use of "E-learning at Home” could assess students’ performance in schools and after schools to have a more complete assessment of students’ learning process, which could better supervise students to pay attention to improve their learning performance after schools to improve their learning autonomy. Finally, one of the important purposes of formative assessment is to promote and improve students' autonomous learning ability.#p#分頁標題#e#
4.2 Suggestions on Future Development of the Practice
The use of "E-learning at Home" in this research has the following problems which need to be improved in the future. First, the use of "E-learning at Home” in the formative assessment was mostly teachers’ assessment on the learning situation of students, and there was no mutual assessment carried out by students with each other, in the future, the use of " E-learning at Home” should be paid attention in terms of assessment conducted by students on each other. Second, the use of "E-learning at Home” in the formative assessment in most cases in this research were one-to-one assessment implemented by teacher on students, the future use may involve more collective assessment, namely, to assess students’ performance in collective class. Finally, in the process of using "E-learning at Home” this time, the information transmission and communication were in the form of text, in the future, more pictures, audio, video and other forms can be used in information transmission.