Onine Assessment – An Effective Tool To Support Learning
– A Case Study Of Middle School Mathematics
Teachers need inputs and training to generate good assessment tools so that schools are able to use them to assess the learning levels achieved by the students in the various school subjects. Generating or developing a good assessment tool (the question paper) is another aspect of assessment which needs improvement.
India is a country with 1.3 billion people . Out of this population, more than 470 million people are below 14 years of age. The number of young people between the age of 6-14 years who attend the school upto upper primary Level are approximately 210 million. Quality education for such a large number is a big concern and there is need to use different strategies to achieve the ultimate goal of universalisation of education qualitatively and quantitatively. Huge efforts and inputs are required to create interesting learning materials, giving classroom teaching-learning processes a more creative and qualitative push and to standardized assessment for all the students across the country.
Importance of Assessment
Assessment is an important aspect of education, which also provides indicators about what the students are learning and how well they are learning. Even if the teaching and learning processes are improved and are implemented in the schools, they will lose their relevance if the assessment is not in consonance with the teaching-learning processes. For example if assessment is consisting of test items which are only knowledge based / memory based , then the whole school system will be moving in that direction promoting knowledge focused learning.
PISA – Programme for International Student Assessment, assesses students in Mathematics and benchmarks the mathematical literacy of a 15 year old to formulate, employ and interpret mathematics in a variety of contexts all around the world. India participated in PISA for the first time in 2009 and ranked 72nd among 74 participating countries. The low scores were explained due to the difference in cultural contexts of the questions used in the assessments.
However, these results and many other surveys done by ASER etc, clearly indicate that assessments in mathematics need to move from memory-based assessment to understanding-based assessment. Assessment agencies like the CBSE have taken a note of these trends and are taking steps to move in this direction.
Also it is a known fact that the teachers need inputs and training to generate good assessment tools so that schools are able to use them to assess the learning levels achieved by the students in the various school subjects. Generating or developing a good assessment tool (the question paper) is another aspect of assessment which needs improvement. Stereotype questions papers which focus on assessing memory/knowledge are no more relevant as the teaching/ learning strategies are moving on to activity based learning or experiential learning. The educational scenario is still struggling to adopt and adapt the relevant assessment strategies due to its complex structure.
Digital Assessment could provide an alternate solution to overcome some of the road blocks and difficulty being faced in assessment. By using digital technologies, each child could be assessed by tapping into a question bank of good quality items prepared by trained teachers & experts. The questions will be available based on various aspects like difficulty levels, domain of learning etc. It is also possible to provide equivalent different assessment tools/question papers to the students with the flick of a click.
Online assessment is slowly emerging as an assessment tool to be used in the next decade. Popularity of online assessment for entrance tests to large numbers of courses has brought it in the forefront.
Using online assessment in the school education is at a nascent stage. Schools are looking at the whole system critically and apprehensively. The major question being how it would help them to assess students, diagnose the gaps in learning and provide indicators for fulfilling the learning gaps and planning remedial measures.
The present case study is based on an initial attempt in this direction to explore the impact of creating an online assessment system, and analyzing the results to look for patterns and indicators to help the schools and students to diagnose the learning gaps of the students, and to provide information and patterns to teachers about these learning gaps.
Mathematics was chosen to be the subject for which online assessment would be planned because the general perception is that it is a tough subject to learn and assess.
- It requires a great deal of effort on the part of the student to learn and do problem solving.
- There is a lurking fear about Mathematics. Lots of students are afraid of learning Mathematics. Online assessment could help them by identifying the learning gaps and making effort to fill them.
- Creating assessment items for assessing understanding of concepts and its applications require creative thinking on part of teachers.
- Then was the question “for which classes the online assessment be planned”. After detailed discussions, middle classes were chosen by the research team. The team felt that:
- Mathematics starts becoming abstract for the students in the middle and upper classes, as concepts like Number Systems, Algebra and Euclidean Geometry are introduced.
- There is no link for transition from an activity based experiential learning approach (in primary) to an approach based on understanding of abstract nature of Mathematics in upper primary or middle classes.
- Teaching/learning in upper primary or middle classes is based on the understanding of Mathematical concepts learnt in primary classes. The students may or may not have understood the basic concepts resulting in further anxiety and fear about Mathematics.
- It was also observed by the research team that a common problem area cutting across all Mathematical themes was problem solving. Whenever a student is confronted with a problem, he is either unable to comprehend it and to create a mathematical model to solve it. Most of the times the class teacher is requested to interpret the problem and tell them how to solve it.It was felt that the questions can be designed in a style or approach in which the students’ comprehension skills, modeling skills and computational skills are assessed along with the conceptual understanding, knowledge and operational skills. Some research findings also supported this strategy of assessment.
Ketterline-Geller, Yovanoff, 2009, concluded that “Assessment results provide information about students’ mastery of relevant prior knowledge and skills within the domain as well as preconceptions or misconceptions about the materials”
How does this information get used to discover the learning gaps? Errors made by learners could provide the indicators towards the learning gaps, and diagnostic assessments could prove useful in this regard.
“Combining diagnostic assessment approaches may prove useful for designing instructional programs to remediate students’ misconceptions. In the approach, multiple-choice items may be strategically designed to incorporate distractors that mirror systematic error in student thinking.” (Ketterlin-Geller, Yovanoff, 2009)
For pilot testing, one topic for each class was selected as per following details:
For each theme, some subthemes were also identified for assessments. For example for the theme Integers, following subthemes were chosen for assessment:
- Need & Representation of Integers
- Addition & Subtraction of integers
- Multiplication of integers
Similarly subthemes for class VI and VIII were identified. Questions generated by team were based on the following model:
All questions were multiple choice questions and options were based on common errors. The various options to each question were created based on the classroom experience of research team with learners. Misconceptions, errors and mistakes generally committed by the students were taken into account to write options.
On the basis of classroom experiences of teachers and the errors made by the students the error codes were determined for respective skill domain.
In addition to such questions, two more sets of questions were prepared. To focus on Modelling and computation skills second set of questions was designed to assess modelling and computation/ calculation question.
Such questions would help to focus assessment of modelling and computation skills of the students. It was designed in such a way that students were required to use little bit or low level of comprehension skill.
After detailed discussions on the various categories of questions , it was also worked out how many questions of each category a student would face so as to assess her/his understanding , comprehension , modeling and calculation skills.
It was also felt that in order not to disturb the time table and class management the total time duration of pilot assessment would be 30 Minutes.
The total time and time taken to attempt one question was worked out and it also helped to finalize the number of questions.
Parallel sets of questions were also created so that all students taking the online assessment would get different equivalent questions.
To have a comprehensive assessment of concepts & skills a judicious mix of questions of various types were chosen.
These questions & their options, combined with error identifications, the software was equipped with identifying learning gaps conceptually and skill wise.
Our presupposition based on the research team’s previous experience and knowledge was as follows:
- The learners will depict learning gaps in the modeling skills in the problem solving contexts maximally.
- A segment of learners will have learning gaps in comprehension, but they will be fewer than those in modelling
- Most of the learners will have better computational skills and would attempt the computational part of the questions correctly.
- Diagnostic online testing will be able to showcase these patterns in learning gaps so as to indicate a remedial learning path.
The research team pilot tested themselves the online tests generated by the software and then issues of student interest and total time taken for the test also came up. It was further discussed and test was further modified keeping in view the time limitations of 30-40 minutes & students interest.
It was found, that for quite a lot of students and teachers, it would be their first exposure to online assessment; it was decided to take up additionally following steps:
- Orientation of teachers to explain and make them aware about online test, if possible make them to take little bit of assessment to experience it, before forming an opinion about.
- Provide comfort and confidence to the students about online testing by planning a demonstrative animated video of short duration (one & half minute or so)and playing it back before the actual online testing started.
Sample for Online assessment
Various schools were contacted to participate in the online test for class VI, VII & VIII. Finally four schools agreed to participate in the experiment and 1500 students (approx) took part in it.
The schools provided the infrastructure i.e. desktop computers connected to a server in a closed loop. Most of the schools were able to provide 30-40 computers connected to a server.
Therefore the assessment had to be staggered into 2-3 days. Also because of limited manpower, the test was conducted in different schools on different dates.
In some schools all the students studying in classes VI, VII& VIII took part but in others only limited students, who were randomly selected, took part.
List of ELO (Expected learning outcomes) selected for classes VI, VII& VIII are as follows:
During the Pilot Testing:
The following responses were observed by the research team during the actual pilot testing of online assessment in the school.
- During the orientation of the teachers, it was felt that the teacher has apprehensions and concerns about the efficacy and impact of online testing. Efforts were made to address those concerns and questions. Some of the concerns were
- Would the students be able to take online test? (Most of them have no experience to take a test on computers)
- Will the software be able to analyse the result and showcase the right details?
- What kind of report it will generate?
- Will such a report be helpful in identifying and filling the learning gaps?
As a first time user of online assessment, such concerns/questions are bound to be raised. The process of online assessment and analysis of the results were explained to the school management,teachers and students.
- It was observed that inspite of being their first online assessment most of the class 7th and 8th students were excited and showed no sign of nervousness about it. However, class 6 students were less confident and needed reassurance and support.
- As the infrastructure spared by the schools taking part in the pilot study was limited in each participating schools, the test was staggered. round 30-40 students took in a 30-40 minutes online test. It took 15-20 days (3-4 times a day) with breaks to let 1500 students (approx) to undergo the experience of online testing.
This experience has created some learning points for the research teams:
- For the online assessment in the school infrastructure – availability of computers and their interlinkages, printers etc. is a must. Only then, it can be operational and successful.
- The students need be given experiences and exposure to online assessment regularly so that it becomes a routine activity in the school.Only then, lots of bottlenecks, hesitations and concerns which were observed during the pilot testing will find their solutions and then online assessment will become an integral part of school’s academic calendar.
- Students/teachers attitude towards online assessment will also change and the assessment will be treated as part of the whole package of assessment for learning and assessment of learning.
Analysis of the data
The software was able to analyse data at different levels
- At individual level for each student about his performance.Various small details about his/her understanding mistakes,learning gaps etc.
- At a large group level for each class or section showing the trends and strength/learning gaps etc.
- At school level, taking into consideration performance of all the students.
The following diagrams illustrate the analysis of the report generated by the software and support the school in improving the teaching/learning of Mathematics; With the help of the reports
- Each child would know the details like time taken to take the test, number of correct responses, mistakes done, strength and learning gaps. Suggestions for going forward/revisiting the concepts, if any. It is possible that students may make individual efforts to plug the learning gap in addition to the efforts being taken up in the class.
- The class report with its trends and patterns in the performance of the students will get insight into the strength of the class-the areas where most of students performing satisfactorily and the areas where learning gaps are depicted. Also other detailed information about mistakes and so on will help the teacher to pinpoint the students who need additional support in the some areas. Then the teacher can plan action to plug the learning gaps.
- The institutional reports will provide the administrators and academy directors an indepth insight into the learning gaps of the whole class in the school and plan corrective action administratively & academically.
Analysis Report of a Large Group
Details provided by the online assessment software
- Analyse by concept
- Analysis by problem solving skills -comprehension modelling calculate
- By the error codes
Analysis by the software based on the online testing provide details based on
- The sub- concepts assessed
- Problem solving skill –Comprehension, Modelling and Calculation
- Mistakes done by the class in the online assessment
Thus the report provides a very comprehensive analysis about the whole class. So the insights can help the teacher to plan and revisit or create remedial activities for the whole class or particular set of student as per the report indicators.
Analysis for the institution
Following graphics gives a detailed view of the performance of the whole class
- Concept wise
- Skill wise
- By Error Codes
If different schools allow academically inter institutional comparisons could be done there by learning from each other’s experiences
Let us illustrate with the help of reports generated by the software. For the sake of privacy protection, the names of the school and the students etc. have been omitted out.
Grade 6 – Assessment Outcome
- Estimation was the weakest link across schools in terms of concepts learnt.
- International System of Numerations was generally well understood across schools
- Students across all schools were found to begood in Calculations
- Students across the schools were found to be weak with Modeling. However, some schools performed better than others.
Grade 7 – Assessment Outcome
- Difference of Integers was the weakest concept across schools
- Need and Representation of Integers was generally well understood across schools
- Students across all schools were good in Calculations
- Modelling was the weakest skill among all students of all schools.
Grade 8 – Assessment Outcome
- Number Between Two Rational Numbers was the weakest link across schools
- Properties of Addition of Rational numbers was generally well understood across schools
- Students across were good in Comprehension.
- Modelling was the weakest skill
- A major difference noted was that students of Grade 8 were better at comprehension than those of Grade 6.
Report for an individual student:
This report is of a class 7th student of one school that participated in theonline assessment.The theme was Integers:
- Need and representation
- Sum and difference of integers
- Multiplication of integers
Such a comprehensive report may help the student to self-realise the efforts she/he has to make to learn the concept very well. Based on the data collected and analysed, these reports can be further customized to various audiences like the students, their tutors and the school administration, keeping in mind their specific needs, ability to understand the complexity of data and intended use of the report.
Further, a remedial action plan may be suggested based on the findings of such an assessment, focusing on the specific learning gaps and error patterns identified.
Learnings from the experiment (Case Study “Online Assessment.”)
- In spite of the limitations of focusing on limited sub concepts of mathematics in middle classes, the potential of online assessment as a supporting tool to teaching learning was clearly evident in the analyse reports of the experiment. It provided detailed inputs to all the stake holders- namely individual student the class teacher and the school management, so that based on the indicators provided each one of them could plan and take remedial action.
- Each individual student, based on the assessment can get specific inputs about the learning gaps and with help of her teacher/ parents peers. The students can think of taking action themselves on remedial in a customised plan.
- With the comprehensive inputs and indicators provided to the teacher online assessment could be very useful for the teacher to modify her teaching /learning strategies and fine tune them to fulfil the needs of the students as highlighted by the assessment.
- School management gets a 360 degree view and may think of taking systematic correction and changes.
- Based on the patterns/ trends generated the academic community at large curriculum planners, textbook developers etc. could provide more specifically focused materials so that the goals enunciated by the national policies are achieved by the students
- This experiment also throws up pointers towards using online assessment in the schools, although this kind of assessment is prevalent in various kinds of competitive examinations for admissions and ranking purposes.
- It’s use in providing inputs, insights and support in further improving the quality of school education largely remains unexplored.
Hence, there is an urgent need to provide good quality assessment through online technology. This experiment is just a beginning. With the advent of newer technological tools and assessment techniques, variety of tests will provide comprehensive inputs and support to teachers to help them achieve the goals of imparting quality education.
Assessment is an important aspect of education which also provides indicators about what the students are learning and how well they are learning. Even if the teaching and learning processes are improved and are implemented in the schools, they will lose their relevance if the assessment is not in consonance with the teaching-learning processes.
Digital Assessment could provide an alternate solution to overcome some of the road blocks and difficulty being faced in assessment. By using digital technologies, each child could be assessed by tapping into a question bank of good quality items prepared by trained teachers & experts.
With the comprehensive inputs and indicators provided to the teacher online assessment could be very useful for the teacher to modify her teaching /learning strategies and fine tune them to fulfil the needs of the students as highlighted by the assessment.
Anita Sharma is Principal of Sanatan Dharam Public School, BU-Block, Pitampura in New Delhi since 2002. She has raised the school standard bilaterally by enhancing the teachers’ competencies in their subjects as well as by facilitating the training programmes for them in usage of technology in the classroom especially the use of open software. She is also a Member of curriculum committee, Mathematics for CBSE and CBSEi (international curriculum launched by CBSE )
She has conducted more than 300 training programmes for teachers in India and overseas about effective transaction of Mathematics curriculum. She has won many awards and accolades including a Presidential Grant from Ministry of Science and Technology to conceptualize, plan and execute the programme “Mathematics Movement.” She is also the recipient of Lifetime Achievement Award Aryabhat Award by All India Ramanujan Mathematics Club and many other prestigious awards.
She has authored a series Math ZING for Primary School Mathematics (Classes I-V)for Encyclopaedia Britannica and a series for Middle School Mathematics (Classes VI-VIII)with S.Chand Publications.
Prof(Dr.) Dharam Parkash, MA ( Maths), MADE, MEd, PhD has worked with School Television, Directorate of Education, Delhi. He has been associated with CIET, NCERT in Film & Video production in various capacities including Head of Media Production. He has been involved in development of Teacher’s training packages for different levels of teachers.
He is the Executive Producer of the Video programme “The Changing Moon” which won Maeda Prize at NHK International Competition, 1999. His Action Research Paper “Teaching Science using Science & Technology Literacy Material”, won Commendation Prize, CASTME, London 2001 based on material developed under UNESCO. He received the B.K. Passi Prize (2008) instituted by All India Association of Educational Research (AIAER) for best paper in Education Technology – based on the pilot study for using mobile devices for delivery of educational content and interaction.