Is our scale for measuring student knowledge broken or good?

It is known that the new assessment system was introduced only recently, previously it was introduced in 500 schools, and this year it was introduced in another 1000 schools. Thus, a new assessment system has been introduced in 1,500 schools, which in turn is causing discussions.

Of course, we need scales to assess the education system, the quality of education, and the knowledge students receive in general. There's this scale now. Now we discussed how well this scale is working, how badly it is working, whether it's broken or in order, in the "Tanaffus" podcast.

Dilrabo Davidova, Director of the Scientific and Practical Center for Pedagogical Mastery and International Assessment, Komil Jalilov, Expert on Education Issues, Khudoyberdiyeva Mukhayyakhon, Teacher of School No. 216 of Yashnabad district, Tashkent city, and Ulugbek Pulatov, Director of the editorial office of xalqtaliminfo.uz took part in this discussion.

Dilorom opa, first of all, if you give me a little idea about the evaluation system. True, here, for example, I didn't go into it, [but] Kamil aka knows well, but the observers may not understand well, so if you briefly tell us how the new assessment works, when it was introduced and plans for the future, then we will move on to the questions, directly.

Before speaking about the evaluation system, I wanted to say that there are huge reforms being carried out in every area, and there should be indicators that show the effectiveness of that area, including education, as you said just now, as in every area. This is true everywhere.

In education, this indicator serves as an assessment. About the new evaluation system, what is the function of evaluation, evaluation in general? Since it is the main indicator, there are questions about how it should be. Research began long before we introduced this evaluation system, and I can confidently say that this evaluation system has not been invented by us.

When studying international experience around the world, almost many countries, almost 80-90% of countries, even more than 90% of countries, have learned to conduct such an assessment system, and in the course of studying this, we have studied it very much and have not taken the assessment system of any country and implemented it for ourselves.

Even when we study foreign experience in many aspects, we are not limited to a highly developed country abroad. It was decided to implement it by studying the experience of developed countries, developing countries, as well as post-Soviet countries like us, and when we localize this evaluation system, the process itself, like many studies, when scientists or specialists study some country, sit down and work on it, and do not reduce it from top to bottom. This was carefully considered and experimented with the example of one school, and then it was seen with the example of three schools.

When transitioning to this process in schools in stages, it was not done at once, and in a word, at each stage, we made the necessary conclusions for ourselves. Yes, if that doesn't work for us, it works for us. For some reason, it is only through research and self-implementation that conclusions are drawn based on which elements work and how they work in us or show results.

What is the difference between this five-point assessment system and the five-point assessment system that has been in effect to this day and is still in effect in some of our schools? The main difference is that in our evaluation, the types of evaluation differ in purpose, and in this evaluation system, we mainly use two main types: formative and summative. Even today, in the same five-point system, there was a formative and summative assessment, a class assessment, and so on. Only in our five-point system, we give a great advantage to current grades, which are placed precisely during the lesson.

I'm asking why the preference is given. As I said, when we studied it on the example of one school, we faced such a reality. The children's grades are not bad, the grades in the journal are not bad, but if we took the children and monitored them according to their own study programs, if we monitored them from outside, the result was completely different.

We have a question: Why are the results of monitoring different from the results of the child in the journal? Gradually, the reason becomes clear. Therefore, even a child who has an excellent grade cannot enter higher education if they do not go to a tutor. This means that his grade in school is clear, not based on criteria, his grade is different everywhere. If we check his knowledge, it gives different results everywhere.

The most interesting thing is that as a result of our observation, the assessment given during the lesson is the assessment given by one teacher, which is a subjective opinion, always the opinion of one person. If the teacher has taken not one, but several classes, he teaches, his assessment of several classes, several classes to students, based on his subjective personal approach, he makes a decision that the student "knows" or "does not know" in this or that subject, and he evaluates it.

Isn't that true in the current system?

This system is the first to try to prevent this subjective thought. In order for one teacher to say "knows, doesn't know" to a class of, say, 40 students, first of all, even if the best specialist has high potential, there can be other factors there.

We've all been teachers, let's face it. For example, suppose we go to a class and we are given three classes. I went to class, evaluated the students, passed the topic, and gradually began to evaluate... I don't know anything about them yet, I don't have an opinion, but there are some of them who are naturally very responsive or quick to communicate, raise their hands, try to answer if they know, and look at the book.

Even if someone knows, there are children of different natures. There are children with all kinds of abilities. Let's admit that three times when we go to class, the students in this class are given an approximate grade. Right?

The general conclusion is that there is distrust in the teacher's grades, is there not?

What is the opinion of Komil Jalilov, an expert on education issues on the new assessment system? Komil aka, what are your general conclusions and opinions on the new evaluation system based on criteria?

Before I answer your question, would you mind if I explained the scientific side of a little situation to make it clearer, to make the subject clearer? Look, there's an evaluation in pedagogy, there's a science called testology. For example, in many countries, higher education is taught, doctoral dissertations are defended, and others. My master's thesis, for example, which I defended in London, was precisely about evaluation, and we studied evaluation as a separate subject for two semesters.

So for the sake of clarity, let me briefly explain the scientific side of this. Look, there are different types of evaluations. Depending on what you're evaluating it for, there are three main types. This is an evaluation system based on criteria, this is an evaluation system based on norms, and this is an ipsative evaluation system.

What is a norm-based evaluation system?

For example, suppose you want to evaluate a fifth-grade boy, for example? You first define a norm, that is, a scientific "representative sample," as it is called in scientific language. That is to say, you're going to gather different groups to show this average fifth grader. For example, a fifth-grade child who studies in one village, studies in one city, studies in one school with good conditions, studies in one school with bad conditions, one rich family, one poor, in short, says "representative sample."

You collect this, and when you give them the same test, you take their score as the norm, and then, the next, the same real score, the same fifth graders, you compare their score with this norm every time you grade them. Now, to what extent it is close to this norm, not close, but deviated, did not deviate, that is, you can compare those results with this norm later. This is a norm-based assessment.

What is a criterion-based assessment?

When evaluating based on criteria, you set some criteria. For example, a fifth-grade child must answer at this level in order to know the same mathematics program as a fifth-grade child, for example, in mathematics. When you set the criteria, it should answer at this level. To know four, he must answer at this level. Each time, you compare the results you get based on this criterion.

What is an Ipsative Assessment?

In the case of an Ipsative assessment, it is often used to assess children from poor families, for example, in America, and to assess school performance. This is compared to the previous result of the same student. You will see how much progress has been made. For example, it was like this before, now it is like this, that is, by comparing it with its previous result. Therefore, this testology has these three different evaluation systems.

Now, look, even in this five-point system, it is still based on some criteria, because no matter how subjective the teacher is, when evaluating the student, it is still based on some criteria. The criteria may not have been written down on paper, they may not have been formalized, but the teacher is still a teacher, he has read it, he has learned it, he knows it, even if he is in his own mind when he puts the grade.

For example, a fifth grader says that it is necessary to know these things in order to know literature or their native language, say, by five, and based on this when evaluating the student. It is true that there is a certain degree of subjectivity here, but it would be wrong to say that the five-point system is not based on criteria at all.

To this expert's question, Khudoyberdiyeva Mukhayyokhon, a teacher from school No. 216 in Yashnabad district, Tashkent city, who introduced a new assessment system, replied as follows:

Yes, for use by teachers. Teachers, for example, have the same requirements, BSB, PSB, each of which is assigned a summative assessment, each of which has its own separate evaluation criterion.

Komil Jalilov: No, here's an apology. I've seen the demo version, and there are questions going on, let's say four points if you solve this one, and one and a half points if you solve this one, and there are three levels of those questions: knowledge, application, reasoning, and each one is evaluated according to that. But, well, for example, this boy solved this option, from that option, let's say, 40 points, for example, 25 points. Is this good, average, or bad? What's this defined for? Are there clear criteria for this, are there clear stone scales?

Teacher: In that case, for example, we will also put an evaluation criterion in front of the child, and then a parental meeting will be held separately in the classrooms, and separately in the school, and each subject teacher should know how many points to how many. For example, before we moved on to the new valuation, there was a [order] for adding the value of 3 itself. Less than 56, less than 55, would be 3. Now that we've moved on to a new evaluation based on those criteria, it has been downgraded.

Komil Jalilov: On the basis of what was it downloaded? There were 56, now there are 30. Based on what? Is there a definite scientific basis for it? Why 30? Why not 31? Why not 35? Why not 28?

Teacher: We asked about this thing when it was passed to us, and we asked why exactly 30 was taken, but it's not an approximate thing to take. When they are said, according to a new assessment based on the same criteria, the same, for example, external controls are taken, according to monitoring, at the time of their receipt, in general, the percentage of assimilation of our own students turned out to be very low.

Komil Jalilov: Then it will not be based on these criteria. As I said earlier, for example, let's take a specific subject, let's say, the subject of literature in the fifth grade. What is the purpose of literary science? Let's say, teach a child to work with an artistic text, right? For example, what should be concrete about it? Must have a criterion. If a child analyzes a literary text at this level, I put 3 or 30 points on it. If he analyzes at this level, I will put 4. Yes, you say that we took the general picture, and according to that...

Dilrabo Davidova: You are drawing a wrong conclusion. May I express my opinion here? Komil aka said that testology has its own requirements, it is taught, but he said one thing: "In a five-point system, the teacher evaluated with some criteria." Somehow? The whole point is that everyone answered "somehow" in their own way.

Now, Kamil aka, don't confuse the standard requirement with the scale. What you're talking about is, look, you've asked us this question many times, we've tried to answer your question, and we've even invited ourselves to the center several times. We've just been to Oxford's AKA, where we've been taking grading courses, senior courses. What will happen when we get there?

There are standards. It is true that we have a standard program in some subjects, in some classes, and a national program. But we are now working on it as a standard for ourselves. Now the work on improving the standards has begun. Now we have worked on it and developed qualification requirements for those standards based on the requirements of the national program.

These curricula were developed based on qualification requirements. A calendar plan has been developed. Based on the calendar plan, questions are being developed in accordance with qualification requirements.