Tamil Nadu Critical Thinking (TNCT) pilot study (2022-2024) tested the hypothesis that adolescent school children can be taught critical thinking skills through a separate curriculum as an effective strategy to counter misinformation and disinformation challenges, among other benefits. The curriculum in Tamil- tailor made for 12-14 year old students in government and government-aided schools of Tamil Nadu- was implemented in classes 8 and 9 in 19 schools of an urban education block, while the remaining 21 schools in the block were the control group. In the end line test results- on average, being part of the treatment group is associated with a 17% percentage increase in critical thinking skills score compared to being part of the control group. This difference is statistically significant at the 95% confidence level and multiple robustness checks indicate a causal relationship between the CT sessions and the end line test scores. The pilot has also helped identify areas for fine-tuning the curriculum and improving the implementation strategy for a scale up.
All six thousand seven hundred students individually tested were for the baseline assessment
Issue
Large scale misinformation and disinformation are today global phenomena affecting societies of all socio-political characteristics. Reports suggest India has been the most affected country in recent times, where fake news has engendered hate crime and social unrest. Susceptibility to information manipulation has debilitating negative impacts on public beliefs about health, science, and intercultural understanding too. The rapid growth of access to the Internet and usage of social media platforms combined with diminishing trust in legacy media has increased the avenues and scope for those interested to propagate fake information and spread baseless narratives. Neither state regulation nor censorship and other measures by social media platforms have been successful in preventing online propagation of lies. In particular, these strategies fail when an authority in power decides to spread disinformation. Therefore, a sustainable solution ought to be to strengthen individual capacity among citizens to reason and distinguish truth from fiction.
Solution- the TNCT Pilot Project Hypothesis
Teaching adolescent school students critical thinking skills will lead to improved intellectual ability to identify poor quality information, evaluate information for reliability and accuracy, differentiate strong arguments from weak reasoning, and draw sound inferences across varied topics. Critical thinking (CT) is the process of skillfully conceptualizing, applying, analyzing, and evaluating information as a guide to belief and action. Teaching it to school students will inculcate the skills required to protect themselves from misinformation, disinformation, and online frauds and instigators by laying the intellectual foundations for developing scientific temper, humanism, and a spirit of enquiry and reform as envisaged by Art 51(A)(h) of the constitution.
Curriculum
The age appropriate culturally relevant curriculum developed by the author focused on building four key critical thinking skill set
- Analyze- a topic, think deeply/ on a complex matter, identify gaps in reasoning and untenable assumptions
- Evaluate- assess available/ new information in a systematic manner; question and reflect one’s priors
- Infer- reach decisions/ opinions/ judgements based on critical analysis and evaluation of available information on related topics; differentiate weak/ flawed arguments from strong ones
- Create- habits and strategy to navigate exposure to unsound ideas, reject unreliable information, reject manipulation of feelings
The curriculum was designed on the basis of the ‘teach at the right level” approach (TaRL). TaRL prioritizes introducing topics that are at the level of a student’s current ability rather than the expected level of ability for a student’s grade. It has been found to be the most productive approach to teaching at schools (World Development Report 2017, n.d.). Through activities based on stories, situations and events that the students could relate to, they learnt to frame questions, gather data, apply data, draw analogies, consider implications, explore different points of view, and communicate standpoints. These transferable skills (Abrami et al., 2015) were expected to not only strengthen reasoning skills but also improve student curricular achievements. The sessions and their delivery were designed such that poor reading or numeracy skills, or lack of resources at home did not much impede student participation in the class. The continuous assessment of student performance too was mostly independent of student ability to read and write proficiently.
Three parameters guided the TaRL approach to designing this curriculum:
(i) Our baseline assessment showed that a large fraction of the students was not able to read or write well in either English or Tamil. Therefore, except for the reading skills, topic, the activities were designed for student participation irrespective of their reading and writing abilities. Where reading was essential to an activity, it was usually a group activity where students could pool their strengths, or it was designed such that the tutor would lead the reading. (ii) Topics for discussion, stories, examples that, etc., where chosen to be age-appropriate and culturally irrelevant rather than based on the assumption that they would be familiar with their textbook content from the current year or the previous year. (iii) Rubrics for assessment of class activities deliberately did not factor in language errors. Feedback to children, too, was focused on improving their understanding of the concepts and application of what they have learned, but not the sophistication in their expression.
Peak into CT Teachers Training 1
Intervention
A total of 3291 students, 49% of the total 6711 students enrolled in grades 8 and 9 of the 41 government and aided middle and high schools in an urban education block, received this intervention. The other 3420 students formed the control group. Randomization was at the school level. In terms of gender, 875 boys and 2416 girls were assigned to the treatment arm while 2100 boys and 1320 girls were assigned to the control group. The gender imbalance, due to many single gendered schools and not statistically significant at the 5% level, is factored in the analysis.
TNCT project Causal Chain
TABLE 1: SAMPLE SIZE BY SCHOOL MANAGEMENT CATEGORY
| Management Type | Treatment | Control | Row Total | |||
| Schools | Students | Schools | Students | Schools | Students | |
| Government | 5 | 320 | 3 | 316 | 8 | 636 |
| Fully Aided | 10 | 2336 | 17 | 2747 | 27 | 5083 |
| Partially Aided | 4 | 635 | 2 | 357 | 6 | 992 |
| Grand Total | 19 | 3291 | 22 | 3420 | 41* | 6711 |
Note* One management runs an aided school and an unaided school with the same school code. Since only one of these two schools received the random treatment assignment, the school code is added both in the treatment and the control groups. Administrative records will show a total of 40 schools only. The project was implemented between September 2022 and April 2024 in three major phases: (i) preparation (pre-baseline) and baseline (ii) random assignment and intervention, and (iii) end line assessment. Due to the unprecedented December 2023 floods that affected school infrastructure and the academic calendar in the district variously, against the planned 30 sessions, the maximum any school received were only 19 sessions.
Results
In the primary measure of outcome- the overall score obtained by the students in the end line test on critical thinking skills- on average, being part of the treatment group is associated with a 17% percentage increase compared to being part of the control group. This difference is statistically significant at the 95% confidence level and multiple robustness checks indicate a causal relationship between the CT sessions and the end line test scores. Adding covariates did not affect the direction or significance but reduced the effect size slightly.
At the most conservative estimate from comparing all the models, on an average, being part of the treatment group is associated with a 14% percentage increase compared to being part of the control group. The Cohen d effect size is 0.409 standard deviations, suggesting clearly perceptible treatment effect where 0.2 standard deviations and above is the norm in education research.
Policy Implications
This project was implemented with the aim of piloting an idea and the results so far have validated the rationale that it is possible to teach adolescent school students critical thinking skills as a discrete subject even when the students are not fully proficient in reading and writing. Subsequently, there are a number of compelling reasons[1] for scaling up the project:
(i) The primary issue that this intervention tried to address continues to be one of the biggest hurdles to democracy (Breakstone et al., 2016). The other solutions such as fact checking units and counter narratives have proved to be insufficient in the face of organized disinformation campaigns. Relying on them alone could also lead to loss of trust among the people for any source of information.
(ii) The TNCT model is a sustainable model. Because its strategy is to enable the citizen to distinguish truth from lies and propaganda, once a person is equipped with the skills, she is potentially protected lifelong. Her primary defense mechanism against deception would be her own intellect with fact checking units and counter narratives etc. aiding her.
(iii) The TNCT model has additional benefits too. CT skills are protection against gullibility in general, which can prevent one from falling prey to health misinformation, online child abuse, financial frauds etc. too, improving the state’s overall crime rates.
(iv) This pilot has demonstrated that both the curriculum and the approach to teaching it are viable policy options. The positive results achieved are based on teaching only a part of the curriculum. Moreover, the per session effect analysis shows that there is a clear impact for each additional session taught to the students. Therefore, a scaled up intervention where the full curriculum is taught is expected to yield an improved achievement, and scope for further refinement of the model.
(v) Scaling up will be in line with existing policies of the state. Rational thinking is the stated foundation of the state’s approach to policy making. The constitution too directs the states to inculcate among citizens a scientific temperament. Tamil Nadu will lead the country in one more education policy innovation (only Kerala has so far announced their plan to introduce a subject for scientific thinking starting from 2025-26). That too with minimal additional resources and without compromising existing focus on other areas of student development.
In view of these factors, progression on the idea could be to roll out and evaluate the intervention in a larger geographical area. Besides further validating the results from this pilot, such an approach would accord us the scope to fine tune the implementation methodology for an eventual state wide roll out. Attention to detail and rigorous customization would help ensure a greater success for the state and the policy.
Limitations
A caveat on the experiment design and another on project implementation are in order. Due to resource constraints and policy principles of the state for piloting new curricular ideas, the experiment was conducted in one education district only. The 41 schools in the study were therefore not decided based on power calculations. All results need to be interpreted allowing for this detail and the additional analyses in the form of bootstrapping and randomization inference.
Second, due to the sudden changes in the curricular calendar of Tamil Nadu schools in the months of March and April 2024, additional end line assessments to test whether the CT sessions impacted student performance in their usual subjects such as math and English could not be conducted. In the parliamentary election schedule announced in March, Tamil Nadu featured in the first phase. This meant curtailing of some academic hours for all subjects as well as rescheduling school final exams. Therefore, only CT skills test could be conducted for the end line.
[1] Based on Flemish Association for Development Cooperation and Technical Assistance
Baseline Assessment
For baseline, tools developed by Annual Status of Education Report (ASER)[1] were used to individually evaluate all available students their proficiency in Tamil, Math and English. The tests were administered in the school premises and the scores used as the baseline natural intellectual ability of the students. Each student was tested individually by a teaching assistant who encouraged the student to attempt a question multiple times, and helped with understanding the questions if need be without helping with the answers. The approach was to ensure that the baseline ability was not underestimated leading to a biased effect estimator in the endline. [1] https://asercentre.org/aser-tools/
End line Assessment
Due to the unexpected changes to the final exam calendar because of the parliamentary elections 2024 schedule, end line student performance could not be individually evaluated for Tamil, Math and English. Their critical thinking skills were assessed in the form of a written exam conducted in their school premises. The questions tested students’ ability to identify false news, instances when they should seek more information before arriving at a conclusion, evaluate information based on evidence, strengthen or weaken an argument, draw correct inferences, draw analogies and prepare to solve a problem. Marking did not penalize language errors. All analyses are at test level in this report.
The end line exams were conducted by Illam Thedi Kalvi (ITK) teachers, who had never been part of the project or aware of the curriculum. Since the focus of the end line assessment was to evaluate student critical thinking skills, the teachers read out aloud the questions so that students who are not proficient in reading could still answer the questions. Some questions in the exam were based on tests developed by universities[1] abroad but adapted for our student age group. Except for one question, the questions were equally new to the two arms of the experiment in both format and substance. [1] William & Mary School of Education https://education.wm.edu/centers/cfge/_documents/resources/tctinstru.pdf
FAQ
- Is this the ideal method to teach critical thinking?
No, the ideal method would be for all teachers to be teaching all subjects using the critical thinking approach- helping students learn how to evaluate and synthesize information, question assumptions, meditate on increasingly complex topics, avoid logical fallacies and reason scientifically, be wary of biases and solve problems rationally.
But integrating CT skills in every lesson across schools and classrooms would be a decades-long project involving rehauled teacher recruitment, a revised teacher graduate curriculum, re-training current teachers, and developing new rubrics for test and exam evaluations. In the meantime, teaching these skills as a co-curricular subject would address the urgent need to incorporate rational thinking abilities in the school curriculum.
- Did this project add to the financial burden of the parents?
No, there was no additional fee or any other charge collected from parents for this project. The CT sessions did not add to the financial burden of the parents or even the take home curricular burden of the students in anyway. There was no textbook or notebook purchase, no homework except one small simple survey in the whole year, and no term/midterm/class tests. The tutor carried the activity sheets for the students and gave them only constructive feedback.