Nova workboard

a blog from young economists at Nova SBE

PISA testing: the perfect tool?

1 Comment

Education is one of those subjects where evaluation of results is most difficult. Not only is it difficult to separate value added from one year of schooling from already existent human capital (or ability), but also due to the fact that most of the effects of education are felt in the long-run. Moreover, when one wants to compare policy, one must also be able to compare results in one school with another, in different years and between countries if possible. Finally, different policies can yield similar gross results albeit with very different cost effectiveness.

One of the methods created to address some of these concerns was the Programme for International Student Assessment (PISA). It was held first in 2000 and currently it evaluates Mathematics, Science and Reading in 65 countries, every three years. This was a huge advance, for it allowed to compare results in different countries, which naturally have different curricula. Also, being somewhat similar in difficulty, it should also be comparable across different years.

However, some caution is needed when doing analysis on this kind of data. One must always remember that this is aggregate data. One of the problems with aggregate data is that, by looking at it, one might interpret relationships at an aggregate level that do not actually hold at the individual level. This is called an ecological fallacy. In the case of education, we are coupling together different schools, students in a country average that might not make sense. What is more, PISA reports look mostly at correlations, but they cannot infer causality from cross-country data.  Finally, there is another problem, which in my view is the most important. Policies in education take time to act, and thus it is not easy to assign results to given policy.

Nevertheless, PISA results (and others, like TIMMS) are still a significant step in gathering better data for accurate analysis. Also, there are indeed some papers like Ramos et al. (2012) do use PISA microdata to make analysis and draw informed conclusions.

#628

José Miguel Cerdeira

Advertisements

Author: studentnovasbe

Master student in Nova Sbe

One thought on “PISA testing: the perfect tool?

  1. On December 2nd, the results for the PISA 2012 were published. As said in the post, it is considered a controversial test as it can raise disagreement about comparisons between countries. I totally agree and, regarding the new ranking, I would like to add that maybe peer effect could also be taken into account to explain the results. Peer effects set the environment for students’ learning and it can either participate in improving the benefits of going to school or in worsening them even if students have the same ability and follow the same curriculum. Furthermore, I think that cultural aspects can influence peer effects a lot.

    If we have a closer look at the PISA results, Asian countries occupy the first seven places, leaving European countries behind. If we look at the example of France, it occupies the 25th place, losing two places since the last PISA. The results show an increase of inequalities among students, enlarging the gap between very good and weak students. In my opinion, the problem is partially due to peer effects, which increase the standard deviation among students’ results. As far as I know, in France, among 15-year old students in particular, good students are sometimes stigmatized and it can be difficult to handle psychologically speaking, especially when good students are in minority. Indeed, due to adolescence, the character is not strong enough at that age and peers’ opinions are very important.
    On the contrary, Asian mind-set is very different, education is highly valued among individuals and I think the success of Asian countries in the PISA can be partially explained by the quality of peers there: respect and discipline are pillars in the Asian culture and this reflects on peer effect.

    Thus, in my opinion, a good peer effect is one of the causes for a better benefit of education and higher scores at the PISA.

    Elise Jaillant #1549

    References:
    Articles “Le Figaro” (French newspaper)
    http://www.lefigaro.fr/actualite-france/2013/12/03/01016-20131203ARTFIG00338-niveau-scolaire-la-france-perd-deux-places-au-classement-mondial.php
    http://www.lefigaro.fr/actualite-france/2013/12/03/01016-20131203ARTFIG00339-baisse-du-niveau-scolaire-la-pedagogie-ne-s-est-pas-adaptee-a-la-generation-internet.php