Nova workboard

a blog from young economists at Nova SBE

The best Portuguese schools: should we trust the rankings?

Every year, the rankings of the best Portuguese schools are published and disseminated throughout the media. The indicator used to organize such ranking is the average national exams score results.

The analysis of schools according to such rankings has received, over the past few years, a large criticism. This year was no different: after the announcement, the controversy came up again.

A thorough analysis of the results would be interesting, but for lack of space, there are two consequent conclusions I would like to emphasize. First, private schools continue to have on average much better results than public schools. To illustrate this idea just notice that there are only 9 public high schools in the top 50, while in the similar ranking for the 9th year there are only 7 public schools. Regarding the 9th grade, this difference is quite visible in the following figure.

Image

Secondly, analyzing only the results of schools where there were over 50 tests, the results show a large difference at the territorial level, being the first places dominated by schools in the North Coast[1].

The results and consequently the corresponding rankings are undoubtedly objectives; but their interpretation is far from consensual. The debate is on whether we should use this indicator as a measure of school performance.

In my point of view, these results should not be used, in a decontextualized way, to assess school performance. In reality, this indicator does not take into account that schools have different characteristics, both in terms of social environment and in terms of human and material resources.

Because of this, there is a huge literature that points out to the weakness of this indicator. In particular, it is interesting to consider the Rita Azevedo’s WP[2] (2011) regarding Portuguese 9th grade exam results in which the author found evidence for a significant causal effect between socioeconomic aspects and school achievement. The conclusion reinforces the weakness of the indicator for the Portuguese case.

The data released by the Ministry of Education shows also socio-economic indicators of public schools (not revealing any information on private schools). Based on this information, the Público’s study divided schools according to economic contexts and calculated an expected value of the average results for each context.

Based on this, I think it would be more appropriate to use an indicator which compares the expected average test scores results with the obtained values  (in this line, Azevedo also proposed that an alternative solution would be to consider the ratio between the expected value and the current average school scores). Although it has some limitations, it would help to draw more truthful conclusions regarding school performances.

However, I cannot deny the importance of the divulgation of the results obtained by the students, because they are a key tool to make comparisons, within the same school (comparing with the previous years) or even between schools at the same context. However, we have to take into account that a comparison over time requires that the tests are comparable between different years which may not be the case of Portugal. Moreover, they are important for the government to be able to identify and combat the causes of the poor performance of some schools and regions.

To conclude, the results should be presented to the public opinion in a careful way; the media should not label the rankings as “the best Portuguese schools”. It is important that the public opinion realize that a school cannot be considered good or bad based only on the average test scores. This conclusion is not only relevant for Portugal since there are other empirical studies that find the same evidence for different countries[3].

Filipe Silvério, #617


[1]This situation is perfectly depicted in the following images, in which is possible to see the average test scores (of Mathematics and Portuguese) for high school and for the 9th year of Mathematics and Portuguese by municipality: http://imagens5.publico.pt/imagens.aspx/809225?tp=UH&db=IMAGENS http://imagens6.publico.pt/imagens.aspx/809226?tp=UH&db=IMAGENS

[2] Azevedo, Rita, “Critical Analysis: Portuguese 9TH Grade Exam Results and Socioeconomic Factors” 2011, Work Project for the Master in Economics, Nova SBE

[3] For instance, a similar study for Chile: Mizala, Alejandra, Pilar Romaguera, and Miguel Urquiola, 2007, “Socioeconomic status or noise? Tradeoffs in the generation of school quality information”, Journal of Development Economics, 84, pages 61-75

Advertisements

Author: studentnovasbe

Master student in Nova Sbe

Comments are closed.