Rev. A. J.

Deignans Dream of Wah Yans Future

Francis T. Lui

(Class of 1964-71, WYHK)

10 December 2006

(1)

Introduction

In his manuscript, Let us dream of Wah Yans future, Fr. Deignan has made several proposals to enhance the quality of education at Wah Yan. These consist of three broad categories of measures, namely, splitting classes into smaller ones, improving the quality of teachers through exchange programs and other means, and providing greater support for extra-curricular activities. All of these measures need additional financial and human resources.

The Wah Yan One Family Foundation wants to consider whether it should raise funds to support Fr. Deignans proposals. To uphold high standards of accountability to potential donors, the Foundation has requested the author of this report, who is an academic economist by training, to conduct an independent research to assess the costeffectiveness of the above proposals. He believes that the best way to perform his duty is to review the literature and draw conclusions based on the evidence from relevant studies at the frontier of research, which are known to be good enough for refuting or justifying national educational policies.

Many questions can be raised about the above proposals. Financial and human resources can be interpreted as inputs to the production of student outcomes. Do we know that these inputs are really very useful? How can we make sure that they will not be wasted? Parents personal experiences may suggest that they give more to each child when they have fewer children to deal with. Can these experiences be extrapolated to drawing conclusions on the usefulness of small-class teaching? How do we measure the success of students? When we see that some students in small classes are doing better, can we be sure that the successes are not due to factors other than reduction in class sizes? How do we measure the quality of teachers? How important are experiences and educational achievements in determining the quality of a teacher? A famous report published in 1966--- the Coleman Report--- seems to suggest that family backgrounds are more important than school differences in determining student outcomes (Coleman et al (1966)). Do schools and teachers really matter? Extracurricular activities yield

benefits, but they are also costly. More time spent on these activities could mean reduction in time spent on academic studies. Do we know that promoting extracurricular activities is a good idea? We can go on and cast more question marks, but the above should have helped us appreciate the complexity of the issues involved. Fortunately, modern research on the economics of education has provided us with reliable answers to many of the questions being raised.

The literature related to the issues above is vast. An early survey paper has reviewed close to 400 relevant studies (Hanushek (1986)). This report is not an attempt to survey the entire literature. Rather, it focuses on four authoritative studies, which are conducted by top researchers in the field. They are done by Krueger (1999) of Princeton, Hoxby (2000) of Harvard, Rivlin, Hanushek and Kain (2005) of Amherst College, Stanford University, and University of Texas at Dallas, respectively, and Carneiro and Heckman (2003) of the University of Chicago. Each paper addresses some conceptual issues related to the economics of education. More importantly, they present evidence based on large-scale quantitative studies that can help us answer many of the questions related to Fr. Deignans proposals. It should be noted that some of the findings of these studies may not and should not come as surprises to frontline teachers and school administrators, who have extensive experiences of observing student performances. The value of the findings lies in the fact that they are obtained by well defined scientific analyses of large databases..

The plan of this report is as follows. In Sections 2 to 5, the studies will be individually described and their results presented. Section 6 discusses the possible implications that we can draw from the studies. Section 7 concludes and makes recommendations. Readers under the constraint of time may focus on Sections 6 and 7 only.

(2)

The Krueger (1999) Paper: Class Size

Educators generally believe that student performances are determined by numerous factors, of which class size may be one. To establish that reduction in class size is indeed beneficial to learning, researchers have to isolate its effects from those of other variables. One way to do so is to conduct the so-called control experiments. This approach is all the more important because many early studies, which have not properly controlled for the influences of other variables, are unable to find any systematic relationship between class size and student outcomes (Hanushek (1986)).

Using data from a control experiment, Krueger (1999) has shown that small-class teaching indeed raises student achievements. It is an econometric analysis of a large-scale randomized experiment on class size, known as the Tennessee Student/Teacher Achievement Ratio experiment, or Project STAR. The project was a longitudinal study in which kindergarten students and their teachers were randomly assigned to one of three groups beginning in the 1985-1986 school year: small class (13-17 students per teacher), regular-size class (22-25 students), and regular-size class with a teaching assistant. After the initial assignment, students have to remain in the same class type for four years. About 6000-7000 students were involved in each year. Over the four years, the sample included 11,600 students from 80 schools, which spanned the entire ranges of poor or well-off, and rural and urban districts. Each school was required to have at least one of each class-size type. Random assignment took place within schools. The students were given standardized tests at the end of each school year to measure their academic achievements.

Table 1 contains a summary of the average scores obtained by students belonging to small and regular class types. The figures in the column Differences represent the differences between the average score in small and regular classes. As we can see, students in small classes, when compared to the control group, i.e., those in regular

classes, on the average always do better.1 This can be interpreted as evidence that reduction in class size does matter.

Table 1: Average Percentile Scores of Students belonging to Different Class Sizes. _____________________________________________________________________ (A) Students who entered STAR in kindergarten Small Class Percentile score in kindergarten 54.7 Regular Class 49.9 Differences 4.8

(B) Students who entered STAR in first grade Percentile score in first grade 49.2 42.6 6.6

(C) Students who entered STAR in second grade Percentile score in second grade 46.4 45.3 1.1

(D) Students who entered STAR in third grade Percentile score in third grade 47.6 44.2 3.4

_______________________________________________________________________

(3)

The Hoxby (2000) Paper: Class Size

The Krueger paper has produced some attractive results, but it has also been criticized for ignoring the so-called Hawthorne effect. The latter is a phenomenon referring to the possibility that some individuals may temporarily increase their productivity at the time when they are being evaluated. The policy experiment of Project STAR implicitly contained incentives for the teachers and school administrators involved to behave strategically. Teachers in small classes might have work harder than others simply because they hoped that better performances of their students would persuade the
1

The Krueger paper contains more sophisticated statistical analyses, which generate similar conclusions. For the sake of avoiding overly technical discussions, these are ignored in this report. We also ignore the results for those classes with teaching assistants.

government to make class size reduction a permanent policy. If this interpretation is true, the improvement in student outcomes reported in the Krueger paper may be due to extra working efforts by the teachers and administrators, rather than the smaller class sizes per se. To remedy this problem, a properly designed control experiment should not have built-in incentives to induce teachers in small classes to work harder than those in regular classes. Ideally, the teachers should not even know that they are subjects being studied.

The study by Hoxby (2000) analyzes the data from a natural experiment where the Hawthorne effect does not play any role. The state of Connecticut has 649 elementary schools belonging to 146 elementary districts. Every year since 1986, its state educational authority has administered statewide tests in the fourth, sixth, and eighth grades. From 1986 to 1991, data of test results are available by school district, and from 1992 onwards, the data are available by school as well as by district. While similar data are available in other states, the study of Connecticut has a special advantage. Detailed data on enrollment, class size, and achievement are available by district, by grade, and by age cohort. Hoxby has also noticed from the data that there are wide variations in class sizes. The first percentile is 8 students per class, and the ninety-ninth percentile is 34 students. This phenomenon results from the fact that class sizes at a particular school for a particular student age cohort depend critically on the population of students available, which varies a lot over time and across different districts. These data form the basis of a natural experiment. Researchers can analyze them to find out whether there is any relationship between class size and student performances. Teachers and administrators participating in Project STAR knew the objectives of the experiment beforehand, but those in Connecticut were not even aware of any experiment being conducted and so there was no basis for the Hawthorne effect to exist.

Hoxbys detailed statistical analyses of the Connecticut data indicate that class size reductions, at least in this state, are not effective for improving student achievements. In the literature, it is sometimes argued that students from low-income or disadvantaged families can benefit disproportionately more than others from class size reduction. Hoxbys study also does not support this hypothesis.

Hoxbys results cast doubts on pursuing a policy of class size reduction, but they have to be interpreted with care. First, it is not clear that these can be carried over to large classes with, say, over 30 students. Second, the results are generated by data from elementary schools. There is a possibility that they are not applicable to secondary schools. Third, we need to understand why there is a difference between the results of Krueger and Hoxby. Smaller classes may make it easier for teachers in small classes to do a better job, but it does not mean that they have the incentives to do so. Class size reduction policies should contain built-in evaluation and incentives.

(4)

The Rivlin-Hanushek-Kain (2005) Paper: Teacher Quality and Class Size

The two papers above deal mainly with the issue of class size reduction. However, experienced educators, such as Fr. Deignan, would understand very well that other factors also matter. As Fr. Deignan has remarked in his Let Us Dream of Wah Yans Future, The secret of a good school is having good teachers who are committed and enthusiastic about their subject. From the empirical studies, what is the evidence on the role of teacher quality? What are the attributes of good teachers and how do we identify them?

The paper by Rivlin, Hanushek and Kain (RHK) is an ambitious attempt to answer a lot of interesting questions related to educational policies. The most important contribution of the paper, however, is its treatment of teacher quality. Since the publication of the Coleman (1966) report, many education researchers have been skeptical about the effectiveness of school and teacher qualities on student performances. The skepticism is partly driven by the findings that some observed attributes of teacher quality, such as their educational attainments and number of years of experience, do not seem to affect student outcomes. There are two interpretations for these negative findings. First, perhaps school and teacher qualities are indeed not important. Second, teacher and school qualities have not been properly measured in previous studies. The

RHK paper follows the second explanation and adopts a new approach to measure teacher quality. Instead of relying on the teachers personal attributes, the paper uses an outcome-based estimator for teacher quality. A teacher is regarded as good if he or she is more capable than others in the same school of improving student achievements. This estimator is not feasible if a typical teacher has only taught one and the same class. In order to come up with reliable estimates of teacher quality, researchers need extensive data involving teachers who have taught classes of different types and students of different abilities.

The data used in RHK come from the UTD Texas Schools Project. Data are compiled for all public school students in Texas. The paper uses data from three cohorts in the mid 1990s, each consisting of more than 200,000 students. The large-scale database contains enough information to permit the researchers to make much more precise estimates than those in previous studies. Statistical analyses of the data have generated a number of interesting results.

First, teachers, and therefore schools, matter importantly for student achievements, once teacher quality is properly measured.2 The estimated variation in the quality of instruction reveals an important role for schools and teachers in promoting economic and social equality. High quality instruction can substantially offset disadvantages associated with low socioeconomic background.

Second, there is absolutely no evidence that having a masters degree improves teaching skills.

Third, there appear to be important gains in teaching quality in the first year of experience and smaller gains over the next few career years. However, there is little evidence of further improvements after the first three years.

Watts and Bosshardt (1991) also find similar results, using a much smaller data set.

Fourth, principals of schools usually know who the good teachers are. Identifying them does not seem to pose a big problem.

Fifth, the effects of class size on mathematics and reading achievement gains appear to be modest but statistically significant. These effects, however, weakens as students progress through school.

(5)

The Carneiro and Heckman (2003) Review: Extracurricular Activities

A vast literature in the economics of human capital has long established that education is a powerful engine of social mobility. There is a very strong and statistically significant relationship, ceteris paribus, between a persons life-time income and the number of years of schooling the person has attended. This indicates that classroom instructions, which help to develop students cognitive skills, are useful in raising their future incomes. However, schooling is not just about knowledge or cognitive skills. Noncognitive skills and attitudes, such as perseverance, sense of responsibility, organization skills, social skills, appreciation of artistic beauty, and good judgments of right and wrong are increasingly recognized and valued in the employment market. These skills are often acquired outside of the classroom through extracurricular activities. A persons education can hardly be deemed complete if he or she scores poorly in non-cognitive skills.

Most educators of course understand the usefulness of extracurricular activities. However, the latter are not costless because they take away the time that can otherwise be used in productive academic studies. The possible tradeoff between extracurricular and curricular activities leads to the question of whether schools are investing too much or too little resources in either type.

Early studies indicate that the tradeoff, if any, is in fact not serious. For example, Mehus (1932) has shown that poor academic performance is usually not caused by active

extracurricular participation. Hill (1944), in another control experiment, has found that extracurricular activities have small, but positive, effects on academic achievement, and they improve social adjustment of students.

Economics Nobel laureate, James Heckman, has made many contributions to the research on the effects of non-cognitive skills. In Carneiro and Heckman (2003), there is a comprehensive review of the modern literature. The weight of the evidence seems to lean heavily to the side that schools should invest more in extracurricular activities, which can stimulate the development of non-cognitive skills. There are several reasons for this.

First, an important component of cognitive abilities, the IQ of a person, has a low degree of malleability after the age of eight. That means, schools can contribute very little in this regard because most of the students are older than this age. On the other hand, at least up to the adolescent years, young people can still acquire non-cognitive skills relatively efficiently. Extracurricular activities in schools therefore can potentially make a difference.

Second, various long-term longitudinal studies of investments in young children from disadvantaged families aimed at improving their non-cognitive skills have shown that they are remarkably successful. For example, analysis of the data resulting from the Perry Preschool program shows that for each dollar invested in the children, the present discounted value of future benefits (during the lifetime of the children) for society is 8.7 dollars! The improvement in non-cognitive skills, among others, can significantly reduce the future crime rate of the affected children. The enormous benefit-to-cost ratio of 8.7 indicates that non-cognitive skills are indeed very important. So long as the ratio is bigger than one, it makes economic sense to invest more.

Third, even though the result above is generated by studies of young children, research projects on adolescents suggest that the latter also benefit a lot from programs that improve their non-cognitive skills. An example of such programs is the Big

Brother/Big Sister program, through which students can be matched to some older mentors.

(6)

Interpretations

The findings discussed above are derived from various studies whose contexts are different from what Wah Yan is facing. The lessons we can draw from them must therefore be interpreted with care. Nevertheless, taken as a whole, these findings do strongly suggest that contributions to support Fr. Deignans proposals would be money well spent. Let us take stock of what we have learned from the studies.

Class Size

Both the Krueger and Rivlin-Hanushek-Kain papers have shown that reduction in class size can improve student performances. Although the Hoxby paper does not come up with the same conclusion, it does indicate that smaller classes can provide teachers with greater flexibilities. These new opportunities alone may not be sufficient. The issue is whether teachers have the incentives to actually make use of them to offer better instructions. Since class size reduction is known to be costly, it is fair that donors want to make sure that resources spent on it can have maximum impact. Incentives created by an effective accountability system, built upon fair firing and promotion practices, are important. Teacher development programs, such as mentoring or participation in exchange programs, are also useful.

Teacher Quality

Fr. Deignan is absolutely right in emphasizing the importance of having teachers who are both dedicated and capable. Teacher quality matters a lot. However, teacher quality is better revealed by the outcomes of teaching rather than by the teachers academic credentials. Relying on the latter is not enough. The school management has

10

the obligation to monitor the effectiveness of the teachers. Since new teachers can learn fast mainly in the first two or three career years, it is a good idea to institute proper mentoring for junior faculty. By the same reason, the probation period for new teachers should be real and serious. The school should not retain teachers who cannot prove themselves during this period. The chance of their becoming great teachers in the future is rather dim, according to the empirical findings. It goes without saying that accountability and proper incentives would go a long way to improve teacher quality.

Extracurricular Activities

Extracurricular activities can contribute a lot to the development of non-cognitive skills, which are highly valued in the market. They also do not hurt academic performance. Wah Yan has done the right thing in upholding its tradition of encouraging extracurricular activities. Since the benefit-to-cost ratio for investing resources to promote these activities is likely to be very high, it is safe to conclude that the school should spend more on these. Such investments do not only provide students with a more complete education, they also contribute to the social mobility of the socio-economically disadvantaged students.

(7)

Recommendations

The literature review in this report suggests that Fr. Deignans proposals to reduce class size, improve teacher quality and promote extracurricular activities make a lot of sense and should be supported. To ensure that money donated has a high degree of costeffectiveness, the school should set priorities. The findings discussed in this report indicate that maintaining the high quality of teachers is really the key issue. Reduction in class size will be much more effective if the teachers are at the same time given the incentives to work hard. They can also help to promote extracurricular activities. The following recommendations are made to improve teacher quality.

11

First, the principal of the school should regularly monitor the performances of the teachers by observing and analyzing student outcomes.

Second, the school should build up a rigorous accountability system. Good teachers should be rewarded, and the less qualified ones penalized. New teachers who fail to demonstrate significant improvements in the first two or three years should not be retained. Donors should consider establishing some funding to support a best teacher award, to be granted every year.

Third, encouraging and funding teachers to participate in teachers exchange programs, as suggested by Fr. Deignan, is a great idea.

Fourth, in recent years, numerous new opportunities for student activities have come into existence. For example, it is now common that universities in Hong Kong, China, or foreign countries offer innovative (and possibly subsidized) summer programs for which Wah Yan students are often eligible. Many students coming from families with educated parents are well informed about these activities. To make them accessible to most students of Wah Yan, the school may consider appointing a teacher who is responsible for collecting and updating the information about these programs and advising students about the opportunities. This practice helps students identify and make use of available outside resources.

Lastly, micro management on the part of donors may be counter-productive. Donors are usually amateurs in education. They should confine themselves to directing donated funds to those activities or programs that are truly worthy of support. The school administration needs a proper degree of autonomy to function efficiently.

12

References
Carneiro, Pedro, and James J. Heckman (2003), Human Capital Policy, Chapter 2 in James J. Heckman and Alan B. Krueger, Inequality in America: What Role for Human Capital Policies? (Cambridge, Massachusetts: MIT Press), 77-239.

Coleman, J. S., E. Q. Campbell, C. J. Hobson, J. McPartland, A. M., Mood, F. D. Weinfeld, and R. L. York (1966), Equality of Educational Opportunity (Washington, D.C.: US Government Printing Office).

Hanushek, Eric A. (1986), The Economics of Schooling: Production and Efficiency in Public Schools, Journal of Economic Literature, vol. 24, 1141-1177.

Hill, Reuben (1944), An Experimental Study of Social Adjustment: An Experimental Study of the Effects of Staff Stimulation to Participation in Extracurricular Activities on the Scholastic Achievement and Social Adjustment of College Students, American Sociological Review, vol. 9, #5, 481-494.

Hoxby, Caroline M. (2000), The Effects of Class Size on Student Achievement: New Evidence from Population Variation, Quarterly Journal of Economics, vol. 115, #4, 1239-1285.

Krueger, Alan B. (1999), Experimental Estimates of Education Production Function, Quarterly Journal of Economics, vol. 114, #2, 497-532.

Mehus, O. Myking (1932), Extracurricular Activities and Academic Achievement, Journal of Educational Sociology, vol. 6, #3, 143-149.

Rivlin, Steven G., Eric A. Hanushek, and John Kain (2005), Teachers Schools and Academic Achievement, Econometrica, vol. 73, #2, 417-458.

13

Watts, Michael, and William Bosshardt (1991), How Instructors Make a Difference: Panel Data Estimates from Principles of Economics Courses, Review of Economics and Statistics, vol. 73, #2, 336-340.

14