education policy analysis archives

: Reducing the socio-economic achievement gap is a key goal of education policy internationally. Since 2017, the stated intention in Ireland is to use the Pobal HP Deprivation Index to identify schools serving high proportions of disadvantaged students. In spite of this, little published research has compared the performance of the HP index to richer survey measures from large-scale educational assessments or examined its association with educational achievement at primary or second level. This paper aims to assess the validity and fitness for purpose of the HP index for use in identifying schools serving high concentrations of socio-economically disadvantaged students. Analyses draw on Ireland’s Program for International Student Assessment (PISA) 2018 dataset matched with administrative data. Findings show a strong correlation between school-average HP and other socio-economic measures examined. However, about one-in-five schools are identified as potential ‘false positives’ or ‘false negatives’ when school -average HP is used for identification purposes. Also, school-average HP explains less variance in reading achievement than other variables considered. Conclusions recognize the benefits of the


Validating School-based Measures of Educational Disadvantage in Ireland
Achieving greater equity in education is a common goal across education systems where the aim is to reduce the association between student socio-economic background and educational outcomes (Organization for Economic Co-operation and Development [OECD], 2020a). Approaches to combating educational disadvantage vary across countries but often involve providing additional resources to schools to support students from socio-economically disadvantaged backgrounds. There is variation internationally in how disadvantaged students are identified but approaches typically involve either the use of individual measures, such as student eligibility for free school meals (e.g., Taylor, 2018), area-level measures, based on the postcode of a student's home address (e.g., Jerrim, 2021;OECD, 2021), or a mix of the two (e.g., Higher Education Access Route, 2022). Some consideration has been given in the international literature to the validity and comparability of these measures (e.g., Domina et al., 2018;Gorard, 2012) although the issue has received comparatively less attention in Ireland. It is timely to address this gap as a new system (based on census data) for the identification of schools serving high concentrations of students from disadvantaged backgrounds was introduced in Ireland in 2017 and finalized in 2022 (Department of Education and Skills [DES], 2017; Department of Education [DoE], 2022a). It is therefore important to examine how schools identified as disadvantaged using one possible application of census data compare to schools identified using alternative measures.
In this paper, we compare one possible application of the census-based measure, based on small area-level data, with two measures based on individual student data (aggregated to schoollevel) in order to assess the validity and fitness for purpose of the area-based variable for use in identifying schools serving high concentrations of students from educationally disadvantaged backgrounds. Specifically, we contribute to an assessment of the validity and fitness for purpose of the Pobal HP deprivation index (henceforth HP index; Haase & Pratschke, 2017) for use in identifying schools serving high concentrations of students from educationally disadvantaged backgrounds by examining how the school-average on this variable is associated with two other socio-economic indicators used in Irish educational research and/or policy. The first of theseeconomic, social and cultural status (ESCS; OECD, 2020b) -is from the Program for International Student Assessment (PISA; OECD, 2019). The second refers to the percentage of students in a school entitled to an examination fee-waiver. This variable has previously been used in Ireland to identify second-level schools for additional funding to support socio-economically disadvantaged students (Weir, 2006).
The paper examines the following research questions: firstly, how is school-average HP related to two other measures of school socio-economic status (SES); i.e., school-average ESCS and percent fee-waiver? Secondly, to what extent does a hypothetical identification system based on school-average HP identify the same schools as an approach using school-average ESCS or percent fee-waiver? Thirdly, how is school-average HP related to school-average reading achievement and is this comparable to the association between school-average reading achievement and other socioeconomic measures? Fourthly, using multilevel modelling, how much variation in student reading achievement is explained by the available socio-economic indicators? And finally, can the amount of explained variation in reading achievement be improved by adding publicly available school structural characteristics to the model?
The paper contributes to the methodological literature on the identification of students from disadvantaged backgrounds by outlining one possible use of area-level data. In Ireland, data on individual student socio-economic indicators are not routinely gathered in education databases at a population level; hence it is necessary to use an alternative such as census data. International readers are likely to be interested in Ireland's approach as the country has been recognized for having a strong focus on equity in education (European Commission, 2019;Hepworth et al., 2021).
The next section of the paper briefly outlines how Bourdieu's (1986Bourdieu's ( , 2005 theoretical contributions have been used to explain the existence of socio-economic differences in educational outcomes. Then, the measurement of SES in PISA is outlined. Thirdly, some approaches are described, which are used internationally to identify students from low socio-economic backgrounds and schools serving high concentrations of such students. Fourthly, we provide a brief overview of primary and second-level education in Ireland. Next, we outline initiatives designed to combat educational disadvantage in Ireland and the procedures used to select schools to participate in the most recent initiative. Methods and results of the current study are then described. Finally, we discuss the implications of our findings.

Understanding Socio-economic Differences in Educational Achievement
With limited exception across educational systems, assessments and outcomes, students from higher socio-economic backgrounds have higher levels of achievement than their counterparts from less advantaged backgrounds (e.g., Bai et al., 2021;Broer et al., 2019;Chmielewski, 2019). For example, across countries/economies participating in PISA 2018, 'advantaged' students scored on average 89 points (almost one standard deviation) higher in reading achievement than 'disadvantaged' students, where 'advantaged' students were those in the top quartile nationally on an indicator of SES (OECD, 2020a).
Various theorists have considered how the education system supports the reproduction of social inequalities, with a key contribution to the field made by Pierre Bourdieu (1986;Bourdieu & Passerson, 1990). Bourdieu used the concepts of capital, habitus, practice, and social fields, to explain the reproduction of cultural and social inequalities through education.
According to Bourdieu (1986), elementary forms of capital are social, cultural, economic and symbolic. These may be combined in various ratios to give field-specific capital, such as scientific or educational capital (Bourdieu, 2005). As middle-class students are purported to be better equipped with the linguistic and cultural capital needed for success in the school environment, they are better placed to succeed in school than their less advantaged peers (Bourdieu & Passeron, 1990). Bourdieu and Wacquant (1992, p. 124) describe habitus as "the product of a particular economic condition, defined by the possession of the minimum economic and cultural capital necessary actually to perceive and seize the 'potential opportunities' formally offered to all". They argued that for middle-class students, there is better alignment between habitus and the requirements of the educational system.
Comparisons have been made between Bourdieu's habitus and Bernstein's (1964) theory of code, which describes speech systems as comprising two codes: the restricted code and the elaborated code, with the latter being characteristic of middle-class students and necessary for educational success (see Harker & May [1993] for a comparison of the work of Bourdieu and Bernstein). In the Irish context, the work of Bourdieu and that of Bernstein have provided a theoretical basis for applied work related to educational disadvantage. For example, Kellaghan (2001) uses Bourdieu's concepts to provide a definition of educational disadvantage that expands on that used in Irish legislation, suggesting that the official definition exhibits a number of inadequacies including a failure to recognize the importance of cultural capital. Skerritt (2017) draws on Bernstein's work in order to examine achievement levels of working-class students, outlining discontinuities between the language of working-class children and the language of the school.

Measuring SES in PISA
The OECD's PISA assesses the ability of 15-year-olds to use their knowledge and skills in reading, mathematics and science (OECD, 2019). In addition, PISA gathers detailed contextual data from students and school principals as well as information on students' skills in collaborative problem solving and global competence. In some participating countries/economies, 1 parents also complete a questionnaire. The study first took place in 2000. The most recent cycle took place in 2018 involving 600,000 students in 79 participating countries (OECD, 2019(OECD, , 2020a. The influence of the OECD on Irish education policy has been noted (McNamara et al., 2021) and PISA itself has provided the basis for literacy and numeracy targets in Ireland (DES, 2016).
Of central relevance to the current paper is the PISA ESCS index, a composite index based on three indices: highest parental occupation, parental education and home possessions, including the number of books at home (OECD, 2020b). The index of home possessions is a summary index of household and possession items, based on student reports of the availability of 16 household items (including three country-specific items) at their home. It is used as a proxy for family wealth as no direct income measure is available from PISA data (OECD, 2020b). Data from the student questionnaire is used in the computation of ESCS, which has been shown to explain a moderate proportion of variance in student achievement. In PISA 2018, 12% of variation in students' reading performance was accounted for by student ESCS on average across the OECD (OECD, 2020a). Avvisati (2020) has reviewed the measurement of SES in PISA and notes that developers of the index suggested that ESCS is closely related to the 'gradient approach,' which emphasizes the relative status of individuals in society. Given PISA's desire to enable comparisons across individuals in different countries/economies, Avvisati considers the PISA measurement of SES to also be influenced by a materialist view of SES. This view places a focus on the attainment of goods and services (American Psychological Association, 2007). Given the influence of both materialist and gradient perspectives, Avvisati proposes a definition of ESCS as "a measure of students' access to family resources (financial capital, social capital, cultural capital and human capital), which determine the social position of the student's family/household" (Avvisati, 2020).
Some concerns about ESCS have been raised in the literature. For example, Rutkowski and Rutkowski (2013) query the extent to which a single measure of socio-economic background is reliable and valid across all countries/economies participating in PISA. Focusing on the home possessions index (a component of ESCS), Rutkowski and Rutkowski identified variation in reliability across countries, poor model-to-data consistency on a number of subscales, and evidence of poor cultural comparability. Also, inconsistencies have been observed in some countries participating in PISA between student and parent reports on family background items, including parental education and occupation (Schulz, 2005). Avvisati (2020) highlights that ESCS may give a 'noisy vision' of the school profile if the number of participating students is small and/or it may give a biased profile if PISA students are atypical (e.g., if they are ahead or behind the expected track for their age). It has also been suggested that the individual components of ESCS are more useful as descriptors than the composite index (see Avvisati, 2020 for details). Despite these concerns, it is recognized that as a composite measure, ESCS has a practical utility in analysis and it has been noted that some improvements to ESCS were implemented in PISA 2018 (Avvisati, 2020).

Identification of Low SES Students/schools: Selected International Examples
While PISA's study design permits a detailed and multi-faceted measure of SES from a sample of schools and students, policy makers have the challenge of identifying disadvantaged students in the population in the absence of detailed information in order to deliver targeted supports in the most effective manner. Indicators used internationally include parents' education, parents' occupation, ethnicity/race, home language, family structure, family income and free schoolmeal (FSM) eligibility (see e.g., OECD, 2021). In many instances, the choice of indicator is dictated by the availability of data for all students, which in turn is driven by operational feasibility. Often these are income proxies most closely related to economic capital using Bourdieu's conceptualization discussed above. Yet, it has been noted that poverty is a complex phenomenon that is not captured by any single measure or indicator (Maître et al., 2021).
FSM eligibility is a commonly used SES proxy in research and policy in many countries, including the USA, Sweden, Finland, Japan, India and the UK (Taylor, 2018). In England and Wales, this information is recorded in the National Pupil Database (NPD) as part of the regular school census. FSM eligibility has been used in the allocation of additional funding to schools via the Pupil Premium (in England) and Pupil Deprivation Grant (in Wales). These supports aim to close the achievement gap between those eligible for FSM and those who are not eligible 2 (Taylor, 2018). Taylor explored the reliability of FSM eligibility and noted that there was a small but significant number of false negatives, i.e., children who were living in poverty but not eligible for FSM. However, the measure is generally successful in identifying children living in socio-economically disadvantaged households. Children are flagged as 'eligible' for FSM in the NPD only if they are both eligible for, and claiming, FSM (Jerrim, 2021). It has been reported that some families may not apply for the scheme due to an associated stigma. As a binary measure, eligibility for FSM has been shown to be a good indicator for the purposes of allocating resources and supports, albeit one that does not fully address the complex and nuanced relationship between SES and educational outcomes (Taylor, 2018; see also Ilie et al., 2017). Gorard (2012) highlights the issue of missing data on FSM, arguing that at least some students missing FSM data represent a deprived, and perhaps extremely deprived, group.
Jerrim (2021) outlines a range of different proxy measures for SES in the context of admissions to university and job recruitment in the UK. He highlights the strength of FSM as an indicator, remarking that it is the best indicator for childhood poverty. However, he notes that data on FSM are typically unavailable to UK universities, necessitating their use of alternatives measures. One such measure is POLAR (participation of local areas), which calculates, in a given area, how likely young people are to participate in higher education (see e.g., Higher Education Funding Council for England, 2005). Jerrim argues that this measure is flawed as it produces a high level of false-positives and negatives; is poorly correlated with income deprivation; and is biased against key demographic groups, including Black, Asian and Minority Ethnic (BAME) groups. Jerrim discusses two other area level measures used in the UK that have moderate to good relationships with low household income: Acorn and the Index of Multiple Deprivation (IMD). Acorn is a geodemographic classification system that combines data from various sources including the Land Registry, administration data and commercial data to assign pupils to one of 62 Acorn types (CACI, 2022). It is currently used by a small number of UK universities. Low income correlates moderately with Acorn (r = .56) and the information is at a very localized level. However, Acorn is not free to use, limiting its widespread use.
IMD is England's official measure of relative deprivation and is freely available. It has a moderate relationship with low income (r = .47) (Jerrim, 2021). Two supplementary indices are available: the Income Deprivation Affecting Children Index and the Income Deprivation Affecting Older People Index (see Noble et al., 2019). Similar to POLAR, Jerrim notes that IMD is biased against those who are BAME, live in single-parent households, or who rent their accommodation. That is, IMD was shown to underestimate the probability that BAME children, those living in London, those living in rented accommodation, single parent families and/or those children with young mothers are in the lowest income group. IMD is only available in England, therefore cannot be used to compare across UK countries. In spite of these limitations, IMD is used in education and other policy spheres in England (Jerrim, 2021).
In Scotland, Laselle and Johnson (2021) propose a marker to identify schools facing higher levels of deprivation than the Scottish average. They combine a location indicator with four relativedeprivation indicators. Two are linked to the Scottish Index of Multiple Deprivation (SIMD), the third refers to the proportion of students in the school eligible for FSM, and the fourth relates to the average progression rate from the school to higher education. They suggest that compared to SIMD alone, their composite indicator better accounts for levels of disadvantage in rural areas.
In the USA, free and reduced-price lunch (FRPL) status is also used as a measure of socioeconomic disadvantage. Federal and many state educational finance systems identify schools for targeted resources based on the proportion of students in the school qualifying for FRPL (Domina et al., 2018). Students whose household income is less than 130% of the poverty line qualify for a free lunch; students with household income between 130% and 185% of the poverty line qualify for a reduced-price lunch (Domina et al., 2018). When compared to Internal Revenue Service (IRS) data on household income, FRPL data have been shown to predict academic achievement more effectively than IRS reported income data, after controlling for household income. FRPL data are based on self-reported household income, which may account for it explaining additional elements of socio-economic disadvantage (Domina et al., 2018).
A recent paper from the OECD (2021) illustrates the use of funding formulas in the French Community of Belgium and in the Netherlands. In both of these regions, additional resources are allocated to schools to support students at risk of educational disadvantage. In the French Community of Belgium, a socio-economic index value is assigned to each student on the basis of their residential area. The index is based on characteristics such as income, qualification levels and unemployment rate (OECD, 2021). In the Netherlands, a student's risk of educational disadvantage is determined on the basis of several variables pertaining to the individual student, including parental educational attainment, mother's origin, and whether or not parents are in debt restructuring. Schools receive additional funding for students at risk of educational disadvantage (OECD, 2021). Changes in the funding formula used in the Netherlands appear to address at least some of the criticisms levelled at the earlier weighted student funding scheme, which by its conclusion was based only on parental education (Driessen, 2017).

Brief Overview of Primary and Second-level Education in Ireland
The Minister for Education has overall responsibility for education in Ireland. The ministerial department, currently called the Department of Education (previously the Department of Education and Skills [2010 to 2020] or the Department of Education and Science [1997 to 2010]), is supported by a number of aegis bodies. In Ireland, children are eligible for two years of free early childhood care and education (ECCE) between the ages of 2 years, 8 months and 5 years, 6 months, or until entry to primary school (DES et al., 2020). A child must be at least 4 years old at the start of the school year (September) to enroll in primary school and must have started formal education by the age of 6 years. There are approximately 3,100 primary schools, with small schools a distinctive feature of the Irish education system, particularly in rural areas (see DES et al., 2020;DoE, 2020a).
After eight years in primary school (two preprimary years followed by Grades 1 to 6), students in Ireland move to a second-level school, almost all of which are state-funded. There are about 730 second-level schools (DoE, 2020a). These can be categorized into three broad groups: voluntary secondary schools, schools (or community colleges) in the Education and Training Board (ETB) sector, and community or comprehensive schools. Voluntary secondary schools are owned by religious groups or organizations or by their trustees and are often single-sex schools. Education and Training Boards own and run ETB schools (previously known as vocational schools) and in some cases, community colleges. These are usually co-educational. Community or comprehensive schools are established by the state and managed by boards of partners and trustees. All school types offer a similar education, which consists of a three-year junior cycle followed by a two-or three-year senior cycle. For further detail, see Coolahan (1995) andDES et al. (2020).
Recent years have seen considerable reform in the first three years of second-level education, moving to the new Junior Cycle program (DES, 2015). This has involved changes to assessment and reporting, a reduced focus on externally assessed examinations and an increased emphasis on classroom-based assessment. The Junior Cycle also features newly developed subjects and short courses. Changes were introduced on a phased basis, with English the first subject to have a new specification introduced. Junior Cycle assessment was impacted by Covid-19, resulting in some adjustments and cancellation of examinations (DoE, 2021).
Following completion of the Junior Cycle, students can complete an optional Transition Year, which involves a more varied educational experience. It is intended to promote personal, social, vocational and educational development, and often includes work placements. In their final two years of second-level education, students follow one of three programs, each leading to a terminal state examination -the Leaving Certificate Established, the Leaving Certificate Vocational Program or the Leaving Certificate Applied. 3 A majority of students follow the Leaving Certificate Established Program (Banks et al., 2017). Review of the senior cycle is ongoing (DES et al., 2020).

Identifying and Addressing Educational Disadvantage in Ireland
In Ireland, educational disadvantage is defined as "the impediments to education arising from social or economic disadvantage which prevent students from deriving appropriate benefit from education in schools" (Ireland 1998, Section 32). The country has a long history of supporting disadvantaged students, dating back to the establishment in 1969 of a collaborative project between the (then) Department of Education and the Bernard van Leer Foundation, which founded a 3 The Leaving Certificate Established aims to provide learners with a broad, balanced education while also offering some specialization towards a particular career option (National Council for Curriculum and Assessment, 2022a). The Leaving Certificate Vocational Programme requires students to take a minimum of five Leaving Certificate subjects (including Irish). In addition, students are required to study two Link modules -Preparation for the World of Work and Enterprise Education. Students are also required to study a modern European language (National Council for Curriculum and Assessment, 2022b). This programme is selected by about 30 percent of students (Banks et al., 2017). The Leaving Certificate Applied is a prevocational program with three main elements: vocational preparation, vocational education and general education. It is undertaken by about 5 percent of school leavers and is not intended for students who aim to directly access third-level education (Banks et al., 2017). preschool for children aged between 3 and 5 in a disadvantaged inner-city area of Dublin (Holland, 1979). The provision of preschool places was also the focus of the Early Start initiative, which was established in eight schools in disadvantaged areas in 1994/1995 and subsequently extended to a further 32 schools (Educational Research Centre, 1998). Also at preschool level, specific funding was provided for Traveler children to attend preschool. For a more in-depth review of initiatives outlined here, see .
The Disadvantaged Areas Scheme (DAS) was introduced in 1984 to support selected primary schools in three Irish cities (Dublin, Cork and Limerick) and provided increased capitation grants for participating schools and a grant for the development of home-school links. DAS was introduced to second-level schools in 1990/1991 and provided for concessionary teaching posts and enhanced capitation grants. Findings from a review of relevant provision, and DAS in particular, informed the development of a later scheme, Breaking the Cycle, which was designed to support primary schools in addressing problems associated with catering for large numbers of pupils from disadvantaged backgrounds .
Breaking the Cycle was introduced as a pilot scheme in 1996/1997 to 33 urban schools and 123 rural schools; all schools in Breaking the Cycle also participated in the Home School Community Liaison (HSCL) scheme, designed to develop positive links between the school and the home (Ryan, 1994). The HSCL scheme was established as a pilot project in 1990 and has since been rolled out more widely to urban primary schools and all second-level schools participating in the current scheme to address educational disadvantage -Delivering Equality of Opportunity in Education (DEIS). 4 The Support Teacher Project (initially called the Teacher Counsellor scheme) was another support at primary level, aimed at supporting schools to manage the behavior of disruptive students. It involved the appointment in September 1996 of 27 'teacher counsellors' and a version of this initiative remains in place in about 40 schools (see DoE, 2020b).
The DEIS plan 2017 (DES, 2017) replaced an earlier DEIS plan, launched in 2005. As with the 2005 plan, the most recent DEIS plan allows for a distinction at primary level between disadvantaged schools in urban areas and those in rural areas. The most disadvantaged urban primary schools are classified as DEIS Urban Band 1. Supports for these schools include reduced class sizes, additional grant aid, access to school meals, access to the HSCL program, and priority access to teacher professional development. Urban primary schools, experiencing levels of disadvantage less extreme than those in Band 1, are classified as Urban Band 2. A key difference between the supports provided to Urban Band 1 and Urban Band 2 schools relates to the more favorable designated staffing schedules in Band 1 schools. Supports for DEIS Rural schools include a DEIS grant, access to the School Meals Program, access to planning and professional development supports and additional funding under the School Books Grant Scheme (DES, 2017).
At second level, schools participating in DEIS receive additional grant aid as well as other supports including an enhanced allocation of guidance counsellors, access to HSCL, and access to specific programs for students such as the Junior Certificate Schools Program and the Leaving Certificate Applied Program (DES, 2017).
Consideration has been given to the extent to which the DEIS program has achieved its aims to date, with some evidence of positive changes in school organization and outcomes for students. However, the lack of a control group in DEIS evaluations limits the conclusions that can be drawn and it has been noted that the multifaceted nature of the program makes it difficult to disentangle which aspects may work best (Smyth et al., 2015). Some have argued that resources provided to DEIS schools are inadequate (Fleming & Harford, 2021). However, the European Commission (2019, p. 15) suggests that initiatives, including DEIS, "have made Irish secondary schools positive forces for inclusion".
Two separate approaches were used for the identification of schools at primary and second level when DEIS was launched in 2005. At primary level, school principals were surveyed and asked to estimate the percentages of students in their schools with various characteristics related to family SES. These included the percentages of students from homes with unemployed parents, singleparent families, of Traveler ethnicity, from a large family ( or more siblings), entitled to a free-books grant, or living in local authority accommodation (Archer & Sofroniou, 2008). From these variables, an index was developed that was used to rank order schools according to their level of disadvantage. Separate rankings were produced for urban and rural settings (Archer & Sofroniou, 2008). In early 2006, schools with the highest levels of disadvantage were invited by the (then) Department of Education and Science to participate in DEIS. Schools admitted to DEIS remain in the program (except in the event of closure).
At second level, centralized data were used in the original DEIS identification process. In contrast to the procedure at primary level, survey-based data were not used to identify second-level schools for DEIS. A school's level of disadvantage was derived from student retention rates, state examination achievement, the percentage of students entitled to an examination fee-waiver, and the percentage of students who dropped out of school. As at primary level, schools were rank ordered and those with the highest levels of disadvantage were invited to participate in DEIS (Weir, 2006). All data used to identify second-level schools were available in databases held centrally by the DES.
An advantage of the identification approaches used for DEIS at both primary and second level was that both of these used a combination of measures in the computation of an index of disadvantage, meaning that the system avoided relying exclusively on a single proxy income-based measure. However, a recognized shortcoming at primary level was the reliance on survey data provided by school principals. Current policy aims to move away from using survey data to standardized population-based data (DES, 2017, Goal 1). Also, one of the measures previously used at second level -the percentage of students with an examination fee-waiver -is tied to changes to the medical card system (Houses of the Oireachtas, 2017). More detail is provided on this measure in the Method section of this paper as the variable is used in the current analysis.
A key aspect of the 2017 DEIS plan was the introduction of a new approach for the identification of schools eligible to receive additional supports; however, the precise details of the revised identification approach were not finalized until 2022 (DoE, 2022a). The new approach draws on the Pobal HP Deprivation Index (Haase & Pratschke, 2017), derived from census data. The HP index identifies three dimensions of affluence/disadvantage: demographic profile, social class composition and labor market situation (Haase & Pratschke, 2017). Each dimension is measured by a number of indicators; some indicators are associated with more than one dimension. The indicators are: age dependency rate; population change over the previous five years; percentage of the population with primary education only; percentage of the population with Third-level education; percentage of households headed by professionals, managerial or technical employees, including farmers with 100 acres or more; mean number of persons per room; percentage of households with children aged under 15 years headed by a single parent; percentage of households headed by semi-skilled or unskilled manual workers, including farmers with less than 30 acres; male unemployment rate; and female unemployment rate.
Constructed using Confirmatory Factor Analysis, the HP index provides an absolute index score (set in 2006 to have a mean of zero and standard deviation of 10 with varying means and standard deviations since then) and a relative index score (set in each census wave to have a mean of zero and standard deviation of 10). The HP index is designed to provide a robust measure of relative affluence and deprivation across both urban and rural areas (Haase & Pratschke, 2017). Deprivation indices in other countries have been criticized for having a lack of sensitivity to rural disadvantage (Fecht et al., 2018) or for being less meaningful for some age groups or minority groups (Fu et al., 2015).
The HP index is based on analysis of Small Area data. A small area has a minimum of 50 households and a mean of just under 100. There were 18,488 small areas defined for the 2011 census (Haase & Pratschke, 2017).
The index has been widely used in policy in Ireland, including in local development, health, transport, the Residential Property Price Index and in the calculation of local property tax (Haase, 2022). For example, the HP index is one predictor of property values in the Residential Property Price Index. Also, a combination of the HP index, an urban-rural classification and the percentage of households residing in local authority rented accommodation are used in the Resource Allocation Model of the Local and Regional Drug and Alcohol Task Forces. In education, the index has been used as one of six criteria applied to assess whether or not a student is eligible to benefit from a college and university admissions scheme designed to support socio-economically disadvantaged school leavers. The authors of the index suggest that some of the most important applications of the index have occurred in the health domain. One example shows how the index has been used to support the identification of disadvantaged areas as part of a process of distributing health resources in the community (Haase, 2022).
Previous research on higher education students (tertiary level) in Ireland has shown substantial differences in mean HP scores between students attending different types of institution, by program type, and by mode of study (full-time or part-time). HP scores have also been shown to be associated with graduate earnings, after controlling for other characteristics (Higher Education Authority, 2019). The current authors are not aware of previously published analyses of the association between HP scores and achievement at primary or second level.
The updated DEIS plan recognizes the improved availability of centrally-held data for primary and second-level schools. This allows the HP index to be used in DEIS identification at both levels (DES, 2017). Advantages of the revised approach include consistency between primary and second levels and a population-based measure that avoids the administrative burden of survey data collection. The removal of achievement outcomes from the identification process addresses the potentially problematic feedback loop between the measurement of educational disadvantage and the successful use of resources and supports to improve educational outcomes.
The revised DEIS identification approach for schools involves matching, for all pupils in the primary and post-primary online databases, their home address data to their appropriate small area HP deprivation score. Specifically, each address is geocoded to small area level and then each geocoded address is assigned a HP small area score derived from the HP index (see DoE, 2022a). HP scores for all matched records are then aggregated to the level of the school. Based on the HP deprivation scores of pupils in the school, a profile is obtained of the level of socio-economic disadvantage in the school. For the purposes of identifying schools for DEIS, the key target group of pupils are those with HP scores at or below -10 (based on the deprivation index having a national mean of zero and standard deviation of 10) and schools with the highest percentages of pupils with scores at or below -10 are those identified for additional supports. In addition, the identification model gives some consideration to pupils with HP scores between -7.5 and -10 who are deemed to be at risk of educational disadvantage (DoE, 2022a).
In the 2021/2022 school year, there were 227 Urban Band 1, 104 Urban Band 2 and 356 Rural DEIS schools at primary level. There were 198 second-level schools in DEIS. Following a period of consultation with the education partners, the refined DEIS identification model was applied in March 2022 (DoE, 2022a), which built on the version introduced in 2017. The latest application of the model results in an additional 284 primary schools and 38 second-level schools that were brought into DEIS from September 2022 (with 39 existing DEIS primary schools reclassified to a different band, i.e., from Urban Band 2 to Band 1 or from DEIS rural to Urban Band 2; DoE [2022b]).
This paper provides a comparative analysis of school-average HP, the percentage of students with an examination fee waiver and PISA ESCS, using the PISA 2018 national dataset for Ireland as the base dataset. In doing so, we aim to demonstrate the extent to which the HP index is suited to the identification of educationally disadvantaged schools relative to these two other available measures. While the validity of FSM as an indicator has been examined extensively in the literature (e.g., Gorard, 2012;Ilie et al., 2017), the authors are not aware of a comparable analysis of the HP index.

Method Data
PISA 2018 data for Ireland were used as the base dataset for this analysis. School-average HP and the percentage of students with an examination fee-waiver were matched to the PISA dataset on the basis of a school identifier held by the PISA national study center.
In PISA 2018, reading was the major domain with 5,577 students in 157 schools participating in Ireland. Ireland's mean reading score was 518.1, significantly higher than the OECD average (487.1). Students in Ireland ranked 4th out of 36 OECD countries. The standard deviation for reading literacy in Ireland was 90.7, compared to an OECD average standard deviation of 99.4 (McKeown et al., 2019).
The PISA population is defined as students who are enrolled in educational programs aged between 15 years and 3 months, and 16 years and 2 months. About three-fifths of participating students in Ireland were in Grade 9 (Third year), one-quarter were in Grade 10 (Transition year) and most others were in Grade 11 (Fifth year). Students in Grades 7 or 8 comprised less than 2% of the sample (McKeown et al., 2019). As students participating in PISA 2018 were drawn from a range of grade levels, some had studied (or were studying) subjects based on Junior Certificate syllabi published prior to 2012 while others were studying subjects as part of new Junior Cycle specifications (McKeown et al., 2019); this variation is not expected to be directly relevant to the current analysis. It is more directly relevant to note that ESCS data pertain to PISA participants only (not all students in the school).
For the current analyses, school-average reading achievement was computed in the International Database Analyzer (International Association for the Evaluation of Educational Achievement, 2021) using all plausible values for reading and grouping output by country and school identifiers. Interested readers should see the PISA 2018 technical report (OECD, 2020b) for a detailed description of the PISA 2018 database, including procedures for computing weights, scaling data, and computing cognitive test plausible values.
Of students participating in PISA 2018 in Ireland, 5,519 (98.9%) had non-missing data for ESCS; data for these students are used in the computation of school-average ESCS and in the multilevel modeling of reading achievement in the current paper. School-average ESCS was computed by aggregating to school level from the weighted student data file. The (unweighted) numbers of students per school ranged between 11 and 44.
Data on the percentages of students in each school with an examination fee-waiver were provided by the State Examinations Commission for the purposes of PISA administration and are used with permission for the current analysis. Data pertain to 2015 and are available for 152 out of 157 schools participating in PISA 2018. As the new Junior Cycle Profile of Achievement was first used in 2017 (see McKeown et al., 2019, for a discussion in the context of PISA participants), the term Junior Certificate (rather than Junior Cycle) examination fee-waiver is used throughout this paper given that data pertain to 2015.
The variable examination fee-waiver, used in the original DEIS identification model and in the current analysis, has been widely used as a proxy for low family income in national studies on education (e.g., Sofroniou et al., 2004;Weir & Kavanagh, 2018) where it was shown that the percentage of students in the school with an examination fee-waiver correlated strongly and negatively with average school achievement (Weir & Kavanagh, 2018). Prior to 2020, 5 students were entitled to a Junior Certificate/Cycle examination fee-waiver if their parent or guardian had a medical card. This is awarded on the basis of low family income (Health Services Executive, 2022).
School-average HP score was provided by the Statistics section of the Department of Education and is used with permission in the current analysis. School-average HP was computed on the basis of students in each school on September 30, 2018, with address data available.
A key difference between school-average ESCS, percent fee-waiver and school-average HP is that school-average ESCS is based only on the cohort of students who completed the PISA reading test, i.e., students in the school aged about 15. Percent fee-waiver relates to the cohort of students in the school who applied for Junior Certificate examinations in 2015. The lack of data on ESCS or examination fee-waiver for students across all grades is one limitation of these measures. In contrast, school-average HP represents students across grade levels in the school, i.e., not just students participating in PISA or State examinations. Although there is unlikely to be substantial variation in the socio-economic profile of students across year groups and over time, some fluctuation may occur, particularly in smaller schools. For the purposes of exploring associations with reading achievement, a strength of ESCS is that data underpinning the measure were gathered at the same time as the test of reading achievement.
Additional explanatory variables examined in the current analysis are: school enrollment size, school location (urban or rural, i.e. population of less than 1,500 persons), and school sector/gender composition (boys' secondary, girls' secondary, mixed secondary, community/comprehensive school; ETB schools).
Although the revised DEIS identification process uses the percentage of students in the school with HP scores in various categories (DoE, 2022a), the current analyses use school-average HP as this more closely mirrors the school-average ESCS measure and has the benefit of being approximately normally distributed. It is also relevant to note that while the revised DEIS identification process gives additional weight to students from a Traveler or Roma family, to students residing in International Protection Accommodation Services centers, and to students experiencing homelessness (DoE, 2022a), no such weighting was applied to HP scores used in the current analysis which was completed prior to finalization of the revised DEIS identification process. Ethical approval was not required for this study, and all work was carried out in accordance with institutional guidelines and requirements.

Analysis
Unweighted and weighted (by school weight) bivariate correlations between school-level variables were conducted in IBM SPSS Statistics (version 26). Application of school-level weights resulted in limited change to the outcome. Therefore, results of unweighted correlations are presented in the results section as the intention is to present findings of a broad exploratory analysis with each school contributing equally to the overall statistics.
Multilevel modeling was conducted in MPlus, which is designed to account for complex sampling (Muthén & Muthén, 2017) and allows for the inclusion of both student-level ESCS and school-average ESCS as explanatory variables. MPlus has the advantage of easily facilitating analysis of all plausible values. Data files for MPlus were prepared in SPSS; a student weight was computed by extracting the school element from the final student weight (Karakolidis et al., 2022). Dummy variables were created where necessary for categorical data and a uniform approach was applied for missing values.
The null model was created to check data preparation and to calculate the intra-class correlation. Fitted models were random intercepts models, the first of which has predictors at both level 1 (student ESCS) and level 2 (school-average ESCS). Subsequent models have level 2 variables only. Statistical significance was determined by examining either the t-value associated with the variable or using a chi-square difference test based on log-likelihood values. With the exception of school enrollment size, continuous variables are grand-mean centered and z-standardized to facilitate interpretation. Additional explanatory variables included in the final model were selected on the basis of being publicly available for the population of second-level schools.
As a final step, non-linearity of relationships was assessed through testing the significance of squared terms associated with continuous explanatory variables and interactions were examined. Non-significant terms were not retained.

Associations Between Socio-economic Indicators
The first research question asks how school-average HP is related to two other measures of school SES. Of interest are the bivariate associations between three indicators -school-average HP score; the percentage of students with an examination fee-waiver; and school-average ESCS. The relationship between school-average HP and school-average ESCS is shown in Figure 1, with 95% confidence intervals shown in dotted lines. For the association between school-average HP and school-average ESCS, R-square quadratic is .54 (R-square linear is .53). Figure 2 shows the association between school-average HP and the percentages of students with an examination fee-waiver. Here, R-square cubic is .66 (compared to .64 for R-square linear).
While Figures 1 and 2 show the association between school-average HP and the two other indicators, of interest also is the strong negative correlation between school-average ESCS and the percentage of student with an examination fee-waiver (r = -.82, p < .01). This correlation is of similar magnitude to the correlations between HP mean and the other variables (HP mean and school-average ESCS: r = .73, p < .01; HP mean and the percentage of students with an examination fee-waiver: r = -.80, p < .01).

Figure 2
A Scatterplot of the Association between School-average HP Score and the Percentage of School Enrollment with an Examination Fee-waiver

Identification of Disadvantaged Schools
The second research question asks to what extent a hypothetical identification system based on quintiles of school-average HP identifies the same schools as disadvantaged as a hypothetical approach using quintiles of school-average ESCS or percent fee-waiver. Table 1 shows the crosstabulation of quintiles of school-average ESCS by quintiles based on school-average HP. For both variables, quintile 1 represents those schools with the most highly disadvantaged cohort. Cells in white in Table 1 show where there is a match between the two approaches. A 'minor mismatch' is illustrated with light grey shading; a 'major mismatch' is shown in dark grey. A 'minor' mismatch indicates a difference of one quintile between two approaches; a 'major' mismatch means a difference of two or more quintiles between the two approaches.
Almost half of schools received the same classification using these two approaches (Table 1). About one-third of schools move one position between the two approaches. For almost one-fifth of schools (19%), there is a difference of two or more quintiles between their classification using the two approaches. A total of 16 schools are identified in the top right of Table 1. These are schools where the socio-economic profile given by average ESCS is higher than that given by school-average HP. These are potential 'false positives'; i.e., the quintile based on school-average HP indicates that the school profile is more disadvantaged than would be anticipated by quintile based on schoolaverage ESCS.
For 14 schools (bottom left of Table 1), school-average HP suggests that the school SES context is higher than that given by school-average ESCS. These are potential 'false negatives'; i.e., the quintile based on school-average HP indicates that the school profile is more advantaged than would be anticipated by quintile based on school-average ESCS. Given the small numbers in the 'major' mismatch group, no further analysis is conducted of the characteristics of these schools. A similar comparison was conducted for quintiles based on the percentages of students with an examination fee-waiver 6 cross-tabulated with quintiles based on school-average HP (Appendix Table A1). Findings show that half of schools have matching quintiles; two-fifths are identified as 'minor' mismatches in classification; and almost one-tenth are identified as 'major' mismatches. Thus the percentage of 'major' mismatches is lower when school-average HP quintiles are cross-tabulated with quintiles based on examination fee-waiver (9% 'major' mismatch, Appendix Table A1) than when school-average HP quintiles are cross-tabulated with school-average ESCS quintiles (19% ,  Table 1).
Comparing quintiles based on the percentage of students with an examination fee-waiver with those based on school-average ESCS shows that 51% of schools are allocated to the same quintile under both approaches, 36% move one position ('minor' mismatch), and 13% differ by two or more quintiles ('major' mismatch).

Associations with Reading Achievement
The third research question considers the association between school-average reading achievement and school-average HP and asks how this association compares to that between school-average reading achievement and other socio-economic indicators. There is a strong positive correlation between school-average reading achievement and school-average HP score (r = .62, p < .01) although this correlation is somewhat weaker than the correlation between school-average reading achievement and school-average ESCS (r = .83, p < .01). There is a strong negative correlation (r = -.79, p < .01) between school-average reading achievement and the percentage of students in the school with an examination fee-waiver.
The fourth research question considers the variation in student reading achievement that can be explained by the available socio-economic indicators. Findings from the null model show that the intraclass correlation is .16. The first multilevel model uses student-level ESCS and school-average ESCS as explanatory variables (Table 2). Findings show that a one standard deviation increase in student ESCS is associated with an almost 20-point increase in reading achievement. A one standard deviation increase in school-average ESCS is associated with an almost 22-point increase in student reading achievement, having controlled for student-level ESCS. A one-standard deviation increase in either student-level ESCS or school-level ESCS is associated with nearly one-quarter of a standard deviation increase in reading achievement (based on Ireland's standard deviation of 90.7 in reading literacy). Model 1 explains just under 5% of within-school variance, about 76% of between-school variance and 16.8% of total variance (Table 3). For the purposes of explaining variation in reading achievement, Model 1 demonstrates the benefit of having data available at both the individual student level and the school level. Models 2, 3 and 4 include predictors at school-level only.
Model 2 shows that in the absence of student-level variables, a one-standard deviation increase in school-average ESCS is associated with a 31-point increase in student reading achievement (Table 2). This model explains 77% of between-school variance and 13% of the total variance (Table 3). The percentage of between-school variance explained by Model 2 is very similar to the percentage explained by Model 1; however, the percentage of total variance explained is somewhat lower (a consequence of the very limited amount of within-school variance explained by Model 2).
Findings from Model 3 show that all else being equal, a one-standard deviation increase in the percentage of students with an examination fee-waiver is associated with a 29-point decrease in student reading achievement (Table 2). This model explains a somewhat lower percentage of between-school and total variance than the earlier models, explaining 69% of between-school variance and 11.7% of total variance (Table 3).
Findings from Model 4 show that a one-standard deviation increase in school-average HP score is associated with a 25-point increase in average reading achievement, all else being equal (Table 2). This model explains 45% of between-school variance and 7.6% of the total variance (Table 3). Of the four models examined, Model 4 explains the lowest percentages of total variance and between-school variance. Finally, publicly available school variables (school sector and gender composition; enrollment size; and school location) were examined to investigate if the explanatory power of Model 4 might be improved in the absence of information on school-average ESCS or percent fee waiver; i.e., to mirror the type of data available to the DEIS identification process. Statistically significant effects were found for the variables examined (HP mean, enrollment size, school location, and sector/gender; Appendix Table A2). There is a statistically significant positive association between school enrollment size and student reading achievement, all else being equal. Having controlled for other variables in the model, students in urban schools score an average of 21 points lower in reading than their rural counterparts. There are also statistically significant differences between school types (based on sector and gender). Interaction terms and curvilinear relationships were not statistically significant. Model 5 explains 64% of between-school variance, less than 1% of within-school variance and 11% of the total variance. This shows that in the absence of detailed socio-economic data at the individual level, it may be helpful to draw on readily available school structural characteristics to explain variation in reading achievement.

Discussion
A challenge for policy makers is how to identify students at risk of educational disadvantage without detailed individual measures on relevant indicators. Income-related measures such as FSM are widely used internationally and considerable attention has been given in the literature to comparisons between FSM and other indicators (e.g., Gorard, 2012;Ilie et al., 2017). The stated intention of the updated DEIS plan in Ireland (DES, 2017, Goal 1) is to move to an identification system for schools serving high concentrations of disadvantaged students that draws on small area deprivation scores for the areas in which students live. However, the measure has not been subject to the same level of scrutiny as FSM and less is known about how it compares to other survey-based socio-economic indicators or educational outcomes at primary or second level.
Previously in Ireland, at second level, a more direct measure of family income was available for DEIS identification purposes through student eligibility for examination fee-waiver. While detailed questionnaire data from students and parents in PISA provides an in-depth picture of individual-level SES, it is not practical or cost-effective to gather data at this level of detail for the population. The current analyses use the detailed information from PISA to examine the performance of the available population-level socio-economic indicators. In doing so, we acknowledge the recognized limitations of ESCS identified in the literature.
In findings that will likely be reassuring for policymakers, there are strong correlations between school-average ESCS, school-average HP and percentage of students with an examination fee-waiver. These analyses demonstrate the potential to use small area data from the small area in which a student lives as a proxy indicator of the student's SES. When aggregated to school-level, these data are shown to provide a reasonable approximation of the school socio-economic context as compared to that indicated by school-average ESCS, which is based on much more detailed student data.
Comparing quintiles based on school-average HP and school-average ESCS, about half of schools are assigned to the same quintile using the two approaches and a further third of schools are within one quintile. For about one in five schools examined, there is a difference of two or more quintiles between the two approaches suggesting that for a minority of schools, school-average HP may a less accurate representation of school socio-economic profile. 7 An alternative interpretation is that school-average ESCS may give a less accurate picture in these schools. Further analysis of these alternatives was not undertaken for the present study, given the limited numbers of schools in the dataset. Future development and validation of the DEIS identification system could usefully give consideration to schools that practitioners indicate may have been misclassified using the HP index. Also, there may be merit in using additional datasets to compare classifications based on the HP index with other socio-economic indicators.
Fewer SES quintile mismatches were identified when quintiles based on school-average HP were compared with quintiles based on percent fee-waiver. While these mismatches were not examined further in the present study, future research could usefully examine data for the population of schools to consider if particular school structural characteristics are associated with identification as false positive or negative. While data are not available at the population level for school-average ESCS, data are much more widely available for both school-average HP and percent fee-waiver.
ESCS has been shown to be related to student achievement levels (OECD, 2020a). National analyses have confirmed a strong association between educational outcomes and the percentage of students with an examination fee-waiver (Sofroniou et al., 2004;Weir & Kavanagh, 2018). While the association between HP mean and educational outcomes has been examined at higher education level (Higher Education Authority, 2019), published analyses are not available for primary or second level. Current findings show a strong correlation between school-average reading achievement and school-average HP score although this correlation is somewhat weaker than that between schoolaverage reading achievement and the percentage of students with an examination fee-waiver. Future analyses could usefully explore the association between school-average HP and achievement in other domains such as PISA mathematics or science.
Multilevel modelling findings show that school-average HP score explains 45% of betweenschool variance in PISA reading achievement. This compares to 77% of between-school variance in reading achievement, which is explained by school-average ESCS and 69% by percent fee-waiver. A model including additional publicly available school-level variables in addition to school-average HP explains 64% of variance in reading achievement, a considerable improvement over the model with average HP only.
A key focus of DEIS is to reduce the achievement gap between students from higher and lower socio-economic backgrounds. Although school-average HP has limitations in the percentage of variance it explains in reading achievement (noting that relationships between PISA mathematics and science, along with other possible achievement measured, have not been explored in this paper), the index has several advantages to policymakers. It is updated following each census; it is universally available for all students; and it does not place any additional data collection burden on school principals. The current analysis illustrates how the explanatory power of school-average HP can be improved by including additional school-level variables. Findings from this paper show that despite some limitations, the HP index represents a reasonable option for use in the DEIS identification process. Earlier international research has highlighted the impact of missing data when using FSM as an indicator of disadvantage with pupils missing FSM falling into two main groups (Gorard, 2012). This underscores the need for technical documentation on the DEIS identification process to provide detail on the level of missing data on the HP index. Considerable detail has been provided in the most recent technical documentation (DoE, 2022a) and this is to be welcomed.
Given that school-average HP is less effective than other socio-economic indicators at explaining variance in the PISA measure of reading achievement, future work could usefully examine the associations between school-average HP and other educational measures such as student engagement, wellbeing or absenteeism. Indeed, there is a need to critically examine the very premise on which validation work rests: Much of this work has traditionally been based on examining associations with various school-level measures and test scores. Equally important are other outcomes such as engagement, wellbeing and progression to further education and work, so a multifaceted approach to validating the HP index is recommended for future work.