Sure Start and its Evaluation in England

Sure Start Programmes

In 1998 a UK government review concluded that disadvantage among young children was increasing and early intervention could alleviate poor outcomes. It recommended a change in service design and delivery, to be area-based, with all children under five and their families as clients. Among the aims were avoiding the stigmatization often associated with targeted programs while fostering child, family and community functioning. From 1999 the first Sure Start Local Programmes (SSLPs) focused on the 20% most deprived areas, including about half of children living below the official poverty line.¹ Sure Start has evolved over time and, while it has the same aims, it has become a more coherent program (children’s centres) with increasing emphasis on service integration.

By 2002, 250 SSLPs were planned, aiming to support 18% of poor children in England under five. A typical program included 800 under-fives. Community control was exercised through local partnership boards, including health, education, social services, private and voluntary sectors, and parents.² Until 2006 funding was directly to individual programs, which were independent of local government.   While the justification for SSLPs was based on early interventions with unambiguous protocols,^3-5 SSLPs themselves did not have a prescribed “protocol”. All were expected to provide: (1) outreach and home visiting; (2) support for families and parents; (3) support for good quality play, learning and childcare experiences for children; (4) primary and community health care and advice about child health and development and family health; and (5) support for people with special needs, but without specific guidance as to how.

The speed and amount of funding was often overwhelming in a sector previously starved of support. Only 6% of the 1999 allocation was spent in that year. Despite this slow start, and without any information on progress, the Treasury expanded SSLPs from 250 programmes in 2002 to over 500 by 2004. Thus SSLPs became a cornerstone of the campaign to reduce child poverty.  

Research Context and Research Results

National Evaluation of Sure Start 

Evaluation began in 2001, continuing until 2012, and was challenged from the outset by the diversity of several hundred unique interventions. Government decisions ruled out a randomized controlled trial; hence a quasi-experimental design with consequent limitations was used to compare SSLP populations with equivalent populations not residing in SSLP areas. The evaluation work up to 2005 was summarised⁶ with detailed reports of all aspects of the evaluation available at www.ness.bbk.ac.uk.^a An independent review of the methodology is available.⁷

Communities and Change: SSLPs had the premise that children and families could be affected by the program directly, and indirectly, via community changes. Community changes over 5 years could not be causally linked to SSLPs, but improvements were noted, significantly different to changes across England.⁸ For example, SSLP areas became home to more young children, while households dependent on benefits decreased markedly and burglary also declined. Child health improved with fewer emergency hospitalisations, severe injuries, and less respiratory infections. For older children, aspects of school functioning improved. Also, the identification of children with special educational needs or disability increased, suggesting improved health screening. Due to the political decisions on Sure Start policy, with a focus on children’s centres rather than communities, community change was not examined after 2006.

Early Effects on Children/Families: An initial cross-sectional study of children and families in SSLP and non-SSLP areas provided mixed findings.^9,10 There were some overall SSLP-related effects, but most effects varied by subgroup. Specifically, three-year-olds of non-teen mothers (86% of sample) in SSLP communities had fewer behaviour problems and greater social competence as compared with those in comparison communities, and these effects for children appeared to be mediated by SSLP effects of less negative parenting for non-teen mothers. Adverse effects emerged, however, for children of teen mothers (14% of sample) in SSLP areas in terms of lower verbal ability and social competence and higher behaviour problems. Also, children from workless households (40% of sample) and from lone-parent families (33% of sample) in SSLP areas scored lower on verbal ability than equivalent children in comparison communities.

Variability in programme effectiveness: The methodology provided estimates of each SSLP’s effectiveness for each assessed outcome and thus allowed investigation of why some programmes might have been more effective. Qualitative and quantitative data on 150 programs were used to rate each SSLP on 18 dimensions of implementation.^11,12 Programs rated high on one dimension tended to score high on others, and better implemented programmes appeared to yield greater benefits.^13,14 In particular, better service integration across agencies was one of the distinguishing features of more effective programs.

Changes to SSLPs: As early evaluation findings indicated that SSLPs were not having the hoped for impact, and evidence from another project, Effective Provision of Pre-school Education (EPPE),¹⁵ showed that integrated Children’s Centres were particularly beneficial to children’s development, the government decided to transform SSLPs into Children’s Centres. An Act of Parliament transferred control of the SSLP children’s centres to Local (government) Authorities, which ensured that they became embedded within the welfare state by statute, making it more difficult for any future government to eradicate. Thus from 2006 SSLPs became Sure Start Children’s Centres (SSCCs) with a more clearly specified set of services and guidelines, and were controlled by local government rather than central government.

Longitudinal Study of Children & Families: Children and families in SSLP areas were compared with those in similar non-SSLP areas followed from 9 months to 3, 5 and 7 years. At 3 years, beneficial effects emerged on 7 of 14 outcomes.^16,17 SSLP children showed better social development, exhibiting more positive social behaviour and greater independence/self-regulation,  partially a consequence of parents in SSLP areas manifesting less negative parenting, and offering a less chaotic and more cognitively stimulating home learning environment for their children. Also, families in SSLP areas used more services.  SSLP children had fewer accidents and were more likely to be immunised, but these latter two effects could possibly have been time of measurement effects and thus not related to SSLPs. 

At age 5, there were mixed effects of SSLPs/SSCCs.¹⁸ Mothers in SSLP areas reported greater life satisfaction, while providing less harsh discipline and a less chaotic and more cognitively stimulating home learning environment for their children. Additionally, their children were less likely to be overweight with better physical health. Mothers in SSLP areas, however, experienced more depressive symptoms and were less likely to attend school meetings. The benefits of SSLPs/SSCCs for child social development found at 3 years were not evident at 5 years of age. Thus, across 20 outcomes, significant main effects of SSLPs/SSCCs emerged for 8 outcomes. 

Considering change from age 3 to 5 years, 5 of 11 outcomes showed evidence of SSLP/SSCC effects. Mothers in SSLP areas manifested greater improvement in life satisfaction and home learning environment, with less harsh discipline. There was also a greater decrease in worklessness for families in SSLP/SSCC areas. Children in SSLP/SSCC areas, however, manifested less positive change in self-regulation than comparison children, which appeared to be because SSLP children manifested greater self-regulation at age 3 and by 5 years, the non-SSLP children had caught up with them. This catching-up by non-SSLP children could have been related to the free early education available for all 3-5 year olds in England from 2004. There was no evidence that the overall SSLP/SSCC effects varied across demographic sub-groups. 

At age 7, beneficial effects were identified for four out of 15 outcomes.¹⁹ For the whole study sample, mothers in SSLP areas used less harsh discipline and provided a more stimulating home learning environment. For sub-populations, mothers of boys in SSLP areas provided a less chaotic home environment; lone parents and those in workless households reported better life satisfaction. Looking at change from 3 to 7 years, mothers in SSLP areas showed greater improvement in the home learning environment and more reduction in harsh discipline. No statistically significant SSLP effects were identified for children.

Cost and Benefit

Value for money evidence is limited. Examination of spending revealed that over the first three years program development was limited. For example, finding suitable skilled staff and delays from dealing with local planning regulations for new buildings, meant that many SSLPs struggled to spend their allotted money.²⁰ Expenditure could be linked to variations in SSLP areas, including size, ethnic minority population, and other local area characteristics.  Small SSLPs appeared not to represent an economically viable model. By the end of the evaluation²¹ it was concluded that some measurable cost-benefits were shown, linked mainly with increased employment of parents in SSLP areas. Projections suggested that future gains might emerge, based on presumed benefits that may accrue from the enhanced parenting identified by the impact study.

Research Gap

Caution is needed in interpreting evaluation results, because of two methodological limitations. Firstly, government decisions to not allow a randomised controlled trial limit causal inferences about effects. Secondly, because data collections in the SSLP and non-SSLP areas had a two-year gap, time of measurement remains a viable alternative explanation for the positive and negative effects detected, although analyses tried to allow for this where possible.  

While SSLPs/SSCCs were associated with more positive parenting when children were 3, 5 and 7 years old, the positive effects on child social behaviour at 3 years disappeared by 5 years. This may have been because from 2004 all 3- and 4-year old children had access to free part-time pre-school education, and 97% took advantage of this.  Hence almost all children would have had pre-school education between 3 and 5. Evidence links high quality pre-school education with improved cognitive and social development.^22-24 Hence possibly developmental advantages associated with SSLPs at age 3 were not detected at ages 5 or 7 because, by this time, almost all children were exposed to pre-school education, which may have resulted in “catch up” for non-SSLP children.

Conclusions

The longitudinal findings differ markedly from earlier findings. Earlier the most disadvantaged 3-year-old children and their families (i.e., teen parents, lone parents, workless households) were doing less well in SSLP areas, while less severely disadvantaged children and families benefited (i.e., non-teen parents, dual parent families, working households). The longitudinal evidence at 3 years indicated benefits for all sections of the population. At age 5 the benefits were less but still exceed any disadvantages and they applied to all sections of the population. Nevertheless, why were there such differences in results between the early cross-sectional, and later longitudinal results? Although it is not possible to entirely eliminate methodological explanations, it seems possible that the contrasting results accurately reflect the contrasting experiences over time. Whereas the 3-year-olds in the cross-sectional study were exposed to ‘immature’ programmes—and probably not for their entire lives—children and families in the longitudinal study were exposed to better developed programmes throughout the children’s entire lives. 

Also programmes probably learned from the earlier phase of the evaluation, and made greater effort to reach the most vulnerable households. Thus differences in exposure to programs and the quality of SSLPs/SSCCs may account for both the initial adverse effects for the most disadvantaged and the subsequent more beneficial effects for almost all children and families in SSLP areas. In addition, change to Children’s Centres placed greater emphasis on multi-agency service integration, which was also a theme in government work linked to the ‘Every Child Matters’ agenda.^b However, it is of concern that long-term effects were limited to parent outcomes.  An in-depth investigation of child care quality in SSLP areas²⁵ found that it was variable but higher pre-school childcare quality was linked with higher child language development. Since there was evidence that children in SSLP areas with higher child care quality were showing greater language development by age 5, an important step would be to improve childcare quality in all locations.

Sure Start evolved and ongoing research partly influenced this process. Developments clarified guidelines and service delivery, with increasing emphasis on service integration and cohesion.  Plausibly the improved evaluation results reflected actual changes in program impact resulting from increasing quality and integration of services, greater attention to the hard to reach, the move to children’s centres, as well as greater exposure to services. The results are modest but suggest that the value of SSLPs/SSCCs has improved. The identification of the factors associated with more effective programmes has informed improvements in SSCCs.

While children’s centres proved popular with parents, after changes in government the number of centres is declining.  The Department for Education reported that more than 350 Sure Start Children’s Centres had closed in England since 2010 with spending in 2015-16 47% less than in 2010 with more cuts planned.²⁶ Nevertheless, some Local Authorities continued to support Children’s Centres, and where this occurred with implementation focussing on service integration, subsequent research has revealed very good outcomes for the academic achievement of children living in disadvantaged communities, thus fulfilling the original Sure Start goals.²⁷

References 

1. Melhuish, E, Hall D. The policy background to Sure Start. In: Belsky J, Barnes J, Melhuish E, eds. The National Evaluation of Sure Start: Does Area-based early intervention work? Bristol: Policy Press 2007:3-21.
2. Glass N. Sure Start: the development of an early intervention programme for young children in the United Kingdom. Children & Society 1999;13:257-64.
3. Ramey CT, Campbell FA, Burchinal M, et al. Persistent effects of early childhood education on high-risk children and their mothers. Applied Developmental Science 2000;4:2-14.
4. Reynolds AJ, Temple JA, Robertston DL, Mann EA. Long-Term Effects of an Early Childhood Intervention of Educational Achievement and Juvenile Arrest: a 15-Year Follow-Up of Low-Income Children in Public Schools. JAMA 2001;285:2339-46.
5. Schweinhart LJ, Barnes H, Weikhart, D. Significant benefits: the High/Scope Perry Pre-School Study through age 27. Ypsilanti, Michigan: High/Scope Press 1993.
6. Belsky J, Barnes J, Melhuish E, eds. The National Evaluation of Sure Start: Does area-based early intervention work? Bristol: Policy Press 2007.
7. Rutter, M.  Is Sure Start an effective intervention? Child and Adolescent Mental Health 2006:11:135-141.
8. Barnes J, Cheng H, Frost M, Harper G, Howden B, Lattin-Rawstrone R, Sack C; the NESS Team. Changes in the Characteristics of Sure Start Local Programme Areas in Rounds 1 to 4 between 2000/2001 and 2004/2005. London: DCSF; Sure Start Report Number 21: 2007. http://www.ness.bbk.ac.uk/local-context/documents/170.pdf. Accessed April 2, 2018.
9. NESS Research Team. Early Impacts of Sure Start Local Programmes on Children and Families. Surestart Report 13. London: DFES 2005.  www.ness.bbk.ac.uk/impact/documents/1183.pdf  (last accessed 2 April, 2018).
10. Belsky J, Melhuish E, Barnes J, Leyland AH, Romaniuk H. Effects of Sure Start Local Programmes on children and families: Early findings from a quasi-experimental, cross-sectional study. British Medical Journal 2006;332(7556):1476-1478. doi:10.1136/bmj.38853.451748.2F
11. Anning A, Ball M. Living with Sure Start: Human experience of an early intervention programme. In: Belsky J, Barnes J, & Melhuish E, eds. The National Evaluation of Sure Start: Does Area-based Early Intervention Work? Bristol: Policy Press 2007:97-112.
12. Anning A, Ball M. Improving Services for Young Children: from Sure Start to Children’s Centres. London, UK: Sage; 2007.
13. NESS Research Team. Variation in Sure Start Local Programmes Effectiveness: Early Preliminary Findings. Surestart Report 14. London, UK: DfES; 2005. http://www.ness.bbk.ac.uk/impact/documents/1184.pdf. Accessed April 2, 2018.
14. Melhuish E, Belsky J, Anning A, et al. Variation in community intervention programmes and consequences for children and families: The example of Sure Start local programmes. Journal of Child Psychology and Psychiatry 2007;48(6):543-551.
15. Sylva K, Melhuish E, Sammons P, Siraj-Blatchford I, Taggart B. Effective pre-school provision (EPPE). London, UK: Institute of Education; 2004.
16. NESS Research Team. The impact of Sure Start Local Programmes on three year olds and their families. Surestart Report 27. London, UK: DCSF; 2008. http://www.ness.bbk.ac.uk/impact/documents/41.pdf. Accessed April 2, 2018.
17. Melhuish E, Belsky J, Leyland A, Barnes A, the National Evaluation of Sure Start Research Team. A quasi-experimental study of effects of fully-established Sure Start local programmes on 3-year-old children and their families. Lancet 2008;372:1641-1647.
18. NESS Research Team. The impact of Sure Start Local Programmes on five year olds and their families. Research report DFE-RR067.  London, UK: DfE; 2010. http://www.ness.bbk.ac.uk/impact/documents/RR067.pdf. Accessed April 2, 2018.
19. NESS Research Team. The impact of Sure Start Local Programmes on seven year olds and their families. Research Report DFE-RR220. London, UK: DfE; 2012. http://www.ness.bbk.ac.uk/impact/documents/DFE-RR220.pdf. Accessed April 2, 2018. 
20. Meadows, P. The costs and benefits of Sure Start Local Programmes. In: Belsky J, Barnes J, Melhuish E, eds. The National Evaluation of Sure Start. Does area-based early intervention work? Bristol, UK: Policy Press; 2007:113-130.
21. Meadows, P and the NESS Research Team. National evaluation of Sure Start local programmes: An economic perspective. Research Report RR073. London, UK: DfE ; 2011. http://www.ness.bbk.ac.uk/cost-effectiveness/documents/DFE-RR073_full.pdf.  Accessed April 2, 2018.
22. Belsky J, Vandell D, Burchinal M, Clarke-Stewart KA, McCartney K, Owen M, The NICHD Early Child Care Research Network. Are there long-term effects of early child care? Child Development 2007;78(2):681-701.
23. Melhuish EC, Sylva K, Sammons P, Siraj-Blatchford I, Taggart B, Phan M, Malin A. The early years. Preschool influences on mathematics achievement. Science 2008;321(5893):1161-1162.
24. Sylva K, Melhuish E, Sammons P, Siraj-Blatchford I, Taggart B, eds. Early Childhood Matters: Evidence from the effective pre-school and primary education project. London, UK: Routledge; 2010.
25. Melhuish E, Belsky J, Macpherson K, Cullis A. The quality of group childcare settings used by 3-4 year old children in Sure Start local programme areas and the relationship with child outcomes. DFE-Research Report 068. London, UK: DfE; 2010. http://www.ness.bbk.ac.uk/impact/documents/RR068.pdf. Accessed April 2, 2018.
26. Walker P. More than 350 Sure Start children's centres have closed since 2010. The Guardian. February 2, 2017.
27. Barnes J. Inter-agency coordination of services for children and families – literature review and case studies of successful inter-agency working. ISOTIS (Inclusive Education and Social Support to Tackle Inequalities in Society) Report 6.2 to the European Union. 2018

Notes:

^a See also the BBK NESS Site. Available at: http://www.ness.bbk.ac.uk/. Accessed April 14, 2018.

^b See also full details of Every Child Matters at https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/272064/5860.pdf. Accessed April 14, 2018.