Tameside Government and Management Support
Activity Guide: Pupil performance (Understanding Attainment and Achievement)
Introduction
Being a “critical friend” i.e. supporting and challenging schools, is an important function of governing bodies, and this is particularly so in the context of pupil performance. In the revised Ofsted inspection framework, for example, it states Inspectors should take account of: “…the extent to which the governing body understands the school’s performance data and has an accurate picture of how well pupils are achieving compared with those in other schools, as well as how different groups of pupils within the school are performing.”
As governors we want all children in our schools to do well and successfully progress to the next level of learning. By doing so, they will enhance their future life chances/ opportunities, including job prospects, and play a more active role as a citizen.
In order to undertake this role effectively governors need to understand their school’s performance data and explore, via a series of questions, the story underlying the figures, particularly strengths and weaknesses. Areas for improvement can then be identified and appropriate strategies put in place to raise standards. This will form part of the governing body’s annual cycle of self-evaluation and school development planning.
Understanding statistics, however, can be quite daunting. They can also be used/ manipulated to illustrate a particular argument/ point of view; hence the phrase – “…lies, damned lies, and statistics!”. Hopefully the activity guide will help improve your confidence levels and make you more aware of some of the more common statistical terms and concepts. It should also help you appreciate both the uses and limitations of data and when other evidence may be more appropriate. Most importantly it will help you ask questions, seek explanations, question assumptions, and discuss strategies for improvement.
The activity guide is structured into four main sections:
- what level of attainment are children expected to make at certain points in their school life?
- understanding attainment and achievement, and the statistics commonly
used
- key questions to ask at meetings
- strategies for raising pupil performance
Appendices contain additional information on
- average points score
- provisional data from the 2010 National Curriculum Assessments at Key Stage 2 & 3 and GCSE examinations
- floor targets
Section 1: Expected pupil performance
Currently (September 2010) schools must assess pupils’ attainment at four key points in their compulsory schooling – when pupils have completed the Early Years Foundation Stage and the National Curriculum programmes of study for Key Stages 1, 2 and 3, usually at the ages of 5, 7, 11 and 14.
Early Years Foundation Stage
Children from birth to the end of the academic year in which they have their fifth birthday follow the Early Years Foundation Stage (EYFS).
This comprises the Early Learning Goals (ELGs), which describe the knowledge, skills and understanding which most (although not all) young children should be able to achieve by the end of the academic year in which they turn five. The ELGs cover six areas of learning/ development, all of which are equally important, and none of which can be delivered in isolation from the other. These include:
-
- personal, social and emotional development
- communication, language and literacy
- problem-solving, reasoning and numeracy
- physical development
- creative development
-
- knowledge and understanding of the world
Statutory assessment for the EYFS takes the form of the Early Years Foundation Stage Profile (EYFSP), which summarises each child’s achievement in the above six areas of learning. There is no testing; practitioners draw on their day-to-day observations of children to build up information throughout the final year of the EYFS. These are 13 assessment scales in total (some areas of learning are sub-divided; for example personal, social and emotional development is divided into three: disposition and attitudes, social development and emotional development) and each scale has nine assessment points, which means a child could achieve a maximum score of 117 points. A child achieving an average of 6 points across each of the 13 areas –78 points in total, would be seen as ready to access the National Curriculum in Year 1.
National curriculum and Key Stage assessments
From age 5 (year 1) to 16 (year 11) all pupils follow the national curriculum. Additionally, maintained schools must offer religious education, sex education, work-related learning and careers education for specified year groups. The national curriculum determines the content of what should be taught – statutory and non-statutory subjects/ areas of learning – and sets attainment targets for learning. The national curriculum also determines how performance will be assessed and reported.
But first a word about Key Stages. The national curriculum is divided into four key stages, which covers the statutory age ranges for school attendance.
| Key stage 1 | Key stage 2 | Key stage 3 | Key stage 4 | |
| Age | 5-7 | 7-11 | 11-14 | 14-16 |
| Year Group | 1-2 | 3-6 | 7-9 | 10-11 |
Assessment is an integral part of everyday teaching and learning. However, by law, schools must assess pupils’ attainment at the end of Key Stages 1, 2 and 3.
Key Stage 1 consists of teacher assessment in speaking and listening, reading and writing, and mathematics and science. Assessments in reading, writing (including handwriting and spelling) and mathematics must take account of results from Key Stage 1 tasks and tests.
Key Stage 2 consists of national curriculum tests in English (reading, writing (including handwriting and spelling) and mathematics and teacher assessment in English, mathematics and science.
Key Stage 3 consists of teacher assessment only, in all national curriculum subjects. Schools have to submit results for English, maths and science.
At the end of Key Stage 4, pupils generally take public examinations, for example GCSEs. Many pupils also undertake vocational courses, foundation learning and apprenticeships.
For each national curriculum subject, there is a programme of study which sets out the subject knowledge, skills and understanding that pupils are expected to develop in each Key Stage. The programmes of study map out attainment targets, which are split into eight levels (1 to 8), plus a description of “exceptional performance” i.e. above level 8.
For GCSEs the range is from A* -G. Other examinations often have pass, merit and distinction.
At the end of each national curriculum Key Stage, pupils are expected to reach a certain level, although many will (and should) exceed it.
-
- at the end of Key Stage 1 most pupils are expected to achieve level 2
- at the end of Key Stage 2 most pupils are expected to achieve level 4
- at the end of Key Stage 3 most pupils are expected to achieve levels 5 or 6
- at the end of Key Stage 4 most pupils are expected to achieve 5 GCSEs A*-C (including English and mathematics)
Sometimes national curriculum levels are expressed in terms of points scores, using the following formula: points score = 6 X level plus 3. Thus Level 4, for example, has a points score of 27 (i.e. 6 x 4 + 3 = 27). Other equivalences are shown below:
| Level | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |
| Points | 9 | 15 | 21 | 27 | 33 | 39 | 45 | 51 |
The National Curriculum standards have been designed so that most pupils will progress by approximately one level every two years, which is equivalent to 6 points.
As there are 6 terms in two years, each point represents one term’s progress for the median pupil.
Many schools have adapted this format by using national curriculum sub-levels with an equivalent points score. A common way of dividing the National Curriculum levels is the use of an a, b, c indicator – where
a – represents a strong level
b – represents a sound/ secure level
c – represents a weak level
| Level | Sub-levels | ||
| c | b | a | |
| 1 | 7 | 9 | 11 |
| 2 | 13 | 15 | 17 |
| 3 | 19 | 21 | 23 |
| 4 | 25 | 27 | 29 |
| 5 | 31 | 33 | 35 |
| 6 | 37 | 39 | 41 |
| 7 | 43 | 45 | 47 |
| 8 | 49 | 51 | 53 |
Whilst the use of sub-level can be helpful in identifying pupil support requirements and tracking progress, caution should be used as the National Curriculum level was designed to indicate attainment at the end of a key stage – a sublevel only gives an indication of the certainty of this achievement.
Summary
| Age | Year group | Stage/ assessment point | Expected level of attainment |
| Birth to five | Reception | Early Years Foundation Stage | 78 points |
| 5-7 | 1-2 | Key Stage 1 | Level 2 |
| 7-11 | 3-6 | Key Stage 2 | Level 4 |
| 11-14 | 7-9 | Key Stage 3 | Level 5/6 |
| 14-16 | 10-11 | Key stage 4 | 5 GCSE Grades A*- C including English and maths |
Section 2: Understanding attainment and achievement, and the statistics commonly used
Analysing pupil performance from the previous summer national tests and examinations is a key item on autumn term agendas. Governors need to understand their school’s performance data and what questions to ask at meetings.
Key sources of data for governors
The schools’ performance data is contained in RAISEonline. This stands for Reporting and Analysis for Improvement through School Self-Evaluation. This is an online resource and schools can provide access for governors to download the full report. However, this is not published until all the data has been validated, usually in the spring term of the following academic year. Nonetheless schools have details of the 2010 national tests and examinations and headteachers should use this information to report the results to governors.
Schools also have access to Fischer Family Trust (FFT) data/ reports which is particularly helpful for target setting and self-evaluation.
The School Improvement Partner’s annual report to governors will also contain a preliminary analysis of the 2010 results and identify areas of concern.
The Head teacher’s termly report to governors should always have a section on standards/ progress. This can be supplemented by reports from subject and key stage leaders.
In terms of questions to ask governors need to know two essential things:
Attainment: the standard of the pupils’ work shown by test and examination results. In other words how many pupils have reached the expected level/grade at each Key Stage or end of year assessment?
Achievement: the progress and success of a pupil in their learning, development or training i.e. distance travelled between two points in time – this can be the start/ end of a term/ academic year or between key stages etc.
Remember – It’s quite possible for a school to have good attainment results i.e. at or above the threshold, but lower scores for achievement. In other words, are overall results at the end of a Key Stage concealing poor progress? Conversely, although a school’s attainment results may be average/ below average it’s quite possible that pupil progress can be good or even outstanding. Ofsted make a judgement on both attainment and pupils’ learning and progress, and remember… pupils’ learning and progress is a limiting grade i.e. it will impact on the overall effectiveness grade.
In order to measure attainment and achievement, a number of statistics have been developed. Apart from percentages (%), these include average points score (APS), value added (VA) and contextual valued added (CVA).
Average points score (APS) is based on the formula referred to above, which links national curriculum levels to points scores, and is used to measure a number of outcomes . These include the performance of individual pupils (or different groups of pupils), different subjects, or a key stage. You can also use APS to work out the average national curriculum level for each subject/ overall at each key stage (see Appendix for worked examples).
Value added (VA)
School A might show high percentages of pupils achieving Level 4 and above, while school B shows lower percentages. However this may be a reflection, in part, of the school’s catchment area rather than the school’s effectiveness. Similarly, pupils at school B may have made more progress than other pupils who were performing at the same level at KS1 for example, and therefore have a higher value added “score” than school A. Therefore, value added measures are designed to allow a fairer comparison between schools with different pupil intakes/starting points, and to give schools greater recognition for the work they do.
Value added is used to measure the progress made by an individual compared with the average pupils make by similar pupils nationally between two points in time, typically key stage assessments. In other words, it’s a measure of relative rather than absolute performance.
The KS1 to KS2 value added measure
Each pupil’s value added (VA) score is based on comparing their KS2 performance with the median – or middle – performance of other pupils with the same or similar results at KS1. The individual scores are averaged for the school to give a score that is represented as a number based on 100. This indicates the value the school has added on average for their pupils.
Scores above 100 represent schools where pupils on average made more progress than similar pupils nationally, while scores below 100 represent schools where pupils made less progress (see note on statistical significance).
The KS2 to KS4 value added measure
Each pupil’s VA score is based on a comparison between their best eight results at GCSE – sometimes referred to as their capped points score – and the median or middle performance of other pupils with the same or similar results at KS2. The individual pupil scores are added together and averaged to produce the school level VA measure. This number is presented as a number based around 1000.
Measures above 1000 represent schools where pupils on average made more progress than similar pupils nationally, while measures below 1000 represent schools where pupils made less progress (see note on statistical significance).
| NB. The accuracy that can be attached to any schools VA measure depends, amongst other things, on the number of pupils in the value added calculation. The smaller the number of pupils, the less confidence can be placed on the VA measure as an indicator of whether the effectiveness of a school is significantly above or below average – see note on statistical significance below. |
Contextual Value Added (CVA)
VA measures take account of prior attainment, which is the biggest single factor affecting pupil results. However, contextual factors which are outside the school’s control, such as gender, mobility and levels of deprivation may have a further impact on pupil results, even after allowing for prior attainment.
In order to introduce more of a level playing field when comparing schools, and take account of these additional factors, a more complex measure known as CVA has been developed. Like VA, this provides an indicator of relative rather than absolute performance and attempts to isolate the “school effect” i.e. whether the school, with the pupils it has, is doing better than, worse than, or broadly the same as other schools, with the pupils they have.
The additional factors which CVA take into account include:
- prior attainment in English and maths
- ethnicity
- gender
- age within age group (month of birth)
- special educational needs
- eligibility for free school meals
- first language other than English
- deprivation based on pupil postcode (using IDACI – that is the Income Deprivation Affecting Children Index)
- looked after children
- geographic mobility
Although CVA calculations use a more complex model than “simple” VA, the basic principle remains the same. Quite simply this involves comparing the peformance of an individual pupil with the performance of children with similar prior attainment and similar circumstances.
An individual’s CVA score will be the difference (positive or negative) between their actual performance and that predicted taking into account the national data for all the factors in the model. These differences are then collated to provide a CVA score for the school. As with “simple” VA, this is based around a medium score of 100 for primary schools and 1000 for secondary schools.
A school’s CVA of more than 100/ 1000 means that, overall, the school has performed better that most schools with a similar mix of students and factors. Under 100/1000 means the performance of this group of students is below average.
Whilst CVA can provide powerful insights into the impact which schools have, the figures need interpreting with caution (see note on statistical significance).
NB: Contextual Value Added scores should not be used to set lower expectations for any pupil/ groups of pupils. DfE advise that schools should, when setting targets for future performance expectations, strive to set equally challenging aspirations for all pupils.
Statistical significance
When using statistics the term “statistical significance” is often used. In everyday language “significant” means important or meaningful; in statistics it’s the likelihood that a finding, result or relationship is caused by something other than just chance.
In other words, what statistical significance tries to show is how confident we can be that a result is reliable or true.
In RAISEonline most reports show attainment or progress scores for your school relative to the national average/ mean. Significance tests have been performed on data using a 95% confidence interval, and where the school value differs significantly from the corresponding national value, sig+ or sig- boxes are used: the sig+ boxes are usually coloured greenwhilst the sig- boxes are coloured blue. Where a school figure is significantly above that of the previous year an up (↑) or down (↓) arrow is displayed to the right of the figure.
Significance tests are heavily influenced by numbers of pupils in a cohort, so large schools are more likely to see Sig+ or Sig- boxes than small schools, even when differences to national averages are the same.
Section 3: Key questions to ask at meetings
All governors need to understand the context of their school and how it compares to other schools. RAISEonline provides this information. A useful summary will also be found in the school’s Self Evaluation Form (SEF).
Q. How well have pupils performed in relation to national standards at the relevant key stage?
Remember, at KS1 pupils are expected to attain Level 2 (or 15 points) whilst at KS2 it’s Level 4 (or 27 points). At KS4 the target is 5 A*-C GCSEs (or equivalent), including English and maths.
Q. How much progress have pupils made between key stages?
Remember the expected level of progress between KS1 and KS2 is two national curriculum levels or 12 points. This is equivalent to 1 point per term.
Progress between KS2 and KS3 is 1.5 levels or 9 points, and from KS2 to KS4 is 2.5/ 3 levels or 15/18 points.
Subsidiary questions might include:
- are children making better or worse than expected progress?
Q. Are there any significant attainment gaps between different groups/ cohorts of pupils?
- gender
- ethnicity
- free school meals
- special educational needs
- able, gifted and talented
- looked after children
- other groups unique to your school
Subsidiary questions might include:
- which are the strongest and weakest performing groups of pupils?
- are all pupils fulfilling their potential or should they achieve more?
- if there are attainment gaps, why?
- is the attainment gap narrowing?
- is there a culture of high expectations in the school?
- are individual pupils’ targets sufficiently stretching and challenging?
- do any pupils lose momentum between point a & b?
- are the most disadvantaged pupils, or those with SEN, making sufficiently rapid progress to catch up with their peers?
Q. Are there any differences between subjects, courses and parallel year groups?
Q. How does your school compare with Tameside/ families within the Greater Manchester Challenge/ national results?
Q. How do actual results compare with the school’s statutory targets? (How achievable/ challenging are the targets)
Q. What are the trends over the last three to five years? Are school standards rising, staying the same or falling?
Q. Are any of the above statistically significant?
Q. Is there a member of staff who has responsibility for assessment/ monitoring results? Who do they report to? How does this feed into the school improvement plan?
Q. How are results communicated to parents/ carers?
Q. How does the school work with parents (particularly of the most vulnerable groups) to help them support their child’s learning?
Section 4: Identifying priorities and actions
Data raises lot of whys; governors also need explanations! What are the reasons underlying the results/ how is the school going to tackle underperformance/ what are the school’s priorities? Some questions you might ask include:
- appropriateness of the curriculum/ curriculum resources?
- quality of teaching and learning/ homework?
- quality of feedback to pupils?
- the deployment of support staff?
- what is the attendance/ absence rate like, both generally and for different cohorts? What are the characteristics of persistent absentees? What is the link between attendance and attainment?
- any issues around behaviour?
- teacher absence?
- assessment for learning / use of pupil tracking data (how is it monitored?)
- what intervention strategies are in use, both generally and for specific cohorts of pupils? How are impacts monitored/ evaluated?
- parental involvement?
- professional development for staff?
Monitoring of priorities/ actions
Once priorities for improvement have been identified, it’s important that governors have in place mechanisms to monitor and evaluate outcomes, particularly their impact (remember, all outcomes must have success criteria).
Autumn term agendas contain an item on test/ examination results which should be fully discussed/ analysed at either full governors or the appropriate committee. Any recommendation for action should then feed into the school development plan and be monitored at subsequent meetings.
It’s also important that pupil progress is reported/ monitored at either at full governors or the appropriate committee. If poor progress is an issue, this needs an explanation and addressing.
Appendix
Average Points Score (APS)
Pupils – a pupil achieving a Level 4 in English and maths would have an APS of 27 (27+27/2=27). If they achieved a Level 5 in English and Level 4 in maths the APS would be 30 (33+27/2=30). A level 5 in both English and maths would result in an APS of 33 (33+33/2=33).
Subjects – the APS for a subject is calculated by finding the total of every pupil’s points scores for that subject and dividing by the number of pupils who were assessed in that subject. For example, what would be the APS for English if three pupils achieved a Level 3, sixteen a Level 4, and eight a Level 5? Again, use the formula referred to above – thus three pupils with Level 3 = 63 (3×21= 63), sixteen at Level 4 = 432 (16×27=432) and eight at Level 5 = 264 (8×33=264), giving a total of 759 points (63+432+264=759). Now divide the total number of points by number of pupils to get the APS: thus 759/27=28.1
Key stage – the overall APS at a Key Stage for a school is found by adding together the points for each subject for each pupil and dividing by the number of pupils who were assessed in subject 1 plus the number who were assessed in subject 2 etc. This enables the points to be included for pupils who were assessed in some, but not all subjects. Thus if the APS for subject 1 = 27.8 and the APS for subject 2 = 28.2 the overall APS would be 28 (27.8+28.1 /2= 28).
Statistical information for 2010
DfE: National Curriculum Assessments at Key Stage 2 & 3 in England, 2009/10 (Provisional)
Key Stage 2
The percentages of pupils achieving Level 4 or above in the 2010 Key Stage 2 tests by subject are as follows:
The percentages of pupils achieving Level 5 in the 2010 Key Stage 2 tests by subject are as follows:
The percentage of pupils achieving level 4 or above in the 2010 Key Stage 2 teacher assessments by subject are as follows:
The percentage of pupils achieving level 5 or above in the 2010 Key Stage 2 teacher assessments by subject are as follows:
The percentage of pupils achieving level 4 or above in combinations of subjects in the 2010 Key Stage 2 tests
The percentage of pupils achieving level 5 or above in combinations of subjects in the 2010 Key Stage 2 tests
Key Stage 3
The percentage of pupils achieving level 5 or above in the 2010 Key Stage 3 teacher assessments by subject are as follows:
The percentage of pupils achieving level 6 or above in the 2010 Key Stage 3 teacher assessments by subject are as follows:
The percentage of pupils achieving level 7 or above in the 2010 Key Stage 3 teacher assessments by subject are as follows:
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
GCSE results 2010
A* - 7.5 per cent of grades awarded
A - 15.1 per cent of grades awarded
B - 20.6 per cent of grades awarded
C - 25.9 per cent of grades awarded
D - 15.9 per cent of grades awarded
E - 7.8 per cent of grades awarded
F - 4 per cent of grades awarded
G - 1.9 per cent of grades awarded
U - 1.3 per cent of grades awarded
Floor targets (minimum expected levels of performance)
Key Stage 2: at least 55% of pupils are expected to achieve L4 or above in English and maths
Key stage 4: at least 30% of pupils are expected to achieve at least 5A*-C GCSE passes including English and maths.
