Methods of data collection
Explore the methods of data collection used in Peter Blatchford’s extensive research in psychology and education
Across all the research projects complementary forms of data collection were developed, including, in CSPAR and DISS, national questionnaires from teachers, headteachers and TAs, and case studies involving semi-structured observations and interviews with staff and pupils. But the main form of data collection used in all the research was systematic observations which categorised and recorded moment-by-moment behaviours and interactions in classrooms. Observation categories were developed to code how children behaved in three ‘social modes’: when with their teachers, when with other children, and when not interacting. Subcategories within each of these three modes covered work, procedural, social and off-task activities. The observation systems used a 10-second time-sampling technique.
The systematic observation methods captured a first- person point-in-time, moment-by-moment description. This allowed a bottom up picture across multiple individual first person moments in time – over 100,000 observation points – and an analysis of the classroom environment in terms of contextual, relational and interconnected dimensions,
In the CSPAR study pupils in their reception year (4-5 years) and when in Year 6 (10–11 years) in large vs small classes were observed (see Blatchford, Moriarty, Edmonds & Martin, 2002, Blatchford, 2003a and b). In Y6 there were 257 observed children in total, and 22,312 observations in total, with an average of 87 observations per child.
In the DISS study, systematic observations were carried out over 2005/06 in 27 primary schools and 22 secondary schools. Observations were conducted in Y1 and Y3 (5–6 and 7–8 years) and Y7 and Year 10 (11–12 and 14–15 years). A total 0f 686 pupils were observed. There were 67,928 observations (data points) in total, collected over 1,132 hours of observation. While the CSPAR study classified class sizes into two groups, large and small, a distinctive feature of the DISS study was that for each 10-second time interval the observers noted class size at that time. More details can be found in Blatchford, Bassett & Brown (2011).
In the MAST study, observations were made on 48 pupils with SEND in Year 5, and in the second phase of data collection (the SENSE study), observations were made of 49 pupils with SEND in Year 9 (see Blatchford and Webster, 2018, for full details). Observations were also collected on a sample of control or typically developing pupils, 151 in Year 5 and 112 in Y9.
Drawing together observation data from both the MAST and SENSE studies resulted in a data set totalling 67,928 observations in all.
For the SPRinG on-the-spot SO analysis, systematic on-the-spot (OTS) observations of pupil behaviour and interactions were conducted in each experimental and control class at the beginning and end of the school year (i.e. in the first and third term of the school year). The OTS method captured behaviour across the school day (and therefore not just in group work), and so showed whether involvement in group work transferred to ‘normal’ classroom activities. There were 12,961 observations in total, 7,023 for the experimental sample and 59,38 for the control sample, with an average of 41.15 observations per child.
These four systematic observation studies were complemented by, in brief:
1.DISS observation analysis of TA and teacher to pupil talk.
The systematic observation methods, described so far, use relatively broad categories of behaviour. In the DISS study, the talk between TAs and pupils, and between teachers and pupils, was described in more fine-grained detail using an ‘instructional talk analysis’ based on research from the effective teaching literature, especially Berliner’s model of effective teaching (1987). Results were for both primary and secondary and there were a large number of utterances coded: for teachers there were 5,226 and for TAs 2,295.
refs
2.DISS Conversation Analysis (CA) of TA and teacher to pupil talk.
A second approach – ‘ Conversation Analysis’ – was an even more closely textured approach Instead of coding single utterances it offers a sequential analysis and takes account of pupils’ talk. In order to further standardise the contexts within which talk takes place, it concentrated on mathematics lessons and on three aspects: 1) turn allocation; 2) topic organisation; and 3) how repairs are accomplished.
A full description of methods and results can be found in Radford, Blatchford and Webster (2011) and Blatchford, Russell and Webster (2012).
Key reference: Radford, J., Blatchford, P. and Webster, R. (2011) Opening up and closing down: how teachers and TAs manage turn taking, topic and repair in mathematics lessons, Learning and Instruction, 21, 625-635
3.SPRinG analysis of video recordings of groupwork
In the SPRinG KS2 study video recordings were analysed to provide a fine-grained description of pupil talk and involvement in groups of 4 pupils working on a researcher-designed group-work activity within their normal classroom context. The video analysis aimed to test whether SPRinG trained groups differed from groups who were not involved in SPRinG training (the ‘control’ sample) when involved in the same group-work task. The tasks were specially designed non-curricular tasks. A total of 31 SPRinG and 29 control groups were filmed.
Key references:
Blatchford, P., Baines, E., Rubie-Davies, C., Bassett, P., and Chowne, A. (2006) The effect of a new approach to group-work on pupil-pupil and teacher-pupil interaction. Journal of Educational Psychology, 98, 750-765.
Baines, E., Rubie-Davies, C., and Blatchford, P. (2009) Improving pupil group work interaction and dialogue in primary classrooms: results from the TLRP SPRinG project. Cambridge Journal of Education, 39, 1, 95-117
Case studies
In the CSPAR project case studies were conducted comprising interviews with practitioners (teachers, headteachers and TAs) and pupils, semi-structured observations and documentary evidence. The case studies aimed to provide a complementary and detailed portrayal of individual classes, which would provide the basis for a more interpretive and grounded analysis of factors relating to class size differences.
Case studies were also conducted as part of the earlier KS1 stage of CSPAR (see Blatchford et al., 2002).
As well as systematic observations of the pupils in the MAST and SENSE studies, there were also case studies of each pupil.
Teacher questionnaires
One form of data collection was from practitioners themselves, and a main source of data was from open-ended responses to the CSPAR and DISS annual national questionnaires completed by teachers, headteachers and TAs. There was a particular focus on responses to the teacher questionnaire (TQ). This was an annual questionnaire from teachers returned by 486 teachers altogether, 206 in year 4, 184 in year 5 and 96 in year 6. There were also annual questionnaires completed by TAs (340 altogether across the three year groups) and Headteachers (437 altogether across the three year groups). In each of the three school years, there was a set of questions which asked teachers to comment on whether, and if so how, the number of children in their class size had affected their teaching, pupil learning and behaviour, grouping practices, pupil relations with other pupils, and classroom tasks and the curriculum. See Blatchford and Russell (2020) for more details on the research methods, including the construction of questionnaire items.
Practitioner views. Also in the SPRinG study there was an analysis of teachers’ experiences of group work and the SPRinG project.
Classroom map.
There were three separate sources of classroom mapping data:
1.The Primary Classroom Groupings Project (PCGP) examined grouping practices in Year 2 (6-7 years) and Year 5 (9-10 years) classrooms. Over 1000 groupings were described in the questionnaires describing 187 classes.
2.The Grouping Research in Secondary Schools (GRIS) Project examined grouping practices in secondary school classrooms (11-12 years and 14-15 years). There were 47 secondary schools and questionnaires were returned from 128 Year 7 and 120 Year 10 classes, providing data on 1767 groupings.
3.Classroom mapping data was also collected as part of The Class Size and Pupil Adult Ratio (CSPAR) Project. This captured grouping practices for the youngest pupils in school – Reception (4-5 years) classes. Questionnaires were returned by 485 Reception class teachers (from 220 schools which were randomly sampled from 8 LEAs). These questionnaires provided data on over 2000 groupings.
A detailed treatment of the classroom mapping results is in: Baines, Blatchford & Kutnick, 2003.
Taken together, these three research projects provided a unique developmental account of grouping practices throughout the child’s life at school.
There were also a number of other methods of data collection developed for the projects, including academic attainment measures, pupil self-ratings, teacher ratings, and ratings of school practices
In general, the methods of data collection were designed to inform each other and be conceptually integrated. The aim was to combine more objective, naturalistic observations with practitioners’ experience. In the book Rethinking Class Size (Blatchford & Russell, 2020) a fuller justification of the data collection methods is provided.