Week 23 - Collecting data

Why collect data?

As practitioners, you have hunches about what is working and what isn’t working in your practice and for your students. However, as the Ministry of Education (n.d) points out “without evidence to back them up, hunches remain subjective.” The Ministry goes further, saying that well-planned data gathering can “reveal new patterns of insight, justify change and motivate colleagues, stakeholders, and school leaders into taking action.”(Ministry of Education, n.d).

Although you have checked your hunches/assumptions with research literature in RESEARCH assessment 1, you are required to include in your Action Plan for the Inquiry how you plan to collect data from the relevant members of your community to meet criterion 3 of RESEARCH 2. You have to explain which data collection tool(s) you plan to use, and why you think these are suitable. You need at least one tool, e.g. survey form. You can have more than one method of data collection, e.g. survey form for students and interview for colleagues if you wish. This week’s class notes will guide you on how to do that.

You also need to consider ethical principles/guidelines for this process which has been discussed last week (week 22).

What type of data can you gather?

The data that you gather can be categorised into two types: quantitative and qualitative. Quantitative data is data that is numerical and helps answer questions like “what?” and “how many?” but is not always able to explain “why” the situation happens (Easton, 2012). Qualitative data is data that is descriptive (i.e. text rather than numbers) and often gives more detail and greater depth in answering the “why?” questions. However, as researchers are not always able to collect qualitative data from a large population, you need to be careful when making generalisations based on qualitative data (Easton, 2012). Here are some indicative examples of quantitative and qualitative data.

Examples of quantitative data include:

• Surveys: the number of responses to close-ended questions in a survey (e.g. multiple choice, Likert scale, Yes/No, True/False)
• Pre and post test: student grades or test scores
• Classroom (or other location) observations: e.g. Behaviour tally sheets
• Students’ just-in-time responses using digital technologies and posts on social media: the number of responses.

Examples of qualitative data include:

• Document analysis such as students’ essays or blogs
• Surveys: the answers you get from open-ended questions in a survey
• Focus groups and/or interviews: The responses from the interviewees
• Classroom (or other locations) observations: field notes
• Students’ just-in-time responses using digital technologies and postings on social media: the content (text-based) of the responses.

You need to indicate what type of data you would like to collect in the Action Plan. When deciding on the data collection methods you plan to use in your Teacher Inquiry (we will go into more detail about the methods below) you need to think about the purpose of your inquiry and how many different data collection methods you should employ. Using mixed methods (often a combination of quantitative and qualitative methods) will give you richer data. If different methods indicate the same findings you can be more confident in those findings. However, you also need to make sure that you don’t try to do too much. You are a single person, don’t spread yourself too thin!

How do you decide on the data collection tools for your Inquiry?

By now you should have decided on your inquiry topic(s) and your teacher inquiry question to guide your plan. You need to choose data collection methods suitable for gathering data to help address this question. The following sections will explain the steps to select data collection tools for your Inquiry.

Remember that you are still able to refine your teacher inquiry question at a later stage when you work on the data collection methods.

Checking your key terminologies

In the section ‘Defining the key terms in your inquiry questions’ of the class notes of week 21, you were asked to provide definitions of the key terminologies so that you and everyone involved in the Inquiry have the same understanding of those terms when the plan is developed and implemented. To decide on which data collection methods you should plan to use, you also need to be clear about the key terms of your inquiry question.

For example in Practitioner teacher inquiry and research, Babione (2015) describes a teacher inquiry project about a “One-to-one iPad initiative”. The research question is “How does the implementation of a schoolwide iPad initiative at a high school affect the perceived level of student achievement across all math curriculums?” (Babione, 2015, p.250). So what does ‘perceived level of student achievement’ mean? Does it mean how students perceive their level of achievement? OR How teachers perceive the student level of achievement? OR parents perceive the student level of achievement OR all of these? And specifically, what is achievement? Is it the students’ improved test scores? OR the students’ improved skills to perform collaborative tasks? OR the students’ improved competency in problem-solving skills?

Different meaning of a terminology requires different data to answer the research question. To work out these type of details, you can revisit the terminology that you have used in your Research Essay (RESEARCH assessment 1), week 21 and decide on a workable definition for your teacher inquiry.

What data do you need to collect?

Once you have a clear definition about what your inquiry is actually looking at, think about the data that you need to collect. For example, if you define achievement as the mean of your class’s test results, you need to collect data from your students’ test scores. Or, if you define achievement as meaning improved skills to perform collaborative tasks, you might want to talk to students (or their teachers) about their skills, or observe the students when they work on their collaborative tasks, or look at their collaborative worksheets. These are data collection methods, which you would choose based on the data that you are interested in.

Starting to work on your data collection plan

To help you think about these questions, begin by viewing the “Data collection” video from the beginning to 6:25 (required) in which Dr Margaret Riel suggests different data that provide information about student learning. This should give you some ideas about what data you think is necessary to gather. Then, from the key terms of your inquiry question, list all the possible data you think could help you answer it.

However, as much as you may want to have all the relevant data, this goal may not be realistic, as it can be time consuming to design, administer, collect and analyse data. You need to consider the constraints and opportunities in your specific context.

To help you decide on your data collection method(s), in the following section, we will provide more detail on their strengths and limitations so you can consider the methods that will best suit your inquiry plan. We will focus on four common methods that are often used in educational research.

The ethical considerations as required in criterion 3 of RESEARCH 2 and discussed in section ‘Ethical Considerations for Your Action Plan’ of the class notes of week 22 should be relevant to the method(s) of data collection you choose for the Inquiry.

Data collection methods

Survey/questionnaire

A survey or questionnaire is used to obtain opinions from respondents to provide information through a set of questions (Babione, 2015).

A strength of this approach is that a survey or questionnaire can be distributed and collected quite easily (Babione, 2015), especially since there are a number of online survey tools such as Google Forms and Survey Monkey which make it fairly straightforward to create, send and collect responses. A survey can be done with a small or large number of respondents and can include both quantitative and qualitative data. Some online survey tools also include some automatic data analysis features.

The downside can be that you are not always able to control the number of responses, especially for an anonymous survey, and the responses can be biased or the responses may not always reflect the real opinion of the respondents.

Now, consider the pros and cons of doing a survey and decide if you would like to use surveys as one of your data collection methods. If you wish, read the section about surveys extracted from Efron and Ravid’s (2013) book (supplementary) related media. The extract provides guides to help you write survey items with good examples. Additionally, this checklist for developing a survey (Peters & Irish, n.d.) (supplementary) related media provides a list of questions to help you check your survey design. Don’t forget to include the tool (the survey questions) in your assessment submission if a survey is one of your chosen data collection methods.

If you plan to conduct a large scale survey, for example, across the whole syndicate or the whole school, you can use the calculator on this web page (http://www.surveysystem .com/sscalc.htm) to determine the appropriate sample size for a survey (e.g. how many responses you should collect) to have the result within a certain margin of error (also called the Confidence Interval) and for a given population (i.e. the total number of students in your school).

Interviews:

Interviews are frequently used to obtain verbal perspectives and opinions from individuals or group about their understandings and experiences, thoughts, feelings, motivations, and the actions of others (Babione, 2015,p.124). Here is an example of an interview schedule including the purpose of the interview and interview questions.

A strength of interviews is that they provide a richer and deeper perspective about the situation and help to verify information from other sources.

A constraint of interviews is that it can take time to arrange and conduct the interview session and analyse the collected data. Because an interview is a person-to-person interaction, interviewees might answer to please the interviewer instead of telling them what they really think and feel, especially if the interviewees are afraid of unfavourable consequences as a result of the interview.

If you choose the interview as your data collection tool to use in your teacher inquiry plan, read this section about interviews from Efron and Ravid’s 2013 book (supplementary) related media. The section outlines different types of interview, the interview process and how to develop the interview questions. Don’t forget to include the tool (the interview questions) in your assessment submission if an interview is one of your chosen data collection methods.

Observations

Efron and Ravid (2013) define observation in research as the act of purposefully and systematically observing the activities, people, and physical aspects of the educational setting.

The strength of observations is, as Babione (2015) points out, that observing humans in natural settings provides insight into the complexity of human behavior and interrelationships among groups. Observations allow you to collect data such as nonverbal behaviours, gestures, and body language, the dynamics of the setting that cannot be obtained through interviews or surveys.

If you choose observation as the data collection tool you would like to use in your teacher inquiry plan, read this section about observation extracted from Efron and Ravid (2013). The extract explains the different types of observation, its process and how to develop an observation form such as a protocol form for qualitative observation and behaviour log, tally sheet and checklist for quantitative observation. Don’t forget to include the tool (the observation protocol form/behaviour log/tally sheet/ checklist, etc) in your assessment submission if observation is one of your chosen data collection methods.

Artifacts and documents

Artifacts and documents are records that provide contextual information for a study (Babione, 2015). Examples of artifacts and documents include student work, school reports, demographic information, grades, test scores, portfolios or teacher journals.

The advantage of this data collection method is that the artifacts and documents may already be available during the course of your teaching and you don’t have to design an extra tool to collect the data.

If you choose artifacts and documents as the data collection tool you would like to use in your teacher inquiry plan, read this section about artifacts and documents extracted from Efron and Ravid (2013). The extract provides a checklist of suggestions and guidelines for gathering and using official and personal artifacts and documents.

Pre and post intervention data

What baseline data do you need to collect? Baseline data is data collected at the start of an inquiry project that allows you to establish the current situation which will enable a point of comparison for data you collect throughout your inquiry.

The video on developing a quantitative research plan (supplementary) related media may be relevant if you are thinking of including experimental methods in your data gathering (e.g. comparing pre-test and post-test student results).This could be used if you are developing an intervention designed specifically to raise student achievement in a particular area, e.g. spelling or mathematics. You could give students a pre test to ascertain their current level and then give them a similar or the same test after the intervention to determine whether there has been improvement.

For each data collection method you need to explain why you have chosen it, the data it will provide for you and why this data will be important.

Integrating the Principles of Kaupapa Māori into your Teacher Inquiry

As you choose the type of data to be gathered and the respective data collection method(s), remember to consider a KM approach in the process. As examples, look at the Taonga tuku iho - the cultural aspirations principle and Ako - a teaching and learning relationship, some questions that you can consider are:

• Taonga tuku iho - the cultural aspirations principle: To what extent do you consider the way Māori people think, understand, interact and interpret the world when choosing the type of data to gather and the method of data collection?
• Ako - a teaching and learning relationship: In what way would you learn back from the students and/or whanau about the pedagogy to be implemented through the gathered data and data collection process?

You can select any other principle and raise your own questions as well.

Write down some key ideas to answer the question in your e-Portfolio.

Checking You’ve Done All That Was Required for This Week

The Tasks-list on the next tab helps you to check you have done all the required activities.

References

Babione, C. (2015). Practitioner teacher inquiry and research. USA: John Wiley & Sons. (e-copy available in Unitec library).

Easton, C. (2012, April). Data for self-evaluation. SecEd. NFER. Retrieved from http://www.nfer.ac.uk/schools/data-for-self-evaluation-seced.pdf

Efron, S. E., & Ravid, R. (2013). Action research in education: A practical guide. New York, NY: The Guilford Press. (e-copy available in Unitec library).

Ministry of Education.(n.d.). Data analysis. Retrieved from http://elearning.tki.org.nz/Teaching/Teaching-as-inquiry/Data-analysis#js-tabcontainer-1-tab-2

Peters, T. and Irish, J. (n.d.). Introduction to Survey Research Methods. Retrieved from http://hms. harvard.edu/sites/default/files/assets/Sites/Academy/files/CTL%20Resource%20Survey%20Research.doc

Riel, M.( 2014, 28 April).T7: Data collection.[Video file]. Retrieved from https://www.youtube.com/watch?v=PyILgB1R5Cc

Shaddock, A (2013). Using Data to Improve Learning: A practical guide for busy teachers, ACER Press.