Data Collection and Analysis

----This research aimed to explore methods to improve critical literacy in my classroom by developing activities to engage students in peer to peer activities that encouraged constructive critical feedback both within the classroom and between year 7 and year 9/10 students. Using the internet as a means to foster local connections in a global context was intended to serve as an effective mechanism to help prepare students for the complex and global nature of their current and prospective world. My students used blogs and nings to change the way they collaborated and interacted, as part of the learning process. They were also able to use the internet to navigate through their desired information sources and hence develop opinions and construct learning avenues for other students. The Internet provided students an immediate method of searching for and capturing multiple perspectives and accounts on a variety of issues, whereas many textbooks confine students to information embedded with stereotypes and narrow perspectives (Maguth 2008). This qualitative action research study, investigated what online social networks (and adolescents engagement with them) have to teach us about both print and multimodal texts and literacies. In order to understand the data collection process I described my research in the following stages (figure 4):

stages.png
Figure 4 – The research stages took 4 weeks to complete. Pre Data Collection took place during the 8 to 10 weeks before the study began.

Collecting and analysing data in this study helped me determine what it was that students found engaging and relevant in my curriculum. It helped my curriculum move a step closer to being more relevant to my students’ lifeworlds.
Figures 5 and 6 give the breakdown of activities for each stage of the research. These are followed by a description of each stage.

y9_10_timeline.png
Figure 5
y7_timeline.png
Figure 6

4.1 Prior to data collection - Introduction to onine environments


All students were introduced to online environments at least eight weeks before the research began. Level C students used the classblogmeister blogging environment, (http://classblogmeister.com/blog.php?blogger_id=97161). Level C students introduced themselves to other overseas schools that were actively using the classblogmeister environment and explored online publishing and collaboration by publishing a range of work including flash, claymation, photostory and moviemaker products as well as by exchanging comments and ideas with overseas students. Some overseas students continued to leave comments on the blogs throughout the research - this communication was noted as data and was collected using screen clips during the stages of data collection, but it did not play an important role in the formal research project. Level C students were also introduced to their Level C ning (http://gcasey.ning.com/) while year 7 students were introduced to a separate ning (http://y7gcasey.ning.com/). All students were given time to explore the different components of the nings which included “My Page”, photos, videos, forums, events, groups and notes, as shown in figure 9.


4.2 Stage 1: Students as Researchers


This stage occurred during week 1 of the study and consisted largely of teacher directed content as shown in the student handout (appendices 1 and 2). When Level C students were given the option to continue using their Class Blogmeister online blog to publish their stage 1 work or to move to the Level C online ning environment, six out of seventeen students chose to use the Class Blogmeister blog for this stage. The Level C students were the researchers of how education and learning had changed over time. They were shown specific questions, detailed in Appendix 1, such as:
  • What is a researcher?
  • How has the Internet changed?
  • What is Web2?
  • How dangerous is the Internet?
They initially gave their immediate opinions/reactions on these questions and then used the Internet to research extended answers.
Students then brainstormed as a group:

  • What has changed?
  • How do you like to learn?
  • If this was your dream school in the 21st Century how would year 7 students be learning right now?

This was followed by two activities shown in Appendix 1: “The Death of Education, the dawn of learning” and “Literacy Debate: Online, R U Really Reading”. These two activities required each student to type their responses as comments on the Level C ning under the appropriate cartoon or photo at http://gcasey.ning.com/photo/r-u-really-reading?context=featured and http://gcasey.ning.com/photo/death-of-education?context=featured. These text based comments were then collected electronically using screen clips. The seventeen year 9/10 Level C students were also asked to keep a reading diary for one week. This gave them an opportunity to document all of their reading from all sources and allowed them to become active participants in the activity “Literacy Debate: Online, R U Really Reading”. This data was collected by our school’s standard method, which involves students uploading their word documents to an allocated area on the school network called the “Teachers Intray”. I spoke with students about developing their own assessment goals and plans for this project but there were no students keen to start this process.
Twenty year 7 students explored and discussed a range of ideas and the concepts of “boring” or “ordinary” in relation to their everyday life. Data collected included class brainstorms, student personal opinions which were collected using SmartArt® and a collection of boring or ordinary photos taken by students and the teacher (Appendix 5). Inspiration software was used to produce and collect class brainstorms. Student opinions and ideas were collected by taking screen clips from the comment sections on the teacher blog and from the student online published work.
Microsoft OneNote® software on the teacher’s laptop was used to collect and organise screen clips of student work. Data collection occurred at the following websites:

At the end of this stage year9/10 Level C students joined the year 7 ning and were allocated three year 7 partners to use for collaboration. Between class times, all students were able to communicate online, solely using this ning, and students did not know each other as each used a fake name.


4.3 Stage 2: Students as Designers


From week 2 all students used the year 7 ning to communicate online. Level C students thought about the things that they liked to do using ICT and which they believed that year 7 students might be interested in learning. They attempted to gain feedback, on their suggestions for learning, from year 7 students by leaving comments on year 7 students’ individual “My Page”. Year 7 students were asked to respond to comments from their older Level C student mentors but most were hesitant to respond to comments from the older students and needed encouragement, from myself, to commence the building of their online relationship.
During this stage, Level C students designed a range of products for year 7 students using SmartArt® in Microsoft Word - some samples are shown in Appendix 9. Here, the teacher was only the facilitator with designs being student generated and each Level C student designed something different - this included multimodal designs where students combined images and sound to produce video clip tutorials. Level C students then used the year 7 ning to gain further feedback, about potential pathways, from year 7 students, through ning comments which were collected as data using screen clips - the number of comments and replies to comments for each Level C student is given in Appendix 20. In this Appendix I have also graded the comments, and the responses to these, to give an indication of their quality. When the student designs were completed they were uploaded to a school network drive and then collected as word documents.
A range of student directed ning activity occurred throughout this stage. These included online discussion forums and purpose-centered groups that were formed, photos uploaded, ning friends being requested and accepted, blog posts and videos uploaded. This data was collected using screen clips and noted on the summary tables given in Appendix 21.
Some year 7 students were happy to make friends with Level C on the ning (just a click of a button) but were very slow to reply to comments. Initially some Level C students did not leave comments with their year 7 partners but only left a friendship request and then wondered why the year 7 students had not replied to them. Year 7 students also explored photo editing using a range of photos that they found boring or ordinary. This they did using Adobe Photoshop CS3®, software. They started to write about their photos during this stage, leaving online comments to others amongst their year 7 peers. As a part of their Photo Journey, year 7 students started to explore podcasting with the aim of verbalising their thoughts about their photos using Audacity® and Adobe Sound Booth® software. This caused a number of problems with noise in the classroom and students first insisted on having playtime - recording all sorts of noises including coughs, squeaks, farts and squeals. None of these, at this stage, were saved or uploaded to the ning.
Stage Two largely involved student directed learning; Level C students were empowered to explore designs that matched their beliefs about good learning and to choose the software that was best for their needs. Their design was determined by negotiation with their three year 7 partners. Appendix 10 provides samples of screen clips from online comments between students. It also gives a range of samples of creative photo editing as completed by year 7 students (taken as screen clips from the year 7 ning) as well as Level C student designs (collected as word documents after students uploaded them on the school network).
Again I spoke with students about taking some charge of their assessment by looking at the appropriate schools VELS (Victorian Essential Learning Standards) assessment items but students avoided conversations in this area.



4.4 Stage 3: Students as Producers


During week 3 informal interviews were conducted with the seventeen Level C students over two class periods. At the same time I was finding it difficult to cope with the number of year 7 students that needed help due to the wide range of activities in which they were involved. These included editing and uploading photos to the ning, recording and uploading podcasts, general ning activity and writing about their photos. To help give year 7 students as much support as possible I started a ning discussion forum asking for students to be mentors to others in their own year 7 class, as shown in figure 7. Six students responded and requested to be mentors which resulted in more students being able to be supported at the time when they most needed it.

figure_7_be_a_mentor.png
As a facilitator of learning, I helped Level C students solve problems or issues that arose - such as technical issues or problems due to lack of communication between year 7 and level C students. At this stage Level C students were using the software of their choice to produce their design. During this stage Level C students published their product to question their three year 7 partners in order to gain feedback for improvement. This data was collected by downloading the tutorials or taking screen clips of online comments and uploaded content. Figure 8 gives a screen clip of three Level C products; video tutorials on Audacity, Photoshop and Moviemaker.

figure_8_3_products.png
In this stage I developed an assessment sheet for Level C students, a sample is shown in figure 9.

figure_9_ict_for_creating.png
Students used these assessment tables to place dots at the level they hoped to achieve with their work – I found it very difficult getting students to set their own targets and have input into their own assessment (this approach was developed over a three week period after being unsuccessful at encouraging student generated assessment).

The development of student centered learning continued in the classroom. A few students wasted their class time playing games - under normal teacher directed classes this would have resulted in students receiving a detention but it was made clear to students that they were responsible for their own learning and that they were to organise their time in a way that was suitable for them. Other students used the opportunity of having responsibility for their own learning with open arms and used their class time responsibly. Appendix 28 gives a summary table showing the approach to work, for each individual Level C student, during the first three weeks.

During week 3 I also investigated some student out-of-school online activity that arose during classroom discussions. Some year7 girls invited me to join their “Spyro” ning. I visited two student MySpace sites. It was interesting that a number of the boys indicated that they had MySpace accounts but did not want me to see their account. All student MySpace sites, that were openly discussed, were set to private and this showed that they were using some protection for themselves outside school in their online sites. One year 7 student also shared her Youtube account details with me and we were able to make “friends” as users of Youtube.



4.5 Stage 4: Students as Publishers


During week 4 (stage 4), Level C students were to publish their final version, of what they had made for year 7 students, by uploading it to the year 7 ning. In total these final products included a range of multimodal work as shown below and as detailed in appendix 24:
1. Video - explaining how to combine clips in Audacity
2. Word document - tutorial for using Flash
3. Video – how to use Photoshop software
4. Set of screen clips published on the ning - tutorial on Pixel Art
5. Set of screen clips published on the ning – how to edit car pictures.
6. Video – how to open a video trailer on the Apple website

This data was collected by downloading the files from the ning and/or taking screen clips. Year 7 students finalised the publication of their edited photos, peer feedback and podcasts. In this stage all students were also involved in reflection, as shown in appendix 19, and self assessment, as shown in appendix 20. During this week Year 10 students (five of the seventeen in the Level C group), although they were at school, did not attend class due to other school commitments. In fact they did have time available at school and, hence, were expected to complete the project work without teacher supervision.



4.6 Data collection across the stages


Field notes

As discussed in 3.4, observations of the student research participants occurred during all classes. This field note data was collected through hand written and typed notes and were completed for each of the sixteen Year 9/10 Level C classes and the ten periods of Year 7 classes. The quantity of data ranged from a half to one page of hand written notes for each lesson. One new A4 page was used for each new lesson hence sixteen pages were collected for Year 9/10 Level C class and ten pages were collected for Year 7.


Data collection tables

A list of student online activities was collated on rough handwritten hardcopy summary tables. These data collection tables were used to record a range of activities including completion of work, number of comments received or given, ning profile changes and other ning activity. The tables were produced in Microsoft Excel® but printed and used in hardcopy format for ease of use in the classroom. Samples of these have been scanned and can be found in Appendix 22 but are not self explanatory.


Hardcopy folders
A series of physical hardcopy folders were setup to hold handouts, printouts, hand written field notes and reflections. Colour-coded plastic section sheets were used to separate lessons from each other. During each classroom lesson I wrote field notes while in the classroom. These were scribbled on note paper while walking around the classroom or, if time permitted, typed. At the end of each class I printed student active blog and ning pages, brainstorms and relevant student work which were placed in the hardcopy folders. I also wrote class reflections which included some planning for the next lesson. Plans for subsequent lessons took into account action research cycles that were apparent, student reactions, standard and quantity of work completed as well as discussions with students and teachers.

Research diary
Throughout all stages I used a research diary and this enabled me to track my thinking which changed over time. I was able to use this diary to track things such as themes, activities and student engagement for later consideration. To organize the collected digital data I set up a OneNote digital notebook. The OneNote digital notebook was set up as series of sections, section groups and pages (section groups contain sections, each section contains pages). By using this digital notebook I was able to simply drag and drop pages, sections and section groups as the quantity of data grew and themes became evident. I organised the data in the OneNote digital notebook into three main section groups as shown in figure 10. These section groups are:
1. Recorded Data: I developed this section group, shown in figure 11, once data collection was completed. I used this section group to organise the significant data, data that would be used for further analysis. This data was further organised into pages, sections and section groups to help clarify and view patterns of activities.
2. Reviews: This section group, shown in figure 12, contains my weekly reviews which include interesting things that were occurring during a given week and further things that I targeted to look for when reviewing the data.
3. Week by Week notes: This section group, shown is figure 13, contains my lesson by lesson snapshots of the happenings in a given lesson, the online activity that occurred and some field notes. This section group was also sub-divided into further section groups – one for each week of the data collection.

figure_10_Onenote1.png

figure_11_Onenote2.png

figure_12_onenote3.png
figure_13_OneNote_4.png

4.7 Classroom Activities


During the four weeks of this study, a large amount of data was collected through a great range of activities. In order to analyse this large quantity of data the classroom activities were broken up into teacher directed activities (table 1) and student directed activities (table 2). The following tables give a summary of the types of data that was collected for each of the activities.
Activities: Teacher directed:
Activity
Teacher directed activities:
Summary of Data Collection
A1
Initial questions given to Level C students:
How has education changed?
How do you like to learn?

If this was your dream school in the 21st Century how year 7 students would be learning right now?
(Student responses on classblogmeister blog via comments on a teacher post.)
Activities A1 to A4 involve student writing – these were collected from blog comments and student blog posts by using screen shots and/or coping and pasting of text.
A2
Level C student research questions- Understanding the world around us and gaining student opinions on questions a to e.
(Students chose to answer these on their Level C ning or their classblogmeiser ning.)

A3
The Death of Education, the Dawn of Learning - cartoon and questions for Level C students.
(Responses on Ms Casey page via comments on the photo uploaded to the Level C ning)

A4
R U Really Reading? Literacy Debate - Level C ning.
(Responses on Ms Casey page via comments on the photo uploaded to the Level C ning)

A5
Student reading Diaries - word documents.
Students kept a reading log for one week using a word table. This was uploaded to the school network where they were collected as data.
A6
Student Designs & Mindmaps.
Students completed these using Inspiration or SmartArt software. These were uploaded to the school network where they were collected as data.
A7
Level C student product for Year 7 students.
Students left comments to each other to gain feedback on project work. These were collected using screen clips. Students also produced a variety of multimodal documents including video clips, screen shots, and word documents. Students published these on the year 7 ning. This data was collected by downloading content and taking screen clips.
A8
Year 7 Production - Photo Journey
Students uploaded their chosen “boring” or “ordinary” photo and a number of edited versions of their photo on their “My Page” on the year 7 ning. They wrote about their photo and encouraged peers to leave comments about their photo. This data was collected using screen shots.
A9
Year 7 Production - Podcast/Voki.
Students used a podcast or voki (Web2 podcast) to verbalise aspects of their chosen photo. This data was collected by recording the podcasts.
A10
Student goals, reflection & self assessment.
Students were given a paper handout to complete using a pen or pencil. These were collected from students and used as data.
Table 1 – Summary of the data collection that took place through teacher directed activities.
Data collection of student directed activities are summarised in the table below.
Activity
Student Directed Activity
Summary of Data Collection
G1
Peer Feedback
Although most peer feedback was teacher directed from the above activities it was hoped that students would start to give constructive feedback without direct instructions – there is no significant additional data collected here.
G2
Ning Group or Discussion Forum activity
Four Level C students created groups in the two ning environments. Two groups were formed on the Level C ning and two on the year 7 ning.
Two Level C students started discussion forums on the Level C ning. Two Level C students started discussion forums on the year 7 ning.
Five year 7 students started discussion forums on the year 7 ning.

G3
Student Ning comments
This data included most of the comments that students exchanged on each other’s ning pages. Most comments collected were unrelated to the specific tasks at hand, with students communicating with each other just because they could.
G4
Other Ning Activity - Membership, avatars, profiles, uploading, applications, photos, general posts
Year 7 ning had the following group memberships:
Jeeves Group – 3 members
Gradients Helping Thing – 5 members
Level C ning had the following group memberships:
People that think Jack should move back – 7 members
Vendetta – 5 members

When students joined a ning they chose their fake name, developed a profile and uploaded an avatar – these three characteristics formed their online identity. Students regulary changed their profile. Level C students chose to use a different avatar on the year 7 ning compared to the Level C ning. All ning activity was recorded in data collection tables that were updated each class. Screen clips were taken of student profiles and avatars for data collection.

G5
Informal interviews
This data was hand written while the interviews were taking place.
G6
Observations
This data was collected in field notes and weekly reviews.
G7
Wider Internet activity
This data was collected by reading student profiles and student reflections. It was also taken from the student interview information.
G8
Overseas communication
This data was collected from the classblogmeister blog environment using screen clips.
Table 2– Summary of the data collection that took place through student directed activities.

4.8 Themes and Categories


This research asked the following questions:
1. What kinds of literacy and learning practices are adolescents involved in as they interact in online social networks; how do they make sense of these, and how are they affected by them?
2. What do online social networks (and adolescents’ engagement with them) have to teach us about both print and multimodal texts and literacies and how can this knowledge be used to update and strengthen literacy curriculum?
3. What kinds of approaches, models and resources are needed to support teachers in the development and implementation of an ICT-based curriculum that addresses multimodal forms of literacy?

The first three themes given in table 3 identify directly with the three research questions above. Each of these three themes is broken down further into categories that could specifically provide data from the classroom activities which are listed in table 1 and 2 (on previous pages). Table 3 (below) shows the relationship between the three main themes/research questions and the data collected. Organising the data in this way allowed each of the research questions to be allocated specific data from the classroom activities.

Four main themes and the 17codes used for data analysis
Theme
Appendix
Activities data was collected from
1. Examining Popular Digital Culture:


1.1 In what are students involved?
Appendix
3A, 3B, 3C, 3D
A4, A5, G4, G5,G7
1.2 How do students make sense of these?
Appendix
4A, 4B, 4C, 4D
A1, A2, A3 & A4
1.3 How are students affected by them?
Appendix
A1
2. What can Educators learn?
Appendix
A1
3. Supporting teachers to address multimodal forms of literacy


3.1 Approaches


3.1.1 Using creativity - through Nings, avatars, friends, groups & forums as well as published work & comments. Podcasts, Vokis etc
Appendix 5
A6, A7, A8, A9, G2, G4
3.1.2 Avoiding problems & issues - spam, bullying, copyright, music uploads etc
Appendix 6
A1, G4
3.1.3 Student Ownership - students taking charge of their work
Appendix 7
G4
3.2 Models for Teaching & Learning


3.2.1 Student research
Appendix 8
A1, A2
3.2.2 Students as designers & producers
Appendix 9
G1, G2, G3, G8
3.2.3 Collaborating with others
Appendix 10
G1, G2, G3, G8
3.2.4 Student Reflection
Appendix 11A, 11B
A10
3.2.5 Student Goals and Assessment
Appendix 12
A10
3.3 Resources


3.3.1 Students' wider use of the internet
Appendix 13
G7
3.3.2 Nings - the value of these
Appendix 14
G2, G4
3.3.3 Classblogmeister the value of these

A1, A2, G8
3.3.4 Using students as resources
Appendix 15
A7, G1
4 Other bits and pieces
Appendix 16

Teacher Weekly Reflections
Appendix 17
G6
Level C Week 1 Activity Summary
Appendix 18
A1, A2, A3, A4, A5, A6
G1,G2,G3, G4, G6
Level C student activity on Level C Ning & approaches to work
Appendix 19
A2, A3, A4, A5, G1,G2,G3, G4, G6
Level C student activity on Year 7 Ning
Appendix 20
A7
G1,G2,G3, G4
Year 7 activity on Year 7 Ning
Appendix 21
A6, A8,
G1,G2,G3, G4, G7
Samples of rough handwritten classroom data tables
Appendix 22

Table 3 – The first three themes in this table relate directly to the three research questions. This table connects the collected teacher directed and student directed data to each of the three research questions. The data in the appendices is organised in these same four themes and seventeen categories, hence when answering the three research questions data can directly be drawn from the above appendices.

4.9 Data Analysis


Analysis and interpretation rises not only for the data but also from the perspectives the researcher holds. Social values and ways of making sense of the world can influence which processes, activities, events and perspectives researchers consider important enough to code. Undoubtedly, my research was influenced by my own education, interests and motivations. In particular, my criticism of the lack of new literacies in the secondary classroom has been recently influenced by twelve months of teaching English as a Second Language (ESL) in South Korea during 2007 and my recent readings of other researchers and critics within the new literacy area such as Dillon (2006), Faber (2006), Jewitt (2008), Johnson & Kress (2003) and Kress (2003). However, my analysis for this research is largely a critical reflection upon the extent to which social networking can help students become more critically aware of the world around us. Like Kellogg (1998) I attempted a meta-analysis of the process at play during my research, paying particular attention to the pedagogical implications of the research experience on my own teaching.

At key places in this paper I intentionally write in the first-person to remind myself and that we cannot truly remove ourselves from our research. Our biases are built into the experimental design and our instruments influence the observations we make. I used Bogdan’s (2003) basic approach in mechanically sorting research data after it had been collected and like Bogdan and Creswell (2007), I developed units of data, coding categories and coding families as tools for sorting data. Decisions were made as to when one unit of data ended and the other began - some units of data had more than one coding number. I then represented the data in figures, tables and discussion areas as recommended by Creswell (2006, p148).

Once I completed data collection, as suggested by Bogdan (2003) and the final observation of the study was written, I took a break of 6 weeks to let the material sit before coming back to it fresh and rested. Hence distancing myself from the details of the fieldwork and allowing relationships between myself and my students to be put into perspective.

I found the Bogdan (2003, p184) interpretation of ideas particularly helpful - including:

· Revisiting writing that helps you understand the basic ideas behind qualitative research
· Reading published studies related to your own research.
· Purposely trying to evaluate your subjects and the situation you are observing.
· Asking: What are the implications of the findings for practice, for current events and for the theoretical orientation from which you think you are operating?
· Speculating about the assumptions that might be held by the audience for which you are writing. Strategizing about how to interpret what you have come to understand.
· Is there a story of an incident from your research that captures a major insight or understanding you have derived from your work? Tell that story and try to work with it. How does it relate to your theory, findings etc?
· Work on writing a clear paragraph summarizing what it is you want to tell readers.

Phillips (1990) discusses objectivity as the label - the stamp of approval that is used for inquires. By analysing student reflections and noting student reactions I attempted on a number of occasions to seek my students’ stamp of approval to ensure my objectivity. Using students as researchers themselves allowed me to continually compare their thoughts, ideas and understandings with my own. I used Wolcott (1990) nine points below to guide my work towards reliability and validity of data:

1. Talk little, listen a lot
2. Record Accurately
3. Begin writing early
4. Let readers “see” for themselves
5. Report fully
6. Be Candid
7. Seek feedback
8. Try to achieve Balance
9. Write accurately

Bland & Atweh (2007), Fielding (2001), Leithch et al.(2007) discuss engaging students’ voices in action research as a means of working collaboratively towards positive outcomes for students and schools sharing the responsibility and achievement. They acknowledge that it takes time for students to adjust to the novelty of having their voices respected – this, I believe, can be seen in the time taken for some students to move from the teacher directed style of learning to student centered learning.