Friday, April 26, 2013

CoCo Seminar - The Digital Education Revolution: initial data analysis of teacher and student reported use of laptops in year 10 science

After the publication of my first paper, I was humbled to be asked to present my work as part of the CoCo Seminar Series at the University of Sydney. My seminar covered the background of the Australian Digital Education Revolution, how it was implemented by the schools I work with, the methodology and results of my first paper, the methodology and results of my second paper (in press) and my current research. Below is the prezi from the presentation and below that is an explanation of each 'slide'. Any feedback or discussion would be greatly appreciated. You can also listen to and watch the actual webcast of the seminar. 

Slide 1 - my ever present microscope graphic (courtesy of Bal√°zs Turai) with tinyurl link in homage to the diggers of Kokoda on the eve of ANZAC day (CoCoDER was already taken!).

Slide 2 - 7.30 Report from 4th Dec 2007 featuring Kevin Rudd, Mark Pesce and principals discussing the DER (post-election). Nice quote from Pesce that providing every student with a device will force the issue with teachers, policy makers and curriculum design.

Slide 3 - A Digital Education Revolution policy document (Rudd, Smith & Conroy, 2007 (pre-election)

Slide 4 - Melbourne Declaration (MCEETYA, 2008)

Slide 5 - back in 2008 I moved from being a Physics teacher in one school  to eLearning Adviser to many schools, right at the outset of the DER. As such I had Principals, Consultants and Directors asking me what would happen to their exam results now every student would have a laptop? My boss said that sounded like a PhD thesis. At the same time I was offered postgraduate study with the Sydney University Physics Education Research  (SUPER) group following my work with them developing an interactive Thomson's Experiment for HSC Physics as part of the Australian Multimedia for Physics Students (AMPS) program. Accordingly I applied to research the impact of the DER in CEO Sydney through SUPER.

Slide 6 - originally, the main phrase banded around was the National Secondary School Computer Fund (NSSCF) rather than the DER, looking to move to a 1:1 computer to student ratio. There were to be two rounds, 1 & 2, beginning in 2008 and 2009 respectively, operating for 4 years each. However, due to negotiations between DETNSW and the federal government, Round 2 was pushed back to later in 2009 and became Round 2.1.

Slide 7 - CEO Sydney got involved straight away, accessing Round 1 funding for half of its secondary schools and Round 2(.1) funding for the other half. Every year 9 student was provided with a Federally funded Macbook or HP laptop. The schools had to bite the bullet and find funds (not budgeted for) to provided the teachers with the same laptops. CEO Sydney had to foot the bill for wireless infrastructure and tech support (though the DETNSW negotiations may have retrospectively covered this). These devices were issued to every Year 9 student over 4 years to be used until the end of Year 12. As such, some will still be in use until 2015, though no new DER devices have been issued since 2012.

Two unique epochs occurred in 2009 and 2011 where, due to only half of the students receiving laptops in 2008, we had a situation where half of the School Certificate candidature in 2009 and again for the HSC candidature in 2011 had been schooled with 1:1 laptops and half without. These epochs represented unique, never to be repeated, dichotomous scenarios to compare students with laptops to those without. Unfortunately, I was unable to begin my research in 2008/2009 (due to not being Australian enough as a then Temporary Resident!). However, after Permanency (and subsequently  Citizenship) I started in July 2010, and thus was able to capture the 2010 data and importantly the 2011 dichotomous HSC epoch data.

Slide 8 - Visible Learning (Hattie, 2009). Most studies have historically compared classes with and without technology and not different ways of learning with technology. We are investigating both.

Slide 9 - Choosing the Wrong Drivers for Whole System Reform (Fullan, 2011). "Pedagogically vapid" is a great phrase. The full statement is important as it emphasises that student performance is about teaching and learning, not the tech. (Fullan was relatively technophobic up to 2011. However, from 2012, in 'Stratosphere', he sees the opportunities technology potentially offers to teaching and learning).

Slide 10 - Laptops for Teachers: Practices and Possibilities (Cowie et al, 2011) - a more positive quote alluding to the new opportunities potentially provided to teaching and learning with laptops.

Slide 11 - The 3 papers so far from DECNSW on DERNSW by Sarah Howard et al from the University of Wollongong. (I envy the time and hence efficiency with which Sarah can work - my 2010 data analyses are appearing in 2013...). Peter Goodyear who leads the USyd Sciences and Technologies of Learning (STL) network and invited me to speak at CoCo was instrumental in setting up the DERNSW evaluation program and is a mentor to Sarah Howard. 

Slide 12 - The 16 secondary schools I work with and surveyed in South-West Sydney and the Shire.

Slide 13 - School Profiles - quite a spread in SES and ESL, all non-selective.

Slide 14 - Response Rates - very nice :) It helps having a relationship with the schools to encourage participation. Anonymisation of data and an obsession with objectivity will ensure no bias (the results published so far demonstrate this, I want to tell an honest story not butter up my colleagues with faux research). 

Slide 15 - First paper published thus far, second one pending. Results of both discussed below.

Slide 16 - Some questions paraphrasing those in the first paper.

Slide 17 - Questionnaire Tool (using Google Doc Forms)

Slide 18 - Bubble Diagrams for 3 teachers in the same department - 2 very similar and well aligned to their students, one very different and quite misaligned.

Slide 19 - Misalignment Index discussed with results

Slide 20 - Graph of Misalignment Index versus average Likert response for all teachers (split as Round 1 (2 years' experience) and Round 2 (1 year's experience)). The spread and trend are very similar for both groups. Those with higher average Likert responses had lower MI i.e. teachers utilsing and expecting greater use of laptops were more highly aligned with their students i.e. the students follow suit.

Slide 21 - Graphs for Student v Teacher responses for the 4 questions. Greater alignment between teacher expectation of laptop use in class/homework and student reported use (bottom two graphs) than between teacher practices of bringing their laptop to school/class and student practices (top two graphs).

Slide 22 - Empirical Alignment Graph - along with the Misalignment Index, the Empirical Alignment Graph could be used by any researcher when comparing teacher responses/activities with student responses/activities. Do you agree?

Slide 23 - Second paper 'in press' - some of the questions discussed.

Slide 24 - Bloom's Digital Taxonomy (reproduced with permission from Andrew Churches - thanks Andrew!)

Slide 25 - Questionnaire Tool (again using Google Doc Forms)

Slide 26 - Frequencies of use for all applications for teachers and students - note the disparity for Simulations in particular.

Slide 27 - Frequencies of use for 3 most enjoyed - note how students out report teachers for blogging, video editing and podcasting (higher order activities)

Slide 28 - Frequencies of use for 3 used most often - word processing, internet research (googling) and electronic text book reported in that order by both teachers and students (usually low order activities)

Slide 29 - Summary of Findings - note, though few students experience simulations, those that do really enjoy them.

Slide 30 - Summary of Findings continued - note issues with BoS. However, in all of these results it is very important to note that this data is from 2010, only one or two years into using 1:1 laptops for these teachers. Three years on from that the classrooms should look quite different (some researchers will have to investigate :)

Slide 31 - Quote from Halverson and Smith (2009) - a great quote highlighting that in some classrooms/schools/systems technology can be manipulated to simply maintain the status quo.

Slide 32 - Current Research - overview of current analysis of 2011 HSC dichotomous data, directly relating to the student performance (dependent variable).

Slide 33 - Whetting people's appetites with the early results - we have significance and correlation, but not saying what :)

Slide 34 - References for this presentation, many more in the papers.

Slide 35 - Acknowledgements - have Ethics Approval form CEO Syd and USyd and greatly appreciate  the participation and openness from the teachers and students!

Slide 36 - love the quote attributed (though not definitely) to Tagore and the augmented reality pic from turkletom.

With the second paper to be published soon there will be far more detail for readers to cogitate if they so wish. Any dialogue/feedback would be greatly appreciated. 

For me, the highlight of presenting at CoCo was the discussion that ensued during the seminar (and has continued subsequently) with the excellent researchers, educators and student teachers that I met. Many thanks to Peter Goodyear for the invitation, Pati Paez and Maryam Khosronejad for the organising and Martin Parisio for the webcasting.

Monday, April 8, 2013

Deep Analytical Thinking

Recently I was asked to run a workshop on 'Deep Analytical Thinking'. This was essentially a session on Critical Thinking and Higher Order Thinking Skills. The primary remit was to get teachers thinking about what it is they are asking of their students, particularly in assessments. Are they giving students the opportunities and challenges of accessing higher order thinking skills?

Working within the New South Wales Board of Studies (but relevant everywhere) it is particularly important for teachers to know and understand the 'verbs' asked of students in the syllabus, exams and consequently, the assessments the teachers write themselves.

This workshop engaged the teachers in a deep analytical thinking task (whilst getting them to collaborate in a Google Doc); tested their knowledge of the verbs in a fun quiz format (demonstrating the power of Google Doc Forms and Flubaroo script); demonstrated the merit in using unique stimulus data e.g. from, thus ultimately allowing for open book/laptop, non-googlable  exams; and discussed Bloom's Taxonomy and Bloom's Digital Taxonomy.

Here is the Prezi, enjoy: