Assessing Socio-Cognitive Interactions in
Online and Face-to-Face Classroom Discussions

Context

This research proposal and pilot of a coding instrument is part of an assignment in a research methodology course in Language Education. The assignment mandate was rather broad. It called for research proposal on an issue in Language Education.

The Study

In keeping with the spirit of the assignment, I chose to look at how language is used in Face-to-Face (F2F) and online classrooms. With this in mind, I looked at Garrison, Anderson, & Archer's (2001) Practical Inquiry Model to code levels of socio-cognitive interactions in online discussions as an indication of knowledge building. I then piloted their coding scheme with both online and F2F classroom interactions.

Pilot Results

The purpose piloting the coding scheme was to find out if there were any major issues involved in using it prior to progressing with a the full-scale study. Many issues arose in this round of coding, many of which could explain the low inter-rater reliability.

In a debriefing session with the raters, different assumptions about the coding scheme were revealed. These differences can account for the three largest areas of disagreement: types of questions, definitions of brainstorming and information exchange, definition between brainstorming and leaps to conclusion. Addressing these three issues could raise percent agreement more than 35%.

While 35% is still very low, rater training and clarification of code definitions could raise the level of rater agreement a good deal. This means that the jury is still out on the applicability of the Practical Inquiry Model to the analysis of face-to-face transcripts.