One of the fruitful outcomes of my postdoc is the sharpening of my academic interest towards peer effects and complex networks in learning settings. As I find myself working on several empirical projects, I thought it may be useful to pull the threads together.
First, the larger question I am curious about is what learning outcomes results from peer interactions. That is, what is it that changes that we can reasonably capture and claim that it was caused by the peer a learner interacted in with an online environment.
1.Peer Effects in Digital Learning Contexts
First set of studies I conducted here examined the effects of peer interactions on grades across the university trajectory. The original intention was to look at the changes in discourse, particularly in relation to subgroups of high and low performers (something that is only now beginning to unfold), but I discovered that prior to that, I needed to work through some foundational analyses first.
Turns out that peer effects on performance (as the most coarse-level learning proxy) have not been examined. Everything done so far is either not methodologically solid or not done in an online environment. Good news was that across two institutional datasets (and the third institutional dataset on the way), we found peer effects on performance as well as social selection processes for both high and low performers. These dynamics though appear to differ greatly across the contexts.
In light of this, I am working on several empirical studies examining peer effects on various learning behaviours captured through discourse and logs to understand when the change occurs due to the peer, and how it can be theoretically explained.
Besides the empirical work, I have been sharpening the argument here around the actual meaning of peer effects and learning behaviour change through relational ties in digital learning environments. It may not be obvious but this actually links to the second strand of work (below) on measurements in networks that represent learning processes.
2. Modelling and Evaluating Social Learning in Networks: Generative Processes, Measures, Null Models, Social Positioning
I continuously struggle to make decision as to how networks should be constructed to capture social learning meaningfully, and then how to ensure if what you observe is not a random process but beyond expected. For the upcoming LAK conference I worked on three parallel projects related to this question.
a) With my collaborators from CRI I looked at comparing randomly simulated networks with the observed forum dynamics. We show that when posting activity is treated as the most primitive generative process, learner degree and edge weight is largely an outcome of individual learning activity, not social effects. We give some examples of the metrics that appear to be more meaningful for the capture of social processes. Bottom line finding: interactions are largely random, social effects are more captured by local cluster-related metrics in post-reply networks, one absolutely needs to control for posting activity at the individual level when modelling forum communication. If this is useful, please reach out, I will share the paper that is currently under review.
b) With another collaborator, we hypothesized that individual presence in forum communication can be split into two interdependent processes: participating indirectly in groups and participating directly with individuals. We showed how thread-co-occurrence and post-reply behaviour are incongruent for more than 1/3 of all posters, but they are temporally inderdependent. Post-reply behaviour is driven by indirect co-occurence, especially in the beginning of the course. This suggests that even though the group-learning processes are random (or driven by the relevance of content), interpersonal learning processes result from social dynamics. A couple of things lined up here for the follow-up around the social roles, which I am teasing out with Nia Dowell’s group communication analysis.
c) Clearly temporal dynamics matters greatly in forum communication, so in this third study, we modelled communication without any demographics or discourse indicators, as a relational event model. The study is rather simple but significant in that it shows that ‘what looks’ like social effects, are the results of accumulated post-reply behaviour. This is much similar finding to the one in the set a) but done through a completely different methodology, on a completely different set of courses, in a completely different pedagogical setting. Also, here since the modelling is at the posting level granularity temporally, the findings actually suggest some of the mechanisms behind how these patterns aggregate into looking as social effects. Another reason why they are not ‘real’ social effects is that our models fail to explain closure – as shown from the confusion network plot, although we do control for structural closure in various ways. This is very interesting as it aligns with the other studies described here – we know very little as to why social effects appear and what mechanisms propel them, especially in digital settings.
d) Not much to be said here besides that I am working on a simple agent-based model to pull the insights from those three studies together into a more theoretical view on the generative processes of forum communication networks in online settings.
3. Applications in Educational Settings
As I am wrapping up the empirical work, there are several clear applications of the foundational research described above.
One – peer matching to help with peer effects exposure.
Another – evaluation of learning design in relation to various network structures.
Third – network experimentation.
I have different small-scale pilots around these interventions in the pipeline, but to be updated once something can be said about the impact.