In their hangout of Week 1, the instructors of DALMOOC engage into the brief discussion about the purpose of Learning Analytics (LA) (sometimes around the 11th minute).
Caroline Rose argues that LA is different from data mining because research questions asked in LA are different from data mining RQ, and picking data for analysis is governed by these “other” questions. Dragan Gasevic refers to LA as helping us understand and optimise learning, and grounds LA as emerging from the new kinds of data, e.g. real-time vs. post-course surveys. He also strongly insists that LA is not only about prediction, but also a take on describing learning, and eventually providing a personalised learning experience. Ryan Baker agrees with them both, pointing that both LA and Educational Data Mining (EDM) aim to transform data to make it meaningful and making learning better. He also adds that LA attempts to bring the stakeholders into the process, i.e learners?, and communicates the findings to them. Finally, George Siemens wraps it up by saying that LA emerged from a new context filled with new data and new methods, and LA is about answering questions educationally, serving students, providing them guidance, but also dealing with broader societal issues, e.g. privacy and ethics.
There is a great edX-forum based exchange about it (not sure if links work without logging in, but one can find this thread on edX forum easily – it is the most active thread anyways). In the forum one of the participants wonders how LA is different from any other educational research, which ultimately “(at least in theory) is to understand the learning process in order to make it better”. Even if you disagree with the comment, like myself, it is worth continuing reading, as the discussion unfolds. Dragan gives a well-rounded answer expanding on his view on LA, and also mentions that LA allows to scale some of the previous qualitative research. That leads to other participants picking up on the tension between quantity and quality, both in relation to quantitative methods vs qualitative methods, as well as quantity of research vs. quality of research. Along the lines, a learner points that LA is different from traditional educational research because “sampling, cleaning data, normalizing and all these techniques are no longer applicable in this context”. Another learner wonders “how to test the claims made by learning analytics, e.g. looking at social network sizes as a predictor of student success in a course, or as a measure of how much a student is learning. How are these claims tested?” Ryan Baker disagrees “with the perspective that big data makes theory obsolete” because, in fact “many core proponents of the “all you need is data” perspective, actually use theory in their model-building, or at least some form of intuition”. Finally, another learner believes LA to be “highly related to the perspective of each individual, participating in that proces. In other words, the focal point of learning analytics is how each individual (e.g. learner, instructor, tutor, and administrator) interpretes and relates to the visualizations and feedback given as they engage in learning activities”. The discussion is quite interesting and full of nice metaphors and academic references.
However, in my opinion, the discussion avoids asking THE question: What is the research paradigm from which LA operates?
LA is a research field because its ambition is to generate knowledge. Research of education and learning is different from physics and math, and thus leads us back to the good old “how to do scientific research in social science?” But before I go en further, I need to say that the entire “methods” discussion is only secondary, and needs to be put aside.
Despite its links to computational methods which is a red flag to many educational researchers, the knowledge LA attempts to generate is situated in the social sciences, and there is no one size fits all solution there, no matter what. I think (and am probably making mistakes here), LA is post-positivist because although the scientific method is similar to a positivist paradigm, LA feeds back to theory, whether it is exploratory or based on hypothesis, and acknowledges the contexts within which observations are made and interpreted. It also does not do interpretivism, or at least if it does, I have not yet seen an example of that. But other than that, LA depends on the methods used and the paradigm of the theory to which it is feeding back.
So far, I have been labelling LA as a pragmatist paradigm (but my thoughts on this are inspired by these workshops on social philosophy, and I am only half way through with the entire course). In my understanding, pragmatism is concerned with research applicability that justifies drawing on multiple methodologies even though based on conflicting epistemologies. Pragmatist reasoning also demands that the problem and research inquiry are theory-embedded, as well as clearly positioned within the social, historical, political, disciplinary and other contexts. Simply put, pragmatist research is clear about the theoretical lens through which the data is selected for analysis, rather than tacks theory unto analysis of available data.
So, I think that LA is essentially pragmatist in its research orientation, because a) its definition includes applicability of research and ultimately problem solving; b) the role of theory in either generating hypothesis or interpretation has been emphasised multiple times. And by the way, I do not include in LA such decontextualised research, as for example, an oft-cited paper by Lytic Lab on learner sub-populations in scaled online courses.
p.s. good to have these in-progress thoughts out of my system, as I have been thinking about this for the last ten days…
p.p.s. And of course, we probably in for more discussions on the issues of theory-drive or data-driven research, as the tension between the paradigms in educational research is likely to become more prominent in the future. With my favorite thoughts on the issue by Simon Buckingham Shum and Amy Colier, as well as Chomsky vs Norvig debate.