麻豆原创

In the recently published chapter in the book  (),  and  outline the importance of using data that are increasingly available to guide decision-making in education institutions, ranging from the federal and state policies at the system level to pedagogical and instructional decisions in schools and classrooms.

Distinction between Educational Data Mining (EDM), Learning Analytics (LA) and Academic Analytics (AcAn)

Educational Data Mining (EDM) applies data mining techniques to data about the determinants of the learning process (e.g., data collected from the knowledge management systems of online learning) to understand the learning patterns and digital trails. Learning Analytics (LA) focuses on using data mining models and other advanced techniques to explore the determinants of student achievement, the output of the learning process, to better inform instructional decisions. Academic Analytics (AcAn) uses statistical analysis and predictive modeling to understand the organizational-level determinants with the purpose of providing guidance for principals and other school leaders on decisions related to the operations and managerial processes of institutions. The main differences of these three approaches are summarized in Table 1.

Table 1. Main differences between educational data mining, learning and academic analytics 鈥� a classification

Source: Authors鈥� elaborations, originally inspired by Romero & Ventura (2010) and Siemens & Long (2011).

Agasisti and Bowers point out that the classification of these three approaches must 鈥渂e intended as provisional, indicative and not prescriptive鈥�, and may need to be revised as the literature evolves. In addition, the three approaches are difficult to separate in practice as they share the analytics techniques, research questions, policy implications and the 鈥渓oop of data鈥� that describes the process of using quantitative data to guide decision-making.

As shown in Figure 1, the first step 鈥淐ollection and acquisition鈥� involves identifying and collecting the relevant datasets to use. Then datasets constructed within institutions (鈥淪torage鈥�) and are cleaned through reconstruction and wrangling (鈥淐leaning鈥�). Multiple datasets may need to be integrated together in order to explore the research problems from a more comprehensive perspective (鈥淚ntegration鈥�). The next step is to apply the statistical, econometric and data mining techniques to the datasets (鈥淎nalysis鈥�). The patterns and results identified from the analysis are then synthesized and visualized in order to facilitate the transition to informing knowledge, such as through policy-makers, school and district leaders and teachers (鈥淩epresentation and Visualization鈥�). Lastly, the results of the analysis can be used to inform the instructional and managerial decisions (鈥淎ction鈥�). The 鈥渓oop of data鈥� only illustrates a typical process of data use. Note that in practice it can be iterative and recursive.

Agasisti and Bowers recommend that an education data scientist can play a key role to bridge the data analysts and educational practitioners in the 鈥渓oop of data鈥�, as the education data scientist 鈥渙wns the technical skills to collect, analyze and use quantitative data, and at the same time the managerial and communication skills to interact with decision-makers and managers at the school level to individuate good ways of using the information in the practical way of improving practices and initiatives鈥�.

Data and tools of data analysis (and analytics) in educational arena

Agasisti and Bowers introduce multiple types of data analytics and tools that are used in the educational arena. First, at the policy level for implementing, managing and evaluating policy interventions, studies based on large-scale international assessment data on student achievement, such as PISA and TIMSS, are particularly informative, as they allow researchers to explore which practices and policies are working in different countries holding other factors constant. There is also a growing number of rigorous 鈥渇ield experiments鈥� that aim to provide evidence on 鈥渨hat works鈥� to improve student learning and school performance. Second, at the level of school practice for principals to improve school operations and for teachers to improve teaching effectiveness, the focus of research is on data-driven decision-making in K-12 schools. Data on student achievement, school administration and digital learning environments are collected and analyzed to help teachers and principals identify the problems and understand the patterns. Research on data use has shown that teacher鈥檚 and leader鈥檚 data literacy and assessment literacy determines whether they can successfully use data to inform their decision-making.

To illustrate how data analyses and analytics can improve student performance, the authors describe four examples. First, for the analyses of system-level determinants of instructional results,  , , and  are widely used to assess the effectiveness of national educational systems. Second, the  and best practices among British Schools are demonstrated as good practices on how to use school-level information to help school leaders to understand students鈥� learning patterns. Third, the tool  developed by Purdue University shows how to use course-specific data to provide timely feedback to teachers for instructional improvement. Fourth,  is one of the interfaces that utilize learning analytics to manage individual student data.

Barriers and impediments to the use of analytics in education

Agasisti and Bowers summarize four barriers that impede the use of data analytics in education and propose potential solutions to lower these barriers. The first concern is the potential threat to student privacy, as many tools are built upon tracking student information. The authors argue that 鈥渙pen code and open access standards must be used when data analytic or machine learning algorithms are used to inform evidence-based improvement cycles in schools, or to the extent that Learning Analytics algorithms make recommendations for content and instruction for student learning鈥�. Second, to address the complexity of data, researchers are recommending using data warehouses for data reporting and analytics. Third, the creation of an adequate platform for data analysis can be costly. However, open access code publication will facilitate the sharing of code and reduce the cost to build such platforms. Fourth, it is challenging to develop methodologies that 鈥減resent the results without excessive simplification (providing an awareness of the complexities of the learning process) but with enough clarity to make the information understandable, and thus usable鈥�.

A way forward: systemic change and the role of education data scientists

To conclude, the authors recommend that education data scientists as a new profession would play a key role to improve data use in schools and higher education. Education data scientists are expected to 鈥渇acilitate the communication between three worlds: (i) one of technical experts in data analyses and analytics, (ii) that of decision-makers at various levels (policy analysts, school principals, institutions鈥� mangers) and (iii) the community of teachers, engaged in frontline instruction鈥�. The authors call for more commitment and resources to train this new expertise as there is currently a lack of training and capacity building in this field.

Agasisti, T., Bowers, A.J. (2017) Data Analytics and Decision-Making in Education: Towards the Educational Data Scientist as a Key Actor in Schools and Higher Education Institutions. In Johnes, G., Johnes, J., Agasisti, T., L贸pez-Torres, L. (Eds.) Handbook of Contemporary Education Economics (p.184-210). Cheltenham, UK: Edward Elgar Publishing. ISBN: 978-1-78536-906-3 

Open Access Version: