Research and Data Practices

last updated 2/28/2020

Improving Student Learning

Improving student learning is the foundation of Lumen Learning’s mission to “enable unprecedented learning for all students” and underlies our commitment to research in teaching and learning. Lumen Learning is particularly interested in generating insights and providing evidence for practices that reduce inequalities in access to education and students’ ability to succeed.

Commitment to Research

Lumen Learning is deeply commitment to research – both to leveraging and extending evidence-based practices already documented in the literature and to discovering and sharing new insights about learning and how to effectively support it.

The foundation of Lumen Learning’s research and data practices is the Asilomar Convention for Learning Research in Higher Education, which asserts the following tenets for learning research:

  1. We are committed to advancing the science of learning for the improvement of higher education.

“The science of learning can improve higher education and should proceed through open, participatory, and transparent processes of data collection and analysis that provide empirical evidence for knowledge claims.” (Asilomar Convention)

  1. We are committed to sharing data, findings and technologies within the learning research community in order to extend the project’s contributions to learning science.

Maximizing the benefits of learning research requires the sharing of data, discovery, and technology among a community of researchers and educational organizations committed, and accountable to, principles of ethical inquiry held in common.” (Asilomar Convention)

Lumen Learning seeks to advance learning science by discovering and sharing insights about learning and how to improve learning using data collected through courseware as well as related learner data from institutions. We will adhere to transparent, responsible and ethical practices around data ownership, sharing and use.  Lumen Learning is also committed to compliance with institutional, state and federal policies regarding appropriate handling and use of learner data.

Consent for Use of Data

In order to function fully and effectively, Lumen Learning’s courseware captures and uses a variety of instructor and learner data within the system for which no explicit permission is required beyond using the courseware. The courseware uses learners’ own data to help them understand their learning process and progress. The courseware shares learner data with instructors to provide visibility into student behaviors and learning so that instructors can facilitate more effective learning. The courseware also uses instructors’ own data to help them track their efforts to support students in their learning. Lumen Learning uses these data to evaluate and improve the effectiveness of its courseware in supporting student learning.

We seek consent from students and instructors to use their learning data for research purposes, following best practices established by Carnegie Mellon University’s Open Learning Initiative (OLI). Implemented with process oversight from Carnegie Mellon University’s Institutional Review Board (IRB), this approach uses an opt-in/opt-out form to confirm user consent for authorized researchers and research communities to use their de-identified data in research studies. Students may opt in or opt out repeatedly, allowing them to change their minds about participation at any point. 

Data Collection

Lumen Learning’s courseware captures comprehensive data that is meaningfully contextualized with semantic markup. These measures strengthen data quality, analytical capability, and searchability. Contextualized data allows us to conduct a full spectrum of analyses and discovery in support of student learning. It future-proofs our work by providing the means to explore new questions or develop new data models as our understanding of the courseware and learning science evolves. Lumen’s Waymaker courseware captures and uses the personally identifiable information (PII) elements of name and email address in order to facilitate a variety of functions within the courseware (e.g. personalized teaching interactions). 

Uses of Data, Data Models, and Analysis

Lumen Learning seeks to identify the most important feedback loops around improving student learning, including the use of data to encourage metacognition and to expose and reinforce study behaviors that lead to increased learning. Data capture, research and analysis focus on:

  • Feedback loops for students
  • Feedback loops for faculty
  • Feedback loops for course design

Current learning science informs our initial hypotheses about these feedback loops. Over time our hypotheses and research strategies evolve along with the courseware and our understanding of its impact on learning. Data analyses drive continuous iterative improvements, accompanied by success measurements to gauge efficacy.

Lumen Learning researchers compile research questions to drive courseware instrumentation for data collection, research methodologies, and data model development. To address these questions, we apply a variety of analytical techniques to better understand and improve student learning including techniques such as:

  • Learning analytics: Analytics that validate learning has taken place
  • Engagement analytics: Analytics documenting measurable activities such as levels of interaction with content, what happens in the classroom, personal interaction, etc.
  • Progression analytics:  Analytics that gauge movement through a course and/or an education program over time
  • Courseware analytics: Analytics that establish how well courseware is supporting student learning

Wherever possible and appropriate, courseware design and research data models support variability and divergent pathways for students to achieve success, rather than “one size fits all.” We employ multiple data models such as:

  • Cognitive models: How are students learning effectively?
  • Adaptation models: What approaches and practices will better support the learner? The instructor? Courseware efficacy?
  • Assessment models: What types of assessments are most effective at demonstrating mastery of learning objectives?
  • Iterative improvement models: What is most effective in facilitating continuous improvement in the courseware?

We are committed to making research findings publicly available in order to broaden the impact of our work on education and learning science, while maintaining privacy protections for learners.

Security, Risk and Liability

Lumen Learning and its partners employ best practices around information security to ensure courseware, integrations, and personally identifiable data remain secure. These practices impact courseware architecture and functionality as well as the behaviors of learners, instructors, institutional staff, and the courseware provider.

Data Architecture, Systems and Technologies

Lumen courseware uses one or more open data stores, like DataShop at the Pittsburgh Science of Learning Center, to make de-identified data available to authorized learning research communities for the purposes of broadening our impact on education generally. We reuse and borrow any tools and assets that can reasonably be reused or borrowed.

Continuous Consideration of Research and Data Practices

We review our strategies, policies, and practices around data and research at least annually to assure that they align with recommended standards among researchers, learners and educational institutions.

Collaborative Research with Institutions

From time to time Lumen Learning collaborates on research with partner institutions. As part of these collaborations we may collect and analyze additional data in partnership with the institution(s). These data are handled according to the principles and processes described above.

Additional Questions?

Additional questions about Lumen Learning’s research and data practices can be directed to Dr. David Wiley, Lumen Learning’s Chief Academic Officer, at