With learning engineering at the core of the Tools Competition, the competition aims to support teams that harness the data their tool generates to support researchers in better understanding learning processes.
In Phase II proposals, competitors are asked to detail how their tool advances learning engineering to improve learner outcomes. Some competitors are also required to form a research partnership that can enhance research in the field.
This post shines light on the requirements for research partnerships and how and why organizers are looking to see all competitors meaningfully address external and team-led research practices.
Continue reading to find answers to the following questions:
- Why is there a research partnership component to the Tools Competition and how do research partnerships support learning engineering?
- What are organizers looking for, broadly?
- What is a research partnership?
- Which competitors are required to form a research partnership and how do I provide evidence of my partnership?
- What can a research partnership look like?
- How do I begin to form a research partnership?
- Where and how do I address the research partnership requirements in the proposal?
Why is there a research partnership component to the Tools Competition and how do research partnerships support learning engineering?
In order to best serve learners, we need to understand how they learn and what works – at scale and in context. With millions of learners around the globe using edtech platforms, and infinite data points about their interactions and outcomes, edtech has the potential to uncover insights and drive innovation in learning when tapped in with the research community; however, few edtech tools leverage the power of their data to support research.
The Tools Competition seeks to change this.
Learning engineering aims to build technologies that both support learning best practices – and help researchers refine those practices by studying how learners engage with and learn from technologies.
This represents a symbiotic relationship where edtech supports researchers in drawing insights about learning through the data generated – and researchers support edtech by driving continuous and evidence-based improvements to technologies.
As a competition, we are looking for tools that capture, leverage, and share robust data to better understand how, why, and what learners are learning. Competitive proposals will not just use data for their own benefit, but will also importantly contribute to knowledge in the field.
Research partnerships are critical to advancing the learning engineering objectives of the competition, facilitating rapid experimentation and continuous improvement, and allowing platforms to respond to real-world data on a tool’s effectiveness across contexts. This makes it possible to scale products and generate high quality data.
By making research partnerships a requirement for some competitors, the Tools Competition seeks to ensure that winning tools are set up for successful, scalable, and iterative research practices that advance learning engineering and improve the knowledge of learning.
What are organizers looking for, broadly?
The Tools Competitions aims to support tools and teams that believe in the importance of learning science research and are prepared to leverage the power of data to iterate on their tool and improve the field’s knowledge of learning.
Through this competition, we are growing a wider community of learning engineers (join a networking call to meet others in the network). Competitors are naturally at different phases. For some, this may be the first time you’ve heard of learning engineering, for others you may have a fully instrumented tool.
Broadly, your proposal should tell us about the potential of your tool to support research on a wide scale, and demonstrate your intention or action addressing that potential. This will look like addressing:
- Understanding: Are you aware of the data your tool generates?
- Infrastructure: How is data generated and shared?
- Quality: Is data high-quality and for robust learning measures?
- Demand: Are there researchers interested in this data?
- Engagement: How is/are researcher(s) engaged on your team?
- Scale: Is the data interesting to several researchers, beyond your team?
Intention is important here, rather than checking a box. We recognize that competitors join the competition at different phases and are looking for competitors that meaningfully think through these questions.
What is a research partnership?
In order for your tool to support learning science research you will need to talk to researchers and work with researchers. For that reason, we require many Tools competitors (see below) to form a partnership with one or more external researchers as part of their proposal.
A research partnership is an informal agreement between a competing Tools team and one or more external researchers at an institution such as a university or research lab, organization, company, or other entity.
Your partnership will demonstrate how your tool generates and shares data that is in demand from the field and how it can support research. What does this mean?
- Demonstrate demand for your data: partnering with external researchers that work with you and plan to study your tool’s data for their own research demonstrates interest from the field in the type of data your tool generates.
- Create a system to support research and share data: A system to support many researchers on an ongoing basis demonstrates your ability to contribute to the field at scale. This infrastructure or system can be simple (like responding to researcher requests to share data) or complex (like fully instrumenting your platform for researchers to run experiments within your tool).
Competitors will be at different stages, and the research partnership will help to advance these objectives. Research partnerships will look different depending on your tool and your phase of development (see examples below); but ultimately, research partnerships are mutually beneficial and collaborative.
To meet the research partnerships requirement, researchers must be external to the immediate organization that is receiving the funds, but they may work for the same institution (e.g., in another university department). If you have collaborated with an external researcher in the past, you may choose to work with them again.
It is crucial that researchers be external to your team, as this demonstrates interest and demand for your tool and data set in the wider research community, telling us that your tool has the potential to impact learning science research at scale.
Which competitors are required to form a research partnership and how do I provide evidence of my partnership?
Growth and Transform competitors in the 1) Accelerating Learning 2) Assessment 3) Teacher Development tracks as well as DARPA’s AI Tools for Adult Learning are required to:
- Either: describe a partnership with one or more external researchers
- Or: reference multiple external researchers who have expressed interest in the data and confirmed the ability of the tool to support research
Transform competitors in the Learning Science Research Track are required to:
- Either: describe a partnership with a digital learning platform with 10,000+ users
- Or: describe their access to a comparable data set that can be used for research
Competitors do not need official documentation or a formal commitment from an external researcher. You may reference the researchers by name who have expressed an interest in partnering on this proposal, and describe the nature of the partnership.
What can a research partnership look like?
Research partnerships will look different for every proposal, but can involve:
- Publishing/sharing data for external researchers to access (e.g., via GitHub)
- Instrumenting the platform to allow ongoing rapid experimentation and A/B testing (read more about instrumentation here)
- Identifying a researcher who plans to use your tools’ data in their research
- Developing an open access dashboard for researchers
- Engaging a researcher on your team to help you understand how your tool can better support research. This may include advising on: structuring or improving data and metrics for effective research; instrumenting your tool; identifying and testing interventions; validating your methodology, etc.
How do I begin to form a research partnership?
Connect with researchers!
Identify researchers that might be interested in the type of data your tool will collect, based on their content or technical area of expertise, and reach out to them. Send them an email describing your tool and project and asking if they would be open to working with you to achieve your research goals.
You may identify researchers that are working in a relevant space by reviewing proceedings from conferences such as Learning at Scale, Learning Analytics and Knowledge, and Educational Data Mining. We also recommend reaching out to junior faculty who may be more responsive, eager for such partnership opportunities, and who can dedicate the time to your partnership. Similarly, senior faculty members may be able to recommend team members with relevant expertise.
You can build costs for the research partnership into your budget. While this can vary depending on the scope of the project and time required of the partnership, +/- 10% of your total budget is a good place to start.
Our team has also compiled a list of researchers who have expressed interest in working with teams in the past. You can also reach out to us via email at toolscompetition@the-learning-agency.com or DARPATools@the-learning-agency.com with questions. Please note, there is no competitive advantage to working with a researcher from this list.
Where and how do I address the research partnership requirements in the proposal?
In the “Advancement of Learning Engineering” section of your proposal, include a description of:
- How you will work with researchers (and whom)
- How you will share learning data
- How your tool can/will support researchers at scale
To answer these questions effectively, you will likely also need to describe the following:
- The type of data your tool collects
- How your tool collects data
- How/where the data will be shared
- The role an external researcher plays on your team
- How your tool will be instrumented for research
Competitors do not need official documentation or a formal commitment from an external researcher. Instead, you may reference the researchers by name who have expressed an interest in partnering on this proposal, and describe the nature of the partnership.
