Last updated: November 2024
With learning engineering at the core of the Tools Competition, a critical component of the Phase II proposal is a learning engineering plan that demonstrates your commitment to rapid-cycle continuous improvement and supporting research at scale.
Learning engineering is a growing field built on partnership between technologists, researchers, and educators to use big data in order to: better understand the learning process, develop more effective interventions, and drive evidence-based product innovation.
In Phase II proposals, competitors are asked to develop a plan detailing how their tool advances learning engineering.
What are organizers looking for in a learning engineering plan?
As a competition, we are looking for tools that capture, analyze, and share robust learning data not just for their own benefit, but also to contribute knowledge to the field at-large.
Your proposal should tell us about the potential of your tool to support research and the steps you will take to leverage your tool and data to do this.
We recognize that competitors join the competition at different phases—for some, this may be the first time you’ve heard of learning engineering, for others you may have a fully instrumented tool. Your learning engineering plan will look different depending on your tool and phase of development.
What should be included in a learning engineering plan?
A learning engineering plan demonstrates how your team follows (or will follow) the learning engineering process (see more here). This will look different for every proposal based on your team, phase of development, and tool.
There are three core components to a learning engineering plan. All three are crucial to a comprehensive plan. You should detail:
- The learning data your tool captures. Learning data can help answer important questions about learning.
- How your team uses data to drive rapid-cycle continuous improvement. Data should be used for real-time or ongoing enhancements to the tool and user experience.
- How you will engage external researchers to support research at scale. Instrumenting your tool for research scales the impact of your tool to the field at large.
For competitors at the Catalyst level with tools in the very early stages, you should speak to how this will be reflected in the development of your tool.
What do you mean by learning data?
Learning data provides rich insights on the learning process. Rather than simply the number of users accessing your platform and their general activity, learning data supports researchers in answering critical questions about learning. This may include data around learner performance, behavior, attitudes, cognitive processes, or other usage and demographic data that can help researchers understand what is working, for which learners, under what conditions.
How do I engage with external researchers to support research at scale?
Engaging with external researchers is a fundamental piece of your learning engineering plan.
This indicates that teams are not only utilizing data from their tools for continuous improvement but also that there’s interest from researchers in using the data and insights your tool captures to respond to important research questions.
You should consider your unique phase of development and goals when determining how to engage with researchers. Some examples include:
- Identifying researchers who will use your tool’s data in their research and describing the research it will support.
- Publishing/sharing data for external researchers to access. Some popular examples include Kaggle, Hugging Face, or GitHub. Repositories like Datashop or the Learning Engineering Hub are education-specific, but may not have the same reach as the popular sites. View this resource for more.
- Instrumenting the platform to allow ongoing rapid experimentation and A/B testing. Read more about instrumentation here.
- Developing an open access dashboard for researchers to utilize.
- Working with a researcher to structure your data for usage for research at scale.
- Engaging a researcher to help you understand how your tool can better support research. This may include advising on: structuring or improving data and metrics for effective research; instrumenting your tool; identifying and testing interventions; validating your methodology, etc.
- Naming researchers who can attest to your tool’s potential to address important research questions and detailing how they have or will study your tool or data.
Researchers you support as part of your plan must be external to your immediate organization or team. This demonstrates that there is interest and demand for your tool and dataset in the wider research community.
Finally, because Learning Engineering is a process of continuous learning and improvement, the strongest plans engage researchers on an ongoing basis.
How can I find and engage researchers?
Get in touch! Identify researchers that might be interested in the type of data your tool will collect—based on their content or technical area of expertise—and conduct outreach.
A great place to start is to identify academic departments focusing on relevant learning outcomes or methodologies. These do not have to be local to your area. You may also find it helpful to refer to conference proceedings (like this one or this one), academic papers, or other major edtech programs for those working in relevant areas.
Send an email describing your tool and project, and inquire about their research objectives and whether they would be open to collaborating. Align together on your goals. Build any costs associated with the research partnership into your proposal budget.
If the researcher you contact is unavailable, they may be able to recommend other colleagues or PhD students within their department.
Finally, our team also maintains a database of researchers with diverse expertise. Let us know if you need support or post a note in the Learning Engineering Google Group.
Where do I address the learning engineering plan requirements in the proposal?
In the Learning Engineering section of the proposal, competitors are asked to detail their Learning Engineering plan. You should include how their proposed tool will support research, the names of any researchers being engaged, how you will work together or support their research, and the nature of the partnership. As mentioned, you are welcome to include costs for developing a research partnership in your proposal budget.
No formal agreement or official documentation is necessary, as plans and structure for each partnership can vary widely. You are welcome to upload a letter of agreement or other evidence, but this is not required.
