Introduction
In the evolving landscape of educational tools, providing teachers with comprehensive insights into student performance is crucial. The first phase of the Knowledge Component (KC) dashboard offered a snapshot of student mastery levels. Now, Phase 2 delves deeper, answering the critical question: "What exactly did students do wrong?"
Building the Attempt History
The enhanced dashboard features a detailed attempt history table, giving teachers a chronological view of each student's attempts. This table includes problem questions, student answers, correctness, timestamps, involved Knowledge Components, and mistake codes when applicable. The backend supports this with a teacher-exclusive endpoint using OFFSET pagination.
The Design Decision
A significant challenge was incorporating MistakeEvent data with Attempt records. To address this, we evaluated two strategies:
-
Option A: LEFT JOIN with GROUP BY
- Simple but risky, as potential bugs could duplicate data without errors.
-
Option B: Scalar Correlated Subquery
- Chosen for its reliability, the subquery ensures one result per attempt, maintaining data integrity even if schema changes occur.
Although slower on large datasets, this approach suffices at current volumes, with future plans for cursor-based pagination.
Importance for Research
The MacLellan framework emphasizes actionable insights for teachers, aligning with the Teacher-in-the-Loop principle. Phase 2 empowers teachers with factual attempt records, allowing them to verify and react to classifier judgments. This transparency is vital for our RCT design, as it examines teacher interventions based on detailed student data rather than model accuracy.
Lessons Learned
OFFSET pagination, while not perfect, is adequate for our current needs. We’ve documented an upgrade path to cursor-based pagination for future scalability. Unlike Phase 1's dual-role endpoint, this teacher-only route simplifies access control, requiring only a straightforward role check.
Future Directions
Phase 3 will introduce an LLM-generated insight feature, leveraging attempt history to provide teachers with comprehensive summaries. The groundwork laid by Phase 2 is essential for this upcoming functionality.
Key Takeaways
- Scalar Correlated Subquery Advantage: Ensures one-to-one data reliability even with schema changes, unlike LEFT JOIN.
- Evidence Before Insight: Providing raw data first allows teachers to interpret before AI-generated insights.
- OFFSET Pagination Limitation: Documented upgrade paths prevent premature optimization.
These enhancements position the dashboard to better support teachers in understanding and addressing student challenges.