Digital Badge Outcomes for STEM Learning in Formal and Informal Contexts: A Comparative Analysis
Digitally-mediated, knowledge-based economies have demanded new knowledge, skill sets and proficiencies for global workers. For a variety of reasons, essential skills required in science, technology, engineering and math (STEM) fields such as critical thinking, modeling, and scientific reasoning are neither adequately taught nor assessed in formal learning contexts. This has resulted in skills deficits with widespread repercussions: “This human capital performance gap threatens our nation’s ability to compete in today’s fast-moving and increasingly demanding global economy. It is emerging as our nation’s most critical business issue” (Deloitte Consulting LLP, 2005, p. 2). The use of digital badges for scaffolding, assessing and communicating STEM learning in connected learning contexts is one possible solution for digital age learning and assessment.
This research explores the impact of digital badge interventions for STEM knowledge acquisition in middle school populations. It will contribute to the emergent body of knowledge of innovative assessments by providing empirical evidence to inform theory and practice for educators in K-12 and higher education contexts. Although there has been significant interest in digital badging systems, to date, only one published research study exists (Abramovich, Shunn & Higashi, 2013) to informing future development and implementation.
The purpose of this study is to understand the roles and potential impact (if any) of digital badge interventions to motivate and scaffold student learning in formal and informal connected learning contexts. Possible learning environments include classrooms, after-school programs, museums, libraries and other informal contexts. The digital badge schemata for the study are designed to promote skills acquisition to proficiency in specific STEM practices, such as analyzing and interpreting data as well as design thinking, information production and communication, which require higher order thinking skills.
An explanatory research design is proposed, implemented as a quantitative study. Specifically, the study design requires the collection of quantitative data from student and teacher participants through pre and post treatment surveys. In addition achievement data, the dependent variables, will be collected from the digital badging system (number of badges attempted and awarded) as well as supplemental data from online contexts such as the social learning platform, Makewaves.
- How does the use of digital badge schemata to support, assess and recognize learning impact student:
- affect (liking for STEM learning);
- learning strategies;
- skills acquisition (Badges earned/attempted).
2. Which (statistically significant) factors of the learning environment affect digital badge acquisition?
Research Design and Methodology
This proposed research follows a sequential explanatory design. Quantitative data will be collected from participant pre and post treatment surveys. The survey instruments will be administered via SurveyGizmo online software. The Connected Learning Model is proposed as the conceptual framework because of its emphases on digital media, peer collaboration as well as participatory learning; these key attributes align the model with social constructivist tenets of learning through discourse with others in socio-cultural contexts. Additionally, the study will also be informed by research in learning theories, assessment, motivation /interest learning, technology-mediated social interactions and instructional design.
Digital badge schemata will be designed which scaffold and assess learning in STEM practices as articulated by the National Academy of Sciences in “A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas” (National Research Council, 2012). The badge designs, are organized into 4 or 5 assessment levels. Performance assessment tasks will be organized by cognitive difficulty loosely aligned with the Revised Bloom’s Taxonomy of objectives. Digital badges will be awarded to participants who demonstrate proficiency of the targeted learning objectives by successful completion of performance tasks.
The pre and post survey instruments will comprise of a variety of questions, some of which already exist in reliable, validated instruments which have been implemented in the target populations in grades 6 – 12 and/or teachers. Items from scales such as the Intrinsic Motivation Inventory (IMI) will be used to assess motivation, self-efficacy and persistence at task. Survey items regarding use of digital media and ICT use will be modeled upon existing surveys such as the “Speak Up” questionnaire or Pew Internet American Life studies. Questions regarding the classroom environment will be modeled upon the What is Happening in the Classroom and similar instruments. Students will be surveyed on preferred learning styles, digital media use, motivations and attitudes towards STEM subjects. Post-test surveys will focus upon learning behaviors promoted by the digital badges, motivation, and affect. Teachers and program leaders or school administrators will also be surveyed to identify attitudes, dispositions and behaviors regarding the concept of digital badges which could possibly impact program implementation.
The units of analysis for the study are 1) individuals and 2) groups of individuals interacting in various learning contexts. It is anticipated that digital badge programs will be implemented for 2-4 weeks. The anticipated study length is 8-10 weeks.
to understand which variables impact learner outcomes and to what degree;
- to construct profiles of learners correlated by extent and motivational aspect;
- to understand the extent to which badges are motivating to elicit the desired learning behaviors, increase effort, and impact the degree of persistence at tasks in members of the target populations;
- to understand links between formal and informal learning using badge schemata to assess, recognize and communicate achievement in participatory cultures and social contexts.
This research is unique for several reasons: its purpose to measure learning in STEM skills along learning paths aligned with badge schema, the measurement of informal learning through recognized theoretical frameworks, the K-12 context and the expected outcomes. It will significantly add to the growing body of knowledge of teaching and assessment of skill sets and proficiencies essential for the knowledge-based workplace. As such, there are myriad possible topics for future research including: linking students’ level of academic success in formal learning environments and informal environments, how can K-12 leaders leverage badge systems for assessment? How can value be added to the badges to potentially interested audiences (the higher education and business communities primarily)? How do badges work to motivate students? Can badge system work in academic environments to assess core knowledge? Will a badge system of assessment motivate learners who struggle in formal contexts? How does learning context affect student learning and badge acquisition (for example, through educational games, the classroom, informal environments such as after school programs)?
Committee on Conceptual Framework for the New K-12 Science Standards; National Research Council. (2012). A framework for K-12 science: Practices, crosscutting concepts, and core ideas. Washington, D.C.: National Academies Press. Retrieved from http://www.nap.edu/catalog.php?record_id=13165
Deloitte Consulting LLP. (2005). Skills gap report: A survey of the American manufacturing workforce. National Association of Manufacturers: Washington, DC.
Comments, questions or suggestions?