February 19, 2020 Delivery WG Call Notes

Attendees

Zac, Jeff, Chris, Nick, Brian, Sarah

Agenda

• Welcome and introduce new WG members
• WG charter discussion
• Review and update task list
• Discuss how to interface with the Coordination WG
• Develop instructors/auditors evaluation instrument

Call Notes

Welcome and introduce new WG members

• Sarah Matthews (EPA)
• Nick Kilby (AAPCA)
• Brian Foster (TCEQ/AAPCA)

WG charter discussion

• Elements
  • Leader
  • Reporting/Meeting Schedules
  • Purpose & Objectives
  • Member Expectations
  • Scope
  • Operating Guidelines
  • Expected Outcomes
  • Decision making process

Review and update task list

• Develop/maintain list of instructors by subject categories and course topics.
• Develop and implement a process for recruitment of new instructors.
• Develop and implement use of improved tools for evaluation of instructors and course materials.
• Oversee interactions with instructors to address needed improvements to instructional materials and to provide other course support.
• Develop and implement delivery format alternatives for all courses including classroom and e-learning/distance learning.
• Provide updates to instructors regarding course library additions and updates.
• Provide information to instructors on use of LMS functionalities and enhancements to aid them in fulfilling their instructor roles for downloading and uploading information to the LMS.

Discuss how to interface with the Coordination WG

Develop instructors/auditors evaluation instrument

• Course summary reports from instructors historically had been collected from EPA
• WESTAR is creating these (so are some other MJOs) reports, but don’t know how they’re getting back to EPA
• Summary reports have space for instructors and course auditors to provide feedback
• One challenge is to identify which materials are being taught and how they compare to what’s in the national LMS
• Think about survey tools, and how to match a training contractor with the materials that they taught from; this should be a functionality in the LMS
• Example with air dispersion modeling: EPA course, Lakes, Trinity, and Providence ??; some overlap in materials, but a lot of propriety too, could cause confusion in associating evaluation with the materials
• Amy hasn’t experienced great surveys in LMSs, star ratings usually; whatever survey you create does it need to be in there, or could it be triggered by the LMS; send link to students; may be a need to have an external software, and then somehow link the results back to the LMS and associate with the training materials/course/instructor
• Preferable to have the eval in the LMS, but not sure how feasible
• Would there need to be separate instruments from the instructor and auditor?
  • Are the questions different between instructor and expert auditor
  • Both instruments could have some common questions, but auditor would provide feedback on delivery and materials; instructor on materials and training logistics/course
  • Amy thinks they should be separate instruments
  • Jeff questions whether it’s generally the case that there is an auditor in the course, probably not;
• Start with some information that we would like to know about; target a single instrument to keep it simple
• What information do we need to know
  • New information that’s not in the class; what’s dated
  • Errors/inaccuracies/typos in the materials
  • Try to get a template/format for the instructors to provide feedback into; need to prompt for specific questions

Next Call

• Wednesday March 18, 2020 @ 11:00 Central