November 20, 2019 Delivery WG Call Notes
Contents
Action Items
- Zac to send Chris new evaluation instrument link
 
Attendees
Zac, Jeff, Amy, Chris
Agenda
- Discuss EPA training delivery webinar
 - Review training evaluation and develop approach for integration into APTI-Learn/new LMS
 - Update on instructor list, and interface with planning and support WG on recruitment
 - Develop instructors/auditors evaluation instrument
 
Call Notes
Discuss EPA training delivery webinar
- EPA contractor developed a report that describes the delivery methods; Chris will send document and ask for comments; send comments back to Chris in an email rather than in document
 - Contractor will give webinar on 12/16 that will summarize the document
 - Next step will be a decision tool, what will that be? Probably web-based but don’t yet have details
 - Borrowing Amy’s charter concept to provide users with a plan to carry out the different training delivery methods for their course
 - It’s important to have a common language that also reflects the vernacular being used in the greater training community
 
Review training evaluation and develop approach for integration into APTI-Learn/new LMS
- Evaluation instrument developed by previous JTC WG's, piloted over course of last year or so
 - Zac to send Chris new evaluation instrument link; Chris to check with EPA on getting the evaluation put into APTI-Learn
 
Update on instructor list, and interface with planning and support WG on recruitment
- Jeff has been maintaining the list; contacted people on the list to see if they’re still teaching; about 40 or so people
 - There are other lists, e.g., LADCO approved contractor list, that Jeff has compared and tried to pare down and make consistent
 - Worked through some of the lists and removed groups/people who don’t make sense to the community;
 - Will starting working on these lists again
 - How is it being communicated? Next step is to ask others in the community about their experience with different people on the training list
 - Jeff also sits on the LMS workgroup and can provide some interaction there
 - Ask curriculum WG what they need in terms of delivery regarding course development
 - When a course development process starts, walk through the tool with the development team
 - How to best used different delivery methods
 
Develop instructors/auditors evaluation instrument
- Course summary reports from instructors historically had been collected from EPA
 - WESTAR is creating these (so are some other MJOs) reports, but don’t know how they’re getting back to EPA
 - Summary reports have space for instructors and course auditors to provide feedback
 - One challenge is to identify which materials are being taught and how they compare to what’s in the national LMS
 - Think about survey tools, and how to match a training contractor with the materials that they taught from; this should be a functionality in the LMS
 - Example with air dispersion modeling: EPA course, Lakes, Trinity, and Providence ??; some overlap in materials, but a lot of propriety too, could cause confusion in associating evaluation with the materials
 - Amy hasn’t experienced great surveys in LMSs, star ratings usually; whatever survey you create does it need to be in there, or could it be triggered by the LMS; send link to students; may be a need to have an external software, and then somehow link the results back to the LMS and associate with the training materials/course/instructor
 - Preferable to have the eval in the LMS, but not sure how feasible
 - Would there need to be separate instruments from the instructor and auditor?
- Are the questions different between instructor and expert auditor
 - Both instruments could have some common questions, but auditor would provide feedback on delivery and materials; instructor on materials and training logistics/course
 - Amy thinks they should be separate instruments
 - Jeff questions whether it’s generally the case that there is an auditor in the course, probably not;
 
 - Start with some information that we would like to know about; target a single instrument to keep it simple
 - What information do we need to know
- New information that’s not in the class; what’s dated
 - Errors/inaccuracies/typos in the materials
 - Try to get a template/format for the instructors to provide feedback into; need to prompt for specific questions
 
 
Next Call
- Wednesday January 15, 2019 @ 11:00 Central
 - Next call brainstorm on evaluation questions for instructors/auditors
 - What else does delivery entail?
 - Debrief on delivery webinar