Child Welfare Certification Impact

The updated certification program standards require a knowledge test and performance assessments. New hires must pass a knowledge exam of child welfare fundamentals before beginning discipline-specific training. After passing the knowledge exam, new hires will complete discipline-specific training, including simulations and field work. During this phase, new hires will be required to demonstrate adequate performance of specific core job functions to hold provisional certification.

Note:

Florida Administrative Code requires any individual holding primary case responsibility to be provisionally or fully certified. This activity is a mechanism to ensure that the new staff member is provided with the training and feedback necessary to build skills before they are assigned to work more independently.

Learning Management System

The Florida Department of Children and Families launched a new learning management system (LMS), My FL Families LEARN, for all DCF employees and 140,000 external users, including all external child welfare professionals and foster parents in April 2023. The eModules for The Academy and skills tracking will be available in the LMS, allowing child welfare professionals across the system of care to train in a shared, virtual environment. The LMS will provide detailed reports and dashboards to easily track training progress.

Evaluation

DCF, in collaboration with the Florida Institute for Child Welfare (Institute), is committed to evaluating all aspects of The Academy implementation phases to ensure that the program is meeting its intended goals and objectives. This comprehensive evaluation process will involve collecting and analyzing data related to the effectiveness and efficiency of the program, including participant feedback, training outcomes, and the overall child welfare program impact. Evaluation information will be used to identify strengths and areas for improvement, and to make data-driven decisions to optimize the training’s effectiveness. In line with this effort, there are three evaluation efforts happening: full implementation evaluation, simulation evaluation, and virtual reality (VR) evaluation.

1. Full Implementation Evaluation:
   In collaboration with the Capacity Building Center for States and the Institute, DCF created an evaluation workgroup to develop a robust evaluation plan for The Academy project. The goal is to ensure that all components of The Academy project are evaluated from initiation to activation to sustainability. The workgroup will coordinate with an independent organization to perform the evaluation.
2. Simulation Evaluation:
   Currently, the Institute is supporting DCF with initial evaluations of the modernization aspects of The Academy project. Beginning in November 2022, the Institute began evaluating the Real Academy Simulation Train-the-Trainer program. The first cohort of trainers has completed the four-series training, and a second cohort has begun training. In general, initial evaluation efforts are focused on trainers' perceptions of the training, the vendor/trainers, support from DCF, and any barriers to statewide implementation. The Institute is sharing session-based reports, including recommendations for consideration, with DCF.
   
   Preliminary feedback from the first cohort of trainers was generally positive for the first two sessions, which cover the use of critical reasoning tools and seeing how to implement a simulation training session. One participant shared, "It was a great training. The simulations were extremely realistic and stress producing even for experienced trainers."
   
   Another said: "This has been one of the most impactful trainings I have been a part of during [my time] as a trainer. The REAL Academy trainers are phenomenal and truly believe in the work they are doing with simulation. I found the training to be a transformational addition to the pre-service curriculum that will better ready the workforce for the realities of the field. The way the [Good Call Tool] is interwoven into the simulations allows for trainees to exercise critical thinking, an opportunity that is often missed as we move through pre-service curriculum."
3. Virtual Reality Evaluation:
   In addition, the Institute is currently developing an initial evaluation plan for the Accenture Virtual Reality Train-the-Trainer program, with the first cohort of trainers slated to undergo training in late March. Similar to the aforementioned simulation evaluation, the Institute will examine trainers’ perceptions of the training, the vendor/trainers, DCF support, self-efficacy, and any barriers to statewide implementation.