Different forms of monitoring are required by laws, regulations, guidance, and the Workforce Innovation Opportunity Act (WIOA) Plan for the State of Colorado (WIOA State Plan) to ensure the appropriate use of Adult Education and Family Literacy Act (AEFLA) funds. Monitoring is conducted by the Office of Adult Education Initiatives (AEI) and Grants Fiscal staff members. Efforts have been made to streamline monitoring processes as much as possible and AEI is interested in feedback from programs about these processes.
Per 2 C.F.R. 200.333, documents and records related to the federally-funded AEFLA Grant must be retained for three years from the date of the record or document. AEI requires the state-funded AELA Grant to follow the same record retention policy.
Program visits are informal, face-to-face visits between an AEI staff member and grantee at the grantee’s program location. These are not monitoring visits. Program visits allow AEI staff to get to know grantee staff, see the program in operation first hand, and discuss areas of technical support the grantee would like to receive. AEI staff will attend as many program visits as possible each year.
Risk Assessment Process
Per 2 CFR 200.331(b), the Adult Education Initiatives Office (AEI) must conduct a risk assessment at the beginning of each program year, using established criteria, to evaluate each grantees risk for purposes of determining a monitoring plan. This risk assessment will be used to determine which programs are prioritized for on-site monitoring each year.
Annually, programs identified as “High Risk” through the risk assessment will receive a formal on-site review visit. At least once within the 3-year grant cycle all grantees will receive a formal on-site review visit (see the 2017-18 On-Site Review Template). On-site reviews include review and discussion of all components of a program relevant to AEFLA funding and may span multiple days. They will include a review of local program records as well as interviews with staff and learners. AEI will notify the local program director if an on-site review visit is required and will work with the director to schedule dates. Areas of concern or findings resulting from on-site reviews may result in Program Improvement Plans (PIP) or Corrective Action Plans (CAP) that the local program must complete.
The AEI Office, as part of the on-site review process and identified in the WIOA State Plan, will support grantees utilizing a Peer Review process. This process will include selecting a reviewer from another adult education program with expertise in an area of quality improvement. The efforts of the Peer Reviewer will support the program in improving practices and enhancing delivery. Also,the process of selecting the area of focus for the Peer Review will be based on the needs of the grantee and decided between the AEI Program Coordinator and the program director. Details on how to request Peer Review and how to become a Peer Reviewer are forthcoming.
Monthly Data Monitoring
Each month, AEI Program Coordinators review data entered in the LACES database for compliance against reporting requirements, progress towards performance targets, and to identify any areas of promise or growth. AEI uses a combination of views, reports, and searches in order to complete this monthly data review. Data reporting errors and concerns are sent to grantees monthly for data cleaning or process improvement. Consistent data monitoring concerns may result in Program Improvement or Corrective Action Plans being assigned to the grantee. More detailed information about monthly data monitoring can be found in the LACES Data Dictionary.
Quarterly Progress Update
Per the WIOA State Plan, grantees will submit a quarterly progress update whereby programs reflect on successes and challenges, outline modifications to grant application plans, and/or share plans for the next 3-9 month period. The updates will be submitted as tracked changes to the 2018-19 Grant Continuation Application. Reference the At-a-Glance calendar for submission deadlines for each quarterly progress update.
Annual Self-Evaluation (Continuation) Process
AEI awarded three-year grants starting in the 2017-2018 fiscal year (July 1, 2017-June 30, 2018). Funding in years two and three (2018-2019 and 2019-2020, respectively) of the grant is not guaranteed as it is contingent upon continued appropriations and upon grantees meeting all grant, fiscal, and reporting requirements.
To receive funding in years two and three, grantees will also need to complete a self-evaluation, or continuation application, outlining their plan for services being provided with AEFLA dollars in the coming year. This process is referenced in the Colorado State Plan. Grantees will self-evaluate the effectiveness of adult education and family literacy activities in their program by using a uniform self-evaluation tool, or continuation application, created by AEI. Grantees must use data collected to develop goals and objectives for improvement in achieving the goals set forth on learner outcomes, professional development plans as well as any other areas of improvement that have been identified during the evaluation process. This plan must be developed and implemented in consultation with AEI.
Program Improvement Plan (PIP) & Corrective Action Plan (CAP)
The AEI Office uses program improvement and corrective action plans to support grantees in their continuous improvement efforts. Each plan provides a template for documenting improvement steps, timelines, and results. Program Improvement Plans may be requested of or assigned to a grantee by the AEI Program Coordinator for a variety of minor AEI policy or guidance concerns. Corrective Action Plans are assigned to grantees for significant or repeated AEI policy and guidance violations or as the result of findings during a formal on-site review visits. Each plan provides an opportunity for thought partnership and dialogue between the AEI Program Coordinator and the grantee. CAP and PIP templates will be provided to grantees by their AEI Program Coordinator as needed.
The Standards of Quality Framework
The Standards of Quality (SoQ) for Adult Education Programs in Colorado is a rubric that all programs may use to ensure the highest quality program and elements may be embedded throughout monitoring processes as appropriate. The SoQ creates a matrix of quality indicators based on evidence and best practices, as required by the WIOA State Plan. This document serves as the framework to determine strengths as well as level and types of support to each grantee. The standards address program management and the delivery of services to adult learners from recruitment to transition into postsecondary education, training, or the workforce. The standards address the roles of the major stakeholders in adult education—including program leadership, teachers, volunteers, office staff, learners, and funders. While priorities and requirements of specific legislation and/or funding sources may change, these SoQ are designed to be universal – i.e. they adhere to the principles of adult education and they are applicable across the spectrum of program size, type and location. The standards encompass best practices for serving adult learners and policies that govern program administration.
Resources, facilities, and community support vary greatly among Colorado’s adult education programs. While it is acknowledged that some programs may be better positioned than others to demonstrate effectiveness in some of the indicators, the SoQ are intended to guide all programs to perform at a consistent and high level of quality. The indicators serve to recognize strengths and areas for growth. The overarching goal of program-level SoQ are to ensure that ABE, ASE, and adult ESL learners in Colorado receive high-quality services leading to positive outcomes for learners and their families and communities. Further information and release of a revised rubric is forthcoming.
The Office of Adult Education Initiatives (AEI) utilizes a handful of required reports to check on grantee progress and to comply with state and federal reporting requirements. Quarterly, grantees will submit a Progress Update. Information from the quarterly progress update should speak to grant activity occurring over the prior 3 month period. These updates are used to inform technical assistance provided to grantees by AEI Program Coordinators and to identify best and promising practices. AEI will provide grantees with items that need to be covered in the quarterly Progress Updates.
Annually, grantees will submit an Annual Performance Report (APR) and an Annual Financial Report (AFR) detailing program or fiscal activity for the prior program year. Templates for each report are provided to grantees at the beginning of July. Completed reports are due to AEI in July and September respectively. Grantee information submitted in the APR and AFR is used to complete statewide APR and AFR reports submitted to the U.S. Department of Education’s Office of Adult, Career, and Technical Education (OCTAE) annually each fall.
The Office of Adult Education Initiatives (AEI) utilizes a handful of required reports to check on grantee progress and to comply with state reporting requirements. AELA grantees will submit a mid-year and an end-of-year performance report, using the AELA Mid-Year & End of Year Performance Report Template, that details progress on items outlined in the grant application and participant data including demographics and learning outcomes. These updates are used to develop the annual report to the state legislature.
Grantees turn in three Interim Financial Reports during the fiscal year in October, January and April. Grantees also turn in an Annual Financial Report (AFR) for the previous fiscal year they were funded 90 days after fiscal year end. For more information about these fiscal reports, please see Grants Fiscal Reference Guide section of the Grantee Handbook.