[
    {
        "id": "osp-2360",
        "type": "article-journal",
        "title": "Programmatic Evaluation: A Prospect in Program Evaluation Design",
        "author": [
            {
                "family": "Al Sinani",
                "given": "Siham"
            },
            {
                "family": "Al-Naamani",
                "given": "Khalid"
            }
        ],
        "URL": "https://omanscience.com/en/articles/programmatic-evaluation-a-prospect-in-program-evaluation-design",
        "language": "en",
        "issued": {
            "date-parts": [
                [
                    2021
                ]
            ]
        },
        "container-title": "Oman Medical Journal",
        "DOI": "10.5001/omj.2021.122",
        "publisher": "Oman Medical Specialty Board",
        "ISSN": "1999-768X",
        "abstract": "Improving the provision of health care is the ultimate goal of health profession education (HPE) programs. Therefore, a well-founded evaluation program is fundamental in the effective planning and implementation of HPE to achieve quality health care. Educational program evaluation is the systematic information gathering and analysis approach related to the educational program’s design, implementation, and outcomes with the aim of monitoring and candid pronouncement of its value. Although much of the attention is on academic program development and implementation, evaluation in education has transformed into a practice-based science over the past few decades. Because educational programs are essentially about bringing change, HPE programs are required to implement procedures that denote pedagogical outcomes resulting in better accountability in the health profession. Consequently, program evaluation would have to be designed to ensure that change, both intended and unintended, has transpired and allow educators and the overseeing bodies to obtain important information on their HPE programs. Furthermore, evidence of undergoing mediocre evaluation can be utilized to decide on the allocation of resources. Accordingly, evaluation-based feedback is an inherent part of purposeful practice concepts of pedagogical processes. Evaluation approaches are contingent on the purpose utilization of the intended evaluation models. Additionally, HPE programs incorporate complex and interactive educational components resulting in diverse outcomes. Nevertheless, most HPE programs utilize one or two evaluation tools and are possibly missing essential parts. Therefore, evaluation systems of complex HPE programs should not be constructed based on conventional approaches assuming a direct cause-and-effect relationship between elements and outcomes. Thus, a change in evaluators’ mindsets to recognize the various combinations of programs’ elements is essential to reconcile such complexity. In addition, combining evaluation theories, approaches, and models to build a comprehensive evaluation model that would suit the complex educational programs may lead to less misleading knowledge and express the pertinent components of evaluation. Evaluation models such as the Action-Logic Model, Bennett’s Hierarchy model, and the Context, Input, Process, and Product model have been utilized in many educational fields. However, HPE specific program evaluation models are limited, and research examining the efficacy of these models in HPE programs is inadequate, especially considering programmatic evaluation as a purposeful mix of evaluation activities to enhance HPE programs’ decision-making and quality-assurance functions, thus enhancing its outcomes. Because HPE programs are regarded as open, complex systems, they cannot be evaluated by examining their components separately. All considered, there is an obvious gap in evaluation practice and knowledge in HPE evaluation. One can adopt one or more existing methods or tools utilizing their strengths and avoiding their limitations in combination with one self’s addition and/or alterations in a programmatic and holistic approach to design a model that fits the purpose and needs. There is increasing demand for HPE programs to pronounce their impact on society. However, meta-analyses demonstrate a minimal effect of HPE programs on physicians’ behaviors and patient care outcomes. This could be attributed to the possibility that the utilized evaluation models do not satisfactorily demonstrate the intended outcomes and/or the fact that programs have little value to add. Therefore, when evaluating HPE programs, a combination model that is attentive to the complexity of such programs and allows change examination is advocated for as the best option in educational program evaluation design. Subsequently, and in parallel to the ‘programs of assessment’ notion and in recognition of the principle that the whole is greater than the sum of its parts, the suggestion to move towards a ‘programmatic medical education program evaluation’ is put forward here. This proposition was attempted to maximize the benefits and minimize the limitations observed in some of the commonly utilized evaluation models. When adopting the concept that the whole is greater than the sum of its parts, we might be able to create evaluation programs that incorporate multiple tools, involving multiple stakeholders, and occurring across the whole program resulting in a more comprehensive evaluation and achieve the goal of improved patient care. Therefore, a programmatic, comprehensive, and holistic evaluation program that determines the educational program’s impact is necessary in the era of rapidly unfolding HPE. In conclusion, models for evaluating learning programs exist but lack evidence of comprehensiveness in complex and intricate HPE programs. In addition, there is limited literature on the programmatic approach to the evaluation of educational programs. This paper might provide one of the initial suggestions for the utilization of fit for purpose programmatic evaluation of educational programs, add to the existing education program evaluation knowledge, and help future programmatic educational program evaluation frameworks in general. The structure of the proposed fit-for-purpose programmatic evaluation model can be context-specific, flexible in approach, and actively involve its stakeholders, and not limited to a particular theory, approach, or methodology. It may help users to determine the best approach and methodology for their program."
    }
]