Email this article 
Hide
Show all
Original Research
Improving inter-rater reliability of the enhancing assessment of common therapeutic factors (ENACT) measure through training of raters
Submitted: 11 April 2024 | Published: 07 September 2022
About the author(s)
Mwamba M. Mwenge, Centre for Infectious Disease Research in Zambia, Lusaka, ZambiaCaleb J. Figge, Johns Hopkins Bloomberg School of Public Health, Baltimore, United States
Kristina Metz, Johns Hopkins Bloomberg School of Public Health, Baltimore, United States
Jeremy C. Kane, Columbia University Mailman School of Public Health, New York, United States
Brandon A. Kohrt, George Washington University School of Medicine, Washington, DC, United States
Gloria A. Pedersen, George Washington University School of Medicine, Washington, DC, United States
Izukanji Sikazwe, Centre for Infectious Disease Research in Zambia, Lusaka, Zambia
Stephanie Skavenski Van Wyk, Johns Hopkins Bloomberg School of Public Health, Baltimore, United States
Saphira M. Mulemba, Centre for Infectious Disease Research in Zambia, Lusaka, Zambia
Laura K. Murray, Johns Hopkins Bloomberg School of Public Health, Baltimore, United States
Full Text:
PDF (425KB)Abstract
Background. As evidence supports task-shifting approaches to reduce the global mental health treatment gap, counselor competency evaluation measures are critical to ensure evidence-based therapies are administered with quality and fidelity. Objective. This article describes a training technique for evaluating lay counselors’ competency for mental health lay practitioners without rating scale experience.
Methods: Mental health practitioners were trained to give the Enhancing Assessment of Common Therapeutic Factors (ENACT) test to assess counselor proficiency in delivering the Common Elements Treatment Approach (CETA) in-person and over the phone using standardized video and audio recordings. A two-day in-person training was followed by a one-day remote training session. Training includes a review of item scales through didactic instructions, active learning by witnessing and scoring role-plays, peer interactions, and trainer observation and feedback. The trainees rated video and audio recordings, and ICC values were calculated.
Results: The training technique presented in this research helped achieve high counselor competency scores among lay providers with no prior experience using rating scales. ICC rated both trainings satisfactory to exceptional (ICC: .71 - .89).
Conclusions. Raters with no past experience with rating scales can achieve high consistency when rating counselor competency through training. Effective rater training should include didactic learning, practical learning with trainer observation and feedback, and video and audio recordings to assess consistency.
Keywords
Metrics
Total abstract views: 2734Total article views: 492
Crossref Citations
1. Assessment Scales in Cerebral Palsy: A Comprehensive Review of Tools and Applications
Chaitanya Kumar Javvaji, Jayant D Vagha, Revat J Meshram, Amar Taksande
Cureus year: 2023
doi: 10.7759/cureus.47939
2. Competency-based training and supervision: development of the WHO-UNICEF Ensuring Quality in Psychosocial and Mental Health Care (EQUIP) initiative
Brandon A Kohrt, Gloria A Pedersen, Alison Schafer, Kenneth Carswell, Fiamma Rupp, Mark J D Jordans, Elise West, Josephine Akellot, Pamela Y Collins, Carmen Contreras, Jerome T Galea, Frezgi Gebrekristos, Muthoni Mathai, Kristina Metz, Naser Morina, Mwamba Mwila Mwenge, Frederik Steen, Ann Willhoite, Mark van Ommeren, James Underhill, Aemal Akhtar, Richard A. Bryant, Tessa Concepcion, Rozane El Marsi, Abdelrhman Elnasseh, Mahmoud Hemmo, Rosco Kasujja, Gergana Manolova, Evgeniy Maslovskiy, Anne W. Mbwayo, Leydi M. Moran, Saphira Munthali-Mulemba, Byamah B. Mutamba, Teresa Mutavi, Mansurat Raji, Ruta Rangel, Manaswi Sangraula, Alejandra Sepulveda, Wietse A. Tol, Edith van t'Hof, Andrey Yurtaev
The Lancet Psychiatry vol: 12 issue: 1 first page: 67 year: 2025
doi: 10.1016/S2215-0366(24)00183-4
3. Piloting competency assessments for an evidence-based brief psychological intervention with Arabic-speaking non-specialists in Switzerland
Mahmoud Hemmo, Aemal Akhtar, Brandon A. Kohrt, Gloria Pedersen, Abdul Fattah Alkamel, Chantal Martin Sölch, Alison Schafer, Julia Spaaij, Richard Bryant, Naser Morina
Cambridge Prisms: Global Mental Health vol: 12 year: 2025
doi: 10.1017/gmh.2025.10023
4. Exploring Empathic Communication Among Community Health Workers: Applying the ENACT Tool in Two South African Sites
Christina A. Laurenzi, Stephan Rabie, Sihle Mamutse, Sarah Skeen, Nicola Jansen van Vuuren, Rosanne Neethling, Sally Field, Simone Honikman
Health Promotion Practice vol: 26 issue: 4 first page: 688 year: 2025
doi: 10.1177/15248399241285888
5. Functional Independence Assessment in Children and Adolescents with Achondroplasia: A Multicenter Cross-Sectional Study Using the WeeFIM Scale
Chung-Lin Lee, Hung-Hsiang Fang, Chih-Kuang Chuang, Dau-Ming Niu, Ju-Li Lin, Mei-Chyn Chao, Yen-Yin Chou, Pao Chin Chiu, Chia-Chi Hsu, Tzu-Hung Chu, Yin-Hsiu Chien, Huei-Ching Chiu, Ya-Hui Chang, Yuan-Rong Tu, Yun-Ting Lo, Hsiang-Yu Lin, Shuan-Pei Lin
Diagnostics vol: 15 issue: 19 first page: 2532 year: 2025
doi: 10.3390/diagnostics15192532
6. Evaluation of the reliability and usability of CARE‐Radiology: A descriptive‐analytic study
Mengshu Wang, Xufei Luo, Janne Estill, Karen Spruyt, Ryo Kurokawa, Nav Persaud, Yasuteru Shimamura, Holly Raison, Paolo Niccolò Franco, Cesare Maino, Hussein Elkhayat, Rehab A. Galal, Daisuke Kimura, Shingo Omata, Sergey A. Ryzhkin, Timur R. Izmailov, Rustem A. Bashirov, Zhaoxiang Bian, Jinhui Tian, Junqiang Lei
Journal of Evidence-Based Medicine vol: 17 issue: 3 first page: 597 year: 2024
doi: 10.1111/jebm.12638
7. Assessment of regenerative hospitality
Jannah Kreb, Wichard Zwaal
Research in Hospitality Management vol: 14 issue: 2 first page: 209 year: 2024
doi: 10.1080/22243534.2024.2411978
8. Perspectives on competency-based feedback for training non-specialists to deliver psychological interventions: multi-site qualitative study of the EQUIP competency-based approach
Abdelrhman Elnasseh, Varun S. Mehta, Gergana Manolova, Gloria A. Pedersen, Shannon Golden, Liyam Eloul, Frezgi Gebrekristos, Pamela Y. Collins, Teresia Mutavi, Anne W. Mbwayo, Muthoni Mathai, Tessa Concepcion, Rozane El Masri, Frederik Steen, Jerome T. Galea, Carmen Contreras, Josephine Akellot, Rosco Kasujja, Samuel Wasereka, Byamah Brian Mutamba, Wietse A. Tol, Mansurat Raji, Sacha Moufarrej, Alison Schafer, Brandon A. Kohrt
BJPsych Open vol: 10 issue: 4 year: 2024
doi: 10.1192/bjo.2024.37
9. Advances in Psychotropic Treatment for Pregnant Women: Efficacy, Adverse Outcomes, and Therapeutic Monitoring
Bárbara Costa, Nuno Vale
Journal of Clinical Medicine vol: 13 issue: 15 first page: 4398 year: 2024
doi: 10.3390/jcm13154398