The MERL Tech Community of Practice

The MERL Tech Community of Practice (MERL Tech CoP) brings together monitoring, evaluation, research, and learning (MERL) practitioners, artificial intelligence (AI) experts, and data responsibility advocates to learn and collaborate. We focus on responsible, appropriate, and effective applications of AI (including Generative AI) to address demand-driven, real-world MERL challenges.

We aim to reach and support an array of audiences who may not have data science backgrounds, presenting complex and opaque concepts in accessible ways. We work together to influence the use of AI for MERL, and we advocate for a stronger role of MERL professionals in evaluating the effects of AI in social sector programming and wider society.

The CoP is part of the wider MERL Tech community, which has focused on the responsible use of digital data and technologies in monitoring, evaluation, research, and learning practices for over a decade. The MERL Tech Initiative convenes and supports the CoP.

The goals of the MERL Tech CoP are to:

Connect and build a diverse network of MERL practitioners committed to learning about and advocating for responsible, ethical applications of emerging AI (especially Generative AI) for MERL in development, humanitarian, human rights, peace building, and philanthropy work.
Democratize understanding of emerging AI technologies and what they can (and cannot or should not) do for MERL, especially in support of problems and practitioners in global majority countries.
Identify, explore, develop and/or pilot ethical and responsible AI tools, systems, code, and models for MERL and share those lessons in the public domain.
Develop guidance, and support materials as public goods to enable responsible uptake and scaling of GenAI and other AI technologies for MERL and to equip MERL professionals to better assess and evaluate AI models and AI uses more broadly.
Influence how the MERL sector (and beyond) understands and views emerging AI at practical and systems levels and raise the alarm on contexts and instances when the use of AI technologies should be actively discouraged or discontinued.

We are grateful to the David and Lucile Packard Foundation for core support that allows the CoP to operate. We also thank the The Hewlett Foundation for their support for our AI+African MERL Learning Group.

Become a member!

Membership is free! To become a member of MERL Tech CoP, fill in this form indicating your agreement and alignment with the CoP Charter and Code of Conduct.

What do I get out of joining The MERL Tech CoP?

Access to meetings and events
First access to online events and ‘in person’ events where there is limited space.
Access to CoP LinkedIn discussions, CoP Slack channels, and/or other discussion forums that the CoP might create.
Opportunity to propose topics and to create, lead, and/or participate in Learning Groups.
First access to specific learning, materials and resources created by Learning Groups.
Access to ‘expert advice’ via set ‘office hours’ or a ‘help desk’ (as it becomes available).
A voice in identifying CoP priorities, strategic opportunities, thematic areas of focus

Get to know our Learning Groups

The CoP has several Learning Groups, which focus on different themes within NLP, MERL, and AI.

Sandbox Learning Group: The Sandbox Group works to identify, test, and compare tools / applications; collaborate on open-source NLP for MERL approaches; and delve into technical nuts and bolts of LLMs, NLP and GenAI applications that are the interface between the models and those who use them.
Ethics and Governance Learning Group. The EGLG focuses on exploring emerging ethics and governance issues and themes, keeping tabs, and sharing AI ethics and governance policies and guidance, further exploring participation and accountability to individuals and communities, and exploring the application of feminist frameworks for AI Ethics and Governance.
Social and Behavior Change (SBC) Learning Group: This group explores ways that GenAI and NLP can support behavior change efforts with a focus on if and how GenAI Chatbots can support SBCC and the MERL of SBCC, The Group also works to test or showcase results of different approaches, explore and compare cost effectiveness and impact, and develop and implement a research agenda in the area of SBCC and NLP/GenAI.
Philanthropy Learning Group: This group convenes MERL teams and other staff working in philanthropic organizations for discussions about the application of NLP for MERL in the foundation world; sharing organizational policy guidance; strengthening foundation capacities to assess the proposed use of NLPs in programs; and considering the role of foundations in the future of AI.
AI in African Evaluation Workstream/Learning Group: This group focuses on conversations and practical learning about AI and NLP for MERL in African contexts and “Made in Africa” AI and NLP tools. It creates space for connection and cross-learning with other NLP / AI and Evaluation groups and networks in and outside of Africa.
Humanitarian AI Learning Group: This group engages with other networks (NetHope, ALNAP, CDAC) to collaborate, co-convene and support work related to greater accountability to affected people in conflict and crisis contexts for AI; to work to open space for voice and choice around the use of affected people’s data for AI decision-making and algorithm training; and to discuss the role of corporate data capture in crisis and humanitarian contexts.
Gender, MERL and AI Learning Group: This group explores a range of topics related to gender, MERL and AI, including uses and critiques of AI through a gender lens and from practical experience; real cases and examples of AI use in MERL of gender programs; resources and support to adopting, testing, and evaluating AI in MERL work for gender programs. The group will intentionally connect with other Learning Groups in the wider CoP offering opportunities to learn from and share and to bring a gender lens to the CoP’s work.
AI and Climate MERL: This group explores the potential of AI for MERL related to Climate Change and Climate Justice. It also looks into the climate and wider environmental issues with AI — including water, energy and other aspects. The group will explore alternative, green AI as a way to reduce and mitigate the ways that AI contributes to environmental degradation and climate change.
UX Design of AI Learning Group: This learning group will aim to meet quarterly and provide opportunities for practitioners with hands-on experience to share insights into who is and isn’t using AI tools and why (trust, awareness, device ownership, data affordability, digital competency, costs, liability, and more); discuss common usage patterns to help inform more intelligent design (onboarding behaviors, informed consent, frequency and duration of use, pain points, guardrails, human-in-the-loop considerations); share and critique emerging design and prompting conventions, with emphasis on data privacy, safety, inclusion, accessibility, and usability; document challenges and promising examples of responsible, human-centered design practices; and increase awareness of safety risks and mitigation strategies (red-teaming, retrieval).
AI, Asia and MERL: This learning group will focus on issues at the intersection of monitoring, evaluation, research, and learning (MERL), artificial intelligence (AI), and data responsibility in Asia.
AI, MERL and Latin America: This learning group will focus on issues at the intersection of monitoring, evaluation, research, and learning (MERL), artificial intelligence (AI), and data responsibility in Latin America.

Learning groups may shift focus over time to adapt to member needs and new ones will likely emerge. CoP members can contact Linda or Bárbara to be added to a Learning Group.

FAQ

When and why was the CoP founded? How has it evolved?

In 2022, members of the MERL Tech Community expressed interested in exploring the application of Natural Language Processing (NLP) and Text Analytics to MERL – including automated transcription, production of log frames, extraction of key themes in qualitative data sets, writing summaries and reports. Many organizations wanted to explore training NLP models on specific data sets that might enable Generative AI (GenAI) to provide outputs that are tailored to specific MERL needs in the social sector. The MERL Tech Initiative began organizing meetings and webinars to support learning and documentation of key NLP + MERL concepts and applications for the social sector and to better understand Large Language Models (LLMs) and how the connected with MERL. The Community of Practice was called the “NLP-CoP” for its first 3 years.

Over time, and as the space evolved, the terms AI and GenAI became more popular catch phrases for this area of work. The term NLP became outmoded and too specific for the work of the group. So we changed the name to the “MERL Tech Community of Practice” to reflect both the fact that different kinds of AI are becoming more mainstreamed, and the fact that overall the CoP looks at a variety of emerging technologies and their intersection with MERL. This also allows for us to continue evolving and adapting to new technologies as they arise.

How can I become a member?

Membership is free! To become a member of MERL Tech CoP, fill in this form indicating your agreement and alignment with the CoP Charter and Code of Conduct.

How can I learn about upcoming CoP events?

We have a dedicated events page, where we share all upcoming CoP events. You can also follow us on LinkedIn and BlueSky.