Behind the Research: How Our Team Tackles Hard Questions

In an effort to direct substantially more funding to the most cost-effective opportunities we can find to help people, GiveWell’s research is expanding in two directions at once: going broader, to cover more areas of global health and development, and going deeper, to improve the inputs that inform our decision-making.

This episode offers a behind-the-scenes look at how this expansion is playing out on GiveWell’s research team, as three researchers each describe a project they have taken on: reassessing how to prevent childhood pneumonia deaths, building a better estimate of how many children die of malaria, and developing an AI tool to check our work for errors.

In this episode, GiveWell co-founder and CEO Elie Hassenfeld speaks with Senior Program Officer Dilhan Perera, Senior Researcher James Watson, and Research Analyst Meghna Ray about the projects they’ve worked on and what it’s like to do research at GiveWell. Elie, Dilhan, James, and Meghna discuss: Changing direction as we follow the evidence: Pneumonia causes around 10% to 15% of deaths among children under five globally. Our New Areas research subteam, which investigates cause areas that are newer to GiveWell, first examined community-based treatment options for reducing these deaths by treating children with amoxicillin, an inexpensive antibiotic. That approach initially looked promising, but conversations with researchers and implementers indicated that many pneumonia deaths today are caused by infections that amoxicillin doesn’t treat. We then shifted our attention toward another strategy for reducing pneumonia deaths: providing medical oxygen for severely ill children in hospitals. This costs more per patient but targets the children at greatest risk of dying and can also treat diseases beyond pneumonia. If expanding access to medical oxygen proves cost-effective, it could become a new way for GiveWell to help prevent these deaths. Improving our estimates for key cost-effectiveness inputs: Assessing the cost-effectiveness of malaria programs generally requires estimating the number of children who die from malaria in a given place, but that figure is surprisingly uncertain. In many places where we fund programs, cause-of-death data come mainly from interviews with families rather than from health system records. Because malaria symptoms are very similar to symptoms for many other diseases, knowing the symptoms a child was experiencing is often insufficient to determine whether malaria was the cause of death. To more accurately determine the share of deaths caused by malaria, we are developing an approach based on the share of school-aged children infected with malaria (which can be measured accurately using rapid fingerprick tests) and the overall child death rate (measured reliably using large household surveys). We then use the relationship between infection levels and deaths to estimate how many are attributable to malaria. Because this approach is based on data that we think can more reliably assess malaria-specific deaths, we expect it will be better at identifying differences in malaria deaths between regions, which could allow us to target malaria funding to the places where it will do the most good. Developing new tools to increase research capacity: Our cost-effectiveness spreadsheets are often very complex, and errors in their formulas or inputs can meaningfully change our estimate of a program’s cost-effectiveness, which is a key factor in our funding decisions. Research analysts have long vetted these analyses line by line to ensure their accuracy. We recently built an AI tool to assist in that process. Early versions reliably caught errors in formulas but often missed issues a human reviewer would catch, such as outdated data or inconsistent assumptions. To address this, we compiled errors analysts had found in past reviews and refined the tool until it caught more than 90% of them. The tool now serves as a first pass, followed by a research anal