Site icon Mark Michaelis' Weblog

GiveDirectly Impact – Questions and Impressions Following Rwanda 2022 Visit

My initial attraction to GiveDirectly, and why IntelliTect decided to support them last year, was the studies they did at the onset to show the positive impact of the cash transfers and verification that, in fact, recipients overwhelmingly made positive choices in what to do with the money. Rather than drinking, drugs, or gambling, the recipients chose to purchase a cow, roof, health insurance, a child’s education, home improvements, or something equally worthwhile. Overwhelmingly, the recipients net worth and quality of life improved significantly. During my visit this tip, it was a blessing to be able to see their gratitude, excitement, and accept their thank-yous. While I only visited four or five households this week, I came away with a strong confirmation that these recipients were exceedingly grateful for the cash transfers and that they were not making unhealthy choices about how to spend the funds. In fact, in the Miller, their choice was especially strategic (at least in terms of income generation).

The GiveDirectly team I met with is relatively young and eager about the mission of GD. Everyone had a college/university degree (the academic discipline of the degree didn’t really matter to GD which makes sense to me). Furthermore, based on the team I interacted with, they were all good hires – meaning the GD employees were hard working, believed in the mission, were thoughtful, and eager to improve. In Rwanda, like Uganda, having a degree doesn’t guarantee you a good job, so GD gets to choose the best applicants. (That said, Rwandan unemployment figures are extremely low – less than 2% – which seems very suspicious.)

Two question areas remain, however, and I’m grateful for the ongoing discussion with GiveDirectly as they help me to resolve them:

Is income improved over time and by how much?
GD’s goal is that people would escape extreme poverty. However, remarkably, they don’t have any data/studies to show this is, in fact, achieved. I’m very surprised by this given my initial evaluation of them at the onset of IntelliTect’s donations was that they were data and research driven.  As a result, the following questions arise:

  • After five years (for example) have recipient households escaped out of extreme poverty? GiveDirectly doesn’t have any data to say that they do.
  • What is the percentage of increased income?
    One study in Uganda, which GD provided for me, shows that over twelve years income goes up a total of 17% (I assume this is after inflation as inflation alone in that time period would be more than 25%). A recipient who started at 1 USD/day would, therefore, be earning less than 1.20/day after 12 years. Another study in Kenya concluded that, “At four years, the families that received GiveDirectly’s cash were not meaningfully better off than control families.”
    • What is a family’s income to start? (Seemingly, GD was not gathering this information in their census – at least for the one I participated in?)
    • What is a family’s income after five years (and more)?
  • Why had GD not been measuring income improvement both before and at a significant amount of time after, in fact why wasn’t it one of their main measures? One obvious reason for not measuring a long time afterwards would be cost, but surely this is the very purpose of the donations and, therefore, you should verify it is working in the long term – even given the expense. At a minimum, use Rwandan government provided data and verify it with sampling.
  • The other organizations I met with during my East Africa visit learned that training in personal financial, gender sensitization, marital conflict (especially when money become available), and income generation were critical to escaping poverty. What was GD doing such that this training wasn’t needed and how did they verify it?
  • I have no doubt that net worth and quality of life improved with cash transfers (even perhaps after five years though I haven’t looked for the research to support this). How do you measure effectiveness of poverty alleviation when considering increased income versus net worth & quality of life improvements?

Why is GiveDirectly’s detection of household conflict 10% of the Rwandan wide figures of domestic GBV in the last 12 months?

  • Why was GD’s detection on conflict across their clients (~2%) so vastly different from UN Women’s (at the time of this writing, 20% experiencing GBV over the 12 months)?
  • How much was the emphasis to not have conflict presented during the community meeting skewing their ability to detect the conflict?
  • Surely GBV was included in what GD considered conflict or, was it not a significant factor in determining who should receive cash transfers? If not, why not?
  • If GBV was not conflict enough to disqualify a household from receiving a cash transfer, what conflicts were significant to disqualify them (however rarely)?

Why does GiveDirectly’s overhead reporting differ from GiveWell’s evaluation on them?

GiveDirectly’s response to my email on this topic stated (and is consistent with their website):

The discrepancy boils down to a difference in calculation. We calculate efficiency as Cash transfers ÷ [Cash transfers + cost of delivery] which historically comes to ~90%. GiveWell calculates it as “Cash grants make up 83.0% of GiveDirectly’s all-time incurred expenses.” The difference is that we break out our measures into two distinct business systems, (1) cash transfers and (2) fundraising to deliver as cash, with a core performance metric for each; whereas GiveWell groups them all together. 

I’m concerned about this for multiple reasons.

  • The most significant is that they receive money that is restricted to a specific expense category (namely fundraising) rather than the generally preferably unrestricted donation. And, as such, they credit the donations from you or me as going directly to the recipient (well, except for transferring the money). In other words, while generally restricted funds are less preferable, because GiveDirectly has restricted funds for fundraising and they are crediting themselves as having a lower overhead for your or my donations than everyone else even though, in fact, their fundraising is nothing special, it is just that the unrestricted funds that you or I are donating, are not going towards fundraising (or perhaps even overhead).
  • GiveDirectly’s fundraising may not be particularly good but they are playing by different rules so, on their website, they look better than other organizations website shows the fundraising as a normal cost. This seems suspiciously disingenuous – and hard to decipher – and outside the spirit of truth in reporting. Help me understand?

Questions:

  • When, on their website, GiveDirectly doesn’t include their fundraising costs in the measure of giving efficiency and yet compares themselves to other organizations, are they subtracting the fundraising costs from other organizations metrics?
  • Where does overhead, such as the cost of management cost or accounting, fit in their spending report?
  • What makes GiveDirectly special such that they believe that not reporting the cost of fundraising the same as everyone else is acceptable? Is it simply that they have a donation restricted to fundraising? What if everyone did the same, would we end up with more truth or less?

Exit mobile version