CAPTURING DATA FOR MULTI-DIMENSIONAL RECOVERY RESPONSES

Authored by Ms. Ayushka Nugaliyadda and Ms. Sarika Warusavitarana of Citra Social Innovation Lab

Looking back almost a year on from the onset of the economic crisis that continues to plague Sri Lanka, the situation at the ground level was unlike anything the country had ever seen. As inflation spiraled and protests erupted in the wake of fuel shortages, extended power cuts and growing discontent, our team continued to sit together, on Zoom or in any conference room we could find on the days we managed to come into office — on foot, by bus, by bicycle, or by exorbitantly priced tuk-tuks — to try and figure out what we could do to help.

But every time we tried to deep dive into one of the many challenges posed by these crises, we kept hitting the same brick wall — we had little evidence beyond our own experiences and what we were seeing and hearing on social media or anecdotally. The dearth of data in the country since the pandemic was a concern that kept cropping up in both internal meetings and with partners, as official statistical institutions faced considerable challenges in collecting robust data during this time. It was during this time that Kanni Wignaraja, Assistant Administrator and Director of the UNDP Regional Bureau for Asia and the Pacific, flagged that, even in the face of multiple urgent needs, a rapidly administered national citizen survey would provide a useful evidence base to inform crisis-related and recovery-focused programming, and posed a question to us as to whether Citra would like to drive this process — and given that we were thinking about the very same need, we jumped at the chance!

Our Process

Conscious of the fact that the team had never administered a survey at this scale before, we first took a bit of time to try to identify what sort of data was available from the period between 2019 and 2022, from government sources as well as from the UN and other non-government institutions. The gap we identified was that while there have been many robust rapid assessments that have been conducted, there were very few larger-scale household-level surveys that allowed for disaggregation at the sub-national level — for instance, the Department of Census and Statistics’ (DCS) Household Income and Expenditure Survey (HIES) is generally administered every three years, but owing to the pandemic, its 2022 round was stalled.

We began by mapping out areas in which we felt that data has been consistently under-reported, unavailable, not updated, or a little muddy — for instance, around persons with disabilities, interhousehold dynamics, the care burden and menstrual hygiene. We spent a few days cooped up in a meeting room, thinking about what kind of data would be not only interesting to dig up in the aftermath of COVID and during the economic crisis, but what would be useful for policymakers in the immediate future. It was also important to think about what kinds of data points were likely to be collected by other agencies during this time, which meant we didn’t have to — for instance, there are technical agencies carrying out frequent assessments focused on very specific areas.

Based on this initial exploration, we settled on a few broad thematic areas we wanted to delve into, which we felt would be able to provide a reasonable snapshot of both the socio-economic and governance impacts felt at the household level during this time: namely, on: (1) demography; (2) employment status; (3) debt status; (4) household assets; (5) access to basic necessities and social services; (6) expenditure; (7) coping strategies; and (8) social cohesion and governance.

Considerations During Designing

Once we had mapped out the broad areas that we wanted to include, and before we got to the actual designing of the survey questions, we listed out a few key considerations we needed to bear in mind and keep reminding ourselves of:

• The length of the survey: It was already looking like it was going to be a rather long survey, but we were conscious of not wanting people to spend too much time on it, being especially sensitive to additional work and household burdens that may have emerged as a result of the pandemic or the crisis.
• The sampling methodology: We knew we wanted: 1) a nationally representative sample; and 2) data that could be disaggregated at the district level. Our conversations with colleagues at the DCS indicated that the latter would be a vital requirement for a survey of this nature — and most importantly, be very useful for policymaking.
• Question design: Certain bits of data that we wanted to capture through the survey, such as on income, employment status and coping strategies, although seemingly straightforward, required much thought on sensitivity and ethics, particularly given the crisis. Other considerations included ensuring that we weren’t asking leading questions, particularly for the modules on governance and trust in public institutions, where the way in which the questions were phrased shouldn’t imply that there were ‘right’ or ‘wrong’ answers for the respondents.
• Data accuracy: Collecting data on household income, expenditure and consumption is difficult, due to inaccuracies due to recall bias and misreporting. Bearing this in mind, we tried to approach these subjects in different ways — for instance, through a set of questions asking the household which religious/cultural festivals they celebrate, if any, how they celebrated this year, and what may have been different from the previous times they’ve celebrated, in order to get a sense of how things have changed for each household. Moreover, we ensured that the DCS took a look at our question bank, not only to ensure that it was methodologically sound, but to raise any issues they traditionally face at the ground level that would be useful for us to keep in mind.

Having taken a crack at the first draft of the survey, we made sure that we brought in expert perspectives — for instance, technical experts at the UNDP Country Office and representatives from other UN agencies to avoid potential areas of overlap with their ongoing work and to get their feedback on what more might be required. Importantly, we made sure to sit down with officials from the Department of Census and Statistics to discuss survey design and sample methodology, to not only ensure the survey was rigorous, but that they would partner with us in advocating for whatever the findings would reveal.

We also brought two experienced technical partners into this process: firstly, the UNDP SURGE Data Hub, which supports UNDP Country Offices in collecting primary data on ground-level socio-economic impacts, came on board with us to provide their valuable expertise in survey design and coding, and to collaborate with us on data analysis and visualization. Secondly, we contracted Nielsons IQ to leverage their vast experience in survey administration to conduct the survey, data cleaning and translations.

Our next step was to run a pilot survey with 50 households across the Colombo, Kurunegala, Jaffna, Kandy, and Batticaloa districts. Results from the pilot were positive, and after making some minor tweaks to the questions, we started rolling out the survey in full.

Progress So Far

Data collection is on track and we have already surveyed over 15,000 households. In the meantime, given the urgent need to address existing gaps in data, we are already working to analyse data from the first 6,000 households surveyed. We understand that although this sub-sample will also be nationally representative, it will only provide us with indicative findings at this stage. However, we strongly felt that getting out the data in this staggered manner would still be useful to supplement other data collection initiatives and inform development planning, even if only very preliminary results could be gleaned.

The full dataset will be available in May 2023. We are looking forward to launching a report with analysis and key findings, as well as an interactive dashboard, which will be shared publicly and freely available, in keeping with our intention to ensure open access to the data collected. We are certain that the data collected will be of use to everyone, including the Government, development partners, researchers, and anyone else interested in better understanding the ground level impact of the economic crisis, especially since the data can be disaggregated and analysed at the subnational level.

In the spirit of working out loud, we’ll continue to share updates on our progress. But, in the meantime, if you are interested in collaborating with us to analyse the data, unpack the findings and explore the different ways in which we might use this data set, we would love to hear from you!

Citra is Sri Lanka’s pioneer Social Innovation Lab that offers human-centered design and systemic design thinking approaches to solve complex development problems. Together with the Prime Minister’s Office of Sri Lanka and the United Nations Development Programme (UNDP) in Sri Lanka, Citra Lab focuses on adopting innovative approaches and tools in its work to build solutions to the challenges faced by the country.