Iman operations Research

Zubia Mumtaz – University of Alberta

Evaluating the Improving Mother and Newborn Health initiative: Are community midwives increasing quality skilled birth attendance rates in Quetta, Gwadar, and Kech districts in Balochistan and are they doing so in a financially self-sustaining manner?

Executive Summary

Complications of pregnancy and childbirth remain the leading cause of death and disability for childbearing women in Pakistan. With a measured maternal mortality rate of 297/100,000 live births, Pakistan is one of the six countries that contribute to more than 50% of all maternal deaths worldwide. To address the issue of high maternal and infant mortality rates and to ensure skilled birth attendance, the government of Pakistan created a new cadre of community based midwives (CMW). Pakistan is also characterized by a difficult terrain and political unrest in some provinces. The Balochistan province in particular poses problems for health services delivery given its vast area (44% of Pakistan’s territory) with only 5% of the nation’s population. One expectation is that CMWs will improve access to skilled antenatal and intra-partum care for the women living in these remote areas.

Recent research shows that these CMWs have largely failed to establish midwifery practice and attract patients. A number of reasons have been identified for this failure which include, but are not limited to: 1) community, in particular women's, lack of trust in CMWs’ capacity to conduct safe births; 2) the CMW's lack of interest in pursuing midwifery practice, partly because they are unable to meet its professional demands in terms of time and resources. In particular, the young CMWs cannot travel to patients’ homes without a chaperone, a woman during daytime and at least two additional men during night visits. 3) The CMWs lack of business skills necessary to establish a private practice; and 4) the CMWs lack of financial support essential to develop practice infrastructure and logistics (Mumtaz et al, 2013).

To address some of these issues, the Improving Mother and Newborn health (IMAN) intervention will be implemented in Quetta, Gwadar, and Kech Districts of Balochistan. To enable the CWMs to establish self-sustaining private practices, they will be provided business skills. A micro-finance institute will offer small loans. CMWs will also liaise with LHWs to undertake an awareness-building campaign using cellular phone SMS technology and through existing women’s support groups. To increase access to emergency transport services, the CMWs will work with their communities to establish a revolving transport fund called the Mamta Fund.

The proposed research aims to investigate whether the CMWs in the IMAN initiative are providing the essential maternal and newborn health care to women and children living in remote Balochistan districts of Quetta, Gwadar, and Kech Districts in a financially self-sustaining manner. Specifically the research will investigate: (1) whether the IMAN initiative is having an impact on CMW service uptake; (2) if any increased CMW service uptake is attributable to the IMAN initiative; (3) whether the IMAN initiative will enable the CMWs to develop financially self-sustainable practices; and (4) what is the level of quality of care the CMWs are providing?

Data will be collected in four interlinked modules over a four year period. Module 1 will consist of a cluster randomized control trial. Proportions of births attended by CMWs in two of the IMAN initiative districts and two control (non-IMAN districts) will be compared at baseline (2013) and three years later (2016) to determine (1) if CMWs have improved their coverage and (2) if this increase can be attributed to the IMAN initiative. The data will be collected using survey methodology. The survey will also assess differentials in birth outcomes/complications by IMAN or non-IMAN CMW status including intermittent preventative treatment during pregnancy, clean cord cutting, active management of third stage of labour, post-partum visit for the mother, thermal care (immediate drying and wrapping), immediate breastfeeding of newborns, and patient satisfaction levels with their maternity care provider.

Module 2 will verify whether the IMAN initiative is delivering services to the targeted participants as intended. To do so, a process evaluation will be conducted. This will consist of a systematic and continual documentation of whether the planned IMAN initiative program functions are being performed in compliance with the program plan. Such functions include appropriate uptake and use of micro-loans, completed business training, establishment of the Mamta fund, and functioning partnerships with Lady Health Workers.

Module 3 will explore if the IMAN initiative has enabled the CMWS to establish self-sustaining practices. Organizational, social, and financial challenges faced by CMWs to establishing and running their practices and attracting new clientele will be explored. A quantitative financial tool will be used to assess the direct and indirect monetary costs as well as opportunity costs and benefits. The size, sustainability and effectiveness of the Mamta fund will also be assessed.

Finally, module 4 will explore the quality of care provided by CMWs in the IMAN ditricts. In-depth interviews with CMWs, women of child-bearing age mothers-in-law and older women and other community members will be conducted. CMW patient-provider interactions during antenatal visits and childbirth will be observed to document CMWs quality of care.

An extensive knowledge dissemination plan will facilitate uptake of research findings to both inform on-going positive developments in the IMAN initiative and contribute to the body of evidence around potentially solutions to improve sustainable coverage of high impact MNCH interventions in vulnerable populations . The plan will include knowledge sharing with IMAN policymakers and planners, Government of Balochistan, MercyCorps, Aga Khan Foundation, civil society advocacy groups, community midwives, USAID and other bilateral donors and international organizations.

1. Study Rationale

Complications of pregnancy and childbirth remain the leading cause of death and disability for childbearing women in Pakistan. With a measured maternal mortality rate of 297/100,000 live births, Pakistan is one of the six countries that contribute to more than 50% of all maternal deaths worldwide (Hogan 2010). To address the issue of high maternal and infant mortality rates, the government of Pakistan created a new cadre of community based midwives (CMW) (PC1, 2005). Pakistan is also characterized by difficult terrain and political unrest in some provinces. The Balochistan province in particular poses problems for health services delivery given vast area (44% of Pakistan’s territory) with only 5% of the nation’s population. One expectation is that CMWs will improve access to skilled antenatal and intra-partum care for the women living in remote areas in Balochistan.

Recent research shows that these CMWs have largely failed to establish their midwifery practices. Four years after deployment of the first batch of midwives, 11.7 % of deliveries in district Jhelum and 3% in district Layyah were attended by a CMW. A number of reasons have been identified for this sub-optimal performance. These include:

1) The community, in particular the women, both young and old, do not trust the CMWs abilities to conduct a safe birth. There is a common perception that CMWs receive poor training, and as a consequence do not receive confidence from community members. In more remote districts where traditional birth attendants (dais) are still the dominant providers, the CMWs are understood to be 'doctors' who should be consulted in the event of a birth complication only. The CMWs however, fail to meet such lofty expectations.

2) The CMW's demonstrate a distinct lack of interest in pursuing midwifery practice, primarily because a) They are unable to meet the professional demands of midwifery practice in remote rural areas. Most are young women. Gender norms of rural Pakistani society preclude their travel alone to homes located at significant distances. During day time they need a chaperone, preferably an older woman and at night at least two men in addition to a woman. This greatly increases the opportunity costs of a CMW's practice.

b) A number of the trained CMW's were unclear of what a midwife was and what she did when they applied for training. Upon graduation, they realized this was not a career path they were interested in pursuing.

c) Income from patient remuneration was insufficient for establishment of self-sustaining practices.

3) The CMWs lack business skills necessary to establish private practices.

4) CMWs lack the financial resources essential for the development of practice infrastructure and logistics¹.

To address some of these issues, USAID through Mercy Corps will implement the Improving Mother and Newborn health (IMAN) intervention in districts Quetta, Gwadar, and Kech in Balochistan. At 785 per 100,000 live births, Balochistan has the highest maternal mortality rates in Pakistan. Balochistan also has some unique needs: it is the largest province of Pakistan, compromising of 44% of the landmass, but houses only five percent of the population². A sparse population dispersed over a large area complicates service delivery, especially for women in remoter regions. A separatist insurgency has created security concerns that further hamper maternal health care services provision.

The IMAN initiative will deploy 90 private-sector CMWs in the three Balochistan districts who will be responsible 5000 population each. To enable the CWMs to establish self-sustaining home-based private practices, they will be provided small loans, standard equipment and business skills training. A micro-finance institute will offer small loans in installments to the CMWs through mobile phones using Telenor’s Easy paisa service. To increase demand for biomedical skilled birth attendance, the CMWs will liaise with local lady health workers (LHWs) to undertake an awareness-building campaign using cellular phone SMS technology and through existing women’s support groups. To increase access to emergency obstetric care, the CMW's with communities to develop a revolving transport fund called the Mamta fund. Mercy Corps will also collaborate with the Pakistan Department of Health to develop a five year MNCH plan, which will include a surveillance plan. See appendix 1 for a logic model of the intervention.

The proposed research aims to investigate whether the CMWs in the IMAN initiative, have increased coverage of essential maternal and newborn health care to women and children living in remote Balochistan districts of Quetta, Gwadar, and Kech Districts in a financially self-sustaining manner. Specifically the research will investigate the following questions:

2. Research Questions:

1. Is the IMAN initiative having an impact on CMW service uptake? Has the initiative increased coverage of the full scope of skilled maternity and newborn care for women living in the interventions districts?

2. If there is an increase in CMW service uptake, is this increase attributable to the IMAN initiative?

3. If there is no change in CMW coverage, is this due to flawed program design or poor program implementation?

4. Have the CMWs become self-sustaining, quality private providers?

(a) Have the financial resources provided been helpful in allowing the CMWs to become self-sustaining? Are the loans being used as intended? How exactly are the loans enabling CMWs to establish their practices?

(b) Has the business skills training enabled the CMWs to establish viable practices and provide high quality care?

(c) What are the barriers hindering CMWs from becoming self-sustaining private health care providers.

5. What is the quality of care the CMWs provide, both from perspectives of evidence-based best practice and women receiving services?
3. Study objectives

Empirical objectives: The fundamental objectives of the research are:

1. To develop evidence that the IMAN imitative has led to increased coverage of high quality maternal and neonatal health care by trained, private-sector community midwives in remote, sparsely populated, insecure districts of Balochistan.

2. To document if the IMAN initiative was implemented as planned.

3. To explore whether CMWs access to business skills training, small loans, and infrastructural support enabled them to develop financially sustainable private midwifery and neonatal practices. To enhance empirical understanding of the ways in which these processes operate.

4. To map women’s experiences of maternal and neonatal health care provided by the CMWs, specifically their perceptions of the quality of care provided and ability to access to emergency maternal and neonatal care.

Knowledge transfer objective:

1. To facilitate uptake of the research findings to inform positive developments in maternal health policy, service design and care delivery in Balochistan, Pakistan more generally and elsewhere.

2. To contribute to the evidence-base of innovative maternal and neonatal health care provision by community-based health care providers.

4. Design and Methods

A mixed methods approach using both quantitative and qualitative methods will be used. Data will be collected in four overlapping modules over a four year period.

1. 
   Module 1 will address research objective 1 will consist of a cluster randomized control trial.
   
2. 
   Module 2 will address research objectives 2 and will consist of a process evaluation.
   
3. 
   Module 3 will address research objectives 3 using a quantitative financial tool.
   
4. 
   Module 4 will address research objective 4 to assess the quality of CMW services using a range of quantitative and qualitative inquiry methods.
   

Figure 1: Diagramatic of Research Design
5. Methods and Measurement of Variables
5.1 Module 1:

Module I will consist of a Cluster Randomized Control Trial (RCT). A RCT is a method of choice for this research for it will enable us to assess if the IMAN initiative has changed coverage of the full scope of maternity and newborn care for women living in the intervention districts and if this increase is attributable to the IMAN initiative. A RCT is the most rigorous way of determining whether the intervention actually produced the intended effects (Rossi 2004). Other quasi-experimental study designs, including pre and post-intervention surveys, can only suggest that the intervention produced the measured outcome. They cannot rule out the possibility that other forces, such as background secular changes, may have led to differentials in outcome (Rossi 2004).

A baseline and end-line survey within a RCT framework will be conducted in both the intervention and control sites. This data will elucidate if trends of CMW service uptake changes before and after the implementation of the IMAN initiative and whether any changes are attributable to the IMAN initiative.
5.1.1 Setting and Population: The study will be conducted in 4 districts in Balochistan, two IMAN districts and two control districts. The two randomly selected IMAN intervention districts are Quetta and Gwader and the control districts are Sibi and Kharan. The criteria for control districts is that they be separated from IMAN districts by at least one district to prevent contamination, but close enough to be similar socio-economically. The target populations will be women aged 15-49, who gave who have given birth in the three years prior to the survey.
5.1.2 Sample size and sampling: Sample size was calculated using the statistical package PASS 2012³ program and is based on CMW coverage rates in districts Layyah and Jhelum, Punjab. Assuming the Layyah CMW coverage rates of 3% are baseline coverage rates in all the study selected districts in Balochistan, we estimate group sample sizes of 760 in group one (IMAN intervention districts) and 760 in group two (control districts) achieve 90% power to detect a difference between the group proportions of 0.0500. The proportion in group one (the IMAN districts) is assumed to be 0.05 under the null hypothesis and 0.1 under the alternative hypothesis. The proportion in group two (the control group) is 0.05. The test statistic used is the two-sided Z test with pooled variance. The significance level of the test was targeted at 0.05. Since cluster sampling will be done, this sample size takes into account a design factor of 1.3.

A cluster in this research will consist of a CMW and her catchment area. To interview 760 women in each arm (total 1520 women), we estimate 36 clusters per arm will be required (total 72 clusters). The sampling frame will be drawn up using a three stage cluster sample design (see fig 2). In the first stage, two tehsils will be randomly selected in all study districts. In Quetta District, which only has two tehsils, both will be selected. In the second stage, 9 union councils will be randomly selected in each selected tehsils (total 36 unions councils). In the third stage, two CMWs (and their catchment areas) will be randomly selected from each selected union council (total 72 CMW clusters). Within each selected cluster, 11 women who gave birth in the three years prior to the survey will be interviewed. ⁴ The first house to be contacted will be randomly selected from the LHWs registers. Every second house will be visited. We estimate that we need to contact at least 124 households per cluster who gave birth in the 3 years prior to the survey. This is based on a demographic assumption that 25% of households will have a woman aged 15-49, 30% of whom gave birth in the last three years¹ and a participation refusal rate of 15% (all conservative estimates) (DHS 2007).

Figure 2: Randomized Control Design Sampling Framework

Sampling Design for IMAN districts (and control districts)

*The two districts were chosen randomly as described in section 5.1.1. A sample design will be used for control districts, which have been selected as described in section 5.1.1.
5.1.3 Data Collection: Data will be collected using a pre-tested questionnaire that will include a set of questions that will capture the full range of essential maternal and neonatal health care. This will include measures of ANC coverage, skilled birth attendance, maternal TT Vaccination, postnatal care, modern contraceptive use, cord care, thermal control, early initiation of breastfeeding, exclusive breastfeeding and socio-demographic characteristics (including indicators of social exclusion) . Women's satisfaction with CMW care will also be assessed. Information will also be collected around the financial costs of seeking CMW care, other social and geographic barriers to seeking her care and the degree and ease of access to Mamta funds. If the woman cannot be interviewed at the time of contact, a mutually agreed upon time and place will be scheduled for another visit and interview. Women will be interviewed by female enumerators. It is anticipated that most interviews will last approximately 45 to 60 minutes.
5.1.4 Data analysis: Data will be analyzed using Stata 12.0, using the program’s ‘svy’ set of commands that take into account the design effect of clustering and stratification.⁵ Univariate and bivariate analyses will be done to assess the proportions of births that were provided a full range of essential maternal and newborn care by the CMWs at baseline and endline. Logistic regression models will then be developed to estimate the odds of attendance by CMW in the IMAN districts compared to control districts, controlled for potential confounders. Are women in IMAN districts more or less likely to use a CMW compared to women living in control districts? How satisfied are women with the care provided by IMAN CMW’s compared to control CMWs? Is there any variability in birth outcomes between the two groups of women under the care of a CMW?

5.1.5 Personnel: A core team of 8 data collectors (6 women and 2 men), and 1 data entry clerk will be trained. We need to hire and train more women than men because female interviewers will the primary data collectors. The male data collectors will interview the husbands or other men if the women are unable to answer all our questions. All enumerators will be trained in the technical details of survey data collection and in cultural sensitivities of the local context (how to approach people, their mannerisms and language, that their personal attire should align with the local cultural expectations and not be offensive). The research manager will oversee the entire survey, ensuring correct coverage and quality of the data. A quality monitoring check list will be used to ensure the quality of data collected. The PI will review the work on a daily basis and monitor data quality weekly. Completed questionnaires will be edited and entered into Stata 12.0 using a double-entry system.

5.2 Module 2:

Module 2 will verify whether the IMAN initiative is delivering services to the targeted participants as intended. A process evaluation will be conducted. This will consist of a systematic and continual documentation of key aspects of program performance (Rossi 2004). Specifically, we will assess whether the following program functions are being performed in compliance with the program plan.

1. Have the CMW's received the planned business training? What proportion of the targeted CMWs successfully completed the required training?

2. Have the CMWs received equipment and access to micro-finance loans. Are they using the loans for the intended purposes?

3. Are the CMW-LHWs partnerships aimed at awareness-building functioning and are they carrying out the planned activities? Are these activities appropriate and doable?

4. Has a Mamta fund being established? What are its mechanisms for collection and disbursal of funds; who has access to these funds; are there people excluded from Mamta; what is the sustainability of the fund?

5.2.1 Data Collection: Questions 1-3 will be answered using the sample of 36 IMAN CMWs (selected in module 1). They will be interviewed in-depth and asked to fill a pre-tested questionnaire at 6-month intervals throughout the project. Question 4 will be addressed using focus group discussions with community members in our IMAN study clusters. Nine FGD will be done annually, 36 in total over the 4-year period. Each focus group discussion will be conducted with 6-10 participants in each, separately for women and men.⁶ Representation of all socio-economic groups will be ensured. Program Records will also be used.

5.2.2 Data analysis Since the interviews and focus groups data will be collected in Balochi and Pushto, data will be translated and transcribed by native speakers. A database of the transcribed interviews , focus groups, and observation notes will be created in Atlas-ti, a qualitative data analysis software program.⁷ Using a social constructivist, interpretative approach⁸, data will be coded and broad themes identified. Initial coding will be guided by the stated research objectives and later by additional concepts as they emerge. Data analysis will be an on-going and iterative process through all phases of data collection, as early identification will allow a fuller probing of unanticipated concepts and variables in upcoming formal or informal interviews and focus group discussions.⁹ Interpretive accuracy will be assessed by triangulation of findings across the four phases, peer debriefing within the research team and other colleagues and respondent validation.

5.2.3 Personnel. The research team for Modules 2 will be comprised of an experienced Anthropologist (Master’s degree holder, woman) and two research assistants (Anthropologist – Master’s degree holder, man).

5.3 Module 3

Module 3 will address research objective 3. It will explore the financial sustainability of the IMAN CMWs midwifery practice. Specifically we will assess if the IMAN initiative of providing the CMWs with business skills training and micro-credit loans has enabled the CMWS to establish self-sustaining practices. A quantitative financial tool will be used to assess the direct and indirect monetary costs as well as opportunity costs and benefits. These costs include financial requirement of establishing practices and maintaining equipment. In addition, we will evaluate the temporal and monetary costs of organizing chaperones (i.e. depending on female family members to accompany young CMWs to deliveries during the day and two or more male chaperones for night deliveries) and private transportation fees. A very important contribution of the CMWs will be the establishment and maintenance of the Mamta Fund which will pay for emergency transportation services, such as ambulances, which are essential for effectively managing complications during delivery. The size, sustainability and effectiveness of the Mamta fund will also be assessed.

5.3.1 Data Collection and Analysis: Data will be collected using a pre tested financial tool. 36 IMAN CMWs who were randomly selected in Module 1 will be interviewed. Data will be collected in three intervals of time; first at baseline, second in year 2 and finally at the end of year 4. Direct or indirect costs shall be calculated on market rates.

5.3.2 Personnel: An individual trained in financial analysis will collect and analyze the data.

5.4 Module 4:

Module 4 will address research objective 4. It will assess the quality of CMW services by mapping women’s experiences of maternity and newborn care, specifically skilled attendance at birth in both normal and emergency obstetrical scenarios.

5.4.1 Methods: Drawing upon the Hulton, Mathews & Stones (2000) framework for assessing quality of maternal health services in developing countries,¹⁰ both women’s experiences of care and key aspects of care provision will be examined. Women’s experiences of maternity care will include an assessment of their impression of the adequacy of CMWs human and physical resources, their trust in her competency to provide safe birth attendance, identify birth complications (or potential for complications), manage an emergency and provide the necessary referral services. The respect they are accorded, their sense of dignity, equity, and the emotional support they receive from the CMW will also be assessed.

Key aspects of CMW quality of care will also be assessed. This will include CMWs knowledge and competencies for a normal birth and identification of birth complications against internationally recognized standards.¹¹ The quality of their referral links and information management systems and the use of appropriate technologies in caring for women will also be assessed.

5.4.2 Data collection: Specifically, the following will be done:

1. Observation and documentation of 12 CMW birth facilities, both physical (e.g., whether the design of the labour room respects women’s privacy) and instrumental (e.g., Availability of instruments, their quality, availability of essential drugs and supplies) using a structured checklist.¹²

2. Observation of 25 moment-in-time CMW-patient interactions. This will include an observation of at least 5 births overall. The points of observation will include, but not be limited to, the provider’s language and behavior toward the patient and her family, time spent with each patient and whether her concerns are addressed. Any complications or adverse outcomes will be explored in depth, which will include interviewing the woman, her family and the CMW.

3. Exit interviews with ten women who delivered in CMW practices (5 routine deliveries and 5 who required emergency obstetric care; 10 interviews total).

4. In-depth interviews with 20 CMWS and 5 referral physicians, both in the public and private sector.

5.4.2 Data analysis. The qualitative data will be analyzed as described in 5.2.2 above.

Table 1: Variables Measured for Different Initiative Levels

Data Collection Method	CMW	Mother	Child	Other Health Service Providers?	Policy-Makers/Program Managers	Community
Module 1: RCT(Baseline and end-line surveys)		-Coverage/receipt of CMW services -Quality of CMW services received (i.e. what interventions and education did the mother receive?)	-Coverage/receipt of CMS services -Quality of CMW services received (which interventions were received; how many visits took place?)
Module 2: Program process evaluation (in-depth interviews, focus group discussions, quantitative tool)	-CMWs completed business training _CMWs using micro-credit for intended purposes			- CMW-LHW partnership conducting awareness building activities	CMWs received equipment and access to micro-credit	Establishment and availability of Mamta Fund -
Module 3: (Financial quantitative tool)	-Direct and indirect costs of a self-sustaining practice. -Opportunity costs of CMW practice					Size, sustainability and effectiveness of the Mamta Fund
Module 4 Quality of CMW care (Observations, in-depth interviews)	Documentation of physical and instrumental Infrastructure -Quality of CMW-patient interaction (CMW behaviour, language, time spent) -Handling birth complications from clinical evidence-based best practices perspective	-Assessment of CMW quality of care received in terms of respect accorded, sense of dignity, equity, and emotional support -Reasons for choosing CMW	Assessment of CMW quality of care received in terms of knowledge of newborn care	-Physicians, LHW perspectives of CMW competency - trust and respect given to CMWs -Provide referral back up to CMSs practice	-expectations of CMWs -barriers faced at policy level to implementing IMAN initiative

6 Research team: The project will be implemented and managed by

1. Dr. Zubia Mumtaz, MBBS, MPH, PhD (Public Health Medicine) (PI), Alberta Heritage Foundation for Medical Research Population Health Investigator and Assistant Professor, School of Public Health, University of Alberta, Canada. She specializes in women's reproductive health with a particular focus on women’s access to reproductive health services and inequities in reproductive health policy, design and delivery of services. Dr. Mumtaz has 15 years of Anthropological and Epidemiological research experience and extensive first hand data collection experience in remote, rural and insecure environments in Pakistan. The present proposal extends one stream of her currently ongoing research project in partnership with the Punjab MCH program (funded by RAF) that is exploring if CMWs in Punjab are providing care to the poor and socially excluded women. As the PI, Dr. Mumtaz will be responsible for the conceptual guidance of the research, including development of research tools, overall project management, ensuring data quality, data analysis and report writing. She will be responsible for ensuring the research complies with all health and ethical requirements. She will be responsible for all correspondence, record keeping, reports and financial monitoring.

2. Afshan Bhatti, Msc Anthropology and MBA is the Research Manager and Anthropologist coordinating University of Alberta’s research projects in Pakistan. Ms. Bhatti has 10-years of extensive first hand ethnographic and survey data collection experience in remote, rural environments. The proposed research extends her past successful management of many of Dr. Mumtaz’s research projects in Pakistan. Ms. Bhatti will manage the proposed project and supervise data collection.

7. Ethical Considerations

Maintaining privacy and confidentiality of all the study participants is an important ethical issue. This will be addressed by only collecting the necessary information.¹³ Only the researchers and project manager will have access to the list of participants’ names and associated codes. The hardcopies of the survey questionnaires, the qualitative data transcripts and the consent forms will be kept in a locked cabinet in Dr. Mumtaz’s office at the University of Alberta for five years. Ethical approval will be obtained from the University of Alberta Health Research Ethics Board and Pakistan Bioethics Board¹⁴

8. Strengths and Limitations of the Study:

A key strength of our study is the use of randomized controlled methodology to assess the impact of the IMAN intervention. Randomization and comparison with controls will allow us to conclude, validly and without bias, whether or not any changes in CMW coverage and quality of care can be attributed to the IMAN initiative. Figure 3 (below) visually demonstrates the importance of teasing apart program effect from background secular changes in outcome.

Fig 3: Outcome level, outcome change and net effect

Another strength of our research is our use of process evaluation (Module 2). A process evaluation combined with a randomized control trial will allow us to identify, precisely, whether the IMAN program strategy (or program theory) led to program outcome (success or otherwise). The third strength is our use of multiple mixed methods research. A key strength of this project is the research teams mixed methods expertise and extensive experience conducting research on the Pakistani CMW program in remote, rural areas.

A limitation of our research is that the RCT is a time-consuming and expensive exercise. It is however, the gold-standard in identifying causality. However, it is important to document causality to ensure effective use of tax-payer dollars. Another limitation is the large scope of the study, which might be expensive to conduct in Balochistan given its low population density and security concerns. There is also the issue of the availability of trained human resources in Balochistan, especially the availabiltiy of personnel with qualitative research method skills.

9. Stakeholders/Partners Engagement Plan Specific for the OR Study:

The key stakeholders of this research are the

1) The Balochistan Department of Health including the Balochistan Maternal and Neonatal Child Health program. The Balochistan government has given top priority to provide the full range of essential maternal and neonatal care to pregnant women living in remote rural areas. They have trained and deployed the CMWs.

2) Mercycorps and the Aga Khan Foundation as the key implementers of the project

3) USAID as the funding body.
One representative of each organization will constitute the Research Advisory Committee (RAC). All RAC members will be engaged in the project from the start. We envision their contribution to include (1) ensuring the research design meets their policy, program and information needs (2) input in development of data-collection tools (3) data collection support, particularly regarding security issues in Balochistan (4) vValidation of research findings.

10. Timeline: See next page
References

1^ Mumtaz, Z, O'Brien, B, Bhatti A and Jhanjri G (2013) Are community midwives addressing the inequities in access to skilled birth attendance in Pakistan? Gender, class and social exclusion. Data analysis on-going.

2^ Government of Pakistan, PCI (1998) 1998 Census. Population Census Organization. Islamabad.

3^ PASS 2012: Power Analysis and Sample Size for Windows. NCSS, Kaysville, Utah.

4^ Aliaga, A. and Ren, R. (2006) Optimal sample sizes for two-stage cluster sampling in Demographic and Health Surveys. DHS Working Papers, No.30. ORC Macro.

5^ STATA (2008) Stata Corporation, College Station, Texas.

6^ Fern, E.F. (2001). Advanced Focus Group Research. Sage Publications, Thousand Oaks, Ca. and London.

7^ Muhr, T. (2001). Atlas-ti. The Knowledge Workbench. http://www.atlasti.de/index.html.

8^ Hammersley, M. (1998) Reading Ethnographic Research. Longman, London and New York.

9^ Tuckett AG. (2005). Part II: Rigour in qualitative research: complexities and solutions. Nurse Researcher 13(1): 29-42.

10^ Hulton, L.A., Matthews, Z., and Stones, R. W. (2000). A framework for the evaluation of quality of care in maternity services. Southampton: University of Southampton.

11^ Afsana, K. and Rashid, S. (2001) The challenges of meeting rural Bangladeshi women's needs in delivery care. Reproductive Health Matters 9(18):79-89.

12^ Abbott, S., Shaw, S., and Elston, J. (2004) “Comparative analysis of health policy implementation: the use of documentary analysis” Policy Studies 25(4):259-266.

13^ Ells C & Gutfreund S. (2006). Myths about qualitative research and the tri-council policy statement. The Canadian Journal of Sociology 31(3): 361-373.

14^ University of Alberta (2011) Human Research Human Research Ethics Policy. [Accessed February 7, 2013]. Retrieved from: https://policiesonline.ualberta.ca/PoliciesProcedures/Policies/Human-Research-Ethics-Policy.pdf