Monitoring
Overall guidelines:
2006 UNAIDS's High Coverage Sites: HIV Prevention Among Injecting Drug Users in Transitional and Developing Countries, Case Studies
2007 WHO/UNODC/UNAIDS's Guide to Starting and Managing Needle and Syringe Programmes
2008 UNAIDS's A Framework for Monitoring and Evaluating HIV Prevention Programmes for Most-At-Risk Populations
2011 USAID/UNFPA/UNDP/UNAIDS/GFATM's Operational Guidelines for Monitoring and Evaluation of HIV Programmes for SWs, MSM, and TG People - Volume I National and Sub-National Levels
2011 UNAIDS's Operational Guidelines for Monitoring and Evaluation of HIV Programmes for PWID: Monitoring and Evaluation at the national and sub-national level or at the service delivery level
2013 WHO/UNODC/UNAIDS's Technical Guide for Countries to Set Targets for Universal Access to HIV Prevention, Treatment and Care for IDUs, 2012 revision
2014 WHO's Consolidated Guidelines on HIV Prevention, Diagnosis, Treatment and Care for Key Populations, page 128
2015 WHO's Tool To Set And Monitor Targets For HIV Prevention, Diagnosis, Treatment And Care For KP - Supplement To The 2014 Consolidated Guidelines Starting at page 38
Guidelines related to specific supplies:
2005 UNFPA/WHO/PATH's Condom Programming for HIV Prevention: A Manual for Service Providers page 35, step 5. Check Progress, Monitor the Quantity of Condoms Distributed and Clients Served
2009 WHO's Toolkit for Monitoring and Evaluation of Interventions for Sex Workers
2015 WHO's Tool To Set And Monitor Targets For HIV Prevention, Diagnosis, Treatment And Care For KP - Supplement To The 2014 Consolidated Guidelines, Overdose and Prevention Management, page 48
Step 1. Estimating the total key population (denominator)
Current and accurate size estimations are needed (at national and sub-national levels) for several reasons:
- Ensuring that the harm reduction intervention does not miss hidden population and is comprehensive in its reach
- Taking into account the changes in population and/or practice.
Sub-national data collection and estimations, preferably at the level of program planning and implementation, may be a better way to proceed. Epidemics are rarely spread across a country uniformly, and prevention programmes, ideally, should be implemented where they can have the greatest impact on the epidemic.
Population estimate (described in the RAR-IDU Guide) should be done at least every two years.
For more details : 1998 WHO’s Rapid Assessment and Response Guide on Injecting Drug Use (IDU-RAR)
2004 WHO’s Rapid Assessment and Response Guide for Work with Especially Vulnerable Young People.
2007 WHO/UNODC/UNAIDS’s Guide to Starting and Managing Needle and Syringe Programmes, page 33, chapter D: Measuring Coverage
2008 UNAIDS’s A Framework for Monitoring and Evaluating HIV Prevention Programmes for Most-At-Risk Populations, page 27, chapter 3: Establishing the Size and Nature of the Problem
2013 WHO/UNODC/UNAIDS’s Technical Guide for Countries to Set Targets for Universal Access to HIV Prevention, Treatment and Care for IDUs, 2012 revision, page 29, chapter 3.2: Measuring Population Size
a. Going further into disaggregated denominators
Collate and analyse data from a range of sources (including data from Public Health agencies) to build reliable local estimates of the types of drugs used and the numbers, demographics, geographics, and other characteristics of PWUD:
- Number of young people aged under 18 who are injecting, or being injected
- Number of male vs. female PWUD
- Number of opioid PWUD
- Number of people injecting other drugs and/or with poly-drug use
- Number of people who inject image- and performance-enhancing drugs
- Number of new psychoactive substance injectors
- Number of people who inject occasionally, for example, when they go to night clubs
- Other overlapping groups, such as men who have sex with men, prisoners or ex-prisoners, female or male sex workers, transgenders, truck drivers, migrant population, or homeless people.
- Etc.
Note : Overlapping circles or Venn diagrams can help mapping these sub-populations and their overlap.
For more details : 2014 National Institute for Health and Care Excellence's Guidelines for Needle and Syringe Programmes, Uk, Recommendation 2 : Collate and Analyse Data on Injecting Drug Use
Step 2. Estimating the population reached (another possible denominator)
There is a need of Unique Identifier to prevent duplication across services/geographic location/interventions. Example: The same new PWUD accessing the mobile van and a fixed site might be accounted as new reach by both interventions.
For more details :
2007 WHO/UNODC/UNAIDS's Guide to Starting and Managing Needle and Syringe Programmes, page 17, chapter F: Standardized Monitoring Processes page 33, D. Measuring Coverage
2007 Unique Identifier Code. USAID-Funded Drug Demand Reduction Program in Uzbekistan, Tajikistan and The Ferghana Valley Region of Kyrgyzstan. DDRP Best Practice Collection.
2013 WHO/UNODC/UNAIDS"s Technical Guide to Set Targets for Universal Access to HIV Prevention, Treatment and Care for IDUs, page 36, chapter 3.4, sub-chapter Unique Identifier Code
Step 3. Computing the number of paraphernalia ordered, or distributed/consumed (nominator)
The number of supplies ordered is a good proxy for the supplies distributed/consumed, if distribution/consumption logs do not exist or are not complete. The difference between supplies ordered and distributed/consumed is the remaining current stock (in the program’s storeroom, in the van and in outreach workers’ bags).
For more details :
2007 WHO/UNODC/UNAIDS's Guide to Starting and Managing Needle and Syringe Programmes,
page 11, chapter C: Stock Orders, Storage and Disposal (collecting data of supplies ordered or used)
page 43, Annex B: Stock Management Form
page 17, chapter F: Standardized Monitoring Processes (to collect data of supplies handed out)
page 47, Annex E: Activity Reporting Form
page 34, chapter E: Using Monitoring and Evaluation for Advocacy And Planning
Step 4 Setting indicators (could be both nominator and denominator)
Note : The idea is to have a selected number of prcise, easy, and useful indicators, instead of numerous imprecise, cumbersome, useless ones.
The useful indicators need to be tested and really help to know whether harm reduction programs have problems, can indentify early problems, changes in trends, etc.
a. Setting indicators related to clients
Ideally, these indicators should be reported from the sites on a monthly basis and be immediately analysed to support evidence based decision making. Donors tend to request quarterly or yearly basis.
- Comparative coverage : Number of needles & syringes supplied/IDU/year (based on the estimated total number of PWID in a specific geographic area).
Note: In case of provision of detached needles and/or detached syringes, take the lowest number. Rationale is that each injection should be performed with a new needle AND a new syringe to fully prevent disease transmission. It is also not overstating and therefore more conservative.
- Service utilization : Number of needles & syringes/unique NSP client/year
- “Ever reached” coverage : Number of PWUD ever reached by the programme.
But as a one-off NSP visit does not provide any significant behaviour change, this figure actually only indicates the reach of the programme.
- “% of IDUs in regular contact”: It means “at least once a month over a period of 12 months”.
- Other: forthcoming
For more details :
2008 UNAIDS's A Framework for Monitoring and Evaluating HIV Prevention Programmes for Most-At-Risk Populations, page 49, Chapter 6: Monitoring Programme Uptake and Coverage
2013 WHO/UNODC/UNAIDS's Technical Guide for Countries to Set Targets for Universal Access to HIV Prevention, Treatment and Care for IDUs, page 45, chapter 4: Indicators pages 50 to 58, sub-chapter 4.3: the Comprehensive Package, 1. Needle and Syringe Programmes
page 83 sub-chapter 6: Condom Distribution Programmes for PWUD and Their Sexual Partners.
b. Setting disaggregated denominator with specific population or specific localisation
Level of application: These indicators can be applied at the national, regional, city, district or service level. In each case the PWUD population estimate used as the denominator should be relevant to the area examined.
Examples of disaggregation :
- disaggregation by gender (male, female, transgender)
- disaggregation by age (≤ 18 years, >18 years and <25 years, and ≥25 years)
- disaggregation by type of drug injected (opioids, stimulants, other)
- disaggregation by sub-recipients, and by outreach worker
- disaggregation by geographic area
- disaggregation by delivery mode (fixed sites, mobile, outreached, vending machine, pharmacy, prison, etc.
For more details :
2013 WHO/UNODC/UNAIDS"s Technical Guide for Countries to Set Targets for Universal Access to HIV Prevention, Treatment and Care for IDUs, page 41, 3.7 Disaggregation
c. Setting disaggregated denominator with specific supplies
i. Injecting equipment
- Young people aged under 18 : Might use thinner needles and low capacity syringes
- Female PWUD : Might use thinner needles (smaller veins) and low capacity syringes
- Injecting other drugs : Specific drug might required specific needle and/or syringe, sterile water for injection, cookers & lighters, or acidifiers (black tar)
- Other overlapping groups of key populations : Additional paraphernalia such as incertive or receptive condoms, lubricants (female or male SWs, MSM)
- Specific sub-groups that can only be accessed in specific localisation
- Allocate special outreach workers to special sub-groups (female workers for female PWUD)
- Other : forthcoming
Note : This is where, offering adequate, distinct, and comprehensive paraphernalia (single or bundled/brown bag) that meet all needs, might help creating data and monitoring sub-population.
ii. Naloxone and opioid users
- Number of harm reduction sites providing training and Naloxone / number of harm reduction sites
- Number of opioid users trained and equipped with Naloxone / number of opioid users
- Number of methadone patients trained and equipped with Naloxone / number of methadone patients
- Number of opioid users' family/friends trained and equipped with Naloxone / number of opioid users
- Number of police officers/first responders trained and equipped with Naloxone / number of police officers/first responders
- Number of liberated prisoners trained and equipped with Naloxone / number of prisoners liberated
- Number of patients leaving detox or other drug free treatment centers trained and equipped with Naloxone / number of patients leaving
- Number of pharmacies trained and equipped with Naloxone / number of pharmacies
- Number of methadone clinics trained and equipped with Naloxone / number of methadone clinics
- Number of persons in need of replacement (depend on the shelve life, % of confiscation/breakage/lost - in each of the category above)
- Number of opioid overdoses averted / number opioid overdoses
For more details :
2015 WHO's Tool To Set And Monitor Targets For HIV Prevention, Diagnosis, Treatment And Care For KP, Overdose and Prevention Management, page 48
iii. Police interference
- Number of instances of positive police interactions with PWUD and/or harm reduction staff (including referral to health services and/or harm reduction services)
- Number of instances of PWUD’s paraphernalia destruction/confiscation by law enforcement
- Number of instances of PWUD being harassed/arrested/detained for paraphernalia possession
- Number of instances of outreach staff being harassed/arrested/detained for paraphernalia distribution
- Number of supplies and types confiscated or desctroyed
- Etc.
Note: This requires a reporting mechanism for police incidence, and training of program staff and clients.
For more details : forthcoming
d. Going further into breadth by disaggregated indicators
- “Injection coverage”: Percentage of injections covered by sterile needles and syringes in each of the sub-groups.
Note : Phe number and percentage of occasions when new sterile equipment was available to use for an injection.
Example : The Integrated Biological and Behavioural Surveillance states up to 3 injections daily in average. If PWUD receive 2 needles and syringes/day/year in average, then “Injection coverage” is 2/3 = 67% of daily need.
- “Pairing coverage”: Tracking if the following pairing as been respected 100% of the time.
Example : 1 syringe with 1 needle with (minimum) 1 swab with 1 cooker with 1 sterile water for injection, etc.
- “Geographic coverage” : Are all key population’s localities being covered?
- Other: forthcoming
For more details : 2014 National Institute for Health and Care Excellence's Guidelines for Needle and Syringe Programmes, Uk, Recommendation 2 : Collate and Analyse Data on Injecting Drug Use
e. Setting indicators that can be related to staff distributing paraphernalia
Such indicators can be tracked to inform about staff effectiveness and adherence to protocol :
- “Pairing coverage” per outreach worker/intervention type
For example : Maybe the mobile van driver keeps forgetting to propose sterile water for injection and therefore break the comprehensiveness of paraphernalia offering
- “Injection coverage” per outreach worker
For example : Some outreach workers might be better at handing out paraphernalia
- Naloxone training and distribution
Other: forthcoming
Step 5. Setting indicator targets
a. Reaching more than 60% of a locality’s PWUD on a regular basis but 90% is the new established target of international agencies :
- Will have an impact,
- Should reduce overdose rate,
- Should help prevent an HIV/ HBV/HCV epidemic,
- Or bring an existing epidemic under control.
Reference: Extremely low and sustained HIV incidence among people who inject drugs in a setting of harm reduction
b. The coverage of NSPs will need to be greater in localities where:
- HIV prevalence among PWUD is already high,
- The PWUD population is concentrated,
- Needles and syringes cannot be accessed elsewhere, for example from pharmacies,
- There are few drug treatment services (no or little opioid substitution therapy available),
- There are few effective programmes addressing prevention, STIs and other health problems among PWUD.
For more details :
2013 WHO/UNODC/UNAIDS's Technical Guide for Countries to Set Targets for Universal Access to HIV Prevention, Treatment and Care for IDUs, page 42, chapter 3.8: Setting targets
Step 6. Collecting and aggregating disaggregated data
a. Organizing monitoring
i. Create an M&E team with at least 1 M&E analyst
ii. Set a Unique Identifier Code protocol common to all harm reduction programs and preferably including HIV and methadone programs
iii. Develop a template (file, googledoc, survey monkey, web-site, software) with the adequate fields to collect data from the harm reduction sites and report to the M&E team
iv. Write a Standard Operating Procedure to collect, collate, analyse, and report data including
1.precise deadlines for sites to report on time and complete
2.deadline for M&E analyst to collate, analyse, and present to management so that decision can be taken
3.deadline for management to report analysis and decision back to harm reduction sites
v. Equip with log books for harm reduction sites and outreach workers
vi. Train staff of pilot sites
vii. Pilot for a couple of months, adjust when necessary
viii. Scale up to all harm reduction sites
b. Collating and analysing disaggregated data
i. The best collection tools (web-base, software, etc.) would automatically aggregate data and might provide graphics. Other tools (MS Excel files, googledoc, survey monkey, etc.) will require a copy/paste in a receptive MS Excel file.
ii. The receptive MS Excel file should be simple with 1 input tab, and several output tabs with pivot tables and graphics for monthly and cumulated results against targets. Ideally it should be printed and distributed during the monthly M&E meeting.
iii. Pasted data should be controlled against outliers, errors, etc.
iv. Analysis should then take place. If needed, specific harm reductions sites can be contacted to provide detailed explanations for variances
c. Presenting output to harm reduction program management and reporting back
i. Ensure rapid presentation
ii. M&E analyst recommends alternative actions
iii. Management decides and concludes, notes are taken
iv. Analysis and decision are reported back to harm reduction sites’ managers
Note : The methods used need to be consistent across all sites. In most countries, however, these data are not collected by a single agency and not centrally organized or collated. Having a single, national-level agency responsible for regularly collating and reporting national data is highly advantageous.
Data must remain confidential and that data and medical records identifying people as users of drugs are not shared with law enforcement agencies, particularly in countries where drug use is criminalized.
For more details :
An example of a system collating data from different services is described in the following report: 2011, HIV/AIDS Asia Regional Program Technical Support Unit, Yunnan Provincial HIV/AIDS Prevention and Control Bureau, Yunnan Public Health Bureau. Making it count: using online data for effective monitoring and evaluation of harm reduction activities in Yunnan Province, China.
2012, Innovative data tools: a suite for managing peer outreach to key affected populations in Viet Nam
2013 WHO/UNODC/UNAIDS"s Technical Guide for Countries to Set Targets for Universal Access to HIV Prevention, Treatment and Care for IDUs, page 37, chapter 3.4, sub-chapter : Collecting and Aggregating Data from Programmes
Step 7. Monitoring indicators and taking action when needed
Remember that monitoring is only an intermediary step, the goal is to take evidenced based actions.
For more details :
2006 UNAIDS's High Coverage Sites: HIV Prevention Among Injecting Drug Users in Transitional and Developing Countries page 10, Recommendations for minimal coverage monitoring and estimation
2011 UNAIDS's Operational Guidelines for Monitoring and Evaluation of HIV Programmes for PWID: Monitoring and Evaluation at the national and sub-national level, page 53, step 6: Output monitoring, including coverage. Are the intended outputs achieved? What proportion of the population in need received services?
2011 UNAIDS's Operational Guidelines for Monitoring and Evaluation of HIV Programmes for PWID: Monitoring and Evaluation at the service delivery level page 48, Step 6: Output monitoring, including coverage. Are the intended outputs achieved? What proportion of the population in the local area received services?
2013 WHO/UNODC/UNAIDS"s Technical Guide for Countries to Set Targets for Universal Access to HIV Prevention, Treatment and Care for IDUs, page 35, chapter 3.4 Measuring intervention coverage
page 38, chapter 3.5 Measuring intervention quality
page 38, chapter 3.6 Measuring the outcome and impact of interventions
Step 8. Is the programme effective in changing specified outcomes?
Note: Further monitoring can be done by testing collected syringes against HIV, HCV and other diseases.
For more details :
2011 UNAIDS's Operational Guidelines for Monitoring and Evaluation of HIV Programmes for PWID: Monitoring and Evaluation at the national and sub-national level page 69, step 7: Outcome monitoring and evaluation. Are there changes in HIV transmission risk? Are these changes due to the HIV prevention programme?
2011 UNAIDS's Operational Guidelines for Monitoring and Evaluation of HIV Programmes for PWID: Monitoring and Evaluation at the service delivery level page 59, Step 7: Outcome monitoring and evaluation. Are there changes in HIV transmission risk? Are these changes due to the HIV prevention programme?
Tools specialised in managing service provison to clients/patients
- SyrEX from International AIDS Alliance in Ukraine, Ukraine
Purpose : Clients registration; recording commodities and services provided
Specifications : Free of charge
Contract type : Proprietry, customization requires investment
- Nebula from Orion, Uk
Purpose : Clients registration and case management including HIV, hepatitis, saliva and tetanus; recording commodities and services provided;stock management and returns, specific to harm reduction supplies and services
Specifications : For a fee
Contract type : Proprietry, customization requires investment
- NEXmanager from webstar-health, Uk
Purpose : Secure, web-based NSP service administration system for pharmacies and harm reduction programs
Specifications : Providers enter data directly, Real time data availability, Convenient web-based interface, Automated invoice and payment schedules
Features : Central/local visibility
Contract type : Proprietry, customization requires investment