Horizontal Evaluation of the Canadian Shellfish Sanitation Program
Final Report
Project number 96744
June 9, 2022

Horizontal Evaluation of the Canadian Shellfish Sanitation Program
(PDF, 2.4 MB)

Table of contents

1. 1.0 Evaluation context
   1. 1.1 Objectives and scope
2. 2.0 Program context
   1. 2.1 Program resources
   2. 2.2 Roles and Responsibilities
   3. 2.3 Program stakeholders and partners
   4. 2.4 Operating environment
   5. 2.5 Regional distribution of CSSP
3. 3.0 Evaluation findings
   1. 3.1 Program Mandate & Evolving Context
   2. 3.2 Program resources
      1. 3.2.1 Funding
      2. 3.2.2 Tracked expenditures
      3. 3.2.3 Estimated current costs
      4. 3.2.4 Resource gaps
   3. 3.3 Governance and leadership
   4. 3.4 Program delivery
      1. 3.4.1 Achievement of results
      2. 3.4.2 Gaps in implementation
      3. 3.4.3 Information for decision-making
      4. 3.4.4 Availability of performance information
      5. 3.4.5 Communication with stakeholders and partners
4. 4.0 Conclusions and Recommendations
   • 4.1 Conclusions
   • 4.2 Recommendations
5. 5.0 Annexes
   1. Annex A – Evaluation methodology
   2. Annex B – Costs analysis methodology
   3. Annex C – Illnesses caused by consuming shellfish contaminated with biotoxins
   4. Annex D – CSSP classifications
   5. Annex E –The legislative mandates of the three federal partners
   6. Annex F – CSSP roles and responsibilities
   7. Annex G – More details on estimated current* CSSP delivery costs
   8. Annex H – Management Action Plan (MAP)
6. Footnotes

1.0 Evaluation context

The horizontal evaluation of the Canadian Shellfish Sanitation Program (CSSP) was led by Fisheries and Oceans Canada’s (DFO) Evaluation Division and was supported by an interdepartmental working group with evaluation representatives from the other two federal organizations responsible for the delivery of the CSSP: the Canadian Food Inspection Agency (CFIA) and Environment and Climate Change Canada (ECCC).

There has not been a comprehensive evaluation of the CSSP since 2007, although components of the program have been covered in evaluations and audits undertaken by DFO and ECCC since then.

The CSSP horizontal evaluation took place between April 2021 and June 2022 and was conducted in compliance with the Treasury Board Policy on Results.

1.1 Objectives and scope

The objective of the evaluation was to identify challenges and opportunities in the management and delivery of the Canadian Shellfish Sanitation Program to provide senior management from CFIA, DFO and ECCC with evidence-based information to support ongoing discussions about future program direction, decisions and improvements. The scope was established through a planning phase, which included consultation with program representatives from all three partner organizations.

The evaluation examined the delivery of the program by all three federal partners and in all three regions where CSSP is implemented i.e., Pacific, Quebec and Atlantic regions (see section 2.5 for more details) over the last five years (2016-17 to 2020-21). Where relevant and feasible, evidence was collected reaching back to 2010 to allow for the examination of trends. Of note, a significant lack of resources to deliver the program was raised as a major concern during consultations, therefore the evaluation included a detailed cost analysis to examine this issue.

Evaluation questions

The evaluation was designed to provide evidence on where the program is working well, as well as to identify where improvements could be made. It included an assessment of program relevance, performance and design and delivery.

Relevance

1. What is the mandate of the CSSP and how has it evolved over time?
2. To what extent is the program responding to existing and evolving needs? Are there any gaps?

Performance

3. What activities and outputs are currently being conducted and are there any gaps in what the agency/departments are able to do? If so, what are the impacts on the program's ability to deliver the program as intended?  Are there any unintended impacts?
4. What resources are allocated to deliver the CSSP and what is used above and beyond what is allocated? How are resource allocations and usage tracked for CSSP? What resources are needed to deliver the program as intended?
5. To what extent is the program meeting its intended results?

Design & delivery

6. To what extent does the program have effective governance within each partner organization; and between partner organizations?
7. To what extent do the partner organizations have information to support effective decision-making?
8. To what extent is the CSSP delivered effectively and efficiently? Are there alternative ways to deliver the program or any of its components and still achieve the intended results?
9. Does the program provide equitable and inclusive access to shellfish harvesting sites for safe consumption?

Evaluation methodology

Six lines of evidence were used to answer the evaluation questions.

• International benchmarking
• Document/legislative review
• Data analysis
• Key informant interviews
• Financial/cost analysis
• Process mapping

Annexes A and B present details on evaluation methodology, including limitations.

2.0 Program context

The CSSP is a federal food safety program that aims to minimize health risks associated with the consumption of contaminated bivalve molluscan shellfish. It has a long history of federal organizations working together and many of the activities of the present-day program (e.g., testing for biotoxins, testing water quality in shellfish growing areas and enforcing regulations in harvest areas) remain the same as those seen throughout the evolution of the program.

While the responsible federal organizations have changed over the years, in February 1990, a Memorandum of Understanding (MOU) was signed between the (then named) Department of the Environment and the Department of Fisheries and Oceans, defining their respective roles and responsibilities for the CSSP. The Canadian Food Inspection Agency was created in 1997 and the MOU was revised in 2000 to reflect the transfer of fish inspection activities from DFO to the new Agency.

The CSSP, which is jointly delivered by CFIA, DFO and ECCC, continues to operate under the MOU (2000) which has the stated goal:

To provide reasonable assurance that molluscan shellfish are safe for consumption as food by controlling the harvesting of all molluscs within the tidal waters of Canada.

CFIA is responsible for providing horizontal coordination to facilitate the development and implementation of the CSSP.

2.1 Program resources

In 2018-19, the three federal partner organizations estimated the cost of the CSSP to be approximately $17.87 million per year for operations and maintenance (O&M), salary and full-time equivalent employees (FTEs). One of the goals of the evaluation was to obtain a more comprehensive understanding of resources spent by the program and any gaps that exist. Updated, more detailed figures based on financial analysis conducted during the evaluation can be found in section 3.2 of the report.

2.2 Roles and Responsibilities

The CSSP controls the harvest of shellfish by classifying waters (see Annex D) as approved, conditionally approved, restricted, conditionally restricted or prohibited. Water can also be unclassified. Classifications are based on comprehensive water quality monitoring and analysis and pollution source assessments. Areas are given either an open or closed status according to whether shellfish may or may not be harvested.

Roles and responsibilities of each partner organization flow from their legislated mandates (see Annexes E and F for details) and are laid out in the MOU (2000)^{Footnote 1} as follows:

• CFIA is the lead regarding the handling, processing, import and export of shellfish; the marine biotoxin monitoring program; and any other microbiological monitoring program not under the responsibility of ECCC. CFIA tests shellfish for biotoxins and, proactively or in response to incidents, the agency also tests for micro contaminants such as salmonella. CFIA operational staff involved in the delivery of the CSSP include, but are not limited to, regional coordinators and program officers, operations specialists, fish inspectors, shellfish specialists, and scientists (e.g., chemists, microbiologists).
• DFO is the lead department regarding the harvesting of shellfish and notifying stakeholders and partners of the openings and closing of harvest areas. The department is responsible for licensing shellfish harvesting, enacting the opening and closing of shellfish harvest areas, and conducting patrols to enforce compliance of harvest. DFO operational staff involved in the delivery of the CSSP include, but are not limited to, biologists (e.g., shellfish specialists), resource managers, reconciliation liaisons, program officers/coordinators, license advisors, policy analysts, and conservation and protection (C&P) staff i.e., detachment supervisors, compliance officers and fishery officers.
• ECCC is the lead department regarding recommending the appropriate classification of shellfish harvest waters based on sanitary and bacteriological water quality conditions. The department is responsible for monitoring water quality in shellfish harvest areas and identifying and evaluating sanitary pollution sources that may affect the quality of the waters where shellfish are being harvested for consumption. This includes undertaking wastewater modelling and recommending emergency closures following significant events such as large rainfalls which could release fecal coliform or other contaminants into the environment through wastewater discharge or agricultural runoff. ECCC operational staff involved in the delivery of the CSSP include, but are not limited to, area coordinators, program officers, biologists, wastewater engineers, laboratory technicians, GIS/data coordinators, technologists, and scientists (e.g., biologists, chemists, hydrologists).

While the three federal organizations have specific roles and responsibilities, their interdependence cannot be understated when it comes to delivering the intended results of the CSSP. Decision-making and program operations within each partner organization are dependent on the information collected and shared between partners. As such, there are four levels of interdepartmental governance to support integrated and collaborative decision-making (see section 3.3). Decisions at interdepartmental committees are made by consensus.

2.3 Program stakeholders and partners

The primary beneficiaries of the CSSP are domestic and international consumers of Canadian shellfish. The program was originally designed for the wild commercial harvest sector to meet requirements for export to international markets. It has since expanded to encompass aquaculture. Export countries, primarily the USA, continue to be the target markets; however, over time there has been increasing pressure to respond to more non-commercial (recreational and subsistence) harvest and Indigenous access for food, social and ceremonial (FSC) purposes. Program stakeholders and partners include:

• Commercial harvesters (aquaculture and wild harvest)
• Recreational harvesters
• Indigenous peoples harvesting for food, social and ceremonial reasons
• Remote coastal communities (Indigenous and non-Indigenous) dependent on the fisheries
• Other levels of government including provinces, and municipalities
• Other federal departments such as Health Canada, Transport Canada, Indigenous Services Canada, and Crown Indigenous Relations and Northern Affairs Canada.

2.4 Operating environment

The program is delivered in three CSSP regions (see section 2.5) with regional differences and challenges that contribute to complexities in program delivery. Below are some examples of complexities, although this list is not exhaustive. 

• The program works with multiple jurisdictions, which have varying regulatory regimes. For example, in British Columbia DFO regulates aquaculture activities, whereas it is a provincial role in other areas in Canada (except for a co-management leasing regime in Prince Edward Island).
• The commercial species that drive local fishing industries differ from one region to another. For example, mussels are the main commercial shellfish species in Prince Edward Island. Geoducks are only harvested on the Pacific coast.
• The risk of biotoxin contamination of scallops differs among some species, depending on the way in which they bioaccumulate toxins. For example, biotoxins do not concentrate in the adductor muscle of the Atlantic “sea scallop” (the giant scallop, Placopecten magellanicus), so only the adductor muscle is harvested and brought to shore. This species is therefore not subject to the same CSSP biotoxin controls as other species of scallops. Further, while scallops are a bivalve molluscan shellfish, most of the scallop fishery occurs in deeper waters where the CSSP is not delivered.
• Stakeholder/partner populations vary in each region. A good example is found in Quebec, where harvesting shellfish from local beaches is a culturally engrained activity. Consequently, there is a large population of recreational harvesters and less commercial activity in the Quebec region compared to the Pacific and Atlantic regions.
• In some regions and communities, the recreational and FSC shellfish harvest have the potential to provide a level of food security, allowing Canadians access to affordable and fresh protein, particularly in the current context of rising food prices.

2.5 Regional distribution of CSSP

The Canadian Shellfish Sanitation Program is delivered in three CSSP regions where there is known shellfish harvesting in tidal and sub-tidal waters: Pacific, Quebec, and Atlantic. It is currently not delivered in Northern Canada or Labrador. Each federal partner has a national headquarters (NHQ) to provide internal support and policy guidance within their respective organization. As the CSSP cannot realistically cover all harvest areas along the 78,000 kilometers of coastline in the three CSSP regions, the program takes a risk-management approach for program delivery, prioritizing the allocation of available resources to areas of highest risk.

Source for growing area numbers and commercial shellfish landings and aquaculture production: DFO

*The CSSP Pacific region only includes the Province of British Columbia.

**The CSSP Atlantic region encompasses three DFO regions: Maritimes, Gulf, and Newfoundland and Labrador regions. For CFIA and ECCC, the CSSP Atlantic region includes New Brunswick, Nova Scotia, Prince Edward Island (PEI), and Newfoundland and Labrador. The program, however, is not delivered in Labrador.

3.0 Evaluation findings

3.1 Program Mandate & Evolving Context

The food safety mandate of the CSSP has remained consistent since 1925.

The CSSP is first and foremost a food safety program. This mandate, as stated in key documents, focuses on the following two objectives:

1. Minimize health risks from the consumption of bivalve molluscan shellfish; provide reasonable assurance that shellfish are safe for consumption by controlling their harvest within the tidal waters* of Canada.
2. Fulfill Canada’s international obligations to meet the terms of bilateral agreements and standards, in particular the bilateral agreement between the USA and Canada^{Footnote 2}, signed April 30, 1948, to improve the sanitary practices prevailing in the shellfish industries of the two countries.

*Note: There are areas on both the Pacific and Atlantic coasts where CSSP resources are used for sub-tidal species (e.g., geoducks). This presents a gap between program delivery and the stated program objectives.

These objectives have been in place since the beginning of the program. In 1924, there was an epidemic of typhoid fever following the consumption of contaminated oysters in the USA, which killed 150 people. In response, Canada passed regulations under the Fish Inspection Act in 1925 requiring that imported oysters be certified as a “safe food product”. Subsequently, the bilateral agreement was signed between the USA and Canada to certify that shellfish moving in either direction across borders were “safe to eat”.

Since then, the USA has continued to be Canada’s most significant trading partner for shellfish. In 2019, Canadian exports of molluscan shellfish** to the USA were valued at approximately $230 million^{Footnote 3}, which accounted for about 48% of all Canadian molluscan shellfish exports. In 2019, the USA accounted for 57% of all of Canada’s scallop exports, 86% of oyster exports, and 91% of mussel exports^{Footnote 4}. These numbers highlight the importance of the American market to the shellfish industry in Canada.

**Note: Data related to molluscan shellfish in this report may include species that are not covered by the CSSP (e.g., giant scallop) as data disaggregated by sub-species is not available.

The CSSP mandate is similar to the mandates of other international jurisdictions examined in the evaluation, all of which focus primarily on the safety of shellfish for consumption.

Washington State, US

“… prevent illness in people who eat Washington-grown molluscan shellfish”

New Zealand

“identify, monitor, evaluate and manage the risks associated with the commercial growing, harvesting, sorting and transporting of [shellfish] intended for human consumption”

Australia

“… assure the food safety of shellfish managed in accordance with its operational guidelines”

United Kingdom

“… ensure the shellfish on the market do not contain unsafe levels”

Chile

“… ensure the safety of the resources, particularly with respect to marine biotoxins”

While the importance of international export to the Canadian shellfish industry is evident, significant domestic demand also exists.

Data shows that Canada is the largest exporter of shellfish within North America, responsible for 43.6% of North American shellfish exports, compared to the USA (38.8%) and Mexico (17.6%)^{Footnote 5}. Canada’s export of shellfish was worth close to $500M in 2019, representing over 35 million kilograms of product (see Figure 1).

Note: This graph excludes some species due to specificity of data. Further, data is presented both with and without scallops as often scallops are not covered under the CSSP but are not disaggregated from the data.

Source: Canadian Trade

Canada’s exports of shellfish have been increasing since 2015, although they have decreased during the COVID-19 pandemic (2020 and 2021). According to the Outlook to 2027 for Canadian Fish and Seafood^{Footnote 6}, the exports of shellfish were expected to reach a value of $600M in 2025.

Nevertheless, according to export and production figures^{Footnote 7Footnote 8}, Canada exports only 25% of its total shellfish production (38,000 metric tonnes out of over 153,000 metric tonnes). Additionally, while imported products are not part of the CSSP, they do provide an indication of a certain level of domestic demand for shellfish consumption (see Figure 1) being equal to a third or more of the volume and value exported. Statistics Canada reports that an average of 1.4 kilograms of shellfish* has been available per person over the last 10 years. Further, according to a survey conducted by the Canadian Aquaculture Industry Alliance in 2011^{Footnote 9}, 43% of Canadians eat shellfish* and average shellfish consumption frequency is 1.9 times a month. In general, access to fresh protein and produce all year was important to over four in five Canadians.

*Note: It is not clear whether species like shrimp and crab are included in the definition of shellfish used in these two studies.

Regarding domestic shellfish consumption, data to understand the portion of commercial shellfish sold interprovincially (under CFIA jurisdiction) or intraprovincially within provinces of origin (under provincial authorities) was not readily available.

The delivery of the CSSP’s mandate is focused on commercial markets and maintaining export.

The CSSP’s first objective (minimizing health risks) encompasses both domestic and international consumers of Canadian shellfish who are protected by the various controls put in place by the program. However, the importance of international consumers to the program is underscored by the mandate’s second objective (fulfilling international obligations to facilitate trade). Given the significance of the domestic shellfish market and level of consumption, the program could be more explicit about the intended delivery to domestic consumers, particularly those who may rely on safe, non-commercial shellfish for food security; for example, recreational harvesters and Indigenous Food, Social and Ceremonial (FSC) harvesters.

The program’s emphasis on commercial harvesters, including for export markets, was reinforced by internal interviewees. When asked if the mandate of the program has changed over time, about half (46%) said that commercial markets continue to be the priority and 29% perceived there is a push to focus more on the commercial/market side and less on other types of harvest.  There were examples of recreational areas, in particular, being given the lowest priority or even removed from program coverage, and there is a lack of guidance on the priority level of Indigenous FSC harvest in the program.

One element that appears to be a principal driver for the program’s emphasis on maintaining the export market is the trading relationship with the USA, which, as explained previously, is Canada’s largest trading partner for shellfish. The bilateral agreement in place with the USA requires regular audits of the CSSP to ensure a USFDA-equivalent* level of protection exists for consumers. Some of the audits have resulted in recommendations to ECCC that are very resource intensive to implement (see below), but the importance placed on the US market means that maintaining access is prioritized despite the challenges this presents. The use of significant ECCC program resources to meet these requirements results in rationalization in other aspects of the program, including the number of areas monitored. However, given that shellfish exported to the USA amounts to just 12% of Canada’s total production, it is possible that the resources dedicated to this aspect of the program is disproportionate. These factors have the potential to directly impact recreational and FSC harvesters, who have not been given the same level of priority as commercial harvesters in designating sites for program delivery (see below).

*Note: USFDA is the United States Food and Drug Administration

The CSSP’s operational context has evolved, resulting in new and increased challenges and pressures, as well as additional demands and interest.

The context within which the CSSP operates has evolved over time in ways that place significant pressures and demands on program delivery, particularly since there has been no known increase in program resources since 2000 (see section 3.2 for more details).

Government priorities have changed

Among the Government of Canada’s key priorities is reconciliation with Indigenous Peoples. The government has increased and accelerated its efforts in this area over the period of the evaluation, recognizing its constitutional and treaty obligations and, in 2021, passing into law an Act respecting the United Nations Declaration on the Rights of Indigenous Peoples. In 2019, all Ministers’ mandate letters included direction to build on the progress made with First Nations, Inuit and Métis people, and the same year Fisheries and Oceans Canada and the Canadian Coast Guard (CCG) released a Reconciliation Strategy committing to the recognition and implementation of Indigenous and treaty rights. In the most recent mandate letters of 2021, reconciliation with Indigenous Peoples has remained a top priority across government.

Indigenous peoples have the right to maintain and strengthen their distinct political, legal, economic, social and cultural institution

External pressures and challenges

• The 2009 USFDA audit report included a formal recommendation that created a significant additional burden on the CSSP with regards to water monitoring requirements, classification and the management of shellfish harvesting around wastewater treatment systems, essentially tripling the amount of water testing required. ECCC continues to feel the burden of this increased workload and it has exacerbated resource constraints faced by the entire program.
• Climate change is amplifying some of the program's current challenges. Increased water temperatures due to climate change results in more frequent harmful algal blooms. These can cause a higher risk of marine biotoxin accumulation in shellfish, which may require more frequent monitoring/sampling.

• Daily extreme precipitation is projected to increase in the future^{Footnote 11}. More frequent severe weather events have a direct impact on shellfish harvest areas, which have to be closed due to increased sanitation pollution concerns. For example, the severe rainfall events of 2021 on the west coast of Canada increased work for all three federal partners (i.e., predicting surges of contaminants entering the water, shellfish and water testing, communications, participation in outbreak response, and increased patrolling to monitor compliance with the closures). The east coast of Canada has experienced similar challenges with an unusually high number of closures in recent years due to climate change-related weather events.
• Increased prevalence of biological contaminants, such as norovirus, has required that CFIA consider the management of new risks.

Increased demands from CSSP stakeholders and partners

• The program is receiving increased requests to harvest for FSC purposes (e.g., requests to harvest horse clams in 35 areas), triggering the requirements for additional bacteriological and biotoxin monitoring, harvest area classifications, and compliance patrols. Many Indigenous communities (and non-Indigenous communities) depend on local fish species, including shellfish for food security.
• Many First Nations are looking to branch out from harvesting solely for FSC reasons into building economic development opportunities for their community members.
• CSSP partners have been approached about program delivery in Northern Canada where there is potential to grow the shellfish industry, including shipping commercially harvested shellfish to southern areas of the country as well as to the USA.
• The needs of those living in rural, coastal communities are increasing, as they seek economic recovery following the COVID-19 pandemic. Meeting these needs is not currently feasible within the CSSP’s current resourcing.
• It is difficult to quantify the demand related to recreational harvest as data on this component was not provided.*

*Note: DFO Pacific region administers a survey to recreational license holders and captures some data on self-reported shellfish harvest through digging and handpicking on beaches. The CSSP Quebec region tracks requests to reopen harvest areas for recreational harvest.

Increased demands from commercial stakeholders and partners

• The CSSP is facing pressure from industry, including aquaculture operators who would like to continue to expand. As shown in Figure 2, the production value* of shellfish from aquaculture has been rising in Canada, almost doubling from 2011-2019.
• Figure 3 shows that the landing values* of Canadian wild shellfish harvest have been rising in both the Pacific and Atlantic regions. This indicates a possible intensification of the commercial shellfish industry which would translate to the CSSP in terms of effort required to support that growth. Note: Quebec’s commercial harvest is mostly from aquaculture.

Source: Aquaculture Production Quantities and Value

Source: Seafisheries Landings

*Note: Production and landing values are often lower than export values (used in Figure 1) due to value-added processing

The program is unable to accommodate increased demand.

When external stakeholders and partners of the program wish to have a new shellfish harvest area classified, they are required to submit a proposal to the Regional Interdepartmental Shellfish Committee (RISC) for review and, once accepted by all three federal partners, for RISC approval. These proposals are referred to as “Section 14 classification requests.” As shown in the graph below, there has been an overall upward trend in new requests for the Quebec and Pacific regions for the years shown. Numbers have remained relatively stable for the CSSP Atlantic region from 2016-17 to 2020-21 (there were no requests for 2020-21). There has been a total of 80 known requests over the last five years.

The evaluation team found these numbers to be low compared to anecdotal reports of demand as described by key informants in interviews; however, it is likely that these numbers are not fully indicative of demand. Internal documentation from 2015 made it clear that the development of any new shellfish harvesting areas would not occur due to a lack of CSSP capacity, unless approved by the National DG Operations Committee. Interviewees from all three federal partners and from industry mentioned there are deterrents to industry or First Nations making requests, such as high costs as a result of user pay initiatives introduced around 2015, internal capacity constraints (e.g., for small businesses), and the low probability of approval.

The program is considering expansion options to address some of the increased demands and changing context for the CSSP, however at the time of writing this report the three partner organizations had not yet reached consensus on how the program could expand.

Source: ECCC (Quebec and Pacific from 2010-11 to 2020-21), CFIA (Atlantic from 2016-17 onward)

While the program has been trying to adjust to its evolving context, progress is very slow

Various initiatives and national and regional working groups have been put in place to try to adjust to the program’s evolving context; however, progress has been slow. One CSSP initiative to note was the “Modernization exercise/Long-Term Sustainability Project” initiated in 2014, aspects of which are ongoing today (see details below). It was intended to review and modernize the CSSP to address increasing demands associated with an expanding shellfish harvesting industry in Canada and ensure future program sustainability by optimizing its use of resources. A more recent (2020-21) review has focused on assessing whether the CSSP is meeting program objectives, identifying challenges and gaps, and identifying areas for program improvement or potential expansion, including introducing appropriate measures to address Indigenous FSC shellfish harvesting in Canada.

Details of Modernization Exercise/Long-Term Sustainability Project (2014 and ongoing)

Harvest area prioritization exercise

In February 2015, in light of resource constraints, CSSP staff were directed to prioritize in the following order, based on the level of food safety risk exposure:

1. Significant and sustained commercial harvesting areas (both wild and aquaculture);
2. Low-production commercial harvest areas; and
3. Areas with only recreational harvest.

No written guidance was provided on the priority level of Indigenous FSC harvest.

This direction has come at a sacrifice of field delivery in lower-priority commercial areas, some FSC areas, and virtually all recreational and non-Indigenous cultural areas (except where there was overlap with prioritized commercial areas).

Growing area inventory and declassification exercise

In order to determine if CSSP resources are allocated appropriately, each shellfish harvesting area was to be evaluated based on indicators such as the volume and value of shellfish harvested, whether leases or licences have been issued, as well as whether invasive species or shellfish disease are present.

See section 3.4 for more details on the growing area inventory.

Alternative service delivery

After an analysis of alternative service delivery (ASD) models was conducted, various ASD options were piloted to address increasing program expansion demands. The main option tested was the implementation of a “user-pay” principle for new stakeholders and partners requesting new services from the program.

A few ASD agreements have been put in place with ECCC and CFIA for sampling and testing. In these agreements industry and Indigenous applicants agreed to bear costs related to water and shellfish sampling and/or testing.

3.2 Program resources

3.2.1 Funding

CSSP total funding allocations are unknown.

No recent and reliable source of information could be found regarding the total funding officially allocated to the CSSP.

Foundational documents indicate that in June 2000, the CSSP was allocated $4 million in annual funding and 25.8 full time equivalent (FTE) employees under the Human Health component of the Program for Sustainable Aquaculture. These resources were intended to increase consumer and market confidence in aquaculture shellfish products and did not target delivery of the program in areas dedicated to recreational, FSC and wild commercial harvest. Figure 5 illustrates how these resources were allocated to ECCC, CFIA and DFO.

In addition, in 2001-02, $51.4M of funding over five years was approved to expand the CSSP to the north. However, only $5.5 million ($4M for ECCC and $1.5M for DFO) was spent, as the project was abandoned in March 2003.

The evaluation was not able to confirm the amount of funding that existed prior to 2000 to deliver the program.

3.2.2 Tracked expenditures

The program does not have an accurate representation of actual CSSP expenditures

At ECCC, the CSSP is delivered entirely under the Shellfish Water Classification Program (SWCP). Operating and maintenance budgets related to the CSSP are accurately captured in their SAP financial system, under specific cost centers dedicated to the SWCP.

There are no dedicated CSSP resources within the CFIA. Funding goes through food business lines. However, since CSSP is a high-priority program due to the risks associated with contaminated shellfish, subject matter experts (SMEs)^{Footnote 13} said that CSSP-related activities are likely to receive the level of resources required (at the expense of lower-risk activities). For the purposes of resource tracking, CFIA’s finance group manually calculated CSSP expenditures through extracted data from various systems using significant assumptions.

Within DFO, most resources dedicated to the CSSP are shared with non-CSSP activities under DFO’s mandate and as such are not specifically allocated to the program. SMEs confirmed that resources currently allocated to the CSSP are unknown, as they are not accurately represented in the financial system.

An internal audit conducted in 2017-18 concluded that DFO lacked adequate information on its financial and human resource allocations to ensure sound CSSP management. The CSSP project code created in 2013 to improve tracking of expenditures had not been used consistently. As recommended by the audit, a memo was circulated to remind staff to use the dedicated project code to track CSSP expenditures accurately. Although there has been some improvement, regional staff seem confused regarding the practices to follow. As most FTEs are not solely dedicated to the CSSP, it is considered complex to use the project code to track salary expenditures. For O&M expenditures, some regions entered actual expenditures against the CSSP dedicated project code only up to the CSSP allocated budgets. In these cases, expenditures greater than budgets are not being captured.

While ECCC’s expenditures are accurately represented, there are concerns with the reliability of CFIA’s and DFO’s tracking of CSSP expenditures. While some tracking is taking place, the fact that certain activities, such as patrolling for compliance or food safety testing, are combined with the same activities for other fish species or foods respectively, makes it more difficult to identify costs.

Despite many limitations, the evaluation was able to present a picture, albeit incomplete, of CSSP expenditures as being around $13 million annually (see Figure 6). Findings further on in the report indicate the cost of delivering the CSSP is significantly higher than what is tracked.

*Because the manual calculation of CSSP expenditures by CFIA requires a significant amount of effort, this exercise was undertaken for two years only (2018-19 and 2019-20).

Source: CFIA, DFO and ECCC financial systems.

While resource tracking issues have been noted in previous reviews, the CSSP has been unsuccessful at addressing them.

The 2007 horizontal evaluation of the CSSP recommended the development of a central tracking and reporting system for CSSP costs:

2007 Evaluation Recommendation

The CSSP Secretariat should develop a tracking and reporting system for CSSP costs and performance.

Management Response and Implementation Strategy

The objective will be to prepare a project plan identifying a proposed process for development and implementation of a tracking and reporting system. This will be developed in consultation with the National Interdepartmental Shellfish Committee.

Fifteen years later, there is still no centralized cost-tracking and reporting system for the integrated program. The implementation of the management response was considered closed in 2012. In their justification, the program stated that “as there are no shared costs for the CSSP, each partner would continue to report on CSSP costs separately as required by their organization”, and “tracking of costing will be integrated into CSSP ADM and DG work planning and will be reviewed annually … Horizontal costing issues will be addressed through the CSSP governance structure as they arise.” Each federal partner still has the same tracking mechanisms and ECCC is still the only federal partner systematically tracking its CSSP expenditures. More rigorous tracking mechanisms are needed to better equip the program to accurately and completely quantify the real cost of delivering the CSSP, including to determine actual resource gaps.

Note: Interviewee responses were classified as “mixed” where they gave responses to the question that were both positive and negative.

3.2.3 Estimated current costs

There is a significant gap between tracked expenditures and what is estimated as the current cost to deliver the program.

An activity-based costing analysis was performed in collaboration with SMEs to estimate the current costs for delivering the CSSP, including those above and beyond what is officially tracked. The evaluation used these estimates as a proxy for current costs.

CFIA’s estimates for current CSSP costs were similar to the figures calculated by CFIA’s finance group for 2018-19 and 2019-20, although FTEs calculated by the CFIA finance group were greater than the number estimated by SMEs.

DFO’s estimates for current CSSP costs were significantly higher than both known funding and tracked expenditures. This confirms that DFO’s tracked expenditures do not provide an accurate representation of all costs associated with carrying out CSSP activities.

ECCC’s costing analysis results for tracked expenditures and current costs estimates are assumed to be identical, given that all CSSP-related costs are fully and accurately recorded in the system.

As shown on Figure 8, there is a significant gap between CSSP tracked expenditures ($13.2 million) and what is estimated as the current cost for the delivery of the CSSP as estimated by SMEs ($21.9 million). DFO’s CSSP delivery costs as estimated by staff ($9.9 million) represent the largest portion (45%) of the total current CSSP delivery cost. ECCC is next with 30% of the costs ($6.5 million).

See Annex G for additional details on estimated costs to deliver the CSSP.

Source: CFIA, DFO and ECCC financial systems and SMEs’ estimates.

87% of internal key informants said the program DOES NOT have the level of resourcing needed to be delivered as intended. (n = 62)

3.2.4 Resource gaps

The CSSP is insufficiently funded.

The significant gap between what is tracked and what is estimated as current costs suggests that resources are being reallocated from other funding sources in order to be able to deliver the CSSP at its current level.

All three federal partners are at maximum capacity:

CFIA laboratory capacity for marine biotoxin testing is reported to be at maximum capacity within current resource levels. Further, the agency is not able to add new monitoring stations and has had to decline some requests from industry to do additional testing. They also divert resources from other CFIA activity areas to meet increasing demands and pressures.

ECCC has experienced internal O&M reductions within its program since 2018. All ECCC regions are facing some program delivery issues, and in several areas the network of monitoring stations has been reduced significantly. This has resulted in extended closures and the reduction of potentially viable approved waters. Pollution sources and wastewater systems are not being reassessed regularly, except in critical harvest areas, meaning the sampling frequency required by the program is not always respected. Continued re-prioritization to fit delivery within resource allocations is not sustainable for ECCC program delivery. Since ECCC is the department that begins the process of opening a harvest area (through recommended classification), lack of resources at ECCC for water monitoring and other activities such as pollution source assessments has been identified as a major bottleneck to CSSP delivery.

Given limited resources, establishing new monitoring sites can mean removing others.

At DFO, funding allocated to CSSP does not cover costs, so resources are reallocated from other regional funding sources on an opportunistic basis, impacting the delivery of other departmental mandated priorities, such as whale protection on both coasts, aquatic invasive species, marine protected areas and more.

In the CSSP, laboratories are used for water quality testing by ECCC, and for biotoxin and microbiological testing by CFIA. While interviewees described capacity challenges around water quality monitoring in Quebec, particularly due to reliance on contractors, the capacity issues are most limiting for the program when it comes to those laboratories used by CFIA for biotoxin testing.

A comparison of the laboratories available to the CFIA to test shellfish for biotoxins and those available to shellfish sanitation programs in other jurisdictions shows that other countries have a larger number of laboratories at their disposal (see details below), which is notable given the potential geographic scope of program delivery in Canada. CFIA has three accredited laboratories conducting marine biotoxin testing, in Burnaby British Columbia; Longueil, Quebec; and Dartmouth, Nova Scotia. Even if each of these labs were able to take on more samples, the time it takes for samples to travel to the labs for testing would present a challenge in serving more remote parts of the coastlines due to required timeframes for samples to arrive at laboratories.

**This does not include Canada’s northern coastline, where CSSP is not currently delivered but where some demand does exist, as described earlier in the report.

*Source: Coastline - The World Factbook (cia.gov)

Insufficient resources adds significant pressures and increases potential risks.

With the limited capacity and uncertainties related to funding, it has not been possible to hire additional indeterminate staff (or retain skilled contractors) for the CSSP. For that reason, the CSSP is using more temporary staff, students, volunteers and/or contractors to do the additional work, rather than using internal capacity. This has impacted operational planning and delivery by creating a dependency on third-party availabilities and rates and removing certain knowledge and expertise from within the program.

The unintended impact of insufficient resources is that the program is pressured to make some decisions based on resource limitations, rather than what is most needed to provide fulsome coverage (e.g., keeping a harvest area closed for a longer time when there is no capacity to monitor the area).

Managing increased demands in the environment of under-resourcing also places considerable stress on core staff, which has had a negative impact on staff morale in some regions. In some cases, significant overtime is required, particularly for DFO’s C&P officers. CSSP program staff on the ground are also the public face of the program and are often required to interact with stakeholders or partners and can bear the brunt of their dissatisfaction.

As previously described, in recent years the CSSP has put in place alternative service delivery (ASD) arrangements with the intent of saving costs while being able to respond to expansion requests. However, each ASD arrangement still requires initial and ongoing program effort to maintain the harvest area classification and related operational requirements. There is also significant overhead and involvement from staff to train and monitor ASD partners.

While ASD does alleviate program delivery pressures, it is not a cost-neutral exercise for any CSSP federal partner.

Due to the Canada’s extensive coastline, the CSSP will always require a risk-based level of prioritization as it is impossible to deliver even a fully resourced program for every potential shellfish harvesting area in the country. However, there are food safety risks associated with recreational and FSC harvesting in areas where the program is not or has ceased to provide coverage. Interviewees explained that in regions where areas are considered “harvest at your own risk” once declassified, people harvesting may not be aware of the change. As well, where long-term closures have been in place without ongoing sampling, people may not feel that the closures are legitimate and may disregard them. These risks may be greater in remote areas, where recreational or FSC harvest of shellfish is both a traditional practice and a way to ensure food security.

3.3 Governance and leadership

There are several governance mechanisms in place to support the implementation and delivery of the CSSP.

As shown in the table below, the CSSP is governed by interdepartmental governance committees at four levels, all of which are chaired by the CFIA members. While there is no formal Deputy Minister committee, Deputy Ministers in the three partner organizations meet on an ad hoc basis.

A secretariat, staffed by one person and housed within CFIA, performs the role of coordination and support for the ADM and DG committees.

There is a need for more strategic coordination, direction, and national leadership.

The evaluation conducted in 2007 recommended “A strengthened role for the ADM Steering Committee that includes clear CFIA leadership …”. While it is understood that the recommendation was fully addressed, there is confusion as to whether CFIA is the federal lead, as there are discrepancies between internal documents and publicly available information. This lack of clarity may explain why some interviewees perceived that there is hesitation by CFIA to play a stronger role in facilitating program level decision-making and initiatives that target program improvements. There were instances of a lack of leadership and direction observed at both the national and regional levels. While high-level decisions for the program are made by consensus among the three partners, and the agency does not have the authority to direct its federal partners or impose decisions, there is a need for more focused effort to resolve long-standing issues, such as how to address increasing demands.

An issue raised by some interviewees was the challenge of delivering a single program by three federal partners with different mandates. Despite the presence of centralized, interdepartmental governance structures, there has been difficulty agreeing on the scope of the program and making key decisions jointly rather than based on each individual organization’s priorities. Decisions made by each partner can have significant impacts on the other partners, which is problematic when they are not made together.

• The latest guidance on how operational staff are to prioritize program delivery appears to exist primarily in emails. The evaluation did not find evidence that this information is accessible in any current guidance document, or that it has been updated since 2015 in response to the evolving context of the program.
• National direction on the program’s role in Indigenous reconciliation thus far seems to have been ad hoc and/or verbal. Despite this, there are examples of operational staff giving priority to Indigenous FSC requests to honour treaties and other commitments made by the government.
• While senior management held similar views to other internal interviewees on many aspects of the program, they tended to have a more positive view of the overall effectiveness and efficiency of the program compared to operational staff, which may suggest a disconnect.
• Interview findings show that program staff and senior management have a divergent view on whose needs should be met by the program (57% of senior management felt that it is not the program’s role to meet all stakeholders’ and partners’ needs, whereas 55% of all other internal interviewees perceived that it is the program’s role to meet these needs).

Many of the issues raised about lack of strategic direction and leadership are not new.

The CSSP has been the subject of several different audits and evaluations since 2007, in which similar concerns were cited. However, of note, CFIA has not conducted any reviews on its portion of program delivery since the 2007 horizontal evaluation of the CSSP.

Some actions were taken by management in response to the 2007 evaluation, in particular the development of a performance measurement strategy; however, this document is no longer current, and the issues persist.

Regional governance is working well.

According to internal key informants, the structure of the interdepartmental governance is, for the most part, working well at the regional level. Internal interviewees said that, due to the longevity of this program, people have developed strong working relationships over time, particularly as members of the RISCs.

Overall, operational roles and responsibilities of the three partners are clear, well understood, and they complement each other. About 70% of all interviewees said there was no duplication between the three federal partners or with other jurisdictions.

The expertise and dedication of operational staff is acknowledged and appreciated by both interdepartmental colleagues and by external stakeholders and partners.

While the program is not without challenges, a few internal key informants said that the CSSP is one of the best examples in the federal government of multiple departments working together to deliver a single program to Canadians.

While the RISCs work well, they are not receiving the national direction they need.

Overall, day-to-day operational decision-making is smooth. However, there are challenges when issues are elevated from the regional to the national level for advice or decision. Challenges range from long delays receiving a response, to the absence of decision-making, to issues being referred back to the RISCs without resolution. Delays receiving national direction in the regions are reported to range from one to four years. The lack of timely policy decisions or direction leads to inefficiencies in the regions. This has been particularly true regarding finalizing decisions about declassification.

A few program personnel indicated that the NISC strikes a lot of working groups. In 2020-21, there were eight national working groups on a range of topics. There is evidence that the working groups are making progress on key initiatives, and the existence of working groups is evidence of the three partners working together on program issues. However, advice and guidance as a result of their work is slow. According to their ToR, “NISC activities should lead to timely advice” … and the committee should “strive to resolve issue within a maximum two-year timeframe.” Given the multi-year timeframes cited by interviewees for the provision of national direction, this objective is not always being met.

One issue is that regional and national personnel do not have the same understanding of the role of the NISC when issues cannot be resolved in the regions. This may be partly attributable to inconsistencies in governance documents. The Terms of References (ToRs) for the RISCs state that when consensus cannot be reached in the region, issues will be brought forward to the NISC for direction and/or decision. Further, the CSSP manual states that the RISCs will provide recommendations to the NISC on all regional aspects that require national direction. However, activities highlighted in the ToR for the NISC do not state responsibilities for regional issues management or decision-making. The NISC ToR  states they will “establish sub-committees and working groups, as required, to deal with specific issues and to develop appropriate policies and/or procedures for dealing with them”; there is no mention of those groups coming to decisions.

As currently structured, the NISC does not operate as a decision-making body – rather, the DG Operations committee is the senior decision-making body for the CSSP. The NISC ToR explains that they bring forward proposals and recommendations to the DG Operations Committee for decision (e.g., related to policy or changes to the program manual).

Some key informants suggested it is time to “dust off” the MOU and program manual through a comprehensive review.  There is evidence to support this need; the evaluation found inconsistencies between the MOU, program manual and the Terms of Reference for the interdepartmental shellfish committees.

Overall, there is consistency across the program where needed, with a few exceptions.

• The program appears to be striking the right balance of consistency and flexibility. Almost half (45%) of internal interviewees stated there is consistency where needed, and it was mentioned that some inconsistencies are by design and consistency is not necessarily the right goal due to differing regional contexts (e.g., related to species harvested or prevalence of recreational harvesters in Quebec).
• There is one area, however, that was raised as an issue by many program staff where flexible regional policy creates inconsistencies in how the program is delivered: regions interpret and apply differently the expectations for monitoring and/or implementing program controls in unclassified and declassified areas. The program manual states “Declassification … results in the area becoming unclassified and therefore no longer subject to CSSP controls.” According to other internal documents, there will be no opportunity to harvest in newly declassified areas. The wording “no opportunity” implies the application of the CSSP control of harvest area closure. The following inconsistencies exist:
  • The Quebec region puts almost all unclassified areas into closed status which obligates patrols.
  • In the Atlantic region, waters that are unclassified or declassified are not required to be controlled for CSSP, however they are regularly patrolled by DFO’s C&P officers during activities for other programs or species. Fishery officers may undertake other activities in unclassified areas such as outreach, engagement and education for CSSP.
  • In the Pacific region, there are widespread closures in unclassified areas to mitigate against possible biotoxin risks.
• When an area is closed, it puts pressure on DFO to monitor for compliance in areas not otherwise subject to CSSP controls. This is particularly an issue in remote northern areas of both coastlines. Further, long-standing closures without monitoring justification causes confusion and frustration for Indigenous and non-Indigenous harvesters who would like to access the areas.

Another issue raised was that of a disconnect between the regional and national level at both CFIA and DFO. At CFIA, it was noted that the national level is not always aware of important regional challenges. At DFO, decisions are sometimes made at the DG level in National Headquarters, but when it comes to implementation at the regional level, the Regional Directors General are not always in agreement and are not obliged to abide by these decisions. This was described to be particularly problematic regarding declassification recommendations and decisions.

It’s difficult to know who has the final say at DFO.

3.4 Program delivery

3.4.1 Achievement of results

Despite challenges and issues in the program, evidence shows the CSSP is achieving its intended results where the program is being delivered.

Both internal and external interviewees were asked to provide a rating in response to the following question:

To what extent is the program achieving its intended results through its current key activities and outputs?

1. Minimizing the consumption of contaminated bivalve molluscan shellfish (thereby minimizing health risks).
2. Ensuring that only shellfish that meet food safety and quality standards reach domestic and international markets.

Most interviewees, both internal and external, agree that the two main objectives of the program are being met to either a great or moderate extent. There have been no deaths as a result of shellfish poisoning for over 30 years and the CSSP continues to successfully enable international export and conveyance of shellfish across provincial borders.

Many interviewees qualified their responses, saying that the program is meeting its intended results to a great extent “in the areas covered by the program.”

Factors that are helping the program to achieve results include:

• Prioritizing delivery of the program to high-risk areas;
• The presence of internal guidance documents within each federal organization (e.g., policies, flowcharts, checklists, standards of practice); and
• Sharing of best practice documents.

Reasons why some interviewees did not rate their responses as “to a great extent” include:

• A perception that more dedicated patrols and enforcement are needed or that there is not enough testing and monitoring;
• Better communication is needed about where it is safe to harvest;
• Perceived gaps in testing for contaminants such as norovirus and vibrio*, the latter of which is not covered by the CSSP but is covered by CFIA complimentary preventive control program requirements and associated inspections; and
• Interviewee awareness about illegally harvested shellfish.

*Note: Vibrio parahaemolyticus is a naturally occurring bacteria that grows rapidly in warming water. Vibrio illness can lead to severe gastrointestinal distress, including abdominal cramps, vomiting and diarrhea, and in serious cases can result in death. While of concern, it is not a contaminant covered by the CSSP.

While ECCC’s monitoring has decreased over the last ten years, they have been able to perform their role by prioritizing delivery in high-risk areas.

ECCC monitors waters to detect contamination levels and identify pollution sources. They make recommendations for classification to the RISCs for review and approval.

As shown in Figure 9, the number of areas monitored by ECCC from 2010-11 to 2020-21 has been in decline in both the Atlantic and Quebec regions while in Pacific region it remained relatively stable. Figure 10 shows that the number of samples being collected has decreased for the three regions over the same time period.

The low numbers for 2020-21 are partly due to the impacts of the pandemic (e.g., laboratory closures, restricted travel). Otherwise, the decreases in areas monitored/samples collected are largely due to resource constraints. The decreased resourcing has been occurring in conjunction with an increase in the number of requests received by program stakeholders/partners to classify new areas for harvest. Therefore, despite a decrease in ECCC’s capacity to deliver, interest in the program and requests for more harvest areas continue to grow, likely widening the gap in meeting stakeholder and partner demands.

ECCC manages the situation by giving priority to high-risk areas, and in some cases, new demands were able to be accommodated through prioritization; however, in the case of new requests for areas that have never been classified, they cannot be answered.

Source: ECCC.

*Information was not provided for the Atlantic region which limited the evaluation’s ability to determine a trend over the ten-year period.

Source: ECCC.

Note: 2020-21 low numbers are due in part to the pandemic.

While CFIA’s biotoxin analyses has decreased over the last five years, it has been effective in monitoring harvest areas covered by the CSSP, providing reasonable assurance of safe shellfish for consumption.

The CFIA tests shellfish from CSSP harvest areas for biotoxins. The CFIA also verifies compliance of shellfish as part of their inspection activities at SFCR-licensed* operators to ensure that shellfish being conveyed across provincial borders or exported abroad are safe for consumption. Regulatory limits for toxins and other hazards in the edible tissues of shellfish are set by Health Canada. Product that exceeds the regulatory limit is subject to controls (e.g., recall, disposal) and cannot be shipped domestically or abroad for consumption.

Figure 11 demonstrates that on average CFIA conducted 20,828 biotoxin analyses per year for the period 2016-17 to 2020-21. The level of analyses has declined 11.7 percent over the past five years.

Although the number of analyses conducted by CFIA each year has been decreasing, the proportion of shellfish biotoxin analyses exceeding the regulatory limit remained constant between 1.5% and 2.5%. Any worsening in these percentages could indicate more frequent or more intense biotoxin blooms, which could result in the need to close areas more frequently or leave them in closed status for longer periods.

CFIA also conducts analyses on shellfish for certain microbiological contaminants. For example, work related to norovirus monitoring and testing led to increased collaborative work and building informal relationships with partners, including Health Canada, the USFDA, American laboratories and others.

*Note: SFCR is the acronym for Safe Food for Canadians Regulations.

Note: The low analyses on samples conducted for the year 2020-21 was likely impacted by the implementation of Covid 19 pandemic restrictions (e.g., no travel, plant closures).

Source: CFIA.

Recalls are a risk mitigation action to prevent illness or death, by removing food from further sale or use at any point in the supply chain. In the case of shellfish, recalls can be initiated as a result of unacceptable test results, or the results of a food safety investigation, and the subsequent heath risk assessment.

As shown in Figure 12, there were a small number of recalled shipments of shellfish between 2012-13 and 2019-20. The number of recalls that were issued per 10,000 metric tonnes of bivalve molluscan shellfish produced/landed were stable, with a peak in 2015-16 mainly associated with a vibrio outbreak in British Columbia.

Source: CFIA.

Note: Program decision resulted in vibrio recalls being excluded starting in 2018-19, as vibrio is not covered by the scope of the CSSP.

Source: CFIA.

Note: 2016-17 and 2017-18 illness data is laboratory confirmed illness only and includes vibrio. Illnesses encompass shellfish consumed that was illegally harvested, as well as that harvested from areas in open status. Provincial reportable disease requirements limit the data.

Although it is understood that illnesses following the consumption of contaminated shellfish are under-reported, a review of several CFIA documents showed the number of confirmed cases of shellfish-related consumer illnesses are relatively low (Figure 13). This information is corroborated from interviews and program data, the latter of which shows that the number of confirmed cases (i.e., through an investigation) of shellfish-related consumer illness per 10,000 metric tonnes has decreased during the 2012-13 to 2020-21 period from 14 to zero.

There were 28 different shellfish-related food recalls from 2016-17 to 2020-21. Half of the recalls (fourteen) were related to waters in British Columbia. The shellfish species most associated was oysters, accounting for 86% of the recalls. Vibrio was the main causative agent (43%) of the recalls, followed by Salmonella (27%), Paralytic shellfish poisoning (17%), Norovirus (7%) and E. coli (6%).

Shellfish traceability

Federal regulations require that bivalve shellfish be tagged for commercial distribution so batches can be traced from harvester to consumer.

Increased attention was given to this issue in the Prime Minister’s 2019 mandate letters to the Minister of Health and the Minister of Fisheries, Oceans and the Canadian Coast Guard, in which the Ministers were specifically asked to develop a boat-to-plate traceability program to help Canadian fishers to better market their products.

Independent of the mandate letters, an initiative was put in place by the program in the Pacific region to address known traceability gaps.

Source: CFIA. (2021). Historical indicator data per province. CFIA. (2021). Vibrio was excluded starting in 2018-19 as risk mitigation measures for vibrio are the responsibility of “Safe Foods for Canadians” licence holders.

DFO’s CSSP compliance activities have increased.

DFO issues licenses for shellfish harvesting. Departmental C&P staff (i.e., fishery officers) patrol waters and beaches to monitor the compliance of commercial operators and the public in their harvest of shellfish. They enact and revoke prohibition orders to close and open harvest areas, respectively.

Over the last 10 years, the average number of hours that fishery officers spent on the CSSP as well as the average number of site checks has slowly been rising (Figures 15 & 16). This aligns with what we heard from DFO interviewees: there has been increased pressure to patrol and enforce compliance in more harvest areas, including sites that get closed for harvest to mitigate risk when they cannot be monitored by partners due to resource constraints.

The number of CSSP violations (Figure 17) has also been increasing in the Pacific and Atlantic regions, especially in the last three years. There have also been more violations per the number of site checks in the same time period (see Figure 18 for the non-compliance index). This data supports the rise in the intensity of the work that was expressed in interviews and the increased burden on C&P staff.

Source: DFO

3.4.2 Gaps in implementation

There are significant gaps in delivery, some of which present barriers for some stakeholders and partners.

Close to 80% of internal interviewees and at least ½ external interviewees perceived that the program does not currently provide equitable and inclusive access to shellfish harvesting sites for safe consumption.

• There is an inability to assess long-term closures for potential re-opening due to the lack of water quality and biotoxin monitoring resulting from capacity constraints. This could limit Indigenous groups’ access to areas they depend on for FSC harvest, or fresh and affordable protein.
• Some interviewees stated that the program is currently an impediment to the growth of the aquaculture industry, a view which has also been expressed by industry through position papers.
• User-pay models to address new demands present financial barriers and/or deterrents to new entrants:
  • Shifting the costs to harvesters risks marginalizing groups with less capacity and fewer resources, which may disproportionally affect Indigenous and recreational harvesters.
  • Several interviewees noted that user-pay models cause a “two-tier system”, where the program is bearing the costs of sampling/testing for some but not others. There is also concern that those bearing the costs may perceive they have the right to exclusive access to an area.
• While the program does not have responsibility to remediate harvest sites, ECCC does have a responsibility to promote remediation together with other jurisdictions and there continues to be demand for decontamination of harvest sites that have been closed long-term.
• Safe access for recreational harvesters is mostly limited to areas already being monitored for commercial harvesters.
• There is a lack of representation for recreational harvesters at interdepartmental governance meetings.
• Geographic barriers exist due to reduced coverage in remote areas. Those living in remote areas are most likely to rely on shellfish as a means to food security.
• A few interviewees also mentioned barriers to communication, such as language barriers.

While there are ad hoc efforts to address barriers, these are not mitigating issues in a comprehensive way.

3.4.3 Information for decision-making

There are some key sources of information that are lacking for operational decision-making and risk management.

• The CSSP is, by nature, a program that must consider risk management in all facets of decision-making and program delivery given the potential risks to shellfish consumers and the economic ramifications of closing borders to trade. As previously described, the program faces resource constraints. In order to make decisions about where to prioritize limited resources, the program has certain information requirements.
• Both interviews and documents reviewed revealed that the lack information on harvest areas appears to be the most significant gap when it comes to decision-making and risk management. Interviewees explained that information from DFO on shellfish locations and harvest activity is incredibly important. Monitoring is resource-intensive, and the information is critical so partners can prioritize their monitoring efforts in areas where it is most needed; for example, focusing on areas with higher-risk species or greater harvest volumes.

• It is not clear how far the 2014 growing area inventory and reclassification exercise (section 3.1) progressed, as the 2016-17 DFO Internal Audit of DFO’s role in delivering CSSP found that DFO lacked reliable, accurate and consistent CSSP information for decision-making. The audit noted that the program was making decisions on prioritization of program delivery based on this information, which was incomplete. The audit included a recommendation that senior management develop processes to obtain accurate, reliable and consistent shellfish harvesting data.
• According to internal documents, as of December 2018 standard operating procedures for harvest data collection and inventory of harvest data repositories were developed and communicated to regions. Further, a long-term plan to integrate CSSP harvest data into a larger DFO Systems Integration Project was established, but this component of the CSSP data repository project was not completed.
• A “refresh” of the “growing area inventory” (GAI) was initiated in 2020 to inform overall program decisions, including for the declassification of areas with no or low levels of harvest. In addition to responsibly aligning public resources with areas of significant active harvest, declassification is a method for reducing resource burdens for all CSSP partners.
• In the refresh, ECCC has been tasked with providing water quality data to each harvest area, and the results of sanitary surveys; DFO with providing information on harvest activity in each harvest area; and CFIA with confirming biotoxin monitoring information.

Limitations to growing area inventory data

While the program’s recent update of data on shellfish harvest areas across the country is an important step in providing the necessary information for prioritization of program delivery and monitoring activities, there are some aspects of the data and data collection processes that limit its usefulness. 

• Data is not available or complete for every area. Not all harvest activity is known or reported to DFO, and in some areas the department relies heavily on C&P officer observations.
• DFO issues licenses and collects fishery data from Management Areas that are significantly larger than the CSSP harvest areas, which means that the link between these datasets is imprecise.
• FSC harvest information is not necessarily specific or complete.
• Privacy Act restrictions mean that landing data for areas with fewer than six leaseholders/commercial fishers may not be reported.
• Data on recreational harvesting activity is not collected.

In conclusion, incomplete information makes it difficult to gather accurate data to support decision-making. Nevertheless, work is ongoing to improve the GAI and the quality of the information being used to support operational decisions.

3.4.4 Availability of performance information

There are gaps in the information available to measure and report on program performance.

The evaluation team requested data to report on program activities and outputs and analyse trends. While enough data was available to provide an assessment of some program activities that are contributing to the achievement of intended results, reliable data was not available for all indicators and several limitations in the data were observed:

• Scattered and/or inaccessible and/or not digitized data requiring significant effort to retrieve;
• Cases of partial information being retrieved, meaning that information was missing for years, regions or variables;
• Inability to confirm what was the best data source (e.g., for emergency closures and spill reports);
• Lack of meaningful targets in some cases (see side bar);
• Regional differences in how data was collected and presented (e.g., not all regions include chemical spills in spill reports;);
• Confusion about who was responsible for collecting certain information  (e.g., Section 14 requests, decisions on declassification); and
• Non-comparable data for key activities; for example, recommendations for classification.

Changing approaches to performance measurement might be linked to some of the data issues:

• the program developed a Performance Measurement Strategy in 2012 which focused heavily on quantitative data (23 indicators).
• In 2017-18, there was a change in approach, where the program identified the need to develop a more narrative approach. A review of two annual performance reports based on qualitative information shows the program is performing adequately, but with no tangible targets the achievements of the program are hard to track through time.
• It is understood that going forward the program will take a hybrid approach to performance measurement, providing both quantitative and qualitative information for decision-making.

A performance measurement regime is needed that identifies areas of both high performance and issues to be addressed.

3.4.5 Communication with stakeholders and partners

The SHELLI tool has improved internal processes to open and close harvest areas and to communicate externally but does have limitations.

The CSSP manual indicates that program partners have responsibilities to communicate externally about the status of harvest areas, for example by posting notices on beaches, conducting outreach, and sending automated notifications to commercial harvesters. The evaluation did not assess in detail the effectiveness of external communications, but it did examine the Shellfish Harvest Extents Latitude Longitude Information (SHELLI)^{Footnote 19} system implemented in March 2015 with two main purposes:

1. To provide a publicly accessible, real-time map of openings and closures of Canadian shellfish harvesting areas

   The public side of SHELLI is an interactive map of the country. Users can zoom in on any area and colours inform them of its status (open or closed). A click on an area allows the user to view additional details, such as the reason for closure, the species included, the public notice, the exact geographic description and the dates of issuance and enforcement. Subscribers receive automatic notifications when status changes. Before the implementation of SHELLI, it regularly took two to three days to implement a closure. This can now happen in as little as an hour.

   44% of external interviewees thought SHELLI was working well for them, however quarter said it was not user friendly and a quarter said the information was not timely.

   Over half (56%) of internal interviewees perceived the data on SHELLI as confusing, and the application is not seen as user friendly. It was stated that the general public sometimes turn to other tools to make informed decisions, such as DFO regional webpages, email, provincial health institute maps (e.g., British Columbia Centre for Disease Control Shellfish Harvesting Status Map)^{Footnote 20} or external apps developed by private companies (e.g.; Can you dig it).^{Footnote 21}

2. To provide an interface for the three federal partners to improve the production of prohibition orders and their revocation

   A prohibition order (PO) is put in place by DFO to close an area for shellfish harvest. Internally, SHELLI offers a map where CFIA and ECCC can select the reason for the PO (e.g., sudden influx of sewage or rainfall runoff; unsafe biotoxin levels detected) from an organization specific list. They can then select in a few different ways (polygons, circles, shape already used in the past, etc.) to display the area on the map where they wish to change the access to all or some species. At this step, the system creates a numbered polygon on the map, as well as a prohibition order number.

   About 40% of internal interviewees mentioned that SHELLI is useful and has improved internal processes.

   However, SHELLI is the only place where all program POs are listed, and while it is not designed as a database, it is used as such by some program personnel. Unfortunately, it is not a reliable source for this purpose; regions enter data inconsistently, and classified areas do not line up with the boundaries of growing areas. This also means that SHELLI is not reliable as a reporting tool.

There are limitations to information shared with stakeholders and partners.

• There can be confusion about classification versus status, as a harvest area can classified as “approved”, but its status could be “closed” in which case people are prohibited from harvesting unless they have a Management of Contaminated Fisheries Regulations (MCFR) license from DFO.
• Like other means of communicating closure information, the timeliness of information in SHELLI is affected by the length of time it takes for testing and closures to occur (information can only be presented after entered in the system). The timelines for these processes can present health risks for Indigenous and recreational harvesters as the time between harvest and consumption is likely shorter for these groups than for those eating commercial product; by the time the closure is put in place the shellfish may already be consumed.
• SHELLI is not necessarily accessible to all users, some of whom do not speak or read English or French as their first language or may not be able to easily access the internet.

The CSSP has implemented some strategies to address inclusivity and accessibility, which include providing outreach and producing posters and signage in multiple languages as well as attempting to provide additional flexibility and accommodation to First Nations communities.

4.0 Conclusions and Recommendations

4.1 Conclusions

The CSSP is a long-standing program. Its food safety mandate has remained the same since 1925, and the program has consistently had a focus on commercial markets and maintaining export. However, the context within which the program operates is constantly evolving, and it continues to face increasing demands and pressures. At the same time, there is no evidence that resource levels have increased since $4M and 25.8 FTEs were added in 2000-01. The program consistently struggles to track its financial resources, however there is strong evidence that the program does not have the level of funding needed to meet current demands, let alone new ones. This issue affects the level of program delivery to some stakeholders and partner groups and increases potential risks.

There are several governance mechanisms in place for the program. Regional governance is working well, and operational roles and responsibilities are clear, distinct and well-understood. However, there is a lack of strategic leadership from senior management and timely guidance to regional staff. The information needed to measure and report on program performance and make decisions about integrated program delivery has limitations. Due to the horizontal nature of the issues raised in the evaluation, recommendations have been addressed to the CSSP ADM Steering committee (comprised of the Vice President, Policy and Programs, CFIA; the ADM, Aquatic Ecosystems, DFO; and the ADM, Science & Technology, ECCC) rather than a specific ADM or Vice President.

Many of the challenges and issues found by the evaluation are not new. They have been documented in previous reviews dating back to the last horizontal evaluation of the program in 2007. The program has made some efforts to resolve some of its long-standing problems; however, there has been limited substantive progress. Despite this, evidence shows the CSSP is achieving its two objectives in the areas where the program is being delivered. While there has been a decrease in some of its activities, prioritization of resources to higher-risk areas has allowed the program to continue to perform its role and provide reasonable assurance of safe shellfish for consumption despite significant delivery gaps.

4.2 Recommendations

Recommendation #1:

It is recommended that the CSSP ADM Steering committee develop, articulate and communicate to program staff renewed strategic guidance about the program’s priorities, scope and reach to address increasing demands and evolving changes in its operating context.

Rationale:

The CSSP has focused on the same objectives since its inception and is missing foundational guidance to direct staff on how to deliver the program in the current environment of increased pressures and demands and a changed operational context. There is a lack of clarity about who the program is intended to serve beyond existing commercial harvesters, and how prioritization should take place. The focus on commercial trade markets affects delivery to other stakeholders and partners, such as: recreational harvesters, Indigenous people harvesting for Food, Social and Ceremonial reasons and/or commercial businesses who do not receive the same level of program delivery or bear additional costs relating to opening new areas for shellfish harvest. This lack of strategic guidance and leadership is not new, having been raised in multiple program reviews including the 2007 horizontal evaluation of the program. Operational staff, who face these pressures on the ground, require more clarity and tools to address current gaps in meeting the needs of program stakeholders and partners.

Recommendation #2:

It is recommended that the CSSP ADM steering committee clarify which federal partner is the lead of the CSSP. Further, it is recommended that they review all levels of interdepartmental governance and establish mechanisms to support more effective decision-making, ensure that long-standing issues are resolved, and operational staff are given timely guidance and advice when needed.

Rationale:

While CFIA, DFO and ECCC have specific roles and responsibilities, their interdependence cannot be understated when it comes to delivering the CSSP. There is a lack of clarity about which federal partner is the lead on the CSSP, given discrepancies between internal documents, naming CFIA as having this role, and publicly available information. Regardless, the agency does not have the authority to direct its federal partners or impose decisions. Partners use consensus to make joint decisions, however, it is clear that mechanisms are needed to improve timely decision-making, resolve long-standing issues and move forward with unified direction that gives the integrated mandate of the CSSP the same importance as individual partners’ mandates.

The current interdepartmental governance committees that support collaborative work between partners do not provide timely support to regions when they are unable to resolve issues on their own. The National Interdepartmental Shellfish Committee (NISC) is designed to discuss issues and make recommendations to the DG Operations committee for final decisions. This arrangement does not provide responsive support to regions, who sometimes wait years before they receive guidance, or in the worst-case scenario do not receive direction at all. Changes to key governance documents (e.g., Terms of References, Memorandum of Understanding, etc.) could help address governance issues. Further, these tools contain inconsistencies and do not currently support issue resolution.

Recommendation #3:

It is recommended that the CSSP ADM Steering committee seek out opportunities to address resource gaps facing the CSSP to meet existing and increasing demands for program services and delivery. Solutions are required to address current risks related to CSSP resource management in order to ensure that health risks are minimized, the shellfish industry remains strong, and stakeholder and partner needs are served appropriately by the program.

Rationale:

The evaluation found that all three federal partners are at maximum capacity and that the program as a whole does not have the level of resourcing needed to respond to existing and increasing demands. This has necessitated prioritization of delivery primarily to high-risk areas and has created unintended impacts such as impeding market growth or the lack of monitoring in long-term closures. While continued re-prioritization to fit delivery within resource allocations is one option for the program, it has the potential to increase risks to shellfish consumers and is not seen as sustainable for program delivery. As demand for services continues to grow and the program considers options for expansion, more dedicated resources are needed to deliver the CSSP effectively without relying on other internal funding sources on an opportunistic basis.

Robust resource tracking mechanisms are also needed to accurately quantify the cost of delivering the CSSP, including to further understand resource gaps to meet existing and increasing demands from stakeholders and partners.

Recommendation #4:

It is recommended that the CSSP ADM Steering committee review the CSSP’s performance measures and develop an integrated performance profile with meaningful performance indicators and targets at the program level. Useful and accurate information on the achievement of results is needed to better support the management of the program. Ideally, program level performance indicators could be integrated into the broader results frameworks of the three partner organizations. CSSP performance data needs to be tracked, collected and reported consistently and in a timely manner across all three federal partners and all regions. Roles and responsibilities for collecting and storing the data should also be established.

Rationale:

As the CSSP proceeds with modernization, there is a need to reassess performance information needs and data collection and tracking mechanisms, with a focus on integrated program results. While the program does collect some useful performance data, there are some limitations related to reliability, inaccessibility, incompleteness and confusion about roles and responsibilities for data collection and storage. Further, while the absence of reported fatalities since 1987 is positive, and could be considered an indicator of program success, in reality, fatalities are the worst-case scenario, and their occurrence would be an indication of program failure. Better information is needed to track and demonstrate where CSSP is performing well and where it needs improvement.

An integrated performance profile with more meaningful indicators and targets would allow for a more nuanced understanding of the program’s ability to achieve its objectives.

5.0 Annexes

Annex A – Evaluation methodology

The evaluation was conducted using an evaluation framework, which included the questions summarized in section 1.1., as well as indicators. Data was collected through the following methods and evidence was triangulated to decrease potential deficiencies with any one method and to develop the overall findings.

Key Informant Interviews

In order to gather the views of CFIA, DFO and ECCC employees, and of external stakeholders and partners to delivery of the CSSP, the evaluation team conducted 21 internal scoping interviews (29 key informants) to help plan the evaluation and 77 internal and external interviews during the conduct phase. All three CSSP regions and national headquarters were represented.

Interviews during the conduct phase:

• CFIA: 16
• DFO: 23
• ECCC: 21
• External: 17

Limitations and mitigation:

• Due to the COVID-19 pandemic, in-person interviews were not possible, thus they were conducted through video-conference or on the phone.
• The Fall 2021 election occurred while interviews were underway. Restrictions during the writ period caused time delays resulting in less time to contact and interview external key informants. This resulted in fewer external interviews than originally planned.
• There were difficulties reaching some external stakeholder and partner groups. For example, recreational harvesters are generally not well organized and so were difficult to identify. Indigenous partners have many demands on their time. Limited capacity to participate in interviews, combined with time constraints faced by the evaluation team (exacerbated by the election), made it difficult to conduct as many interviews as planned. Therefore, there are fewer overall informants from these two groups meaning their perspectives are not as widely represented in the findings. The evaluation team relied more on internal perspectives about their respective issues and needs.

International Benchmarking

The evaluation team completed a review of alternative design and delivery models in foreign programs related to shellfish sanitation - including those in the United States, United Kingdom, Australia, New-Zealand, Mexico, and Chile – in order to compare the CSSP to similar programs elsewhere. A literature review of published articles was conducted, as well as extensive review of publicly available Canadian and international reports, regulations, and other information on foreign programs.

Limitations and mitigation:

• It is difficult to find complete, publicly available data on all relevant parameters for foreign programs, making a full comparison impossible.
• Information is particularly difficult to find where countries do not have English or French as an official language.
• For these reasons, the evaluation did not attempt a true comparison between the CSSP and other programs, but rather sought information that could provide global context and useful contrasts to the CSSP.

Financial/Cost analysis

The evaluation team completed an analysis of expenditures and internal costs associated with program delivery from data provided by CFIA, DFO and ECCC. Extensive discussions with subject matter experts provided additional input  and allowed for a thorough understanding of the program’s resources.

Limitations and mitigation:

Annex B includes a detailed description of the methodology and limitations for this line of evidence.

Data Analysis

Program statistics and performance data from the CFIA, DFO and ECCC were analyzed to inform the evaluation about activities (including trends) contributing to the performance of the program.

Limitations and mitigation:

• There were limitations in some of the data received from the federal partners for the evaluation, which are explained in the performance section of the report (section 3.4). Some of the limitations described there made it difficult to draw conclusions about some of the indicators the evaluation intended to examine.
• The evaluation team mitigated limitations by focusing on the most reliable data, and by completing or replacing some incomplete or limited data with data gathered from multiple other sources (other Government of Canada databases and publications; research institutes/think-tanks), as well as by excluding some pieces of data that were judged unreliable.

Document / Legislative review

In order to understand the context and delivery of the CSSP in the three organizations, the evaluation team conducted a review of over 150 internal and publicly available documents from CFIA, DFO and ECCC and external sources (e.g., shellfish harvest industry)., Materials reviewed included, but were not limited to, previous reviews, internal governance documents, applicable legislative and regulatory documents, agreements, mandate and priority-setting documents, and in the case of external organizations, public websites.

Limitations and mitigation:

• Due to the large number of documents received from the program it was necessary  to prioritize and sample certain categories.

Process mapping

The evaluation team developed a process map to identify the steps and decision points in the program’s processes and better understand program complexities. A draft process map was created based on information gathered in the document review, and it was subsequently reviewed and validated by subject matter experts from CFIA, DFO and ECCC. Three group discussions (one in each CSSP region) with members from the three federal organizations took place after all other lines of evidence were complete and thus focused on areas of interest informed by the process mapping and identified during the evaluation. Overall, this line of evidence informed the evaluation team on the day-to-day operations of the program, explored complex processes, and allowed for an examination and better understanding of where processes are not working well or where the program faces challenges.

Limitations and mitigation:

• Due to the COVID-19 pandemic, in-person group discussion were not possible for this evaluation. They were thus conducted through video-conference.

Annex B – Costs analysis methodology

The costs analysis examined Canadian Food Inspection Agency (CFIA), Fisheries and Oceans Canada (DFO) and Environment and Climate Change Canada (ECCC) financial, human and materiel resources in support of the Canadian Shellfish Sanitation Program (CSSP) as follows:

*DFO CSSP project codes extracted from Abacus were: 90255 – CSSP; 96256 - CSSP Wastewater Treatment Module Project; and 96195 - Aquaculture Management for the CSSP.

**Bottom-up refers to breaking down a program into components and estimating the costs of each component based on the best available information. Activity-based costing uses historical cost information to assign direct costs to the program as well as allocate an appropriate portion of indirect costs.

Annex C – Illnesses caused by consuming shellfish contaminated with biotoxins

While there are several illnesses that can be caused by eating contaminated shellfish, three illnesses can be caused by eating shellfish contaminated with biotoxins that are not destroyed by cooking: paralytic shellfish poisoning, amnesic shellfish poisoning and diarrhetic shellfish poisoning.

Paralytic shellfish poisoning (PSP) is an illness that may have serious and potentially fatal effects. It is caused by eating bivalve shellfish and other molluscan shellfish that have been contaminated by toxins produced by certain species of microscopic marine algae found in coastal waters.

Amnesic shellfish poisoning (ASP) is an illness caused by domoic acid, a naturally occurring acid which is produced by some marine algae. Domoic acid can accumulate in filter-feeding bivalve molluscan shellfish such as clams, mussels, scallops and oysters. ASP was unknown in Canada until November 1987, when an outbreak in Eastern Canada resulted in four deaths.

Diarrhetic shellfish poisoning (DSP) is an illness caused by toxins that are produced by certain microscopic plants and can bio-accumulate in the tissues of shellfish. DSP is often short-lived and non-life-threatening; however, for some people, especially young children, the elderly, pregnant women and those with weakened immune systems, DSP can be very serious.

Source: Marine biotoxins in bivalve shellfish: Paralytic shellfish poisoning, amnesic shellfish poisoning and diarrhetic shellfish poisoning

Annex D – CSSP classifications

Approved

The classification assigned to a shellfish harvest area as determined by the shellfish control authority* from which shellfish can be harvested for direct consumption.

Conditionally approved

The classification assigned to a shellfish harvest area which has been determined by the shellfish control authority to meet approved area criteria for a predictable period. The period is conditional upon meeting established requirements and/or performance standards specified in a conditional management plan.

Restricted

The classification assigned to a shellfish harvest area as determined by the shellfish control authority where harvesting shall be by licence under the Management of Contaminated Fisheries Regulations and the shellfish, following harvest, is subjected to a suitable and effective treatment process through relaying or depuration.

Conditionally restricted

The classification assigned to a shellfish harvest area which has been determined by the shellfish control authority to meet, at a minimum, the restricted classification criteria for a predictable period. The period is conditional upon meeting established requirements and/or performance standards specified in a conditional management plan.

Prohibited

The classification assigned to a shellfish harvest area as determined by the shellfish control authority where shellfish harvesting is not permitted.

Generally speaking, restricted, conditionally restricted and prohibited areas are classified as such due to sanitary survey or water quality results that document a persistent pollution source risk.

*Shellfish control authority - The departments or agencies of the Government of Canada that are signatories to the interdepartmental MOU between CFIA and DFO and ECCC concerning the CSSP or provincial shellfish leasing bodies

Source: Canadian Shellfish Sanitation Program manual

Annex E –The legislative mandates of the three federal partners

CFIA

The Canadian Food Inspection Agency has responsibilities under the Safe Food for Canadians Act (an Act respecting food commodities, including their inspection, their safety, their labelling and advertising, their import, export and interprovincial trade) to regulate grades and standards for food; quality management and control or safety programs, and preventive control plans; and the design, construction, hygiene, sanitation and maintenance of equipment and facilities; as well as the conveyance across provinces and import/export; and food traceability requirements. The Minister of Agriculture and Agri-Food also has the authority to register/issue licenses authorizing the conveyance across provinces, or import/export, as well as conducting activities in respect of food commodities. They also have the authority to issue certificates/documents setting out information deemed necessary to facilitate export.

DFO

Fisheries and Oceans Canada has responsibilities under the Fisheries Act and associated regulations for implementing measures to maintain fish stocks; issuing/authorizing leases and licenses for fishing; and making fisheries management orders when required in order to address a threat to the proper management and control of fisheries and the conservation and protection of fish. The Minister is required to provide notice of fisheries management orders. The Minister may authorize rivers or other waters to be set apart for natural or artificial propagation of fish and grant special licenses/leases for planting/forming oyster beds and for the cultivation and production of oysters. Regulations lay out season dates for each area, minimum measurements, harvest limits, and the tools that may be used.

ECCC

Environment and Climate Change Canada does not have any regulatory responsibilities under the CSSP specific to fisheries management or public health protection. However, the department has administrative responsibility for Section 36-42 of the Fisheries Act, which relates to its role in pollution prevention under the Act’s General Provisions and Regulations. ECCC maintains recognized expertise in evaluating pollution sources and assessing pollution control methods and mechanisms. Furthermore, the department has the mandate to monitor and report on environmental quality as per its responsibilities under Part 3 of the Canadian Environmental Protection Act.

Annex F – CSSP roles and responsibilities

The roles and responsibilities are defined in the Memorandum of understanding between the Canadian Food Inspection Agency (CFIA) and Fisheries and Oceans Canada (DFO) and Environment Canada (EC) concerning the Canadian Shellfish Sanitation Program ("CSSP").

Responsibilities of the CFIA

The CFIA shall be the lead agency in the administration of the CSSP with regard to: the handling, processing, import and export of shellfish; the marine biotoxin monitoring program; and any other microbiological monitoring program not described in section 13.1.4 – "Responsibilities of EC".

The CFIA shall be responsible for:

1. Inspecting and issuing certificates of federal registration to plants that meet federal regulatory requirements and are engaged in the processing, holding and export of shellfish;
2. Licensing fish importers and inspecting imported shellfish;
3. Administering the marine biotoxin monitoring program and any other shellfish micro-biological monitoring program not described under EC's responsibilities in section 13.1.4;
4. Recommending to DFO the closing of shellfish areas because of unacceptable marine biotoxin, microbiological and chemical levels in shellfish stock, and advising DFO when shellfish areas are acceptable for the harvesting of shellfish;
5. Reviewing referrals from DFO for the issuing of licences for harvesting from closed areas, for relaying or depuration purposes;
6. Maintaining records, data bases and other documents in support of marine biotoxin, microbiological and chemical closures, recommended closure actions, and administrative evaluations by internal and external auditors;
7. Ensuring proper application of prescribed analytical and reporting procedures in CFIA laboratories and private laboratories approved in accordance with the CSSP Manual of Operations, including adequate quality assurance, performance standards and quality control of the laboratory-generated data;
8. Ensuring proper application of prescribed sampling procedures by qualified parties, including adequate quality assurance and quality control of the collected samples;
9. Supporting DFO in its notification activity related to section 13.1.3(e), and providing or making available to interested parties' information on program activities;
10. Implementing CFIA elements of jointly developed Management Plans for "Conditionally Approved" areas; and
11. Participating in the CSSP audit program, as well as in external audits by such bodies as Health Canada and the United States Food and Drug Administration.

Responsibilities of DFO

DFO shall be the lead agency in the administration of the CSSP with regard to the harvesting of shellfish and shall be responsible for:

1. Opening and closing shellfish areas on the basis of:
   • classification recommendations from EC, based on the sanitary and bacteriological water quality of the shellfish areas, and agreed to by the regional Shellfish Area Classification Committees; and
   • recommendations from the CFIA on marine biotoxin levels, and microbiological and chemical levels in shellfish areas;
2. Posting, patrolling and enforcing shellfish closures in accordance with the Fisheries Act;
3. Controlling shellfish relaying operations and harvesting for depuration operations;
4. Implementing DFO elements of jointly developed Management Plans for "Conditionally Approved" areas;
5. Providing notification to the CFIA, EC, stakeholders and other interested parties, on locations, boundaries and timing of harvesting closures and openings;
6. Maintaining records of the opening and closure of shellfish areas, as well as records of enforcement patrols, in support of reviews by external or internal auditors, and providing the CFIA and EC with annual patrol enforcement activity reports;
7. Consulting with the CFIA and EC prior to the commencement of any new developmental or exploratory shellfish fisheries, and/or the issuance of any new licences or permits thereto; and
8. Participating in the CSSP audit program, as well as in external audits by such bodies as Health Canada and the U.S. Food and Drug Administration

Responsibilities of ECCC

1. ECCC shall be the lead agency in the administration of the CSSP with regard to recommending the appropriate classification of shellfish harvest waters based upon the sanitary and bacteriological water quality conditions of the area, and shall be responsible for:
2. Conducting comprehensive sanitary and bacteriological water quality surveys of the shellfish areas in Canada, in accordance with the CSSP Manual of Operations criteria;
3. From the surveys, determining the sources of point and non-point pollution, the degree of contamination and the extent of area contamination, and recommending the location of closure lines;
4. Recommending to the regional Shellfish Area Classification Committees specific classifications of areas and their boundaries, on the basis of survey results and the classification definitions in the CSSP Manual of Operations;
5. Maintaining records, data bases, sectoral maps, survey reports, central files and other documents in support of classification action and administrative reviews by internal and external auditors;
6. Ensuring proper application of prescribed analytical and reporting procedures in EC laboratories, private laboratories approved in accordance with the CSSP Manual of Operations, and laboratories under contract to EC, including adequate quality assurance and quality control of the laboratory-generated data;
7. Ensuring proper application of prescribed sampling procedures by qualified parties, including adequate quality assurance and quality control of the collected samples;
8. Promoting pollution prevention, regulatory compliance, remediation and restoration of shellfish areas, together with federal/ provincial/ municipal agencies and other stakeholders;
9. Supporting DFO in its notification activity pursuant to section 13.1.3(e), and providing or making available to interested parties information on program activities;
10. Upon request, providing to DFO available information on water quality for areas proposed;
11. Implementing EC elements of jointly developed Management Plans for "Conditionally Approved" areas; and
12. Participating in the CSSP audit program, as well as in external audits by such bodies as Health Canada and the U.S. Food and Drug Administration.

Source: Memorandum of understanding between the Canadian Food Inspection Agency (CFIA) and Fisheries and Oceans Canada (DFO) and Environment Canada (EC) concerning the Canadian Shellfish Sanitation Program ("CSSP")

Annex G – More details on estimated current* CSSP delivery costs

43 staff were consulted for the estimates exercise:

ECCC: 4

CFIA: 17

DFO: 22

*Subject matter experts were asked to estimate "current" costs. Given the influence of the pandemic, they used numbers which were representative of a normal year.

Annex H – Management Action Plan (MAP)

Horizontal Evaluation of the Canadian Shellfish Sanitation Program(CSSP) (Project # 96744)
Performance Measurement and Evaluation Committee (PMEC) Date: June 9, 2022
MAP Completion Target Date: November 2024
Lead ADM/DC: CSSP ADM Steering Committee

Recommendation 1: Completion date: November 2023

Recommendation: It is recommended that the CSSP ADM Steering committee develop, articulate and communicate to program staff renewed strategic guidance about the program’s priorities, scope and reach to address increasing demands and evolving changes in its operating context.

Rationale: The CSSP has focused on the same objectives since its inception and is missing foundational guidance to direct staff on how to deliver the program in the current environment of increased pressures and demands and a changed operational context. There is a lack of clarity about who the program is intended to serve beyond existing commercial harvesters, and how prioritization should take place. The focus on commercial trade markets affects delivery to other stakeholders and partners, such as: recreational harvesters, Indigenous people harvesting for Food, Social and Ceremonial reasons and/or commercial businesses who do not receive the same level of program delivery or bear additional costs relating to opening new areas for shellfish harvest. This lack of strategic guidance and leadership is not new, having been raised in multiple program reviews including the 2007 horizontal evaluation of the program. Operational staff, who face these pressures on the ground, require more clarity and tools to address current gaps in meeting the needs of program stakeholders and partners.

Management Response

It is agreed that increasing program demands and evolving operational demands requires an updated strategic approach for CSSP partners and stakeholders.

Government of Canada controls for shellfish sanitation have existed for nearly a hundred years. Throughout the program’s evolution, the CSSP partners have been unified in the goal of protecting consumers at home and abroad from unsafe shellfish.  While the CSSP partners believe that design and delivery of a science-based domestic program enables trade, the program also contributes to the safe harvest of shellfish for Food Social and Ceremonial purposes as well as Recreational uses. CSSP partners recognize the benefits of transparent policies and communication.

This recommendation aligns with the on-going review of program delivery priorities and approaches and provides additional context and information for partners to consider. This work will enable a shared understanding of priorities and goals so that we can determine and communicate how best to continue to provide food safety protection for harvesters, support industry and address program expansion requests.

Link to larger program or departmental results (if applicable)

Fisheries and Oceans Canada:
2020-2021 Departmental Results Framework
Result 1.1: Canadian Fisheries are sustainably managed
Result 1.2: Canadian aquaculture is sustainably managed

Canadian Food Inspection Agency:
Program Result 9: Risks are managed

Environment and Climate Change Canada:
2021-22 Departmental Results Framework
Result: Canadians have clean water

Recommendation 2: Completion date: November 2024

Recommendation: It is recommended that the CSSP ADM steering committee clarify which federal partner is the lead of the CSSP. Further, it is recommended that they review all levels of interdepartmental governance and establish mechanisms to support more effective decision-making, ensure that long-standing issues are resolved, and operational staff are given timely guidance and advice when needed.

Rationale: While CFIA, DFO and ECCC have specific roles and responsibilities, their interdependence cannot be understated when it comes to delivering the CSSP. There is a lack of clarity about which federal partner is the lead on the CSSP, given discrepancies between internal documents, naming CFIA as having this role, and publicly available information. Regardless, the agency does not have the authority to direct its federal partners or impose decisions. Partners use consensus to make joint decisions, however, it is clear that mechanisms are needed to improve timely decision-making, resolve long-standing issues and move forward with unified direction that gives the integrated mandate of the CSSP the same importance as individual partners’ mandates.

The current interdepartmental governance committees that support collaborative work between partners do not provide timely support to regions when they are unable to resolve issues on their own. The National Interdepartmental Shellfish Committee (NISC) is designed to discuss issues and make recommendations to the DG Operations committee for final decisions. This arrangement does not provide responsive support to regions, who sometimes wait years before they receive guidance, or in the worst-case scenario do not receive direction at all. Changes to key governance documents (e.g., Terms of References, Memorandum of Understanding, etc.) could help address governance issues. Further, these tools contain inconsistencies and do not currently support issue resolution.

Management Response

It is agreed to review the CSSP governance, both at the national and regional levels. The CSSP partners are committed to delivering a cohesive program that reflects both the mandates of the partner departments/agency, as well as the ultimate objective of the CSSP.

The interdependence required for effective decision-making and delivery of the program will be a key consideration in the governance review. Clear governance structures will provide mechanisms for support, timely decision-making and issue resolution, which can then be communicated to partners and stakeholders.

Link to larger program or departmental results (if applicable)

Fisheries and Oceans Canada:
2020-2021 Departmental Results Framework
Result 1.1: Canadian Fisheries are sustainably managed
Result 1.2: Canadian aquaculture is sustainably managed

Canadian Food Inspection Agency:
Program Result 9: Risks are managed

Environment and Climate Change Canada:
2021-22 Departmental Results Framework
Result: Canadians have clean water

Recommendation 3: Completion date: March 2024

Recommendation: It is recommended that the CSSP ADM Steering committee seek out opportunities to address resource gaps facing the CSSP to meet existing and increasing demands for program services and delivery. Solutions are required to address current risks related to CSSP resource management in order to ensure that health risks are minimized, the shellfish industry remains strong, and stakeholder and partner needs are served appropriately by the program.

Rationale: The evaluation found that all three federal partners are at maximum capacity and that the program as a whole does not have the level of resourcing needed to respond to existing and increasing demands. This has necessitated prioritization of delivery primarily to high-risk areas and has created unintended impacts such as impeding market growth or the lack of monitoring in long-term closures. While continued re-prioritization to fit delivery within resource allocations is one option for the program, it has the potential to increase risks to shellfish consumers and is not seen as sustainable for program delivery. As demand for services continues to grow and the program considers options for expansion, more dedicated resources are needed to deliver the CSSP effectively without relying on other internal funding sources on an opportunistic basis.

Robust resource tracking mechanisms are also needed to accurately quantify the cost of delivering the CSSP, including to further understand resource gaps to meet existing and increasing demands from stakeholders and partners.

Management Response

It is agreed that the CSSP needs a solid foundation for funding to address resource gaps, thus enabling respective partner activities and existing and future program delivery. An improved understanding of CSSP expenditures, such as through use of a common financial code would provide information to support resourcing conversations.

Link to larger program or departmental results (if applicable)

Fisheries and Oceans Canada:
2020-2021 Departmental Results Framework
Result 1.1: Canadian Fisheries are sustainably managed
Result 1.2: Canadian aquaculture is sustainably managed

Canadian Food Inspection Agency:
Program Result 9: Risks are managed

Environment and Climate Change Canada:
2021-22 Departmental Results Framework
Result: Canadians have clean water

Recommendation 4: Completion date: June 2024

Recommendation: It is recommended that the CSSP ADM Steering committee review the CSSP’s performance measures and develop an integrated performance profile with meaningful performance indicators and targets at the program level. Useful and accurate information on the achievement of results is needed to better support the management of the program. Ideally, program level performance indicators could be integrated into the broader results frameworks of the three partner organizations. CSSP performance data needs to be tracked, collected and reported consistently and in a timely manner across all three federal partners and all regions. Roles and responsibilities for collecting and storing the data should also be established.

Rationale: As the CSSP proceeds with modernization, there is a need to reassess performance information needs and data collection and tracking mechanisms, with a focus on integrated program results. While the program does collect some useful performance data, there are some limitations related to reliability, inaccessibility, incompleteness and confusion about roles and responsibilities for data collection and storage. Further, while the absence of reported fatalities since 1987 is positive, and could be considered an indicator of program success, in reality, fatalities are the worst-case scenario, and their occurrence would be an indication of program failure. Better information is needed to track and demonstrate where CSSP is performing well and where it needs improvement.

An integrated performance profile with more meaningful indicators and targets would allow for a more nuanced understanding of the program’s ability to achieve its objectives.

Management Response

It is agreed that a renewed performance framework, with meaningful indicators and targets, and data management practices, will improve our ability to reliably and accurately report on the program. Each of the CSSP partners has its respective performance approaches under the GOC policy on results; however, none of these link clearly to the CSSP. The CSSP Performance Measurement Strategy needs to be reviewed with this performance lens.

Clear and specific measures and targets will enable improved analysis of relevant data, thus supporting continuous improvement. Up to date Performance information and processes for compiling this information more efficiently and effectively, will enable the program to improve how we assess our overall performance, as well as demonstrate achievement of outcomes and goals to partners and stakeholders.

Link to larger program or departmental results (if applicable)

Fisheries and Oceans Canada:
2020-2021 Departmental Results Framework
Result 1.1: Canadian Fisheries are sustainably managed
Result 1.2: Canadian aquaculture is sustainably managed

Canadian Food Inspection Agency:
Program Result 9: Risks are managed

Environment and Climate Change Canada:
2021-22 Departmental Results Framework
Result: Canadians have clean water