Summary of the Evaluation of the Certification and Market Access Program for Seals (CMAPS)
Evaluation of the Certification and Market Access Program for Seals graphic summary
(PDF - 228 KB)
About the Evaluation
The evaluation was conducted between March and November 2019 to provide information for senior management decision-making. Evidence was gathered through a document review, 13 interviews, 2 case studies, a project file review (n=28) and a literature review. The evaluation objectives were to examine stakeholder needs, assess program effectiveness, identify factors impacting performance, and document lessons learned.
About the Program
The Certification and Market Access Program for Seals (CMAPS) was launched in 2015 to mitigate the impacts of the total seal products ban imposed by the European Union (EU). CMAPS was designed to assist Canadian Indigenous peoples access the EU market using the Indigenous exemption, and to assist the commercial seal products industry develop and renew other market access. Activities and resources aligned to three pillars that reflected program objectives.
Figure 1: Three Pillars target specific points on the seal products value chain
Description
Pillars target specific points on the seal products value chain.
The graphic depicts a row of 6 interconnected arrow boxes. Each arrow represents a point along the value chain for seal products. From left to right, they are:
- Pre-harvest
- Harvest
- Processing
- Product development
- Promotion and branding
- Sale and export
Each of the three pillars of CMAPS target different sections of the value chain.
Pillar 1 activities target sale and exports at the end of the chain.
Pillar 2 activities target all points along the entire value chain from pre-harvest through to sale and export.
Pillar 3 activities target the two last steps of the value chain: promotion and branding, and sale and export.
Contributions supported eligible activities under each pillar as follows:
Pillar 1 - Activities such as software development, maintenance and upgrade; and training to operate software to certify and track Indigenous origin in order to enter the EU
Pillar 2 - Eligible activities included business training and mentoring; acquisition and maintenance of equipment; feasibility studies; and marketing and business development
Pillar 3 - Market access related initiatives such as feasibility studies directly related to market access, and promotional activities for seal products.
Key Findings
To varying degrees, the CMAPS has made progress on meeting its objectives. Perhaps the greatest accomplishment is that positive results were realized given the challenging context in which the program operates.
CMAPS is the only federal program solely focused on market access for seal products, and is seen as a demonstration of support at a critical point for the Canadian seal products industry. CMAPS is designed to address two broad sets of needs: 1) market access and 2) capacity building within Indigenous communities to access and leverage the Indigenous exemption. These require long term attention. CMAPS provides an opportunity for governments, industry, and communities to work in tandem to address challenges impacting market access.
Under Pillar 1, capacity to enable the Indigenous exemption was established. Nunavut and the Northwest Territories became Recognized Bodies. However, tracking systems to support certification remain largely as they existed prior to CMAPS, and do not extend to processed products. Uncertainty over compliance with the EU regulation created challenges, and Recognized Bodies are still seeking clarification from the EU before investing in further systems development. In addition, information about the certification process has not been widely distributed, and stakeholders outside the process have limited understanding of requirements.
Under Pillar 2, there was some improvement in capacity to support exports. Project funding resulted in a Circumpolar Crafters’ Network whose goal is to support trade and product development among crafters. Market intelligence specific to opportunities for leveraging the Indigenous exemption was obtained through direct experience and meetings. Some product development and skill-building workshops were held, but were not widespread. The key challenge in this regard is the initial underestimation of what was needed to increase capacity to leverage the Indigenous exemption.
Under Pillar 3, the objectives related to the broader seal products industry were well advanced. The contributions supported promotional events; participation in international trade shows; and market research. For the first time, there is a high level of consensus among industry players on a marketing strategy to securing market access, and a funding application to the Canadian Fish and Seafood Opportunities Fund received a high level of support from stakeholders. However, this does not replace the need for funding to support individual marketing activities.
Factors Impacting Performance
Facilitators
- Networking and sharing information
- CMAPS expertise and focus
- Eligible activities
- Local stakeholder involvement
- Evidence of personal impact
Challenges
- Timing of project decisions
- Management of contribution agreements
- Leveraging resources of other federal programs
- Lack of capacity in the program and among stakeholders
Resources
Figure 2: CMAPS Expenditures, Planned and Actual, 2015-2016 to 2018-2019
Description
Figure 2 is a vertical bar graph comparing the planned and actual expenditures for the program over three different categories as well as the overall total of the three categories. An accompanying table below shows the values associated with each bar on the graph. Below this, an additional table shows the percentage of funds used per category.
The first of the three categories is salaries. The planned salaries was $1,042,436 and the actual was $978,261. This means that 94% of the funds were used that were planned for salaries.
The second category is operating. The planned operating was $666,889 while the actual was only $227,101. This means that only 34% of the planned operating funds was actually spent.
The third category is contributions. The planned contributions was $2,019,615 and the actual was $1,848,854. This means that 92% of the contributions funding was actually allocated.
Finally, the total spending for the program during the period of evaluation was planned to be $3,728,940 while the actual spending was $3,054,217. This means that overall the program actually spent 81% of the planned amount.
A marked drop in operating spending occurred in 2017-18 coinciding with a planned reduction in operating costs due to lower activity in supporting applications for Recognized Bodies. However, this is insufficient to explain the difference, and suggests that other planned activities did not occur. As operating costs were not tracked by pillar, it is challenging to isolate specific causes of the gap between planned and actual resources.
Figure 3: Annual Operating Costs, Planned and Actual 2015-16 to 2018-19
Description
Figure 3 shows a line graph with two lines that represent the planned and actual operating costs for the program. The gap between the lines illustrates the variation between planned and actuals costs.
Planned operating costs exceeded the actual in each of the four years presented. In 2016-17, the gap narrowed considerably from the previous year as actual costs approached planned costs. In 2017-18, actual costs dropped steeply, while planned costs rose sharply. This created the highest variation between planned and actual amounts and is depicted by an arrow. The gap narrowed somewhat in 2018-19, but is still the second widest of the four years presented
Lessons Learned
Description
Lessons Learned
The graphic depicts five rows of interconnected arrow boxes. Each row starts with a lesson learned followed by three supporting points, for a total of four arrow boxes per line. The arrows point forward to illustrate that the lessons learned are considerations for future programming.
Lesson 1: Consider multi-year contribution agreements
- To utilize resources more efficiently
- To enable cohesiveness of results in capacity building
- To accommodate factors of geography, gender, and language
Lesson 2: Consider simplifying the reporting requirements under contribution agreements
- To reduce capacity required to administer projects
- To better identify achievements of program objectives
- To streamline the collection of performance data
Lesson 3: Build capacity by engaging directly with stakeholders in local communities
- Face to face interaction is the most effective method to communicate about the program
- Shepherding proponents through government processes is necessary with CMAPS stakeholders
- Develop a plan to build capacity that reflects updated needs assessment
Lesson 4: Apply program management practices and resources consistently to the program
- Improve training in managing contribution agreements
- Provide a consistent approach to reporting requirements
- Organize program files so that effective knowledge transfer can occur
Lesson 5: Engage the knowledge and expertise of DFO regions and other government entities more fully
- Facilitate access for stakeholders with low capacity and awareness of government programs
- Address needs that are outside the scope of DFO with respect to the seal products industry
- Increase stakeholders capacity and program effectiveness
- Date modified: