Evaluation of the Fisheries and Aquaculture Clean Technology Adoption Program (FACTAP)

Final Report
Project Number 96395
November 2020

Table of Contents

1. 1.0 Evaluation context
   1. 1.1 Purpose
   2. 1.2 Scope
   3. 1.3 Methodology
2. 2.0 Program context
   1. 2.1 At the Federal level
   2. 2.2 At the Departmental level
   3. 2.3 Program governance
   4. 2.4 Financial summary
3. 3.0 Evaluation findings
   1. 3.1 Summary of key findings
   2. 3.2 The FACTAP addresses ongoing needs
   3. 3.3 Contribution agreements have been signed
   4. 3.4 Cleantech is being adopted by recipients
   5. 3.5 The FACTAP contributes to environmental performance
   6. 3.6 The role of the FACTAP is unique
   7. 3.7 Recipients receive support during application processes
   8. 3.8 Evaluation of proposals is efficient
   9. 3.9 The FACTAP leverages provincial government funding
   10. 3.10 Funds are distributed to recipients efficiently
   11. 3.11 Use of FACTAP performance information is limited
   12. 3.12 Promotion, communication and awareness of the FACTAP is limited
   13. 3.13 The FACTAP applies good practices in program delivery
4. 4.0 Conclusions and recommendations
5. 5.0 Annexes
   1. Annex A - Methodology
   2. Annex B - Management action plan (MAP)

1.0 Evaluation context

1.1 Purpose

This evaluation assessed the , effectiveness and efficiency of Fisheries and Oceans Canada’s (DFO) Fisheries and Aquaculture Clean Technology Adoption Program (FACTAP).

The evaluation was conducted as required by the Policy on Results (2016) and the Financial Administration Act (1985). The purpose of the evaluation was to provide senior management with information for decision-making and to identify lessons learned and good practices that may be applied to other grant and contribution programs within DFO.

1.2 Scope

This is the first evaluation of the FACTAP. It covers the three-year period December 2017 through March 2020 and included all DFO regions: Newfoundland and Labrador, Maritimes, Gulf, Quebec, Ontario and Prairie, Pacific, Arctic^{Footnote 1}, and National Headquarters.

The evaluation was conducted by DFO’s Evaluation Division between January 2020 and November 2020.

1.3 Methodology

The evaluation was designed to respond to the questions listed below. Information gathered from multiple lines of evidence was triangulated to address the evaluation questions. The methodology included interviews, document review, literature review, a survey to recipients, and documented examples. The evaluation methodologies, limitations and mitigation strategies are discussed in Annex A.

Evaluation questions

1. To what extent does the FACTAP address ongoing needs?

Effectiveness

2. To what extent are operators in the Canadian fisheries and aquaculture sectors aware of the FACTAP?
3. To what extent is clean technology being adopted by operators in the Canadian fisheries and aquaculture sectors?
4. To what extent does the FACTAP contribute to the environmental performance of the fisheries and aquaculture sectors?

Efficiency

5. To what extent is the FACTAP efficient?
6. To what extent are Gender-based Analysis Plus (GBA+)^{Footnote 2} considerations included?
7. To what extent does the FACTAP address challenges and explore opportunities for improvements?

2.0 Program context

2.1 At the Federal level

The FACTAP is part of the Government of Canada’s Clean Growth Hub (CGH),^{Footnote 3} a federal ecosystem of support for clean technology across six stages of an Innovation Spectrum, as illustrated below. Cleantech refers to any process, product, or service that reduces negative environmental impacts.

Description

The diagram depicts the six stages of the Clean Growth Hub’s Innovation Spectrum including research and development, pilot and demonstration, commercialization and market entry, growth and scale up, export, and clean technology adoption.

Sixteen federal departments and agencies form the Hub and allocate program funds. Of these, DFO’s FACTAP budget represents 0.8% of CGH funding.^{Footnote 4}

The Hub also advances the Clean Technology and Innovation pillar of the Pan-Canadian Framework on Clean Growth and Climate Change (PCF) and is part of the Innovation and Skills Plan.

2.2 At the Departmental level

The FACTAP supports the Clean Technology Adoption Stage of the Innovation Spectrum by funding projects to implement market-ready clean technologies, processes and sustainable practices^{Footnote 5} into the day-to-day operations of operators in the fisheries, aquaculture and processing sectors. The fund is intended to improve industry environmental performance, reduce negative impacts on aquatic ecosystems, and contribute to sustainability management.

The FACTAP reports to the CGH on environmental results and the information is collected as part of the Clean Technology Data Strategy.^{Footnote 6}

Eligible groups under the FACTAP include: commercial enterprises, such as companies and individuals; Indigenous organizations, such as groups, associations or communities; non-profit organizations; and industry or professional association.

The FACTAP falls under the Aquaculture Management Program within DFO’s Departmental Results Framework.

2.3 Program governance

The FACTAP is a grant and contribution program that was established in 2017-18 and is set to sunset in 2020-21. The program is administered by five FTEs. This includes one regional program coordinator (RPC) per FACTAP administrative region (Pacific, Central & Arctic, Atlantic), as well as a National Program Manager and a National Program Officer located in the national headquarters (NHQ), as illustrated below.

Description

The figure displays a map of Canada depicting the FACTAP’s Pacific, Central & Arctic, and Atlantic administrative regions. The map also depicts the number of FTE’s per region. There is one FTE in each administration region, as well as two FTE’s in the program’s national headquarters.

Similar to other programs in DFO and CCG, there is no line reporting between FACTAP’s administrative regions and the NHQ. This team is supported by two levels of decision-making committees: a Regional Proposal Review Committee (RPRC) for each region, as well as a National Steering Committee (NSC).

2.4 Financial summary

The FACTAP will receive a total budget of $19 M over the lifespan of the program. This includes salaries for 5 FTEs, operations and maintenance (O&M), and grants and contributions (Gs&Cs), as illustrated below.

Description

The figure depicts FACTAP total actual expenditures ($ in millions) for salaries, O&M, and contributions.

In 2017-18, the program spent $1.2 million.

In 2018-19, the program spent $4.7 million.

In 2019-20, the program spent $6.4 million.

In 2020-21, the program planned to spend $6.5 million. Only planned information was available for this year.

The FACTAP contributes up to 75% of eligible project costs. All projects must also include a recipient contribution of 15% and a minimum 10% provincial and territorial (P/T) government funding contribution (see section 3.8 for further details).

3.0 Evaluation findings

3.1 Summary of key findings

Under its current mandate and design, the FACTAP performs well. However, FACTAP regional program coordinators face administrative workload that limits the ability of the program to pursue improvements that might generate broader adoption and improvements in environmental performance within the fisheries and aquaculture sectors.

The FACTAP contributes to federal and departmental priorities

The FACTAP supports the adoption of clean technology within the fisheries and aquaculture sectors, particularly among small and medium enterprises, and the number of signed contribution agreements have increased yearly. Through the program’s current activities, the FACTAP contributes to federal and departmental priorities such as promoting clean growth, reducing greenhouse gas (GHG) emissions and, addressing marine pollution.

In addition to funding the adoption of market-ready cleantech, there is a need in the fisheries and aquaculture sectors to support cleantech at other, early, stages of the Clean Growth Hub’s Innovation Spectrum. Most notably, the Research and Development, Pilot and Demonstration and Growth and Scale-up stages.

The FACTAP can achieve greater impacts at the sector level

Through the program’s current activities, the FACTAP is supporting improvements in the environmental performance of the recipients it funds. For example, by reducing energy consumption and greenhouse gas emissions.

Generating broader adoption and improvements in environmental performance among operators at the sector-level remains a challenge. Operators lack of knowledge of available cleantech and of funding opportunities. Also, the FACTAP largely funds projects that are less than $100,000 which limits the program’s impact at the sector level. The FACTAP can engage in a proactive and strategic funding approach to increase the program’s impact on operators’ environmental performance.

There are factors that limit the FACTAP’s ability to generate program awareness

Overall, the FACTAP is administered efficiently. Nevertheless, FACTAP regional program coordinators face a significant administrative workload due to the effort required to see applicants through the project application, evaluation of proposals, establishment of contribution agreements, distribution of funds, and reporting processes. This is particularly true given the size of FACTAP’s administrative regions and the limited technical capacity of applicants to fulfill application and reporting requirements, such as proposals. As a result, program promotion is not a priority for regional program coordinators and this contributes to a low awareness of the program among operators.

3.2 The FACTAP addresses ongoing needs

The FACTAP addresses Government of Canada priorities and issues of importance to Canadians, as listed below. Survey respondents and key informants interviewees said that the FACTAP addresses their business needs by supporting improvements of environmental performance (e.g. reductions in GHG emissions, energy efficiency).

Pan Canadian Framework for Clean Growth and Climate Change

• The FACTAP is helping Canadian companies deliver on greenhouse gas emissions reductions beyond 2030 and meet current commitments to invest in technology that bends the cost curve on GHG reductions.

Ocean Plastics Charter

• The FACTAP supports reductions in marine pollution. Under the Ocean Plastics Charter, Canada commits to taking concrete and ambitious action to address marine plastic litter.

Ghost Gear Reduction

• The FACTAP contributes to Departmental and Federal commitments towards the Global Ghost Gear Initiative^{Footnote 7} and to the United Nations’ Sustainable Development Goal 14 – Life Below Water.

Supporting the protection of sensitive marine areas

• The FACTAP supports efforts to protect sensitive marine areas, such as Marine Protected Areas, where the Government of Canada, provinces and First Nations are working to minimize impacts to ecologically sensitive areas.

Most survey respondents and key informants also mentioned that there is a need to support cleantech across some stages of the innovation spectrum, specifically Pilot and Demonstration, Research and Development, and Growth & Scale-up.

• Notably, there is strong support for the FACTAP to be expanded to include Pilot and Demonstration. According to interviewees, it would be an opportunity to identify excellent pilot projects, and funding those activities would align with the original purpose of the program.
• There is also a need to fund the most applied stages of Research and Development.

3.3 Contribution agreements have been signed

As of March 2020, the FACTAP has signed 86 contribution agreements totalling $20,882,275 in total project value to support improvements in clean technology for the fisheries, aquaculture and fish processing sectors. This includes Total FACTAP contributions of $11,324,970.

For every $1.00 of FACTAP funding, $0.84 is invested through private sector and provincial investments.

The majority (52%) of projects were funded in the Pacific administrative region, followed by the Central and Arctic (26%) and Atlantic (22%), as depicted below. No projects have been funded in the Territories.

Description

The figure depicts the total number of projects and their dollar value per FACTAP administrative region.

In the Pacific region, 45 contribution agreements have been signed for a total of 5.3 million.

In the Central & Arctic region, 22 contribution agreements have been signed for a total of 3.7 million.

In the Atlantic region, 19 contribution agreements have been signed for a total of 2.1 million.

The FACTAP funds the same number of projects between fisheries and aquaculture sectors, as depicted below. A few projects target processing operators.

Description

The pie chart depicts the distribution of signed contribution agreements by sector. Forty-seven percent of projects have been signed in the Aquaculture and Fisheries sectors, each. Seven percent of contribution agreements were signed in the processing sector.

The majority (58%) of contribution agreements focus on projects less than $100,000 in total project value. In three cases, the FACTAP funded large projects greater than $1 million in total project value in each sector, including two individual recipients and one industry association, as depicted below.

Description

The bar graph depicts the distribution of signed contribution agreements by size.

50 contribution agreements funded projects of less than $100,000.

33 contribution agreements funded projects greater than $100,000 and less than $1 million.

3 contribution agreements funded large projects greater than $1 million.

3.4 Cleantech is being adopted by recipients

The FACTAP has met some of its performance targets.^{Footnote 8}

• >100% aquaculture/processing sites have adopted clean technology funded by contribution agreements (57/54 sites).
• >100% fishing vessels have adopted clean technology funded by contribution agreements (310/54 vessels).
• 108 additional businesses or sites should be adopting cleantech funded by the FACTAP through self-funding or other mechanisms by March 31, 2021. However, data on this performance indicator is not available and there is no evidence yet that this is taking place.^{Footnote 9}

The number of signed contribution agreements has increased over the span of the program, as depicted below.

Description

Bar graph depicts the number and proportion of signed contribution agreements, by year.

In 2017, 8 contribution agreements were signed (9% of all agreements).

In 2018, 33 contribution agreements were signed (38% all agreements).

In 2019, 45 contribution agreements were signed (56% all agreements).

Nevertheless, interviewees and survey respondents perceive that cleantech is only being adopted to a limited and moderate extent by fisheries and aquaculture operators. For example, cleantech is costly, particularly for small enterprises with limited profitability and resources. Also, sector-wide knowledge and communication of cleantech and of available funding opportunities is lacking.

3.5 The FACTAP contributes to environmental performance

A majority of survey respondents and interviewees experienced positive environmental impacts since implementing cleantech,^{Footnote 10} as demonstrated in the administrative data review. The greatest environmental impact of cleantech adoption across the sectors has been a reduction in energy consumption and greenhouse gas (GHG) emissions, as listed below.

• 78% of respondents reported reduced energy consumption
• 48% of respondents reported reduced GHG emissions

Survey respondents and interviewees also indicated that FACTAP-funded cleantech generated a variety of additional environmental impacts including improved waste management, reduced use of water, plastics, chemicals, and impacts on marine life and surrounding ecosystems, as listed below.

• 30% of respondents reported improved waste management
• 26% of respondents reported reduced water use
• 19% of respondents reported reduced plastic use
• 19% of respondents reported reduced chemical use

The FACTAP has exceeded its environmental performance target of 90% of stakeholders, or recipients, that report a reduction in environmental impact; 100% of stakeholders, or recipients, report a reduction in environmental impact.^{Footnote 11}

Examples of projects funded by the FACTAP that contribute to the environmental performance are listed below:

Photobioreactors in the aquaculture sector

The FACTAP funded six projects to support the acquisition of energy efficient photobioreactors for algae production in aquaculture hatcheries. Older models are energy inefficient and are associated with high energy costs. In this case, environmental performance includes reduced energy consumption, reduced water use and reduced plastic use.

Description

Top view of installed photobioreactor used for phytoplankton production.

Side view of installed photobioreactor used for phytoplankton production.

Styrofoam replacement in the aquaculture sector

The FACTAP funded five projects for the replacement of exposed styrofoam used in aquaculture rafts with durable alternatives to reduce plastic marine pollution. In this case, environmental performance includes reduced plastic use.

Description

Collection of waste styrofoam rafts collected and going to disposal.

Durable styrofoam-raft replacements installed.

Mapping cameras in the fisheries sector

The FACTAP funded 10 projects to support the acquisition of mapping cameras for commercial trap fisheries to reduce negative impacts on sensitive marine ecosystems and the incidence of ghost gear. In this case, environmental performance includes reduced GHG emissions, reduced energy consumption, and reduced marine ecosystem impacts.

Description

Installation of sonar mapping camera technology aboard a fisheries vessel.

Imaging software used to map the benthic seafloor.

Effluent treatment equipment in the processing sector

The FACTAP funded a number of projects to improve waste management across all sectors. For instance, by supporting the acquisition of effluent treatment equipment to improve waste water discharge from a fish processing plant. In this case, environmental performance includes improved waste management.

Description

Installation of an advanced oxidation process to remove color and solids from wastewater.

Clear effluent discharge as a result of installed wastewater treatment process.

3.6 The role of the FACTAP is unique

Evidence suggests that the FACTAP funds small and medium enterprises (SMEs) in the fisheries and aquaculture sectors that tend to lack alternative funding opportunities for cleantech. Indeed, 95% of contribution agreements have been stablished with SMEs and a majority of survey respondents and key informants said that the FACTAP mitigates financial risks associated with the adoption of new cleantech for recipients.

A majority of recipients stated that, in the absence of the FACTAP funding, they either would not have implemented their cleantech projects or would have done so but across a longer time-frame.

In theory, fisheries and aquaculture operators are also eligible to apply to federal fisheries funds such as the Atlantic Fisheries Fund (AFF), Quebec Fisheries Fund (QFF), and British Columbia Salmon Restoration and Innovation Fund (BCSRIF). Fisheries funds^{Footnote 12} provide funding to support growing market demands and sustainable industry growth. However, in practice the FACTAP collaborates with the AFF, QFF, and BCSRIF to avoid duplication of funding efforts (See Section 3.9 for more details on provincial collaboration).

A majority of interviewees (83%) said they were not aware of any other programs that also provided funding for cleantech.

Committee members said that the FACTAP also generates socio-economic impacts for funding recipients to a moderate extent. The greatest socio-economic impact experienced by survey respondents was related to cost savings associated with lower electricity and fuel use, as listed below.

• 73% of respondents reported cost savings

Survey respondents indicate that additional socio-economic benefits include: increased production, and business competitiveness as well as improved productivity, ability to continue operations, and social license to operate, as listed below.

• 40% of respondents reported improved productivity
• 37% of respondents reported an ability to continue operations
• 37% of respondents reported an increase in production
• 37% of respondents reported increased business competitiveness

3.7 Recipients receive support during the application process

FACTAP applications are accepted based on an continuous intake process by each regional program coordinator.

Description

The figure depicts the following seven FACTAP activities: Application; Evaluation of proposals; Establishing CAs; Disbursement of funds; Reporting; Collection and use of performance information; and Promotion and communication.

The figure depicts an arrow between the Evaluation of proposals and Collection and use of performance information activities to highlight that the FACTAP takes into consideration the performance of previously funded projects during the evaluation process.

Application

A majority of interviewees and survey respondents (91%) consider the submission of proposals (i.e. application requirements) to be efficient.^{Footnote 13}

While application requirements are sometimes viewed as complicated and the application process lengthy, regional program coordinators provide support to applicants before, during, and after proposals have been submitted.

There is a significant time investment required during the application stage. Regional program coordinators' involvement varies from one day in the best-case scenario (e.g. if the applicant works with a consultant) to a full week of work in the worst-case scenario (e.g. when the project is highly technical or when communication is complicated). (See Section 3.13 for additional details on the support provided by RPCs).

3.8 Evaluation of proposals is efficient

Evaluation of Proposals

A majority of interviewees (92%) consider the evaluation and ranking of proposals to be efficient.^{Footnote 14}

Proposals are screened through the RPC, RPRC, and NSC committees prior to approval. Committee members and senior managers agree that committee decision-making, composition, joint provincial and territorial governance, frequency of meetings, and recommendations to the Director-General are somewhat efficient or very efficient.

The FACTAP informs recipients of a funding decision within 90 business days in 97% of the cases, exceeding the departmental service standards.

GBA+

The FACTAP gives consideration to proposals submitted by indigenous recipients during the evaluation of proposals process, as illustrated in the figure below.

Description

The figure depicts a map of Canada showing FACTAP’s three administrative regions and the number of Indigenous recipients that have been funded in each.

4 Indigenous participants have been funded in the Pacific region.

2 Indigenous participants have been funded in the Central & Arctic region.

1 Indigenous participant has been funded in the Atlantic region.

Most interviewees indicated that informal acknowledgement of Indigenous applications takes place.

• 8% of funded projects are of Indigenous Status.

No other consideration or formal scoring is given to other diverse population groups recognized under GBA+, such as women operators and operators in remote communities.*

*Departmental documents indicate a requirement to extend consideration to indigenous and women participants.

Establishing contribution agreements

A vast majority of interviewees and survey respondents (96%) consider that contribution agreements are established efficiently.^{Footnote 15}

3.9 The FACTAP leverages provincial government funding

Most survey respondents (62%) said it was either easy or very easy to receive financial or in-kind support^{Footnote 16} from provinces.

All provinces provided the mandatory 10% financial contribution towards FACTAP projects. This includes total P/T contributions of $2,951,229. Quebec provided the highest proportion of P/T funding, on average, as illustrated below

Description

The figure contains a bar graph showing the proportion of provincial support provided by each province, on average, alongside a map of Canada highlighting all contributing provinces.

Quebec provided a 30% financial contribution towards FACTAP projects, on average.

Ontario provided a 12% financial contribution towards FACTAP projects, on average.

Prince Edward Island provided a 12% financial contribution towards FACTAP projects, on average.

Nova Scotia provided an 11% financial contribution towards FACTAP projects, on average.

British Columbia Quebec provided a 10% financial contribution towards FACTAP projects, on average.

Newfoundland and Labrador provided a 10% financial contribution towards FACTAP projects, on average.

Manitoba provided a 10% financial contribution towards FACTAP projects, on average.

New Brunswick provided a 10% financial contribution towards FACTAP projects, on average.

Most respondents do not perceive the provincial financial leveraging as an issue, yet program staff and committee members indicate this has been a challenge for the program, to different extent across provinces.

Certain provinces have created dedicated funds to support clients’ access to funding through the FACTAP:

• Quebec applicants benefit from the support of three provincial ministries^{Footnote 17}, including a funding program dedicated to developing SMEs in the commercial fisheries and aquaculture sector.
• BC provides dedicated funding through the Ministry of Agriculture.
• P.E.I. created a dedicated funding program called PEI-FACTAP in 2019.

Recipients in provinces lacking a dedicated fund faced challenges obtaining P/T contributions:

• In the Central and Arctic region, in-land provinces (i.e. Manitoba and Ontario) have relied on in-kind contributions to meet this requirement.
• In the Atlantic region, New Brunswick, Nova Scotia, and Newfoundland and Labrador also faced challenges receiving provincial support due, in part, to provincial funding being dedicated towards the Atlantic Fisheries Fund. The Atlantic Fisheries Fund requires that 30% of the funding comes from participating provinces. It has attracted most Atlantic provincial funding, and it has been a barrier for FACTAP applicants in obtaining funding.

Although most surveyed recipients do not think the FACTAP should make any change to the minimum contribution requirement, four out of six survey respondents in the Atlantic region would like to see some changes to the provincial funding requirement. Changes could take the form of a transition to a letter of support rather than a financial contribution from the province.

3.10 Funds are distributed to recipients efficiently

Distribution of funds

A vast majority interviewees and survey respondents (96%) consider the distribution of payments to be efficient.^{Footnote 18}

In 2017-18 and 2018-19, 100% of FACTAP payments were made in accordance to the payment schedules. During this period service standard targets for requisition for payment within 30 days were also exceeded.

Recipients receive 90% of project funding once contribution agreements are signed. The remaining 10% percent is held back until final reports have been approved, which happens in the next fiscal year.

Reporting

A majority of interviewees and survey respondents (83%) consider the final reporting to be efficient.^{Footnote 19}

A review of administrative data shows that 69% of completed reports in 2018-19 were delayed by anywhere from one month in the Atlantic region, to three months in the Central and Arctic region.

Interim and final reports from funded recipients can be delayed due to missing deliverables, signatures, required documentation, or requests for extensions. As well, fisheries operators prioritize their activities over reporting requirements when they coincide with the fishing season.

3.11 Use of FACTAP performance information is limited

Collection and use of performance information

The main sources through which the FACTAP collects performance and financial information are project interim and final reports.

At the Program level:

Notwithstanding delays in the collection of reports, performance information of successful cleantech that has been previously funded is taken into consideration when proposals are evaluated.

At the Departmental level:

Within DFO, anecdotal evidence suggests that performance information from FACTAP-funded semi-closed containment systems was used to support policy development related to the responsible plan to transition from open net-pen salmon farming in British Columbia.

At the Federal level:

Areas of improvement remain with regards to how overall performance information could be used to support reporting and decision-making outside DFO. There is no evidence that other departments are using overall FACTAP performance information to support decision-making.

While the FACTAP reports to initiatives under the Pan-Canadian Framework on Clean Growth and Climate Change, the primary recipient of FACTAP information is the Clean Growth Hub (CGH). As part of the Clean Technology Data Strategy (CTDS), the CGH gathers information that is already collected by the FACTAP since the program’s data collection strategy predates the CTDS. Part of the information on the environmental benefits collected by the FACTAP is qualitative. The CGH is developing guidance focused on estimating GHG reductions to help support the collection and reporting of quantitative data.

3.12 Promotion, communication and awareness of the FACTAP is limited

Promotion and communication to the public

More than half of interviewees and survey respondents (59%) said the promotion of the FACTAP is somewhat efficient.

The FACTAP carries out various promotional activities, including conferences, industry association meetings, tradeshows, and DFO’s notice to fisheries.

Fisheries and aquaculture operators are, in general, somewhat aware of the FACTAP. Survey respondents strongly indicated that the program can raise awareness of the FACTAP among businesses in the fisheries and aquaculture sector through more direct outreach. Other ways to raise awareness include more involvement in association meetings and more public announcements.

Survey respondents reported having learned about the FACTAP through industry associations, conference events and DFO’s Notice to Fishers as listed below. An additional venue was through word-of-mouth as news is shared by FACTAP recipients among their networks. Federal program websites, like the FACTAP website and the Clean Growth Hub website are the least used sources through which operators learn about the program.

• 50% of survey respondents learnt about the FACTAP through Industry associations;
• 27% of survey respondents learnt about the FACTAP through conference events;
• 23% of survey respondents learnt about the FACTAP through word-of-mouth;
• 20% of survey respondents learnt about the FACTAP through DFO’s notice to fishers;
• 10% of survey respondents learnt about the FACTAP through the FACTAP website; and
• 0% of survey respondents learnt about the FACTAP through the CGH website.

Program staff and survey respondents consider public announcements and description of projects on the FACTAP’s website somewhat to not very efficient. Due to delays in internal processes, a number of projects have not been announced; therefore, the FACTAP website is underutilized as a communication tool. Only 56 projects have been posted on the FACTAP website, of which 41 are from the Pacific region (73%), as illustrated below. In addition, of all 19 FACTAP projects in the Atlantic region, 26% are posted online.

Description

The bar graph depicts the percentage of projects posted on the FACTAP website, per region.

91% of projects funded in the Pacific region have been posted on the FACTAP website.

45% of projects funded in the Central and Arctic region have been posted on the FACTAP website.

26% of projects funded in the Atlantic region have been posted on the FACTAP website.

3.13 The FACTAP applies good practices in program delivery

The FACTAP staff developed multiple good practices in delivering the program.

Flexibility in program delivery

Thanks to the training received from the G&C Centre of Expertise, program staff demonstrated more flexibility in program delivery. For instance, the eligibility criteria for cleantech are flexible to some extent, as are the deadlines, which are adjusted to fit the reality of the recipients.

In addition, the FACTAP created a second regional committee in the Central and Arctic region to address language diversity. Specifically, the FACTAP oversees one Anglophone and one Francophone committee in the C&A region.

Collaboration with provincial representatives

There is evidence of good collaboration with the provincial representatives. The FACTAP is working hand to hand with provincial representatives who often sit on the RPRC. They bring their experience and network and facilitate the promotion of the program in the regions.

Provincial representatives support committee decision-making. There is evidence of provincial input being key for assessing management risk based on previous experiences working with the applicant.

Regional coordinators provide support to clients

FACTAP staff are available to provide guidance and answers to clients throughout the application process. There is a significant time investment required during the application stage, especially at the regional level. The time invested was hard to quantify as it can differ from project-to-project. Nevertheless, most FACTAP recipients are positive about the support they have received during different phases of the funding cycle.

• Regional coordinators reached out to recipients when guidance or clarification needed during the application stage
• 90% of survey respondents felt that staff provided direction and clarification on the type of information that is expected
• They received clarification regarding contribution agreements, workplans, budget adjustments, and amendments

4.0 Conclusions and recommendations

Conclusions

The FACTAP is a relevant and well-performing program and is supporting the adoption of clean technology in eight provinces to improve the environmental performance of the operators it funds. The FACTAP has facilitated access to funding, but there are some limitations concerning the environmental impact the FACTAP has at the sector level. Moreover, the FACTAP is unable to quantify improvements in environmental performance at the program level. Specifically, the FACTAP’s performance indicators do not measure quantified outcome-specific metrics, but rather rates of adoption by operators and whether operators experienced environmental impacts.

Evaluation findings showed that the FACTAP is generally administered efficiently. However, interviewees commented this results from the efforts of RPC’s leading applicants through the various FACTAP processes, which results in internal challenges associated with the delivery of the FACTAP. For instance, the administrative workload caused by the continuous intake of projects takes priority over promotion activities. Overall, the FACTAP promotion has been difficult for a number of reasons, including limited O&M funding, the turn-over of regional program coordinators and the size of program administrative regions, particularly the Central and Arctic region. As a result, there is a lack of awareness and understanding of the types of cleantech that are eligible.

According to the evaluation evidence, the FACTAP’s contribution could be strengthened by three means:

• Taking a proactive and strategic approach towards project selection, so as to fund projects that will maximize the FACTAP’s impact on cleantech adoption at the sector level;
• Expanding FACTAP activities by making cleantech at other stages of the Innovation Spectrum eligible; and,
• Increasing FACTAP marketing and promotion activities to target a broader audience of fisheries and aquaculture operators.

Recommendations

Recommendation #1: Strategic funding

It is recommended that the Senior Assistant Deputy Minister, Strategic Policy work with fisheries, aquaculture and processing operators and associations to adopt a more proactive and targeted approach to address key environmental issues associated with these industries thus increasing the overall impact and reach of the Program.

Rationale: Establishing a proactive and targeted funding approach should address outcome-specific challenges, for example, by demonstrating reduced greenhouse gas emissions or reduced marine plastic pollution. This could be done by addressing industry pain-points or reaching a larger number of operators, for example through industry associations to realize even greater improvements in environmental performance.

Recommendation #2: Eligible activities

It is recommended that the Senior Assistant Deputy Minister, Strategic Policy, consider expanding the range of eligible FACTAP activities to include additional stages of the cleantech Innovation Spectrum beyond funding of market-ready technologies.

Rationale: Expanding the range of eligible FACTAP activities was raised throughout this evaluation as important to recipients and program managers. For example, the program would benefit from supporting cleantech proposals at the Research and Development, and Pilot and Demonstration stages of the Innovation Spectrum as such projects are currently ineligible.

Recommendation #3: FACTAP promotion

It is recommended that the Senior Assistant Deputy Minister, Strategic Policy, engage in additional promotional activities for the FACTAP including direct outreach activities to potential recipients of the program and collaboration with industry associations.

Rationale: There is a low awareness of the FACTAP among operators at the sector-level despite the FACTAP’s promotional efforts. The FACTAP can raise awareness among businesses in the fisheries, aquaculture and processing sectors through more direct outreach and greater involvement in association meetings and industry-led events.

5.0 Annexes

Annex A - Methodology, Limitations and Mitigation Strategies

Although the evaluation encountered some methodological challenges, methodological limitations were mitigated, where possible, through the use of multiple lines of evidence and the triangulation of data. This approach was taken in order to establish the reliability and validity of the findings and to ensure that conclusions and recommendations were based on objective and documented evidence.

Administrative Data

A review of the FACTAP administrative documents was undertaken to understand the FACTAP funding process. This review focused on Contribution Agreement process, on how the program supports funded recipients to adopt of the cleantech, on monitoring and reporting processes. A sample of 22 projects (7 in Atlantic, 7 in Pacific and 8 in Central and Artic) was used to analyze the above aspects.

Given that the program started in 2017-2018, not all interim or final reports were available for review at the time of the evaluation. The reports also used various metrics to report on project results. This meant that information on project results was not available for all projects.

To mitigate that limitation, the evaluation used a combination of available year-end reports, and information gathered through discussions with project recipients to develop findings related to the impact of FACTAP projects. The information was also used with other data collection methods (e.g. interviews and survey) and triangulate findings from other lines of evidence.

Interviews

A total of 33 FACTAP Program Management, Regional Proposal Review Committee (RPRC), National Steering Committee (NSC) Members, Senior Management, Recipients and a Clean Growth Hub representative were interviewed. By conducting those interviews, the goal was to gather the view of the interviewees on the need of the program, on how the funded projects impacted the environment by adopting the clean technology in fisheries and aquaculture sectors, and if the program activities are provided efficiently.

Observations

In March 2020, the Evaluation Team participated in a National Steering Committee (NSC) Meeting as observers to understand the NSC evaluation process. In addition, during the data collection phase, the Evaluation Team planned to do some sites visits, but due to COVID-19, the sites visits were not possible. To mitigate this limitation, the Evaluation Team asked more detailed questions while doing the interviews for the documented examples. The information was used to triangulate findings from other lines of evidence.

Survey

A survey with FACTAP's recipients provided an understanding of their perceptions of cleantech adoption and environmental impacts. They also expressed their perspective on the support supplied by the FACTAP and on the program's and performance.

The survey was made available online to 72 FACTAP recipients, and 30 completed survey responses were received, representing a 42% response rate. The survey was administered online between June 15 and August 8, 2020, and the responses were used to triangulate findings from other lines of evidence.

Literature Review

A two-part literature review on cleantech and innovation was completed. The first part focused on the connections between governmental investment, innovation and economic growth. The second part highlighted the role of long-term government support and networks in adopting cleantech in the Fisheries and Aquaculture sectors. The information generated by the literature review was used to triangulate findings from other lines of evidence.

Documented Examples

A brief, focused analysis was undertaken on four examples, which contributed to emphasize our findings of some environmental and socio-economic impact, success factors such as awareness, degree of innovation, DFO support, provincial contributions, as well as FACTAP challenges and lessons learned. Nine interviews were conducted with recipients and program managers.

The examples analyzed were:

1. Reduction of plastic marine pollution
2. Implementation of clean-technology in an Indigenous community
3. Adoption of innovative clean-technologies
4. Environmental and economical impacts of a waste treatment system

Document Review

The evaluation team reviewed DFO and external documents to understand the context and background of the FACTAP, also to assess the general need of the program, and best practices.

Annex B - Management action plan (MAP)

Evaluation of the Fisheries and Aquaculture Clean Technology Adoption Program (project #996395)
PMEC Date: November 2020
MAP Completion Target Date: March 2022
Lead ADM/DC: Jean-Guy Forgeron, Senior Assistant Deputy Minister, Strategic Policy