national research council’s institute for research in ......national research council’s...

National Research Council’s Institute for Research in Construction (NRC-IRC)

Canadian Construction Materials Centre (CCMC) Business Review

F I N A L R E P O R T A N D

R E C O M M E N D A T I O N S

April 28, 2010

National Research Council’s Institute for Research in Construction (NRC-IRC) Final report and recommendations

CCMC Business Review Date: April 28, 2010

Brio Conseils inc.

Page: 2 / 21

TABLE OF CONTENTS

1.0 INTRODUCTION .......................................................................................................................................... 3

2.0 METHODOLOGY ......................................................................................................................................... 4

3.0 KEY ISSUES ................................................................................................................................................. 5

4.0 RECOMMENDATIONS ................................................................................................................................. 7

5.0 CONCLUSION ............................................................................................................................................ 10

APPENDIX A: STATEMENT OF WORK, NRC-IRC

APPENDIX B: INTERIM REPORT, BRIO CONSEILS

APPENDIX C: INTERVIEWS WITH PTPACC MEMBERS, AD HOC RESEARCH

APPENDIX D: WEB SURVEY WITH BUILDING OFFICIALS AND MANUFACTURERS, AD HOC RESEARCH

APPENDIX E: FOCUS GROUP RESULTS, AD HOC RESEARCH

APPENDIX F: SUMMARY OF SEPTEMBER 2009 NRC WORKSHOP, AD HOC RESEARCH

APPENDIX G: GOVERNANCE REVIEW OF THE CCMC, INSTITUTE ON GOVERNANCE

APPENDIX H: SUMMARY REPORT, BRIO CONSEILS

APPENDIX I: OPEN CONSULTATION – LIST OF CONTACTED GROUPS, BRIO CONSEILS

APPENDIX J: CHBA DISCUSSION PAPER, CHBA



Brio Conseils inc.

Page: 3 / 21

1.0 INTRODUCTION For the past 20 years, the Canadian Construction Materials Centre (CCMC), part of the National Research Council’s Institute for Research in Construction (NRC-IRC), has offered the construction industry a national evaluation service for innovative, non-standardized construction products on a cost-recovery basis. The CCMC provides an opinion when a product/system complies with the minimum performance level established by the National Building Code (NBC). It also provides an opinion on compliance of a product to recognized standards. In spring 2009, the CCMC embarked on a Business Review project aimed at examining its business model for evaluating construction products, its mission and if it was meeting the needs of its clients. Many changes have occurred in the construction market and the regulatory framework over the past few years. The CCMC has continually responded to such changes by enhancing its organization and services to meet demands. The CCMC wants to promote a proactive approach that is both serious and objective so it can make changes as required to meet future expectations. To that end, the NRC-IRC has engaged Brio Conseils, a firm specializing in transformation management, entrusting it with heading and managing, in a transparent manner, a consultation with the various players involved in the construction industry. The main goal of this consultation is to identify stakeholders’ needs and determine if the CCMC is meeting those needs with its current business model. This report provides an understanding of how the key stakeholders perceive the CCMC’s role, mandate and services and its relationship to the Canadian construction industry. It identifies key issues that have been identified by the key stakeholders and recommends areas of improvements.



Brio Conseils inc.

Page: 4 / 21

2.0 METHODOLOGY

Based on the Statement of Work (Appendix A), the CCMC’s business review involves three phases, as illustrated below:

Project Timeline and Deliverables

The NRC has appointed a Business Review project team, comprising representatives from provincial ministries, regulatory authorities and manufacturers—as well as the NRC and the Chair of the Canadian Commission on Construction Materials Evaluation (CCCME). Their role is to review the results, prioritize the findings into groups of ideas and make recommendations as to what can be moved forward on.

The outcome of the phase 1 was to identify CCMC key strengths, weaknesses, opportunities and threats (SWOT) based on the internal assessment and an environmental scan. An interim report has been written including the findings for this first phase (Appendix B).

The second phase was done in two parts. The purpose of the first part was to better understand the primary

1 stakeholders’ needs and their satisfaction level with the CCMC’s services. To do

that, surveys were conducted with primary stakeholders: ministry officials responsible for construction policy (Appendix C), building officials and manufacturers (Appendix D). The purpose of the second part was to consult stakeholders through focus groups on specific issues in order to validate and have an open discussion about the findings from the first part (Appendix E). All surveys were conducted in partnership with Ad hoc Research who has more than 25 years experience conducting consultation studies.

1 A direct stakeholder is a group of individuals who are immediately affected by the service offered by the CCMC and who stand to

personally benefit or loss from any changes made to the CCMC business model.



Brio Conseils inc.

Page: 5 / 21

Concurrently with all the surveys conducted with the stakeholders, a workshop was held in Calgary last September with members from the World Federation of Technical Assessment Organizations (WFTAO) and IRCC (Inter-Jurisdictional Regulatory Collaboration Committee) as well as with participants representing stakeholders from the Canadian construction industry. The objective of the workshop was to share experiences and learn from international visitors about their business models for evaluating innovative construction products. The focus was on what works in their models and the problems encountered. The theme of the day was: Process Improvements to Obtain Faster Market Acceptance (Appendix F). Also, a Governance Review was conducted by the Institute On Governance. The purpose of the governance review was to examine certain aspects of the governance of the NRC and the CCCME to ensure that they are in line with contemporary principles and exemplary practices (Appendix G).

So everybody has an opportunity to give their opinion, a summary report (Appendix H) was posted on CCMC’s Web site in January and February 2010 to get feedback from the construction industry. Different groups (Appendix I) were contacted and invited to provide input to the business review. The Canadian Home Builders’ Association (CHBA) had published a discussion paper on CCMC in March 2005 and submitted a supplement to the paper in October 2009 (Appendix J). In late February, the Business Review project team, comprising representatives from provincial ministries, regulatory authorities and manufacturers—as well as the NRC and the Chair of the Canadian Commission on Construction Materials Evaluation (CCCME), reviewed the results, prioritized the findings into groups of ideas and made the recommendations presented in this report.

This report will be submitted to the CCCME for review before going to NRC-IRC management for the final decision.

3.0 KEY ISSUES

There are many areas for improvement that were mentioned in all surveys that were conducted with stakeholders. The following are the key issues that should be addressed to meet their needs.

.1 Misperception regarding the role of the CCMC and the service it provides

The CCMC’s stakeholders can be split into two groups: manufacturers or proponents on one side and decision makers

2 or regulators/specifiers on the other. Both groups recognize the value of

the services provided by the CCMC. The manufacturers view the CCMC as a service provider, whereas the decision makers tend to view the CCMC as a partner. Decision makers may not always have all the information they need – or the time and resources to analyze it - to decide whether or not to approve an innovative product, and the evaluation report from the CCMC has a lot of value for them. For the manufacturers, the CCMC reports and listings make it easier to access the Canadian market through a nationally recognized agency. This recognition ensures that they can use the report as a marketing tool to help get their products to market.

2 In most Canadian provinces, the municipal building official is ultimately responsible for approving what the engineer,

architect or builder (specifier) has specified for use in the building. In Quebec, the specifier on a project tends to be the party who is ultimately legally responsible for the construction materials chosen. Despite these differences, the end result is similar across Canada: those who specify the materials for innovative products tend to prefer specifying a product with a CCMC number because they feel that it limits their vulnerability to liability. For the purpose of this report, the term ―decision makers‖ will be used when referring to building officials and specifiers (engineers, architects, builders, etc.).



Brio Conseils inc.

Page: 6 / 21

From there, it seems that a proportion of manufacturers and decision makers believe that the CCMC approves products, when in fact it provides an opinion on code or standard compliance to assist decision makers in approving construction products. It also seems that they do not understand the distinction between a listing and a report. For example, some of them are unaware that the listing does not necessarily evaluate standards referenced in the National Building Code and that it is not an opinion on code compliance. Also, it seems that the re-evaluation process is not well understood. Many building officials experience difficulties determining the validity of a listing or report that has an expired date or that has an ―under re-evaluation‖ status.

All that confusion about the CCMC’s role and the service it provides can lead to the perception that the service it provides does not completely satisfy the stakeholders’ expectations or that the service it provides is not properly communicated to the stakeholders. In either case, if the opinions produced by the CCMC are not properly used, this could lead to improper use of products and the decision makers may put themselves at risk in terms of liability.

.2 The evaluation process is too long/Manufacturers do not understand the evaluation

process.

There is a general consensus regarding the long time it takes the CCMC to go through the evaluation process. Sometimes, the process is so long that by the time the report comes out, the industry has changed. This situation goes against the CCMC’s role, which should be to facilitate the acceptance of new and innovative construction products and systems.

Other organizations like the CCMC around the world are facing the same issue. The topic was discussed in detail at the NRC workshop held in Calgary in September 2009 (Appendix F). Many ideas for improvements were identified and have been considered in this report’s recommendations to improve the efficiency of the CCMC’s evaluation process.

In general, the degree of trust in the output provided by the CCMC is quite high. However, it seems that there are a few misperceptions about the CCMC evaluation process. Some people believe that the evaluation process depends on a single person and is not reviewed by a committee, when in fact all reports/listings are reviewed by at least one peer and a manager. The CCMC evaluation and decision making process needs to be better communicated.

.3 Manufacturers are not satisfied with the CCMC’s response time and follow-up.

Several manufacturers feel that response time and better follow-up on inquiries are among the top priorities for improvement. It seems that the blame should not always be placed on the CCMC because manufacturers also have responsibilities in the process and they are not always fulfilled properly for different reasons. Regardless of who is responsible, the CCMC could surely benefit from being more transparent when it comes to the evaluation process, including the time it may take, the costs, the expectations and the outcomes. Also, several manufacturers claim to be uncomfortable about complaining to the CCMC when it comes to its lack of responsiveness and follow-up. Because of this climate, the CCMC is most likely unaware of existing problems which can create frustration and give the CCMC a bad reputation.

.4 Building officials’ use of CCMC reports.

There is a perception in the construction industry that building officials rely heavily on the CCMC’s evaluation services for product approval. The survey findings clearly show that not all building officials have the same amount of knowledge and/or resources to interpret the information



Brio Conseils inc.

Page: 7 / 21

submitted by the proponent of an innovative product to determine if it can be approved or not. The CCMC reports give them a comfort level when approving the use of a product. They feel they are exposing themselves to less risk by relying on the expertise of the CCMC.

Although it may be beyond CCMC’s role and responsibility to monitor building officials’ use of their evaluations, CCMC should at least make sure the information it produces is easily accessible and is well understood by users in terms of content and format. The website is the main tool used by building officials to check if a CCMC number is still valid or not. While some find the website relatively easy to use, others complain that the new website is far less intuitive, which contributes to the confusion. Furthermore, some building officials seem to have a false sense of security when a product has a CCMC number. Some assume that a CCMC number is a guarantee that the product satisfies the intent of the Building Code, without being aware of the conditions and limitations of CCMC’s opinions.

4.0 RECOMMENDATIONS

The project team members met on February 24-25, 2010 to review all information generated by this business review exercise and make recommendations to address identified issues.

.1 Review the evaluation services:

There is confusion around CCMC evaluation services.

Listings: The value of evaluating products to standards has been confirmed. However a key finding is that the listing is misunderstood and misinterpreted. Measures shall be taken to eliminate this confusion:

- Keeping listings as currently defined should be re-examined.

- Because of the confusion about what a listing really means there are strong arguments in favour of offering exclusively evaluation reports, i.e. evaluation to the NBC. If this approach were adopted:

(a) Products under the scope of a standard referenced in the National Building Code (NBC) should be evaluated for compliance to the NBC, not only compliance to the standard. Therefore, if the NBC has any additional requirements or restrictions that apply to this product, the CCMC report would address them.

(b) Products that are covered under the scope of a standard not referenced in the NBC should be handled as an alternative solution and should be evaluated for compliance to the NBC.

Re-evaluations: re-evaluation is an internal process to ensure that the product evaluated has not changed in quality. To eliminate confusion, the re-evaluation date and ―under re-evaluation‖ status should be removed from documentation, and only the date from the last update should appear. If a report is on the website, it should mean that it is valid for the information published.

Capacity: internal resources and processes should be optimized to increase efficiency and help meeting increasing and changing demands for CCMC services. Consideration should be given to accrediting other organizations in providing evaluation services.



Brio Conseils inc.

Page: 8 / 21

.2 Improve the evaluation and decision – making process:

All organizations like CCMC are under pressure to deliver an increasingly broad and complex range of services in a manner that is time- and cost-efficient, transparent and un-biased. Increasing and improving communication with stakeholders will be very important, but process and organizational change are also needed.

Expectations regarding the evaluation service should be clearly defined in terms of timeframe, cost and quality. They should be measurable and realistic.

CCMC processes should be reviewed to identify areas for improvements (speed, service, cost, and maintain quality)

- Review activities and tools to support the evaluation process in regards to their practicality and efficiency;

- Review roles, responsibilities, staffing and core/bottleneck resources.

Here are some ideas to improve the evaluation process:

- Develop a means to identify the level of risk associated with an innovative product: some innovative products might be classified as having a marginal risk (and fast tracked) while products classified as high risk would have to follow a more thorough evaluation process and the process for products with a lower risk could be streamlined.

- Making greater use of external or alternate resources and knowledge should be considered, for example:

o Collaboration with other organizations should be considered, such as collaborative agreements with SCC accredited bodies, MOU agreements with other evaluation services and labs with required expertise, use external consultants when adequate staff resources are unavailable, etc.

o Consider using or accepting evaluation criteria developed by other recognized technical assessment organizations for same or similar products (e.g. ICC-ES).

o Consider recognition (as supporting evidence for example) of evaluations/approvals from other countries in Canada.

o Explore the value of field experience in the evaluation process. To reduce frustration, clearly communicate the limitations (regional climate differences, scientific rigor, timeframe) of field experience.

- Post the report more quickly after test results are received.

- Consider better integrating and reducing overlap between the SEP and technical guide development processes.

- Participate earlier in proponent’s product development (pre-commercialization) to increase chances of an end product that will meet CCMC performance criteria.

The re-evaluation timing and process should be examined:

- Should it be on a particular schedule (3 or 5 years) or prompted by a change to the product or to National Building Code requirements affecting the product? However, any changes to a product or the Code should be documented in the listing/report.

- Better manage the re-evaluation process with the manufacturer. Investigate whether the process could start a year from the expiry date and set the criteria for re-evaluation when the notice and request for payment is set.



Brio Conseils inc.

Page: 9 / 21

Here are some ideas to improve the decision-making process:

- Review the role of the CCCME and SCTE in the evaluation process - consider increasing their role particularly with respect to approval of technical guides and reports - to improve transparency, impartiality and consistency of evaluations and opinions.

- Consider more opportunities for input from regulatory authorities at the SEP, technical guide and report phases, particularly with respect to the scope, conditions and limitations of the evaluations.

- There should be a mechanism to resolve disagreements between the proponent/AHJ and the evaluation officer (e.g. disagreement on limitations).

- Emphasis on internal peer review and staff cross-training for consistency of evaluations and opinions.

- Develop and publish a risk assessment and management protocol.

.3 Ensure clear communication with stakeholders.

One quality of successful organizations is that they understand and are responsive to their stakeholders’ needs. In order to serve them as effectively as possible, different approaches should be implemented to provide more personalized services. As mentioned in the previous section, manufacturers view the CCMC as a service provider, and decision makers view the CCMC as a partner. It is therefore essential to develop two different approaches: one to better serve manufacturers and another to better support decision makers.

Implement a manufacturer/client care program:

- Develop a customer care policy. The policy should establish roles, responsibilities and procedures for first contact, informing the proponent of the whole process and how decisions are made (to avoid any misunderstanding and the perception of individual bias) ensuring regular updates, dealing with any complaints or questions the proponent may have, etc.

- Before the proponent gets into the process, clearly communicate the different steps, deliverables, chances for success, timeframe and costs to ensure a clear understanding of all the requirements and outcomes.

- Manage the process from beginning to end, play a one-stop shop role: use the project management process to plan time and costs, manage risks and manage expectations and changes. Follow-up on execution.

- Clearly communicate the proponent’s own responsibilities (client accountability protocol) with respect to sharing of required information, respecting testing and re-evaluation schedules, contract terms and expiration, report cancellation, delays, etc.

Develop and implement a decision maker support program:

- To improve the ability of decision makers to properly use the information provided by the CCMC, develop a decision maker support program including communications and training.

- Improve CCMC documentation to ensure that the opinion, description, conditions and limitation sections of reports are clear.

- Improve the website to better direct users to the Web Registry, reduce/remove information which is not related to CCMC evaluations and add a ―FAQ‖ page.



Brio Conseils inc.

Page: 10 / 21

Develop a customer relationship approach with all stakeholders:

Satisfied customers are easier to serve and deal with. An important aspect of service delivery is to understand the needs of the stakeholders and address their concerns. After that, there should be mechanisms in place for handling and expediting solutions to problems as they are identified. - Implement an inquiry management policy. This policy should establish roles,

responsibilities and procedures for inquiries from: Manufacturers/clients, Decision makers and General public.

- Direct all calls from the general public (not from manufacturers or decision makers) to the appropriate source of information (e.g. CMHC).

- Have an impartial and transparent mechanism where complaints can be logged and acted upon to address immediate concerns.

- Train technical personnel in customer service and the importance of rapid, proactive communication.

.4 Opportunities:

Energy efficiency, water conservation, green and sustainability/conservation are all opportunities for the CCMC to expand its services. CCMC should play a role in these areas. Since these topics are under development or being considered in the National Building Code, the CCMC needs to plan for it.

5.0 CONCLUSION

The implementation of the above recommendations will surely contribute to improving the stakeholders’ satisfaction with the service provided by the CCMC. It should contribute to provide:

Services that meet the needs and that are clearly communicated and understood;

Time- and cost-efficient processes based on risk;

Customer approach with clearly communicated expectations and responsibilities;

Transparent and impartial decision-making.

Customer focus is often challenged by public sector cultures, hierarchical organizational structures and differing priorities. Overcoming these challenges means aligning CCMC priorities with stakeholder requirements to fulfill their expectations. Finally, it will be important to inform stakeholders and the public on CCMC Website to follow-up on the outcomes of the business review.



Brio Conseils inc.

Page: 11 / 21

APPENDIX A: Statement of Work, NRC-IRC



Brio Conseils inc.

Page: 12 / 21

APPENDIX B: Interim report, Brio Conseils

National Research Council’s Institute for Research in Construction (NRC-IRC)

Canadian Construction Materials Centre (CCMC) Business Review

I N T E R I M R E P O R T

December 22, 2009

National Research Council’s Institute for Research in Construction (NRC-IRC) Interim report CCMC Business Review Date: December 22, 2009

Brio Conseils inc. Printed on: May 6, 2010

Page: 2 / 33

TABLE OF CONTENTS

1.0 EXECUTIVE SUMMARY ............................................................................................................................ 3

2.0 INTRODUCTION ...................................................................................................................................... 6

3.0 METHODOLOGY ...................................................................................................................................... 7

3.1. PHASE 0: PROJECT SCOPE AND METHODOLOGY ........................................................................................................ 7 3.2. PHASE 1: SWOT ANALYSIS .................................................................................................................................. 8 3.2.1. Internal Environmental Assessment ...................................................................................................... 8 3.2.2. Environmental Scan ............................................................................................................................... 9

3.3. PHASE 2: STAKEHOLDERS’ CONSULTATION .............................................................................................................. 9 3.3.1. Surveys with Primary Stakeholders ........................................................................................................ 9 3.3.2. Focus groups with primary stakeholders: .............................................................................................. 9 3.3.3. Broader Stakeholder Consultation ....................................................................................................... 10

3.4. PHASE 3: FINAL REPORT .................................................................................................................................... 10

4.0 BUSINESS REVIEW .................................................................................................................................. 11

4.1 CURRENT BUSINESS MODEL ................................................................................................................................. 12 4.1.1. CCMC Mandate/role ............................................................................................................................ 12 4.1.2. Stakeholders ........................................................................................................................................ 14 4.1.3. Services ................................................................................................................................................ 14 4.1.4. Processes .............................................................................................................................................. 15 4.1.5. CCMC strengths and weaknesses ......................................................................................................... 18

4.2. FUTURE TRENDS ............................................................................................................................................... 19 4.2.1. Key findings .......................................................................................................................................... 19 4.2.2. CCMC threats and opportunities .......................................................................................................... 20

4.3. SWOT ........................................................................................................................................................... 20 4.4. STAKEHOLDER NEEDS ........................................................................................................................................ 22 4.4.1. Ministry officials ................................................................................................................................... 22 4.4.2. Building officials ................................................................................................................................... 25 4.4.3. Manufacturers ..................................................................................................................................... 27 4.4.4. Opportunities ....................................................................................................................................... 29 4.4.5. Summary of the findings ...................................................................................................................... 30



Page: 3 / 33

1.0 Executive summary

In spring 2009, the CCMC embarked on a Business Review project aimed at examining its business model for evaluating construction products, its mission and if it was meeting the needs of its clients. This report summary provides an understanding of how the key stakeholders perceive CCMC’s role, mandate and services and its relationship to the Canadian construction industry.

The focus of the surveys with Stakeholders was to identify primary stakeholders’ needs and their satisfaction with the CCMC’s current services.

In the late 1980s, provincial/territorial representatives agreed to support a national evaluation body. Today, the consensus is that there are advantages to having a national product evaluation to prove code conformity.

CCMC VALUE FOR THE STAKEHOLDERS

Ministry officials believe that CCMC provides an invaluable service. Many provinces do not have the resources to test products and/or determine if they are compliant.

Building officials are far more comfortable approving standardized products (average of 9.3 out of 10) than non-standardized products (rated at only 3.2 out of 10). Many building officials admit to lacking expertise or having no time to review the necessary documentation when it comes to innovative products. Both CCMC listing and report are rated the highest of 5 sources of information when it comes to product evaluation with or without standard. The main reason is the quality of the information provided.

Although manufacturers seem to find the CCMC listings useful (84% very or somewhat), they seem to agree that the reports are more useful (96% very or somewhat). They believe that if CCMC would stop offering the listing service, it will be more complicated to obtain approval for their products. In fact, manufacturers have the perception that the construction industry and professionals currently seem to rely heavily on the CCMC listings for product approval.

NEEDS FOR IMPROVEMENTS

Difference between listing and report:

Almost half the building officials (44%) as well as manufacturers (44%) admit to not knowing the distinction between both services.

The scope of the listing seems misunderstood:

- Many building officials assume that a CCMC listing implies that the product is NBC compliant (38% completely agree and 43% somewhat agree with the statement).

- More than half (54%) of building officials do not know that the listings do not necessarily reference standards in the NBC.

- Two-thirds of building officials (67%) do not know the CCMC does not evaluate any additional requirements imposed by the national or provincial building code.

- More than half the building officials are unaware of the existence of a preface (51%). Among those who are aware of it, only 47% claim to always read it.

The building officials may not necessarily using the CCMC report as they should:

- Most building officials very likely read the report to ensure that the product/system meets the conditions and limitations stated (73%) and that it has a valid CCMC number (73%).



Page: 4 / 33

- Only half (49%) are very likely to read the full report thoroughly. - Only 23% are actually aware of the format change introduced 2 years ago. Those who are aware

of it seem to feel that the 2005 report format is an improvement over the 1995 format (94% agree completely or somewhat).

- Less than half the building officials are aware of the review of the installation manuals (42%) or of the review of the quality control procedures (40%).

- A total of 80% of respondents find the optional information at the end of the report useful, although only 27% find it very useful.

Lack of communication:

Several ministry officials mentioned regularly receiving e-mails from other government agencies but rarely receiving any news from the CCMC. They would prefer a more regular dialogue between the CCMC and themselves as well as with the building officials.

Lengthy process for the report service for innovative products:

Ministry officials are of two minds when it comes to the time aspect since they feel it is important to evaluate the products rigorously—which in turn can be a time-consuming process.

There seems to be a general dissatisfaction among manufacturers with the time it takes the CCMC to complete a report (only 35% are very or somewhat satisfied). The two elements that seem to drive this dissatisfaction are:

- the time it takes to develop the technical guide; - the time it takes to receive the final report after successful testing.

Re-evaluations cycle:

Most ministry officials prefer a three-year re-evaluation cycle but question the CCMC’s capacity to achieve it.

Building officials are satisfied (93% very or somewhat) with the current 3 year cycle, only 57% would completely or somewhat agree with a 5 year cycle. The main concern revolving around the 5 year cycle is that the information in the reports would be outdated considering the rate at which technology changes.

Although many manufacturers are satisfied with the current 3 year re-evaluation cycle (78% are very or somewhat satisfied), there seems to be an openness to changing the cycle to a 5 year one (87% completely or somewhat agree with the idea).

Services:

The vast majority of ministry officials feel that CCMC invests in creating a client relationship by calling back/writing back very quickly and engaging in an open and direct dialogue when discussing issues/problems.

Of building officials who did contact the CCMC, 97% were very (46%) or somewhat (52%) satisfied with the support/service they received. The most common complaints regarding the service are: slow response time and quality of information provided.

The satisfaction level with the support/service respondents received when they contacted the CCMC is relatively good (27% very and 49% somewhat satisfied); however, there seems to be room for improvement. Almost a quarter (24%) of manufacturers claim to be not very or not at all satisfied. The main reason mentioned is because it is ‘time consuming/has a slow response time’).



Page: 5 / 33

Opportunities:

During the survey, opportunities were evaluated with both building officials and manufacturers:

The manufacturers seem to think that making the progress of their file open to themselves (91%) or to building officials (78%) could be useful; however, their level of comfort is less when it comes to all the other potential entities to whom the information could be given (builders, architects, other manufacturers).

Manufacturers seem favorable to having a progress report provided (91%) during the re-evaluation process; however, only 21% of building officials feel that a “partial” report would be very useful. Building officials confirmed through the individual interviews and the focus group sessions that they would avoid using such reports due to concerns related to liability issues.

The value added by the CCMC’s offering on-site visits is questionable. 63% of building officials seem to disagree with the idea of CCMC not visiting construction sites but, since their confidence level in current CCMC reports/listings is high, the added value of on-site visits is questionable.

Building officials and manufacturers agree that the CCMC could extend its services to include:

- energy efficiency (84% business officials and 67% manufacturers); - water conservation (75% business officials and 50% manufacturers); - green initiatives (69% business officials and 65% manufacturers); - sustainability/conservation (69% business officials and 61% manufacturers).



Page: 6 / 33

2.0 Introduction For the past 20 years, the Canadian Construction Materials Centre (CCMC), part of the National Research Council’s Institute for Research in Construction (NRC-IRC), has offered the construction industry a national evaluation service for innovative, non-standardized construction products on a cost-recovery basis. The CCMC provides an opinion on whether a product/system complies with the minimum performance level established by the National Building Code (NBC). In spring 2009, the CCMC embarked on a Business Review project aimed at examining its business model for evaluating construction products, its mission and if it was meeting the needs of its clients. Many changes have occurred in the construction market and the regulatory framework over the past few years. The CCMC has continually responded to such changes by enhancing its organization and services to meet demands. The CCMC wants to promote a proactive approach that is both serious and objective so it can make changes as required to meet current and future expectations. To that end, the NRC-IRC has engaged Brio Conseils, a firm specializing in transformation management, entrusting it with heading and managing, in a transparent manner, a consultation with the various players involved in the construction industry. The main goal of this consultation is to identify stakeholders’ needs and determine if the CCMC is meeting those needs with its current business model. The main objective of this report is to provide a clear understanding of the CCMC’s role, mandate and services and its relationship to construction community-regulatory and industry stakeholders, both in Canada and internationally. More specifically, this report will: define the CCMC’s current role, mandate, services, processes, strengths and weaknesses; identify future trends that could be a threat to or an opportunity for the CCMC; identify current and emerging client needs.



Page: 7 / 33

3.0 Methodology The CCMC’s business review involves three phases, as illustrated below:

Project Timeline and Deliverables

3.1. Phase 0: Project Scope and Methodology

The NRC has appointed a team of approximately 10 people to direct the project and consult with and report to NRC-IRC management. The project team consists of:

NRC management and corporate support (3)1; CCMC manager assisted by staff (2); chair of Canadian Commission on Construction Material Evaluation (1); provincial-territorial regulatory authority (1); building officials (1); manufacturing industry (2); consultant recruited by this contract (1).

This Business Review is being managed using project management best practices. The role of each team member is illustrated in the diagram below.

1 In December 2009, Mr. Bill Cowley has been replaced by Mr. Dino Zuppa



Page: 8 / 33

Project Team Organizational Structure

Bob BowenProject sponsor

John FlackProject manager

Denis BergeronProject client

Francine BelzileBrio Conseils

Bill CowleyCentral business

support

Michelle GagnonC&E

Communications Officer

NRC-IRC Management Committee

CCCME

Kim CampbellAd hoc research

Chris TyeMunicipal affairs

(Edmonton)

Gary ChuDow Chemical Canada ULC

Don FigleyCCCME Chair

Mark AlbrightEvaluation officer

Jim DonovanBuilding permits

(Halifax)

Jim WhalenPlastifab corp.

Working Team

Executive

A project scope including the objectives, deliverables, action plan and schedule, was presented to the project team on June 2, 2009 for approval. During that meeting, the methodology proposed by Brio Conseils was reviewed based on the Statement of Work supplied by the NRC-IRC. The purpose of the review was to ensure that NRC-IRC’s main goal were met, which is to obtain an objective assessment of the current situation, identify issues that need to be addressed and allow the NRC-IRC to make decisions and recommendations. It was also important for the NRC-IRC to conduct a broad, open and transparent consultation with key stakeholders. During that meeting, the fact that the team members came from different areas of the construction community helped in developing a methodology that took into account their perspectives.

3.2. Phase 1: SWOT Analysis

As stated in this document’s introduction, the objective of phase 1 of this project was to achieve a clear understanding of the CCMC’s role, mandate and services and its relationship to the construction community-regulatory and industry stakeholders, both in Canada and internationally. The outcome was to provide a SWOT Analysis. The main deliverables of phase 1 were:

Internal Environmental Assessment Environmental Scan

3.2.1. Internal Environmental Assessment

The Internal Environmental Assessment was conducted through personal interviews with CCMC evaluation officers (9), administrative staff (3), code officers (2), NRC-IRC researchers (2), IRC-NRC management (2) and external partners (consultants, laboratories, university researchers) (3), who are collaborators in the evaluation process. The interviews were conducted by Francine Belzile of Brio Conseils, from April to September 2009. A discussion guide was used for the interviews. The Internal Environmental Assessment allowed us to identify the CCMC’s key strengths and weaknesses, which were validated with the project team.



Page: 9 / 33

3.2.2. Environmental Scan

The Environmental Scan looked at the CCMC's place in the external environment to identify future trends (political, technological and socioeconomic) that could be threats or opportunities. Input came from interviews with stakeholders, Web sites, news and analyst reports.

3.3. Phase 2: Stakeholders’ Consultation

The objective of Phase 2 of the Business Review was to identify stakeholders’ key issues. This phase had two parts. The purpose of the first part was to better understand stakeholders’ needs and their satisfaction level with the CCMC’s services. To do that, primary stakeholders were surveyed. The purpose of the second part was to consult stakeholders on specific issues in order to validate and have an open discussion about the findings from the first part.

3.3.1. Surveys with Primary Stakeholders

All surveys with primary stakeholders were conducted in partnership with Ad hoc Research. Over the past 25 years, Ad hoc Research has built up extensive experience conducting consultation studies aimed at better understanding the needs and expectations of clients and assessing their satisfaction. All interviews and surveys were conducted by Ms. Kimberley Campbell, Vice-President and Partner at Ad Hoc Research. The surveys were used to gather qualitative and quantitative data and identify stakeholder needs from three main groups: Provincial and territorial ministry officials responsible for construction policy, municipal building officials and manufacturers. All the information related to how the data was collected for each group is included in the table below:

Ministry Officials Building Officials Manufacturers Who

Members of PTPACC (11)

Members of each provincial building officials associations (229)

Clients who have used CCMC services the past two years (54)

How

Stakeholder interviews Web-based survey Web-based survey

When May through August August 19 to October 21 September 15 to October 10

Brio Conseils analyzed the data collected, which will form the basis of the second part.

3.3.2. Focus groups with primary stakeholders: The focus groups were held across the country from late September to November 2009. Focus group meetings consisted of half-day workshops (morning or afternoon) in four locations: Halifax, Toronto, Calgary and Vancouver. Attendance was by invitation only and included representation mainly from manufacturers and building officials but also from provincial ministries and other stakeholders.

Focus group meetings explored specific issues in order to validate results from the first part. Depending on the number of participants, one or two groups met in each of the following cities:



Page: 10 / 33

Location Date

Calgary (1 group) Tuesday, September 29, 2009

Vancouver (2 groups) Monday, November 16, 2009

Toronto (2 groups) Wednesday, November 18, 2009

Halifax (1 group) Wednesday, November 24, 2009

Because the Province of Quebec’s participation rate in the Web surveys was particularly low, it was decided to conduct personal interviews instead of focus groups to better understand the evaluation process for innovative construction products in that province.

3.3.3. Broader Stakeholder Consultation

So everybody has an opportunity to give their opinion, the summary report can be read on the Web site, and feedback can be provided via the Web, mail or telephone in January and February 2010.

3.4. Phase 3: Final Report

The Business Review project team (see 3.1) will review the results, prioritize the findings into groups of ideas and make recommendations as to what can be moved forward on.

The Final Report will be available in March 2010 and provide a prioritized list of recommendations. It will be submitted to the CCCME for review before going to NRC-IRC management for consideration and decision.



Page: 11 / 33

4.0 Business Review A business model is the means by which an organization creates value for its various stakeholders. In the case of the CCMC, these stakeholders would include, but not be limited to, manufacturers, building officials and ministry officials. To help prioritize how it creates value for its stakeholders, the IRC has set out a strategic and operational plan that outlines the organization’s vision/mission and values as well as the goals and activities used to fulfill the organization’s strategic direction.2

The CCMC’s credibility is a direct indication of the significant value it has managed to create for its various stakeholders over the past 20 years. Even though the organization is currently in a good position, potential opportunities and issues exist that the CCMC should address so it can continue to provide exceptional value to its stakeholders. The business model review looked at three elements of the organization: current business model (mandate/role, stakeholders, services, processes, strengths and weaknesses), future trends (threats and opportunities) and stakeholder needs. This review will allow the NRC-IRC to determine if any changes are required to the existing CCMC model to meet future expectations.

2 Institute for Research in Construction, Strategic Plan 2004-09.



Page: 12 / 33

4.1 Current business model

The CCMC’s mandate is to facilitate market access to innovative construction products. To fulfill that mandate, the CCMC offers two services (Listings and Reports) to their stakeholders. Processes have been developed to ensure that the services are provided in step with stakeholders’ needs. This section describes each element to provide a better understanding of the current CCMC model. Section 4.4 will cover stakeholder needs. This section will review the CCMC’s mandate/role, stakeholders, services and processes. Strengths and weaknesses will be presented at the end of the section.

CCMC Business Model

4.1.1. CCMC Mandate/role In the NRC-IRC’s strategic plan, the mission statement is “To improve the lives of Canadians through a vibrant construction sector that provides a quality and cost-effective built environment”, and the vision is to be “A recognized leader in the development of a quality built environment through research, innovation and the creation of integrated solutions.” The Institute is built around five programs, one of which is the “Codes and Evaluation Program”. The Codes and Evaluation Program is a client- and stakeholder-driven program with two significant deliverables for the Canadian construction industry: the first is a set of National Model Codes, the second is the CCMC, which is described as a technical evaluation service that facilitates acceptance of new and innovative construction materials, products and services nationally and internationally. The program promotes industry development and the commercialization (nationally and internationally) of innovative construction products through flexible, technically credible evaluations and linkages with foreign assessment organizations. It is also an international leader in the advancement of common approaches to objective-based codes.

CCMC Current

Business Model



Page: 13 / 33

When we asked employees and their partners what they thought about the CCMC’s mission/role was, they generally expressed the same understanding:

• provide an opinion on innovative construction materials/systems based on the building

code; • help building officials make decisions; • help manufacturers or proponents bring innovation to the marketplace.

Employees were asked to identify what is helping them fulfill their role and what is hindering them from doing so. Here is the outcome:

Help Hinder

External • Network of experts: labs, universities, research centers

• CCMC’s recognition

• Increasing complexity of products and systems under evaluation

• Accelerating innovation makes manufacturers want answers on code compliance faster

• Liability issues: nobody wants to take responsibility if a product goes wrong



Page: 14 / 33

Help Hinder

Internal • Access to expertise and testing facilities for product evaluation

• Engineers who have a code of conduct; they are professionals

• Recognized as neutral • Close to the code.

• Process has been enhanced and is expanding but nothing has been removed

• Bottlenecks in the process (bureaucracy)

• Lack of consistency between evaluation officers

• Lack of availability of internal researchers

• Evaluation officers need a good grasp of different kinds of expertise: technical, management, client-oriented, etc.

4.1.2. Stakeholders

The project team members reached a consensus as to who should be a direct or indirect stakeholder. First, a direct stakeholder is a group of individuals who are immediately affected by the service offered by the CCMC and who stand to personally benefit or lose from any changes made to the CCMC business model. Direct stakeholders are:

ministry officials (provincial); building officials (municipal); manufacturers.

An indirect stakeholder is a group of individuals or organizations that are removed from immediate involvement but are either linked to those who are directly affected or are benefiting indirectly from the services offered by the CCMC. Indirect stakeholders are:

construction firms; engineers; architects; architectural technicians; spec writers.

During this Business Review, extensive research was conducted to gather information about primary stakeholders’ needs and their satisfaction with the services offered by the CCMC.

4.1.3. Services The CCMC offers two different services: Listings and Reports.

Listings Evaluation Reports

Purpose For products/systems for which there is a standard

For products/systems for which there are no standards

Definition Consists in a single page document containing the following information

Consists of a document ranging from 4 to 25 pages and divided into the following sections:

Section 1 States the standard used for Provides a CCMC opinion on



Page: 15 / 33

Listings Evaluation Reports product evaluation compliance with the NBC 2005

Section 2 Provides a general description of the product that was evaluated

Provides a general description of the product that was evaluated

Section 3 Provides additional information on the standard, as well as any relevant regulatory information

Outlines the conditions and limitations associated with the evaluation as well as the limitations on the use of the product

Section 4 Provides administrative and contact information

Outlines the technical data associated with the product or system. The CCMC uses the data in 4.1 as the basis for the opinion. All additional data are either requested by the proponent or related to health and safety issues not addressed by the Code. In specific jurisdictions, additional data may be required by the regulatory authority

Section 5 Provides administrative and contact information

All Listings and Evaluation Reports bear a unique evaluation number, such as CCMC XXXXX-L or CCMC XXXXX-R, and are published in the CCMC Registry.

Follow-up of Evaluation Reports: Every year, the CCMC requires the applicant to affirm that the product has not been modified in any way. Every three years, the CCMC re-evaluates the product, and full or partial tests are performed as necessary. 4.1.4. Processes Organizational structure The CCMC is made up of 1 manager, 9 evaluation officers and 3 support staff. They work closely with NRC researchers and code officers as well as external consultants (researchers, universities, laboratories, etc.). CCMC operations report to the NRC-IRC and get guidance from the Canadian Commission on Construction Materials Evaluation (CCCME) representing the geographic regions of Canada and diverse sectors of the construction industry. For example, they provide input to support direction in cases of outside appeal or conflict resolution. Note that governance is beyond the scope of this project and will be covered by the Governance Review under the responsibility of the Institute on Governance.



Page: 16 / 33

Evaluation processes Proponents should submit an application through either the Web site or by fax or mail. When the CCMC receives a request to evaluate a product, an evaluation officer examines a sample of the product, notes its basic characteristics and then determines whether the product is eligible, either as an innovative product (Evaluation Report) or as a standardized product (Listing). The CCMC follows a seven-step evaluation process for each eligible client. This process is almost the same for the report and the listing. The main difference is that for a listing, the evaluation is done according to existing standards while for a report, the test methods and evaluation criteria need to be developed. These processes are illustrated below:

Listing3

3 Flack, John, Ph.D.. Evaluating to objective-base codes. November 8th, 2007.



Page: 17 / 33

Evaluation Report

• published

• Product/proponent specific

• Code, other, requested• Benchmark defined

Preparation of anEvaluation Report

• Results review• Limitations applied

• Recognized

Evaluation of Resultsand Opinions

Laboratory Testingby a Third Party

Preparation of a Technical Guide

• P/T input• Code analysis• Defined scope (NBC)• Compliant or alternative

Scope andEvaluation Plan

Eligibility Reviewed

Application byProponent

CCMC performance

When looking at the number of closed files that have been handled by the evaluation officers, we realized the trend has been pretty stable over the years. The CCMC believes the workload is heavier because the products/systems are more complex and require more time to evaluate.

2006 2007 2008 2009 (actual) New listings 28 20 15 16 New reports 24 10 11 20 Re-evaluations 209 306 260 221

CCMC’s key challenges

Aware of the evolution of the construction industry and conscious of keyholders’ needs and preoccupations, the CCMC decided to conduct a Business Review to answer some of these challenges:

Increasing complexity of products and systems under evaluation; More clients’ complaints going through top management regarding:

- Time and cost, - Don’t agree, don’t understand evaluation process, - Don’t get the opinion they want;

Frequency of evaluation (3 years vs. 5 years); Misuse or misunderstanding of CCMC reports and opinions;



Page: 18 / 33

Risk management: health & safety (CCMC) vs. liability (building officials); Existing processes in place are not optimized:

- Process has been enhanced, but nothing has been removed, - Complex process to evaluate standard products (listing), - Capacity of internal and external resources; - Bureaucracy is growing every year;

Lack of relationship with building officials, don’t know if CCMC is meeting their needs, not really sure what those needs are.

4.1.5. CCMC strengths and weaknesses Based on all the interviews conducted, the strengths and weaknesses most often mentioned are: Strengths: Credibility, NRC resources; Unbiased opinion: federal non-profit, more respected, more independent; As a team, scope of knowledge across the board; Employees motivated to provide good service; National presence; Very thorough technically – how products need to be evaluated; Network of experts: labs, universities, research centers; Reports good, reliable.

Weaknesses: Slow service; Cost of product qualification (CCMC + R&D + testing); Existing processes/tools in place are not optimized.



Page: 19 / 33

4.2. Future trends

These future trends were identified through individual interviews with CCMC stakeholders, Web searches and Web surveys with building officials and manufacturers. We focused on political, socioeconomic and technological aspects.

Objectives:

- Provide an understanding of the innovative construction product market. - Identify trends in the construction industry and understand their impact on the CCMC

business model. - Support and improve strategic decision making through better data and information

capture and analysis.

4.2.1. Key findings

• Political – Several provinces are moving ahead with green and sustainability legislation.

There is a concern that the regulatory structures to support this are fragmented and insufficient;

– Pressure is mounting for some degree of harmonization of building product standards, Canada is not that different from other countries;

– NAFTA and other international trade agreements are looming. It could open the door to foreign code or standard recognition.

• Socioeconomic

– Innovation is a strong growth driver for the economy: today’s market is defined by the number of innovative products and materials that appear on the market;

– There is a significant move to pre-fab, semi-built units, particularly units fabricated on site;

– The crisis in skilled trades may force the adoption of different building products that demand lower skill levels from the trades that install them;

– The extent of building officials’ knowledge and experience varies greatly within any province. The turnover rate among building officials is very high;

– As budgets tighten and workloads increase, cities have contracted with private-sector firms to administer permitting and plan review and inspection services related to building safety.

• Technological

– More innovative products are using more sophisticated materials that perform multiple roles in the building;

– The push is on for greater utilization and value added from raw materials—wood composite beams rather than solid lumber;

– Sustainable building products are being used; – Pre-assembled construction is on the rise.

Future

Trends



Page: 20 / 33

4.2.2. CCMC threats and opportunities

The opportunities and threats listed below come not only from the environmental scan to identify the future trends, but also from the internal assessment presented in section 4.1 CCMC Current Business Model. Opportunities: Several provinces are moving ahead with green and sustainability legislation. There is a

concern that the regulatory structures to support this are fragmented and insufficient; Standard recognized internationally; Innovation is a strong growth driver for the economy: today’s market is defined by the

number of innovative products and materials that appear on the market.

Threats: Loosing expertise for evaluation and testing; Building Officials’ knowledge and expertise varies greatly within province; Time pressure on quality and cost; Increasing complexity of products and systems under evaluation; Misuse or misunderstanding of CCMC reports and opinions; Push for new green and energy efficient technologies may have an impact on the

demand for evaluation of innovative products.

4.3. SWOT

A SWOT Analysis is a strategic planning method used to evaluate the Strengths, Weaknesses, Opportunities, and Threats related to the CCMC’s objective, which is to facilitate market access to innovative construction products. The internal (strengths and weaknesses) and external (opportunities and threats) factors taken from the internal assessment and environmental scan have been used to identify what is favorable and unfavorable to achieving that objective. Strengths: Credibility, resources from NRC Unbiased opinion: federal non-profit, more

respected, more independent As a team, scope of knowledge across the

board Employees motivated to provide good

service National presence Very thorough technically – how products

need to be evaluated Network of experts: labs, universities,

research centers Reports are good, reliable

Weaknesses: Slow service Cost of product qualification Existing processes/tools in place are not

optimized



Page: 21 / 33

Opportunities: Several provinces are moving ahead with

green and sustainability legislation. There is a concern that the regulatory structures to support this are fragmented and insufficient;

Standard recognized internationally; Innovation is a strong growth driver for the

economy: today’s market is defined by the number of innovative products and materials that appear on the market.

Threats: Loosing expertise in evaluation and testing Lack of expertise among building officials Time pressure on quality and cost Increasing complexity of products and

systems under evaluation Misuse or misunderstanding of CCMC reports

and opinions Push for new green and energy efficient

technologies may have an impact on the demand for evaluation of innovative products.



Page: 22 / 33

4.4. Stakeholder Needs

A code requirement or referenced standard exists for most of the products installed in new construction. When an innovative product is developed, many times it doesn’t fit with an existing standard. In that situation, the manufacturer can use the service from the CCMC to provide the proper information to the building official so he or she can accept or reject the installation of the product in a new construction. In the late 1980s, provincial/territorial representatives agreed to support a national evaluation body. Today, the consensus is that there are advantages to having a national product evaluation service to help determine code conformity.

Process to market an innovative product

The focus of the surveys with Stakeholders was to identify primary stakeholders’ needs and their satisfaction with the CCMC’s current services. Preliminary recommendations will be based on the gap between stakeholders’ needs and current CCMC services and the changes required to address reported issues.

4.4.1. Ministry officials

Brio Conseils mandated Ad Hoc Research to interview various ministry officials, which took the form of eleven (11) in-depth telephone interviews (the detailed profile of respondents is in an Appendix A). The participants were all members of the Provincial/Territorial Policy Advisory Committee on codes (PTPACC) responsible for articulating provincial/territorial views on the content and development of the model National Building Code (NBC), National

Client

Needs



Page: 23 / 33

Fire Code (NFC) and National Plumbing Code (NPC), and other national model codes, advisory and progeny documents. The PTPACC provides a forum for the provinces and territories to discuss policy and share information on the administration of these codes (see the overall methodology in section 2.0 for more details). The PTPACC members are also advise the CCMC on matters related to the need, impact and utility of construction material evaluation services.

The ministry official interviews surfaced information that would be difficult to obtain using more structured research methods. They provide an accurate picture of the overall satisfaction with CCMC services and help identify elements with which stakeholders are satisfied as well those needing improvement.

Almost all of those interviewed have relatively limited contact with the CCMC throughout the year. The contact most often mentioned is that of checking the Web site: to verify the status of a product (to see if it has a listing or report number and what

stipulations are mentioned in the CCMC report or listing); before a case of appeals; to get more information on a product evaluated by the CCMC following an industry

complaint or an issue regarding a material that had been ‘approved’ by the CCMC.

Ministry officials’ satisfaction level

Ministry officials seem to be satisfied overall with CCMC services. The CCMC provides an invaluable service. Many provinces do not have the resources to test products and/or determine if they are compliant, so the vast majority prefers that the CCMC simply stay with its current scope and not compete with private industry.

The positive aspects of CCMC services are presented below in order of importance.

Credible: The CCMC is considered to be a very credible organization as a source of

information.

A rigorous and thorough, standardized process for product evaluation. As such, they tend to put a lot of faith in any resulting CCMC evaluation.4

Client relationship: speed of response and quality of communication with the CCMC.

Knowledgeable staff: All agree that the people who answer their questions are

knowledgeable and capable of understanding and discussing the various issues at hand.

Technical details: Several respondents spontaneously mentioned appreciating the amount of technical information provided (especially in the reports).

Situation of CCMC within NRC: All agree that the CCMC is well placed under the

umbrella of the NRC. Indeed, none of them had really questioned the soundness of its current placement.

4 The words “approval” was used when the respondents were referring to the service offered by the CCMC.



Page: 24 / 33

Consultation during the early stage and the evaluation process: The vast majority

seem relatively satisfied with the level of consultation regarding innovative products. The vast majority appreciate the fact that the CCMC communicates with them and asks for their comments on what to include in the technical guidelines for innovative products. Consultation during the process was requested by only a few: the vast majority feels that once the process has begun, they do not need to provide input.

Needs for improvements

One of the main issues does not seem to be the CCMC’s scope itself but rather ensuring that everyone is aware of that scope.

Lack of communication: Several ministry officials mentioned regularly receiving e-mails

from other government agencies but rarely receiving any news from the CCMC. They would prefer a more regular dialogue between the CCMC and themselves as well as with the building officials.

Lengthy process for the report service for innovative products: Many claim to hear

negative feedback from manufacturers. Many ministry officials are of two minds when it comes to the time aspect since they feel it is important to evaluate the products rigorously—which in turn can be a time-consuming process.

Wrongful use of CCMC reports: The process of approving innovative products is

exposed to a high level of liability since building officials are not necessarily using the CCMC reports as they should be.

Many re-evaluations are not up-to-date: Several mentioned there seems to be a

backlog in terms of re-evaluations: most prefer a three-year re-evaluation cycle but question the CCMC’s capacity to achieve it. However, several commented that the CCMC already has difficulties re-evaluating products on a three-year cycle and wondered if the proposed five-year cycle might give the CCMC an opportunity to “catch up” on the backlog.

Other aspects probed in detail

Ministry officials have little interest in partially updated re-evaluations, but many suggest a

progress report.

Needs for future: Respondents were asked what areas the CCMC might need to consider in their evaluation process in the future. The following elements were mentioned most often: - energy efficiency; - water conservation (grey water); - water consumption; - length of product life (especially for northern communities).



Page: 25 / 33

Awareness of CCCME (Canadian Commission on Construction Materials Evaluation). The vast majority knows the CCMC and is completely unaware of its CCCME counterpart.

4.4.2. Building officials

The Web survey involved 229 building officials. The majority (46%) of respondents were from Western provinces, 26% were from Central provinces and 4% were from the Yukon, Northern Territories and Nunavut. Despite several efforts to get Quebec building officials to participate, only 1% of respondents were from Quebec. As a result, alternative approaches were developed to obtain feedback from the Province of Quebec. The detailed profile of respondents is in an Appendix B. This Web survey focused on clearly identifying needs and issues reported by building officials. Building officials’ needs:

High level of comfort approving standardized products: Building officials are far more

comfortable approving standardized products (average of 9.3 out of 10) than non-standardized products (rated at only 3.2 out of 10).

Many building officials admit to lacking expertise to review the necessary

documentation when it comes to innovative products: A total of 77% agree to some extent (22% completely agree and 55% somewhat agree) with the statement that “in many cases I feel that I do not have the technical expertise/support to review all the documentation necessary to approve innovative construction products/systems for which no standards exist”.

Many building officials admit to having no time to properly review documentation

when it comes to innovative products: A total of 77% agree to some extent (30% completely agree and 44% somewhat agree) with the statement “I would like to review all the documentation necessary to approve an innovative construction product but I have very little time to do so”.

Building officials’ satisfaction level Listings: A CCMC listing was rated the highest of 5 sources of information when it comes to

products for which a standard exists. In fact, 79% claimed to be very likely to approve a product based on a CCMC listing. The other sources and their likelihood of being used as a source of information are: a label referencing a recognized industry standard (50%), certification programs (24%), manufacturer label and technical data sheet (13%), laboratory report (9%).

Almost all building officials agree that the CCMC listings are useful (62% very useful and 35% somewhat useful). What they find useful about the listings are: - quality of information provided (56%); - reliability of trustworthiness of the CCMC (29%); - ease of access to information (12%).



Page: 26 / 33

Reports: A CCMC report was rated the highest source of information when it comes to products for

which a standard does not exist. A total of 77% are very likely to consider the CCMC as a source of information. The other sources and their likelihood of being used as a source of information are: - an alternative solution submitted by a proponent as per objective-based code (40%); - assessment by an engineer (34%); - assessment by an architect (21%); - approval by another body/authority having jurisdiction (17%); - ICC_ES Report (12%); - independent analysis of manufacturer’s literature (11%); - information on product’s performance in another location (11%).

Almost all building officials agree that the CCMC reports are useful (66% and 32% somewhat useful). What they find useful is the quality of the information provided.

Services Although the perception of the CCMC’s services is quite positive, the fact respondents do

not agree totally with the following statements showing some opportunities for the CCMC to improve: The CCMC offers an impartial third party verification that is of value to my activities

(66% completely agree and 30% somewhat agree). The CCMC’s services address my needs as a building official (40% completely agree

and 52% somewhat agree). It is easy to obtain the information I need from the CCMC (38% completely agree and

46% somewhat agree). I rely on the CCMC for my approval of innovative construction products/systems for

which no standard exists (45% completely agree and 39% somewhat agree). If the CCMC has issued a number to a product, I assume that the product is NBC

compliant (38% completely agree and 43% somewhat agree). Of those who did contact the CCMC, 97% were very (46%) or somewhat (52%) satisfied

with the support/service they received. From those who were not very satisfied (3%).The most common complaints regarding the service are: slow response time; quality of information provided.

Issues There is a misconception regarding the difference between a listing and a report: Almost half the building officials admit to not knowing the distinction between both

services. The scope of the listing seems misunderstood:



Page: 27 / 33



- Two-thirds of building officials (67%) do not know the CCMC listing does not evaluate any additional requirements imposed by the national or provincial building code.


The building officials may not necessarily be using the CCMC report as they should: Most building officials very likely read the report to ensure that the product/system meets

the conditions and limitations stated (73%) and that it has a valid CCMC number (73%). Only half (49%) are very likely to read the full report thoroughly. Only 23% are actually aware of the format change introduced two years ago. Those who

are aware of it seem to feel that the 2005 report format is an improvement over the 1995 format (94% agree completely or somewhat).

Less than half the building officials are aware of the review of the installation manuals (42%) or of the review of the quality control procedures (40%).

A total of 80% of respondents find the optional information at the end of the report useful, although only 27% find it very useful.

4.4.3. Manufacturers A total of 54 manufacturers who currently have a CCMC listing or report number responded to the invitation. They were invited to participate in the survey directly through e-mail. Most of the respondents came from a company with more than 100 employees. More than half the respondents have their head office in the United States and 46% in Canada. The detailed respondent profile is in Appendix C.

The focus of this Web survey was to clearly identify needs and issues reported by manufacturers.

Manufacturers’ needs

Most respondents (69%) distribute their product nationally, and Canada is their primary market. The majority of manufacturers identify code compliance requirements in the product development process:

- in the development stage of their products (77%); and - before deciding to distribute it (78%).

Most respondents recognized that the services provided by the CCMC are useful: On average, 58% of manufacturers’ innovative products (without a standard) have a

CCMC report number, and 56% of innovative products (with a standard) have a CCMC listing number.

98% of manufacturers agree (53% completely and 45% somewhat) that having a CCMC number gives them better access to the Canadian market.



Page: 28 / 33

96% of manufacturers agree (52% completely and 44% somewhat) that having a CCMC report for innovative products or a listing makes it more likely that their product/system will be selected by architects, engineers or construction firms.

96% of manufacturers agree (33% completely and 63% somewhat) that a CCMC number will most likely provide an immediate approval for use.

Although manufacturers seem to find the CCMC listings useful (84% very or somewhat), they seem to agree that the reports are more useful (96% very or somewhat). The majority of the reasons given for considering the termination of CCMC listings as having a significant impact relate to the manufacturer’s perception that the construction industry and professionals currently seem to rely heavily on the CCMC listings—as such they are concerned that it will be more complicated to obtain approval for their products.

Satisfaction level

100% of manufacturers are very or somewhat satisfied with the current level of

confidentiality offered by CCMC. 83% of manufacturers find the current 3 year protection period for a technical guide to be

acceptable. Although many manufacturers are satisfied with the current 3 year re-evaluation cycle

(78% are very or somewhat satisfied), there seems to be an openness to changing the cycle to a 5 year one (87% completely or somewhat agree with the idea). During the focus group sessions, several manufacturers and building officials made the point that the actual re-evaluation cycle is not a central point since, in a constantly evolving market, the chances are that the manufacturer will change their quality control process well within the 3 or 5 year period.

The satisfaction level with the support/service respondents received when they contacted the CCMC is relatively good (27% very and 49% somewhat satisfied); however, there seems to be room for improvement. Almost a quarter (24%) of manufacturers claim to be not very or not at all satisfied. The main reason mentioned is because it is ‘time consuming/has a slow response time’).

Issues

Time is of the essence in today’s world. Every day a manufacturer cannot get access to the market is lost money. For that reason, it is no a surprise that timeliness is the major issue. There seems to be a general dissatisfaction among manufacturers with the time it takes

the CCMC to complete a report (only 35% are very or somewhat satisfied). The two elements that seem to drive this dissatisfaction are: - the time it takes to develop the technical guide; - the time it takes to receive the final report after successful testing.

Almost a third of manufacturers (30%) claim to have had a product refused by a building official despite having a CCMC number.



Page: 29 / 33

4.4.4. Opportunities During the survey, opportunities were evaluated with both building officials and manufacturers: The manufacturers seem to think that making the progress of their file open to

themselves (91%) or to building officials (78%) could be useful; however, their level of comfort is less when it comes to all the other potential entities to whom the information could be given (builders, architects, other manufacturers).



Building officials and manufacturers agree that the CCMC could extend its services to include: - energy efficiency (84% business officials and 67% manufacturers); - water conservation (75% business officials and 50% manufacturers); - green initiatives (69% business officials and 65% manufacturers); - sustainability/conservation (69% business officials and 61% manufacturers).



Page: 30 / 33

4.4.5. Summary of the findings

Stakeholder Needs CCMC Value Issues Manufacturers: Product compliance with code A CCMC number gives them better

access to the Canadian market. Having a CCMC report or listing makes

it more likely their product/system will be selected by architects, engineers or construction firms.

A CCMC number most likely enables an immediate approval for use.

Dissatisfaction with the time it takes CCMC to complete a report.

Slow response time to queries.

Building Officials: More comfortable approving standardized products than non-standardized products.

Lack of expertise to review documentation for innovative products.

Lack of time to properly review documentation.

Listings and reports are the highest source of information when it comes to evaluating innovative products with or without standards.

Misconception regarding difference between a listing and a report.

Building officials may not necessarily be using the CCMC report as they should.

Although the perception of the CCMC’s services is quite positive, the fact building officials are not totally satisfied shows some opportunities for the CCMC to improve:

Ministry Officials: Want to provide support to the building industry for determining acceptability of innovative products in regards to building code requirements.

Credible. Rigorous and thorough, standardized

process. Client relationship. Knowledgeable. Provide technical detail. Situation of CCMC within NRC. Consultation during early stages of

evaluation process.

Lack of communication. Lengthy process. Wrongful use of CCMC reports. Many re-evaluations not up-to-date.



Page: 31 / 33

Appendix A Interviews with PTPACC Members

Respondent profile

Posit ion Division Location

Building code coordinatorPublic safety division Nova Scotia Labour & W orkforce Development

Halifax, Nova Scotia

Manager, Building Safety Branch Dept of Community Services, Consumer & Protective Services, Branch Building Safety

W hitehorse, Yukon

Vice Chair , PTPACC, Executive director

Building & Safety Policy Branch, Ministry of Housing & Social Development

Victoria, BC

Director & Chief Building OfficialBuilding & Development Branch, Ministry of Municipal Affairs & Housing

Toronto, Ontario

Fire MarshalMunicipal & Community Affairs, Government of the Northwest Territories

Yellowknife, Northwest Territories

Senior architect, building regulation Direction de la normalisation et de la qualification, Régie du bâtiment du Québec

Montréal, Québec

Chief Building Official Building Fire and Safety

Regina, Saskatchewan

Director, Codes & Standards Safety Services, Municipal Affairs Edmonton, AlbertaDeputy Fire Commissioner Office of the Fire Commissioner W innipeg, ManitobaDirector Technical Inspection Services

Public Safety/Securite publique New Brunswick/Noveau-Brunswick



Page: 32 / 33

Appendix B Web Survey with Building Officials

Respondent profile

A1. What is your training background? *Technologist 45% Trade back-ground 32% Building off icial 18% Contractor 14% Designer 9% Engineer 9% Building inspector 8% Architect 6% Safety codes off icer 1% Manager 1% Other 4%

A2a In what Province/Territory do you work as a building official?TOTAL - West 45%

Alberta 11% British Columbia 29% Manitoba 2% Saskatchew an 3%

TOTAL - Central 26% Ontario 25% Quebec 1%

TOTAL - East 25% New Brunsw ick 11% New foundland - Labrador 1% Nova Scotia 13%

TOTAL - North 4% Yukon 2% Northw est Territory / Nunavut 3%

A2b Do you work as a building official…Full time position 93% Part time position 7%

A2c What are your responsibilities as a building official?*The f ield inspector 76% Plan analyst w ho is involved in issuing the building permit 70% Fire Inspector 13% Responsible for building maintenance 8% Chief building off icial 7% Manager / Managerial duties 4% Regulation policy developmentand implementation 3% Other 19%

n=229

The maximum margins of error at a confidence level of 95% for the total sample is ± 6.48%.



Page: 33 / 33

Appendix C

Web Survey with Manufacturers Respondent profile

A1. In what field does your company specialize? Thermal Protections and Weather Barriers 41% Roofs 30% Plastics and Composites 22% Wood (including Wood Adhesives) 11% Concrete 11% Doors and Window s 7% Finishes 7% Metals 6% Walls 6% Special House Construction Systems 4% Utilities 4% Joint protection 4% Insulation 4% Other 10%

A2. In which department do you work? Technical Services 37% Product Development 22% Ow ner / Administration 19% Marketing/Sales 13% Operations 2% Purchaser 2% Codes and standards 2% General manager 2% Research & development 2%

A3. How long have you been working in this field? TOTAL - Less than 10 years 32% 1 - 5 years 22% 6 - 10 years 9% TOTAL - 11 to 20 years 28% 11 - 15 years 15% 16 - 20 years 13%

TOTAL - More than 20 years 41% 21 - 25 years 13% 26 - 30 years 15% Over 30 years 13%

A4. What size is your company?Less than 10 employees 9% Betw een 11 and 100 employees 33% More than 100 employees 57%

n = 54

A5. In what Province / Territory do you distribute your products? All Canadian Provinces and territories 69% TOTAL - West 15% Alberta 13% British Columbia 15% Manitoba 13% Saskatchew an 13% TOTAL - Central 24% Ontario 24% Quebec 15% TOTAL - East 17% New Brunsw ick 9% New foundland and Labrador 2% Nova Scotia 11% Prince Edw ard Island 6% TOTAL - North 4% Northw est Territories / Nunavut 2% Yukon 4%

A6. Is your primary market…Local 15% Provincial 28% National 69% International 44%

A7. Which of the following does your company conduct in Canada? Manufacture products/systems 72% Distribute products/systems 69% Export to USA 44% Export outside of USA and Canada 24% Install products/systems 15% Sell products to Canadian dealers / distributors 4% Distributes US made products to Canada 2% Sells US manufactured productsto distributors 2%

A8. Where is your head office located?Canada 46% United States 52% Outside of United States or Canada 2%

n = 54

The maximum margins of error at a confidence level of 95% for the total sample is ± 11.75%.



Brio Conseils inc.

Page: 13 / 21

APPENDIX C: Interviews with PTPACC members, Ad hoc Research

CCMC Business ReviewInterviews with PTPACC Members

September, 2009

Objectives and MethodologyObjectives and Methodology

Objectives and Methodology

Objectives

Interviews with Ministry Officials are one of several tools used to consult with the various CCMCstakeholders.

Objectives

The main objectives of these interviews are to: Draw out information that would be difficult to obtain using more structured research methods.

Determine overall satisfaction with CCMC services.

Identify elements which stakeholders are satisfied with as well as those which need to be improved upon.

There are also some specific elements that were deemed important to broach during the interviews:

Determine preferences when it comes to the basis on which listings should be provided.

Determine satisfaction with the layout of the CCMC Report for innovative products.

Ascertain preferences when it comes to dealing with situations in a report in which variations exist between the NBC and the provincial regulations/requirements regarding the use of certain productsprovincial regulations/requirements regarding the use of certain products.

Evaluate interest in a 5-year re-evaluation cycle vs. a 3-year cycle.

Assess interest in a partially updated report when extenuating circumstances cause a delay in a re-evaluation.

Collect any ideas for improving the timing for report.

Establish to what extent Ministry Officials are satisfied with the level of consultation in the early stages as well as during the y y g gevaluation process.

Ascertain the current satisfaction with the scope of CCMC services.

A secondary objective was to determine the awareness of CCCME and satisfaction with the level of representation of this committee.

3

Methodology

Overall Methodology Brio Conseil and CCMC mandated Ad Hoc Research to conduct interviews with various Ministry Officials, which

took the form of eleven (11) in-depth telephone interviews.

Position Division LocationPosition Division Location

Building code coordinatorPublic safety division Nova Scotia Labour & Workforce Development

Halifax, Nova Scotia

Manager, Building Safety BranchDept of Community Services, Consumer & Protective Services, Branch Building Safety

Whitehorse, Yukon

Vice Chair , PTPACC, Executive director

Building & Safety Policy Branch, Ministry of Housing & Social Development

Victoria, BCdirector Housing & Social Development

Director & Chief Building OfficialBuilding & Development Branch, Ministry of Municipal Affairs & Housing

Toronto, Ontario

Fire MarshalMunicipal & Community Affairs, Government of the Northwest Territories

Yellowknife, Northwest Territories

Senior architect, building regulation Direction de la normalisation et de la qualification, Régie du bâtiment du Québec

Montréal, Québecg Q

Chief Building Official Building Fire and Safety

Regina, Saskatchewan

Director, Codes & Standards Safety Services, Municipal Affairs Edmonton, AlbertaDeputy Fire Commissioner Office of the Fire Commissioner Winnipeg, ManitobaDirector Technical Inspection Services

Public Safety/Securite publiqueNew Brunswick/Noveau-Brunswick

Recruitment Names of Ministry Officials to be contacted were provided by CCMC.

4

Methodology (cont’d)

Discussion Guide The discussion guide was developed by Ad Hoc Research and approved by Brio Conseil and

CCMC. Readers are invited to refer to Appendix I to view the detailed discussion guide and exercise sheets used during the discussionexercise sheets used during the discussion.

Individual Interviews All interviews were conducted by Ms. Kimberley Campbell, Vice-President, Partner at Ad Hoc

Research. Interviews lasted on average 52 minutes each and were audiotaped.g p Unless otherwise mentioned in the text, findings in this report reflect the views of all those

interviewed.

5

Main LearningsMain Learnings

Main Learnings

1. The level of knowledge and experience of building officials varies greatly within anyprovince.

Some Overall Issues Which Can Affect CCMC Were Raised During the Discussion.

Almost all agree (with the exception of the province of Quebec) that there is a variation in knowledge and experience among building officials which, in turn, tends to affect their ability to interpret data provided by CCMC.

Rural areas are often cited as being particularly vulnerable to building officials who are part time employeesand do not necessarily have the background (engineering or architecture) to interpret much of thei f ti id d b th CCMC A lt th b ildi ffi i l h h l t l t i t tinformation provided by the CCMC. As a result these building officials who have less tools to interpretinformation tend to react in two different ways:

Either they tend to approve any product that has a CCMC number (listing or report);

Or they will tend not to approve a product for which CCMC has not specifically approved a particularuse for (even though they have been provided data which could have been used to make a decision).

“In major municipalities it is not an issue, but in smaller municipalities up north, the building officials don’t have the competencies required and the sophistication to interpret the data provided by CCMC.”

2. The variety of new materials and the rapidity with which they appear on the market. All pointout that the market today is defined by the amount of innovative products and materials thatappear on the market.

3. Many provinces do not have the resources to test the products and/or determine if they arecompliant. As such the CCMC provides an invaluable service.

4. Pre-assembled construction: One Ministry Official mentioned that pre-assembled constructionis on the rise and that CCMC will have to adapt to this reality in the future.

7

Main Learnings

The quasi totality of those interviewed have relatively limited contact with CCMC throughout the year.

The contact most often mentioned is that of checking the website to verify the status of a product

The Extent of Contact With CCMC is Limited for Most of Those Interviewed.

The contact most often mentioned is that of checking the website to verify the status of a product(does it have a listing or report number and what stipulations are mentioned in the CCMC report orlisting).

Other ad hoc reasons for contacting CCMC include: Appeals process: A few mention that they tend to refer to CCMC documents when a case of appeals is beforeAppeals process: A few mention that they tend to refer to CCMC documents when a case of appeals is before

them.

Industry complaint: A couple mention that if there is a complaint from the industry regarding a particularmaterial, they will tend to contact CCMC to obtain more details regarding what was tested and how.

An issue regarding approved material: Similarly to the previous point, a few mention contacting CCMC if anissue arises regarding a material that had been approved by CCMCissue arises regarding a material that had been approved by CCMC.

Indeed, direct contact with CCMC is usually reserved for: Requests for additional information regarding a report or listing.

Comments regarding a new report request (only mentioned by a few).

LEARNING

The Ministry Officials generally have limited contact with CCMC and as such their comments on the various questionstend to stay high level and do not tend to address the details of how listings and/or reports are presented.

8

Main Learnings

All except one of the Ministry Officials are aware of the differences between listings and reports.

Possible lack of awareness among building officials: Many mention that they doubt if localbuilding officials are aware of the differences between both services.

The Vast Majority are Aware of the Differences Between Listings and Reports.

bu d g o c a s a e a a e o e d e e ces be ee bo se ces

There Seems to be an Overall Satisfaction with CCMC Services.

Whether it be for listings or reports, the vast majority of Ministry Officials seem quite satisfied withthe services offered by CCMC. Below are some of the positive aspects mentioned:y p p

Credible: CCMC is considered to be a very credible organization and reliable source of information.

A i d th h t d di d All i t th t CCMC i i t i d

POSITIVE ASPCECTS RELATED TO CCMC SERVICES (in order of importance)

A rigorous and thorough, standardized process: All are in agreement that CCMC insists upon a rigorous andstandardized process for the approval of products. As such, they tend to put a lot of faith into any resulting CCMCdocuments approval.

“They don’t leave much un-examined! You can be sure that a CCMC report is covering all bases.”

The rapidity of response and the quality of communication with CCMC: The vast majority of Ministry Officials feelp y p q y j y ythat CCMC invests in creating a client relationship by:

calling/writing back very quickly.

engaging in an open and direct dialogue when discussing issues/problems.

Knowledgeable staff: All agree that those who are answering their questions are knowledgeable and capable of

9

g g g q g punderstanding and discussing the various issues at hand.

Provide technical details: Several spontaneously mention appreciating the amount of technical information that isprovided (especially in the reports).

Main Learnings

Aspects Mentioned as Possible Improvements (in order of importance)

1. Lack of Communication: The most commonly mentioned improvement is that of CCMC’s lack of communication with building officials.

“Wh CCMC f il t i t l b i th b ildi ffi l h ti ith CCMC d l“When CCMC fails to communicate on a regular basis, the building officers no longer have a connection with CCMC and are no longer motivated to keep themselves up-to-date.”

Many Ministry Officials feel that there is a huge lack of communication between CCMC and building officials. There is a relatively high turnover when it comes to building officials, and they are not always fully aware of:

CCMC’s mission;

th diff b t t d li ti the difference between reports and listings;

how to interpret reports and to a lesser extent listings (see point page 11);

the purpose of the re-evaluation of reports and listings (see pages 13 and 16).

Several Ministry Officials mention regularly receiving emails from other government agencies but rarely from CCMC. They would prefer a more regular dialogue with CCMC as well as withrarely from CCMC. They would prefer a more regular dialogue with CCMC as well as with building officials.

LEARNING

One of the main issues does not seem to be that of the scope of CCMC, but rather ensuring that all are aware of CCMC’s scope.

Although CCMC may provide information on its website, it seems abundantly clear that most building officials expect to receiveinformation directly. In their busy schedules, they do not tend to check the CCMC website to see if there is any news, they much prefer anewsletter format that links to sections of the website and as such facilitates the task of finding the information.

10

In addition, there seems to be an issue regarding who is responsible for informing building officials. Most Ministry Officials seem to feelthat the responsibility falls upon CCMC as opposed to the individual provinces/territories.

Main Learnings

Aspects Mentioned as Possible Improvements (in order of importance) (cont’d)

2. Lack of Knowledge Among Building Officials on How to Interpret a CCMC Listing/Report:Although Ministry Officials are somewhat aware of how listings and reports are constructed, the

t j it th t th t b bl i t f h b ildi ffi i l dvast majority agree that there seems to be a problem in terms of how building officials read aCCMC report and/or listing. As a result of this lack of knowledge there tends to be a huge lack ofunderstanding when it comes to what a CCMC number entails specifically in terms of: Listings: many mention that building officials tend to assume that a listing means that the product complies

with NBC. Indeed, many do not understand the nuance between an evaluation to standard and a complianceto code.

Difference between provincial and NBC: it is also assumed that most building officials are unaware that a CCMC report does not provide a specific recommendation when a province imposes limitations of use. They are unaware that CCMC simply provides the relevant data to be examined by the building official.

LEARNING

If this lack of knowledge among building officials proves to be true, the whole process of approving products, especially innovative ones, isexposed to a high level of liability as the building officials are not necessarily using the CCMC reports as they should be – thus unwittinglyexposing themselves to the wrongful use of materials.

11

Main Learnings

Aspects Mentioned as Possible Improvements (in order of importance)

3. Lengthy Process in the Report Service for Innovative Products. Although the length of time it takesto assess an innovative product does not directly affect Ministry Officials, many claim to hear negativefeedback from manufacturers.feedback from manufacturers.

It remains important to be rigorous: Many Ministry Officials are of two minds when it comes to thetiming aspect as they feel that it is important to evaluate the products rigorously – which in turncan be a time-consuming process.

“It’s a necessary evil - if you want thorough reports it’s going to take time.”

Arguments in favour of speeding up the process: Several voice the following arguments in favour ofspeeding up the process: CCMC is the gatekeeper when it comes to new and innovative products and has the responsibility to approve the

products in a timely manner as they are sometimes being used on the market before report numbers are issued.

In a context in which innovative products are more and more prevalent it is important to ensure a relatively rapidIn a context in which innovative products are more and more prevalent, it is important to ensure a relatively rapidturnaround time when it comes to the reports – otherwise there is a chance that they will be outdated before theyeven enter the market.

“A 3 to 5 year process for the introduction of a new product in this context results in it being almost too late to roll out a new product.”

Suggestions for speeding up the process: Considering that they do not fully understand the process,the vast majority do not have any suggestions on how to speed it upthe vast majority do not have any suggestions on how to speed it up. A few mention that the process leading up to entering into a contract with CCMC seems to be the most time-

consuming and an area that should be examined in terms of how it could be optimized.

One Ministry Official suggests that the reports may provide too much information – by maintaining the focus on aparticular use, the reports could be turned around faster and be clearer in terms of the intended use of theproduct

12

product.

Better mutual recognition of standards and prior evaluations in other jurisdictions: Several mention thatCCMC should further consider recognizing certain evaluations done in other countries. The perception being thatCCMC is always starting from scratch when it could be benefiting from evaluations done elsewhere.

Main Learnings

Aspects mentioned as Possible Improvements (in order of importance)

4. Many Re-evaluations are Not Up-to-date: Several mention that there seems to be a backlog in termsof the re-evaluations. Two cases in point are referred to: When the re-evaluation date is past due and there is no mention whether it is being re-evaluated or not.

There are several listings/reports that mention that the re-evaluation is in process with no further information.

In both cases, Ministry Officials are somewhat confused as to the status of the product. If a product hasnot been reviewed, a few wonder whether there is a chance that it is still being used and shouldn’t be.That having been said most simply assume that if the product has not changed substantially since theThat having been said, most simply assume that if the product has not changed substantially since thelast evaluation, it still conforms to regulation.

“It seems that most listings/reports are up for review or way past their date of renewal. It makes me question whether or not the product still conforms.”

13

Main Learnings

Several options regarding listings were examined with each of the respondents.

Of note is that many of the respondents are unaware of the subtleties that surround each of thevarious options. Indeed, none of the four options are universally acclaimed by all respondents asthere are several issues that are brought up regarding each option.there are several issues that are brought up regarding each option.

Although this is a current way of proceeding

LIST TO ANY RECOGNIZED STANDARD LIST TO STANDARDS REFERENCED IN THE CODE

Although most feel that this would limit confusion withAlthough this is a current way of proceeding,many feel that building officials misuse thelisting.

Provides an openness to internationalstandards that may be ahead of Canadian

Although most feel that this would limit confusion withrespect to the use of materials, the issue of many perceivingthe listing to imply compliance to the code remains.

Provides a more strict definition of the codes being usedand limits those used in Canada.

standards.

Evaluating to standard is oftenmisunderstood to mean complying to code.As such, this option opens CCMC and thebuilding officials to the misuse of materials.

Provides a listing that is Canadian and as such protects usfrom eventually simply using foreign standards.

It remains that a listing is often misunderstood to meancomplying to code. As such, this option still leaves CCMCand the building officials open to the misuse of materialsbuilding officials to the misuse of materials. and the building officials open to the misuse of materials.

The trend is towards the internationalization of standards,and this option seems counterintuitive to many as it confinesto Canadian standards only.

14

Main Learnings

LIST TO THE LATEST VERSION OF THE STANDARD

LIST TO CODE REFERENCED STANDARDS AND SHOW COMPLETE COMPLIANCE TO CODESTANDARD

This option would eliminate the risk of building officialsmisusing the listings. However, many voice concerns that theCCMC does not have the resources to carry out such anoption in a timely and cost effective manner.

COMPLETE COMPLIANCE TO CODE

Although many are open to this idea, there areconcerns that building officials will perceive apotential conflict with the code if an alternativeversion is used. p y

Would provide a service that most building officials assumealready exists.

Many question if CCMC has the resources to offer thisservice in a timely manner.

Standards change quickly in today’s marketand this option would ensure that listings aretaking this reality into consideration.

Some building officials may not accept alisting that does not conform to their currentcode.

In rare cases, the latest version of a standardmay not be accepted into the code.

15

Main Learnings

The quasi totality feel that providing a partially updated re-evaluation report will be too litigious. Too much liability: Most agree that a full report is necessary to gain a clear understanding of when to use

th d t t A ti l t ld l t l d d t iti f i i t

Little Interest in Partially Updated Re-evaluations

the product or not. A partial report would leave too many loose ends and opportunities for inappropriate use of the product.

It would add to the current confusion: In a context where several building officials are already confused as to how to interpret the information in a report, having a partial report would just add to the confusion.

“It would likely add to frustrations regarding CCMC and how to interpret their report.”

In light of these views, many feel that a partially updated re-evaluation would be of little use tobuilding officials.

Many suggest a progress report: However, many suggest that CCMC inform them of where the product is in the re-evaluation process. The mention “re-evaluation in process” does not indicate

if th d t h h d i th l t l ti …if the product has changed since the last evaluation;

… where they are in the process (re-testing, evaluation of the data, report is being written, etc.).

“It’s a big black box and we have no idea what is going on… I don’t think building officials really know what to do when they are told the product is being re-evaluated because they don’t seem to know what it means.”

A few suggest a “black list” of products that do not obtain re-evaluation: A few mention the gg pusefulness of a list of products that do not obtain re-evaluation, which would be circulated to Ministry and building officials.

LEARNING

16

It is abundantly clear that the vast majority do not seem to be aware of what “re‐evaluation” refers to and why it isrequired. CCMC should consider finding tools to better inform potential users of the “raison d’être” of the re‐evaluation process and how it is done.

Main Learnings

Awareness of the change: The vast majority of Ministry Officials are aware that the report format

The Revamped 1995 Report is Much Appreciated. However, There are a Few Suggestions for Improvement.

was revamped in 1995.

Appreciation of the changes: The changes are perceived positively and most feel that therevamped report is now easier to read and consult.

A few suggestions for improvement: An executive summary: Several feel there is a need for a summary, which could be included in the report.

They feel that it is difficult to decipher in what cases the product is approved for use vs. the cases it needs to be examined further by the building official. Many feel there should be a front page to each report mentioning:

For what use it is approved.

If there are provincial/territorial exceptions (if there are, simply reference in the document).

Better visual representation: A couple of respondents feel that the reports should have more visual representation to make it easier to understand the context of use.

LEARNINGLEARNING

The most often requested change to the report is that of a summary at the beginning of the document. This is a strongindication that there seems to be some confusion when it comes to rapidly finding the required information in thereport.

17

Main Learnings

Most Prefer a 3-Year Re-evaluation Cycle but Question CCMC’s Capacity to Achieve it.

The majority feel that a 3-year cycle is ideal especially in the context of a market that is constantlyevolving.

“Product development is fast paced and we need to be able to keep up with the market.”

However, several remark that CCMC already has difficulties re-evaluating the products on a threeyear cycle and wonder if the proposed 5-year cycle might give CCMC the opportunity to “catchup” on the back log.

CCMC Should Not Compete With the Private Sector.

In the same vein, several mention that the re-evaluation must be cyclical and not periodical (at the3- or 5-year mark all must be re-evaluated) as they are acutely conscious that CCMC does not havethe resources to re-evaluate all the products in one year.

The vast majority prefer that CCMC simply remain within its current scope and not compete with the private sector.

In fact, the vast majority feel that it is not the role of a government agency to compete with the private sector

Areas That Should be Considered In the Future By CCMC.

Respondents were asked what areas CCMC might need to consider in their evaluation process in the future. The following were the elements mentioned most often:

private sector.

18

Energy efficiency

Water conservation (grey water)

Water consumption

Length of product life (especially for northern communities)

Main Learnings

The Vast Majority Seem Relatively Satisfied With the Level of Consultation for Innovative Products.

Consultation at the beginning is essential: The vast majority appreciate that the CCMCcommunicates with them and asks for their comments on what to include in the technicalguidelines of innovative products. They agree that CCMC has shown a willingness to give theprovinces/territories an opportunity to comment and has demonstrated an openness todiscuss issues with the provinces/territories. This having been said many of the smaller provinces and territories mention that although they appreciated This having been said, many of the smaller provinces and territories mention that although they appreciated

this effort from CCMC, they do not have the resources or the time to provide input. This however does notreflect badly on CCMC, it is simply a reality that they live with. In any case, these provinces/territories tend tovoice the opinion that other larger provinces are more apt to point out issues and feel confident that, at theend of the day, they will profit from the comments made by their counterparts.

Consultation during the process is requested by only a few: The vast majority feel that once Consultation during the process is requested by only a few: The vast majority feel that oncethe process has begun, they do not need to provide input. However, several mention that they would like an update in terms of where the innovative product is in the

process (e.g.: have the technical guidelines been established, has testing started, has CCMC begun toevaluate the tested data, has CCMC begun to write the report).

19

Main Learnings

NRC Seems to be A Logical Place For CCMC to be.

All agree that CCMC is well placed under the umbrella of the NRC. Indeed, none of them had really questioned the soundness of its current placement.ea y quest o ed t e sou d ess o ts cu e t p ace e t

Most comment that NRC is intimately involved with the industry, therefore it is a good fit with CCMC.

One respondent mentioned the interest in having regional CCMC offices (e.g.: West / Ontario / Quebec / Atlantic / Territories). The hope is that by having regional offices, the CCMC would offer: Better communication / dialogue with building officers;

Faster service to the manufacturers;

Better knowledge of regional issues.

20

Main Learnings

CCCME is Relatively Unknown.

Unless the person being spoken to was on the CCCME committee or knew someone on it, they are unaware of its existence.a e u a a e o ts e ste ce

The vast majority know the CCMC and are completely unaware of its CCCME counterpart.

LEARNING

CCMC should consider communicating more information regarding the CCCME if it is deemed useful in helpingMinistry/building officials better understand the mission of the CCMC.

21

Appendix I – Research Tools

Appendix I – Research ToolsAppendix I Research ToolsPlease click on the icon to view the desired research tool.

Discussion Guide

Discussin Guide

22



Brio Conseils inc.

Page: 14 / 21

APPENDIX D: Web Survey with Building Officials and Manufacturers, Ad hoc Research

CCMC Business ReviewCCMC Business ReviewWeb Survey with Building Officials and

ManufacturersNovember 2009

Table of Contents

Objectives and Methodology 3

Executive Summary 7

Findings – Building Officials Survey 14Participant Profile 15Participant Profile 15

Responsibilities and Practices of Building Officials 17

Level of Comfort for Approving Products 20

Sources of Information/Factors of Influence For Establishing Compliance to the NBC 23

Knowledge of CCMC 30

Perceptions Regarding CCMC Listings 33

Perceptions Regarding CCMC Reports 40

Additional Questions 49

Findings Manufacturers 58 Findings – Manufacturers 58Participant Profile 59

Practices of Manufacturers 62

Knowledge of CCMC 66

Evaluation of Listings and gReports for Innovative Products 73

Perceived Timeliness of Reports 80

Issues Related to the Reports 83

Re-Evaluation Process 91

Additional Services 94

2

Additional Services 94

Satisfaction With Contact With CCMC 97

Appendices

Objectives

In the context of the CCMC Business Review one of the objectives is to consult stakeholders in terms

Objectives

In the context of the CCMC Business Review, one of the objectives is to consult stakeholders in terms of their perceptions of CCMC.

The Web survey allowed CCMC to consult several building officials across Canada as well as manufacturers.

The main objectives of these surveys are to:

Validate certain learnings from the interviews with provincial officials.

Gather opinions regarding various issues that CCMC has identified and produce quantitative data.

4

Methodology

Canadian Building Officials

Manufacturers who currently have a CCMC Listing or Report number.

Target Population

y g p

Data collection was carried out online with

Data Collection

a questionnaire for building officials between August 19 and October 21, 2009,

a separate questionnaire for manufacturers between September 15 and October 10, 2009.

Building officials were contacted through various associations across Canada and were invited to participate in the survey.

Manufacturers were invited directly through e-mail to participate in the survey.

In all,

Sample Size and Accuracy of Results

229 building officials and

54 manufacturers responded to their respective surveys.

The maximum margins of error at a confidence level of 95% for the total sample and for sub-samples are detailed in below.

Building officials: ± 6.48%

M f t 11 75%

5

Manufacturers: ± 11.75%,

Methodology

Methodological note:Please note that the term “innovative product” referred to products for which no current standard exists.

6

Executive SummaryExecutive Summary

Executive Summary – Main Findings

ParticipationThe Building Official survey was filled out by 229 participants.

The Manufacturer Web survey was filled out by 54 participants.

High level of comfort approving standardized products: Building officials are far more comfortable approving standardized products (average of 9.3 out of 10) than non-standardized products (rated at only 3.2 out of 10).

Level of involvement in approving innovative products is mixed: A quarter (25%) of building officials either “do not approve innovative products for construction projects” or “are not at all involved in the decision to approve an

Building Official Profile Information

not approve innovative products for construction projects or are not at all involved in the decision to approve an innovative product”.

Many building officials admit to having a lack of expertise to review the necessary documentation when it comes to innovative products: A total of 77% agree to some extent (22% completely agree and 55% somewhat agree) with the statement that “in many cases I feel that I do not have the technical expertise/support to review all the documentation necessary to approve innovative construction products/systems for which no standards exist”.the documentation necessary to approve innovative construction products/systems for which no standards exist .

Many building officials admit to lacking time to properly review documentation when it comes to innovative products: A total of 77% agree to some extent (30% completely agree and 44% somewhat agree) with the statement “I would like to review all the documentation necessary to approve an innovative construction product but I have very little time to do so”.

8

Executive Summary – Main Findings (cont’d)

CCMC is a Likely Source of Information to Base Acceptance of a Product for Which Standards Exist.

CCMC as a Source of Information for Building Officials

For Products For Which Standards Exist:

The likelihood of considering a CCMC Listing: A CCMC Listing was rated the highest of 5 sources of information (CCMC Listing: a label referencing a recognized industry standard, certification programs, manufacturer label and technical data sheet, a laboratory report) when it comes to products for which standards exist. In fact, 79% claim to be very likely to approve a product based on a CCMC Listing.

Proportion of standard products for which a CCMC Report is taken into consideration: When it comes to the approval of a product for which a standard exists, building officials claim that they took a CCMC Listing into consideration, on average, for 68% of all the products they were faced with.

The usefulness of a CCMC Listing:

The quasi-totality of building officials agree that CCMC Listings are useful (62% very useful and 35% somewhat useful).

As for manufacturers, they tend to find CCMC lists less useful than their building official counterparts who consult them more often (34% find listings are very useful and 50% find the somewhat useful).

9


For Innovative Products:

The likelihood of considering a CCMC Report: Similarly to the products for which standards exist, a CCMC evaluation report far outranks the other available sources of information when it comes to a building official approving an innovative product. A total of 77% are very likely to consider CCMC as a source of information vs only 40% who would be very likely to consider an alternative solution submission by ainformation vs. only 40% who would be very likely to consider an alternative solution submission by a proponent as per the objective-based code.

The importance of a CCMC Report: With an average importance rating of 9.2 out of 10, CCMC Reports were rated the highest among 5 sources of information (CCMC Report: sign off/seal of approval from a professional, analysis of the risk to health and safety, evaluation of the potential liability to the municipality and analysis of technical performance) in terms of importance when it comes to a product for whichand analysis of technical performance) in terms of importance when it comes to a product for which standards do not exist.

Proportion of innovative products for which a CCMC Report is taken into consideration: When it comes to the approval of innovative products, building officials claim that they took a CCMC Report into consideration, on average, for 67% of all the products they were faced with.

The usefulness of CCMC Reports: The usefulness of CCMC Reports:

The quasi-totality of building officials agree that the CCMC Reports prove to be useful (66% very useful and 32% somewhat useful). This proportion is very similar to the perceptions regarding listings (see page 40).

As for manufacturers, they tend to find reports less useful than the building officials; 48% find listings are very useful and 48% find them somewhat useful This is not surprising considering that buildingare very useful and 48% find them somewhat useful. This is not surprising considering that building officials are more likely to use the report as a tool.

10


A Misconception Regarding the Difference Between a Listing and a Report: Building officials: Almost half (44%) admit to not knowing the distinction between both services.

Manufacturers: 44% admit to not knowing the distinction between both services

A Need For Communication From CCMC

Manufacturers: 44% admit to not knowing the distinction between both services.

A Misconception Regarding the Listings:The scope of the listing seems misunderstood:

Many building officials assume that a CCMC Listing implies that the product is NBC compliant (38% completely agree and 43% somewhat agree with the statement).

Unaware that a listing does not reference an NBC Standard:

More than half of building officials 54% admit to not being aware that the listings do not necessarily reference standards in the NBC (43% No and 14% Do not know).

42% of manufacturers were also unaware.

Unaware that a listing does not evaluate any additional requirements imposed by the NBC Unaware that a listing does not evaluate any additional requirements imposed by the NBC

67% of building officials do not know the CCMC does not evaluate any additional requirements imposed by the National or Provincial Building Code.

48% of manufacturers were also unaware.

Unaware of the preface: More than half of building officials are unaware of the existence of a preface (51%). Among those who are aware of it only 47% claim to always read it.

Unaware of the review of installation manuals and quality control procedures: Less than half of building officials are aware of the review of installation manuals (42%) or of the review of the quality control procedures (40%).

11


Awareness of changes to the 2005 report format is low: Manufacturers seems to be more aware than building officials of the changes.

Building officials: Although only 23% of building officials are actually aware of the format change, those whof it t f l th t th 2005 t f t i i t th 1995 f t (94%

Issues Surrounding the Report Changes

are aware of it seem to feel that the 2005 report format is an improvement over the 1995 format (94% agreecompletely or somewhat).

Manufacturers: 44% of manufacturers are actually aware of the format change, those who are aware of it seemto feel that the 2005 report format is an improvement over the 1995 format (100% agree completely orsomewhat).

H th t i f i t id i th t l 20% l t l However, there seems to remain some room for improvement considering that only 20% completely agree with the statement.

Pertinence of the optional information at the end of the report:

A total of 80% of respondents find the optional information at the end of the report useful, although only 27% find it very useful.

The timeliness of the report is more of an issue with manufacturers than with building officials.

Building Officials: 51% claim to be very or somewhat satisfied with the timeliness of the reports

Satisfaction with the Timeliness of the Report

Building Officials: 51% claim to be very or somewhat satisfied with the timeliness of the reports.

Manufacturers who have paid for the CCMC Report service are less satisfied with only 35% who claim to be very or somewhat satisfied.

12


The idea of a partial report is of interest, especially for the manufacturers.

Building Officials: 72% feel that a “partial” report during the re-evaluation process would be “very” or “somewhat” useful

Usefulness of Partial Report when a Re-evaluation is Being Held-up

somewhat useful.

Manufacturer: 92% claim that a “partial” report would be “very” or “somewhat” useful.

Usefulness of Information Regarding the Process of the Evaluation of a Report

The idea of providing an update regarding the progress of a specific project is well received.

Building Officials: 84% feel that this “progress report” would be “very” or “somewhat” useful.

The manufacturers seem to think that rendering the progress of their file open to themselves or to the buildingofficial could be useful (91% and 78% respectively), however rendering the information available via a registry seemsto garner less enthusiasm (57%).to garner less enthusiasm (57%).

Building officials: While 93% of building officials are satisfied (very or somewhat) with the current 3 year cycle, only

Changing the 3 Year Re-evaluation Cycle to a 5 Year Cycle

57% would completely or somewhat agree with a 5 year cycle.

The main concern revolving around the 5 year cycle is that the information in the reports would be outdated considering the rate at which technology changes.

Manufacturers: are more enthusiastic about the 5 year proposition, 87% completely or somewhat agree with the proposal.

13

During the focus group sessions, several manufacturers and building officials make the point that the actual re-evaluation cycle is not a central point since, in a constantly evolving market, the chances are that the manufacturer will change their quality control process well within the 3 or 5 year period.


The value added of CCMC offering on-site visits is questionable

63% of building officials seem to disagree with the fact that CCMC does not visit construction sites.

When asked when CCMC should conduct on-site visits, 59% would prefer it be done at both the initial

On-site Visits by CCMC

evaluation and at re-evaluation.

This having been said, building officials seem to have a high confidence level in current CCMC Reports/Listings. As such, the added value of on-site visits is questionable.

Expansion of Services

Building Officials and Manufacturers agree that CCMC could extend its services to include:

Energy efficiency (84% building officials and 67% manufacturers);

Water Conservation (75% building officials and 50% manufacturers);

Xxx Green (69% building officials and 65% manufacturers), and;

p

Xxx Green (69% building officials and 65% manufacturers), and;

Sustainability / Conservation (69% building officials and 61% manufacturers).

M f t t

Satisfaction with Contact with CCMC

Means of contact Almost half (46%) of the building officials have contacted CCMC through the website in the past year.

Manufacturers tend to communicate more so through email (87%) or the phone (74%).

Satisfaction with contact Building officials: Of those who did contact CCMC 97% are very or somewhat satisfied with the support/service they received.

M f t M f t l ti fi d ith th t/ i i d f CCMC I d d l t t (24%) f

14

Manufacturers: Manufacturers are less satisfied with the support/service received from CCMC. Indeed, almost a quarter (24%) of manufacturers claim to be not very or not at all satisfied.

Findings – Building Officials SurveyFindings Building Officials Survey

BUILDING OFFICIALS

Participant ProfileParticipant Profile

Participant Profile

Training Backgroundn=229

BUILDING OFFICIALS

Training Background

The vast majority of respondents are technologists or have a trade background.

Of note that only 9% have an engineering background and 6% have a background as an architect.

A1. What is your training background? *Technologist 45% Trade back-ground 32% Building off icial 18% Contractor 14% Designer 9% Engineer 9% Building inspector 8%

n=229

Full-time vs. Part-time

The quasi-totality (93%) of respondents work full-time.

Responsibilities

Architect 6% Safety codes officer 1% Manager 1% Other 4%

A2a In what Province/Territory do you work as a building official?TOTAL - West 45%

Alberta 11% Responsibilities

The two main responsibilities of respondents are (of note that responsibilities are not mutually exclusive):

Field inspector (76%), and;

Plan analyst who is involved in issuing the building permit (70%).

British Columbia 29% Manitoba 2% Saskatchew an 3%

TOTAL - Central 26% Ontario 25% Quebec 1%

TOTAL - East 25% New Brunsw ick 11% New foundland - Labrador 1%

Regional Representation

The majority (46%) of respondents are from Western provinces.

26% are from Central provinces.

4% are from Yukon, Northern Territories, Nunavut.

Nova Scotia 13% TOTAL - North 4%

Yukon 2% Northw est Territory / Nunavut 3%

A2b Do you work as a building official…Full time position 93% Part time position 7%

, ,

Despite several efforts to get Quebec building officials to participate, only1% of respondents are from Quebec. As a result, alternativemethodological approaches have been developed to obtain feedbackfrom the province of Quebec.

A2c What are your responsibilities as a building official?*The field inspector 76% Plan analyst w ho is involved in issuing the building permit 70% Fire Inspector 13% Responsible for building maintenance 8% Chief building off icial 7% Manager / Managerial duties 4% Regulation policy developmentand implementation 3% Oth 19%

17

Base: All respondents

*Note: The sum of the proportions presented for this question may exceed 100% as respondents could provide more than one answer

Other 19%

BUILDING OFFICIALS

Responsibilities and Practices of Building OfficialsResponsibilities and Practices of Building Officials

Level of Involvement in Approval Decision of Innovative Products

BUILDING OFFICIALS

More than 6 out of 10 building officials claim to be relatively involved in the decision making process for innovative products in a construction project (35%

D1 What is your level of involvement in taking the decisions to approve an innovative product in a construction project?

n= 229

35%

28%

innovative products in a construction project (35% very involved and 28% somewhat involved).

This having been said, a quarter (25%) either: Do not approve innovative products for construction projects

(17%);

Or are not at all involved in the decision to approve an

62%

13%

17%

innovative product (8%).

21%

8%

Very involved Somewhat Not very Not at all Don't approveVery involved Somewhatinvolved

Not veryinvolved

Not at allinvolved

Don t approveinnovative

products forconstruction

projects

19Base: All respondents

Capacity to Review Documentation for Product Approval

BUILDING OFFICIALS

D5 Please indicate to what degree you agree with the following statements.

I always review all the documentation necessary before

Completely agree Somewhat agree

n= 173

TOTAL % of Agree(Completely+somewhat)

58%

22%

33%

55%

approving the use of an innovative construction products /systems for which no standards exist.

In many cases I feel that I do not have the technicalexpertise / support to review all the documentation

necessary to approve innovative construction products /t f hi h t d d i t

91%

77%

30% 44%

systems for which no standards exist.

I would like to review all the documentation necessary toapprove an innovative construction product but I have very

little time to do so.74%

Although almost 6 out of 10 building officials (58%) claim to review all the necessary documentation before approving the Although almost 6 out of 10 building officials (58%) claim to review all the necessary documentation before approving the use of an innovative construction product, 5 out of 10 (52%) completely agree with the following statements: They don’t have the time to review all the documentation (30% completely agree), or;

They don’t have the technical expertise to do so (22%).

LEARNINGLEARNING

There seems to be a relatively high number of building officials who admit to not necessarily having the time or the expertiseto properly review all the documentation necessary to approve an innovative construction product. As such, some buildingofficials are putting themselves at risk in terms of liability.

20Base: Respondents involved in the approval decision for innovative products

BUILDING OFFICIALS

Level of Comfort for Approving ProductsLevel of Comfort for Approving Products

Overall Level of Comfort for Approving ProductsBUILDING OFFICIALS

B1 Please evaluate your level of comfort when it comes to approving one of the following products for a construction project.

Use a scale of 1 to 10 where 1 means you do not feel at all comfortable approving them for use and 10 means you feel very comfortable in approving them for use in a construction project.

n= 229

9% 23% 61%standardized products

8 9 10

n= 229

Average

9 392%

% Top 3 boxes

9% 23% 61%

3%1%1%

standardized products

non-standardizedproducts

9.3

3.2


92%

6%


Building officials are far more comfortable approving standardized products than non-standardized products.

Indeed, the average level of comfort attributed to non-standardized products (on a scale of 1 to 10) is far less(3.2/10) than for standardized products (9.3).

22

Level of Comfort for Approving Products by Country or Origin

BUILDING OFFICIALS

B2 Please indicate if you would be comfortable or not in approving the use of products from the countries for a construction project.

Use the same scale of 1 to 10 where 1 means you do not feel at all comfortable approving them for use and 10 means you feel very comfortable in approving them for use in a construction project.

n= 229

19% 21% 21%North America

8 9 10 Average

7.4

% Top 3 boxes

4%

North American products far outperformother countries in terms of the level ofcomfort that building officials have inapproving them for use in a constructionproject (average of 7.4/10).

14%

11%

8%

8%

7%

4%

Europe

New Zealand and/orAustralia

5.4

5.1

29%

23%

p j ( g )

The two countries that inspire the least amount of comfort are South America (average of 3.1/10) and Asia (average of 2.9/10)

2%0%2%

Australia

South America 3.260%

2%0%1%Asia 2.9


4%

23

BUILDING OFFICIALS

Sources of Information/Factors of Influence For Establishing Compliance to the NBC

Most Popular Standards Referred ToBUILDING OFFICIALS

57%ANSI

C1 Excluding standards mentioned in the NBC which of the following standards do you notice construction products refer to for compliance?*

n= 229

Most Popular Standards Referred To A quarter of building officials (25%) claim not to use aproduct unless it shows compliance to the NationalB ildi C d (NBC) Thi fl t th l k f

47%

25%

15%

ISO

EPA

HVI

Building Code (NBC). This reflects the lack ofcomfort that many seem to have in approvingnon-standardized products.

The two standards that are most often mentioned are(these mentions are not mutually exclusive): ANSI (57%)15%

14%

8%

HVI

APA

AAMA

ANSI (57%)

ISO (47%)

6%

3%

2%

TECO

CSA

ULC

ASTM 1%NFPA 1%OBC 1%CCMC 1%

Other Standards Referred To

2%

25%

9%

CGSB

I do not use a product that does not showcompliance to NBC

I cannot remember the names of thet d d

CCMC 1%CEN 1%ASSE 1%

Note: For a complete listing of other standards mentioned, please consult the Detailed Statistical Tables in Appendix I

25

standards.


*Note: The sum of the proportions presented in this table may exceed 100% as respondents could provide more than one answer

Tables in Appendix I

Sources of Information for Acceptance of Products for Which Standards Exist

BUILDING OFFICIALS

C2 When approving a product for which standards already exist in a construction project what is the likelihood that you would accept it based on the following source of information.

n= 229

Very likely Somewhat likely TOTAL % OF LIKELY(very+somewhat)

79%

50%

14%

31%

CCMC listing service

A label referencing a recognized industrystandard

93%

81%

24%

13%

9%

41%

40%

37%

Certification programs

Manufacturer label and technical data sheet

A l b t t

65%

53%

9% 37%A laboratory report45%

A CCMC Listing seems to be a reference when it comes to products for which standards exist. In fact, 79% of building officials claim to be very likely to approve a product based on a CCMC Listing.

Interestingly, a label referencing a recognized industry standard garners far less likelihood of acceptance than a CCMCListing (50% vs 79% very likely to accept)


Listing (50% vs. 79% very likely to accept).

Although a “laboratory report” and/or a “manufacturer label and a technical data sheet” should be sufficient information to base acceptance, these two sources of information garner substantially less likelihood of being accepted.

LEARNING

26

Base: All respondentsMany building officials, in face‐to‐face consultations, claim to have far more confidence in a CCMC Listing due to the factthat CCMC offers the added value of having looked at the technical data and validated if it corresponds to a recognizedindustry standard. Several claim that they do not necessarily have the technical expertise to go through with this analysis(which is corroborated by their technical background (see page 17).

Sources of Information for Acceptance of Innovative Products

BUILDING OFFICIALS

D3 When approving an innovative construction products / systems for which no standards currently exist what is the likelihood that you would consider the following sources of information when making your decision.

n= 172

Very likely Somewhat likely

Similarly to the products for which standards exist, a CCMC

l ti t f t k

TOTAL % OF LIKELY(very+somewhat)

77%

40%

22%

47%

CCMC Evaluation Report

An alternative solution submission by aproponent as per objective-based code

98%

87%

evaluation report far outranks the other available sources of information when it comes to a building official approving an innovative product (77% are very likely to consider CCMC as a

34%

17%

52%

54%

Assessment by an engineer

Approval by another body/Authority havingjurisdiction

86%

71%

likely to consider CCMC as a source of information vs. only 40% who would consider an alternative solution submission by a proponent as per the objective-based code).

21%

12%

47%

48%

Assessment by an architect

ICC-ES Report

67%

60%

LEARNING

The high proportion of “very11%

11%

39%

35%

Independent analysis of the manufacturer’sliterature

Information on the performance of the product inanother location

49%

46%

e g p opo o o e ylikely” for CCMC as compared tothe other solutions demonstratesthe high level of confidence thatmost building officials seem tohave in a CCMC evaluation report.

27

Base: Respondents involved in the approval decision for innovative products

have in a CCMC evaluation report.

Sources of Information for Acceptance of Innovative Products (cont’d)

BUILDING OFFICIALS

D4 What other sources of information do you use for acceptance of innovative construction products / systems?

n = 172 More than 7 out of 10 building officials (72%) do not I use no other sources of information 72% Internet searches / info obtained via Internet 5% Comparison w ith other codes /standards 3% Minister's orders 3% Past experiences 3%

use any other sources of information than those listed on the previous page.

Among the other sources listed are Internet searches as well as reports from other agencies such as ULC, BMEC, Warnock Hersey, Intertek, etc.ULC (material listing, reports) 2%

BMEC (authorization, reports) 2% Warnock Hersey (listings, reports) 2% Manufacturers' data 2% Objective based solutions / codes 2% Intertek 2% Know n uses in other projects w ith good performances 2% Other 22% DNK-DNA 1%

Note1: Responses with mentions of 2% and more are not presented in this table. For a complete listing of other sources referred to, please consult the Detailed Statistical Tables in Appendix I

Note2: The sum of the proportions presented in this table may exceed 100% as respondents could provide more than one answeranswer.


28

Importance of Various Factors that Influence a Decision to Approve an Innovative Product

BUILDING OFFICIALS

D2 Please rate the importance of each of the following factors in your overall decision to approve the use of an innovative construction product / system for which no standard currently exists.

Use a scale of 1 to 10, where 1 means not at all important and 10 means very important in your decision to approve use.

n= 172

6%

14%

15%

16%

65%

50%

CCMC Reports

Sign off/ seal of approval from a professional

8 9 10 Average

9.2

8 7

% of top 3 boxes

87%

80%14%

12%

11%

16%

10%

12%

50%

56%

52%

Sign off/ seal of approval from a professional

Analysis of the risk to health and safety

Evaluation of the potential liability to thei i lit

8.7

8.6

8 4

80%

77%

75%


16% 15% 35%

municipality

Analysis of technical performance

8.4

8.0

75%

66%

All f th f t t t d id d i t t i th ll d i i t th f i ti All of the factors tested are considered important in the overall decision to approve the use of an innovative construction product /system. Indeed, all of five factors score an average of 8.0/10 or higher by building officials. Of note that CCMC Reports rank the highest of all 5 factors tested (with an average of 9.2/10).

29

Use of CCMC Reports/Listings for Approval of Products

BUILDING OFFICIALS

E3a Thinking of the innovative products that you approve for projects, what proportion of innovative products do you take a CCMC reportinto consideration when deciding to approve its use?

Please provide an answer out of 100%

E3b Thinking of the products that you approve for projects, what proportion of products for which standards exist do you take a CCMC listing into consideration when deciding to approve its use?


20%

35%

n= 229

Average: 68%


n= 229

Average: 67%

18% 19%

38%

4%

19%

9%

13%

20%

7%

18%

10% 10%

None 1-25% 26-50% 51-75% 75-90% AlmostNone(0%)

1-25% 26-50% 51-75% 75-90% Almostall (90-100%)

Base E3a and E3b : Respondents who are aware of CCMC’s services

(0%) all (90-100%)

B ildi ffi i l t k CCMC R t f li ti i t id ti f l t 7 t f 10 d t th t Building officials take a CCMC Report of listing into consideration for almost 7 out of 10 products that are approved for use.

68% for innovative products, and;

67% for standard products.

30

BUILDING OFFICIALS

Knowledge of CCMCKnowledge of CCMC

Awareness of CCMC ServicesBUILDING OFFICIALS

E1 Which of the two services are you aware that CCMC offers?

n= 229

E2 Before answering this survey, were you aware of the distinction between the two services offered by CCMC (listing and reports)?

n= 229

89%

83%

Listing: A listing is issued for productsthat have demonstrated

Reports: an evaluation of innovativeconstruction products/systems which

yes56 %

no44 %

83%are not referenced in the NBC

(Reports)

Base: All respondents Base: All respondents

Although the vast majority of building officials claim to be aware of the two services offered by CCMC, almosthalf (44%) admit to not knowing the distinction between both services.

LEARNING

!

One‐on‐one interviews and focus group sessions uncovered that the terminology used (listing vs. report) tends tocontribute to this confusion as both are “listed” on the CCMC website.

There is a need for further communication on behalf of CCMC when it comes to the differentiation between a listing

32

and a report as each one has a specific “raison d’être” and, as such, should be analyzed by the building officialdifferently.

Perceptions Regarding CCMC’s ServicesBUILDING OFFICIALS


E6 Please indicate to what extent you would agree with each of the following statements.

n= 229

% of Agree

This series of statements indicatesome opportunities for CCMC toimprove: Impartiality: Although a full 66%

40% 52%

66% 30% CCMC offers an impartial third party verification

that is of value to my activities

CCMC services address my needs as abuilding official

96%

92%

Impartiality: Although a full 66%completely agree that CCMC is animpartial third party, one would expectthis proportion to be higher consideringit is a government agency.

The ease of obtaining information:only 38% completely agree with this

38% 46%It is easy to obtain the information I need from

CCMC

I rely on CCMC for my approval of innovative

85%

only 38% completely agree with thisstatement.

Almost 4 out of 10 building officials(38%) admit that they will assume thata product is NBC compliant if it has aCCMC number.

45% 39%

38% 43%

y y ppconstruction products / systems for which no

standards exist

If CCMC has issued a number to a product Iassume that the product is NBC compliant.

85%

81%

LEARNING

This is yet another indication thatCCMC needs to bettercommunicate how reports andlistings should be used.

33

Base: Respondents who are aware of CCMC’s services

BUILDING OFFICIALS

Perceptions Regarding CCMC ListingsPerceptions Regarding CCMC Listings

Listings: Usefulness of ListingsBUILDING OFFICIALS

62%Very useful

F1 How useful do you think the CCMC listings are when approving products/systems for use in your projects?

n= 214F2 Why do you find the listings to be useful?

SUBTOTAL - Quality of thin information 56% Technical data on products (limitations, uses, restrictions) 25% I f ti / P id d / f l i f ti 15%

n = 207

35%

2%

Somewhat useful

Not very useful

3%

97%TOTAL % useful

TOTAL % of Not Useful

Informative / Provides good /useful information 15% Good tool to identify a product 8% They are a trustw orthy / credible source 13% Reliability / reliable information given 7% It's a reference guide / references building codes 2%

SUBTOTAL - Reputation of CCMC 29%

1%Not at all useful

3% TOTAL % of Not Useful

F3 Why do you not find the listings to be useful?More information should be available 3

n = 7

Products listed have been evaluated / review ed by them 19% Independent / non-bias / thirdparty review s 14%

SUBTOTAL - Easy acces to information 12% Convenient / helpful resource 6% Quick to use / f ind the neededinformation 4%

More information should be available 3Listing / reports are sometimes w ithdraw n later 2Final approval of products is not up to them 1Only uses products approved byprovincial building codes 1Misleading information 1Do not know 1

The quasi-totality of building officials agree that the

Comprehensive / easy to understand 2% Accessible / easy to access 2%

Other 1% DNK 32%

CCMC Listings are useful (62% very useful and 35% somewhat useful).

When probed as to what they find useful, most responses evoked can be categorized under: the quality of information provided (56%);

35

Base F1: Respondents who have taken CCMC listing into consideration when approving a product

Base F2*: Respondents who find the CCMC listing to be Useful

Base F3*: Respondents who find the CCMC listing to be Not Useful


the reliability and trustworthiness of CCMC (29%); and

the ease of access to information (12%).

Listings: Awareness and Use of PrefaceBUILDING OFFICIALS

47%Always

F6 Were you aware of the existence of a preface for each 2005 listing?

n= 104

F7a How often do you read the preface of a listing (check one answer only)

n= 214

99%

52%

1%

Sometimes

Never

no51 %

yes49 %

n= 103

F7b How would you characterize the manner in which you read the prefaceLEARNING

Always readSometimes read All

respondents

16%

37%

47%

56 %

11 %

33 %

I quickly skim through the preface

I read the preface thoroughly

Check preface to ensure that use ofthe product is compliant

There needs to be additional communication when itcomes to the existence of the preface and itsimportance.

Indeed, only 19% actually are 1) aware of the preface,2) always read it and 3) read through it thoroughly or at

37%

23%

40%33 %the product is compliant2) always read it and 3) read through it thoroughly or at

least check it to ensure that the use of the product iscompliant xxx(49% aware X 47% always read it X 84%).

36

Base : F6. Respondents who have taken CCMC listing into consideration when approving a product

F7a. Respondents who have taken CCMC listing into consideration when approving a product

F7b. Respondents who read the preface always or sometimes.

Listing: Awareness of CCMC Listing Relative to NBC

BUILDING OFFICIALS

F8a The CCMC listings show compliance to a recognized standard (i.e. put together by a recognized standard writing body through a consensus based process). These standards are not necessarily referenced in the NBC. Were you aware of this?

LEARNING

The answers to these two questionsclearly indicate that CCMC needs to

n= 214

Yes46 %

No54 %

better communicate the basis of alisting to the building officials. In bothcases, more than half of respondentsadmit to not knowing that:

The listings do not necessarily

F8b The CCMC listings show compliance to a recognized standard. However, when the national or provincial Building Code impose additional requirements to the industry standard, CCMC does not evaluate to these requirements. We provide guidance to the location of reference in the NBC 2005. ie A CCMC listing to a standard is only that and not compliance to the NBC.

The listings do not necessarilyreference standards in the NBC (54%).

The CCMC does not evaluate anyadditional requirements imposed bythe National or Provincial Building

Base : Respondents who have taken CCMC listing into consideration when approving a product

in the NBC 2005. ie A CCMC listing to a standard is only that and not compliance to the NBC.

Were you aware of this? Code(57%).

However, those who are aware tend tobe aware of both statements. A totalof 79% of those who answered yes toth fi t ti l d t

Yes43 %

No57 %

n= 214

the first question also answered yes tothe second question.

37

Base : Respondents who have taken CCMC listing into consideration when approving a product

Listings: Suggestions for Improvements BUILDING OFFICIALS

F9 How could CCMC improve their listings for products / systems in terms of either format or content?

The listing is f ine as it is 40% Cl / i t d t d i f ti 6%

n = 214 A total of 40% of building officials claim that the listings need no improvement. While another 47% cannot think of any

ti f i tClearer / easier to understand information 6% More detailed / technical info on products / tests 3% Updated information / Listings should be up to date 2% Identify materials tested for NBC 2% Specify w hat is not tested but complies w ith standards 1% Specify w hether a standard is referenced or not 1% List products that are in the process of approval 1% Specify the limits of the listing up front 1%

suggestions for improvement.

Although there are several suggestions for improvement, no single improvement garners a large proportion of respondents’ spontaneous mentions.

p y g pProducts should not be available until suitably tested 1% Make available documents / reports pertaining to lists 1% List other recognized certif ications and listings 1% Standardize the format 1% Specify w hat geographical regions it w as approved in 1% Do not know 47%

*Note: The sum of the proportions presented in this table p p pmay exceed 100% as respondents could provide more than one answer

38 Base : Respondents who have taken CCMC listing into consideration when approving a product

BUILDING OFFICIALS

Perceptions Regarding CCMC ReportsPerceptions Regarding CCMC Reports

Reports: Usefulness of CCMC ReportsBUILDING OFFICIALS

66%Very useful

G1 How useful do you think the CCMC reports are when approving innovative building products /systems?

n= 214G2 Why do you find the reports to be useful?

Detailed / technical info (limitations, uses, restrictions) 35% Informative / Provides good /quality / important info 20% Independent / non bias / thirdparty review s 11%

n = 214

32%

2%

Somewhat useful

Not very useful

3%

97%TOTAL % of Useful

% of Not Useful

Independent / non-bias / thirdparty review s 11% Provides all the info requiredto make a decision 9% Reliable / credible information given 9% Availability / Info is easilyavailable 3% Information is clear / easy tounderstand 3% Additional info by a recognized industry standard 2% Other 4% DNK 32%

1%Not at all useful

3% % of Not Useful

G3 Why do you not find the reports to be useful?

n = 6

Note: Responses with mentions of 2% and more are not presented in this table. For a complete listing of other sources referred to, please consult the Detailed Statistical Tables in Appendix I

*Note: The sum of the proportions presented in this table may exceed 100% as respondents could provide more than one answerNot used in my small rural tow nship 1

Listing / reports are sometimes w ithdraw n later 1Do not know 4

more than one answer

The quasi-totality of building officials agree that the CCMC Reports prove to be useful (66% very useful and 32% somewhat useful). This proportion is very similar to the perceptions regarding listings (see page 35)35).

When probed as to what they find useful, most responses relate to the quality of the information provided.

40

Base G1: Respondents who have taken CCMC report into consideration when approving products

Base G2: Respondents who find the CCMC reports to be Useful

Base G3: Respondents who do not find the CCMC reports to be Useful

Reports: Use of CCMC ReportsBUILDING OFFICIALS

G4 When consulting a CCMC report for an innovative construction product/system what is the likelihood that you would do the following:

n= 220

Very likely Somewhat likely TOTAL % of Likely

73%

73%

24%

21%

I read the report to ensure that theproduct/system meets the conditions and

limitations stated.

Verify that the product / system in question has avalid CCMC number

(very + somewhat)

97%

94%

49%

20%

38%

30%

valid CCMC number

Read the full report thoroughly

Skim through the report

87%

49%g p 49%

Although most building officials consider it very likely to read the report to ensure that the product system meets the conditions and limitations stated (73%) and that it has a valid CCMC number (73%), it is slightly disconcerting to note that it i l “ h t lik l ” th t th t t i th t (24% d 21% ti l )it is only “somewhat likely” that they entertain these acts (24% and 21% respectively).

In addition, only about half (49%) are very likely to read through the full report thoroughly.

41Base : Respondents who have taken CCMC report into consideration when approving products

Report: Perceptions of Report Changes BUILDING OFFICIALS

G5 Were you aware of these changes? G6 To what extent do you agree that the new 2005 report format is an improvement to the 1995 format?

In 2005, CCMC changed its report format. The previous format was in use prior to 2005.

n= 220

TOTAL % of Agree (completely+somewhat)

94%

41%

53%

n= 51Yes23 %

No77 %

6%

Completelyagree

Somewhatagree

Somewhatdisagree

77 %

Although only 23% are actually aware of the format change, those who are aware of it seem to feel that the 2005 report format is an improvement upon the 1995 format (94% agree completely or somewhat).

42

Base G5: Respondents who had received both 1995 and 2005 reports

Base G6: Respondents who were aware of the changes made in the 2005 report

Report: Evaluation of the Format of CCMCReports

BUILDING OFFICIALS

G7 Please indicate your level of agreement for the following statements with regards to the format of the CCMC reports for innovative construction products / systems.

n= 220

Completely agree Somewhat agree TOTAL % of Agree

33%

36%

63%

52%

The reports provide complete information

I can easily find the information I am looking forin the reports

Completely agree Somewhat agree TOTAL % of Agree(very+somewhat)

96%

88%

21% 65%

in the reports

The reports are current with the code? 86%

Although most building officials agree that the reports provide complete information (96%) and they can easily find the information they are looking for (88%), the relatively low proportion of respondents who “completely agree” indicate that there is room for improvement on both fronts.

The question addressing whether or not the reports are current with the code also provokes mixed reviews in that only 21% completely agree with the statement.

43

Base : Respondents who had received both 1995 and 2005 reports

Report: Perceptions Regarding the Optional Information

BUILDING OFFICIALS

G8a In the new Format, how useful do you find the optional information at the end of the report presented without any opinion on Code compliance?

n= 220

G8b How could this section be improved?

% of Useful

80% No improvements needed to this section 31%

n = 220

27%

52%

% of Not Useful

More detailed info on product limitations 2% State code compliances 1% Should have a better synopsisof overall product 1% Information on how other provinces deal w ith new products 1% More illustrations / a picture is w orth 1000 w ords 1% Improve spelling / grammar used 1% Be more relevant to today's standards 1% DNK 63%

7%

1%

13%

Very useful Somewhat Not very Not at all I never use

8%DNK 63%


useful useful useful thissection

A total of 80% of respondents find the optional information at the end of the report useful, although only 27% find it very useful. However, those who were aware of the change to the report are more likely to find the optional information “very or somewhat useful” (90%

vs. 76% who were not aware of the change).

Of note that 13% never use the section at all (only 2% among those who were aware of the change and 16% among those who were not aware).

Very few suggestions are given regarding how to improve the optional information section (31% consider that it needs no improvements and 63% choose not to answer the question).

44Base : Respondents who had received both 1995 and 2005 reports

Report: Suggestions for ImprovementBUILDING OFFICIALS

G9 How could CCMC improve their reports for innovative construction products / systems in terms of either format or content?

The report is f ine as is 44% More detailed info on specif ic/ approved uses of products 5%

n = 220 Either respondents claim the report format is fine as it

is (44%) or they simply choose not to answer the More detailed info on specif ic/ approved uses of products 5% Simpler / easier to understandinformation 3% More illustrations / a pictureis w orth 1000 w ords 1% Test / evaluate products for various uses / intended uses 1% Should receive information directly 1% Information / reports should be easier to f ind 1% Information should be better organized / categorized 1% Provide more help to inspectors 1% State code compliances 1%

( ) y p yquestion (46%).

Of note that several suggestions were brought up during the focus group sessions (see focus group session reports).

State code compliances 1% Test / evaluate more products 1% More public aw areness of CCMC reports 1% Website should provide hyperlinks to the manufacturer's site 1% More generalized over-view s ofproducts 1% DNK 46%

*Note: The sum of the proportions presented in this table may exceed 100% as respondents could provide y p pmore than one answer


Report: Satisfaction with the TimelineBUILDING OFFICIALS

13%Very satisfied

G10a What is your level of satisfaction with the time it takes CCMC to complete an evaluation and publish a CCMC Report?

n= 220

G10b What do you think CCMC could do to speed up the time it takes to complete a report?

Hire more staff 5% Charge more / have more money available for evaluations 3%

n = 192

%

38%

12%

6%

Very satisfied

Somewhat satisfied

Not very satisfied

Not at all satisfied

Charge more / have more money available for evaluations 3% Suppliers need to become more familiar w ith the process 2% Allow various qualif ied sources to submit information 1% Submit entire product lines for evaluation 1% Educate people / the public on the approval process 1% Test products at key times during the year 1% Applicants should have their products tested before hand 1% Give an estimated date for completion of review s 1% Send the information directly 1% 6%

32%

Not at all satisfied

It has no impact on my work

yOutsource w orkload to meet deadlines 1% Accommodate customers more quickly 1% Have a detailed checklist forapplicants to follow 1% Follow the same process as ICC-ES 1% Build more testing facilities 1% Release English report before it gets translated to French 1% DNK 87%

*Note: The sum of the proportions presented in this table may exceedNote: The sum of the proportions presented in this table may exceed 100% as respondents could provide more than one answer

Approximately a third of building officials (32%) feel that the time it takes to publish a CCMC Report has no impact on their work. In fact, this is far more of an issue with the manufacturers who pay a fee to CCMC than it is for those who use CCMC as a resource. This having been said, 18% claim to be very or somewhat unsatisfied with the time frame.

Suggestions: It is not surprising to find 87% unable to give specific suggestions for improvement considering that the internal CCMC process is not known

to building officials.

The suggestions that are offered tend to mirror those that were submitted by the working sessions during the CCMC International Conference in September 2009 (see report).

46

Base G10a: Respondents who had received both the 1995 and the 2005 reports

Base G10b: Respondents who were less than very satisfied with the time it takes CCMC to complete a report

Report: Usefulness of Partial Reporting

G11 I f t ti i t th t l th d ti d bli ti f t d i th l ti CCMC ld

BUILDING OFFICIALS

G11 In case of extenuating circumstances that prolong the updating and publication of a report during the re-evaluation process CCMC could provide a “partial” report –i.e a partial evaluation covering only the results of tests that have been completed and identify the elements that are pending.

To what extent would such a partial report be useful to you?

% of Useful72% Only 21% of building officials feel that a “partial” report

21%

51%

21%

n= 220

% of Not Useful

28%

during the re-evaluation process would be “very” useful.

Indeed, individual interviews and the focus group sessions confirm that most building officials would avoid using such reports due to concerns related to li bilit i21% 21%

7%

Very useful Somewhatuseful

Not veryuseful

Not at alluseful

liability issues.

G12 How useful would you find it if CCMC was producing an update to let manufacturers and building officials know at what stage in the evaluation process, a product is at?

46%

% of Useful84% Knowing at what stage the evaluation process is at is

of more interest to building officials, with almost 4 out of 10 building officials who would find it very useful

n= 220 38%

11%

% of Not Useful

17%

(38%).

During individual interviews and focus group sessions, the lack of information regarding the evaluation process (e.g.: what triggers the evaluation process, why some products have not yet been evaluated, etc.)

f ( f


11%

6%

Very useful Somewhatuseful

Not veryuseful

Not at alluseful

is a concern for many (see report on focus group sessions for more information).

BUILDING OFFICIALS

Additional QuestionsAdditional Questions

Evaluation of Re-Evaluation Time Frame

H1 C tl CCMC l t t f d t / t

BUILDING OFFICIALS

H2 If CCMC t h it l ti l t 5H1 Currently, CCMC re-evaluates reports for products / systems every three (3) years. How satisfied are you with this time frame?

n= 229

% of Satisfied93%

H2 If CCMC were to change its re-evaluation cycle to 5 years.

To what extent would you agree with such a change?

n= 22957%

% of Agree

% of Disagree

35%

58%

% of Not Satisfied16%

41%

31%

43%

g

5%2%

Verysatisfied

Somewhatsatisfied

Not verysatisfied

Not at allsatisfied

7% 16%

12%

Completely Somewhat Somewhat Completelyagree agree disagree disagree

H3 What concerns do you have about changing to a 5 year cycle?

n = 98

While 93% of building officials are satisfied (very or somewhat) with the current 3 year cycle, only 57% would be satisfied with a 5 year cycle

List w ould no longer be current / info may be outdated 39% Changes in product specs / quality w ould not be know n 36% Technology changes faster / products are constantly evolving 34% Too long 21% Too many new products 8% Would need to publish updates/ changes regularly 5%

be satisfied with a 5 year cycle.

The main concern revolving around the 5 year cycle is that the information in the reports would be outdated considering the rate at which technology changes.

49

Would need to publish updates/ changes regularly 5% 3 years seems to w ork f ine 1% Less oversight w ould reduce confidence 1% Do not know 15%


Base H1 and H2: All respondents

Base H3: Respondents who do not agree with a change to a 5 year cycle

Awareness of Listing and Report ComponentsBUILDING OFFICIALS

Are you aware that CCMC includes the following in their listings and reports for innovative construction products

H4a A review of installation manuals to determine that the installation does not conflict with NBC.

H4b A review of the quality control procedures to determine if they have control of the product.

42%yes 40%yes

Are you aware that CCMC includes the following in their listings and reports for innovative construction products…

59%no

n= 229

60%no

n= 229

Once again, there seems to be a certain lack of building official knowledge when it comes to the content of the listing reports Less than half of them are aware of the review of installation manuals (42%) or of the review of the qualityreports. Less than half of them are aware of the review of installation manuals (42%) or of the review of the quality control procedures (40%).

Do you think CCMC should continue to offer these services?

H5a A review of installation manuals to determine that the installation does not conflict with NBC

H5b Do you think CCMC should continue to offer these services?

A review of the quality control procedures to determine if they have control of the product.

93%yes

229

91%yes

229

7%no

n= 229

9%no

n= 229

Despite the low awareness of these services, the vast majority of building officials would like CCMC to continue offering them:

50Base : All respondents

review of installation manuals (93%), and;

review of the quality control procedures (91%).

Reaction to Eliminating Quality Control Reviews

H7 If CCMC t t i t th lit t l i i f f i l ti i if th i ISO 9000 i t d

BUILDING OFFICIALS

32%

H7 If CCMC were to terminate the quality control review in favour of simply mentioning if the company is ISO 9000 registered.

n= 229% of Agree

40% % of Disagree

50%

The majority of building officials seem to be in favour of the status quo regarding the quality control. Indeed, only 8% completely agree with the proposed ISO 9000 reference.

Thi h i b id t b ildi ffi i l ill t d32%

29%

21%

This having been said, most building officials will tend toward proposals that lighten their work load vs. add to their work load. Simply referring to whether or not a company is registered as ISO 9000 is additional work for the building official who must now check if the particular reference is applicable

8%10%

Completely Somewhat Somewhat Completely Do not

applicable.

agree agree disagree disagree know

51

Base : Respondents who had received both 1995 and 2005 reports

Reactions to On-site Visits

H8 C tl CCMC d t i it t ti it t h i ti H9 At hi h t ( ) d thi k CCMC h ld d t it i it t

BUILDING OFFICIALS

38%

H8 Currently CCMC does not visit construction sites to see how an innovative construction product / system is installed or is performing. To what extent do you agree with this policy?

n= 229

% of Agree

% of Disagree63%

H9 At which stage(s) do you think CCMC should conduct on-site visits to see how a product is performing?

27%At initial evaluation

n= 207

25%

38%

25%

% of Agree

35%20%

59%

At re-evaluation

At initial evaluationAND re-evaluation

10%

2%

Completelyagree

Somewhatagree

Somewhatdisagree

Completelydisagree

Do notknow

22%Only if a problem

is reported

*Note: The sum of the proportions presented in this graphic may exceed 100% as respondents could provide more than one answer

Base H8: All respondents

Base H9: Respondents who do not completely agree with CCMC not visiting construction sites to see how innovative products/systems are performing

More than 6 out of 10 (63%) building officials seem to disagree with the fact that CCMC does not visit construction sites.

When asked when CCMC should conduct on-site visits, 59% would prefer it be done at both the initial evaluation and at re-evaluation.

This having been said, building officials seem to have a high confidence level in current CCMC Reports/Listings. As such, the added value of on-site visits is questionable.

52

Additional Requirements for Products/SystemsBUILDING OFFICIALS

H10a Does your Province / Territory / Region building code impose additional requirements on innovative construction products / systems beyond the NBC for acceptance (ex.: Green, Energy efficiency, etc.)?

H10b Which ones?

n= 220 Energy eff iciency 89% Green 66% Water conservation 61%

n = 116

Yes51 %

No49 %

Water conservation 61% Sustainability/Conservation 40% Resistance to climate 2% Alternative solutions (unspecif ied) 1% Minister's ruling 1% Good Building Practice for Northern Facilities 1% Best practice handbook for government funded construction 1% DNK/DNA 1%DNK/DNA 1%


The provinces for which building officials are more likely to claim there are additional requirements are:p g y q British Columbia (71%), and;

Ontario (62%).

The provinces for which building officials are less likely to claim there are additional requirements are: Saskatchewan (0%);

Maritime provinces (31%), and;

NWT / Nunavut (17%).

53

Base H10a: All respondents

Base H10b: Respondents whose province, territory or region impose additional requirements

Evaluation of Expanded Services BUILDING OFFICIALS

84%Energy efficiency

H11 In your opinion should CCMC extend its service to include the following service?

n= 229

% of “Yes”

75%

69%

69%

Waterconservation

Green

Sustainability /69%

Conservation

The majority of all building official respondents agree that CCMC should extend its services to include the four j y g p grequirements tested (almost 7 out of 10 respondents).

The most popular service is Energy Efficiency.

54

Base : All respondents

BUILDING OFFICIALS

Contact with CCMCContact with CCMC

Means of Contacting CCMCBUILDING OFFICIALS

46%Through the Web-

site

I1 In the past year, have you contacted CCMC for assistance…

n= 229

% of “Yes”

Almost half (46%) of the building officials have contacted CCMC through the website in the past year.

Of those who did contact CCMC 97% are very or somewhat satisfied with the support/service they received.

The most common complaints regarding the service are:

23%

20%

site

By e-mail

By phone

The most common complaints regarding the service are: Slow response time;

The quality of the information provided, and;

The Website is not user/friendly.

7%Any other means

Base : All respondents

I2a How satisfied were you with the support / service you got when you contacted CCMC?

I2b Why were you not satisfied with the service / support when contacting CCMC?

n= 132

46%

52%

TOTAL % of Satisfied

Time consuming / slow responsetime 21% Incomplete answ er / information given 19% Website is hard to use / not user friendly 18% No reasons / Not dissatisf ied 11%

n = 7297%

No reasons / Not dissatisf ied 11% Hard to f ind the right peopleto contact 8% Couldn't f ind the product I w as looking for 6% Information w as unclear / not to the point 6% Conflicting info / opinions given 4% Follow up on queries / returncalls 3% Takes too long to produce andpublish a report 1% Has never called / contacted them 1%

56

3%

Verysatisfied

Somewhatsatisfied

Not verysatisfied

Has never called / contacted them 1% Long delays 1% Information is not updated / needs to list new products 1% DNK/DNA 18%


MANUFACTURERS

Findings – ManufacturersFindings Manufacturers

The reader is advised that the sample size for several questions is quite small and that results should therefore be interpreted with caution

!

MANUFACTURERS

Participant ProfileParticipant Profile

Respondent Profile

A1. In what field does your company specialize?

n = 54

A5. In what Province / Territory do you distribute your

n = 54 MANUFACTURERSy p y p

Thermal Protections and Weather Barriers 41% Roofs 30% Plastics and Composites 22% Wood (including Wood Adhesives) 11% Concrete 11% Doors and Window s 7% Finishes 7% Metals 6%

y y yproducts? All Canadian Provinces and territories 69% TOTAL - West 15% Alberta 13% British Columbia 15% Manitoba 13% Saskatchew an 13%

TOTAL - Central 24% Walls 6% Special House Construction Systems 4% Utilities 4% Joint protection 4% Insulation 4% Other 10%

A2. In which department do you work?

TOTAL Central 24% Ontario 24% Quebec 15%

TOTAL - East 17% New Brunsw ick 9% New foundland and Labrador 2% Nova Scotia 11% Prince Edw ard Island 6% TOTAL - North 4%A2. In which department do you work?

Technical Services 37% Product Development 22% Ow ner / Administration 19% Marketing/Sales 13% Operations 2% Purchaser 2% Codes and standards 2% General manager 2%

TOTAL North 4% Northw est Territories / Nunavut 2% Yukon 4%

A6. Is your primary market…Local 15% Provincial 28% National 69% International 44%General manager 2%

Research & development 2%

A3. How long have you been working in this field? TOTAL - Less than 10 years 32% 1 - 5 years 22% 6 - 10 years 9%

TOTAL - 11 to 20 years 28% 11 15 years 15%

International 44%

A7. Which of the following does your company conduct in Canada? Manufacture products/systems 72% Distribute products/systems 69% Export to USA 44% Export outside of USA and Canada 24% Install products/systems 15%11 - 15 years 15%

16 - 20 years 13% TOTAL - More than 20 years 41% 21 - 25 years 13% 26 - 30 years 15% Over 30 years 13%

A4. What size is your company?Less than 10 employees 9%

Install products/systems 15% Sell products to Canadian dealers / distributors 4% Distributes US made products to Canada 2% Sells US manufactured productsto distributors 2%

A8. Where is your head office located?Canada 46% United States 52% Outside of United States or Canada 2%

59

Base:All respondents

Less than 10 employees 9% Betw een 11 and 100 employees 33% More than 100 employees 57%

Outside of United States or Canada 2%

A5, A6, A7: The sum of the proportions presented in this graphic may exceed 100% as respondents could provide more than one answer.

The reader is advised that the sample size for several questions is quite small and that results should therefore be interpreted with caution.!

Countries of Operation

MANUFACTURERS

A9. Please indicate in which countries the following are carried out.

Research and development of new products/systems

Manufacturing of products/systemsn = 54

61%

67%

69%Canada

United States

A10. Approximately what proportion of… is done in Canada?

42%

33%

11%

70%

19%

United States

Abroad


O f t l i th t


On average, manufacturers claim that: 42% of their research and development of new products/systems is done in Canada, and;

33% of their products are manufacturer in Canada.

60The reader is advised that the sample size for several questions is quite small and that results should therefore be interpreted with caution.!

MANUFACTURERS

Practices of ManufacturersPractices of Manufacturers

Identification of Code Compliance Requirements

MANUFACTURERS

B1a In the development or production stage of a new product/system do you identify code compliance requirements?

Yes, always Yes, sometimes

77% 23%n=39

Base: Respondents whose company manufactures products and systems.

100%

The majority of manufacturers identifycode compliance requirements: In the development stage of their products

(77%) as well as,

Before deciding to distribute it (78%)

B1b Do you identify code compliance requirements for a product before deciding to distribute it?


Before deciding to distribute it (78%).

Base: Respondents whose company distributes products and systems

78% 16%n=37


95%

Base: Respondents whose company distributes products and systems.


Most Popular Organisations Used As Reference when Developing a New Product/System

MANUFACTURERS

B2 Which organisations do you refer to most often as a reference when developing a new product/system?

87%ASTM

n = 54

Most Popular Organisations Referred To ASTM is by far the most referenced

organisation (87%).

Other popular organisations are ULC

56%

52%

46%

32%

ULC

UL

CSA

ANSI

(56%), UL (52%) and CSA (46%).

32%

30%

20%

19%

ANSI

CGSB

ASHRAE

NFPACEN 6%APA 6%

Other Organisations Referred To

17%

11%

ISO

AAMA

BNQ 6%EPA 6%ICC 4%AWPA 4%ICC-ES 4%NBCC 4%TECO 2%

N t Th f th


DADE 2%Miami Dade 2%FM Global 2%National Building Code of Canada 2%CLS 2%ALSC 2%NLGA 2%

Note: The sum of the proportions presented in this graphic may exceed 100% as respondents could provide more than one answer.

63

Base: All respondentsFM 2%I do not use any standards for innovative products 2%


Organisations Used As Reference (cont’d)

MANUFACTURERS

B3a Have you ever refered to other sources or used other means to demonstrate compliance of a product/system?

n = 54

B3b Please specify what

More than 7 out of 10 manufacturers(72%) also use other sources of

liYes72%

No28%

B3b Please specify what sources or means n = 39

ICC-ES / ICC ES reports 36%Independent testing lab / accredited 3rd party lab reports 10%CCMC 8%Intertek 5%

compliance.

The most popular is ICC-ES (36%).

Only 8% spontaneously mention CCMCas a source used to demonstrate thecompliance of a product/system.

B All d t

Intertek 5%Actual testing / field testing 5%Miami Dade 5%Florida building code 5%Engineering evaluations / 3rdparty engineer opinions 5%BBA 5%Other 75%

Note: For a complete listing of other sources referred to, please consult the Detailed Statistical Tables in Appendix I




MANUFACTURERS

Knowledge of CCMCKnowledge of CCMC

Manufacturer Familiarity with CCMCMANUFACTURERS

C1 To what extent do you consider yourself familiar with the services offered by the CCMC?

57%n = 54

83% 83% of manufacturers claim to be familiar

with CCMC. However, only 26% are veryf ili

26%

15%

2%

17%familiar.

C2 Please indicate to what extent you agree with the following statements

Very familiar Somewhatfamiliar

Not very familiar Not at all familiar


y g gregarding the CCMC listings and reports.

The reports and listings …

54% 39%...are tools to demonstrate code compliance.


93%

TOTAL% Agree

Many manufacturers simply assume thatCCMC demonstrates code compliance orcompliance to a standard (in both cases54% completely agree).

35% completely agree that a CCMC

n = 54

54%

35%

37%

44%

...are tools to demonstrate compliance to astandard.

...provide an approval for use in construction.

91%

80%

35% completely agree that a CCMCListing or report will provide an approvalfor use in construction.

66



Awareness of CCMC’s Services

MANUFACTURERS

C3 Please indicate if you are aware of each of the services offered by the CCMC?

a) Listing: A listing is issued for products that have demonstrated compliance with recognized standards.

b) Reports: an evaluation of innovative construction products/systems that are not referenced in the NBC.

Although the vast majority claim to beaware of the two services offered byCCMC, 44% admit to not having known

96%

82%

Yes, aware of listing

Yes, aware of reports

n = 54the distinction between both servicesbefore answering the survey.

LEARNING

C4 Before answering this survey, were you aware of the distinction between the two services offered by CCMC (listing and reports for innovative products)?

Base: All respondents One‐on‐one interviews and focus groupsessions uncovered that the terminologyused (listing vs. report) tends to contributeto this confusion as both are “listed” on the

Yes No

n = 54

CCMC website.

56% 44%

67



Awareness of Listing Standards

MANUFACTURERS

E1a The CCMC listings show compliance to a recognized standard(i.e. put together by a recognized standard writing body through a consensus-based process). These standards are not necessarily referenced in the NBC. Were you aware of this?

n = 31

LEARNING

Yes58%

No42%

The answers to these twoquestions clearly indicate thatCCMC needs to bettercommunicate what service theyare offering to the manufacturer

E1b The CCMC listings show compliance to a recognized standard. However, when the Building Code imposes additional requirements

h f d d d h CCMC i f li i DOES

Base: Respondents whose company has obtained a listing in the past 5 years.

gwith a listing. In both cases, morethan 4 out of 10 respondentsadmit to not knowing that:

The listings do not necessarilyf d d i h NBCto the referenced standard the CCMC service for listings DOES

NOT evaluate to these additional requirements. Were you aware of this?

n = 31

reference standards in the NBC(42%).

The CCMC does not evaluate anyadditional requirements imposedby the National or Provincial

Yes52%

No48%

by the National or ProvincialBuilding Code (48%).

68



Use of CCMC Services

MANUFACTURERS

C5a Has your company ever obtained a CCMC listing for a construction product/system (used for products for which standards already exist)?

C5b Has your company ever obtained a CCMC report for an innovative construction product/system (used for products for which standards do not already exist)?

C5c Has your company ever initiated an inquiry or request to CCMC for a listing or a report for innovative products and did not go through with it?

C5d Are you currently in the process of initiating a request for a listing or report?

54 % f “Y ”

67%

48%

44%

Obtained a CCMC listing

Obtained a CCMC report

Ever initiated an inquiry or request to CCMC

n = 54 % of “Yes”

61%Is currently in the process of initiating a request


On average 58% of manufacturers’ innovativeC6a Considering all the innovative products that you

submit for approval for building projects, what is the percentage for which you have a CCMC report? n = 26None (0%) 8% Some (50% or less) 35% Majority (51% to 100%) 58%

Average 58%

On average 58% of manufacturers innovativeproducts have a CCMC number.

When it comes to listings, on average, 56% ofmanufacturers’ existing products have a CCMCListing.

C6b Considering the products that you submit for approval for building projects, what is the percentage for which you have a CCMC listing? n = 36None (0%) 6% Some (50% or less) 44% Majority (51% to 100%) 50%

Average 56%

!Of note that one-on-one interviews and focusgroups have demonstrated that the majority ofmanufacturers find it difficult to differentiatebetween a report and a listing even afterrepeated clarifications.

69

Base: C6a - Respondents whose company has obtained a CCMC report.C6b - Respondents whose company has obtained a CCMC listing.

repeated clarifications.


Use of CCMC Services (cont’d)

MANUFACTURERS

C7 Why do you not use the CCMC listing service more often? Not needed / rarely needed 8

Expensive 4 Time consuming / takes too long to f ind the needed info 3 Uses other listing services instead 2 Do not know 2

n = 19

Do not know 2 Other documentation provided 1 Unfamiliar w ith it / unaw are of it's existence 1 Canadian products / doesn't cover imported products 1 Uses them only for new productinfo / reports 1 We don't sell our products in Canada 1 Not adapted to our needs 1 Info is too technical / confusing / hard to understand 1

Base: Respondents whose use of the CCMC listing is less than 75%.

C8 Wh d tC8 Why do you not use the CCMC reportfor innovative construction products/systems more often?

Time consuming / takes to longto search for info 5 Do not know 4 Not needed / rarely needed 3 Lack of timely responses fromstaff 2 Technical guide is not reflective of conditions 2 Takes too long before the guide is published 2 E i 2

n = 14

Base: Respondents whose use of the CCMC report is less than 75%.

Expensive 2 Technical charts make it easier for competitors to copy tech 1 Lack of updated listings 1 Decision not to commercialize Canada 1


Use of CCMC Services (cont’d)

MANUFACTURERS

C9a When was the last time your company obtained a CCMC listing or a re-evaluation.

C9b When was the last time your company obtained a CCMC report or modification/re-evaluation.

C9c When was the last time your company initiated an inquiry or request to the CCMC for a listing or a report for innovative products and did not

More than three quarters of manufacturers (78%)who responded to the survey have applied for a newlisting/report of a re-evaluation quite recently (withinthe past 2 years).

the CCMC for a listing or a report for innovative products and did not complete it?

listing reportinitiated inquiry

n = 36 n = 26 n = 24

C10a Were you involved in the decision not to go through with the request

for a report of listing? n = 24

Less than a year ago 64% 54% 21% Betw een 1 and 2 years 14% 23% 29% Betw een 3 and 5 years 8% 12% 25% More than 5 years ago 8% 4% 13% Do not know 6% 8% 13%

Yes62%

No38%

Base: Respondents who initiated an inquiryBase: Respondents who used the service

n = 14

C10b Reasons not to follow through with the initial request for a listing or a report for innovative products.

Changing priorities/product was never commercialised 5 Requirements were too stringent and impossible to accomplish 3 Cost / too expensive 2 Lack of proper communication 1 Our customer did 1 Tests in technical guide were not related to inteded use 1 Could not meet our testing needs 1

n = 14


Could not meet our testing needs 1

Base: Respondents who were involved a decision not to go through With the request.

MANUFACTURERS

Evaluation of Listings and Reports for Innovative Products

Reasons for Using CCMC’s Services

MANUFACTURERS

D1 Please rate to what extent you agree with each of the following statements.

Completely agree Somewhat agreen = 40TOTAL

% Agree

53%

52%

45%

44%

Having a CCMC number gives us better access to the Canadian market.

Having a CCMC report for innovative products or listing makes it more likely thatmy product/system will be selected by architects, engineers or construction

firms.

98%

96%

50%

55%

35%

28%

Having a CCMC number provides an un-biased, third party verification of ourproduct.

Having a CCMC number is an effective way to prove and support our productclaims.

85%

83%

35%

23%

40%

43%

Having a CCMC number provides us with an edge over our competitors.

We generally apply for a CCMC number because our competitor(s) alreadyhave a CCMC number and we are at a competitive disadvantaged by not having

one

75%

65%

Base: Respondents whose company has used CCMC’s services in the past 5 years.

one.

The two statements that receive the highest level of agreement from manufacturers refer to CCMC giving better access to the Canadian market (98% agree completely or somewhat) and that a CCMC Report or listing makes it more likely that a


the Canadian market (98% agree completely or somewhat) and that a CCMC Report or listing makes it more likely that a product be selected (96% agree completely or somewhat).

Usefulness of CCMC’s Services

MANUFACTURERS

D2 To what extent would you agree that having a CCMC report number gives your product(s) immediate approval for use?

33% 63%n=40


95%

TOTAL % Agree

95% of manufacturers agree (completely orsomewhat) that a CCMC number will most likelyprovide an immediate approval for use. Of note,however, that only a third (33%) agree completely

D3 How useful do you find the…

33% 63%n 40 95%

V f l S h f l

Base: Respondents whose company has used CCMC’s services in the past 5 years.

TOTAL

with this statement.

Although manufacturers seem to find the CCMC

34%

48%

50%

48%

CCMC listings (n=32)

CCMC reports (n=25)

Very useful Somewhat useful

84%

96%

TOTAL% Agree Listings useful (84% very or somewhat useful).

They seem to agree that the reports are moreuseful (96% very or somewhat useful).

D8 Have you ever had a product refused by building officials even if you had a CCMC number?

n = 54

Base: Respondents whose company has obtained a CCMC listing / report

Almost a third of manufacturers (30%) claim tohave had a product refused by a building officialdespite having a CCMC number

Yes30%

No50%

despite having a CCMC number.

74


Do not know20%


Listing Strengths and Areas for Improvement

MANUFACTURERS

D4a What are the main strengths of the listings?

Technical data on products (limitations, uses, restrictions) 31% / %

n = 32

D4b How could the CCMC improve their listing service?

The listing is f ine as it is 44% / f %

n = 32

They are a trustw orthy / credible source 22% Good tool to identify a product 16% Independent / non-bias / thirdparty review s 16% It's Canadian / ensures the product meets Canadian standards 13% Convenient / helpful resource 9% Marketing 9% Comprehensive / easy to understand 6%

Too time consuming / Process should be faster 19% Product limitations should be more visible / easier to see 9% More information should be available 9% Not enough visibility / Needs more promotion / visibility 6% Misleading information 6% Not enough follow -ups / statusupdates 6% Other 28%

Reliability / reliable information given 6% Products listed have been evaluated / review ed by them 6% It's a reference guide / references building codes 6% Required for product sales to Canada 6% Other 28%

Base: Respondents whose company has obtained a CCMC listing

Do not know 6%

Note: The sum of the proportions presented in these tables may exceed 100% as respondents could provide more than one answer.

Base: Respondents whose company has obtained a CCMC listing.

The strengths and areas for improvement mentioned for the listings reflect much of the input we have obtained through one-on-one interviews and focus groups as well.


Report Strengths and Areas for Improvement

MANUFACTURERS

D5a What are the main strengths of the CCMC reports for innovative products?

D5b How could the CCMC improve their reports for innovative products?

Detailed / technical info (limitations, uses, restrictions) 32%

n = 25

The report for innovative products is f ine as is 52%

n = 25

Test products to ensure they meet established standards 28% Reliable / credible information given 20% Independent / non-bias / thirdparty review s 20% Provides all the info requiredto make a decision 8% Additional info by a recognized industry standard 8% Other 28% Do not know 20%

Quicker turn around time / speed of execution 28% More detailed info on specif ic/ approved uses of products 8% Other 28% I do not know 8%

Base: Respondents whose company has obtained a CCMC report

%

Note: The sum of the proportions presented in these tables may exceed 100% as respondents could provide more than one answer.

Base: Respondents whose company has obtained a CCMC report.

Since many manufacturers find the difference between listings and reports confusing, any difference in the list of strengths of reports vs. listings should be interpreted with extreme caution.


Perceived Timeliness of ReportsPerceived Timeliness of Reports

Satisfaction with Timeliness of CCMC Reports for Innovative Products

MANUFACTURERS

F1 What is your level of satisfaction with the time it takes the CCMC to complete a report for innovative products (from the moment an agreement with the CCMC is signed to the moment a report is issued)?

F2 More specifically, how satisfied are you with the time it takes to complete the following steps in the report process:

n = 23TOTAL % SatisfiedVery satisfied Somewhat satisfied

35%

91%

78%

n 23 Satisfied

9%

30%

26%

26%

61%

52%

The time it takes the CCMC to complete a report

The ease of initiating a request with the CCMC.

The responsiveness of the CCMC to the initial request

y

78%

65%

48%

48%

26%

13%

4%

9%

52%

52%

44%

39%

The responsiveness of the CCMC to the initial request.

The time it takes to carry out the tests by your selected third party laboratory.

The time it takes to develop a technical guide.

The time it takes to receive the final report after successful testing

Base: Respondents whose company has obtained a CCMC report in the past 5 years.

48%9% 39%The time it takes to receive the final report after successful testing.

There seems to be a general dissatisfaction among manufacturers with the time it takes CCMC to complete a report. g g p pIndeed, only 35% of manufacturers claim to be satisfied with this aspect.

The two elements that seem to drive this dissatisfaction are: The time it takes to develop the technical guide;

The time it takes to receive the final report after successful testing.

Focus group sessions also suggest that CCMC’s responsiveness during the whole process often leaves something to be


g p gg p g p gdesired (some complain of e-mails going unanswered for weeks, and sometimes months at a time).

Suggestions for Improvements for Timeliness

MANUFACTURERS

F3 What do you think the CCMC could do to speed up the time it takes to complete a report for innovative products?

74%n = 23

n = 11

F4 If CCMC were to offer an expedited service for an additional fee would you take advantage of this service?

52%Hire more staff 5

Outsource workload to meet deadlines 3

Follow a set time frame 3

They should not create their own standards 2

Use standards / criteria for other countries as a base 2

22%

13% 13%

Do not know 2

Follow the building codes 1

List minimum code requirement as well as optimal ones 1

Accommodate customers more quickly 126%

13% 13%

Definitely Maybe Probably not Definitely not

Base: Respondents less satisfied with the time it takes to complete a report. Base: Respondents whose company has obtained a CCMC report in the past 5 years.

There seems to be a certain openness to paying anadditional fee for a more expedited service. However, only22% of manufactures feel CCMC should definitely offer


22% of manufactures feel CCMC should definitely offerthis service.

Issues Related to the ReportsIssues Related to the Reports

Perceptions of Changes to Report Format

MANUFACTURERS

F5 Were you aware of these changes?

No

n = 23

CCMC 1995 Report Format CCMC 2005 Report Format

In 2005, the CCMC changed its report format for innovative products. Below are the modifications that were made to the sections:

56%Section 1 Purpose of evaluation Section 1 Opinion

Section 2 Opinion Section 2 Description

Section 3 Description Section 3 Conditions and limitations

Yes44%

Section 4 Usage and limitations Section 4 Technical evidence

Section 5 Performance

F6 To what extent do you agree that the format for the 2005 report for innovative products format is an improvement over the 1995 format?

n = 10

Although only 44% of manufacturers are actually aware of the format change, those who are aware of it seem to feel that the 2005 report format is an improvement upon the 1995 format (100% agree completely or somewhat)

Completely agree20%

Somewhat agree80%

1995 format (100% agree completely or somewhat). However, there seems to remain some room for improvement

considering that only 20% completely agree with the statement.

81



Evaluation of Report Format

MANUFACTURERS

F7 Please indicate your level of agreement with the following statements with regards to the format of the NBC 2005 CCMC reports for innovative construction products/systems.

Completely agree Somewhat agreen = 23TOTAL

% Agree

48%

35%

22%

48%

61%

61%

The reports' format makes it accessible to the readers.

The report clearly states the results of analysis in order to prove my products'possible uses.

The reports provide complete information about my product.

96%

96%

83%


There seems to be some room for improvement considering that less than half of the respondents completely agree with any of the proposed statements.


Perceived Usefulness of Certain Ideas

F8b When extenuating circumstances that prolong the updating and F8a In the case of a new evaluation request, how useful would you find it if CCMC were to provide a method to facilitate the proponent’s access to information related to the progress being made on a particular evaluation file?

n = 23

%

F8b When extenuating circumstances that prolong the updating and publication of a report for innovative products during the re evaluation process, the CCMC could provide a “partial” report (i.e a partial evaluation covering only the results of completed tests as well as identification of pending elements). To what extent would such a partial report be useful to you?

n = 23

92%

35%

44%

89%

70%

92%

Base: Respondents whose company has obtained a CCMC report in the past 5

22%


CCMC report in the past 5 years.

84% of manufacturers believe that better access to information related to the progress of their


Manufacturers seem more favourable than their building official counterparts to having a partial report provided (manufacturers 91% agree vs

83

report file would be useful.report provided (manufacturers 91% agree vs. 72% building officials).

Who Should Have Access to Report Information Before it is Published

MANUFACTURERS

F10 Before a new report for an innovative product is published who would you feel comfortable giving access to the information*?

78%Building Officials

n = 23

57%

30%

22%

Provincial Ministry Officials

General Public

Do not provide the information to anyone but the proponent

17%

4%

4%

Other Manufacturers

Builders

Architects


4%Do not know

* CCMC has completed the technical work and the proponent has accepted the report. Between this point in the process and inclusion in the Registry there are many additional steps that may take months, for example: translation to second language or final edits if required.


Almost 8 out of 10 manufacturers are comfortable rendering non-published information available to building officials (78%).

However, the level of comfort is less for all the other potential entities to whom the information could be given (especiallywhen it comes to other manufacturers, builders and architects). In fact, 22% feel that the information should be provided


, ) , ponly to the proponent.

Who Should Have Access to Information Regarding Re-Evaluation of a Report

MANUFACTURERS

F8 In the case of the re-evaluation of a report (the proponent already has a published report which is being modified or updated). How useful would it be if CCMC were to provide a method to facilitate access to information related to the progress being made on a particular evaluation file? Please answer for each of the following parties.

n = 23

TOTAL

65% 26%The proponent would access

information related to the progressbeing made on a particular file


91%

TOTAL % useful

44%

30%

35%

26%

The building officials would accessinformation related to the progress

being made on a particular file.

The information related to the progressbeing made on a particular file would

78%


g pbe available to all via a registry. 57%

Th f t t thi k th t d i th f th i fil t th l t th b ildi ffi i l ld The manufacturers seem to think that rendering the progress of their file open to themselves or to the building official could be useful (91% and 78% respectively), however rendering the information available via a registry seems to garner less enthusiasm (57%).


Report Confidentiality

MANUFACTURERS

F11 The CCMC process keeps all information confidential and only allows access to those who need to know until the final report is accepted by the client. To what extent are you satisfied with this level of confidentiality?

n = 23 100% of manufacturers are very or somewhat satisfied with

Very satisfied

87%

Somewhat satisfied

13%

100% of manufacturers are very or somewhat satisfied with the current level of confidentiality offered by CCMC.



Appreciation of Protection Period

MANUFACTURERS

F13 As it has become more complex and costly to develop technical guides for innovative products/systems the protection period for new technical guides has been increased from 1 to 3 years. Do you feel that a three years protection is acceptable?

n = 23Why? n = 3

Yes83%

No13%

Do not know4%

It almost takes 3 years to complete the cycle 1

Content needs to be open for revisions and appeals 1

5 years would probably be better 1


83% of manufacturers find the current 3 year protection period for a technical guide to be acceptable.


MANUFACTURERS

RE-EVALUATION PROCESSRE EVALUATION PROCESS

Re-evaluation Process

MANUFACTURERS

G2 To what extent do you feel the re-evaluation of products/systems by the CCMC is necessary?

n = 54

39%

70%

30%

70% of manufacturers feel that the CCMC re-evaluation is very or somewhat necessary.

32%

26%

30%

4%


Very necessary Somewhatnecessary

Not verynecessary

Not at allnecessary

89



Re-evaluation Process (cont’d)

MANUFACTURERS

G1 Currently, the CCMC re-evaluates reports for products/systems every three (3) years (this places reports between code cycle 5 years). How satisfied are you with this time frame?

Base: All respondents78%

n = 54

G3 If the CCMC were to change its re-evaluation cycle to 5 years, to what extent would you agree with such a change?

63% 87%

n = 54


48%

78%63% 87%

30%

17%

22%24% 13%

6%

Very satisfied Somewhatsatisfied

Somewhatdissatisfied

Verydissatisfied

11%

2%

Completelyagree

Somewhatagree

Somewhatdisagree

Completelydisagreeg g g g


During the focus group sessions, several manufacturers and building officials make the point that the actual re-evaluation cycle is not a central point since, in a constantly evolving market, the chances are that the manufacturer will change their


quality control process well within the 3 or 5 year period.

MANUFACTURERS

Additional ServicesAdditional Services

Interest in Expanded Service

MANUFACTURERS

G5 In your opinion, should the CCMC expand its service to include the following services?

67%Energy Efficiency

n = 54 % of “Yes”

65%

61%

50%

Green

Sustainability / Conservation

Water Conservation


Similarly to the building officials, manufacturers have ranked Energy Efficiency among the top services that CCMC could y g , gy y g pconsider offer.


Suggestions for Added Services

MANUFACTURERS

G6 Do you have any suggestions as to any other services CCMC could offer?

7%Become the standard for all listings

n = 54

4%

4%

2%

Comparative CCMC / NBC evaluations

Faster response time to demands

Cover more subjects / info like Air Barrier Systems

2%

2%

2%

2%

Get rid of the reports

Increase efficiency of currentservices first

Need to be more market driven/ respond to industry needs

General testing facility and guidance to innovative products


2%

2%

2%

2%

General testing facility and guidance to innovative products

Evaluation of other products approved by other countries

Ensure products conform with US standards

More rigorous testing / evaluation of new products

2%

2%

78%

Better assistance to manufacturers

No suggestions

Do not know

93



MANUFACTURERS

Satisfaction With Contact With CCMCSatisfaction With Contact With CCMC

Contact with CCMCMANUFACTURERS

H1 In the past year, have you contacted the CCMC about a file?

87%By e-mail

n = 54 % of “Yes”

74%

17%

9%

By phone

Through the web-site

Any other means


E-mail and phone are the most popular means of communicating with CCMC for manufacturers.


p p p g


Satisfaction With CCMC Service

MANUFACTURERS

H2a How satisfied were you with the support / service you received when you contacted the CCMC?

n = 49

There seems to be room for improvement when it comes to the level of satisfaction with the support/service received from CCMC.

49%76%

Why? n = 12

Time consuming / slow response time 5

Follow up on queries / return calls 2

Other 12

support/service received from CCMC.

Indeed, almost a quarter (24%) of manufacturers claim to be not very or not at all satisfied.

27%24%

20%

4%

Base: Respondents who have contacted CCMC.

Very satisfied Somewhatsatisfied

Not verysatisfied

Not at allsatisfied

96

Base: Respondents who have contacted CCMC.




Brio Conseils inc.

Page: 15 / 21

APPENDIX E: Focus Group Results, Ad hoc Research

CCMC B i R iCCMC Business ReviewFocus Group Results

December 2009

Table of Contents

Objectives and Methodology 3

Executive Summary 6

Main Learnings 12Learnings Regarding the Perceived Role of CCMC 13Learnings Regarding the Perceived Role of CCMC 13

The Main Strengths of CCMC 14

Areas for Improvement 15

General Learnings 27

Appendix (research tool) 32pp ( )

2

Objectives

In the context of the CCMC Business Review one of the objectives is to consult stakeholders in terms

Objectives

In the context of the CCMC Business Review, one of the objectives is to consult stakeholders in terms of their perceptions of CCMC.

The focus groups allowed CCMC to consult several building officials and manufacturers across Canada.

The main objectives of these focus groups were to provide a forum in which building officials and manufacturers alike could:

Identify what their perception of CCMC’s role is;

Express their level of satisfaction with CCMC (strengths and weaknesses);Express their level of satisfaction with CCMC (strengths and weaknesses);

More specifically:

Understand their perceptions of CCMC listings vs. CCMC reports;

Discuss how CCMC should offer listings in the future;g ;

Discuss CCMC’s re-evaluation process (understanding of the process, suggestions for improvement);

Discuss how CCMC could improve their communication with building officials and manufacturers

4

manufacturers.

Methodology

Canadian Building Officials

Manufacturers (the majority of whom have a CCMC listing or report number and some who are in the process of applying).

Target Population

( j y g p p pp y g)

Names of manufacturers and building officials to be contacted were provided by CCMC.

Profile of Participants

In all, 6 focus groups were held:

1 in Calgary (September 29, 2009)

2 in Vancouver (November 16, 2009)

2 in Toronto (November 18 2009)2 in Toronto (November 18, 2009)

1 in Halifax (November 24, 2009)

In addition to the 6 focus groups, 6 individual interviews were carried out in Quebec with manufacturers.

A total of 49 persons participated in the focus group sessions and the individual interviews.

18 building officials participated 18 building officials participated

29 manufacturers participated

1 representative from a laboratory participated

1 ministry official (who had previously participated in phase 1 (individual interviews with ministry officials)

5

Executive SummaryExecutive Summary

Executive Summary

The following pages summarize the main strengths as well as suggested improvements identified during the focus groups.

Of interest, the vast majority of the improvements relate to: improving means of communication with manufacturers andimproving means of communication with manufacturers, and

providing more information to building officials.

Main Strengths

Manufacturers Building Officials

CCMC provides a national scope of acceptance

Having a CCMC Listing or Report number ensures betteracceptance by building officials

Saves time – since CCMC is a third party that interprets thecertified laboratory data and determines if a product iscompliant to a standard listed in the code or the intent of thecode The existence of CCMC provides an even playing field for

smaller manufacturers

CCMC’s personnel is very knowledgeable

code.

CCMC is considered to be a reliable source of information:All agree that CCMC is a reliable source of information when itcomes to providing information on products that they need toapprove for use.

CCMC’s personnel is very knowledgeable.

Th CCMC St ff ll id d f i dl d The CCMC Staff are generally considered friendly andapproachable.

Provide assurance and a certain comfort level whenapproving the use of a product (listing and reports) –many feel there is less risk when a product has a CCMCnumber.

CCMC i t t i l f / t

7

CCMC is present at regional conferences/events.

Executive Summary (cont’d)

LEARNING

Many manufacturers are Several manufacturers feel that this element (improved response time and

Main Areas for Improvement

Many manufacturers are dissatisfied with CCMC’s response time and follow-up

Several manufacturers feel that this element (improved response time and better follow-up on inquiries) is among the top priorities for improvement.

CCMC should also address the climate of fear that seems to revolve around making any kind of complaint. Due to this climate, CCMC is most likely unaware of several existing problems.

The length of time to obtain a report

Baring being able to speed up the existing report process, CCMC would surely benefit from being more transparent when it comes to the process of applying for a report, the time it may take and the associated costs.

A certain confusion when it comes to the re-evaluations

Building Officials: CCMC needs to clearly communicate to building officials what a re-evaluation entails, what prompts it and how to interpret a listing/report that is past its re-evaluation or under re-evaluation.

Manufacturers: The process for re-evaluation needs to be better managed with the manufacturers (systematic reminder prior to re-evaluation, clear communication as to what it entails, prompt re-evaluation of products).

8


LEARNING

The website is widely used but Some suggest that CCMC should re-evaluate the website and render it f f

Other Areas for Improvement

not considered to be user-friendly

more user-friendly, especially when it comes to emphasizing how to find the registry of product evaluations.

Suggested Improvements to the Reports/Listings

See page 22 for suggested improvements.Reports/Listings

Reports/listings are dependent on a single person and not reviewed by a committee

CCMC should consider communicating exactly how decisions are made to avoid any misunderstandings and perceptions of individual bias.

y

Lack of an official review process CCMC could consider communicating the review process.

CCMC is perceived as CCMC needs to clearly explain why in some cases it is suggesting theCCMC is perceived as overstepping its mandate when it comes to establishing the technical guide

CCMC needs to clearly explain why in some cases it is suggesting the technical guide over-reach the original intended purpose so as to limit frustrations on behalf of the proponent.

9


Oth A fLEARNING

CCMC does not take field experience into account

CCMC needs to clearly communicate why field experience cannot be taken into account so as to reduce frustration on behalf of the proponent.


There are a variety of issues that surfaced directly related to the ICF industry

CCMC should consider addressing the issues with the ICF industry in a transparent and proactive manner so as to re-establish its lost credibility.

ICF industry

Other CCMC Learnings

Lack of Knowledge Among Some Building Officials

Certain building officials seem to feel a false sense of security when a product has a CCMC number. Indeed, many simply assume that a CCMC number is a guarantee that the product meets the intent of the building code.

A Thorough Confusion When it Comes to Defining a Listing and a Report

CCMC needs to communicate to building officials what exactly a CCMC Listing and or Report entails and how each can be interpreted. By not clearly communicating the scope of each CCMC number to building ffi i l CCMC i i li itl i b ildi ffi i l t b li th t

10

officials, CCMC is implicitly encouraging building officials to believe that a CCMC number is a guarantee of code compliance.


Should CCMC Continue to Offer a Many of the arguments for CCMC keeping the listing service (see page 29)

Other CCMC Learnings

S ou d CC C Co t ue to O e aListings and Report Service?

a y o e a gu e s o CC C eep g e s g se ce (see page 9)are based on a broader reaching problem: that of building officials who donot all have the same level of knowledge and/or resources to carry outevaluations of third party laboratory reports to ensure that the products doindeed comply to a recognized standard.

This results in some building officials relying too heavily on CCMC as ag y g ythird party resource. As such, the question becomes whether or not it istruly CCMC’s role and responsibility to address the shortcomings of thebuilding officials.

11

Main LearningsMain Learnings

Main Learnings

Learnings Regarding the Perceived Role of CCMC

Differing relationships depending on the point of view: Building officials tend to view CCMC as a partner whereas manufacturers view CCMC as a service provider.p

The role of CCMC from a building official’s perspective tends to revolve around the following points:

A facilitator for the introduction of products to market (especially innovative products);

CCMC provides a resource for building officials if they have questions regarding the scope of use of a product;

CCMC provides a centralized list of products that have met CCMC “approval”;

T d t i h th d t t th i t t f th b ildi d To determine whether products meet the intent of the building code;

To develop technical guidelines for innovative products when no standards exist.

The role of CCMC from a manufacturer’s point of view tends to revolve around the following points:

To provide a marketing tool to help introduce their product into the market.

CCMC makes it easier for the manufacturer to penetrate the Canadian market through an agency that is p g g y“nationally” recognized.

Guarantee the safety of the Canadian public by ensuring that the products used in construction are compliant to the intent of the code.

The use of the word “approval”: Several building officials and manufacturers alike tend to use the term “approve” when speaking of the role of CCMC.when speaking of the role of CCMC.

LEARNING

CCMC needs to better communicate its role so that building officials and manufacturers alike are not under the impression

13

that CCMC actually evaluates and/or approves products.

Main Learnings – Main Strengths

The Main Strengths of CCMC

Manufacturers Building Officials

Provides a national scope of acceptance: Having a CCMCnumber offers a relatively universal access to building sites acrossCanada and allows the manufacturer to avoid having to deal witheach province separately and/or with each building projectseparately.

Ensures better acceptance by building officials: The majority

Saves time: Most building officials agree that having a CCMClisting saves them time as they do not need to interpret thelaboratory data themselves and determine if a product iscompliant to a standard listed in the code or the intent of the code– they can rely on CCMC’s expertise and knowledge.

In addition several building officials mention that they Ensures better acceptance by building officials: The majorityof manufacturers have had to deal with situations in which abuilding official refuses to use their product (for both listings andreports) because it did not have a CCMC number. Listings: Although most admit that this type of situation does not

happen often for CCMC listing products, it can happen severaltimes a year – as such, they prefer to have the CCMC listing and

In addition, several building officials mention that theywould not have resources or the know-how to interpret thedata and evaluate an innovative product.

Trusted opinion: Building officials have a very high esteem ofCCMC’s capacity to determine if a product is safe to use.

Reliable source of information: All agree that CCMC is areliable source of information when it comes to providing

y y p gavoid losing a contract due to uninformed building officials.

Reports: Many building officials insist on a CCMC number when itcomes to an innovative product as they rely on CCMC’s expertiseto reassure them that the product can indeed be used accordingto the intent of the NBC.

Provides an even playing field for smaller manufacturers: If asmaller manufacturer obtains a CCMC listing or report it will likely

reliable source of information when it comes to providinginformation on products that they need to approve for use.

Technical knowledge: Most agree that CCMC’s personnel isvery knowledgeable and able to answer any questions they haveabout the scope of use of a product.

Friendly and approachable staff: Many of the building officialsfeel that they have a good relationship with the person they dealsmaller manufacturer obtains a CCMC listing or report, it will likely

be accepted by a building official regardless of the fact that themanufacturer is not a large multinational company with a knownreputation.

Technical knowledge: Many agree that CCMC’s personnel isvery knowledgeable in determining technical guide criteria.

feel that they have a good relationship with the person they dealwith.

Provide assurance: Several building officials feel that CCMC’slistings and reports provide them with a certain comfort level whenapproving the use of a product. As such they feel that they areexposing themselves to less risk when approving products thathave a CCMC number

14

have a CCMC number.

Good representation at regional conferences: Several buildingofficials mention that they appreciate that CCMC is present atmany of their regional conferences and events.

Main Learnings – Areas for Improvement

There are many areas for improvement that were mentioned in the group sessions. The following pages address each of the suggested areas for improvement in order of importance:

Main Areas for Improvement

Many manufacturers are dissatisfied with CCMC’s response time and follow-upy p p

The length of time to obtain a report

A certain confusion when it comes to the re-evaluations


Th b it i id l d b t t id d t b f i dl The website is widely used but not considered to be user-friendly

Suggested improvements to the reports/listings

Reports/listings are dependent on a single person and not reviewed by a committee

Lack of an official review process

CCMC is perceived as overstepping its mandate when it comes to establishing the technical guide CC C s pe ce ed as o e s epp g s a da e e co es o es ab s g e ec ca gu de

CCMC does not take field experience into account

Offer more accessible listings/reports

15


There is a clear divide in terms of perceptions when it comes to the quality of communication with CCMC.

Manufacturers are generally very satisfied with the ease of obtaining information from CCMC and the rapidity with which CCMC personnel answers their inquiries (indeed in the Web survey1 97% claim to be very (46%) or

Many Manufacturers are Dissatisfied with CCMC’s Response Time and Follow-up

with which CCMC personnel answers their inquiries (indeed in the Web survey1 97% claim to be very (46%) or somewhat (52%) satisfied with the support/service they receive).

On the other hand, several manufacturers complain regarding a slow response time and a lack of follow-up oninquiries. (This lack of satisfaction is reflected in the Web survey1 in which only 25% of manufacturers claim to be“very satisfied” and a full 24% are “very or somewhat dissatisfied”). Some specific examples are: CCMC has been known to taking two months to answer specific questions CCMC has been known to taking two months to answer specific questions.

E-mails and voice-mails going unanswered for weeks. Many mention that if they did not hound their CCMC contact they would probably never receive an answer to their question.

“It is simply unacceptable that an agency to whom I pay money for a service, provide such a poor follow-up on calls and e-mails.”

“It is common courteously to return one’s calls on a timely basis – CCMC simply doesn’t do it.”

“There is a double standard. Why do the building officials get swift responses and we, as paying customers, are getting the short end of y g g p , p y g , g gthe stick.”

Fearful of Complaining: Several manufacturers claim to be uncomfortable formulating any complaints to CCMC when itcomes to their lack of responsiveness and follow-up. Many are convinced that if they complain, their request will bedropped to the bottom of the priority list of the already overworked professional and will further delay their acceptance.pp p y y p y p

“It’s common knowledge that if you complain too much your file is simply going to end up at the bottom of the pile. So what do we do, we stay quiet and say nothing for fear of not getting what we need.”

161 “CCMC Business Review, Web Survey with Building Officials and Manufacturer”, by Ad hoc research, November 2009


Suggestions:

Hiring more personnel: Several are acutely aware that CCMC is short staffed and that this is a contributing

Many Manufacturers are Dissatisfied with CCMC’s Response Time and Follow-up (cont’d)

factor to the lack of timely responses.

Having an experienced customer service representative who coordinates the requests and ensures there is a follow-up internally. Although several are interested by this concept, some are concerned that adding another person in the hierarchy will simply slow the process down even more. In addition, many comment that the person would have to be technically knowledgeable.

“I am not sure how adding another layer in the communication process is going to actually speed things up.”

Training the technical personnel in customer service / management and the importance of rapid and proactive communication.

LEARNING

Several manufacturers feel that this element (improved response time and better follow-up on inquiries) is among the top priorities for improvement. CCMC should also address the climate of fear that seems to revolve around making any kind of complaint. Due to this climate, CCMC is most likely unaware of several existing problems.

17


The Length of Time to Obtain a Report

Several mention having issues with the time it takes to get a report. Specific issues include:

Oftentimes, by the time the report comes out, the industry has changed:

One manufacturer claims to have waited 3 years for the report and ended up deciding to drop out of theprocess due to the lengthy delays.

Others feel they are hostage to CCMC as they have already invested too much money to drop out of theprocess – but wonder if it was worth the initial investment considering the amount of time it is taking toget a report.

Those abandoning the process mention that they tend to revert to part 4 of the code and obtain theirinnovative product approved as an equivalent product through an engineer. This however can be costlyas they must complete the process on a project by project basis.

Suggestions:

Take into consideration evaluations that have been carried out by other international evaluation agencies.y g

Be more transparent when it comes to steps being taken, who is accountable for each step and how much time each step can take.

Take existing field experience into account.

Simplify the process for a technical guide by taking risk assessment into consideration (if the product holdslittle or no risk the process for obtaining a report should be streamlined)little or no risk, the process for obtaining a report should be streamlined).

LEARNING

Barring being able to speed up the existing report process, CCMC would surely benefit from being more transparent when it comes to the process of applying for a report the time it may take and the associated

18

transparent when it comes to the process of applying for a report, the time it may take and the associated costs.


A Certain Confusion When it Comes to the Re-evaluations

Confusion regarding the re-evaluations is twofold:

Manufacturers: Although most manufacturers feel that the re-evaluation process is easy, transparent andstraightforward there are some complaints when it comes to the logistics of the re evaluation:straightforward, there are some complaints when it comes to the logistics of the re-evaluation:

Slow turnaround: a few complain that although CCMC has cashed their check for the re-evaluation oftheir product several months ago, the process of obtaining a product to be re-evaluated has not beeninitiated.

Lack of follow-up: A few complain that when they inquired as to the status of their paid re-evaluation,they obtain little or no response from CCMCthey obtain little or no response from CCMC.

Lack of proactivity: In some cases, it is the manufacturer who was obliged to remind CCMC that theyare up for re-evaluation and request information as to when and how it will be initiated.

Building Officials: The quasi totality of building officials do not understand the re-evaluation process. Morespecifically they are unaware of:

h t t l ti what prompts a re-evaluation;

what exactly a re-evaluation entails;

at what frequency re-evaluations done;

how to interpret a listing/report that has an expired re-evaluation date (the date for re-evaluation is pastdue by several years) – but has no indication that it is under re-evaluation;

how to interpret a listing/report that is indicated “under re-evaluation”.

LEARNING

Building officials: CCMC needs to clearly communicate to building officials what a re-evaluation entails, what

19

prompts it and how to interpret a listing/report that is past its re-evaluation or under re-evaluation.Manufacturers: The process for re-evaluation needs to be better managed with the manufacturers (systematic reminder prior to re-evaluation, clear communication as to what it entails, prompt re-evaluation of products).


A Certain Confusion When it Comes to the Re-evaluations (cont’d)

5 Year Re-evaluation Cycle vs. A 3 Year Cycle: When asked if the re-evaluation cycle should be changed to 5years, reactions are mixed.

Some applaud the idea as the re-evaluation cycle is already behind schedule and, as such is, for allpractical purposes, already on a 5 year cycle.

Others comment that the official re-evaluation should be not be on any particular schedule and shouldsimply be automatically prompted if:

the NBC changes and affects the said product in any particular manner;

the manufacturer claims in their yearly affidavit that their product has changed or that their manner ofmanufacturing it has changed.

20


Lack of Communication When it Comes to the Steps Required to Apply for a Listing/Report

Several building officials comment that they are often left in the dark when it comes to the steps required toapply for a listing or report.

In addition, CCMC does not generally respond to requests regarding the process in a timely manner (e.g.: onemanufacturer has been waiting for 5 months just to get a price on how much the report would cost).

Some also mention that the pricing seems to be relatively arbitrary. Those who have several listings/reports donot understand why some listings/reports are more expensive than others. The information is never clearlycommunicated to them by their contact at CCMC.

Suggestions

Provide a list of each step that will be required for each application (listings and reports) on the website.

When a manufacturer contacts CCMC to apply, they should be provided with:

A list of the various steps that will be required.

An idea of how long each step can takeAn idea of how long each step can take.

A ballpark figure of how much it could cost as well as aspects that can influence the cost as well as a feeschedule.

“I have budgets to plan and to respect, with CCMC they come out of the blue with bills for $10,000. I am willing to pay for it, but it would make it a lot easier if they had warned me in advance how much it was going to cost and I could include it in our annual

budget.”

LEARNING

CCMC needs to be more transparent when it comes to the process of applying for a listing/report, the time it can take and the related costs.

21


Suggested Improvements to the Reports/Listings

Several building officials suggest improvements to the reports/listings:

Change the names and clearly indicate:

report for an innovative product

listing for a product for which a standard exists

Simply putting an L or R at the end of the number is not sufficient.

In the case of listings: Clearly indicate to what standard the product is being compared to.

In the case of both reports and listings: Clearly indicate the intended application in an executive summary so In the case of both reports and listings: Clearly indicate the intended application in an executive summary sothat the building official does not need to fish for information.

In the case of both reports and listings: Clearly indicate if limitations exist (yes/no) and where the limitationsare further explained (on what page / what paragraph).

In the case of reports: Include a section that clearly indicates if there are any regional specifications.

I th f t A f f t ti th t if tt ib t t bl l d i t i th NBC th In the case of reports: A few manufacturers mention that if attribute tables already exist in the NBC, thereport should not include different attribute tables – the differing information tends to confuse some buildingofficials and can lead to them simply refusing the use of a product.

Ensure that all reports are updated to reflect the 2005 code – a few comment that some reports are so farpast due their re-evaluation and that they no longer reflect the reality of the industry.

f CC C f f / A few mention that CCMC should not charge for the inclusion of a hyperlink in the Listing/Report.

22


The Website is Widely Used But Not Considered to be User-friendly

The website is widely used by building officials to check if a CCMC number is still valid or not.

While some find the website relatively easy to use, others complain that the recent revamping of the website isfar less intuitive than the last version of the website. Contributing to the confusion is:

The overall structure of the NRC website that is considered difficult to navigate.

The difficulty of finding the registry of product evaluations: this is the tool that is most often used andbuilding officials and manufacturers alike complain that it is very difficult to locate this information on thewebsite.

Difficulty in finding concise information on the steps required to apply for a listing/report.

LEARNING

Some suggest that CCMC should re evaluate the website and render it more user friendly especially when itSome suggest that CCMC should re-evaluate the website and render it more user-friendly, especially when it comes to emphasizing how to find the registry of product evaluations.

23


Reports/Listings are Dependent on A Single Person and Not Reviewed by a Committee

Obtaining a CCMC report is especially important to manufacturers as they have very few alternatives to a CCMCreport when it comes to obtaining approval for the use of their innovative products in the Canadian market.

A few manufacturers comment that the process of obtaining a report or listing seems to rely on the judgement ofa single person. The concerns raised are the following:

It is impossible for every CCMC employee to have extensive expertise of all industries – as such, a few raiseconcerns that the current business model is flawed.

If a single person is in charge of the decision, their individual bias can taint the decision making process.

It is suggested that each decision be reviewed by a committee and, as such, any bias that could be attributed to asingle person would be eliminated.

Lack of An Official Review Process

A few manufacturers who have experienced issues with the process of developing the technical guide commentthat there is no official review process.

This perceived lack of transparency inevitably leads to a lack of confidence in the process as a whole.

24


CCMC is Perceived as Overstepping its Mandate When it Comes to Establishing the Technical Guide Some manufacturers (from different industries) mention that CCMC will occasionally tend to define the technical

guide for broader use than the use originally intended for the product. In these cases, the manufacturers feel thatg g y pCCMC is overstepping its mandate. These manufacturers feel that if they want their product to be tested for aspecific use and not a multitude of other uses, it should be the choice of the manufacturer and not the prerogativeof CCMC.

CCMC Does Not Take Field Experience Into Account

Some of the manufacturers complain that CCMC does not seem to have any procedures related to takingpractical field experience of a product/system into account.

In some cases products have been used worldwide and even in Canada for decades – but CCMC insists on In some cases products have been used worldwide and even in Canada for decades – but CCMC insists ontrying to establish tests that will prove what years in the field have already proven. This seems to be veryfrustrating for the manufacturers.

Offer More Accessible Listings/Reports

Some building officials suggest that they are sometimes confronted with having to check a CCMC number while directly ata construction site. In these cases, there is an enormous amount of pressure to approve the use of a product in a timelymanner (almost instantaneously) so that the construction can continue.

Some suggest that having access to the listings via a website that is smart-phone friendly would be advantageous and

g p

25

gg g g p y gsave them from having to return to the office to check on the website.


Issues with the ICF Industry

There were several representatives from the ICF industry who were represented during the Toronto sessions whoexpressed acute dissatisfaction with their dealings with CCMC. The following is a summary of their maindi ti f tidissatisfactions:

Lack of Cooperation: CCMC has demonstrated little or no willingness to provide any solutions to theindustry in a situation in which the technical guide has taken an inordinate amount of time to establish.

Lack of Transparency: There is a general lack of communication with CCMC. The latter does not provideinformation on what steps are being taken to resolve issues that have been raised with them. In cases in

hi h th h t t d CCMC d i i th i t t twhich they have contested CCMC decisions, the review process was not transparent.

Files are Under the Responsibility of a Single Person: There were accounts of an ICF product beingapproved by one person while another person refused to approve a similar product. This type of situationinevitably leads to mistrust of CCMC as an agency and its credibility.

Loss of Files/Information: CCMC has, on a few occasions, claimed to have lost a request and/ori f ti th t id d t th O i thi di i i h th i dibilitinformation that was provided to them. Once again this diminishes their credibility.

LEARNING

CCMC should consider addressing the issues with the ICF industry in a transparent and proactive manner so as to re-establish its lost credibility.

26

Main Learnings – General Learnings

Although the majority of the building officials who participated in this phase of the research feel they have the competencesto properly decipher a CCMC report or listing, several point out that in smaller municipalities where the resources are morelimited, building officials do not necessarily have the knowledge to properly understand the terms and conditions of a

Lack of Knowledge Among Some Building Officials

CCMC listing or report.

As such, some building officials rely on the depth of analysis offered by CCMC to approve a product and tend to take aCCMC number (listing or report) as a guarantee of the quality of the product.

LEARNINGLEARNING

Certain building officials seem to feel a false sense of security when a product has a CCMC number. Indeed, many simply assume that a CCMC number is a guarantee that the product meets the intent of the building code.

27


A Thorough Confusion When it Comes to Defining a Listing and a Report

Confusion as to the difference between each CCMC number: Most building officials and manufacturers are thoroughlyconfused when it comes to the difference between listings and reports. Although most claim to understand the difference,during the discussions the moderator continually had to reiterate the difference between one and the other. Indeed, severalhad never noticed that some CCMC numbers have an “L” designation while others have an “R”.

Terminology is Confusing: One of the elements that contributes to this confusion is the term “listing”. In the mindsof most participants, CCMC is providing a listing of products (for which standards exist or for innovative products)and as such listings apply to both types of CCMC numbers.

Lack of Knowledge: Many of the building officials were surprised to find out that a CCMC Listing does not necessarilyevaluate any additional requirements imposed by the NBC. Indeed, several assumed that a CCMC Listing was anindication that the product is code compliant. This reflects results from the recent Web survey1 where more than half of building officials admitted to not knowing that CCMC does not

evaluate any additional requirements imposed by the National or Provincial Building Code (57%).

Suggestions:

C f CC C ( ) Clearly define each CCMC number (Listings and Reports) on the website. Ensure that the links are visible and easyto find.

When CCMC visits regional conferences/events it should take the opportunity to meet with the building officials andclarify the situation.

LEARNINGLEARNING

CCMC needs to communicate to building officials what exactly a CCMC Listing and or Report entails and howeach can be interpreted. By not clearly communicating the scope of each CCMC number to building officials,CCMC is implicitly encouraging building officials to believe that a CCMC number is a guarantee of codecompliance

28 1 “CCMC Business Review, Web Survey with Building Officials and Manufacturer”, by Ad hoc research, November 2009

compliance.


Report Service: The quasi totality agree that CCMC has a key role to play in the industry when it comes to aiding newand innovative products to market.

Few Other Alternatives: Since most feel that they have little or no viable alternatives when it comes to getting their

Should CCMC Continue to Offer a Listings and Report Service?

y g ginnovative products used in projects, they feel that CCMC has a certain responsibility in continuing to offer thisservice.

Listing Service: The response as to whether or not CCMC should continue to offer a listing service is far more complex.The table below summarizes the arguments in favour and those against.

In Favour of Abolishing CCMC Listings Arguments Against Abolishing CCMC Listings

Little value added: Many manufacturers feel thatCCMC offers little value added as they are simplyvalidating information that is already provided by acertified laboratory.

Many building officials are dismayed at the thought of CCMC no longeroffering the Listing service. Reasons behind this dismay include: More time consuming: Many admit that they lack the resources to be able to

undertake the job of checking the laboratory reports against a recognized

Level playing field: Some manufacturers also mentionthat by abolishing the Listing service, the playing fieldamong competitors would be leveled (there would beno advantage to those who have a Listing vs. thosewho do not have one).

Allows CCMC to concentrate on its original

standard. The fact that CCMC offers a third party service saves many buildingofficials precious time.

Do not necessarily have the knowledge: Some admit that they do notnecessarily have the knowledge to interpret the data provided by accreditedlaboratories. As such, they would be at a certain loss without a CCMC Listing.

Less ease of being approved: Some manufacturers worry that theprocess of having a product approved may be longer as the building official

mandate: By not spending time on the Listings and there-evaluation of Listings, CCMC’s personnel wouldundoubtedly have more time to spend on Reports andtheir re-evaluations.

process of having a product approved may be longer as the building officialmust now spend more time examining the data provided by manufacturersand will be less likely to consider products that they are less familiar with.

No common list of products: Some point out that many certified labs donot necessarily offer a list of the products that they have tested to recognizedstandards and, as such, finding the information on products would be moredifficult.

29

difficult. Indeed, one participant mentioned that he would prefer that ALL listed

products be centralized with CCMC in the future to ensure a “national”reference system for products for which standards exist.


TO CONSIDER

Many of the arguments for CCMC keeping the listing service are based on a broader reaching problem: that ofbuilding officials who do not all have the same level of knowledge and/or resources to carry out evaluations ofg g ythird party laboratory reports to ensure that the products do indeed comply to a recognized standard. This resultsin some building officials relying too heavily on CCMC as a third party resource. As such, the question becomeswhether or not it is truly CCMC’s role and responsibility to address the shortcomings of the building officials.

30


Those who specify: In most provinces the consensus seems to be that those who specify thed t hit t i d i f th b ildi

Who is Involved in Specifying Innovative Products?

products are: architects, engineers or designers of the buildings.

Who is ultimately responsible for approving the use of a product? In most Canadian provinces, the municipal building official is ultimately responsible for approving what the

engineer architect or designer has specified for use in the buildingengineer, architect or designer has specified for use in the building.

In Québec the engineer of a project tends to be the person who is ultimately legally responsible for theapproval of construction materials chosen.

Despite these differences the end result is similar across Canada: that those who are specifying thematerials for innovative products (for which standards do not exist) tend to prefer specifying aproduct with a CCMC number because they feel that it limits their vulnerability to liability.

31

AppendixAppendix

Please click on icon to view a copy of the discussion guide

Discussion guide



Brio Conseils inc.

Page: 16 / 21

APPENDIX F: Summary of September 2009 NRC Workshop, Ad hoc Research

Summary of 2009 NRC SeminarSeptember 28, 2009p

Calgary, Alberta, Canada

Table of contents

Part 1 - Summary of the morning session presentations

Page 4: Overview of CCMC’S business review and product evaluation in Canada: Dr. John Flack, Manager CCMC

Page 6: Overview of product evaluations in New Zealand and Australia: Mr. Paul Shortis, General Manger, SciencePage 6: Overview of product evaluations in New Zealand and Australia: Mr. Paul Shortis, General Manger, Science and Engineering Services, BRANZ Ltd

Page 9: Overview of product evaluations in the United States of America: Mr. John Nosse, President Emeritus, ICC-Evaluation Services Inc.

Page 10: Overview of product evaluations in France – fast-track model: Mr. Bruno Mesureur, Direction Technique CSTB

Part 2 - Summary of Afternoon Working Session

Page 14: How to streamline the evaluation process of innovative products .

Page 17: Identify key success factors when collaborating with other organizations for the recognition of innovativePage 17: Identify key success factors when collaborating with other organizations for the recognition of innovative products.

Page 19: Identify the key success factors to a good communication between regulatory bodies, the proponent, and the technical evaluation organization.

Appendix: electronic version of each of the morning sessions

2

Part 1 :Summary of the Morning Session

Copies of each of the presentations can be found annexed to this document


Introduction: Hosted by the NRC Institute for Research in Construction, this workshop brought together

members from the World Federation of Technical Assessment Organization (WFTAO) and IRCC (Inter-Jurisdictional Regulatory Collaboration Committee) as well as invited participants

ti t k h ld f th C di t ti i d trepresenting stakeholders from the Canadian construction industry. The objective of the workshop was to share experiences and learn from international visitors

about their business models for evaluating innovative construction products. The focus was on what works in their models and problems encountered. The theme of the day was: Process Improvements to Obtain Faster Market Acceptance.

In the morning, presentations were given by representatives from New Zealand/Australia, USA, France, and Canada. There was also a presentation from a Canadian manufacturer to get their perspective. In the afternoon, group discussions took place to get input from participants’ own experience with examples of both the positive and negative aspects of their experience.

Opening remarks: Mr. Ivan Moore, Assistant Deputy Minister, Public Safety Division of the Government of Alberta,

opened the seminar. In his speech, Mr. Moore emphasized the importance of true collaboration to promote innovation. The goal should be to help new products to get into the market as fast and as safely as possible All jurisdictions should support national harmonization of the Building Code Insafely as possible. All jurisdictions should support national harmonization of the Building Code. In that regard, Alberta and British Columbia have already signed an agreement to harmonize their practices.

4


For the past 20 years, the Canadian Construction Materials Centre (CCMC), part of the National Research

Overview of CCMC’S business review and product evaluation in CanadaBy: Dr. John Flack, Manager CCMC

For the past 20 years, the Canadian Construction Materials Centre (CCMC), part of the National Research Council’s Institute for Research in Construction (NRC-IRC), has offered the construction industry a national evaluation service for innovative non-standardized construction products on a cost-recovery basis. CCMC provides an opinion whether a construction product complies with the minimum performance established by the National Building Code. The services offered by CCMC are Evaluation Listings and Evaluation Reports. A listing is issued for products that have demonstrated compliance with a recognized standard. A report is written for product/system for which there are no standards. Over the past few years, some changes have occurred: Objective based National Building Code was introduced in 2005;

Increased in the number of complex products;

Increased number of complaints.

CCMC has responded to these changes by enhancing its services to meet the demands however, proponents still h di th i ff d b CCMChave some concerns regarding the services offered by CCMC: Cost

Time frames

Scope

Increased technical challenges

The CCMC business review is being undertaken to obtain an objective assessment of the current situation identify The CCMC business review is being undertaken to obtain an objective assessment of the current situation, identify issues that need to be addressed and allow NRC-IRC to make changes to the CCMC business model, as required, to meet future expectations.

5


Plasti-Fab Ltd is a manufacturer of moulded expanded polystyrene (EPS) insulation products for

Manufacturer’s experience in the Canadian regulatory systemBy: Mr. James Whalen, Technical Marketing manager, Plasti-Fab Ltd.

Plasti Fab Ltd. is a manufacturer of moulded expanded polystyrene (EPS) insulation products for building construction applications. Vertically integrated from manufacture of expandable polystyrene resin to EPS product incorporated into prefabricated building systems.

Plasti-Fab Ltd. uses both services offered by CCMC: Evaluation Listings and Evaluation Reports. A listing is issued for products that have demonstrated compliance with recognized standards. A report is written for a product/system for which no standards exist.

The challenges specific to each service are the following: Evaluation listings:

Primarily time as evaluation against National Standard

Li it ti f l ti t ll d t d Limitation of evaluation scope not well understood

Evaluation Reports:

Testing required to demonstrate “alternative solution ” is often an unknown factor;

Innovative products require development of evaluation guidelines – i.e. time and cost – that manufacturers have limited input into;

Setting limits on evaluation scope challenging for CCMC and manufacturers;

Time and expense to complete evaluation process and bring innovative products to market leads to frustration with evaluation process.

6


The Building Research Association of New Zealand (BRANZ) was established in 1969 under the

Overview of product evaluations in New Zealand and AustraliaBy: Mr. Paul Shortis, General Manger, Science and Engineering Services, BRANZ Ltd

g ( )Building Research Levy Act. BRANZ’s main activities consist of supporting research, knowledge transfer and services such as a library. BRANZ also does ”commercial work”. BRANZ is designed to support a relatively small manufacturing industry in a relatively unique timber framed construction environment.

I 1992 N Z l d d t d th f b d B ildi C d Thi B ildi C d t In 1992, New Zealand adopted the performance based Building Code. This Building Code sets out ”Acceptable Solutions” making reference to more than 600 NZ standards. If a product does not meet the standard, an ”Alternative Solution” can be identified with a Verification Methodology.

Some important changes have occurred in the past few years: Reduction in building trades that are qualifiedReduction in building trades that are qualified

Rise of “developers”

Building boom

New styles of building – “NZ Mediterranean”

Inadequate skills of Building Consent Authorities

Use of untreated timber

Liability sits with “last man standing” = BCA..

Which leads to a highly conservative environment

Significant barriers to innovation

7


These changes raised the following issues: Difficult to prove Acceptable Solution;

Overview of product evaluations in New Zealand and Australia (cont’d)

Difficult to prove Acceptable Solution;

Costly & slow to produce evidence of an Alternative Solution;

No “quick to market” methodology.

In response to that situation, a new approach has been developed: the Product Evaluation Staircase

This approach has been designed to fulfill the following needs: A framework for risk assessment

A network of qualified/certified assessment agencies

Understanding in the application of each level of riskUnderstanding in the application of each level of risk

Skills back in the industry

A collaborative approach to innovation

Common sense8


ICC Evaluation Services (ES) is a non-profit corporation and a subsidiary of the International Code Council ICC-ES’ business model is similar to the Canadian model and has been designed

Overview of product evaluations in the United States of AmericaBy: Mr. John Nosse, President Emeritus, ICC-Evaluation Services Inc.

Code Council. ICC-ES business model is similar to the Canadian model and has been designed to provide: a process for technical recognition of innovative building products based on life-safety and preservation of

property;

a program addressing the sustainability of building products;

the stated services in a business-like manner to be financially self-sufficient.

To minimize complaints and improve communication, ICC-ES created a Customer Care Unit. This unit stays in touch with the applicant during all aspects of the evaluation process. This unit relieves the project engineer of non-technical work to the degree possible.

The Technical Review Process starts with the establishment of acceptance criteria with input from The Technical Review Process starts with the establishment of acceptance criteria with input from the applicant and outside, recognized experts on the subject. Then, a draft is posted on the ICC-ES website for public comments 30 days before the public hearing. Based on required hearings (several iterations may be required), a committee composed of building authorities having jurisdiction (AHJ) is responsible for approving the acceptance criteria. A report is written and posted on the official ICC-ES website with a notice to the applicant. Because this process is open and transparent, it protects ICC against liability. This process can take anywhere between 4 months to 3 years. The comments and impact of current process are: Challenge of time verses technical responsibility for a successful business model responding to

manufacturer and AHJ needs;manufacturer and AHJ needs;

Acquisition of technically qualified staff and consultants;

Competition which creates positive and negative results.

9


Overview of product evaluations in France – fast-track modelBy: Mr. Bruno Mesureur, Direction Technique CSTB

The CSTB is a state owned public establishment. They offer services in Research, Consultancy, Technical assessment Tests and Factory inspectionTechnical assessment, Tests and Factory inspection.

There are different assessment types depending on product life cycle to support product development (innovation) to product growth. A lot can be done to reduce the evaluation time:

Partnerships with other approval bodies or partial sub-contracting.

Partial reports.p

CSTB has internal test labs but also develops partnership with institutes from other countries. Mixed approach is also used: physical tests/numerical simulation.

Tests data from other approval bodies are more and more accepted for the Atec. The accreditation of the test labs is necessary but not sufficient. Inter laboratories comparison tests must be carried out to assess the confidence intervals and reduce the scatter.

ICC ES recognizes test data and evaluation reports issued by the CSTB. This kind of partnership shall be developed in the future to improve and speed up the assessment process.

Agencies involved in Green have signed a partnership agreement in order to align assessment methods. Can we develop similar co-operation schemes for the voluntary certification of construction products?p

The French low carbon policy will impact the building sector. To be successful, this policy will push innovation at a large scale that has never been seen before. In order to face this new market, the CSTB has developed the Innovation Pass to allow a reduced time to market. One of the objectives is to allow the construction stakeholders to trust the innovative products and processes.

10


The Innovation Pass: it is a one step diagnostic based on a single technical file;

Overview of product evaluations in France – fast-track model (cont’d)

it is a one step diagnostic, based on a single technical file;

the assessment is carried out by CSTB experts, based on an engineering judgment;

it is issued in 3 months;

it is delivered for a period of 2 years and cannot be renewed.

So far, the impact is positive:, p p More manufacturers apply for an Innovation Pass

CSTB expertise is recognized by the professionals (insurance companies, Grenelle de l’environment, supervisors)

Partnership with the main stakeholders

Training of the manufacturers ; Training of the manufacturers ;

Positioning of the IP towards other assessment systems;

Improvement of the communication towards the insurance companies and other construction stake holders.

11

Part 2 :Summary of Afternoon Working Session

Summary of Afternoon Working Session

The afternoon session was a workshop that focused on innovative construction product evaluations in Canada and WFTAO countries. More specifically participants were asked to brain-storm in groups of 7 or 8 on the following subjects:

How to streamline the evaluation process of innovative products .p p

Identify key success factors in collaborating with other organizations for the recognition of innovative products.

Identify the key success factors to a good communication between regulatory bodies, the manufacturer, and the technical evaluation organization.

13

H t St li th l ti f i ti d t


During the morning session, several of the presenters mentioned that proponents tend to complain that the process forevaluating innovative products tends to be long. The purpose of the first exercise in the working session was to identifyareas in which the evaluation of innovative products could be streamlined to improve upon the time line

How to Streamline the evaluation process of innovative products

1. Properly Managing the Expectations of the Proponent Many agreed that proponent complaints that the process takes too long seem to stem from a lack of communication between both parties

areas in which the evaluation of innovative products could be streamlined to improve upon the time line.

The following summarizes the areas that were identified as being subject to possible improvement.

Many agreed that proponent complaints that the process takes too long seem to stem from a lack of communication between both parties in terms of:

What steps need to be addressed

Who will be responsible for each step

The estimated time for each step

This particular point was addressed more in-depth in the third part of the working session (see page 19).

2. Collaboration Between Technical Evaluation Organizations Many agreed that mutual recognition agreements between various technical evaluation organizations could help accelerate the evaluation

process.

This particular point was addressed more in-depth in the second part of the working session (see page 17).

3. Risk OptimizationS l f th ti d th t t h i l l ti i ti h ld d l t id tif th l l f i k i t d Several of the groups mentioned that technical evaluation organizations should develop a means to identify the level of risk associatedwith an innovative product.

It was suggested that there are several innovative products that could be classified as having a marginal risk and as such could be “fasttracked” – similarly to the French model discussed by Bruno Mesureur from the CSTB in the morning session. The products classified ashigh risk would have to undergo a more thorough evaluation process.

14


H t St li th l ti f i ti d t ( t’d)How to Streamline the evaluation process of innovative products (cont’d)

4. Standardize Evaluation Methods Some suggested that standardizing evaluation methods amongst technical evaluation organizations could help

speed up the evaluation process.

Consider a process to enable the acceptance of a previously issued evaluation criteria (e.g.: ICC-ES in the US for the Canadian CCMC).

5. Possible Improvements to the Application Processa. Ensure that the proponent is provided with an application guide. Among other things, this guide should:

list and explain each of the steps that will be undertaken to ensure an evaluation; … list and explain each of the steps that will be undertaken to ensure an evaluation;

… provide an explanation of “preparatory stage” vs. the actual “evaluation stage”;

… clearly indicate the proponent’s responsibility when it comes to the “preparatory stage” – ensure that theproponent understands that the faster they respond during the “preparatory stage”, the faster the overallevaluation process can be.

b C id th id f t t Th id f h i t t tb. Consider the idea of customer care agents. The idea of having a customer care agent to ensure a propercommunication between the technical evaluation organization and the proponent (as is currently done by ICC-ESInc) was widely referred to during the session.

c. Technical evaluation organization should be involved at an earlier stage. Often times, the proponentapproaches the technical evaluation organization at the end of their product development process – once theproduct has been finalised and they are ready to go to market If the technical evaluation organization were toproduct has been finalised and they are ready to go to market. If the technical evaluation organization were tobecome involved earlier, there may be less frustration on the part of the manufacturer in terms of how long it takesto obtain an evaluation. Some suggested that the technical evaluation organization should try and communicatethis to manufacturers.

d. Pre-qualification of the risk involved with the product (see risk optimization on previous page).

When determining the eligibilit of a prod ct for e al ation prod cts co ld be classified as either high or lo When determining the eligibility of a product for evaluation, products could be classified as either high or lowrisk and dealt with accordingly.

e. Attempt to integrate the scope and evaluation plan with the preparation of the technical guide: Whenpresented with the CCMC’s evaluation process, a few suggested that these two steps be streamlined into one asthere appears to be a certain amount of overlap between these two steps (ex.: code analysis).15

H t St li th l ti f i ti d t ( t’d)


6. Possible Improvements to the Preparation of the Technical Guide a. More collaborative process: Some suggested that the technical guide should be prepared with more input from

the manufacturer to ensure that the manufacturer is “buying into” how the technical guide is being developed.

How to Streamline the evaluation process of innovative products (cont’d)

Some even suggested that a third-party technical committee be in charge of determining the current codeguidelines that may or may not apply to the innovative product in question.

b. Use of third-party consultants: If lack of resources is an issue, in terms of being able to do the researchnecessary to prepare the technical guide, some suggested that some of the legwork could be outsourced to athird-party consultant.

c Properl determine the se of the prod ct in q estion Man fact rers ill often tr to obtain an e al ation forc. Properly determine the use of the product in question: Manufacturers will often try to obtain an evaluation forthe broadest possible use without realizing that the broader the use, the more complex and lengthy the testing willbe. In the same realm, the suggestion that material characteristics and optional characteristics be properly

identified so as to be able to test the base product first with the option of adding on the optionalcharacteristics.

7. Post Report Quickly Attempt to instate a 30 day policy: 30 days after results are received, a report must be posted. A few suggested that the report should be posted even though it has not been translated into other official

languages.

16


The purpose of the second exercise was to identify specific key success factors in terms of having technical evaluation organizations from various countries collaborate. The following summarizes the areas that were identified as being subject to possible improvement.

Collaborating with Other Technical Evaluation Organizations

The key success factors of such a collaboration are:

1. Development of common assessment criteria and methods of assessment.

2. Ensuring that there is a common access to information in the form of: central data base

common manner in which data is stored

3. Develop standardized results by:St d di i th t ti f t t lt b th t h i l l ti i ti Standardizing the presentation of test results by the technical evaluation organizations

Establishing a protocol for validation of non-standardized tests (try to harmonize technical requirements).

Defining climate zones worldwide to establish product performance criteria accordingly (by climate zones)

Obtaining a global accreditation of laboratories (ILAC: SCC, IAS, etc.). All should refer only to test data thatcome from accredited sources (EA, ILAC, etc.)

Developing a base document that all members need to fill out so that any country can refer to the information in astandardized format and can pick and chose the information they will need/use.

4. Ensure access to other technical evaluation organizations’ previous experience with similarproducts. This can be achieved through a common data base, or simply having access to the peopleand being able to ask them questions (what were the main concerns were there any surprises etc )and being able to ask them questions (what were the main concerns, were there any surprises, etc.).

5. Develop a collaboration agreement with other countries. These agreements could be bi-lateral and/ormulti-lateral.

17


Collaborating with Other Technical Evaluation Organizations (cont’d)

There are some barriers that were identified and that would need to be addressed in order for aninter-country collaboration to be successfuly

Information data is difficult to access and often not transferable electronically

Difficulty of translating country regulatory requirements into product characteristics

There is no common format for test results, as such, it is often impossible to read a document / report / data from another jurisdiction.p j

Language Barriers

Different levels of expertise between jurisdictions

Differences in installation procedures between countries.

Standards vary considerably from one country to the nextStandards vary considerably from one country to the next

Testing methods vary considerably from one laboratory facility to the next

Political support required – if there is no political will for such collaboration, it will be difficult for the regulatory bodies to obtain the financial support from their government needed for this type of collaboration.

18


The purpose of the third exercise was to identify specific key success factors in terms of improving communication with proponents. The following summarizes the areas that were identified as being subject to possible improvement.

Key areas for improvements in the communication with proponents

1. Ensure that there is a designated person responsible from both the technical evaluationorganization and the proponent.

2 I t t j t ( l f d t t ) h i i h f2. Instate a project manager (also referred to as customer care person) who is in charge ofcommunications with the proponent. This person would… …be the first person (and preferably the only one) who is in contact with the proponent. Some mentioned that the

first meeting should preferably be in a face-to-face meeting to establish a business relationship;

…inform the proponent of the whole process:p p p

How many steps are involved in the process

What each step entails

How long each step can take

Who is responsible for each step

C t th t b i l d Costs that can be involved;

… be in charge of dealing with any complaints or questions that the proponent may have;

…provide the proponent with a timeline that is specific to their project with specific deadline dates associated tothe various steps along with who is responsible for each of the steps;

… communicate on a regular basis (at least monthly) with the proponent to validate their interest in continuing withthe process, follow-up on the status of what each party needs to be doing.

19


Who is the project manager? Most believed that the project manager should be someone who…h i t l

Key areas for improvements to communication with the proponent (cont’d)

• … has a senior management role• … has people skills• … acts as a facilitator between the technical evaluation organization and the proponent• … is knowledgeable of the technical side of the evaluations• … is different from the personnel in charge of the technical evaluations; many suggested that the

project manager should be a coordinator while technical evaluations should be coordinated

3. Create an accountability protocol to ensure that the proponent is aware of what is under his/herresponsibility.

E th t th i d dli i t d ith h t

internally by another team of personnel specialised in the evaluation portion.

Ensure that there is a deadline associated with each step.

Ensure that the technical evaluation organization / project manager follows-up with the proponent ifdeadlines are not met and communicates how this delay will affect the overall evaluation process.

4. Ensure that there is an impartial ombudsman or official appeals process such that the proponentcan easily communicate any dissatisfactions.y y

20

Click on the presentation that you wish to view.

Overview of CCMC’S business review and product evaluation in CanadaBy: Dr. John Flack, Manager CCMC

Overview of product evaluations in New Zealand and AustraliaBy: Mr. Paul Shortis, General Manger, Science and Engineering Services, BRANZ Ltd

Overview of product evaluations in the United States of AmericaBy: Mr John Nosse President

Overview of product evaluations in France – fast-track modelBy:Mr Bruno Mesureur Direction

Présentation Microsoft PowerPoint


By: Mr. John Nosse, President Emeritus, ICC-Evaluation Services Inc.

By:Mr. Bruno Mesureur, Direction Technique CSTB





Brio Conseils inc.

Page: 17 / 21

APPENDIX G: Governance Review of the CCMC, Institute On Governance

Governance Review of the Canadian Construction Materials Centre of the National Research Council (NRC-CCMC) Phase 3 Final Report March 29, 2010 Prepared by Laura Edgar Institute On Governance

© Copyright, year, Institute On Governance (if applicable)

For further information, please contact:

[IOG 2010-5107]


Laura Edgar Institute On Governance

122 Clarence Street Ottawa, Ontario, Canada K1N 5P6

tel: 1 (613) 562-0090 fax: 1 (613) 562-0097

[email protected]

The contents of this paper are the responsibility of the author(s) and do not necessarily reflect a position of the IOG or its Board of Directors.

The Institute On Governance (IOG) is a Canadian, non-profit think tank that provides an independent source of knowledge, research and advice on governance issues, both in Canada and internationally. Governance is concerned with how decisions important to a society or an organization are taken. It helps define who should have power and why, who should have voice in decision-making, and how account should be rendered. Using core principles of sound governance – legitimacy and voice, direction, performance, accountability, and fairness – the IOG explores what good governance means in different contexts. We analyze questions of public policy and organizational leadership, and publish articles and papers related to the principles and practices of governance. We form partnerships and knowledge networks to explore high priority issues. Linking the conceptual and theoretical principles of governance to the world of everyday practice, we provide advice to governments, communities, business and public organizations on how to assess the quality of their governance, and how to develop programs for improvement.

You will find additional information on our activities on the IOG website at www.iog.ca

NRC-CCMC Governance Review Phase 3FINAL REPORT Prepared by the Institute On Governance 1

Governance Review of the Canadian Construction Materials Centre of the National Research Council (NRC-CCMC) – Phase 3 Final Report

I. Background

For the past 20 years, the Canadian Construction Materials Centre (CCMC), part of the National Research Council's Institute for Research in Construction (NRC-IRC), has offered the construction industry a national evaluation service for innovative non-standardized construction products on a cost-recovery basis.

The National Research Council engaged Brio Conseil to conduct a business review of the Canadian Construction Materials Centre (NRC-CCMC). In conjunction with the business review, the National Research Council also commissioned a governance review. The purpose of the governance review was to examine certain aspects of the governance of the NRC and the CCCME to ensure that they are in line with contemporary principles and exemplary practices.

The governance review has run concurrently with the business review. During Phase 2 of the governance review the Institute completed a document review of CCMC governance arrangements, completed two short case studies of other governance models and completed eight one-on-one interviews, including representatives from NRC management, NRC legal services, the CCCME Chair and CCMC provincial stakeholders regarding the current governance arrangements of CCMC as well as options for improvement. The results of the review, including strengths and weaknesses of the current model, options for governance improvement under the current model, and additional options for consideration, were summarized in a Phase 2 report to NRC-CCMC submitted in February 2010.

The objective of Phase 3 of the governance review is, based on the results of the governance review and an examination of the Brio Conseil business review results, to provide a final set of recommendations for governance improvement for consideration by NRC.

II. Summary of the Strengths and Weaknesses of the Current Governance Model

CCMC, as it is currently situated within NRC, has a reasonable set of governance structures, processes and policies, many of which reflect best practices in the governance of organizations. These policies and procedures extend to those that define NRC-CCMC’s relationship with its advisory body, the CCCME. Moreover, these processes and policies appear to be consistently followed in practice. As noted earlier, and as has been found in the business review, there appears to be strong support for product evaluation work to continue to be housed within NRC. NRC-CCMC, as an independent, third-party with a public purpose, is well-positioned to find a balance between


supporting innovation and managing risk. Other strengths of the current model include CCMC staff expertise and quality of work, CCMC’s ability to access external expertise through the Canadian Commission on Construction Materials Evaluation (CCCME) and its Standing Committee on Technical Evaluations (SCTE), the resources provided by NRC to support the evaluation work, and the belief that CCMC is a source of independent and ‘neutral’ advice, separate from manufacturers, builders and regulators. Finally, the “CCCME Policies and Procedures 2007” manual is a clear and thorough document that provides comprehensive information and guidance on the work of CCMC and the role of the CCCME. While the NRC-CCMC has a range of good governance strengths, the governance review also identified some governance challenges to be addressed. Most of these identified challenges, including communication to end-users, managing risk and the timeliness of service delivery, and better engagement with the provinces and territories, were also observed in the business review. Two other challenges, related to the roles and engagement of the CCCME and the SCTE, were identified through the governance review.

A. Communication As identified in the results of the business review, and supported by the interviews undertaken through the governance review, CCMC has significant challenges with effectively communicating to, and educating, the end-users (municipal building officials, builders, designers etc) of CCMC evaluation reports and listings. There are clear indications from both the governance and business reviews that end-users at the regional / municipal level do not always understand the difference between a report and a listing, and often believe that CCMC opinions on products amount to an ‘approval’ or a certification of the product, when in fact a CCMC opinion is neither. Nor are there any mechanisms, beyond limited engagement on the CCCME and the SCTE, to seek input from these key stakeholders on evaluation processes and products.

In addition, the business review also identified a lack of awareness or knowledge of the Commission and its advisory role to the CCMC. In fact, the CCCME and SCTE, although key components of the decision-making framework, were seldom mentioned by participants in the business review.

B. Risk Management Accountability and liability are directly linked to who does the work and who makes decisions regarding the work. One component of managing liability is to manage risk, and NRC-CCMC has recently introduced some components of a risk management tool. However, the consistency of the results of the tool, and the application of results, are unclear since it is left up to the judgment of individual staff members. There is no internal review panel in place to ensure oversight and consistency in risk management decisions, nor is input sought from the CCCME or its Standing Committee on Technical Evaluations.

C. Clarifying Role and Engagement of CCCME Currently the CCCME meets once a year. A number of interviewees questioned the infrequency of meetings, both in terms of CCMC being able to receive effective advice with such limited engagement and the difficulty in keeping the focus of Commission members. In addition, some interviewees wondered whether CCMC could be making better use of the expertise at the


CCCME table. Several interviewees expressed a desire for more communication between meetings.

D. Engagement of Standing Committee on Technical Evaluations While there are guidelines, there is no formal policy or protocol that governs decisions on when CCMC should engage the Standing Committee on Technical Evaluations (SCTE) on Technical Guides. In addition, unlike the Commission, the SCTE does not, in practice, work to consensus but rather provides web-based input to CCMC on a member by member basis. This less cohesive approach to seeking input from the Committee potentially limits dialogue on technical evaluation issues and therefore the effectiveness and clarity of the advice given to CCMC. Finally, the SCTE is not engaged in evaluating the results of testing against the Technical Guides or in reviewing evaluation reports.

E. Engaging the Provinces and Territories CCMC has many stakeholders whose voices should be reflected in decision-making, including manufacturers, municipal building officials, builders and designers, and the provinces and territories. The provinces and territories, as the regulators who use CCMC opinions to take decisions on products,1 should also have significant engagement and influence on CCMC processes. While some provincial interviewees were satisfied with the current level of engagement and influence, others believe that more frequent and comprehensive engagement, as well as transparency, with the provinces and territories is desirable, and that the provinces and territories should have greater influence on decisions regarding evaluation processes.

In addition, consistent with the results of the business review, many interviewees expressed concern with the time required for CCMC to complete evaluations, as well as the challenges placed on smaller manufacturers as a result of the time and cost of CCMC evaluations. These issues are two of the main reasons that the provinces and territories would like more influence. However, the provinces and territories also recognize that liability concerns and the need for due diligence within NRC-CCMC are significant drivers of the current process, and that the timeliness of evaluations is a challenge for product evaluation services in many other countries as well.

III. Recommendations for Governance Improvement under the Current Model

Good governance rests on a framework of structures, policies, processes and traditions that define how power is allocated, who has voice or input into decision-making, how key relationships are maintained, and how decision-makers are held to account. In governance, form should follow function. The governance model for CCMC must therefore be effectively linked to the objectives of NRC-CCMC and the Government of Canada, and the business model to the governance of CCMC. As the results of the business review are now complete and are consistent

1 The results of the business review indicate that in Quebec the decisions on design and construction, and the associated liability, rest with the designers and builders.


with the findings of the governance review, the Institute offers the following recommendations for consideration to improve the governance of the CCMC. As noted earlier, there appears to be strong support for product evaluation work to continue to be housed within NRC. NRC-CCMC, as an independent, third-party with a public purpose, is well-positioned to find a balance between supporting innovation and managing risk. Other strengths of the current model include CCMC staff expertise and quality of work, CCMC’s ability to access NRC research and expertise, the resources provided by NRC to support the evaluation work, and the belief that CCMC is a source of independent and ‘neutral’ advice, separate from manufacturers, builders and regulators. While the governance of the CCMC is strong overall, the Institute has a number of recommendations for improvement under the current model. In addition, where Brio recommendations have clear governance implications, these are identified and supported as well.

1. Improve communications to end-users. Regular proactive communication with, and education of, user groups of CCMC products is required, and in particular communication of the fact that the CCMC offers opinions and not approvals. In addition, greater transparency in the evaluation process, perhaps through a web-based tracking model, would improve understanding and engagement by clients and other stakeholders in the evaluation process. While communications is primarily a management rather than a governance role, the CCMC governing body has a responsibility to provide oversight of this communications function because it also acts as a risk management tool and improves transparency.

In addition, more effort should be made to communicate the existence and role of the CCCME, as its value-added to the evaluation process is significant and has the potential to be more so. The Institute also supports the Brio recommendations related to clearly defining and communicating timeframes, costs and deliverables for the evaluation services offered. This Brio recommendation is consistent with the Institute’s own recommendations for improved communication to end-users, and is a way of further improving transparency and accountability.

2. Enhance risk management practices. While CCMC has implemented some aspects of a

risk management tool, more can be done. First, the Institute recommends that NRC-CCMC, in collaboration with the CCCME, develop a strong risk management protocol that speaks to the balance of innovation and risk. The Institute also recommends that, at a minimum, an internal, committee-based, oversight mechanism be established to ensure consistency in the application and implementation of results from the risk management protocol used by staff. NRC-CCMC may wish to consider having the SCTE provide input on, or review, risk management decisions. However, recognizing that SCTE members are volunteers with limited time for engagement, NRC-CCMC could consider other options, including (a) offering an honorarium for service to SCTE members; (b)


creating a second Standing Committee to deal with Risk Management; or (c) consider engaging external advice on risk management issues.

The Institute also supports several of the Brio recommendations under “improve the evaluation and decision-making process” including developing a means to identify the level of risk associated with an innovative product and adjust evaluation processes accordingly, and clearly defined collaborations with other organizations to access additional external expertise. These two Brio recommendations are useful risk management tools and will help better manage potential liabilities. The Brio recommendation to consider the creation and/or use of other NRC-CCMC accredited bodies to conduct some of CCMC’s work is another interesting option for improving the efficiency and timeliness of service delivery. The implications of this recommendation for risk management are unclear since while CCMC may do fewer evaluations in-house there may be some risk associated with being an accreditation body. This issue would need to be explored further.

3. Improve engagement and communications to CCCME. The limited number of interviews

completed with CCCME members and others seem to indicate some desire for more frequent engagement between CCCME and CCMC and/or better utilization of CCCME expertise by CCMC. At a minimum, the Institute recommends that CCMC provide CCCME members with more regular communication on its work, and that CCMC also consider the types of issues that are brought before the Commission and whether the expertise of the Commission could be more fully drawn upon. For example, the Commission could play a useful role in contributing to the development of an NRC-CCMC risk management protocol that balances innovation and risk. NRC-CCMC may also wish to consider conducting a brief survey, or holding a short facilitated session with CCCME members, to gather and assess how members feel they can and could contribute to the work of CCMC.

4. More effective use of the CCCME Standing Committee on Technical Evaluations. As

noted earlier, the Standing Committee on Technical Evaluations does not, in practice, work to consensus but rather provides web-based input to CCMC on a member by member basis. In addition, while there are guidelines, there is no protocol or policy governing what evaluations the Standing Committee is to be engaged on. The Institute therefore recommends that such a protocol be developed and implemented. The Institute also recommends that additional web-based mechanisms be put in place to allow all members of the SCTE to have a dialogue on the evaluation criteria in a Technical Guide and to come to some consensus on the advice given to CCMC. Finally, NRC-CCMC may wish to consider engaging the Standing SCTE on the review of test results and evaluation reports, particularly for those that are considered high risk.

The Institute supports the Brio recommendation to review the role of the CCCME and the SCTE in the evaluation process, particularly with respect to approval of technical guides and reports. This recommendation is consistent with the Institute’s own recommendations, and would help improve the consistency, impartiality and transparency


of the processes, increase the use of already available expertise, and better manage the risks associated with evaluations.

5. More effective engagement with Provinces and Territories. The limited number of

interviews with provincial representatives indicate a desire for increased communication, engagement and influence on the work of CCMC. One interviewee suggested that the Commission should have a structure that better reflects provincial perspectives. That said, it is unclear whether increased representation on the CCCME is the best way to address this concern, and other interviewees had no other specific recommendations on how to address the issue. Greater transparency in the evaluation process, perhaps through a web-based tracking system, may be part of the answer. The Institute recommends that NRC-CCMC consult with all of the provinces and territories to discuss the best method and frequency of seeking their input into evaluation processes. Greater engagement with the Provincial/Territorial Policy Advisory Committee (PTPACC) or some other provincial / territorial advisory committee may be one option. A Provincial/Territorial Standing Committee of the CCCME may be another. Regardless of the mechanism chosen, MOUs with the provinces and territories will then have to be adjusted to reflect any agreed changes in the relationship.

IV. Closing Remarks The Institute believes that implementing the above recommendations will improve the effectiveness of the governance of the CCMC under its current governance model. In some cases the recommendations will also help in mitigating the risks associated with evaluation work by improving communication to stakeholders, better involving the input of external expert advice and enhancing risk management practices. The Institute recognizes that more engagement may also lead to higher costs, although additional costs could be minimized through the use of web-based communication technologies. While this report provides recommendations for improving governance under the current model only, it is worth noting that a number of interviewees of the governance review were interested in exploring other models. Several options for other models were put forward in the Institute’s Phase 2 report. However, additional research into the specific nature of the liability concerns to NRC-CCMC would need to be completed before considering a move to another governance model. In addition to issues of liability, NRC-CCMC and its key stakeholders would need to explore and come to some form of agreement on a range of governance issues, including how to best engage key stakeholders, how to maintain independence and transparency, and how to ensure a continued balance between innovation and risk. That said, in the longer-term there are a number of other governance models that may do more to address liability issues, and associated potential media coverage, that are of concern to NRC-CCMC.

Governance Review of the Canadian Construction Materials Centre of the National Research Council (NRC-CCMC) Phase 2 Report February 1, 2010 Prepared by Laura Edgar Institute On Governance

© Copyright, year, Institute On Governance (if applicable)


[IOG 2010-5107]


Laura Edgar Institute On Governance

122 Clarence Street Ottawa, Ontario, Canada K1N 5P6

tel: 1 (613) 562-0090 fax: 1 (613) 562-0097

[email protected]

The contents of this paper are the responsibility of the author(s) and do not necessarily reflect a position of the IOG or its Board of Directors.

The Institute On Governance (IOG) is a Canadian, non-profit think tank that provides an independent source of knowledge, research and advice on governance issues, both in Canada and internationally. Governance is concerned with how decisions important to a society or an organization are taken. It helps define who should have power and why, who should have voice in decision-making, and how account should be rendered. Using core principles of sound governance – legitimacy and voice, direction, performance, accountability, and fairness – the IOG explores what good governance means in different contexts. We analyze questions of public policy and organizational leadership, and publish articles and papers related to the principles and practices of governance. We form partnerships and knowledge networks to explore high priority issues. Linking the conceptual and theoretical principles of governance to the world of everyday practice, we provide advice to governments, communities, business and public organizations on how to assess the quality of their governance, and how to develop programs for improvement.

You will find additional information on our activities on the IOG website at www.iog.ca

Governance Review of the Canadian Construction Materials Centre of the National Research Council (NRC-CCMC)

Table of Contents

I. BACKGROUND 1

Introduction 1

Objectives & Approach 1

Organization of Report 3

II. FINDINGS 3

The Current Model 3

Strengths of the Current Model 5

Identified Challenges of the Current Model 6

III. LESSONS FROM OTHER JURISDICTIONS 8

BRANZ 8

ICC-Evaluation Service (ICC-ES) 8

Comparative Review of Key Elements of CCMC, BRANZ and ICC-ES 9

IV. OPTIONS FOR CONSIDERATION 10

Annex A – Interview Guide 14

Annex B – Case Studies 15

NRC-CCMC Governance Review REPORT Prepared by the Institute On Governance 1

Governance Review of the Canadian Construction Materials Centre of the National Research Council (NRC-CCMC)

I. Background

Introduction For the past 20 years, the Canadian Construction Materials Centre (CCMC), part of the National Research Council's Institute for Research in Construction (NRC-IRC), has offered the construction industry a national evaluation service for innovative non-standardized construction products on a cost-recovery basis. CCMC provides an opinion on whether a construction product complies with the minimum performance established by the National Building Code (NBC).

CCMC’s opinions are derived through the development of technical requirements that provide performance requirements for a particular product and systems based on the minimum acceptable measures established by the NBC. CCMC opinions on compliance expressed through their reports have gained a high level of credibility and recognition both in Canada and in International markets. The reports are used by stakeholders to facilitate market access to these innovative construction products by many professionals, including architects, specifiers, designers, regulators and owners.

In the past few years, CCMC has become aware of pressures and demands related to their businesses that have initiated a reflection on its role, mandate, continued relevance of its services and relationship with the construction community. The National Research Council engaged a consulting firm to conduct a business review of the Canadian Construction Materials Centre (NRC-CCMC). The Canadian Commission on Construction Materials Evaluation (CCCME) and NRC management will use the study’s findings to set the future direction for the centre and make necessary changes to meet stakeholders’ short- and long-term needs.

In conjunction with the business review, the National Research Council also commissioned a governance review. Some of the questions and issues raised with the current governance model include the risks and liability to NRC in the provision of its services, the obligations upon NRC as a federal government research organization, the ability of NRC-CCMC to manage pressures and attention from the media and politicians and the influence of key stakeholders on decisions regarding CCMC services and operations.

Objectives & Approach The purpose of the governance review was to examine certain aspects of the governance of the NRC and the CCCME to ensure that they are in line with contemporary principles and exemplary practices. This review was conducted on a confidential basis.


The governance review has run concurrently with the business review conducted by Brio Consulting. To date, Phases 1 and 2 of the governance review have been completed. Phase 1 of the governance review included participation in an international workshop in Calgary, and the inclusion of a governance question in the business review survey. The governance-specific results of the survey, while limited, were reviewed by the Institute and discussed with the CCMC Business Review Project Team.

In Phase 2, the Institute completed a document review of CCMC governance arrangements, completed two short case studies of other governance models and completed eight (8) one-on-one interviews, including representatives from NRC management, NRC legal services, the CCCME Chair and CCMC provincial stakeholders regarding the current governance arrangements of CCMC as well as options for improvement. The interview guide can be found in Annex A. The results of this report are based on the document review and interviews conducted as part of Phase 2 of the governance review. The analysis provided in this report, along with any recommendations and options to be explored, are based on the Institute’s knowledge and experience working on governance issues of public-purpose organizations and is not a legal analysis. Governance and Good Governance The Institute On Governance defines governance as the process whereby societies or organizations make their important decisions, determine who has voice, who is engaged in the process and how account is rendered. Good governance rests on a framework of structures, policies, processes and traditions that define how power is allocated, who has voice or input into decision-making, how key relationships are maintained, and how decision-makers are held to account. Effective governance ensures objectives are realized, resources are well managed, and the interests of stakeholders are reflected in key decisions. The Institute On Governance works from a set of good governance principles. According to this approach, good governance exists where those in positions of power are perceived to have acquired their power legitimately, and there is appropriate voice accorded to those whose interests are affected by decisions. Further, the exercise of power results in a sense of overall direction that serves as a guide to action. Governance should result in performance that is responsive to the interests of citizens or stakeholders. In addition, good governance demands accountability between those in positions of power and those whose interests they are to serve. Accountability cannot be effective unless there is transparency and openness in the conduct of the work being done. And finally, governance should be fair, which implies conformity to the rule of law and the principle of equity. In exploring these principles it is important to note two points. First of all, it is not enough to meet only some of the principles; all of the principles need to be present, to at least some degree, to ensure good governance. Second, there are inherent conflicts within the principles (for example, between performance and accountability) and it is importance to find balance among them.


Organization of Report The report is organized into five parts. Following the introduction, Part II summarizes the findings of the governance review, which include the results of the interviews and document review, and outlines some of the strengths and challenges of the current model. Part III provides a brief synopsis of two other models – one from New Zealand and another from the United States – and compares key elements of these models to CCMC. Finally, Part IV provides some options for consideration.

II. Findings

The Current Model The Canadian Construction Materials Centre (CCMC) is a part of the National Research Council's Institute for Research in Construction. The CCMC offers the construction industry a national evaluation service for innovative, non-standardized materials, products, systems and services in all types of construction. CCMC will also evaluate products that are covered under the scope of recognized standards. An evaluation is an impartial, technical opinion. On the building side, that opinion often relates to the requirements of the National Building Code of Canada and of provincial codes. For products that have specific standards industry often relies on CCMC's evaluations to demonstrate conformity. Opinions on these products are published in Evaluation Listings. For products that have no specific standards CCMC provides evaluations, which are undertaken for products that are manufactured, assembled, or processed at a particular site or manufacturing facility and the evaluation applies only to those products. The resulting Evaluation Reports and Evaluation Listings specify the ways in which the products are to be used for the evaluation to be valid. CCMC provides an opinion on whether a construction product or system complies with the requirements of the National Building Code of Canada. Manufacturers, regulators and others all benefit from the service, which is recognized by all provincial and territorial building regulatory bodies across the country. While some of the larger Canadian provinces also provide their own provincial evaluation process, CCMC has become the main provider of evaluation services in Canada. Some have expressed concern that there is over-reliance on the services that CCMC provides. Members of the Provincial/Territorial Policy Advisory Committee on Codes (PTPACC), composed of senior officials from provincial and territorial departments responsible for building regulation, recognize CCMC evaluations as a useful basis for determining the acceptability of products within the context of building code requirements under their jurisdictions. The acceptance of individual CCMC evaluations is made on a case by case basis by the appropriate jurisdiction. No jurisdiction is bound to accept CCMC reports.


Governance of CCMC The governing body of the CCMC is NRC, and, ultimately, the Government of Canada. CCMC is responsible for most aspects of the evaluation of products, including developing criteria for the evaluation of products, reviewing and analyzing test results, and providing an opinion on whether the product complies with the requirements of the National Building Code. The actual testing of the product against the set criteria is conducted by independent laboratories.

NRC-CCMC has an advisory board – the Canadian Commission on Construction Materials Evaluation (CCCME). CCMC “operates under the general policy and technical advice”1 of the CCCME. CCCME has a membership representing regulators, industry and general interest categories and is a source of independent advice that supports decision-making with NRC-CCMC. The CCCME operates under the “CCCME Policies and Procedures 2007”, which provides detailed terms of reference, policies and operating procedures for the CCCME. The Commission currently meets approximately once a year and its work focuses on broader policy issues of CCMC related to risk management, quality control and information management and sharing. While the CCCME provides advice on policy issues, all policy decisions are taken by NRC-CCMC. Secretariat support for the CCCME is provided by CCMC.

The Commission also has the capacity to act as an informal, neutral appeals mechanism for those who are in some way dissatisfied with evaluation processes or decisions taken by CCMC. However, as the Commission is advisory only, their role in appeals is to review and provide advice to NRC. NRC is responsible, and accountable, for any decisions taken.

The Commission has an Executive Committee which meets once a year about ½ way through the year to address any emerging issues. Any decisions taken by the Executive Committee are ratified by the full Commission at their next meeting.

The Standing Committee on Technical Evaluations of the CCCME is charged with reviewing CCMC Technical Guides and providing guidance on other technical issues that may arise. While the membership of the Executive Committee is drawn from the Commission, members of the Standing Committee on Technical Evaluations (SCTE) are appointed primarily based on technical expertise rather than representation. Appointments to the SCTE are made by the Chair of the CCCME on the recommendation of the Executive Committee.

CCMC has a guideline that specifies when CCMC staff are to engage the SCTE. This guideline indicates that the SCTE is engaged in the review of all new Technical Guides and Scope and Evaluation Plans, and that the SCTE also reviews all major modifications or upgrades to Technical Guides. Finally, the SCTE may be requested to review Technical Guides where the staff have identified issues for which additional technical advice would be of value. CCMC staff work through and with the support of the CCMC Manager to engage the SCTE when seeking their input on both reviews of major modifications to Technical Guides or in seeking SCTE advice on specific issues.

The SCTE does not meet in person or according to any regular schedule. Rather, each Committee member is asked to provide their individual input on the development of CCMC Technical Guides via electronic means. While Committee members do see each other’s input,

1 “CCCME Policies and Procedures”, Canadian Commission on Construction Materials Evaluation, National Research Council of Canada, 2007. p. 8.


there is, in practice, little online dialogue or exchange on evaluation criteria. The SCTE is not engaged on all evaluations, and decisions on when to engage the SCTE are made by CCMC.

Strengths of the Current Model

The results of the interviews revealed an overall belief that CCMC and the CCCME function well and provide a valuable service to their key stakeholders (manufacturers, the provinces and territories, municipal building officials and builders). Interviewees highlighted a range of strengths of the current governance structure that demonstrate the principles of good governance. There is a strong and clearly-defined relationship between the CCCME and NRC, with a representative of the Institute for Research attending Commission meetings, and an annual report from the Commission submitted to the NRC. Efforts made to link CCMC work to other related activities, including the work of the Canadian Commission on Building and Fire Codes, as well as to international standards and codes, were noted as a strength by a number of interviewees.

In keeping with the initial results of the CCMC Business Review, interviewees generally expressed strong support for product evaluation work to continue to be housed within NRC. NRC-CCMC, as an independent, third-party with a public purpose, is well-positioned to find a balance between supporting innovation and managing risk. Some of the reasons given by interviewees for their support of CCMC include CCMC expertise and quality of work, CCMC’s ability to access NRC research and expertise, the resources provided by NRC to support the evaluation work, and the belief that CCMC is a source of independent and ‘neutral’ advice, separate from manufacturers, builders, municipal building officials and provincial/territorial regulators. There was also recognition that NRC-CCMC has taken on the liability associated with conducting evaluations.

Interviewees highlighted a large range of strengths related to the functioning of the CCCME. These strengths include the broadly representative nature of the Commission, consensus-based decision-making, and fair, thorough and frank discussion at Commission meetings. Participation is considered to be generally good, with individual Commission members participating more or less in any particular discussion based on their knowledge and skills. Commission meetings are open to the public, though attendance is generally limited and this opportunity is not actively promoted. Meetings are considered well-run and agendas put together with input from CCCME members and CCMC staff. Finally, there is an effective working relationship between the CCCME and CCMC, with generally reasonable timeliness of materials provided to the Commission and responsiveness to Commission inquiries and requests.

The “CCCME Policies and Procedures 2007” manual is a clear and thorough document that provides comprehensive information and guidance on the work of CCMC and the role of the CCCME. The manual also specifies the policies and procedures that the CCCME must follow, including scope of work, mode of operation, membership, conflict of interest, responsibilities of the CCCME and guidelines for liaison with other organizations. The manual outlines the membership, roles and responsibilities of the Executive Committee, Standing Committees and Task Groups, as well as requirements for meetings, minutes and other administration. The membership matrix for the CCCME, along with Terms of Reference for both the SCTE and the Chair’s Selection Committee are also provided.


Identified Challenges of the Current Model

A. Communication As identified in the business review, and supported by the interviews undertaken through the governance review, CCMC has significant challenges with effectively communicating to, and educating, the end-users (municipal building officials, builders, designers etc) of CCMC evaluation reports and listings. There are clear indications from both the governance and business reviews that end-users at the regional / municipal level do not always understand the difference between a report and a listing, and often believe that CCMC opinions on products amount to an ‘approval’ or a certification of the product, when in fact a CCMC opinion is neither. Nor are there any mechanisms, beyond limited engagement on the CCCME and the SCTE, to seek input from these key stakeholders on evaluation processes and products.

In addition, Phase 1 of the business review also identified a lack of awareness or knowledge of the Commission and its advisory role to the CCMC.

B. Risk Management Accountability and liability are directly linked to who does the work and who makes decisions regarding the work. As both the work completed on evaluations – with the exception of independent laboratory testing – and the decisions regarding CCMC work are taken within NRC-CCMC, liability for the evaluation work also rests with NRC-CCMC. One component of managing liability is to manage risk, and NRC-CCMC has recently introduced some components of a risk management tool. This tool assists individual CCMC staff members in identifying and categorizing the potential level of risk of any particular product, and encourages use of the results to help determine the scope and depth of a product evaluation. While staff can seek input and advice from one another and CCMC management on risk management, it is not a requirement. The consistency of the results of the tool, and the application of results, are therefore unclear since it is left up to the judgment of individual staff members. There is no internal review panel in place to ensure oversight and consistency in risk management decisions, nor is input sought from the CCCME or its Standing Committee on Technical Evaluations.

C. Clarifying Role and Engagement of CCCME Currently the CCCME meets once a year. While interviewees agreed that the time allotted for meetings was sufficient given the business brought before the Commission, a number of interviewees questioned the infrequency of meetings, both in terms of CCMC being able to receive effective advice with such limited engagement and the difficulty in keeping the focus of Commission members. Related, several interviewees questioned some elements of the current agenda, which has a significant amount of reporting, and wondered whether CCMC could be making better use of the expertise at the CCCME table. Several interviewees expressed a desire for more communication between meetings.


Finally, while the “CCCME Policies and Procedures” manual is a strong document, some of the language could give the impression that the CCCME has more decision-making influence than is, in fact, the case. For example, the Policies and Procedures manual states that CCMC operates under the general policy and technical advice of the CCCME. “Operates under” could imply a responsibility for CCMC to follow the advice of the CCCME.

D. Engagement of Standing Committee on Technical Evaluations While there are guidelines, there is no formal policy or protocol that governs decisions on when CCMC should engage the Standing Committee on Technical Evaluations (SCTE) on Technical Guides. In addition, unlike the Commission, the SCTE does not, in practice, work to consensus but rather provides web-based input to CCMC on a member by member basis. This less cohesive approach to seeking input from the Committee potentially limits dialogue on technical evaluation issues and therefore the effectiveness and clarity of the advice given to CCMC. Finally, the SCTE is not engaged in evaluating the results of testing against the Technical Guides or in reviewing evaluation reports.

E. Engaging the Provinces and Territories CCMC has many stakeholders whose voices should be reflected in decision-making, including manufacturers, municipal building officials, builders and designers, and the provinces and territories. For manufacturers, their voice is heard through the evaluation process. For municipal building officials and builders there are some significant challenges with communications, as noted earlier. The provinces and territories, as the regulators who use CCMC opinions to take decisions on products,2 should also have significant engagement and influence on CCMC processes. Currently the CCCME has a minimum of four regulatory authority representatives, including a representative from the Provincial/Territorial Policy Advisory Committee on Codes (PTPACC), and each province and territory has a signed Memorandum of Understanding with the National Research Council of Canada (NRC). While some provincial interviewees were satisfied with the current level of engagement and influence, others believe that more frequent and comprehensive engagement, as well as transparency, with the provinces and territories is desirable, and that the provinces and territories should have greater influence on decisions regarding evaluation processes.

Consistent with the results of the business review, many interviewees expressed concern with the time required for CCMC to complete evaluations, as well as the challenges placed on smaller manufacturers as a result of the time and cost of CCMC evaluations. These issues are two of the main reasons that the provinces and territories would like more influence. However, the provinces and territories also recognize that liability concerns and the need for due diligence within NRC-CCMC are significant drivers of the current process, and that the timeliness of evaluations is a challenge for product evaluation services in many other countries as well.

2 The results of the business review indicate that in Quebec the decisions on design and construction, and the associated liability, rest with the designers and builders.


III. Lessons from Other Jurisdictions The Institute conducted short case studies of two other construction materials evaluation organizations – BRANZ (New Zealand) and ICC-ES (United States). The full case studies can be found in Annex C. Below is a brief summary of each case study, followed by a brief analysis of how CCMC governance compares to BRANZ and ICC-ES.

BRANZ BRANZ Group is owned and directed by New Zealand's building and construction industry. BRANZ is an independent and impartial research, testing, consulting and information company providing services and resources for the building industry. Funding for BRANZ comes from industry funding through a mandatory Building Research Levy, the Foundation for Research, Science and Technology and revenues generated through commercially contracted research projects for private, government and international clients. The BRANZ board is a governing board, with the usual fiduciary and strategic responsibilities of a board of directors. The board has seven members, all of whom are appointed by and from the membership of the Building Research Advisory Council (BRAC). Generally the board of BRANZ has significant building experience (builders, architects) and some business experience. There tend to be gaps in board expertise related to governance and research. The board tends to dedicate more time to business concerning the levy, which is legislation-driven and is mandatory for industry to contribute to, whereas the board generally pays less attention to the commercial side of the operations. The board’s role is to ensure that there is sufficient capacity and expertise within BRANZ to carry out appraisals in a way that minimizes liability. Decision-making regarding all appraisals therefore rests with management. All liability associated with appraisals is taken on by BRANZ Ltd.

ICC-Evaluation Service (ICC-ES) ICC-Evaluation Service (ICC-ES) is a non-profit, public-benefit corporation. ICC-ES does technical evaluations of building products, components, methods, and materials. The ICC-ES board is a governing body, with the usual set of fiduciary and strategic responsibilities, including setting policy for the organization. The Board of Directors for ICC (International Code Council), ICC-ES’ parent company, appoints the ICC-ES board. The ICC-ES board consists of 10 members, 9 of whom are voting members and represent governmental jurisdictions. The 10th member is non-voting, and acts as a liaison to the parent company. The board of directors is directly accountable to the ICC-ES parent organization (ICC). ICC-ES’ key stakeholders are the customers who pay for evaluation reports and the building jurisdictions who receive information from ICC-ES.


The ICC-ES Board of Directors recently created the ICC-ES Industry Advisory Committee (ESAC).3 The Board selects members that represent a broad range of companies and industry associations. The objectives of ESAC are (1) to advise the ICC-ES Board of Directors on matters affecting the working relationships and cooperative efforts between ICC-ES and its report holders and users; and (2) to offer recommendations for improved communication and better customer service on the part of ICC-ES. There is significant transparency and opportunity for public input in ICC-ES draft acceptance criteria. Criteria are posted on the ICC-ES website, and then the criteria go through a public hearing process headed by an Evaluation Committee (EC) composed of Code Officials. There is public input into the acceptance criteria, and deliberations by the Committee leading up to a decision are done in the presence of the audience so there is total transparency. All materials are made public, including letters submitted to the Committee. However, there are also potential challenges related to this approach, including those related to protecting proprietary information and the time required for the engagement process.

Comparative Review of Key Elements of CCMC, BRANZ and ICC-ES Below is a chart that highlights key similarities and differences in the CCMC, BRANZ and ICC-ES governance models.

Governance Element CCMC BRANZ ICC-ES

Type of Organization

Government Incorporated Not-for-Profit

Incorporated Not-for-Profit

Governing Body NRC, Government of Canada

Board of Directors Board of Directors

Governing Body Representation

Federal public service Building Industry of New Zealand

Government jurisdictions within the United States

Resources for Evaluation Services

Fee for service, subsidized by the federal government

Fee for service Fee for service

Internal and External Expertise

Strong staff expertise. Access to additional expertise via. CCCME and its Standing Committee on Technical Evaluations Other expertise brought in on contract basis as needed.

Organization ensures sufficient expertise on staff.

Some expertise on staff. Significant effort to engage external expertise through Industry Advisory Committee and public hearing process.

Liability Rests with NRC and Rests with BRANZ. Rests with ICC-ES.

3 The committee will be down sized from the current 20 members at the request of the chairman and vice-chairman who feel that the sheer size of the committee makes it difficult to operate


the Government of Canada. Managed primarily through due diligence in CCMC work and engagement of external expertise.

Managed through sufficient expertise on staff and due diligence in work. Some indications that liability of board of directors has led to risk avoidance in decision-making.

Managed by due diligence in staff work and the public hearing process, which is intended to draw out external expertise and ensure transparency in decision-making.

IV. Options for Consideration Good governance rests on a framework of structures, policies, processes and traditions that define how power is allocated, who has voice or input into decision-making, how key relationships are maintained, and how decision-makers are held to account. In governance, form should follow function. The governance model for CCMC must therefore be effectively linked to the objectives of NRC-CCMC and the Government of Canada, and the business model to the governance of CCMC. As the business review is not yet complete, the Institute offers the following options for consideration to improve the governance of the CCMC. Specific recommendations at this point would be premature. CCMC, as it is currently housed within NRC, provides a number of benefits including in-house expertise, access to NRC expertise and resources, and stakeholders’ perception of CCMC as a neutral, independent and effective service provider. One option is therefore to improve the governance of the CCMC within the existing NRC framework. Additional research into the specific nature of the liability concerns to NRC-CCMC would need to be completed before considering a move to another governance model. In addition to issues of liability, NRC-CCMC and its key stakeholders would need to explore and come to some form of agreement on a range of governance issues, including how to best engage key stakeholders, how to maintain independence and transparency, and how to ensure a continued balance between innovation and risk. That said, in the longer-term there are a number of other governance models that may do more to address liability issues, and associated potential media coverage, that are of concern to NRC-CCMC. Options 2 and 3 provide two such alternatives, but their treatment here should be considered very preliminary in nature. Option 1 - Improvements to the Current Governance Model CCMC, as it is currently situated within NRC, has a reasonable set of governance structures, processes and policies, many of which reflect best practices in the governance of organizations. These policies and procedures extend to those that define NRC-CCMC’s relationship with its


advisory body, the CCCME. Moreover, these processes and policies appear to be consistently followed in practice. As noted earlier, there appears to be strong support for product evaluation work to continue to be housed within NRC. NRC-CCMC, as an independent, third-party with a public purpose, is well-positioned to find a balance between supporting innovation and managing risk. Other strengths of the current model include CCMC staff expertise and quality of work, CCMC’s ability to access NRC research and expertise, the resources provided by NRC to support the evaluation work, and the belief that CCMC is a source of independent and ‘neutral’ advice, separate from manufacturers, builders and regulators. While the governance of the CCMC is strong overall, the Institute offers the following suggestions for improvement under the current model.

1. Improve communications to end-users. Regular proactive communication with, and education of, user groups of CCMC products is required, and in particular communication of the fact that the CCMC offers opinions and not approvals. In addition, greater transparency in the evaluation process, perhaps through a web-based tracking model, would improve understanding and engagement by clients and other stakeholders in the evaluation process. While communications is primarily a management rather than a governance role, the CCMC governing body has a responsibility to provide oversight of this communications function because it also acts as a risk management tool and improves transparency.

In addition, more effort should be made to communicate the existence and role of the CCCME, as its value-added to the evaluation process is significant and has the potential to be more so.

2. Enhance risk management practices. While CCMC has implemented some aspects of a

risk management tool, more can be done. First, the Institute suggests that NRC-CCMC, in collaboration with the CCCME, develop a strong risk management protocol that speaks to the balance of innovation and risk. The Institute also recommends that, at a minimum, an internal, committee-based, oversight mechanism be established to ensure consistency in the application and implementation of results from the risk management protocol used by staff. NRC-CCMC may wish to consider having the SCTE provide input on, or review, risk management decisions. However, recognizing that SCTE members are volunteers with limited time for engagement, NRC-CCMC could consider other options, including (a) offering an honorarium for service to SCTE members; (b) creating a second Standing Committee to deal with Risk Management; or (c) consider engaging external advice on risk management issues.

3. Improve engagement and communications to CCCME. The limited number of interviews

completed with CCCME members and others seem to indicate some desire for more frequent engagement between CCCME and CCMC and/or better utilization of CCCME expertise by CCMC. At a minimum, the Institute recommends that CCMC provide CCCME members with more regular communication on its work, and that CCMC also


consider the types of issues that are brought before the Commission and whether the expertise of the Commission could be more fully drawn upon. For example, the Commission could play a useful role in contributing to the development of an NRC-CCMC risk management protocol that balances innovation and risk. NRC-CCMC may also wish to consider conducting a brief survey, or holding a short facilitated session with CCCME members, to gather and assess how members feel they can and could contribute to the work of CCMC.

4. More effective use of the CCCME Standing Committee on Technical Evaluations. As

noted earlier, the Standing Committee on Technical Evaluations does not, in practice, work to consensus but rather provides web-based input to CCMC on a member by member basis. In addition, while there are guidelines, there is no protocol or policy governing what evaluations the Standing Committee is to be engaged on. The Institute therefore suggests that such a protocol be developed and implemented. The Institute also suggests that additional web-based mechanisms be put in place to allow all members of the SCTE to have a dialogue on the evaluation criteria in a Technical Guide and to come to some consensus on the advice given to CCMC. Finally, NRC-CCMC may wish to consider engaging the Standing SCTE on the review of test results and evaluation reports, particularly for those that are considered high risk.

5. More effective engagement with Provinces and Territories. The limited number of

interviews with provincial representatives indicate a desire for increased communication, engagement and influence on the work of CCMC. One interviewee suggested that the Commission should have a structure that better reflects provincial perspectives. That said, it is unclear whether increased representation on the CCCME is the best way to address this concern, and other interviewees had no other specific recommendations on how to address the issue. Greater transparency in the evaluation process, perhaps through a web-based tracking system, may be part of the answer. The Institute recommends that NRC-CCMC consult with all of the provinces and territories to discuss the best method and frequency of seeking their input into evaluation processes. Greater engagement with the Provincial/Territorial Policy Advisory Committee (PTPACC) or some other provincial / territorial advisory committee may be one option. A Provincial/Territorial Standing Committee of the CCCME may be another. Regardless of the mechanism chosen, MOUs with the provinces and territories will then have to be adjusted to reflect any agreed changes in the relationship.

The Institute believes that implementing the above recommendations will improve the effectiveness of the governance of the CCMC under its current structure. In some cases the recommendations will also help in mitigating the risks associated with evaluation work by improving communication to stakeholders, better involving the input of external expert advice and enhancing risk management practices. The Institute recognizes that more engagement may also lead to higher costs, although additional costs could be minimized through the use of web-based communication technologies. However, even with continued due-diligence in CCMC’s work, more effective risk management, better communication and engagement, and more effective use of external expertise to guide decision-making, there will always be some risk to NRC and the Government of Canada as long as CCMC is part of NRC.


Based on a more comprehensive analysis of the nature of the risk, the NRC may wish to analyze additional options, including those provided below. Option 2 - The Canadian Commission on Building and Fire Codes (CCBFC Model): The CCBFC, unlike the CCCME, has decision-making authority and is more arms-length from NRC. Provincial and other stakeholder engagement and influence in the work of CCBFC is much stronger. Secretariat support for the work of the CCBFC is provided by the Canadian Codes Centre (part of NRC), much like the CCMC provides secretariat support to CCCME. However, CCBFC and the work of CCMC vary in two significant ways. First, the CCBFC, with its focus on building codes, has a mission that is primarily safety-driven, while the NRC-CCMC must balance safety with innovation. As form should follow function, the CCBFC governance model is not a clear fit for CCMC. Second, CCMC work goes beyond secretariat support to the conduct of evaluations. Because evaluation work is completed within NRC-CCMC, and because the CCCME is an entity set up by the Government, most, if not all of the liability, may still remain with NRC even if the CCCME was given more authority. In addition, having CCMC staff report to the Commission while the NRC is their employer may create difficulties in reporting and accountabilities that would need to be carefully thought through. Finally, the composition of the CCCME and its Standing Committee on Technical Evaluations would have to be reviewed closely if the CCCME were given more decision-making authority. Option 3 – Independently Incorporated Not-for-Profit Corporation: Another option is to have evaluation work conducted and overseen by a separately incorporated not-for-profit corporation totally arms-length from government. Such a model would limit liability for work conducted and decision-making to the not-for-profit corporation, and could allow for more voice from the range of stakeholders that make use of evaluation processes and products. In addition, a not-for-profit model, with balanced and fair representation from all key stakeholders on the board of directors, could achieve the required independence and neutrality that is currently seen as a real strength of the CCMC model. Issues related to funding of the organization would need to be addressed. Second, given that liability would rest on a board of directors of the corporation, likely making the organization more risk adverse, the possibility of impacts on innovation and on the timeliness of evaluations would also need to be considered. Finally, the composition of the board of directors, and in particular the necessary skills and expertise related to the governing of a not-for-profit corporation, as well as how to effectively engage the provinces and territories, would need to be explored.


Annex A – Interview Guide

NRC-CCMC Governance Review Interview Guide

1. What are the strengths of the current governance structure? What are the principle weaknesses? 2. Please comment on the following relationships in terms of authorities, responsibilities and

accountability:

a. CCMC and the CCCME (Advisory Board) b. CCMC and the NRC c. CCCME and the NRC d. Provinces and the CCCME/CCMC

3. Please comment on the following elements of the CCCME: a. Commission size and composition b. Committee structures c. Recruitment and selection d. Skills and capacities e. The effectiveness of Commission meetings

4. Do NRC-CCMC and CCCME allow for sufficient voice from stakeholders and demonstrate accountability to their stakeholders? If not or insufficient, how could voice and accountability be improved?

5. What do you see as the major governance risks facing CCMC? What approaches does CCMC

have in place to address these risks? Are there other approaches or structures that you feel would better mitigate against these risks?

6. What options would you suggest for improving the overall effectiveness of CCMC governance?


Annex B – Case Studies 1. BRANZ Case Study BRANZ Group, comprising BRANZ Inc and BRANZ Ltd and their partly-owned subsidiaries, is owned and directed by New Zealand's building and construction industry. BRANZ is an independent and impartial research, testing, consulting and information company providing services and resources for the building industry. BRANZ two main areas of activity are: (1) to research and investigate the construction and design of buildings that impact the built environment in New Zealand; and (2) enable the transfer of knowledge from the research community into the residential and non-residential building and construction industry. Funding Funding for BRANZ comes from industry funding through a mandatory Building Research Levy, the Foundation for Research, Science and Technology and revenues generated through commercially contracted research projects for private, government and international clients. The Building Research Levy funds a continuous program of research and information transfer activities for the benefit and education of the building and construction industry, and funds activities such as research projects, seminars, a technical advice helpline and material testing projects. Commercial income is generated through activities such as the testing and appraisal of products, the sale of publications and the provision of educational information and professional development activities. BRANZ customers are located in New Zealand, Australia and around the world. The BRANZ Board and Membership The BRANZ board is a governing board, with the usual fiduciary and strategic responsibilities of a board of directors. The board has seven members, all of whom are appointed by and from the membership of the Building Research Advisory Council (BRAC), itself an industry-based association. The membership of BRAC itself is made up of appointees from across industry, and the BRANZ board is therefore representative in nature. The only official duty BRAC has with regard to BRANZ is the appointment of the Board. That said, while BRANZ board is accountable to the broader industry, it is BRAC who will be the primary judge of BRANZ’s effectiveness, efficiency and accountability. Generally the board of BRANZ has significant building experience (builders, architects) and some business experience. Possible gaps include someone with experience in research, as well as someone who has expertise/experience with governance who can provide a broader perspective to board discussions. 40% of BRANZ income is from the levy with the remaining 60% generated by its commercial services. Despite this, the board in practice tends to dedicate more time to business concerning the levy, which is legislation-driven and is mandatory for industry to contribute to, whereas the


board generally pays less attention to the commercial side of the operations (which runs well on its own). The board has the usual committees, including Finance & Audit and Compensation, and has the ability to form other ad hoc committees as well. Membership on these committees is drawn from the board. There is no board committee that deals with appraisals. The appraisal service is considered to be a ‘stand-alone’ business unit within BRANZ. As such, its reporting responsibilities lie primarily with management, while the board retains an oversight role, particularly with regard to liability. The BRANZ board also has two Research Guidance committees that report to it. The Research Guidance committees - one deals with the research agenda and the other with research proposals – are both appointed by the board. Membership on the Research Guidance committees is not drawn from among board members, but rather from industry experts. These committees make recommendations to the BRANZ board on its research agenda and proposals. While the expertise that is on these committees is highly valued, there has been some reflection on whether the two Research Guidance committees could merge, and whether this would provide a more holistic approach to BRANZ’s research. BRANZ ‘members’, as specified in the Act, includes all builders who pay the levy and apply for membership. The broader membership may attend the AGM and vote, and members receive some free services and access to publications. The Board’s Role in Appraisal Decision-Making The board’s role is to ensure that there is sufficient capacity and expertise within BRANZ to carry out appraisals in a way that minimizes liability. Decision-making regarding all appraisals therefore rests with management. All liability associated with appraisals is taken on by BRANZ Ltd. Managing Risk and Liability Appraisals are a service charged to manufacturers. There is no regulatory framework that defines or limits BRANZ role in appraisals, and as such there is no regulatory framework to protect against the risk associated with appraisals. For this reason, BRANZ evaluations need to be first, robust, which from a BRANZ perspective means assessing both the production system and product. Second, BRANZ adopts evaluation criteria to test products against the building code. For appraisals BRANZ always tests to the complete building code and not just one clause or element of the code. The appraisal process can therefore be expensive for the manufacturer. BRANZ appraisals do not have mandatory recognition from the New Zealand Building Consent Authority (BCA), although it is rare for a BCA to question a BRANZ appraisal.


BRANZ can face challenges with its appraisals being treated like ‘approvals’. In particular BRANZ has had difficulties with builders not considering the context the product is meant to be used in when using a product that has been appraised. Note that New Zealand also has a product certification scheme where products can be certified against a particular (one) clause of the building code. This certification scheme does have mandatory recognition from Building Consent Authority (BCA). BRANZ Relationship to Government BRANZ was set up by the New Zealand building industry with support from the central government, but is completely independent. A portion of the levy is budgeted toward supporting the development of the Building Code and relevant national standards. There is an expectation by industry that BRANZ will participate in discussions with government around the Building Code and Standards at a technical level, and BRANZ participation in these discussions is recognized by government. Other Of note, in New Zealand the laws regarding personal liability of directors changed approximately four years ago. While the old law was considered by some to ‘lack teeth’ the new law, which puts all liability on the shoulders of directors personally, has made both the volunteer directors, and organizations as a whole, much more risk adverse.


2. ICC-Evaluation Service (ICC-ES) Case Study ICC-Evaluation Service (ICC-ES) is a non-profit, public-benefit corporation. ICC-ES does technical evaluations of building products, components, methods, and materials. The evaluation process culminates with the issuance of reports on code compliance, which are made available free of charge to code officials, contractors, specifiers, architects, engineers, and anyone else with an interest in the building industry and construction. ICC-ES evaluation reports provide evidence that products and systems meet code requirements. ICC-ES came into being on February 1, 2003, when America's four building-product evaluation services officially combined their operations. The four "legacy" evaluation services that came together to form ICC-ES were the National Evaluation Service, Inc.; BOCAI Evaluation Services; ICBO Evaluation Service, Inc.; and SBCCI Public Service Testing and Evaluation Services, Inc. Through the legacy evaluation services, ICC-ES has a history that goes back more than seventy years. The Board of Directors The ICC-ES board is a governing body, with the usual set of fiduciary and strategic responsibilities, including setting policy for the organization. The responsibility of the President and his staff is to implement policy. The Board of Directors for ICC, ICC-ES’ parent company, appoints the ICC-ES board. The ICC-ES board consists of 10 members, 9 of which are voting members and represent governmental jurisdictions. The 10th member is non-voting, and acts as a liaison to the parent company. The ICC (International Code Council) board makes an effort to draw the ICC-ES board from ‘known quantities’ - those who have been active in their jurisdictions and are well recognized, at least within their region. Currently the ICC-ES board is overseeing changes that are taking place in the organization, particularly with regard to how they operate. The goal is for ICC-ES to be more market conscious in serving its report holders while retaining its technical expertise. The board’s primary focus is in providing services to building enforcement agencies primarily representing government at various levels. The ICC-ES Board of Directors recently created the ICC-ES Industry Advisory Committee (ESAC).4 The Board selects members that represent a broad range of companies and industry associations. The members were chosen based on activity with ICC-ES operations and staff, involvement with industry activities, and the type of products, materials and/or services in which candidates functioned. The objectives of ESAC are (1) to advise the ICC-ES Board of Directors on matters affecting the working relationships and cooperative efforts between ICC-ES and its report holders and users; and (2) to offer recommendations for improved communication and better customer service on the part of ICC-ES. A requirement of eligibility is that each of the newly appointed members have an established record of participation in ICC-ES activities and efforts to promote improved service to report holders and users. All meetings are open to the public.

4 The committee will be down sized from the current 20 members at the request of the chairman and vice-chairman who feel that the sheer size of the committee makes it difficult to operate


The board of directors is directly accountable to the ICC-ES parent organization (ICC). ICC-ES’ key stakeholders are the customers who pay for evaluation reports and the building jurisdictions who receive information from ICC-ES. Other stakeholders include engineers, specification writers and all users of ICC-ES products. ICC-ES has very little direct engagement with government agencies, particularly at the federal level. ICC-ES does have some engagement with larger states who have their own evaluation systems. For example, Florida has its own Building Code where technical requirements closely follow the International Building Code. ICC-ES is able to provide an evaluation compliance report for the national code and also provide a supplement that evaluates compliance with the state code. The point of offering the service is to demonstrate a customer service mentality. There is a nominal fee for the supplement for state codes. ICC-ES also has contact with municipal governments. Generally any contact with them is related to municipal governments seeking clarification on reports or seeking advice in interpreting reports in terms of how to use the information in the field. Decision-Making with Regard to Evaluations ICC-ES staff will draft acceptance criteria for an evaluation if existing International Codes do not adequately address a product. These criteria are posted on the ICC-ES website, and then the criteria go through a public hearing process headed by an Evaluation Committee (EC) composed of Code Officials. Members of the Evaluation Committee are appointed by the ICC-ES board of directors and are representatives of enforcement bodies. The Evaluation Committee has geographic representation, but greater emphasis is placed on technical expertise and knowledge of the Codes. There is public input into the acceptance criteria, and deliberations by the Committee leading up to a decision are done in the presence of the audience so there is total transparency. All materials are made public, including letters submitted to the Committee. There are two appeal mechanisms for decisions of the Evaluation Committee – one for appeals on procedures and one for technical appeals. Appeals on procedure will go directly to the board of directors for final decision. Technical appeals have a more complex process whereby if an appeal is ‘sustained’ by the board, it will go back to the Committee and another public hearing will be held. ICC-ES does not do testing. To be recognized by ICC-ES, test facilities must be accredited under ISO Standard 17025 for the specific testing required by the acceptance criteria. ICC-ES does not provide a list of accredited test facilities to manufacturers. However, the information is available from International Accreditation Service, another subsidiary of ICC, or any other accreditation organization with like accreditations traceable to the International Laboratory Accreditation Cooperation. Upon receipt of technical data, the ICC-ES technical staff will review the submitted data for compliance with the code and acceptance criteria. The staff engineer will approve or hold the


application for product recognition pending resolution of comments by the engineer. There must be a technical basis for decisions taken. There is also an appeal for staff decisions. Appeals are rare, but if they occur they can be brought to a Panel of Experts5. Managing Risk and Liability The riskier the product (based on experience with aspects of the product, assumptions that might have been taken by product manufacturer etc.) the more conservative the ICC-ES will be in setting acceptance criteria. Because of public hearings risks will be identified and evaluated in a public forum. In addition, the public hearing process allows access to a broader range of expertise than can be kept on staff, as well as access to the most current information. In ICC-ES’ view, open hearings ensure the most current information is available to the Evaluation Committee. Secondly, they help protect ICC-ES from liability by showing effort to gain the most significant information available on a product type. The Committee will not deny anyone from speaking at the hearings. Acceptance criteria are approved by representatives of enforcement bodies to assure independence in decision-making and regard for public safety. The reality that concerns about the risks associated with a product will come out in hearings, as well as the United State’s legal environment, motivates all involved to protect themselves through due diligence in their work. The Evaluation Committee does not delve into proprietary information when dealing with innovative products, so the risk associated with a competitor accessing proprietary information is very low. Other One of the biggest challenges facing ICC-ES is the element of competition. There are certification agencies that are now issuing their own evaluations that make reference to ICC-ES evaluation criteria. There is no copyright violation involved, so competition can make free use of ICC-ES work. In addition, because the competition is not involved in the development of acceptance criteria they can issue their reports much more quickly.6

5 To date ICC-ES has had only one formal technical challenge which dealt with fire protection for raised computer floors. In concert with the U.S. National Council of Structural Engineers Associations (State bodies) a pool of technical experts in various structural fields has been created from which the ICC-ES Board of Directors can appoint panel members. The structural pool will also be used by the Evaluation Committee to provide recommendations on difficult structural acceptance criteria. 6 Evaluation criteria are only developed if there are applications filed and fees paid to help defray costs.



Brio Conseils inc.

Page: 18 / 21

APPENDIX H: Summary Report, Brio Conseils

National Research Council’s Institute for Research in Construction (NRC-IRC) Summary report CCMC Business Review Date: December 22, 2009


Page: 1 / 4

INTRODUCTION

In spring 2009, the CCMC embarked on a Business Review project aimed at examining its business model for evaluating construction products, its mission and if it was meeting the needs of its clients.

This report summary provides an understanding of how the key stakeholders perceive CCMC’s role, mandate and services and its relationship to the Canadian construction industry.

METHODOLOGY

For this project, the NRC-IRC has engaged Brio Conseils, a firm specializing in transformation management, entrusting it with heading and managing, in a transparent manner, a consultation with the various players involved in the construction industry. The CCMC’s business review involves three phases, as illustrated below:

Project Timeline and Deliverables The NRC has appointed a Business Review project team, comprising representatives from provincial ministries, regulatory authorities and manufacturers—as well as the NRC and the Chair of the Canadian Commission on Construction Materials Evaluation (CCCME). Their role is to review the results, prioritize the findings into groups of ideas and make recommendations as to what can be moved forward on.

The outcome of the phase 1 was to identify CCMC key strengths, weaknesses, threats and opportunities (SWOT) based on the internal assessment and an environmental scan.

The second phase was done in two parts. The purpose of the first part was to better understand the primary1 stakeholders’ needs and their satisfaction level with the CCMC’s services. To do that, surveys were conducted with primary stakeholders: ministry officials responsible for construction policy, building officials and manufacturers. The purpose of the second part was to consult stakeholders through focus groups on specific issues in order to validate and have an open discussion about the findings from the first part. All surveys were conducted in partnership with Ad hoc Research who has more than 25 years experience conducting consultation studies.

Recommendations from the project team will be based on the gap between stakeholders’ needs and current CCMC services and the changes required to address reported issues.

1 A direct stakeholder is a group of individuals who are immediately affected by the service offered by the CCMC and who stand to personally benefit or loss from any changes made to the CCMC business model.



Page: 2 / 4

KEY FINDINGS

The focus of the surveys with Stakeholders was to identify primary stakeholders’ needs and their satisfaction with the CCMC’s current services.

In the late 1980s, provincial/territorial representatives agreed to support a national evaluation body. Today, the consensus is that there are advantages to having a national product evaluation to prove code conformity.

CCMC VALUE FOR THE STAKEHOLDERS

Ministry officials believe that CCMC provides an invaluable service. Many provinces do not have the resources to test products and/or determine if they are compliant.

Building officials are far more comfortable approving standardized products (average of 9.3 out of 10) than non-standardized products (rated at only 3.2 out of 10). Many building officials admit to lacking expertise or having no time to review the necessary documentation when it comes to innovative products. Both CCMC listing and report are rated the highest of 5 sources of information when it comes to product evaluation with or without standard. The main reason is the quality of the information provided.

Although manufacturers seem to find the CCMC listings useful (84% very or somewhat), they seem to agree that the reports are more useful (96% very or somewhat). They believe that if CCMC would stop offering the listing service, it will be more complicated to obtain approval for their products. In fact, manufacturers have the perception that the construction industry and professionals currently seem to rely heavily on the CCMC listings for product approval.

NEEDS FOR IMPROVEMENTS

Difference between listing and report:

Almost half the building officials (44%) as well as manufacturers (44%) admit to not knowing the distinction between both services.

The scope of the listing seems misunderstood:



- Two-thirds of building officials (67%) do not know the CCMC does not evaluate any additional requirements imposed by the national or provincial building code.


The building officials may not necessarily using the CCMC report as they should:

- Most building officials very likely read the report to ensure that the product/system meets the conditions and limitations stated (73%) and that it has a valid CCMC number (73%).

- Only half (49%) are very likely to read the full report thoroughly.



Page: 3 / 4

- Only 23% are actually aware of the format change introduced 2 years ago. Those who are aware of it seem to feel that the 2005 report format is an improvement over the 1995 format (94% agree completely or somewhat).

- Less than half the building officials are aware of the review of the installation manuals (42%) or of the review of the quality control procedures (40%).

- A total of 80% of respondents find the optional information at the end of the report useful, although only 27% find it very useful.

Lack of communication:

Several ministry officials mentioned regularly receiving e-mails from other government agencies but rarely receiving any news from the CCMC. They would prefer a more regular dialogue between the CCMC and themselves as well as with the building officials.

Lengthy process for the report service for innovative products:

Ministry officials are of two minds when it comes to the time aspect since they feel it is important to evaluate the products rigorously—which in turn can be a time-consuming process.

There seems to be a general dissatisfaction among manufacturers with the time it takes the CCMC to complete a report (only 35% are very or somewhat satisfied). The two elements that seem to drive this dissatisfaction are:

- the time it takes to develop the technical guide; - the time it takes to receive the final report after successful testing.

Re-evaluations cycle:

Most ministry officials prefer a three-year re-evaluation cycle but question the CCMC’s capacity to achieve it.

Building officials are satisfied (93% very or somewhat) with the current 3 year cycle, only 57% would completely or somewhat agree with a 5 year cycle. The main concern revolving around the 5 year cycle is that the information in the reports would be outdated considering the rate at which technology changes.


Services:

The vast majority of ministry officials feel that CCMC invests in creating a client relationship by calling back/writing back very quickly and engaging in an open and direct dialogue when discussing issues/problems.

Of building officials who did contact the CCMC, 97% were very (46%) or somewhat (52%) satisfied with the support/service they received. The most common complaints regarding the service are: slow response time and quality of information provided.

The satisfaction level with the support/service respondents received when they contacted the CCMC is relatively good (27% very and 49% somewhat satisfied); however, there seems to be room for



Page: 4 / 4

improvement. Almost a quarter (24%) of manufacturers claim to be not very or not at all satisfied. The main reason mentioned is because it is ‘time consuming/has a slow response time’).

OPPORTUNITIES

During the survey, opportunities were evaluated with both building officials and manufacturers:

The manufacturers seem to think that making the progress of their file open to themselves (91%) or to building officials (78%) could be useful; however, their level of comfort is less when it comes to all the other potential entities to whom the information could be given (builders, architects, other manufacturers).



Building officials and manufacturers agree that the CCMC could extend its services to include:

- energy efficiency (84% business officials and 67% manufacturers); - water conservation (75% business officials and 50% manufacturers); - green initiatives (69% business officials and 65% manufacturers); - sustainability/conservation (69% business officials and 61% manufacturers).



Brio Conseils inc.

Page: 19 / 21

APPENDIX I: Open Consultation – List of contacted groups, Brio Conseils

Company name Contact Title

AFA Forest Products Inc. Andy Dolan National Sales & Marketing Manager, Allied Products.

Air Ins R. Jutras

Alberta Building Officials Association (ABOA) Bob Clarke President

Alberta Building Officials Association (ABOA) Mike Bos Membership Director

APCHQ André Gagné Directeur technique

BC Building & Safety Policy Branch, Ministry of Housing & Social Development

Jeff Vasey Executive Director

Building Official′s Association of British Columbia (BOABC)

Frank Hill Vice-President - Member Services


Ron E. Dickinson President


Richard Bushey Executive Director

Canadian Home Builder Association Don Johnston Senior Director, Technology & Policy

Canadian Roofing Contractors' Association Peter Kalinger Technical Director

Canadian Urethane Foam Contractors Association (CUFCA)

Laverne Dalgleish

Canadian Wood Council Peggy Lepper

CanBest Brampton Elie Alkhoury

City of Markham Mr. Boyko

Yukon Dept of Community Services, Consumer & Protective Services

S. Dueck Manager, Building Safety Branch

FPInnovations -- Forintek East Richard Desjardins

FPInnovations -- Forintek West Conroy Lum

Irwin Energy Consulting Services Arthur A. Irwin

ITS Vancouver G. Nisho

Manitoba Building Officials Association (MBOA) John Barnes President - Executive Board

Manitoba Building Officials Association (MBOA) Michael S. Anderson

Past-President / Membership Chair

Ministry Official from Alberta James Orr

New Brunswick Building Officials Association (NBBOA)

John Jackson President

Nova Scotia Building Officials Association (NSBOA)

Jack Leedham Administrative officer

Ontario Building & Development Branch, Ministry of Municipal Affairs & Housing

David Brezer Director & Chief Building Official

Ontario Building Officials Association (OBOA) Ronald M. Kolbe Chief Admin Officer

Order of Architects Quebec Pierre Frisco

Public safety division Nova Scotia Labour & Workforce Development

Ted Ross Building Code Coordinator



Brio Conseils inc.

Page: 20 / 21

Régie du bâtiment du Québec Georges Tessier Direction de la normalisation et de la qualification

Ministry Official from Saskatchewan William Hawkins Chief Building Official Building Fire and Safety

Sageos Eric Blond

Saskatchewan Building Officials Association (SBOA)

Mr. Knutson President

The EIFS Council of Canada/Dryvit Peter Culyer

Ministry Official from Manitoba Christopher Jones Office of the Fire Commissioner

Municipal & Community Affairs, Government of the Northwest Territories

Stephen Moss Fire Marshal



Brio Conseils inc.

Page: 21 / 21

APPENDIX J: CHBA Discussion Paper, CHBA

CHBA Discussion Paper on the Canadian Construction Materials Centre

and Code Conformity Assessment

March 2005

Rowena E. Moyes

[email protected]

Table of Contents

Executive Summary i

1. What is the CCMC? 1

2. How did the CCMC get started? 2

3. Where do CCMC evaluations fit into the system? 3

4. CCMC: one of the traditional means of assessing compliance 4

5. Users’ requirements and expectations 7

6. CCMC is becoming the only game in town 9

7. Some of the issues are complex 11

8. Objective-based Codes will have an impact 17

9. Recommendations 20

- Structural

a) Fix the problem of liability.

b) Apply the principles of Smart Regulation.

- CCMC/NRC and Advisory Commissions

c) Limit the scope of CCMC evaluations.

d) Define and use a simple process for risk rating, management and due

diligence.

e) Clearly communicate the actual scope of CCMC evaluations.

f) Create a practical avenue for review and/or appeals.

g) Investigate issues of “Equivalent” or “Acceptable” performance.

h) Provide a functional alternative to the traditional acceptances offered by

local/provincial officials.

i) Review CCMC’s capacity to perform the growing amount of work, and ensure

it has the necessary resources.

- Other jurisdictions

j) Resolve the issue of non-CCMC evaluations, and non-evaluated products

k) Investigate expanded use of alternative compliance acceptance methods

i

Executive Summary

1) The Canadian Construction Materials Centre (CCMC) is designed to

address a real need

Canadian building and fire codes define what products and systems must be used

by law to protect health and safety in housing construction. Many of the

requirements refer to product, performance, design or installation standards, or to

traditional approaches. However, as new products and systems are developed or

brought in from other countries, they may use different approaches. The CCMC is

a national body established to evaluate whether these products or systems meet

the requirements of the building and fire codes.

2) The informal system of equivalency acceptance is breaking down

Traditionally, local building officials have accepted alternative products and

systems which can show “equivalent” performance to code, often based on

manufacturers’ tests, etc. However, with the growing complexity of construction

and with growing awareness of liability, officials in a number of localities have

begun to back away from accepting equivalents. Or, they will only accept them a

single project at a time, with code conformity confirmed by professional

engineers or architects. At the same time, provinces / territories have given their

support to a national system of evaluation. In some cases, provinces have

discontinued or restricted their own evaluation services.

3) Pressures are increasing

CCMC evaluations can take a long time. Evaluations and the associated tests can

be extremely expensive – particularly if no performance data currently exists for a

traditionally accepted product or system. In that case, someone wanting to use a

new approach must not only test their own product or system but the traditional

ones as well. As the informal acceptance system dries up, attempts to provide an

“interim acceptance” process have foundered because of liability concerns.

Meanwhile, some products or systems which have just been accepted in the past

with no specific code requirements, or those only partially covered by existing

standards, or even those with industry-recognized but not code-referenced

standards, are suddenly being asked to provide evaluations.

4) Definition of role and scope is under revision

The existing policies do not contain clear enough definitions of what the CCMC

evaluations cover and do not cover, or what they should require or not require.

The Canadian Commission on Construction Materials Evaluation has been

examining this issue. New policies should address practicality, cost efficiency and

how to relate tests and research to a “likelihood of problems” factor. They should

be based on simple risk assessment and management procedures which allow

CCMC to do what is “reasonable” and not “everything they can think of”.

ii

5) Recommendations

Structural

a. Fix the problem of liability.

b. Apply the principles of Smart Regulation.

CCMC/NRC and Advisory Commissions

c. Limit the scope of CCMC evaluations.

d. Define and use a simple process for risk rating, management and due

diligence.

e. Clearly communicate the actual scope of CCMC evaluations.

f. Create a practical avenue for review and/or appeals.

g. Investigate issues of “Equivalent” or “Acceptable” performance.

h. Provide a functional alternative to the traditional acceptances offered by

local/provincial officials.

i. Review CCMC’s capacity to perform the growing amount of work, and

ensure it has the necessary resources.

Other jurisdictions

j. Resolve the issue of non-CCMC evaluations, and non-evaluated products

k. Investigate expanded use of alternative compliance acceptance methods

1

1. What is the CCMC?

The Canadian Construction Materials Centre is a national body which evaluates

innovative products, materials, equipment, installation methods, design standards, etc., to

see whether they can be accepted as complying with the building, fire and plumbing

codes in use in Canada. The service is voluntary, offered on a fee-for-service basis to

manufacturers who request it. CCMC produces a registry of completed reports and

listings, which is available online. Building department staff, designers, builders,

authorities and other interested parties can check the registry on a voluntary basis to

determine which products/systems have been evaluated for use in their projects.

CCMC is staffed by a small department within the National Research Council (Institute

for Research in Construction: Codes and Evaluation). In addition to the manager, there

are eight evaluation officers, one additional evaluation officer who concentrates on

infrastructure-related requests, and three support staff.

There were more than 500 reports/listings in the registry in 2003. Over the course of the

year, 250 new files were opened, and of those, 120 were still active in 2004. The shortest

time taken for an evaluation is about six months – and that is for a product where a

standard already exists. For products/systems which are very innovative, the time period

is often much longer, as the work can be quite complex and sometimes extremely

expensive. In some cases, no listing results even after years of discussions and/or tests –

sometimes because the products could not pass the performance tests, or proponents and

CCMC could not agree on requirements, or the proponent decided not to proceed with

their innovation – or because a change in a standard makes the work unnecessary.

When a proponent applies for evaluation of an innovative product or system, the process

starts with a Scope and Evaluation plan (SEP). This looks at relevant standards and code

provisions and identifies the performance issues to be addressed in the official evaluation.

CCMC also consults with technical experts to identify the strategy for evaluating

equivalent performance.

If the proponent accepts the plan, the next step is to prepare a Technical Guide (TG). For

this, CCMC staff, with input and advice from relevant experts, develop test methods and

technical criteria, as well as criteria for lab reports and in-plant quality control. Approved

TGs are sent to the proponent, who can then contract with a laboratory of their choice

complying with CCMC policy to have the testing performed. Test results must be sent

directly to CCMC. The proponent is required to submit other supporting administrative

criteria.

The laboratory submitted test data will be reviewed by a CCMC Evaluation Officer

against the criteria established in the Technical Guide to determine if, in the opinion of

CCMC, the product or system complies with the requirements of the building code. If

CCMC is satisfied that the criteria has been met, an evaluation report is prepared. These

reports include a complete description, installation requirements and any restrictions on

use. Every year, the proponent must affirm that the product has not been modified. Every

2

three years, the product is re-evaluated, including reviews of plant quality control and

reported problems.

Product evaluations are provided on a fee basis: proponents must pay for the preparation

of plans, technical guides, reviews and listings. As such, it is “ protected” on behalf of the

proponent for the first year after completion. If a new proponent requests an evaluation

using the same technical guide during that year, they are required to pay some of the TG

development costs and the first proponent is provided with a rebate of some of the costs.

After the protection period expires, the guide becomes available to other proponents for

free.

CCMC operations are overseen by the Canadian Commission on Construction Materials

Evaluation (CCCME). Appointed by NRC, this group is chosen to represent diverse

sectors of the construction industry, testing and research sector, and the building

regulation community.

One important CCCME group is the Standing Committee on Technical Evaluations,

which reviews initial SEPs, and TGs. During 2004, that group has been re-examining the

goals and scope of CCMC evaluations. A report with initial recommendations has been

prepared, endorsed by CCCME and recommended to the NRC Management Committee

for approval (see Section 7.8).

2. How did the CCMC get started?

For many years, Canada Mortgage and Housing Corporation used to run a product

acceptance service. This group would review product information and produce a listing

of products accepted for use in housing projects using NHA mortgages. These products

would then be required in the field by CMHC’s own inspectors and accepted by most

local building departments for other projects as well.

In the 1980s, the CMHC service and its product evaluation functions were transferred to

the National Research Council. The new arrangement reflected the fact that CMHC was

no longer relying on a separate inspection service running in parallel with building

departments. It was also intended to take advantage of the NRCs respected research staff

and facilities. And it was a response to calls for a single, national, science-based

evaluation service, to serve the larger building industry beyond residential.

At the time, several provinces ran their own evaluation services – some more formal than

others. Director’s rulings and interpretations or their equivalent were also sometimes used

for new product acceptances. Building industry spokesmen say that until fairly recently,

if the provinces thought a product looked acceptable, they would accept it while the

manufacturers sought a CCMC number for a year or two.

The Provincial Territorial Policy Advisory Committee on Codes is a committee of senior

officials responsible for building regulation, which provides policy advice to the

Canadian Commission on Building and Fire Codes, and its supporting Canadian Codes

3

Centre at NRC. The PTPACC has agreed to support the use of CCMC evaluations as a

basis for determining acceptability of products in regards to building code requirements.

That has substantially happened.

3. Where do CCMC evaluations fit into the system?

The short answer is “between one-off product or system approvals at the local level and

full standards developed by an accredited Standards Development Organization and

referenced in codes”.

As soon as governments start regulating construction through building code laws and

regulations, they have to define what is acceptable. That can be done by straight

description, or by referencing prescriptive product standards and/or installation standards,

sometimes with third party certification systems and some form of marking or

identification backing them up. It can also be done through performance requirements, or

design standards, which establish methods of calculation and/or testing.

No matter which system is used, as soon as an official body describes how something

must be done, there will be a class of products and systems that don’t meet the

requirements – or can’t demonstrate that they meet them. Most innovative ideas for

products or systems regulated in codes, for example, will not exactly meet the existing

code requirements or the referenced standard. Products manufactured, accepted and used

in other countries also may not meet codes here.

If they don’t meet the requirements because they are dangerous or unreliable, that is a

good thing. But if they don’t meet the requirements because the wording doesn’t apply,

or they use a different approach that isn’t covered, or because you can’t measure them or

test them the required way1, that works against the desired objective.

Around the world, in countries which regulate construction through a code, some kind of

review and acceptance system has evolved to cover the issue of new and/or non-

conforming products and system.

1 Where there are enough different producers of similar products, the appropriate route is

preparation of a new standard by a recognized standards-writing agency. (This process is intended to

provide one set of requirements for products or systems – not to evaluate the appropriateness of a new,

innovative product only produced by one or two manufacturers.)

4

4. CCMC: one of the traditional means of assessing compliance

In Canada, decisions on code compliance have been made in five main ways:

Evaluation and acceptance process Current

availability

Local review

and acceptance

of equivalency

Relatively informal, one-off decisions generally

made by individual building officials, based on

manufacturers’ information and supporting

documents such as:

test results and/or design calculations (in-

house and/or third party)

evaluations from other jurisdictions

performance history in other jurisdictions

related industry standards and/or certifications

design and statements of code conformance by

a professional architect or engineer

These are made under the building codes’

“equivalency” provisions – which give the

authority having jurisdiction the right to accept

non-conforming solutions which are “equivalent”

to the code requirements

ranges from

restricted to

zero

availability

Professional

evaluation/

certification

All provinces accept some non-standard products

or systems in individual projects where a

professional architect and/or engineer as

appropriate signs a certificate or letter of

assurance that the proposal meets the intent of the

code.

increasingly

required for

more minor

variances

Provincial ruling

or acceptance

More formal ruling or acceptance historically

issued at the provincial level, based on similar

information, sometimes with additional tests or

research; this would apply in general across the

jurisdiction subject to whatever conditions are

placed on the acceptance.

becoming rare

CCMC

evaluation

Formal code conformance evaluation and

registration of non-standard products and systems

by the Canadian Construction Materials Centre,

which is part of the National Research Council.

the main

avenue for

new approvals

ORDs issued by

certification

bodies

The oddly-named “Other Recognized

Documents” or ORDs, are issued by a few of the

groups accredited by the Standards Council of

Canada to certify products to standards within

currently rare

except for

ULC, but may

increase

5

certain areas of expertise. These take less time

that a full standard to prepare (typically three to

six months) and are intended for situations when a

recognized Canadian standard does not cover a

new product to be certified. SCC policies say an

ORD “shall provide an equivalent level of safety

or performance as provided for similar functions

in existing standards”, and must be approved by

an applicable regulatory authority or (in

unregulated areas) acceptable to the appropriate

industry associations. Where the product or

system is not subject to other specific regulations,

it is to be certified to a national standard of

Canada (NSC) or an industry consensus standard

prepared in accordance with ISO Guide 7. The

issuer has to have procedures for tracking misuse

of the certification mark or hazards appearing in

the products. If the ORD has not been

incorporated into revisions to a recognized

standard within five years, it has to be resubmitted

to the regulatory body for reapproval, or

withdrawn. ORDs have been used frequently by

Underwriters Laboratories of Canada (ULC) to

accommodate non-standard and innovative

approaches, subject to any stated conditions.

Note: A subcommittee of the PTPACC is

currently discussing whether and how ORDs

should continue to be referenced in the codes

themselves.

New or updated

standards

Most construction standards are prepared by one

of four Standards Development Agencies

accredited by the SCC (CSA, CGSB, ULC and

BNQ), U.S. agencies (e.g. ASTM and NFPA), as

well as several industry groups with specific

expertise in their own subject areas, both in

Canada and the US. Some of these groups’

standards are referenced directly in the building

codes. In that case, the edition number is also

spelled out. Obviously, the manufacturer of a

successful innovative product will eventually seek

either a new standard or a revision to

accommodate it. But the process can take years.

Other standards are not referenced directly in the

codes, but proponents may use them to support

acceptability.

ongoing, but

less

acceptance by

authorities of

non-

referenced

standards (by

industry

groups or

SCC-

recognized

SDOs) and of

upcoming

revisions

6

Local and provincial officials, as well as

engineers and architects, have traditionally given

non-referenced industry standards and

certification systems, as well as proposed or

completed changes in referenced standards,

serious consideration in assessing code

acceptability of new products and systems.

Change in

building codes

New standards and revisions to standards already

referenced in building codes may be accepted (in

whole or in part) during the subsequent review

and updating of building codes.

Alternatively, where a new or innovative product

is covered by other code requirements rather than

a referenced standard, those may be amended

directly during the next code review. Obviously,

the review would deal with the issue of whether

the new product or system meets the intent of the

code. These reviews are conducted by code

committees under the oversight of the Canadian

Commission on Building and Fire Codes and the

Provincial-Territorial Policy Advisory

Committee, with full public review.

Where a material or construction is not addressed

in the Code or by a standard, new requirements

can be incorporated into the Code.

For comparison, in a March 2002 Standata (97-IB-020R1), Alberta identified the

following methods as the basis for acceptance under the Alberta Building Code:

Acceptance for broad application:

certification of compliance with a recognized standard

evaluation and listing by a recognized evaluation agency such as CCMC

Acceptance restricted to a single application only:

overall professional involvement in whole building (where it is understood that

the professionals have determined acceptability of products and systems,

documentation required)

restricted professional involvement in certain elements of the building (letters of

compliance required)

variances issued by the local safety codes officer (“should only be needed under

exceptional circumstances”)

variances issued by an appeals panel of the Safety Codes Council

7

There is actually another possibility, which is to bypass the Canadian market and its

product approvals system completely. The U.S. market is big enough and close enough

that a Canadian manufacturer can decide to base its whole marketing plan on shipping all

its production to that country. The U.S. product evaluation system is somewhat different

than in Canada – with various pros and cons2. But if U.S. acceptance is granted, the

manufacturer may use that approval to build up a product history which can be used to

support later applications in Canada and elsewhere. This has a cost for Canadian

innovation and productivity, as Canadian innovators lose access to their home markets, at

least in the initial period, and Canadian consumers lose early access to their innovations.

5. Users’ requirements and expectations

Various groups have different needs and expectations of the product evaluation/code

conformity process. The needs of any one group can conflict internally (e.g. proponents’

requirements for access to work already completed by others, which can conflict with

their desire to protect their own proprietary information). The following table identifies

some of the key requirements and expectations of different groups.

Users Key requirements and expectations

Proponents of

innovative

products and

systems

process(es) allowing good products to enter the market

quickly and inexpensively

avoidance of unnecessary duplication across the country

process which provides a clear “pass” result

clear requirements outlining what must be done and why for

the technical evaluation, including tests, etc.

requirements limited to those necessary to predict conformity

to code requirements

scope and type of testing limited to the least amount and

expense reasonably necessary to predict conformity

access to information about any other concerns raised by

experts which may affect product performance or proponent

liability

way of limiting testing/research which can be demanded

because someone thinks some possible concerns should be

examined, where no problem is known

recognition of existing research

access to similar work already completed

protection of proprietary information about the innovation

effective appeal process

Competing

manufacturers no loss of existing regulatory acceptance based on traditional

performance, because of new listings for competing products

2 See Section 7.6 for more discussion on US product evaluations.

8


no loss of existing regulatory acceptance based on non-CCMC

processes such as industry standards, because of new listings

for competing products

high enough requirements for testing to make sure new

competing products or systems don’t get a market advantage

because of lower required performance levels

Builders



no unnecessary documentation/ certification requirements at

building permit stage

no loss of existing regulatory acceptance for products and

systems (based on traditional performance or non-CCMC

processes), including because of new listings for competing

products

evaluation system which weeds out products or systems that

will create callbacks or failures with possible builder liability

means of showing the builder took reasonable precautions to

avoid problems with non-standard products or systems (due

diligence)

Home buyers /

occupants




will create problems or failures

(unfounded expectation that these evaluations will deal with

issues such as appearance, etc.)

(unfounded expectation that failures will be paid for by public

sector agencies involved in review of products)

Local building

officials

easy-to-administer-and-document system for showing code

compliance (standards, certification, listing, etc.)

acceptance system procedures which the official can clearly

demonstrate were followed, should problems arise later


quickly and inexpensively, as long as that process does not

expose the municipality or building official to unreasonable

liability


will create problems or failures

Provincial

departments with

responsibility for



access to CCMC-NRC level science for evaluating products

9


building

and systems

process(es) they can offer or recommend to local industry

proponents and building officials, without duplicating

evaluation processes or taking on unreasonable liability

6. CCMC is becoming the only game in town

Local

Even though the codes were never intended to require certification or product evaluations

listings for all products and systems, building/safety codes officials are requiring them for

more and more things. This trend seems to have sprung from several large product and

systems failures in recent years3. Also, there is more understanding of the way

components must work together as a system, and some new products and systems are

becoming extremely complex.

Acting on the advice of lawyers, insurers and sometimes provincial departments,

building/safety codes officials have dramatically reduced their willingness to accept

anything not clearly covered by codes unless it is:

supported by recognized standards or CCMC product evaluation/listing, or

designed and stamped by a professional architect or engineer, or

both the above.

As a result, the informal system of code conformity acceptance through local variances is

drying up.

For a long time, building/safety codes officials have been increasing demands for

professional design and assurance of code compliance for non-standard structural

systems. Now, they are looking for proof of code compliance for other products and

systems as well. The overall “tightening” has been worsened because new CCMC listings

seem to create the perception that existing products and systems which have traditionally

just been accepted on the basis of historical performance now have to get an evaluation as

well. CCMC points out that this is not their intent.

Alberta points to the example of rimboards. When one product received a new CCMC

listing, some officials refused to accept other rimboard products which didn’t fit the

CCMC criteria. This ignored an apparently functioning APA standard and full

certification program by APA-The Engineered Wood Association. The APA is currently

considering preparing an ORD and asking PTPACC to act as a liaison body. CCMC

notes that its rimboard evaluation was initiated by request of a rimboard manufacturer

that is not a member of APA. The APA service is exclusive for its members. So the

3 Leaky condos in BC, pine shakes in Alberta, plastic venting for mid efficiency furnaces in

Ontario, etc.

10

CCMC evaluation criteria, which is largely based on the APA protocol except for some

additional Canadian requirements (i.e. limit states design), is open to all manufacturers,

non-APA and APA members. However, CCMC does not run a certification programs.

(The APA standard is not referenced in National or Provincial Codes, although the

certification had been commonly accepted by building officials.)

This kind of response at the implementation level is putting more pressure on

manufacturers and other proponents to seek unnecessary CCMC approvals for products

which have previously been accepted. This is not CCMC’s intention, but the outcome is

not good, and should be addressed.

Provincial/Territorial

In the late 1980s, provincial / territorial representatives agreed to support a national body

to provide evaluations. In accordance with that decision, provinces and territories have

taken steps to encourage the use of CCMC evaluations as a basis for determining

acceptability of products in regards to building code requirements within their

jurisdictions. This seems to have been accelerated by the same concerns about liability as

mentioned above.

In 1992, for example, Alberta terminated its own product listings service in favour of

recognized CCMC listings. Any manufacturers or other proponents who asked the

province for evaluation were referred to CCMC. With its 1997 update to the Alberta

Building Code, the province also specifically said that “evaluation reports issued by the

Canadian Construction Materials Centre, National Research Council of Canada, or an

organization approved by the Director may be used in determining compliance” with the

requirements of the Code (2.4.1.1.(2)). So far, apparently, no other organizations have

sought or been given approval by the Director.

Similarly, Ontario had an active Building Materials Evaluation Commission. The BMEC

is still functioning. However, in the wake of the PTPAC decision, the province directed

BMEC not to accept applications for new reviews unless CCMC confirms that it is not

working on or about to work on an evaluation of that type of product or system.

When Ontario first introduced provincial building code legislation in 1975, it did not

adopt the concept of equivalencies. As a result, the code did not identify any groups as

accepted to provide product evaluations. Now, it has specifically named CCMC, but

without an avenue for products evaluated by other organizations. Already, some product

manufacturers with CCMC evaluation listings have claimed they have the “only products

recognized by the Ontario Building Code” – despite existing non-CCMC certification

systems for alternate products or traditional acceptance.

11

National

There are advantages to having one single national products evaluation service serving

the need to prove code conformity – as long as it can meet the requirements of its many

users, and facilitate introduction of desirable innovations and non-standard products.

It is worth noting that the United States has only recently consolidated three of its main

code compliance product evaluation services. The services associated with the three

former model building code development organizations now cooperating as the

International Code Council (ICC) are now known as ICC-ES. The arguments in favour of

this consolidation, as explained in a recent article4, were strong: “a single national listing

service verifying the code compliance of new building products and design methods

would facilitate the reduction of trade barriers, both regionally and internationally,

without diminishing safety levels.”5

However, it seems unlikely that the goal of a single national products evaluation service

in Canada contemplated the changes in underlying approval systems, and the severe

restrictions on local approval of equivalents, taking place today.

7. Some of the issues are complex

Determining code compliance might not at first sound like a hugely complicated task. But

it’s not quite as simple as it seems. Issues include:

7.1 How will a change affect the rest of the system?

There is increased understanding that a building functions as a system. A large number of

elements work together to create the structure, the envelope, the heating/cooling systems,

etc. – and changes to any one of these may affect the others. Incompatibility of materials

has caused several problems. Adhesives and fasteners accepted for one material may not

be appropriate for another.

One presentation on building envelope design issues6 gave several examples of other

changes that may jeopardize other elements. Here is one: “There has been a fundamental

shift in window design over the past 15 years, from frames complete with brick molds

and self-flashing sills (sloped and with a drip on the underside) to frames with nailing

flanges, no brick mold or brick mold that is a clipped-on accessory, and little or no

overhang at the sill. This means that external sealant is often relied upon as the only

means of protection against all water transfer mechanisms. Internal window leakage at

the jamb to sill joint and at thermal breaks has been observed more often in the past 20

4 News and Report magazine, Spring 2002, Building Officials Association of British Columbia

5 There are still other evaluation bodies such as the National Fire Prevention Association (NFPA).

6 Driving Beyond the Headlights, a presentation to the 9th Canadian Conference on Building

Science & Technology in February 2003, by David L Scott, B.Arch., Vice President Building Engineering,

Morrison Hershfield Limited.

12

years compared to previously. The deposition of water behind the line of the sheathing

membrane almost always results in corrosion of steel studs or rotting of wood framing.”

When a proponent brings forward a new product or system for evaluation, how far should

evaluators go in assessing its impact on the rest of the building system? For how long

should it have to show acceptable performance, and with what cautions as to care and

maintenance?7 Given that CCMC and scientists at NRC may be able to think of any

number of research projects into potential areas of impact, how should those be assessed

so that only the necessary and cost-effective ones are required?

7.2 Unstated assumptions

Scott’s presentation on building envelope design identified some of the issues for

designers 8

– and these can affect evaluations as well.

1. Building codes only cover certain issues: the code “only provides

requirements where the construction community has agreed that some aspect

of design or construction needs to be regulated”.

2. Prescriptive solutions in the code “may depend on unknown assumptions and

therefore might not be suitable” for other uses.

3. Codes often contain performance requirements along with prescriptive ones –

and compliance with the latter does not necessarily imply conformance with

the former.

4. Standards referenced in codes may have their own written or unwritten

assumptions or limitations.

We could add to this the issue of time. The Codes do not spell out how long products and

systems have to perform – and those countries who have tried to do so have run into

serious problems. For how long should an innovative product or system have to

demonstrate acceptable performance, and using what kind of research tools?

At the other end of the scale, aesthetics is another issue clearly not covered by codes.

Some evaluations have looked at colourfastness as a potential issue, for example.

Several CCMC evaluations have suggested other “implicit performance requirements” of

codes. There are often arguments about these, especially if they do not seem to relate to

the proponent’s vision of the product or system and will lead to expensive and/or time-

consuming tests.

Is CCMC the appropriate place to define those? How will their conclusions be fed back

into the codes themselves? Since home builders and home owners may eventually bear

the liability for problems with new products, how far do they want evaluators to go in

assessing unwritten assumptions that may affect performance? As above, how should

possible research/tests be assessed so only the necessary and cost-effective ones are

7 Codes do not typically address this issue.

8 ibid.

13

required? If CCMC scope is limited so only explicit code requirements are tested, how

will that be communicated to builders? With the publication of the intent statements for

the code requirements, the accepted implicit requirements have in effect become

documented and therefore explicit – what are the implications?

7.3 Equivalent or Acceptable performance

Some industry representatives feel that the main problem in getting timely, cost effective

evaluations is the insistence that new products and approaches provide performance

which is equivalent to the traditional approaches.

Building Codes have for a long time contained wording similar to this Section 2.5 of the

model National Building Code of Canada (on which all Provincial and Territorial

building codes are based):

2.5.1 Alternate Materials, Appliances, Systems and Equivalents Permitted

The provisions of this Code are not intended to limit the appropriate use of materials,

appliances, systems, equipment, methods of design or construction procedures not

specifically described herein.

2.5.1.2 Evidence of Equivalent Performance

Any person desirous of providing an equivalent to satisfy one or more of the

requirements of this Code shall submit sufficient evidence to demonstrate that the

proposed equivalent will provide the level of performance required by this Code.

2.5.1.3 Equivalence Demonstrated by Past Performance, Test or Evaluation

Materials, appliances, systems, equipment, methods of design and construction

procedures not specifically described herein, or which vary from the specific

requirements of this Code, may be used if it can be shown that these alternatives are

suitable on the basis of past performance, tests or evaluations.

Many traditional products and systems are accepted in the codes because of their past

performance – there are no actual “performance” requirements, but there is a successful

history. There are good reasons for requiring new products and systems to provide an

equivalent level of performance – because that successful history is a strong argument

and other systems (modeling, design calculations, simulators, etc. to determine acceptable

performance) can’t necessarily give the same degree of assurance. But it is possible that

traditional products and systems may provide a higher level of performance than is

actually required. Naturally, proponents of those traditional products and systems do not

want to face competition that can provide lower levels of performance. Equally naturally,

proponents of the innovative products and systems feel they are being unfairly excluded

and that the criteria should be “acceptable” performance instead.9

One of the key decisions in framing the objective-based codes was whether to require

equivalent performance to traditional approaches, or allow performance which is “good

enough”. The Canadian Commission on Building and Fire Codes decided that it was

9 Proponents can, of course, propose a code change to accommodate their innovative approach.

14

important to continue the concept of “equivalency”; otherwise, local building officials

would be put into the position of deciding what is “acceptable” building performance.

This type of decision, the group decided, should be addressed at the national level – as a

proposed code change – subject to the full system of policy guidance, consensus process,

technical and public review.

Should this apply to CCMC as well, or could that group address “acceptable”

performance? Would opening the issue of “acceptable” performance jeopardize existing

requirements in the codes, particularly under the objective-based system?

7.4 Performance of traditional systems

Some products or systems are alternative to traditional construction which has never been

tested. In that case, an evaluation may require extensive tests of the traditionally accepted

systems, in order to provide the benchmarks against which the new one can be measured.

And in deciding the performance of traditional approaches, what is not in the code

because it is assumed (performance of traditional materials, installation quality, even

level of expertise and training) can create serious problems and disagreements. As above,

how far should CCMC go in examining implicit requirements for these systems?

CCMC requires the proponent (or consortium of proponents and sometimes other

interested groups such as CMHC and NRC) to pay for the testing of the traditional

systems as part of the Technical Guide. This can be extremely expensive and take a very

long time. Are there better ways to fund research into traditional systems?

Proponents are given a 12-month “protection” on the resulting Technical Guide (if no

other manufacturers join in and share costs). Is this too long? Long enough?

7.5 Interim acceptances (R & D process)

Proponents say there should be a special process for interim approvals. This would be

used, for example, if the initial investigation does not show any major concerns but

additional testing is being done to confirm assumptions, or on non-critical issues, before

the full evaluation is completed. Or it could be used where the product or system has

already been accepted or used under similar conditions elsewhere without hazards, but

requirements here are different in some respect so further testing is being done.

A proposal for an interim system, Regional Assessment Service, has foundered because

CCMC already has a full assessment service, which it considers to provide the

appropriate level of investigation for determining the suitability of innovative

approaches. Provinces in those discussions wanted CCMC to accept full liability if

problems arose, and the Centre did not think that was appropriate for a fast-track, less

exhaustive review.

The proposed system would have had four main components:

a panel of relevant experts would examine the manufacturer’s existing

information, research and test results on its proposed product or system

15

they would then give an informed opinion as to whether the products should be

allowed to be used while the formal process of full product evaluation took place

that decision would be backed up with appropriate legal and financial assurances

from the manufacturer (taking liability for product failures, backed up with

possible bonds or other financial assurances)

the decision would also spell out conditions for monitoring results, restrictions on

use, etc.

Are the liability concerns “real”? Would review or approval of interim acceptances by a

wide regulatory group such as PTPACC help? If the product failed the tests, how would

that be handled? Can interim approvals be issued with additional measures to track and

respond to potential problems in the field?

7.6 Information provided by others

a) Manufacturers’ own information:

Proponents are naturally inclined to present their “best-case” scenario. Their test

data may not cover all use situations. Many test methods are available: they may

not be equally consistent or technically challenging. Laboratories are accredited

as having the facilities and staff to perform tests, not necessarily to choose the

most appropriate ones to conduct for code equivalence. And in the market, as

Scott points out10

“Application instructions are limited to the product itself and

avoid discussion of adjacent connecting systems for fear the proponent will

assume some liability for the performance of the system … Most reliable in the

author’s opinion is the use of recognized evaluation programs such as that

established at the Canadian Construction Materials Centre.” On the other hand,

appropriate tests done for the manufacturer by SCC-accredited testing agencies

should be accepted unless there is a compelling reason for further investigation.

b) U.S. Evaluations:

While CCMC and the International Code Committee Evaluation Service

exchange information and try to use each others’ work, CCMC states that product

evaluations prepared in other jurisdictions such as the US do not always meet

their criteria. There is a perception that the US process may rely much more on

manufacturer-defined performance criteria and testing regimes. This may work

within that country’s much more litigious environment where manufacturers

eventually bear more of the final costs of errors, they say, but is not suitable for

Canada. In the US legal environment, US evaluation bodies appear to have no

liability for product acceptances; people who suffer damages because of faulty

products sue the manufacturers11

. A comparison of the US and Canadian systems

of liabilities – and whether the US product evaluations produce higher failure

rates than Canada’s system – was beyond the scope of this paper.

10

ibid

11

Judging by some recent court cases, the Canadian system may work in a very similar way.

16

It is important to look at what the countries’ building codes require. Canada and a

number of other countries use a different approach to design calculations (limit

states) than the US, for example. Other questions are what information was

considered, and what tests were done and how. Was the same sample or set of

samples used for all tests, or can manufacturers use the best results from different

samples? Also, most of the U.S. system is apparently open for public input and

comment12

. This is more transparent than the CCMC approach, which places

more emphasis on protecting proponents’ proprietary information. However,

some competitors are thought to use the U.S.’ openness to delay new products and

systems and/or develop similar ones.

There may well be differences – but are they important enough to justify a

separate set of evaluation tests and requirements for products already accepted

through a US product evaluation? Perhaps this question should be referred to

PTPACC as well as the CCCME.

7.7 Time and Costs

All of the above issues can affect the time and cost of product evaluations. So can access

to appropriate experts’ time (NRC scientists, university researchers and other outside

experts), differences of opinion about the kinds of tests required, performance targets to

be met, etc., and time taken by proponents to arrange consortiums, funding, analyze

whether the evaluation is worth pursuing, etc. Time and cost are crucial issues for new

products and systems. A risk rating and management system, as well as limits on scope of

evaluations (see below), may be helpful.

7.8 Scope of CCMC evaluations

During 2004, the CCCME’s Standing Committee on Technical Evaluations has examined

the goals and scope of CCMC product evaluations. At the moment, CCMC uses seven

key guidelines to guide the scope of its evaluations:

1. All proponent-requested performance issues could be addressed.

2. Code-explicit requirements (e.g. health and safety) must be addressed.

3. Performance issues should be addressed if there is some relationship to the

original health and safety expectations, explicit and implicit, of the code.

4. Performance issues should be addressed if they are explicitly covered by a

code-referenced standard for similar product applications.

5. Evaluation criteria could be established to assess performance levels based on

extensive scientific justification or in anticipation of code changes. There

should be consultation with the CCMC standing committee.

6. Once a performance issue is addressed for a family of products, or a specific

material, then all similar products or materials need to be subjected to that

issue, other than proponent-requested claims.

7. Performance issues must be addressed in relation to a product’s intended

application (use, climate, geography, etc.).It has proposed some changes to the

current system.

12

Also, one interviewee pointed out that the system in the U.K is apparently all public, while that in

France is all private.

17

Wording of these principles is unclear. They include some questionable elements

(implicit performance issues which have “some relationship to the original health and

safety expectations”, and evaluation criteria included “in anticipation of code changes”

leap out as requiring reconsideration, or at the very least strict limits). They also leave out

some issues which appear to be important. For example, nothing in this list specifically

addresses cost issues, practicality, or how to relate tests and research to a “likelihood of

problems” factor.

The Subcommittee on Technical Evaluations has proposed redefining these criteria to

four simpler, more direct ones, as follows:

1. Code explicit requirements shall be addressed. These requirements shall be

addressed in relation to a product’s intended use, based on scientific justification

and linked to objectives.

“Addressed” is achieved by scientific justification (such as testing, calculable

solutions, modeling, field studies) or professional judgement that they attain the

minimum level of performance against acceptable Code solutions with respect to

the applicable objectives and the functional statements of the Code.

2. Evaluation criteria could be established on anticipated Code changes that have

been approved by the appropriate standing committee of the Canadian

Commission on Building and Fire Codes.

3. Once a requirement (other than proponent-requested claims) is established for an

intended use, then that requirement shall be addressed for all similar products or

materials, with the same intended use.

4. Proponent-requested performance claims beyond Code explicit requirements shall

be confirmed based on scientific justification.

(Performance issues beyond the Code should be identified separately in the

evaluation strategy.)

Do these proposed changes go far enough? Do they exclude all analysis of implicit

requirements? How will that be communicated to builders, so they do not expect more

from the products than has been tested? Are the identified code objectives written tightly

enough to limit unnecessary testing and performance requirements?

8. Objective-based Codes will have an impact

The Canadian Commission on Building and Fire Codes and NRC/IRC’s Codes Centre are

in the final stages of preparing a new objective-based version of the National Building

Code. This takes a new approach to regulatory requirements with the specific goal of

encouraging innovation.

The front of the book (Division A) is a short section which sets out the broad

purposes behind the code requirements:

18

o the set of four broad “objectives” behind building regulations, together with

the more detailed sub-objectives, plus

o “functional statements”, which give a brief description of what the code is

trying to achieve in certain areas (emergency egress requirements, for

example, are “to facilitate the timely movement of persons to a safe place in

an emergency”)

The objectives and sub-objectives define the scope of the Code much more precisely

than has ever been the case in the past; an issue that can’t be related to one of the

objectives will not be dealt with in the Code.

Division A also defines exactly how compliance is determined. Clause 1.2.1.1 (1)

states that : “Compliance with this Code shall be achieved by

a) complying with the applicable acceptable solutions in Division B (see

Appendix A), or

b) using alternative solutions that will achieve at least the minimum level of

performance required by Division B in the areas defined by the objectives and

functional statements attributed to the applicable acceptable solutions.

The second and much longer part of the book (Division B) contains acceptable

solutions (which also become the basis for evaluating alternatives). This consists of a

complete set of practical, enforceable building regulations, including:

o most of the traditional code requirements everyone is used to, renamed

“acceptable solutions”, plus

o lists of all the specific sub-objectives and functional statements behind each

one

The CD-Rom (but not the printed code) contains a huge amount of advisory material,

including:

o statements detailing the specific intent(s) of each requirement

The new system has been specifically designed, among other issues, to make the

acceptance of products and systems at the local level more accommodating to innovative

products.

As mentioned above, the NBC has always had provisions for allowing equivalents.

However, a number of building officials concerned about potential liability have

preferred to ignore them. Clause b) in Subsection 1.2.1.1 makes it clear that “alternative

solutions”, which is how innovative products and systems are classified, must be

considered.13

Then, the new objectives, functional statements and intent statements set

limits on what the consideration has to include – i.e., in which areas the new products and

systems have to show equivalent performance.

To take one very simple example, for Balconies, the NBC says:

13

A reasonable system to limit potential liability would help to support this goal.

19

Residential balconies not used as passageways shall be designed to carry the

design roof snow load or 1.9 kPa, whichever is greater.

The identified Functional Statement is F20 …

F20: To support and withstand expected loads and forces

as it relates to the Safety objective OS2, and the Fire and Structural Protection of

Buildings objective OP2.

OS2 Structural Safety

An objective of this Code is to limit the probability that, as a result of the design

or construction of the building, a person in or adjacent to the building will be

exposed to an unacceptable risk of injury due to structural failure.

One of the sub-objectives is attributed to the requirement:

OS2.1 loads bearing on the building elements that exceed their load-bearing

capacity

OP2 Structural Sufficiency of the Building

An objective of this Code is to limit the probability that, as a result of its design or

construction, the building or part thereof will be exposed to an unacceptable risk

of damage or loss of use due to structural failure or lack of structural

serviceability.

One of the sub-objectives is attributed to the requirement:

OP2.1 loads bearing on the building elements that exceed their load-bearing

capacity

For the short term, it’s not unreasonable to expect everyone to be cautious in evaluating

innovative solutions until they adjust to the new wording and structure. And, unless the

ongoing concerns about potential liability are addressed (see below), building officials

may continue to be unwilling to accept equivalents for some time to come.

It is not certain how much impact the transition to the new system will have on CCMC’s

own work, but Centre personnel have stated that the new approach will help them to

define issues and narrow the focus in preparation of Technical Guides.

There has been some discussion of introducing two different classes of CCMC

evaluation:

“acceptable solution” – meets the Code’s prescriptive requirements (e.g. Division

B of the OBC) – this would be equivalent to today’s product listing

“alternative solution” – addresses the same issues as solutions in Division B, and

meets the explicit Code objectives and functional statements

20

9. Recommendations

Structural Change:

9.a Fix the problem of liability

The prospect of taking on responsibility which far outweighs their actual level of

involvement or fault is casting a “liability chill” on acceptances – and may

eventually result in an “innovation freeze”. Fixing the liability issue is not an easy

task, but it should be addressed. A concurrent step should be a new assessment of

the real extent of the risk. It appears that courts are willing to accept processes

which have been based on agreement by a diverse group of informed experts.

Judges seem to support them as a “reasonable standard” for setting requirements

and evaluating alternatives. A policy statement by governments to that effect

might help as well. As mentioned below, a similar policy statement to exempt

persons preparing or approving the interim acceptances from liability except in

cases of gross negligence might also be appropriate. If the risk issue can be

brought under control, some of the pressures on CCMC and alternative forms of

evaluations/acceptance will disappear as well.14

9.b Apply the Principles of Smart Regulation

Recently, a new approach to government regulatory action has begun to include

risk assessment and management as a key test of regulatory actions. This aims at

ensuring that the chosen mix of regulatory instruments responds to the actual risk

and need for public sector intervention.

For example, the External Advisory Committee on Smart Regulation reported to

the government of Canada in September 2004. This group took a broad approach

to the subject, saying that: “regulation encompasses a range of instruments that

includes formal rules, such as statutes, subordinate legislation (regulations) and

ministerial orders, as well as less formal instruments, such as standards,

guidelines, codes, and education and information campaigns.”

Here are some of the Advisory Committee’s key recommendations:

Regulators should understand how the different levels of tools work, and

choose a mix of appropriate (least intrusive and most flexible) ones.

Precautionary measures – those based on a possibility of problems instead

of evidence of problems – should be carefully controlled and explained.

Decisions should be based on a risk management framework that:

o assesses risks

14

People raise the issue of how homeowners are to be compensated if problems do slip through this

net of reasonable care and diligence, and arise later. Accepting a low level of risk is essential to the smooth

functioning of any economy. Traditionally, these risks (not due to fault or negligence) have been borne by

the homeowners themselves. If society as a whole decides that this is not fair, then a form of disaster

compensation should be provided by government, out of the taxes paid by all citizens, including

homeowners and industry.

21

o prioritizes them

o communicates risk levels and considerations

o consults widely

Performance of regulations should also be defined and assessed.15

These principles should be applied to the evaluation of building materials and

systems, on a common sense basis.

Changes directly affecting IRC/CCMC:

9.c Limit the scope of CCMC evaluations

The CCMC consists of a small group of dedicated technical evaluators, housed

within a body dedicated to scientific research. Their scientific consultants are

trained to see possible difficulties and implications and to explore all possible

hypotheses. They need a policy which will allow them to do what is reasonable

rather than everything they can imagine, without having to worry about people

second guessing their actions later. The publication of the intents of code

requirements will help to alleviate this situation.

The proposals drafted by the Subcommittee as amended by the CCCME should be

supported. In addition, some specific guidelines should be drafted to ensure that

any required research and testing is based on the most practical and cost effective

methods available.

9.d Define and use a simple process for risk rating, management and due diligence

From the outside, it appears that the most important requirement may be a simple,

defensible process for managing the potential risk posed by new products and

systems, and their evaluation. Legal tests of negligence are usually based on what

a “reasonable man” would have done in the circumstances. As mentioned above,

courts will often accept adopted policies, standards and/or consensus views in

determining what is reasonable.

That does not mean throwing out the baby along with the bathwater. A lot of

people, builders among them, expect the evaluation system to identify products

and systems which will meet Code objectives and not lead to later problems. But

it does mean limiting the investigations to those areas which are absolutely

necessary.

Therefore, where there are disagreements over the need for proposed performance

requirements and tests, these should be resolved using a fairly simple matrix

rating of the likelihood of certain problems occurring in the future, and the degree

of hazard they would pose if they did occur.

15

See http://www.smartregulation.gc.ca/en/index.asp

22

These ratings should be completed on the basis of existing technical knowledge

and opinion, by a group of appropriate practitioners representing not just

scientific/research approaches but also practical on-site expertise. Only those

where the ratings fall in the lower right area of the chart should be addressed.

The proposed restatement of the scope of CCMC evaluations may already exclude

all consideration of implicit requirements. If it does not, those should also be

determined by a simple risk rating approach. Nothing beyond strict compliance

with code wording and intents of requirements should be considered unless a

group of appropriate experts with scientific/research and site experience agrees it

has both:

1) a high probability of problems and

2) a high potential for harm (health and safety issues) if those problems

materialize.

CCMC also needs a procedure to justify using less expensive research and testing

methods wherever possible. The strategy contained in SEPs and TGs should

always be reviewed against the principles of need and cost efficiency, again using

a simple risk matrix rating. So should the evaluation process itself. One positive

step might be to have an outside review of performance and testing issues very

early in the process, instead of just once the TG is already drafted (which is when

it currently goes to the Subcommittee on Technical Evaluations for comment).

This review, which was also recommended by the task group on scope, should

have the express aim of ensuring only necessary and cost efficient tests are

required. CCCME recently has directed the CCMC to form a task group with the

chair of technical standing committee to examine how the TSC can have earlier

input into the process.

Another useful step might be a government policy directive that CCMC must

accept evaluations performed in other areas with similar health and safety

concerns, unless it can show specific risk-based reasons for not doing so.

Very Low Low Moderate High Very High

Very Low

Low

Moderate

High

Very High

Lik

elihood o

f Pro

blem

s

23

9.e Clearly communicate the actual scope of CCMC evaluations

As CCMC moves from more broadly-based performance evaluations to

evaluations based on code compliance, it will be important to inform people in the

building industry and in the wider public of this change. Builders, for example,

will need to know the new extent so they can assess if they need to ask additional

questions of their manufacturers on non-code issues that may still affect customer

complaints or callbacks. In addition, the note to the proposed new principle 4,

which says performance issues beyond the Code should be identified separately in

the evaluation strategy, is important, and should be supported. These non-code

issues should be clearly separate in the report as well. Access to a list of topics

raised but dismissed, along with the reasons for dismissal (e.g. not explicit in

code, or low risk rating), might also be helpful.

9.f Investigate issues of “Equivalent” or “Acceptable” performance

A recurring complaint about the product and system evaluation process is the

requirement to demonstrate “equivalent” performance to the current acceptable

solution. CCBFC has considered whether it would be appropriate to enable local

building officials to determine an “acceptable” level of performance, and

recommended against it. That role may be appropriate for CCMC, if requested by

a proponent. On the other hand, taking this step may open up large areas of

liability for all concerned, including manufacturers and builders.

This process can identify areas where the code requirements should be changed. If

a new approach provides an acceptable level of performance which is less than

the code-defined level, the code requirement does not truly reflect minimum

acceptable performance levels. However, as for requests for changes to referenced

standards, changing code requirements can be a very long process. In the interim,

CCMC evaluations and reports – which are designed to help innovations onto the

marketplace more quickly – could be appropriate.

CHBA may want to ask the Provincial/Territorial Policy Advisory Committee on

Codes (PTPACC) and Canadian Commission on Construction Materials

Evaluation (CCCME) to look at these issues. It may also want to engage Canadian

Commission on Building and Fire Codes (CCBFC), as the group responsible for

the Building and Fire Code requirements against which CCMC evaluates products

and systems.

9.g Provide a functional alternative to the traditional interim acceptances offered

by local/provincial officials The concept of interim acceptances has bogged down over which group is willing

to put its name on the resulting document – and potentially attract liability if

anything should go wrong.

It is important to innovators that this role gets filled. As mentioned above, there

are distinct advantages to having more complex evaluation functions performed

by one national body. Since it is well on its way to achieving that status, CCMC is

24

the obvious candidate to fill this role. Its concerns about liability should be re-

examined.

It should prepare and follow a flexible process of accepted procedures designed to

rate and manage risk with due diligence. This would include procedures to

categorize risk levels, as discussed above, and support approval where risks are

deemed acceptable. In some cases, it could also include increased requirements

for manufacturers to limit initial production runs, track installations (where this is

possible), report problems, provide security or assurances, have procedures in

place to respond to any hazards that arise, etc. This would be an extension to

CCMC’s current role: perhaps it could work with existing SCC-accredited

certification groups for the monitoring functions. This increased tracking and

monitoring would result in increased costs.

Review of interim acceptances by the PTPACC might also help spread the risk

and responsibility – as long as that group could take on this role without the

system grinding to a halt. Legal changes to make those people who prepare,

comment on or approve the interim acceptances exempt from liability (possibly

except in cases of gross negligence) might also be appropriate.

If CCMC is for some reason unable to take on this role, the CCBFC and PTPACC

must find another acceptable method of filling this need.

9.h Review CCMC’s capacity to perform the growing amount of work, and ensure

it has the necessary resources

The CCCME has recommended some changes to focus and streamline CCMC’s

work, and this paper has some further suggestions. These changes may be

sufficient to accommodate the predicted growth in demand for CCMC’s services,

given uploading of evaluation. Within the next year, the situation should be

reviewed, to ensure the Centre has sufficient capacity and resources. It should be

reviewed again once the impact of the new objective-based codes can be

determined.

Changes affecting other jurisdictions

9.i Resolve the issue of non-CCMC evaluations, and non-evaluated products

This is more of an issue of how provinces and territories deal with accepting

equivalencies than what CCMC does. There are a number of groups with

standards and/or certification programs whose information or approvals are being

undermined by current policies. The list of groups whose information can be used

to support applications for “equivalency / code conformity” should be expanded.

Directives clarifying that traditional non-evaluated methods continue to be

acceptable may be required. This problem should be referred to the PTPACC for

resolution, possibly jointly with the CCBFC and the CCCME.

25

At the same time, CCMC must make sure to apply its policy of referring

proponents to appropriate other standards and evaluation/certification groups,

where those groups exist. It should also be aware of the impact its listings may

have on traditional competing products and structure its reports/listings to avoid

unnecessary problems, wherever possible.

9.j Investigate expanded use of alternative compliance acceptance methods

Some groups are currently considering preparing ORDs and asking the PTPACC

to act as its regulatory liaison body. Although a somewhat convoluted approach,

this could provide a good form of national regulatory acceptance for certain

equivalents – assuming the PTPACC decides to continue supporting the concept

of ORDs. The accredited certification bodies should then be included by the NBC

and individual provinces in their lists of acceptable support for equivalents.

CHBA Discussion Paper on the

Canadian Construction Materials Centre and Code Conformity Assessment

October 2009 SUPPLEMENT

October 2009 Rowena E. Moyes

[email protected]

Executive Summary CHBA’s 2005 discussion paper on the CCMC and Code Conformity Assessment is still relevant today. Builders need evaluation systems to provide: • process(es) allowing good products to enter the market quickly and inexpensively • process(es) which weed out products or systems that will create serious callbacks or

failures with possible builder liability • no unnecessary documentation/ certification requirements at building permit stage • no threat to regulatory approval for products and systems which have traditionally

been accepted based on longstanding performance, non-referenced standards, etc. • means of showing the builder took reasonable precautions to avoid problems with

non-standard products or systems (due diligence) There have been some changes to processes since 2005, including new guidelines on the scope and practice of product evaluations, risk assessment to guide product evaluations, some shifts to improve consistency, and an effort to clear out dormant files. Definition of objectives and functional statements in the 2005 codes appears to have helped focus CCMC evaluations, although there still appear to be cases of unnecessary requirements. A Business Review currently under way offers an opportunity to look at new options. A number of issues identified in the 2005 paper are still outstanding. It is still difficult to get local inspectors’ approval for alternatives, although the objective-based codes may have helped somewhat. Many people still seem to see the CCMC as the “only game in town” for innovative product and systems approval. This is likely to be exacerbated by the growing demand for action on greenhouse gasses and climate change, especially reductions in use of fossil fuels and other carbon generators. Energy efficiency and water conservation have already been added as objectives to the Ontario and British Columbia building codes and are expected in the National Building Code no later than 2012. It is possible that significant numbers of innovative products and systems from here and abroad will be seeking approvals. Some new approaches may be worth further examination, including:

• bringing building officials back into the process, with guidelines/protocols based on levels of risk

• recognizing good, accepted risk reduction tools • allowing CCMC and others to prepare risk assessments as a stand-alone product • accepting other test results • anticipating new demand (core requirements for new classes of products,

facilitating cross-jurisdictional approvals, etc.) • reviewing manufacturing plant operations (for higher risk products) • creating a new volunteer-based decision-making (rather than advisory) body,

similar to the CCBFC • accrediting additional competing code evaluation and certification bodies

Table of Contents

Executive Summary.....................................................................................................ii Table of Contents ...................................................................................................... iii 1. Scope......................................................................................................................1 2. Summary from the March 2005 paper.....................................................................1 3. Some changes to procedures ...................................................................................3

3.1 Limits to the scope of CCMC Evaluations.................................................3 3.2 Process for risk rating, management and due diligence ..............................3 3.3 Shift to reduce inconsistency .....................................................................5 3.4 Clearing out the backlog............................................................................5 3.5 Additional process concerns ......................................................................5 3.6 Business review ........................................................................................7

4. Experience with objective-based codes ...................................................................7 5. Potential impact of new ‘green’ products and technologies .....................................8 6. Responding to new challenges ..............................................................................10

6.1 Bringing more building officials back into the process ............................10 6.2 Providing guidelines/protocols for building officials ...............................10 6.3 Recognizing good, accepted risk-reduction tools in the protocols ............11 6.4 Allowing CCMC and others to prepare risk assessments as a stand-alone product...........................................................................................................12 6.5 Accepting other test results......................................................................12 6.6 Anticipating new demand........................................................................13 6.7 Using outside expertise for full evaluations .............................................13 6.8 Reviewing manufacturing plant operations for higher risk products.........13 6.9 Creating a volunteer-based decision-making (rather than advisory) body.14 6.10 Accrediting additional code evaluation and certification bodies.............14

7. Conclusion ............................................................................................................15 Appendix A: More Details of the CCMC Process for Risk Rating .............................16

1

1. Scope This paper is issued as a supplement to the Canadian Home Builders’ Association’s 2005 discussion paper on the Canadian Construction Materials Centre (CCMC) and Code Conformity Assessment. It is not a broad new investigation, but provides an update on specific process changes, challenges and opportunities, with particular emphasis on:

• changes to procedures • experience with the objective-based codes which have come into use since 2005 • potential impact of new ‘green’ products and technologies

2. Summary from the March 2005 paper CHBA’s initial discussion paper was a wide-ranging look at the CCMC, which is still relevant today. Anyone with an interest in this topic is encouraged to read the full 2005 document. We have reproduced its Executive Summary below.

1) The CCMC is designed to address a real need Canadian building and fire codes define what products and systems must be used by law to protect health and safety in housing construction. Many of the requirements refer to product, performance, design or installation standards, or to traditional approaches. However, as new products and systems are developed or brought in from other countries, they may use different approaches. The CCMC is a national body established to evaluate whether these products or systems meet the requirements of the building and fire codes.

2) The informal system of equivalency acceptance is breaking down

Traditionally, local building officials have accepted alternative products and systems which can show ‘equivalent’ performance to code, often based on manufacturers’ tests, etc. However, with the growing complexity of construction and with growing awareness of liability, officials in a number of localities have begun to back away from accepting equivalents. Or, they will only accept them a single project at a time, with code conformity confirmed by professional engineers or architects. At the same time, provinces / territories have given their support to a national system of evaluation. In some cases, provinces have discontinued or restricted their own evaluation services.

3) Pressures are increasing

CCMC evaluations can take a long time. Evaluations and the associated tests can be extremely expensive – particularly if no performance data currently exists for a traditionally accepted product or system. In that case, someone wanting to use a new approach must not only test their own product or system but the traditional ones as well. As the informal acceptance system dries up, attempts to provide an ‘interim acceptance’ process have foundered because of liability concerns. Meanwhile, some products or systems which have just been accepted in the past with no specific code requirements, or those only partially covered by existing

2

standards, or even those with industry-recognized but not code-referenced standards, are suddenly being asked to provide evaluations.

4) Definition of role and scope is under revision

The existing policies do not contain clear enough definitions of what the CCMC evaluations cover and do not cover, or what they should require or not require. The Canadian Commission on Construction Materials Evaluation has been examining this issue. New policies should address practicality, cost efficiency and how to relate tests and research to a ‘likelihood of problems’ factor. They should be based on simple risk assessment and management procedures which allow CCMC to do what is ‘reasonable’ and not ‘everything they can think of’.

5) Recommendations

Structural a. Fix the problem of liability. b. Apply the principles of Smart Regulation.

CCMC/NRC and Advisory Commissions c. Limit the scope of CCMC evaluations. d. Define and use a simple process for risk rating, management and due

diligence. e. Clearly communicate the actual scope of CCMC evaluations. f. Create a practical avenue for review and/or appeals. g. Investigate issues of ‘Equivalent’ or ‘Acceptable’ performance.1 h. Provide a functional alternative to the traditional acceptances offered by

local/provincial officials. i. Review CCMC’s capacity to perform the growing amount of work, and

ensure it has the necessary resources.

Other jurisdictions j. Resolve the issue of non-CCMC evaluations, and non-evaluated products. k. Investigate expanded use of alternative compliance acceptance methods.

1 In many cases, it is an advantage for new products and systems only to have to prove that their performance is ‘equivalent’ to current accepted solutions in Part B of the Codes. However, sometimes it would be easier to show that performance of a new product or system is ‘acceptable’ under the Codes’ objectives and functional statements. This is particularly true where the performance of traditional construction methods either has not been analyzed (which opens the question of who should pay for that analysis), or goes beyond those levels (i.e., the old methods would be considered ‘gold plated’ if introduced today).

3

3. Some changes to procedures

3.1 Limits to the scope of CCMC Evaluations When CHBA’s initial paper was written in early 2005, the Canadian Commission on Construction Materials Evaluation (CCCME)2 was discussing draft simplified guidelines to help to define and focus the scope of CCMC’s evaluation services. Those have now been put into place. Here is the final wording:

1. Code explicit requirements shall be addressed. These requirements shall be addressed in relation to a product’s intended use, based on scientific justification and linked to objectives. (NOTE: Addressed is achieved by scientific justification (such as test evidence, calculable solutions, modelling, field studies) or professional judgement that they attain the minimum level of performance against acceptable Code solutions with respect to the applicable objectives and the functional statements of the Code.

2. Evaluation criteria could be established on anticipated Code changes that have been approved by the Canadian Commission on Building and Fire Codes.

3. Once a requirement (other than proponent-requested claims) is established for an intended use, then that requirement shall be addressed for all similar products or materials, with the same intended use.

4. Proponent-requested performance claims beyond Code explicit requirements shall be subjected to the same scientific rigour as in point 1. (NOTE: The performance issues beyond the Code be identified separately in the evaluation strategy.)

These guidelines are very similar to the proposals mentioned in CHBA’s 2005 paper. It appears that these guidelines are helping to reduce unnecessary work and delays, as long as the personnel involved do not apply them too conservatively.

3.2 Process for risk rating, management and due diligence The National Building Code’s concept of “limiting the probability (... that people ...) will be exposed to an unacceptable risk” acknowledges that there are acceptable levels of risk. CHBA’s 2005 paper recommended a simple risk assessment process be used to guide CCMC’s decisions and requirements and define ‘due diligence’. The Association’s suggested model contained five levels of ‘likelihood of problems occurring’ and five levels of ‘probable hazard if they were to occur’. We suggested that scope should be limited and extensive tests should not be required in lower risk situations. 2 The CCCME is a body established by the National Research Council to provide policy for and technical advice to the CCMC. Its members represent a matrix of interests and expertise from across the country.

4

A new CCMC risk rating procedure was approved at the end of 2008, and has been applied as of January, 2009. A risk matrix form is filled out collectively by the technical team associated with the product type, as part of the initial review of the client’s application. The risk assessment is revisited during the code analysis, and at other milestones. If the scope of the project changes, the risk analysis will also be reviewed. Ratings of 1 to 5 are applied on four criteria: probability of failure, severity of failure, impact on adjacent elements, and difficulty of detection. Weightings are prepared by multiplying the scores; i.e. probability of failure x severity of failure x impact on adjacent elements x detecting failure. Totals are scored into one of three categories, as follows:

• 1-81 = low risk (described as products which “typically have no risk to the health and safety of the public”)

• 82-255 = medium risk (“having a minimal risk to the health, safety and welfare of the public”)

• 256-625 = high risk (“having a high potential to harm in regards to the health, safety, and welfare of the general public”)

The CCMC’s new risk assessment responds to issues raised by the CHBA and others. It can identify areas of concern early, and help to focus attention on key factors. However, some of the terms and definitions seem a bit worrisome, as is the use of “occupant discomfort” as a risk rating factor. Also, some outcomes of the formula appear perverse. Taking an exaggerated example to make the point, it appears that a balcony guard made of corrugated cardboard would be rated 5 for likelihood of failure and 5 for severity of failure, but if it was very easy to detect could still end up with a rating well within the low-medium risk category.3 On the other hand, a sample risk assessment for a four-component system calculated risk by adding all the low to low-medium rating numbers for each individual component – 90, 72, 72 and 96 – together. This gave the system an overall score of 330, which is considered well into the high risk category. These concerns can be magnified if personnel implement the system too conservatively. At the least, experience and outcomes should be monitored. See Appendix A for more discussion of this subject.

3 CCMC says it would assess the balcony guard as two risk components (attachment to the structure and railing). Failure of either would have a high impact on the other, resulting in a 5 rating for impact on adjacent elements. (If so, the rolling risk total would add up to 125, well within the 82 to 255 “medium risk” level.) CCMC also says the attachment to structure of this hypothetical corrugated cardboard guard would be given a 3 rating (moderate) for ease of detection, as “it is moderately reasonable to expect that the average person observing would consider complete failure imminent if part of the guard were to detach from the structure”. However the railing itself would be given a 5 (defined as “no chance of detection before failure”), as “a tear in the cardboard for example, may not necessarily lead to the same conclusion”.

5

3.3 Shift to reduce inconsistency CCMC reports that there has been an ongoing shift towards more of a ‘team’ approach among evaluators. Since the introduction of the 2005 objective-based codes, new staff, new computer programs and the risk assessment process, evaluators dealing with the same areas of interest have been discussing issues together on a more frequent basis. As staff have realized that they have the support of their colleagues, they are reported to be getting more comfortable with this approach. This has not yet removed all complaints of inconsistent decisions by different staff members.

3.4 Clearing out the backlog CCMC reports that they are making headway on reducing the backlog of applications for evaluation. New staff coming on board have been tasked with investigating old files and either moving them forward or closing them out. A new computer system also flags dormant/inactive files and prompts for decisions on action. As of mid 2009, CCMC reports that its new document tracking system4 shows about 700 job files, including product listings (to existing standards), re-evaluations, and innovative product/system applications (alternatives to acceptable solutions listed in the National Codes). Approximately 100 of those are in the process of being closed. Reasons for closure can vary widely: some files go dormant because the person in charge at the applicant’s office has left the firm; some because applicants have given other projects higher priority; some because of mergers and acquisitions; some because applicants perceive the CCMC requirements as being impractical or too expensive, some because the applicant does not sign the contract or pay for the services. A number are shown as in ‘admin’, which includes some which are completed and in the process of being loaded to the website. Of the slightly more than 500 active files, approximately half are waiting for action from the applicant. That leaves about 250 files being worked on by the staff of 14.

3.5 Additional process concerns Some additional issues have arisen in the interviews for this paper. Concerns over Re-Evaluations. The CCMC website has an ‘Important Notes’ page, which says:

“Some of the Evaluation Reports and Listings appearing in this Registry show ‘Re-evaluation due’ dates that are past. CCMC is currently in the process of re-evaluating these products. In the interim, the evaluations remain valid for the conditions under which they were originally published.”

4 The computerized tracking system is still in the start-up phase and there may be some inconsistencies in the numbers.

6

Some manufacturers have found that potential users see the due dates, and the note on the listing that “Re-Evaluation is in Process”, and avoid the product. At the very least, the listing should repeat the statement that evaluations remain valid. The online material should be updated as soon as possible. In addition, products must be re-evaluated every three years whether or not the Technical Guides (TGs) setting out tests and other requirements have been updated. This can be useful for identifying and checking impact of any changes to the production process. But if production hasn’t changed, re-evaluation is perceived as a waste of money. Need for more outside review Initial scope and evaluation plans are generally sent only to the Standing Technical Committee for quick outside review. For complex issues, there may be a need to go beyond this group, to draw on more specific expertise and knowledge of implications and costs. Also, when the TGs are revised, those plans do not go to the committee or other outside group. Where significant changes are planned in a TG, outside input should be sought. Opportunities to facilitate standards development may be missed In some cases, work on a TG can affect more companies than just the applicant. CCMC occasionally works with small groups of manufacturers to develop TGs. That process can help to identify wider concerns and issues any later product standard would have to address. There is an opportunity to ensure work on the TG could also be used towards a later product or system standard – which should encourage more manufacturers to pay a share of the TG costs. Reviewing work on more applications in the light of format and requirements for future product standards development could be very helpful. Manufacturers and CCMC are both concerned about out-of-date Code contents – either acceptable solutions and/or referenced standards CCMC is directed to compare performance of alternative solutions against the ones deemed acceptable in Part B. Sometimes it isn’t even possible to obtain samples of products meeting those standards any more. Sometimes traditional construction methods identified in the codes have attributes or performance levels which go beyond performance requirement, or are irrelevant. A CCMC listing number does not necessarily mean that a product or system will comply with the Code requirements These listings just say the product conforms to an existing standard. Sometimes the Codes can require more than what is covered by the standard – sometimes only part of a standard could be referenced. This has spurred complaints that the format and contents are not useful either for the industry or for building inspectors. CCMC has been consider-ing adding another level attesting to conformance with both the standard and the code.

7

3.6 Business review A consulting firm has been retained to review CCMC’s business practices in light of changes over the past 20 years. Particular interests include “the advent of green products, the concept of sustainability and pressures to increase energy efficiency”. Phase 1 consists of an environmental scan, web survey of manufacturers and building officials and select one-on-one interviews. Stakeholder consultations make up Phase 2, scheduled for completion by the end of 2009. The final report is expected by the spring of 2010.

4. Experience with objective-based codes The 2005 national model codes took a brand new approach in a brand new format. A new Division A defines the Objectives and Functional Statements behind all the traditional prescriptive and performance requirements. The traditional contents were then renamed ‘acceptable solutions’, linked to appropriate objectives and functional statements, and placed in Division B. This was done for two main reasons:

• to facilitate acceptance of alternative acceptable solutions, by spelling out exactly what goal(s) and function(s) they would be expected to meet

• to define the codes’ core purpose (for the National Building Code, for example, the four Objectives deal with Safety, Health, Accessibility, and Fire and Structural Protection of Buildings)

As pointed out in the last paper, building departments’ concerns about possible liabilities and the closing of provincial evaluation groups has significantly reduced the traditional avenues for acceptance of alternatives. There are some anecdotal reports that this situation may have eased slightly. This is attributed to the new objective-based codes format, or to some reduction in anxiety about liability, or both. However, the number of building departments demanding CCMC reports or listings – or individual site-specific engineering reports – for any deviation from strict code compliance is still perceived to be high. As expected, introduction of objective-based codes does appear to have had some impact on the volume of CCMC applications. The number of applications to CCMC has varied from highs of 500+ per year in the early days to lows of 250 in 2001-4. In 2005, the number jumped to 314, possibly reflecting a desire by some manufacturers to get their evaluations for the new Code requirements completed early. In 2006, volume dropped back slightly to 261, then jumped again to 336 in 2007. The 2008 total was 284. CCMC notes that numbers alone do not tell the whole story; they do not show the complexity of the applications – which has increased appreciably. There is a general perception that the 2005 National Codes’ definition of objectives, sub-objectives and functional statements, combined with the new CCMC guidelines, has helped to keep the scope of new product evaluations focused on explicit code requirements. On the other hand, there are still reports of evaluation officers demanding tests apparently beyond that level.

8

With many building officials apparently refusing to accept anything other than CCMC reports for acceptance of alternative solutions, it is hard for manufacturers to fight unreasonable demands. Builders support justified caution – they do not want to use inadequate products – but CCMC staff must be very careful not to overstep or appear to overstep their role. It is impossible to quantify the number of potential new products or systems which do not apply – through whatever possible acceptance route – because the process is considered too difficult, lengthy or expensive.

5. Potential impact of new ‘green’ products and te chnologies Almost immediately after the 2005 National Codes were introduced, CCBFC received requests for changes to the National Building Code which did not fall within the scope of the four objectives identified in the new objective-based code. The Canadian Commission on Building and Fire Codes and the Provincial-Territorial Policy Advisory Committee on Codes have been looking into the issue, and CCBFC is expected to add objectives for both of these issues by the time the 2015 codes are published. The Ontario Building Code and the BC Building Code already include them. The push for new green technologies is definitely gathering strength. A key driver is climate change and the desire to address greenhouse gasses (GHGs). North America has lagged behind Europe in this, but appears ready to take new steps. Energy efficiency technologies for buildings are one example. Carbon pricing and caps on GHG emissions by large emitters are another. Introduced by the European Union in 2005, cap and trade has recently become one of the Obama administration’s key policy goals in the United States. British Columbia, Manitoba, Ontario and Quebec are members of a cross-border group called the Western Climate Alliance, which is drafting core model policies for a widely based regional ‘hard’ cap and trade scheme to come into effect in 2012. The federal government has plans for a less ambitious ‘intensity cap’ scheme. A number of Canada’s other trading partners are introducing or investigating carbon pricing as well. If these initiatives result in robust new programs, demand for new GHG-reducing technologies, especially fossil fuel energy conservation measures, can be expected to surge across the economy. Even if Cap and Trade schemes do not take off, the pressure for action on the environment means governments at all levels are likely to pursue ongoing measures to encourage or regulate green technologies, resource efficiency and reduced carbon emissions.

9

In this environment, manufacturers and importers are already working hard to come up with better ‘mousetraps’. As a result, pressure to get new products, equipment and systems into both new and existing buildings is expected to grow. A surge in demand would definitely create challenges for the CCMC. It is appropriate for the Centre to prevent unsafe new products coming onto the market based on a solid scientific rationale. It is not appropriate for it to prevent good and necessary products because of inadequate funding or staffing, excessive technical requirements, or undue risk aversion/exposure. It’s not uncommon to hear people say that the National Building Code doesn’t cover green technologies, so CCMC doesn’t have to worry about them – or not yet. But some of the most interesting new approaches are integrated into traditional building components, which the building codes do regulate. Photovoltaics, for example, may be a key technology for reducing energy consumption in buildings – and helping electrical utilities achieve new government requirements for a growing percentage of electricity to come from renewable sources. New products in Europe include photovoltaics as an integral part of roofing or cladding systems, or sandwiched in glass in place of traditional glazing in windows, skylights, etc. In France, the technical assessment body for building products apparently has assembled a new group of experts and produced core performance test requirements for these building-integrated-photovoltaics (BIPVs) in order to speed up evaluation of individual products. An assessment of this approach is beyond the scope of this report, but may be worth examining. How well could it work here? Should CCMC be considering similar approaches? There is also an issue of cross-jurisdictional product evaluations. Building, fire and plumbing codes are not the only regulatory acceptance obstacles faced by new technologies. Products and systems to collect and reuse grey water, for example, also need approval from public health and other departments. It may be very useful to have a small group that can coordinate and facilitate cross-jurisdictional new product evaluations. Is this a role for CCMC itself? If not, who could take it on? Should CCMC or another branch of IRC be promoting the concept and helping to bring together appropriate departments and agencies?

10

6. Responding to new challenges A full examination of options for CCMC and evaluation of alternative approaches is well beyond the scope of this paper. Instead, this section raises a few issues, possible approaches, and questions.

6.1 Bringing more building officials back into the process The first recommendation in CHBA’s 2005 paper was: “Fix the problem of liability”. Potential liability appears to be one of the key reasons building officials are either refusing to consider alternative solutions without CCMC ‘approval’, or demanding engineers’ stamps on a site-by-site, application-by-application basis, or both. This has dramatically reduced access to an important avenue for alternative solutions to become accepted. Bringing more building officials back into the process could do much to address unnecessary irritants and support innovation. It is definitely possible to address building departments’ risk of liability. It requires a well-understood process to:

• identify risk factors • identify tools to reduce the risks, and • use the best, most cost effective tool or combination of tools in a reasonable,

scientifically supportable way. Unfortunately, it is the builders and manufacturers who have an incentive to go through this kind of process. There is far less incentive for municipal or provincial/territorial departments with responsibility for codes to move from a no-risk situation to a low-managed-risk situation (unless a surge in demand for green technology suddenly casts them as ‘obstacles in the race to fix climate change’.) Who could champion this initiative? Who should prepare appropriate guidelines (see below)? What kind of regulatory support would they require, if any?

6.2 Providing guidelines/protocols for building officials Courts will not assign negligence liability to an inspector or building department that approved a project which properly used the ‘deemed-to-comply’ acceptable solutions in Part B of the Code, even if the product or system fails. The inspector/department did what a reasonable person with the expected level of expertise would have done in the same circumstances (took due diligence) – which is the test courts use for assessing negligence. However, if the inspector has accepted an alternative solution, the situation gets more tricky. How can they prove what a reasonable person would have done then? Apparently, there is not a lot of guidance on this – for the building departments or for the courts. Would it be helpful to produce notes, protocols or guidelines based on risk management principles? This would have to be done carefully, or it could worsen an already difficult

11

situation. But if done well such documents could become the standard for due diligence, reducing concerns about liability and improving acceptance of good alternative solutions. Should this be done through the Appendix of the National Codes or through a separate document?

6.3 Recognizing good, accepted risk-reduction tools in the protocols It’s important to remember that the National Model Codes don’t operate in a vacuum. As illustrated in the CCBFC report Canada’s Construction System: the context for model codes5, there are multiple layers of responsibilities and activities playing a part. These include manufacturers’ own information, in-plant quality control, third-party testing, product, design and/or management standards, certification systems, design and best practice guides, detailed drawings and specifications, site tests, first- and third-party warranties, warranty and insurance program conditions, training and education (as well as contract obligations, other legal requirements, and individual and broader systems to protect consumers, respond to problems, and provide recourse if things go wrong.) A protocol for acceptance of alternative solutions should recognize and support these systems as means of risk management. Checklists could include advice on scoring/ evidence for such issues as: Are there tests to show performance? Were they done by reputable laboratories (listings of labs in Canada and other countries)? How often are they updated? Is there a recognized standard (not already called up in the National Codes)? Is the product/system certified by an accredited certification body, or other reputable group? Is it covered by an Other Regulatory Document (ORD), recognized by the Standards Council of Canada? Is there a history of successful product use? Has a recognized professional approved the product/system application? Can that approval be used generically rather than site-by-site? Is there a Standata for it? Has it been accepted for use in another jurisdiction with similar construction methods and expectations for health and safety? Does the application meet a design guideline from a reputable body? Is there a quality management program in effect? Is there a training program for installers? How strong is the proponent? Is there a warranty or insurance program? The aim would be to balance the level of risk with the value of the kinds of risk mitigation measures listed above. Scoring should be designed so that low risk products where the proponent is a longstanding, reputable manufacturer would only have to show the company’s product information, test results from a reputable laboratory, and an affidavit saying that the product or system conforms with the building code and there is a plant quality assurance program in place. For medium-risk products or systems, more risk mitigation could be required. Trade-offs should be possible: perhaps a new manufacturer’s lack of track record could be balanced by generic engineering approvals and review of quality assurance, a new type of insurance, or similar.6

5 http://www.nationalcodes.ca/ccbfc/constructionsystem_e.pdf 6 Over the past couple of years, as provinces have adopted the new objective-based codes approach, chief building officials have begun approving a small number of alternative solutions. They should be collected as case studies, and used to identify process and analysis improvements.

12

Who should be the lead agency to prepare this kind of protocol? Would building departments recognize such a system? What regulatory support would be required, if any?

6.4 Allowing CCMC and others to prepare risk assessments as a stand-alone product Any risk management starts with an assessment. If some building departments are not comfortable with assessing risk – or the proponent believes they are too conservative – could CCMC or other acceptable experts do so instead? This should be offered to manufacturers on a voluntary basis. Under such an approach, new products or systems where the risk of problems is assessed as low would be deemed acceptable alternative solutions without CCMC-type third-party code compliance evaluation, provided they have an acceptable score under the protocols discussed above. CCMC or the approved experts would also need a guideline setting out the steps to assess risk levels (current process, with amendments, might be acceptable). For higher risk products or systems, conditions could be applied, such as limiting the number of installations, requiring reports of any problems to be sent to the risk assessment body or expert for a period of years, etc. Would this help create a more efficient system? Which bodies should be qualified to prepare the outside risk assessments? Should they have to be third-party? Would building departments recognize such a system? What regulatory support would be required, if any?

6.5 Accepting other test results An ongoing friction point in new product evaluations is the non-acceptance of tests performed to somewhat different standards than referenced in the codes, or performed in other countries by labs not listed by the Standards Council of Canada. There are a number of elements to this issue, including possible competitive advantage given to offshore manufacturers, and impact on Canadian testing labs if other countries’ results are accepted here without reciprocal acceptance of Canadian results in those other countries. However, it does appear that some accommodations could be made. For example, even strictly equivalent tests are only accepted if they were performed in the 1 – 1 ½ years preceding the application. Barring changes in test standards or product formulation, there seems little reason why tests up to three years old or even more should not be accepted. For tests using somewhat different standards, perhaps a simple risk assessment should be performed to examine what is the increased risk of problems because of the differences in the test and the increased severity? Where the increase in risk is assessed to be low, why can’t the test results simply be accepted? In marginal cases, could the test results be accepted but a condition be added that any problems must be reported to CCMC?

13

6.6 Anticipating new demand As noted above, France has apparently produced core performance test requirements for evaluation of building-integrated photovoltaics in the wake of government policy to significantly increase renewable energy use. Alberta’s Standata can perform a similar role. Should CCMC be anticipating demand for similar steps? If not, should another body? How would ‘core requirements’ be approached? Should it also be directed to design its work on certain TGs so as to support later standards development, where those are likely to be useful? Similarly, should the Centre encourage cross-jurisdictional approaches for products and systems which require approval from several different agencies?

6.7 Using outside expertise for full evaluations Demand for evaluations is expected to grow in areas where CCMC does not currently have on-staff expertise. It might be very helpful for CCMC to be able to designate certain laboratories, research agencies or specific experts to perform all of the work for new product or system evaluations, including creation of the Technical Guides and determination of compliance. This would also give the Centre flexibility to respond to changed volumes in more traditional areas. Would this increase the efficiency of the system? Are there any concerns about using outside experts this way? How can they be addressed? Who would the outside laboratories/agencies/experts be working for?

6.8 Reviewing manufacturing plant operations for higher risk products Unlike most certification agencies, CCMC does not do regular plant visits to review operations. Interestingly, an Australian article7 says that Japan recently surveyed the manufacturers of 13,965 products which had received government approvals to find out if actual material specifications still matched the initial test results. “Around 50 companies reported 130 products differed from the ministry approved specification,” the article says. And of a sample of 52 products tested to date, five did not meet the performance requirements. While there is no quantification of how much these products differed from the performance requirements, Japan is apparently introducing “continued monitoring, random checking and greater scrutiny by evaluation bodies”. Should CCMC be performing plant visits? Should they be limited only to high risk products? How frequently should they be performed?

7 International Code Collaboration with Japan, in Australian Building Regulation Bulletin, Autumn 2009, Australian Building Codes Board, Canberra.

14

6.9 Creating a volunteer-based decision-making (rather than advisory) body CHBA’s 2005 paper called for new policies to be “based on simple risk assessment and management procedures which allow CCMC to do what is ‘reasonable’ and not ‘everything they can think of’.” There are some concerns that because CCMC is part of the National Research Council it could be held to a very high level of accountability in any case that went to court. As mentioned earlier, in determining liability courts usually base their decisions on what a ‘reasonable person’ with equivalent expertise would have done in the same circumstances. Does that mean the courts would use the highest level of expertise inside the whole NRC to decide ‘reasonableness’? This potential liability appears to have choked up at least some of the decision-making process at CCMC. Apparently, the current volunteer-based Canadian Council on Construction Materials Evaluation (CCCME) is an advisory body, rather than a decision-making one. It only gives input to the CCMC’s decisions. Should there be a freestanding, volunteer-based, decision-making body, somewhat similar to the Canadian Commission on Building and Fire Codes (CCBFC) instead? Members of such a group could hear representations from proponents, opponents, and various research- and industry-based groups. CCMC would act as secretariat and give advice, but not be responsible for final decisions. Such a board should have more flexibility to make decisions, and its members should be protected from undue liability exposure under the federal government’s existing policies. Could such a board function more effectively than the current system? What are the pros and cons?

6.10 Accrediting additional code evaluation and certification bodies The Australian Building Code Board and New Zealand Department of Building and Housing have introduced a new system for evaluating whether innovative products and systems qualify as ‘acceptable alternatives’ under the two countries’ building codes8. In this system, the government-established Joint Accreditation System of Australia and New Zealand (JAS-ANZ) accredits qualifying certification bodies to evaluate innovative products and systems’ suitability under the codes, and to certify them with the CodeMark logo and certificate of conformity. The rules set out a system of risk analysis and requirements based on risk levels. High risk products and systems, for example, require frequent plant monitoring plus construction site surveillance. Legislation in Australian states and territories says that building approval agencies must accept products with this certification as alternative solutions. (Other avenues are

8 See http://www.abcb.gov.au/go/products/codemark/whatiscodemark for more information. Full rules for accreditation and certification are available here: http://www.abcb.gov.au/index.cfm?objectid=846439B7-C0EC-007A-DDF09E3A4DF6BE6B

15

available for acceptance of innovative products and services, but officials are not obliged to perform the analyses or accept them.) There are now two accredited CodeMark certification bodies in Australia. Interestingly, the Commonwealth Scientific and Industrial Research Organization, Australia’s equivalent of the National Research Council, has decided to discontinue its building product and systems appraisals program (somewhat similar to CCMC) by December 2009. Apparently, this is part of a larger-scale reorganization of the CSIRO. The Australia-New Zealand system appears to introduce competition into evaluations. Is this desirable? While there are reports of problems in Australia-New Zealand, are they due to the competitive approach or other issues? What are the criteria for evaluation agencies, such as expertise, staffing, insurance, oversight, etc.? How can competition in the evaluation sector improve access, timeliness, etc.?

7. Conclusion CCMC fills an important role in the Canadian construction system. It has taken good steps to improve processes and move to a risk-based approach. There are still some areas which need to be addressed for those products and systems going through the CCMC system. These include review of the risk analysis process, handling of re-evaluations, occasions when additional outside review may be advisable, opportunities to facilitate future standards, removing outdated requirements and standards from the national Codes, and addressing the information problem created when a CCMC product listing does not necessarily show code compliance. An ongoing issue arises where lower- and medium-risk innovative products and systems do not require the full CCMC review. The less formal acceptance systems they need have been disappearing, and nothing has yet arisen to replace them. This situation is expected to get worse as new code objectives create new demand for products and systems from Canada and elsewhere to be approved for code use. This paper raises several alternative approaches which may be useful within CCMC or as parallel processes. These include: providing guidelines and/or protocols for building officials on approving alternative solutions; recognizing good, existing and innovative risk reduction tools; allowing CCMC and others to prepare risk assessments as a stand-alone product; accepting other test results; anticipating new demand; using outside expertise for full evaluations; reviewing manufacturing plant operations for higher risk products; creating an outside volunteer-based, decision making (rather than advisory) body; and accrediting additional code evaluation and certification bodies. CHBA suggests that the business review of CCMC should include discussion of potential usefulness of these alternatives.

16

Appendix A: More Details of the CCMC Process for Ri sk Rating The building code’s concept of “limiting the probability (... that people ...) will be exposed to an unacceptable risk” acknowledges that there are acceptable levels of risk. CHBA’s 2005 paper recommended a simple risk assessment process be used to guide CCMC’s decisions and requirements. The suggested model contained five levels of “likelihood of problems occurring” and five levels of “probable hazard if they were to occur”. CHBA suggested that scope should be limited and extensive tests should not be required in lower risk situations. The sample risk assessment chart from that paper is shown below. CHBA’s Proposed Risk Assessment Form

A new procedure has been approved and applied recently at CCMC. It is filled out collectively by the technical team associated with that product type after the evaluation officer does the initial review of the client’s application. The risk assessment is revisited during the code analysis, and at other milestones. If the scope of the project changes, the risk analysis will also be reviewed. Ratings of 1 to 5 are applied on four criteria as shown below (it appears there is no zero rating):

CCMC’s new Risk Rating Chart

Column 1: Probability of Failure Score Extent Description

5 Frequent Likely to occur frequently 4 Probable Likely to occur several times in the service life of a product 3 Occasional Likely to occur at least once in the service life of a product 2 Remote Unlikely to occur but possible 1 Improbable Occurrence very unlikely

Very Low Low Moderate High Very High

Very Low

Low

Moderate

High

Very High

Hazard level if problems do occur

Likelihood of problem

s occurring

17

Column 2: Severity of Failure Score Extent Description

5 Catastrophic Product fails and results in sudden fatal injury (i.e. Structural) 4 Major Product fails and likely results health and safety risk 3 Critical Product fails and may result in health and safety risk 2 Significant Product fails with likely impact to occupant discomfort 1 Functional

Failure Product fails with unlikely impact to occupant discomfort

Column 3: Impact on Adjacent Elements Score Extent Description

5 Catastrophic Adjacent element fails and results in sudden fatal injury (i.e. Structural) 4 Major Adjacent element fails and likely results health and safety risk 3 Significant Adjacent element fails and may result in health and safety risk 2 Critical Adjacent element fails with likely impact to occupant discomfort 1 Functional

Failure Adjacent element fails with unlikely impact to occupant discomfort

Column 4: Detecting Failure Score Extent Description

1 Very Obvious Almost certain to be detected before failure 2 Easily detectable Highly likely to be detected before failure 3 Reasonably Detectable Moderate chance of detection 4 Difficult to Detect Rarely detected before failure 5 Unlikely to Detect No chance of detection before failure

Weightings are prepared by multiplying the scores; i.e. probability of failure x severity of failure x impact on adjacent elements x detecting failure. Totals are scored into one of three categories, as follows:

• 1-81 = low risk • 82-255 = medium risk • 256-625 = high risk

CCMC says (italics added) that “In general ...:

• Low risk products typically have no risk to the health and safety of the public. Low risk products that have built in redundancies or a high factor of safety typically do not require the experience of skilled or specialized trades to employ in the field.

• Medium-risk products will be defined as having a minimal risk to the health, safety, and welfare of the general public. These types of products may or may not require specialized trades or a high level of skill to employ in the field.

• High-risk products will be defined as having a high potential to harm in regards to the health, safety, and welfare of the general public. These types of products generally require specialized trades or a high level of skill to employ in the field. Clients whose products are considered and quantified as high risk will be directed to forego an evaluation or review alternatives to mitigate their risks.”

18

The CCMC’s risk assessment does respond to issues raised by the CHBA and others. However, there are some concerns about:

• definitions used There seems to be a wide range between category 1 (“no risk”), category 2 (“minimal risk”) and category 3 (“a high potential to harm”), for example. Also, the use of “critical” and “significant” as descriptors for the middle category of risk ratings – normally used for “average”

• use of “occupant discomfort” as a risk factor • the assumption that a product failure must cause adjacent elements to fail • outcomes from the formula

It seems appropriate that a product with ratings of 1 in probability of failure, 1 in severity of failure, 5 in impact of failure on adjacent elements and 5 in difficulty of detection would score 25, for example, or very low risk. That this same rating would be given to a product with ratings of 5 in probability of failure, 5 in severity of failure, 1 in impact on adjacent elements and 1 in difficulty of detection seems perverse. There is a question whether columns 3 and 4 should even be considered for products with low scores in columns 1 and 2. Scoring for composite systems may also create counter-intuitive results. See below.

Scoring for systems with several components In a sample risk assessment for a composite product recently provided by CCMC, the four components individually were scored at 90, 72, 72 and 96. (Using CCMC’s characterizations above, that would be minimal risk, no risk, no risk and minimal risk.) To calculate the rating for the product as a whole, those numbers were simply added together. That resulted in a final score of 330, or high risk. See below:

Sample CCMC Risk Assessment Table

Risk Assessment Results Item

(System) Risk Area

(Component) Probability of Failure

Severity of Failure

Impact on Adjacent Elements

Detecting Failure

Total Score1 (Risk)

Item 1 Component 1 2 3 3 5 90 Component 2 4 3 3 2 72

Component 3 2 3 3 4 72 Item 2 Component 4 2 4 3 4 96 Total 330

Adding risk factors together appears to produce a perverse result. The risk analysis process as a whole should be examined for shortcomings and modified as appropriate.