proficiency testing final report tracking test 16e31 june 2016 · 2016. 10. 26. · june 2016...

© IFM Quality Services Pty Ltd

This report is only for the use of participating laboratories

Proficiency Testing FINAL REPORT

Tracking test 16E31

June 2016

Proficiency Testing Provider Certificate Number 3189-02.

Program Coordinator: Ingrid Flemming IFM Quality Services Pty Ltd PO Box 877 Ingleburn NSW 2565 Australia Tel: +61 (0)2 9618 3311 Fax: +61 (0)2 9618 3355 Email: [email protected] Samples Dispatched: 26 April 2016 Results Due: 15 June 2016 Report Issued: 26 October 2016

We gratefully express our thanks to IECEE CTL WG2 for their support. In particular, thanks to Ted Gaertner and Dan O’Shea for their assistance in the planning, conduct and reporting of this program.

mailto:[email protected]



Operational Summary

NUMBER OF ROUNDS AND FREQUENCY OF TESTING

This is the first of 2 rounds to be operated in 2016.

APPLICABLE STANDARDS

• IEC 60112:2003 Edition 4.1 2009

• Product standards calling up IEC 60112.

• ISO/IEC 17025

SAMPLE MANUFACRURE AND ASSURANCE OF SAMPLE INTERGRITY

Samples were supplied by a subcontracted organization. Homogeneity was confirmed by a subcontracted organization. IFM also held data from previous observations. All required homogeneity criteria were met.

METHOD OF DISPATCH

All samples were sent via express national/international courier.

ASSESSMENT PROCEDURE

Please refer to the following pages for the assessment criteria.

DISPUTE PROCEDURE

Participants wishing to dispute their assessment, or appeal an evaluation on other grounds should submit

their appeal in writing to the program coordinator.

Please refer to http://www.ifmqs.com.au/ifm_terms_and_conditions.htm for the detailed policy.

CONFIDENTIALITY

The identity of participants and any identifying information supplied by participants is treated as confidential. Appropriate policies and procedures have been put into place to ensure this confidentiality is maintained.

Please refer to http://www.ifmqs.com.au/ifm_policy_on_confidentiality.htm for the detailed policy.

TEST INSTRUCTIONS

Instructions applicable to this testing round may be found at:

http://www.ifmqs.com.au/proficiency/instructions/16e31_Instructions.pdf

INDIVIDUAL RESULTS

Should individual test results be required, these will be provided upon request.

CORRECTIVE ACTIONS

IFM follows up on corrective actions of laboratories for a period of 3 months. Details for the closure

of actions can be at :

http://www.ifmqs.com.au/proficiency/references/QPF086-01_Corrective_Action_Management_for_Electrical_PT.pdf

http://www.ifmqs.com.au/ifm_terms_and_conditions.htm

http://www.ifmqs.com.au/ifm_policy_on_confidentiality.htm

http://www.ifmqs.com.au/proficiency/instructions/16e31_Instructions.pdf

http://www.ifmqs.com.au/proficiency/references/QPF086-01_Corrective_Action_Management_for_Electrical_PT.pdf



Participation Summary

⇒ 75 organisations enrolled

⇒ 73 organisations participated

⇒ 96 sets of results were submitted

⇒ 5519 individual test results were received

⇒ 2107 results were assessed

⇒ 3412 results were not assessed

⇒ 150 results were assessed as outliers

Test Assessment Criteria

• Failure of the sample (flaming or tracking)

Participants were required to report if the sample tracked or flamed. If the sample either tracked or flamed it was considered to have

failed the 50 drop test. Consensus of whether the sample passed or

failed was used to assess results. Tests were only assessed where greater than 70% consensus was met.

• Drops to failure (flaming or tracking) Where the sample failed, the 50 drop test robust statistics were

used to assess participants based on the number of drops reported

to obtain tracking or flaming, whichever occurred first. Participants

received Z scores based on the median and NIQR of submitted

results. Acceptable Z scores were |Z|<3.0.

For laboratories with equipment audit outliers, all test results were excluded from calculation of statistics, including median, inter-quartile range and consensus. All participants received assessments.

Variations to the assessment procedure will be detailed in the discussion.

• Depth of erosion When the sample did not fail the 50 drop test robust assessment was based on Z-scores as above. All results less than 1 mm were

considered acceptable when the statistically acceptable high was

less than 1 mm. Z-score are not shown when the statistically

acceptable high is less than 1 mm.

Available Tests

Samples Sample A Circuit board

Sample B Polycarbonate

Sample C Green plastic

Sample D Polystone



Discussion

EQUIPMENT AUDIT

Prior to commencing testing, participants were required to ensure their equipment was compliant with

IEC60112 by completing an equipment audit. A list of the responses received from this equipment audit

can be found at:

http://www.ifmqs.com.au/proficiency/reports/16e31_EquipmentAudit_Outcomes_20160429.pdf

For each Audit question being assessed, Table 1 summarises the number of laboratories submitting responses which were deemed to be incorrect, and subsequently required an explanation to be provided to

IFM. The question: Do you check the temperature of electrodes before starting each test? recorded the

greatest number of incorrect responses. IEC60112 specifies the electrodes must not be hot enough that

they have an adverse effect on the specimen. Touching the electrodes during cleaning was considered an

acceptable method of checking the temperature of the electrodes. The majority of laboratories required to provide an explanation reported that they were handling the electrodes during cleaning. Outliers for these

laboratories were closed without the need for corrective action as handling the electrodes is a suitable

method of checking the temperature.

The question: In normal use, how often is the distance separating the electrodes checked? recorded the next highest number of incorrect responses. It is a requirement of IEC60112 that electrodes are cleaned

before every sample. Some participants misinterpreted before every testing session and before every

specimen type to mean before every specimen. Laboratories that misunderstood this question, and

subsequently confirmed they clean the electrodes before every specimen, were not required to submit a corrective action.

The current status of equipment audit issues is attached as the first appendix to this report.

http://www.ifmqs.com.au/proficiency/reports/16e31_EquipmentAudit_Outcomes_20160429.pdf



ANALYSIS OF RESULTS

The range of results submitted for both samples A and C was larger than expected. It was found that there was more variation in results received from laboratories with equipment audit outliers. Laboratories with

equipment audit outliers were excluded from statistical calculations. Additionally, due to the large range of

results, the level of consensus required for assessment of results has been decreased from 75% to 70%. A

summary of the assessments can be seen in Table 2.

Analysis of results revealed the following:

• Participants reporting sample A failing the 50 drop test at 175 V also reported the sample failing at

higher voltages. Participants reporting sample C failing the 50 drop test at 425 V also reported the sample failing at 450 V. These consistent, incorrect results may indicate systematic testing errors

within the laboratories

• However, participants reporting sample A failing at any voltage were not found to be more likely to report sample C failing at 425 V or 450 V. Within affected laboratories, the inconsistency of the false

positive “fail” test results may indicate an inability to reliably reproduce test results.

• Laboratories receiving outliers may wish to examine there methods for possible systematic errors as

well as confirming the reproducibility of their testing.

Despite this, no factor other than an equipment non-conformance was found to affect laboratories’ results.

DEPTH OF EROSION

IEC60112 states that “erosion depths of less than 1 mm are reported as <1 mm”. The majority of

participants correctly followed these instructions. Some participants reported numeric results less than 1 mm (e.g. 0.4 mm). These participants did not report results in accordance with the standard.

• In this PT program, participants reporting actual measurement values of less than 1 mm will be assessed as if they had reported “<1 mm”.

• In future rounds, participants reporting measurements less than 1 mm will be required to explain why they did not report results as specified by the standard.



Question Number of incorrect

responses The electrodes are made from 1 The width of the left electrode is (mm) 1 The thickness of the left electrode is (mm) 0 The width of the right electrode is (mm) 1 The thickness of the right electrode is (mm) 0 The angle of the left electrode's chisel edge is (degrees): 0 The thickness of contacting surface of the left electrode (mm): 2 The angle of the right electrode's chisel edge is (degrees): 0 The thickness of contacting surface of the right electrode (mm): 2

Ensuring the entire width of chisel edges make contact with the surface of the specimen? 1

Do you check the temperature of electrodes before starting each test? 14 How often are electrodes cleaned? 7 How is the distance separating the electrodes normally checked? 4 In normal use, how often is the distance separating the electrodes checked? 12 How is the force exerted on the specimen by the electrodes determined? 3 The force is? N 2

How often is the resistivity of the dropping solution checked? 7

How long does it take for 50 drops to fall from the dropping device? (minutes) 0 What is the weight of 50 drops? (grams) 4 The resistivity of the dropping solution is (Ohm.m): 9 The vertical distance from the base of the dropping tube to the surface of the sample is (mm) 0 Is the test equipment located in an essentially draught free environment? 1 The typical rate of air movement around the sample is (cm per second) 7 Are fumes removed? 10 How are personnel protected from fumes that are generated during testing? 3 What frequency is the power supply? (Hz) 0 How many kVA is the power source? 0

What voltage drop is observed at the electrode tips between no load at 600V and when a current of 1.00 A flows through a load connected to the electrode tips? 9

What is the accuracy of the instrument that measures the applied voltage? (% of relevant scale) 5

At what current value does the over-current device trip when the current is increased at about 1 mA per second? (A in true r.m.s) 7

What is the accuracy of the instrument that measures the current? (% of relevant scale) 3

When a current equal to 0.50 A is suddenly initiated by a switch operation, how long does the current flow before activation of the over-current device (seconds). Perform 5 measurements and provide oscillograms.

7

On what type of material are the specimens supported?: 2 The thickness of the specimen support is (mm) 3 Is the specimen support level? (Please check both from the front and the side.) 1

TABLE 1: SUMMARY OF EQUIPMENT AUDIT



TABLE 2: SUMMARY OF ASSESSMENTS Sample A Sample B Sample C Sample D

175V Sample failure Correct Pass Pass Incorrect Fail Fail

175V Drops to failure Acceptable range NA NA

175V Depth of erosion Acceptable range <1 <1




225V Sample failure Correct Fail Pass Incorrect Pass Fail

225V Drops to failure Acceptable range 0-36 NA

225V Depth of erosion Acceptable range NA <1












Results

SAMPLE A 175V

0102030405060708090

<1 mm >1 mm NotAssessed

16e31 Tracking TestQ06 175V Depth of erosion

Sample A (mm)

Freq

uenc

y

Excluded

Included

01020304050607080

Pass(Correct)

Fail(Incorrect)

NotAssessed

16e31 Tracking TestQ06.1 175V Pass/FailSample A (Pass/Fail)

Freq

uenc

y

Excluded

Included



Results

SAMPLE A 200V

010203040506070

<1 mm >1mm NotAssessed


Sample A (mm)

Freq

uenc

y

Excluded

Included

0

10

20

30

40

50

60

Pass(Correct)

Fail(Incorrect)

NotAssessed


Freq

uenc

y

Excluded

Included



16e31 Tracking TestQ15.1 225V Drops to Fail

Sample A (Drops)Number of Results Received 78Number of Results Analysed 25Quartile 1 11.0000Quartile 3 20.0000IQR 9.0000NIQR 6.6717Median 16.0000Acceptable High:Median + (3*NIQR) 36.0151Acceptable Low:Median - (3*NIQR) -4.0151Statistically Acceptable Range 40.0302

Results

SAMPLE A 225V

0

5

10

15

20

25

30

3 7 12 16 21 25 30 34

Mor

e

Result (Drops)

Freq

uenc

y


Sample A

Excluded

Included

0102030405060708090

Pass(Incorrect)

Fail(Correct)

NotAssessed


Freq

uenc

y

Excluded

Included



Results

SAMPLE B 175V

0102030405060708090

100



Sample B (mm)

Freq

uenc

y

Excluded

Included

0102030405060708090

100

Pass(Correct)

Fail(Incorrect)

NotAssessed

16e31 Tracking TestQ06.1 175V Pass/FailSample B (Pass/Fail)

Freq

uenc

y

Excluded

Included



Results

SAMPLE B 200V

0102030405060708090

100

<1 mm >1mm NotAssessed


Sample B (mm)

Freq

uenc

y

Excluded

Included

0102030405060708090

100

Pass(Correct)

Fail(Incorrect)

NotAssessed


Freq

uenc

y

Excluded

Included



Results

SAMPLE B 225V

0102030405060708090

100



Sample B (mm)

Freq

uenc

y

Excluded

Included

0102030405060708090

Pass(Correct)

Fail(Incorrect)

NotAssessed


Freq

uenc

y

Excluded

Included



Results

SAMPLE C 425V

0102030405060708090

100



Sample C (mm)

Freq

uenc

y

Excluded

Included

0102030405060708090

Pass(Correct)

Fail(Incorrect)

NotAssessed

16e31 Tracking TestQ21.1 425V Pass/FailSample C (Pass/Fail)

Freq

uenc

y

Excluded

Included




Sample C (Drops)Number of Results Received 56Number of Results Analysed 23Quartile 1 20.5000Quartile 3 37.5000IQR 17.0000NIQR 12.6021Median 29.0000Acceptable High:Median + (3*NIQR) 66.8063Acceptable Low:Median - (3*NIQR) -8.8063Statistically Acceptable Range 75.6126

Results

SAMPLE C 450V

02468

10121416

4 12 21 29 38 46 55 63

Mor

e

Result (Drops)

Freq

uenc

y


Sample C

Excluded

Included

0

10

20

30

40

50

60

Pass(Incorrect)

Fail(Correct)

NotAssessed


Freq

uenc

y

Excluded

Included




Sample C (Drops)Number of Results Received 80Number of Results Analysed 27Quartile 1 4.0000Quartile 3 16.5000IQR 12.5000NIQR 9.2663Median 8.0000Acceptable High:Median + (3*NIQR) 35.7988Acceptable Low:Median - (3*NIQR) -19.7988Statistically Acceptable Range 55.5975

Results

SAMPLE C 475V

05

10152025303540

2 8 14 21 27 33

Mor

e

Result (Drops)

Freq

uenc

y


Sample C

Excluded

Included

0102030405060708090

Pass(Incorrect)

Fail(Correct)

NotAssessed


Freq

uenc

y

Excluded

Included



Results

SAMPLE D 425V

0102030405060708090

100



Sample D (mm)

Freq

uenc

y

Excluded

Included

0102030405060708090

100

Pass(Correct)

Fail(Incorrect)

NotAssessed

16e31 Tracking TestQ21.1 425V Pass/FailSample D (Pass/Fail)

Freq

uenc

y

Excluded

Included



Results

SAMPLE D 450V

0102030405060708090

100



Sample D (mm)

Freq

uenc

y

Excluded

Included

0102030405060708090

100

Pass(Correct)

Fail(Incorrect)

NotAssessed


Freq

uenc

y

Excluded

Included



Results

SAMPLE D 475V

0102030405060708090

100

Pass Fail NotAssessed


Sample D (mm)

Freq

uenc

y

Excluded

Included

0102030405060708090

100

Pass(Correct)

Fail(Incorrect)

NotAssessed


Freq

uenc

y

Excluded

Included



�End of Report�

Lab Code Reason Withdrawn / Comment Due DateAction Alert

Tracking Test - 16e31 (201604)

Electrical Program Status Report

Y Y1 CBTL - Certification Body Testing Laboratory

O 26/10/2016 Part01: 2 results assessed as outlying




O 26/10/2016 Part02: 1 result assessed as outlying




O 26/10/2016 Part01: 1 result assessed as outlyingO 26/10/2016 Part02: 1 result assessed as outlying

















14 CBTL - Certification Body Testing Laboratory








O 26/10/2016 Part01: 2 results assessed as outlyingO 26/10/2016 Part02: 2 results assessed as outlying









26/10/2016 Page 1 of 4Produced by IFM Quality Services Pty Ltd

WWW.IFMQS.COM.AU - Ph: +61 2 9618 3311

http://www.IFMQS.COM.AU





Y Y24 Private















Y Y34 CTF - Customer’s Testing Facility



















































Y Y58 Private



Y Y60 Private







64 Private









Y Y69 Private












73 Private

74 Private


76 Private

>> Report Indicator Columns and Action History Codes

- "Y" when a submitted result was determined to be an outlier requiring corrective action(s) or additional information is required to clarify a result. Action history will then follow.- "Y" when corrective or other action(s) have not been completed- Date by which corrective or other action(s) are required to be completed. This date is not set until the FINAL report has been issued.

X - Reminder sent to laboratoryC - Action Closed (Completed)F - Overdue action referred to IEC-EE

Action History Codes:

O - Action Opened by IFMN - Notification of action sent to laboratoryA - Acknowledgement received from laboratoryR - Response received from laboratory

Column:

Action

AlertDue Date





Tracking Test (Audit) - 16e31 Audit (201603)



Y2 CBTL - Certification Body Testing Laboratory

O 28/04/2016 Equipment Audit Outlier - Action RequiredC 4/05/2016 clarification received and OK. However, IFM has requested lab has corrective action report on

hand to explain submission of typographical errors.



N 28/04/2016 Equipment Audit Questions to be answered.R 3/05/2016 clarifications provided.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 14/06/2016 Missing values added. Staff trained to clean electrode between each use.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 14/06/2016 Ventilation system turned on when test is over.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 21/06/2016 Large door allows fumes to be vented quickly. Active ventilation has also been added.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 9/06/2016 Procedures updated.


N 28/04/2016 Equipment Audit - Action RequiredR 9/05/2016 suitable clarifications provided


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 31/05/2016 Electrodes are handled between tests, therefore they cannot be too hot.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 9/06/2016 Misunderstanding of question, typo and electrodes allowed to cool between tests.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 9/06/2016 Resistance of dripping solution checked regularly. Outlier due to typo. Equipment adjusted.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 14/06/2016 Electrodes allowed to cool between tests. Fumes actively removed from the room, instead of

the enclosure. Voltage drop question misunderstood.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 19/05/2016 Test engineer entered wrong value. Equipment was compliant.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 30/05/2016 Outliers the result of misunderstanding of questions.


N 28/04/2016 Equipment Audit - Action RequiredR 4/05/2016 Suitable clarification of audit response received.



O 28/04/2016 Equipment Audit Outlier - Action RequiredC 9/05/2016 Ventilation fan is in enclosure.








O 28/04/2016 Equipment Audit Outlier - Action RequiredC 14/06/2016 Wrong solution measured. Electrodes cleaned and therefore at a low temperature before the

next test.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 6/06/2016 Equipment readjusted.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 16/06/2016 Electrode separation now checked before every test. Resistivity of solution measured in wrong

units. Adjustments made to over-current advice.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 8/06/2016 Electrode separation is checked regularly.


Y24 Private

O 28/04/2016 Equipment Audit Outlier - Action RequiredC 18/05/2016 Solution used based on manufacturers expiry date.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 11/05/2016 Outlier due to misunderstanding of the question.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 9/05/2016 Airflow is less than 20 cm/s.

Voltage drop was misunderstood.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 18/05/2016 Correct responses revised after revised version sent.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 6/05/2016 Outliers corrected.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 9/05/2016 Conductivity now tested every day.





O 28/04/2016 Equipment Audit Outlier - Action RequiredC 31/05/2016 Misunderstanding of questions and equipment recalibrated.

Y34 CTF - Customer’s Testing Facility

O 28/04/2016 Equipment Audit Outlier - Action RequiredC 16/06/2016 Equipment not properly adjusted. Equipment to be revived to ensure proper adjustment.

35 CTF - Customer’s Testing Facility


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 6/05/2016 Outliers due to misunderstanding.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 5/05/2016 Support changed to glass.







38 CTF - Customer’s Testing Facility


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 1/06/2016 Checking chisel edge makes contact with surface is now part of their procedure.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 9/05/2016 Policy updated and staff trained.




N 28/04/2016 Equipment Audit - Action RequiredR 9/05/2016 suitable clarification provided


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 14/06/2016 SOP updated so that electrodes are cleaned between every test.


N 28/04/2016 Equipment Audit Outlier - Action RequiredR 3/05/2016 suitable clarifications received.


N 28/04/2016 Equipment Audit - Action RequiredR 5/05/2016 Suitable clarifications received.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 9/05/2016 Outliers due to misunderstanding of questions.


N 28/04/2016 Equipment Audit QuestionsR 3/05/2016 Suitable clarifications received.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 1/06/2016 Question misunderstood.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 11/05/2016 Outliers due to typos.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 13/05/2016 Revised answer correct.



O 28/04/2016 Equipment Audit Outlier - Action RequiredC 11/05/2016 Outliers due to misunderstanding of the question.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 9/05/2016 Change to procedure and mistranslation.



O 28/04/2016 Equipment Audit Outlier - Action RequiredC 14/06/2016 Questions misread.


58 Private








Y60 Private

O 28/04/2016 Equipment Audit Outlier - Action RequiredC 18/05/2016 Question misunderstood.



O 28/04/2016 Equipment Audit Outlier - Action RequiredC 9/05/2016 Dropping solution has a set shelf life.


64 Private




O 28/04/2016 Equipment Audit Outlier - Action RequiredS 7/07/2016 No response received.F 7/07/2016 Referred to IEC and Reporting NCB(s) as applicable


O 28/04/2016 Equipment Audit Outlier - Action RequiredR 4/05/2016 Email received with corrected answers. Requested additional explanations as to why different

results were initially submitted.F 7/07/2016 Referred to IEC and Reporting NCB(s) as applicable

Y69 Private

O 28/04/2016 Equipment Audit Outlier - Action RequiredC 5/05/2016 Incorrect data entry.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 5/05/2016 Outliers corrected.


O 28/04/2016 Equipment Audit Outlier - Action RequiredC 31/05/2016 Eqipment and procedures updated.

73 Private

Y74 Private

O 28/04/2016 Equipment Audit Outlier - Action RequiredC 4/05/2016 corrected sheet received. Not in CB scheme, therefore ot required to submit CAR.


76 Private







>> Report Indicator Columns and Action History Codes

- "Y" when a submitted result was determined to be an outlier requiring corrective action(s) or additional information is required to clarify a result. Action history will then follow.- "Y" when corrective or other action(s) have not been completed- Date by which corrective or other action(s) are required to be completed. This date is not set until the FINAL report has been issued.

X - Reminder sent to laboratoryC - Action Closed (Completed)F - Overdue action referred to IEC-EE

Action History Codes:

O - Action Opened by IFMN - Notification of action sent to laboratoryA - Acknowledgement received from laboratoryR - Response received from laboratory

Column:

Action

AlertDue Date




proficiency testing final report tracking test 16e31 june 2016 · 2016. 10. 26. · june 2016...

Documents