i

Report of the Subcommittee on Examination Quality

Management

(FY2016)

April, 2017

Subcommittee on Examination Quality Management Intellectual Property Committee

Industrial Structure Council

i

Introduction The subcommittee was established under the Intellectual Property Committee of the Industrial Structure Council in August 2014 in order to make recommendations for improvements of quality management concerning examinations of patents, designs, and trademarks in the JPO through verifications and evaluations of the implementation system/ the implementation status of quality management; for example, the Subcommittee verifies and evaluates whether or not policies and procedures of quality management including quality manual policies or other necessary manuals have been properly formulated, whether or not the quality management system has been properly established, and whether or not quality management has been properly implemented in compliance with the formulated policies and procedures. With the globalization of business and R&D activities by Japanese companies, it is required that examination results produced by the JPO be highly evaluated abroad as well, and contributed to smoothly obtaining IP rights. It is also required to improve predictability of businesses utilizing the industrial property rights system so as to prevent unnecessary disputes. In order to satisfy these needs, it is crucial to maintain and improve the level of quality of examinations, which industrial property rights are based on. In responding to such circumstances, the JPO formulated and announced its "Quality Policy" for "robust, broad, and valuable establishment of rights" in FY2014. Based on this, the JPO has constructed the quality management system across the examination departments so that patent, design and trademark examinations may be conducted in compliance with the Quality Policy. In order for the quality management system to work effectively for maintaining and improving the quality of examinations, it is important to effectively operate a PDCA cycle, which is a quality management method that the JPO has adopted as its internal initiative for examination quality improvement, and thereby to continuously improve the quality of examinations. Aiming to realize the world’s leading quality management by reflecting objective evaluations and improvement recommendations from external experts on such internal efforts of the JPO, the Subcommittee verified and evaluated the implementation system/ the implementation status of the quality management conducted by the JPO in FY2016, based on the quality management report provided by the JPO, according to the evaluation items and criteria established for examination quality management in FY2014, and then considered on what needed to be improved.

ii

Subcommittee on Examination Quality Management Intellectual Property Committee under Industrial Structure Council

Past Meetings

1st Subcommittee Meeting December 16, 2016 Agenda:

1. Implementation status of initiatives for quality management 2. Intermediate evaluations based on evaluation items and criteria 3. Improvement recommendations concerning the implementation system and the implementation status based on intermediate evaluations

2nd Subcommittee Meeting March 27, 2017 Agenda:

1. Implementation status of initiatives for quality management 2. Evaluations based on evaluation items and criteria 3. Improvement recommendations concerning the implementation system and the implementation status based on intermediate evaluations

4. FY 2016 Report of the Subcommittee on the Examination Quality Management (Draft)

iii

Subcommittee on Examination Quality Management

under the Intellectual Property Committee of the Industrial Structure Council

List of Members

Chairperson Hidetaka Aizawa Professor, Hitotsubashi ICS, Hitotsubashi University

Setsuko Asami Professor, Graduate School of Innovation Studies, Tokyo University of Science

Toshiaki Iimura Attorney, YUASA and HARA Wataru Inoue Economic News Editor, The Nikkan Kogyo

Shimbun, Ltd. Sumiko Ohara Patent Attorney, ITOH International Patent

Office Harumi Kojo Attorney, Sakurazaka Law Offices Kenji Kondo President, Japan Intellectual Property

Association Kenichi Nagasawa President, International Association for the

Protection of Intellectual Property Takeshi Nakajo Advisor, The Japanese Society for Quality

Control / Professor, Chuo University Keiko Honda Patent Attorney, Honda International Patent

and Trademark Office Tomoko Watanabe Patent Attorney, Watanabe Tomoko

International Patent Office

(Titles omitted; listed in Japanese syllabary order)

iv

Table of Contents

I. Implementation Status of Initiatives for Examination Quality Management .................... 1 1. Patents ............................................................................................................................. 3 2. Designs ............................................................................................................................. 9 3. Trademarks .................................................................................................................... 15

II. Evaluation Results Concerning Implementation Systems/Implementation Status in Quality Management ............................................................................................................. 23

1. Patents ........................................................................................................................... 25 2. Designs ........................................................................................................................... 29 3. Trademarks .................................................................................................................... 33

III. Recommendations for Improvement in Implementation System/Implementation Status of Quality Management ......................................................................................................... 38

1. Patents ........................................................................................................................... 38 2. Designs ........................................................................................................................... 39 3. Trademarks .................................................................................................................... 40

IV. Conclusion .................................................................................................................. 42

(Reference 1) Evaluation Items and Criteria Concerning Examination Quality Management

1

I. Implementation Status of Initiatives for Examination Quality Management The Japan Patent Office (JPO) has constructed the quality management system shown below and has managed quality control.

Overview of the Quality Management System of the JPO

Commissioner and Deputy Commissioner are in charge of the maintenance and implementation of the quality management system (Director General of the Trademark and Customer Relations Department is responsible for trademark matters, instead of Deputy Commissioner). Quality management is carried out by the Examination Divisions that conduct substantive examination, the Policy Planning and Coordination Department that plans policies and makes proposals for quality management initiatives, and the Quality Management Office that assesses and analyzes quality all working collaboratively while maintaining separation of their own duties. The Subcommittee on Examination Quality Management under the Intellectual Property Committee of the Industrial Structure Council (hereinafter, the "Subcommittee") was established under the Intellectual Property Committee of the Industrial Structure Council in order to make recommendations for improvements of quality management in the JPO through verifications and evaluations of the implementation system and implementation status of quality management.

Deputy Commissioner/ Director-General

2

As shown below, the JPO makes self-regulatory efforts for continuous improvement of examination quality by following a PDCA cycle in performing its examination quality management. At the same time, the Subcommittee makes evaluations and recommendations for improvement in the implementation system and implementation status of quality management in the JPO. Such evaluations and recommendations are reflected in the internal PDCA cycle of the JPO, which will contribute to further improvement in examination quality.

Relationship between the JPO's Internal Quality Management and the

Subcommittee on Examination Quality Management The quality management system of the JPO has been documented into the Quality Management Manuals for Patent Examination, Design Examination, and Trademark Examination (Quality Manuals), and published on the website of the JPO. Under such quality management system, the JPO implemented the following major initiatives in FY2016 in order to properly assess and maintain/improve examination quality. The major initiatives and the results are as follows.

3

1. Patents (1) Enhancing Initiatives for Improvement in Examination Quality and for Quality Verification - In order to enhance initiatives for quality verification, it is important to listen carefully to users' voices. Thus, while the content of the user satisfaction survey and the exchanges of opinions with users was enhanced, opinions regarding examination quality collected through the points of contact (such as an opinion submission form for improving the quality of examinations) were utilized to reflect users' voices to the quality management initiatives. As major initiatives, the following initiatives were implemented to execute examinations in accordance with the fundamental principle of "We meet wide-ranging needs and expectations" specified in the Quality Policy.

The user satisfaction survey of FY2016 was conducted using the questionnaires to which necessary modifications were made to reduce the load on respondents while maintaining continuity with the surveys conducted for the last four years.

The Examination Divisions exchanged opinions with industry organizations and other entities to grasp industry trends and users' needs.

As a result, in the user satisfaction survey of FY2016, the response rates improved compared to the previous fiscal year for all the four types of questionnaires1 that were used. In the user satisfaction survey of FY2016, the level of satisfaction with the overall quality of examinations of national applications and PCT applications improved, with an increased ratio of responses giving an evaluation of 4 or 5 in a 5-point scale compared to the previous fiscal year. - Prior art search is one of the essential processes in patent examination, and thus improvement in search capabilities is required. Thus, efforts were made to enhance the usability of searches on foreign documents or other literature, and to accumulate and share know-how on the searches. In addition, by taking into consideration the issues identified through the system of opposition to grant of patent, which started in FY2015, prior art searches were further enhanced.

1 The four types of questionnaires are as follows: a questionnaire on the overall quality of patent examinations on national applications, a questionnaire on the quality of patent examinations on specific national applications, a questionnaire on the overall quality of the international search and international preliminary examination on PCT applications, and a questionnaire on the quality of the international search and international preliminary examination on specific PCT applications.

4

As major initiatives, the following initiatives were implemented to execute examinations in accordance with the fundamental principle of "We grant robust, broad and valuable patents" specified in the Quality Policy.

Although search ranges and other factors are usually rationalized by consultations among examiners or other staff, the number of consultations focusing on searches, which was held once within a half-year term for each examiner in the previous fiscal year, was increased to three times within a half-year term.

The results of examination on the filed oppositions to grant of patent or other issues were fed back to the Examination Divisions, for which factor analysis was conducted by the examiners in charge or other staff, and information was shared among the Examination Divisions as necessary.

As a result, in the user satisfaction survey of FY2016, the level of satisfaction with the searches on foreign patent documents for national applications improved, with an increased ratio of responses giving an evaluation of 4 or 5 in a 5-point scale compared to the previous fiscal year. Also, regarding the level of satisfaction with the searches on foreign patent documents for PCT applications, the ratio of responses giving an evaluation of 4 or 5 in a 5-point scale was almost the same as that of the previous fiscal year; however, the ratio of responses giving an evaluation of 2 or 1 in a 5-point scale decreased compared to the previous fiscal year. - In order to enhance examination quality, it is necessary for each examiner to steadily perform examination according to the Examination Guidelines for Patent and Utility Model revised in FY2015. Thus, consultations among examiners were enhanced by promoting the exchange of opinions and the sharing of knowledge in making judgment on patentability requirements and descriptive requirements in the revised Examination Guidelines. As major initiatives, the following initiative was implemented to execute examinations in accordance with the fundamental principles of "We grant robust, broad and valuable patents" and "We all dedicate ourselves to improving quality, cooperating with concerned persons and parties" specified in the Quality Policy.

In order to steadily conduct examinations in accordance with the revised Examination Guidelines or other standards, consultations among examiners were enhanced. Specifically, consultations were enhanced for the cases requiring special attention in application of the revised Examination Guidelines or other standards,

5

such as a judgment on the cases in which a claim concerning an invention of a product recites a manufacturing process of the product.

As a result, in the user satisfaction survey of FY2016, the level of satisfaction with the judgment regarding "descriptive requirements for the description and the claims" including the judgment regarding product-by-process claims increased compared to the previous fiscal year, with an increased ratio of evaluations of 4 and 5 in a 5-point scale. - In order to more accurately understand the current level of examination quality, it is also important to enhance quality audit. Thus, quality audit was enhanced by, for example, improving the infrastructure for allowing the Quality Management Office to conduct appropriate quality audit. As major initiatives, the following initiatives were implemented to execute examinations in accordance with the fundamental principle of "We continually improve operations" specified in the Quality Policy.

Improvement was made to the method of providing the results of quality audit to the managerial staff in the Examination Divisions.

Currently (as of March 2017), domestic applications are selected with high flexibility though the number of times of selecting has an upper limit. Regarding PCT applications, however, selection with high flexibility has not been achieved due to the constraints on the system. For this matter, preparation was made to provide a new system from FY2017 through which applications can be selected with high flexibility without limitation on the number of times of selecting.

As a result, audit result information that is updated daily has become available to the managerial staff in the examination departments, thus enabling speedier action in the course of the initiatives on quality implemented in the examination divisions. In FY2017, selection of applications with high flexibility without limitation on the number of times of selecting is expected to be achieved. - In order to conduct appropriate quality management, it is important to run a PDCA cycle effectively. Thus, it is necessary to perform continual improvement, while evaluating how much objectives of quality management initiatives have been achieved through following the PDCA cycle.

6

As major initiative, the following initiatives was implemented to execute examinations in accordance with the fundamental principle of "We continually improve operations" specified in the Quality Policy.

For the initiatives for quality management specified in the Initiatives for Examination Quality Management (FY2016), the relation between these initiatives and their results was evaluated.

Search for foreign patent documents had been identified as an issue through the user satisfaction survey of the previous fiscal year or other studies. As a result of implementing corresponding initiatives, however, in the user satisfaction survey of FY2016, the level of satisfaction with the searches for foreign patent documents on national applications improved, with an increased ratio of responses giving an evaluation of 4 or 5 in a 5-point scale compared to the previous fiscal year. Also, regarding the level of satisfaction with the searches for foreign patent documents on PCT applications, the ratio of responses giving an evaluation of 4 or 5 in a 5-point scale was almost the same as that of the previous fiscal year; however, the ratio of evaluations of 2 and 1 decreased compared to the previous fiscal year. Through the comparison between the initiatives for quality management specified in the Initiatives for Examination Quality Management (FY2016) and the results, it was confirmed that the PDCA cycle has been effectively functioning and the quality of patent examinations has been improved continuously, with the enhancement of "searches for foreign patent documents" and other initiatives. (2) Establishing Policies, Procedures, and Structures aiming for High Quality Examination - In order to improve examination quality, it is crucial to enhance the organizational structure for examinations. Thus, continuous efforts need to be made to secure the necessary number of examiners in order to ensure that the JPO's examination implementation system can be compared favorably with that of major intellectual property (IP) offices, taking into account the efficiency of examination. As major initiatives, the following initiative was implemented to execute examinations in accordance with the fundamental principle of "We grant robust, broad and valuable patents" specified in the Quality Policy.

The examination implementation system for efficient and appropriate patent examinations has been ensured.

7

As a result, the number of permanent examiners and fixed-term examiners was increased, and 39 permanent examiners and 104 fixed-term examiners were newly employed. - In order to improve examination quality, it is also crucial to enhance the quality management system. Thus, efforts need to be made to further enhance the quality management system, including the improvement of the infrastructure for quality audit, while making continuous efforts to secure the necessary human resources. As major initiatives, the following initiatives were implemented to execute examinations in accordance with the fundamental principle of "We continually improve operations" specified in the Quality Policy.

Quality audit was conducted with the use of applications that had been selected in an appropriate and timely manner.

An efficient application selection system with high flexibility is expected to be launched by FY2017.

As a result, quality audit on about 400 to 600 applications for each of 6 types of drafts (about 3,200 cases2 in total) are expected to be conducted by 94 Quality Management Officers in FY2016. - In order to manage the quality of examination, it is crucial that each of the personnel who engages in examination has a good understanding of quality management. Thus, training and seminars on quality management were provided to examiners and other staff to promote their understanding of quality management. As major initiative, the following initiative was implemented to execute examinations in accordance with the fundamental principles of "We grant robust, broad and valuable patents," "We raise the knowledge and capabilities of our staff," and "We all dedicate ourselves to improving quality, cooperating with concerned persons and parties" specified in the Quality Policy.

In order to promote examiners' understanding of quality management, intensive discussions were conducted in each examination division for a fixed period of time, and lectures were conducted on quality management in patent examinations through

2 The six types of drafts are as follows: the first notice of reasons for refusal, a decision to grant a patent after the first office action, a decision to grant a patent after the second or later office action, a PCT international search report and a written opinion of an international searching authority, a final notice of reasons for refusal, and a decision of refusal.

8

training exercises (first-period training of examiner course, later-period training of examiner course, etc.) for training fresh examiners.

As a result, in the user satisfaction survey of FY2016, the level of satisfaction with "consistency of judgments among examiners," "communication with examiners (such as face-to-face interviews and telephone conversations)," and other items improved compared to the previous fiscal year, with an increased ratio of evaluations of 4 and 5 in a 5-point scale. (3) Communication of information on initiatives for examination quality improvement - In order to support global businesses of companies, it is necessary to build an environment where examination results produced by the JPO are highly appreciated among the international IP community, which allows users to obtain patent rights smoothly in foreign countries. Thus, the JPO needs to actively communicate its initiatives for examination quality improvement to overseas IP offices including those in emerging countries through international meetings or other occasions, as well as to build cooperative relations with overseas IP offices, and then to collect information on their initiatives for examination quality. As major initiatives, the following initiatives were implemented to execute examinations in accordance with the fundamental principle of "We contribute to improving the quality of patent examination globally" specified in the Quality Policy.

Through international meetings or other occasions, efforts were made to increase trust in examination at the JPO, and to spread high-quality examination to overseas IP offices including those in emerging countries.

Examiners of the JPO were dispatched to the Europe Patent Office (EPO) and the United States Patent and Trademark Office (USPTO) for mid- to long-period to communicate quality management methods of the JPO such as quality audit as well as to collect information on quality management methods and the latest trends in the IP offices to which the examiners were dispatched.

Training was conducted to the personnel of IP offices in India and the ASEAN countries by the examiners of the JPO. The JPO dispatched its examiners as trainers for new-examiner training in India and Thailand, for example.

9

As a result, the five IP offices agreed to the IP5 Joint Statement in Tokyo 2016 (Tokyo Statement) stipulating efforts including "providing high-quality and reliable examination results." The five IP offices shared their understanding of their quality management systems one another. Further, the JPO shared the understanding, with the EPO and the USPTO, on their quality audit methods and other issues. Furthermore, efforts were made to contribute to improvement in practical examination capabilities of IP offices in the countries into which Japanese companies have been expanding their business, and to spread the examination practices of the JPO to such IP offices. 2. Designs (1) Enhancing Initiatives for Improvement in Examination Quality and for Quality Verification - In order to enhance initiatives for quality verification, it is important to listen carefully to users' voices. Thus, while the content of the user satisfaction survey and the exchanges of opinions with users was enhanced, opinions collected through a contact for receiving opinions regarding the quality of examinations (such as an opinion submission form for improving the quality of examinations) were utilized to reflect users' voices to the quality management initiatives. As major initiatives, the following initiatives were implemented to execute examinations in accordance with the fundamental principle of "We meet wide-ranging needs and expectations" specified in the Quality Policy.

The user satisfaction survey of FY2016 was conducted using questionnaires to which necessary modifications were made to reduce the load on respondents while maintaining continuity with the surveys conducted for the previous fiscal year.

The examination divisions exchanged opinions with industry organizations and other entities to grasp industry trends and users' needs.

As a result, in the user satisfaction survey of FY2016, response rates improved from the previous fiscal year for all the two types of questionnaires3 that were used. The level of satisfaction with the overall quality of examinations improved from the previous fiscal year, with an increase in the number of evaluations of 4 and 5 on a scale of 1 to 5.

3 The two types are as follows: a questionnaire on the quality of overall design examinations and a questionnaire on the quality of design examinations on specific applications.

10

- Prior design search is one of the essential processes in design examination, and thus improvement in search capabilities is required. Thus, efforts were made to enhance the usability of searches on designs including graphic images for which the Examination Guidelines for Design was revised, and to accumulate and share know-how on prior design searches, and prior design searches were further enhanced. As major initiatives, the following initiative was implemented to execute examinations in accordance with the fundamental principle of "We grant robust, broad and valuable design rights" specified in the Quality Policy.

In the examination on design applications, search know-how on prior design search were accumulated and shared among examiners, and operation was started in which a scope of the search is written in a notification and sent with a notice of decision of registration.

As a result, in the user satisfaction survey of FY2016, the level of satisfaction with prior design searches improved from the previous fiscal year, with an increase in the number of evaluations of 4 and 5 on a scale of 1 to 5. - In order to enhance examination quality, it is necessary for each examiner to steadily perform examination according to the "Examination Guidelines for Design" revised in FY2015. Thus, consultations among examiners were enhanced by promoting the exchange of opinions and the sharing of knowledge in examining international applications for design registration. As major initiatives, the following initiatives were implemented to execute examinations in accordance with the fundamental principles of "We grant robust, broad and valuable design rights" and "We all dedicate ourselves to improving quality, cooperating with concerned persons and parties" specified in the Quality Policy.

In order to steadily conduct examinations in accordance with the revised examination guidelines or other codes, consultations between examiners and their approvers were enhanced for all international applications for design registration.

The Working Group on the Design Examination Standards for this fiscal year was held. Through discussions on the clarification of operation regarding request for application of the provisions on exception to lack of novelty, on the descriptive requirements of applications and drawings and the handling of reference views, new revisions in the Examination Guidelines for Design were drafted.

The newly developed examination system for international applications for design registration has enabled paperless and more stable operation of examination.

11

As a result, it was confirmed that examiners share the validity of determinations on international applications for design registration, and other factors. - In order to more accurately understand the current level of examination quality, it is also important to enhance quality audit. Thus, quality audit was enhanced by, for example, improving the infrastructure for allowing the Quality Management Division to conduct appropriate quality audit. Furthermore, aspects of quality audit and other points were considered in light of what was discussed through consultations in examining international applications for design registration. As major initiatives, the following initiatives were implemented to execute examinations in accordance with the fundamental principle of "We continually improve operations" specified in the Quality Policy.

The method of verifying quality audit, including whether a search range is appropriate, was reviewed.

Currently (as of March 2017), a quality audit system has not been created; however, preparation was made to provide a new system from FY2017 that provides functions for detailed setting of selection conditions for a case to be audited.

As a result, the verification of the appropriateness of a search range for examination specified by an examiner was enabled. In FY2017, it is expected that the use of the quality audit system will enable quality audit to be conducted before notices are sent out to applicants for all cases subject to audit, further increasing efficiency in quality audit. - In order to conduct appropriate quality management, it is important to run a PDCA cycle effectively. Thus, it is necessary to perform continual improvement, while evaluating the relation between the objectives of quality management initiatives and the results through following the PDCA cycle. As major initiatives, the following initiative was implemented to execute examinations in accordance with the fundamental principle of "We continually improve operations" specified in the Quality Policy.

For the initiatives for quality management specified in the "Initiatives for Examination Quality Management (FY2016)," the relation between these initiatives and their results was evaluated.

12

As a result, knowledge was shared and accumulated by means of consultations among examiners for all international applications for design registration, a system of providing the results of prior design searches as feedback to applicants was strengthened with the start of operation in which a scope of the search is written in a notification and sent with a notice of decision of registration, and active communication with applicants was performed. Consequently, in the user satisfaction survey of FY2016, the score of "improved" increased for changes in overall quality of design examination, prior design searches, and communication compared to other items, regarding changes in impression of quality of design examination. Furthermore, in the user satisfaction survey of FY2016, the level of satisfaction with prior design searches increased for 4 and 5 on a 1 to 5 scale. By the comparison between the initiatives for quality management specified in the "Initiatives for Examination Quality Management (FY2016)" and their results, it was confirmed that the PDCA cycle has been effectively functioning, and the quality of design examinations has been improved continuously. (2) Establishing Policies, Procedures, and Structures Aiming for High-Quality Examination - In order to improve examination quality, it is crucial to enhance the organizational structure for examinations. Thus, continuous efforts need to be made to secure the necessary number of examiners, taking into account the efficiency of examination. As major initiatives, the following initiative was implemented to execute examinations in accordance with the fundamental principle of "We grant robust, broad and valuable design rights" specified in the Quality Policy.

The examination implementation system for efficient and appropriate design examinations has been ensured.

As a result, two examiners were newly employed to improve and strengthen the organizational structure for examinations. Also, a way for examiners with years of experience to be reemployed so that they can continue working on examination was introduced (one examiner was reemployed in FY2016).

13

- In order to improve examination quality, it is also crucial to enhance the quality management system. Thus, efforts need to be made to further enhance the quality management system, including the improvement of the infrastructure for quality audit, while considering the efficiency of quality management practices. As major initiatives, the following initiatives were implemented to execute examinations in accordance with the fundamental principle of "We continually improve operations" specified in the Quality Policy.

The quality management system and infrastructure were enhanced, and quality audit and information provision to the examination divisions were performed in a more appropriate and timely manner.

Preparation was made to provide a case selection system from FY2017, which is highly flexible and efficient.

As a result, efforts were made to enhance the system for planning and making proposals for quality management, and one coordinator for planning was assigned to the Design Division in June 2016. In addition, four Quality Management Officers conducted 168 cases of quality audit in FY2016 (112 registered cases and 56 refused cases). In FY2017, it is expected that the use of the quality audit system will enable quality audit to be conducted before notices are sent out to applicants for all cases subject to audit, further increasing efficiency in quality audit. - In order to manage the quality of examination, it is crucial that each of the personnel who engages in examination has a good understanding of quality management. Thus, training and seminars on quality management were provided to examiners and assistant examiners to promote their understanding of quality management. As major initiatives, the following initiative was implemented to execute examinations in accordance with the fundamental principles of "We grant robust, broad and valuable design rights," "We raise the knowledge and capabilities of our staff," and "We all dedicate ourselves to improving quality, cooperating with concerned persons and parties" specified in the Quality Policy.

In order to raise the awareness of personnel on quality management and to improve the level of satisfaction on communication between users and examiners, lectures were conducted on quality management in design examination through training exercises for developing examiners.

14

As a result, the knowledge and capabilities of examiners were improved, which form the basis of the improvement in examination quality, and knowledge was shared among examiners. Furthermore, in the user satisfaction survey of FY2016, the level of satisfaction with "consistency of judgments among examiners," "communication with examiners (such as face-to-face interviews and telephone conversations)," and other items improved from the previous fiscal year, with an increase in the number of evaluations of 4 and 5 on a scale of 1 to 5. (3) Communication of information on initiatives for examination quality improvement - In order to support global businesses of companies, it is necessary to build an environment where examination results produced by the JPO are highly regarded among the international IP community, which allows users to obtain design rights smoothly in foreign countries. Thus, efforts need to be made to actively communicate the JPO's initiatives for examination quality to overseas IP offices through international meetings or other occasions, as well as to build cooperative relations with overseas IP offices to collect information on their initiatives for examination quality. As major initiatives, the following initiatives were implemented to execute examinations in accordance with the fundamental principle of "We contribute to improving the quality of design examination globally" specified in the Quality Policy.

Through international meetings or other occasions, efforts were made to increase trust in examination at the JPO, and to spread high-quality examination implementation to overseas IP offices including those in emerging countries.

The initiatives of the JPO for quality management of design examination were introduced at the Industrial Design 5 Forum (ID5), and the information on the initiatives was actively communicated to overseas IP offices.

Examiners of the JPO were dispatched as trainers for a training program for examiner development in Myanmar.

The initiatives of the JPO for the quality management of design examination were introduced at the Japan-China-Korea Design Forum, and the information on the initiatives was actively communicated to the IP offices of China and Korea.

As a result, shared understanding on the quality management system was created at the ID5 and the Japan-China-Korea Design Forum. Furthermore, efforts were

15

made to contribute to improvement in practical examination capabilities of the IP Office in Myanmar, and to spread the examination practices of the JPO to the IP office. 3. Trademarks (1) Enhancing Initiatives for Improvement in Examination Quality and for Quality Verification - In order to enhance initiatives for quality verification, it is important to listen carefully to users' voices. Thus, while the content of the user satisfaction survey and the exchanges of opinions with users, etc. was enhanced, opinions collected through a contact for receiving opinions regarding the quality of examinations (such as an opinion submission form for improving the quality of examinations) were utilized to reflect users' voices to the quality management initiatives. As major initiatives, the following initiatives were implemented to execute examinations in accordance with the fundamental principle of "We conduct consistent and objective trademark examination" specified in the Quality Policy.

The user satisfaction survey of FY2016 was conducted using questionnaires to which necessary modifications were made to reduce the load on respondents while maintaining continuity with the surveys conducted for the last two years.

The examination divisions exchanged opinions with industry organizations and other entities to grasp industry trends and users' needs.

As a result, in the user satisfaction survey of FY2016, response rates improved from the previous fiscal year for all the two types of questionnaires4 that were used. In the user satisfaction survey of FY2016, the level of satisfaction with the overall quality of examinations improved from the previous fiscal year, with an increase in the number of evaluations of 4 and 5 on a scale of 1 to 5.

- Trademark search on distinctiveness is one of the essential processes in trademark examination, and thus improvement in search capabilities is required. Thus, efforts were made to, for example, accumulate and share search know-how, including that on non-traditional trademarks, and trademark searches on distinctiveness were further enhanced.

4 The two types are as follows: a questionnaire on the quality of overall trademark examinations and a questionnaire on the quality of trademark examinations on specific applications.

16

As major initiatives, the following initiatives were implemented to execute examinations in accordance with the fundamental principle of "We conduct consistent and objective trademark examination" specified in the Quality Policy.

As part of improvement of examination materials for non-traditional trademarks, the manual for substantive examination (color trademark, sound trademark, and position trademark) was reviewed.

For the trial cases for which opposition of trademark registration was filed in FY2015 and consequently decision of rescission was made (49 cases), analyses were conducted on the factors or other elements of the discrepancy between examination decisions and appeal/trial decisions, and the results of the analyses were shared in the examination departments as necessary.

As a result, in the user satisfaction survey of FY2016, the level of satisfaction with "judgments of distinctiveness" and "consistency among examiners" improved from the previous fiscal year, with an increase in the number of evaluations of 4 and 5 on a scale of 1 to 5.

- Properly judging similarity of trademarks is one of the most important pillars, and thus judgment in examination needs to be conducted on the basis of the latest actual trade conditions and business trends. Thus, the Examination Guidelines for Trademarks, including the provisions on similarity of trademarks, were reviewed on the basis of users' needs and actual trade conditions obtained by making efforts to enhance exchanges of opinions with users and representatives. As major initiatives, the following initiatives were implemented to execute examinations in accordance with the fundamental principle of "We conduct consistent and objective trademark examination" specified in the Quality Policy.

The examination divisions responsible for the applications filed by companies exchanged opinions with industry organizations and other entities to grasp industry trends and users' needs.

Factors or other elements of discrepancy were analyzed mainly for cases for which trial decisions were different from examination decisions in the trials against the final rejection of the application, opinions were exchanged between the examination departments and the Trial and Appeal Department, and information was shared in the Trial and Appeal Department on newly stipulated processing policies and other procedures.

17

As a result, in the user satisfaction survey of FY2016, the level of satisfaction with "judgments of similarity" improved from the previous fiscal year, with an increase in the number of evaluations of 4 and 5 on a scale of 1 to 5.

- In order to enhance examination quality, it is necessary for each examiner to steadily perform examination according to the Examination Guidelines for Trademarks revised in FY2015. Thus, consultations among examiners were enhanced by promoting the exchange of opinions and the sharing of knowledge on judgment on distinctiveness of trademarks, including non-traditional trademarks. As major initiatives, the following initiatives were implemented to execute examinations in accordance with the fundamental principle of "We conduct consistent and objective trademark examination" specified in the Quality Policy.

In order to conduct a comprehensive review on the Examination Guidelines for Trademarks, the Working Group on Trademark Examination Standards, which was established in Trademark System Subcommittee of Intellectual Property Policy Committee under the Industrial Structure Council, deliberated on the matters, and drew up the revised examination guidelines in March this year.

In order to increase the efficiency of trademark searches on distinctiveness, a framework was provided to collect and share consultation notes among all examination divisions.

The Trademark Division, the directors of the examination divisions, and other staff had consultations and discussions on cases requiring deliberate decisions (cases that may have a profound impact on the society).

As a result, in the user satisfaction survey of FY2016, the level of satisfaction with "consistency with the examination guidelines and the examination manual", and "judgments of distinctiveness" improved from the previous fiscal year, with an increase in the number of evaluations of 4 and 5 on a scale of 1 to 5.

- In order to more accurately understand the current level of examination quality, it is also important to enhance quality audit. Thus, quality audit was enhanced by, for example, improving the infrastructure for allowing the Quality Management Division to conduct appropriate quality audit. As major initiatives, the following initiatives were implemented to execute examinations in accordance with the fundamental principle of "We continually improve operations" specified in the Quality Policy.

18

Through the enhancement of quality audit, quality was understood properly, and the understanding was utilized for improvement. Quality audit was conducted in a specified period of time (three months) in the previous fiscal year; however, the operation was changed to conduct quality audit throughout the year from June FY2016.

Currently (as of March 2017), a quality audit system has not been created; however, preparation was made to provide a new system from FY2017 that provides functions for detailed setting of selection conditions for a case to be audited.

As a result, the transparency and reliability of the results of quality audit increased due to the change in the execution method of quality audit. In FY2017, it is expected that the systemization of quality audit will enable quality audit to be conducted before notices are sent out to applicants for all cases subject to audit, and efficiency in quality audit will be further increased. - In order to conduct appropriate quality management, it is important to run a PDCA cycle effectively. Thus, it is necessary to perform continual improvement, while evaluating how much objectives of quality management initiatives have been achieved through following the PDCA cycle. As major initiatives, the following initiatives were implemented to execute examinations in accordance with the fundamental principle of "We continually improve operations" specified in the Quality Policy.

For the initiatives for quality management specified in the "Initiatives for Examination Quality Management (FY2016)," the relation between these initiatives and their results was evaluated.

As a result, by the comparison between the initiatives for quality management specified in the "Initiatives for Examination Quality Management (FY2016)" and their results, it was confirmed that the PDCA cycle has been effectively functioning, and the quality of trademark examinations has been improved continuously. In the user satisfaction survey of FY2016, the level of satisfaction with the overall quality of examinations improved from the previous fiscal year, with an increase in the number of evaluations of 4 and 5 on a scale of 1 to 5. Furthermore, the percentage of non-conformance in the quality audit in FY2016 decreased.

19

(2) Establishing Policies, Procedures, and Structures Aiming for High-Quality Examination - In order to improve examination quality, it is crucial to enhance the organizational structure for examinations. Thus, continuous efforts need to be made to secure the necessary number of examiners in order to ensure an examination implementation system for enabling efficient and appropriate examinations of trademarks including non-traditional trademarks. As major initiatives, the following initiatives were implemented to execute examinations in accordance with the fundamental principle of "We contribute to the protection and enhancement of brands and the smooth consumption of goods and services" specified in the Quality Policy.

For improving the organizational structure for examinations for conducting efficient and appropriate trademark examinations, eight examiners were newly employed.

A way for examiners with years of experience to be reemployed so that they can continue working on examination was introduced (15 examiners were reemployed in FY2016).

To accelerate the examination process for non-traditional trademarks, a dedicated examination team has been further established.

While the number of trademark applications has been increasing, the current quality of examinations was maintained and examinations were conducted steadily. At the same time, the range of applicable applications for the accelerated examination system was broadened so that it is available to applicants who wish earlier grant of rights.

As a result, in the user satisfaction survey of FY2016, the level of satisfaction with the overall quality of examinations improved from the previous fiscal year, with an increase in the number of evaluations of 4 and 5 on a scale of 1 to 5. In addition, in the user satisfaction survey of FY2016, for the level of satisfaction with "timeliness of examinations (examination period)," 90% or more users responded as being satisfied with the current state or having no problem. - In order to improve examination quality, it is also crucial to enhance the quality management system. Thus, efforts need to be made to further enhance the quality management system, including the establishing of a section dedicated to quality management.

20

As major initiatives, the following initiatives were implemented to execute examinations in accordance with the fundamental principle of "We continually improve operations" specified in the Quality Policy.

In order to enhance the system for planning and making proposals on quality management, the Quality Management Section was newly established in the Trademark Division in April 2016 for managing the quality of examinations objectively and consistently.

Quality audit was conducted, with cases subject to quality audit selected in an appropriate and timely manner.

An efficient case selection system with high flexibility is expected to be provided by FY2017.

As a result, within the organization structure of examination quality management, the roles of persons in charge, persons conducting examinations, persons planning and making proposals for initiatives, and persons analyzing and evaluating the quality of examinations were clearly divided. Thus, the organizational and staffing structure that allows more efficient and effective quality management was established. In addition, cases were assigned to 10 Quality Management Officers by industrial field, and about 3,000 cases of quality audit in total (1,000 registered cases and 2,000 refused cases) were conducted. Furthermore, in FY2017, it is expected that the systemization of quality audit will enable quality audit to be conducted before notices are sent out to applicants for all cases subject to audit, and efficiency in quality audit will be further increased. - In order to manage the quality of examination, it is crucial that each of the personnel who engages in examination has a good understanding of quality management. Thus, training and seminars on quality management were provided to examiners and other staff to promote their understanding of quality management. As major initiatives, the following initiatives were implemented to execute examinations in accordance with the fundamental principles of "We raise the knowledge and capabilities of our staff," "We conduct consistent and objective trademark examination," "We promote the utilization of the trademark system by closely communicating with applicants," and "We contribute to the protection and enhancement of brands and the smooth consumption of goods and services" specified in the Quality Policy.

21

In order to promote examiners' understanding of quality management, lectures were conducted on quality management in trademark examinations through training exercises (first-period training of examiner course, later-period training of examiner course, etc.) for developing examiners.

Through the enhancement of quality audit, quality was understood properly, and the understanding was utilized for improvement.

As a result, in the user satisfaction survey of FY2016, the level of satisfaction with "consistency of judgments among examiners," "telephone/FAX contact," "interviews," and other items improved from the previous fiscal year, with an increase in the number of evaluations of 4 and 5 on a scale of 1 to 5. (3) Communication of information on initiatives for examination quality improvement - In order to enhance users' understanding of JPO's initiatives for examination quality management and to increase its presence regarding quality management in the international IP community, it is crucial to provide information appropriately to overseas IP offices including those in emerging countries. Thus, efforts need to be made to actively communicate the JPO's initiatives for examination quality to overseas IP offices, including those in emerging countries, through international meetings or other occasions, as well as to build cooperative relations with overseas IP offices to collect information on their initiatives for examination quality, in order to contribute to enhancing quality of trademark examination globally. As major initiatives, the following initiatives were implemented to execute examinations in accordance with the fundamental principles of "We actively share information with relevant persons inside and outside Japan in order to improve the quality of trademark examination" specified in the Quality Policy.

Through international meetings or other occasions, efforts were made to increase trust in examination at the JPO, and to spread high-quality examination to overseas IP offices including those in emerging countries.

The initiatives of the JPO for the examination quality management were introduced at the International Trademark Association (INTA) Annual Meeting, and the information on the initiatives was actively communicated to overseas IP offices including those in emerging countries.

22

The JPO made a proposal for launching a project concerning quality management at a meeting of the Five Trademark Offices (TM5), and the five offices agreed to share the information on quality management initiatives of the five offices in the future.

The initiatives of the JPO for the examination quality management were introduced through an examiner exchange program between Japan and Taiwan, dispatch of experts, and acceptance of other trainees (Columbia, the Philippines, India, Kazakhstan, Mexico, etc.), and the quality management initiatives of respective IP offices were discussed.

As a result, the offices have created shared understanding of their quality management systems one another at the Annual Meeting of the INTA. In addition, it is expected that the participants in the meetings of the TM5 will create shared understanding of their quality management systems one another in the future. Furthermore, the exchange of examiners, dispatch of experts, acceptance of other trainees, and other programs have created shared understanding among participants on their quality management systems one another.

23

II. Evaluation Results Concerning Implementation Systems/Implementation Status in Quality Management Evaluation was conducted according to the Evaluation Items and Criteria Concerning Examination Quality Management (See Reference 1 at the end of the report) formulated by the Subcommittee in FY2014. As evaluation criteria for each evaluation item, a scale of 1 to 4 ("Very Satisfactory," "Satisfactory," "Generally Achieved," and "Requiring Improvement") was adopted so that it allow appropriate evaluations without unnecessary complexity. In addition, because examination processes are common among patents, designs, and trademarks, common evaluation criteria for assessing the implementation systems/implementation status in quality management are used among them. For each evaluation criterion, the objectives and aspects of evaluation for each evaluation item were clarified, and the actions, status, and other items to be achieved at each stage were specifically defined. In particular, evaluation items (f) and (g) were evaluated as "Very Satisfactory" only in the case where both of the conditions "initiatives necessary for the improvement of quality had been planned and implemented as planned, and their objectives had been achieved" and "it was recognized to have effects that would contribute to further improvement of quality" were achieved. In addition, evaluation item (h) was evaluated as "Very Satisfactory" only in the case where both of the conditions "analysis of examination quality and identification of issues had been conducted sufficiently" and "identification of issues had been conducted from a comprehensive perspective" were achieved. Furthermore, evaluation items (i) and (j) were evaluated as "Very Satisfactory" only in the case where the condition that improvement in "policies, procedures, and structures" as well as "quality management initiatives" had been made "at an excellent level" was achieved, instead of "sufficiently been made." Before the deliberation by the Subcommittee, the JPO provided its members with materials outlining the outcomes and status of the implementation of examination quality management on patents, designs, and trademarks, respectively, regarding the eleven evaluation criteria (Handouts 2 to 4 of the first meeting of FY2016, and Handouts 2 to 4 of the second meeting of FY2016). Points that were unclear regarding the outcomes and status of implementation in the handouts, if any, were followed-up in

24

the Q&A session with the JPO on the same day as the deliberation. Upon request from the Subcommittee members, a visit to the sites where examination or quality management is performed in the JPO was available. The Subcommittee members evaluated each evaluation item on a scale of 1 to 4 according to the evaluation criteria, based on, for example, descriptions of the handouts mentioned above. Subsequently, the Subcommittee deliberated to compile an official evaluation of the Subcommittee based on each member's evaluation and the results of the Q&A session. The median value of the scores given by the Subcommittee's members was used as a Subcommittee's official evaluation. However, different evaluations by some members from the official Subcommittee's evaluations were, if any, also described in association with the evaluation criteria. The Subcommittee's evaluation results are summarized as follows.

25

1. Patents Evaluation item (a) (Status of creation of the Quality Policy, Quality Manual, and other documents) The Quality Policy, the Quality Manual, and other documents indicating specific procedures for quality management were created and appropriately managed, and thus this item was evaluated as "Very Satisfactory." However, some members gave an evaluation of "Satisfactory" because it was unclear which activity was the basis for the revision of the Quality Manual; a quality assurance system chart or a document system chart was not clearly shown in the Quality Manual, and thus the relation between the Quality Manual and the documents specifying the procedures for quality assurance was ambiguous. In addition, other members gave an evaluation of "Satisfactory" because simpler and more comprehensible description was desirable for documents of which main purpose was external communication from the JPO. Evaluation item (b) (Clarity of procedures for examinations and quality management) This item was evaluated as "Very Satisfactory," because the Examination Guidelines for Patent and Utility Model stipulates what needed to be done in the examination process and the specific procedures and that the Quality Manual clearly describes who should be responsible for establishing and implementing the quality management system, and the procedures and the persons in charge of the implementation of quality management. However, some members gave an evaluation of "Satisfactory" because the reason for the revision of the documents specifying the processes was not clear, and the relation between the results of each process and each procedure was not clear. It was thus difficult to judge whether the current procedures were clear enough for assuring examination quality and which points should be improved. Other members gave an evaluation of "Generally Achieved" because, though the Examination Guidelines for Patent and Utility Model stipulates what needed to be done in the examination process and the specific procedures, the stipulated procedures needed to be disseminated thoroughly to examiners. Evaluation item (c) (Publication of the fundamental principles of quality management etc. to users of IP systems and dissemination of such information to the staff)

26

The Quality Policy and the Quality Manual were published so that users including overseas users easily access them. They were also disseminated to all the staff members who engaged in examination through multiple ways. Training exercises and seminars for the staff were provided on a regular basis. In addition, the understanding of the staff members on the knowledge needed for examination was checked. Moreover, intensive discussions on quality were conducted by all the examiners in each examination division for a fixed period of time. Through these, the current status of understanding on quality management was assessed and their understanding on quality management was promoted. Accordingly, this item was evaluated as "Very Satisfactory." However, some members gave an evaluation of "Satisfactory" because they felt that dissemination and publication to overseas users were not enough. Other members gave an evaluation of "Generally Achieved" because strengthening communication of information to users was still desirable. Evaluation item (d) (Examination implementation system) The examination system and staffing structure was established to an internationally comparable level as a result of initiatives for enhancing the examination system and increasing the efficiency in examination, which allowed examiners to achieve their quota efficiently. However, the world's utmost organizational and staffing structure for examination had not completely been established yet. Thus, this item was evaluated as "Satisfactory." However, some members gave an evaluation of "Generally Achieved," because the load of examinations has increased mainly due to the increase in the number of PCT applications. Evaluation item (e) (Quality management system) The JPO established an organizational structure of examination quality management, in which persons in charge, persons conducting examinations, persons planning and making proposals for initiatives, and persons analyzing and evaluating the quality of examinations were all independently positioned. For example, written notices were assigned to the Quality Management Officers based on the major types of the notice, aiming for better audit practices. Overall, the organizational and staffing structure was established in order to enable planning and making proposals for initiatives for quality management in an efficient and effective manner so that quality management initiatives be conducted at an internationally comparable level. Thus, this

27

item was evaluated as "Satisfactory." Evaluation item (f) (Initiatives for quality improvement) Initiatives necessary for improvement of quality (approval, checking drafted notices before approval, consultations, setting targets for each examiner and evaluations on his/her achievements, interviews or telephone contact, enhancement of searches of foreign patent documents, collection and provision of quality-related information, training exercises and seminars, evaluations and guidance on prior art searches conducted by registered search organizations, and provision and maintenance of search indexes), which started in FY2015, were continuously implemented as planned in FY2016. As an initiative that would contribute to further improvement of quality, consultations were enhanced for the cases requiring special attention to application of the revised examination guidelines or other standards, such as judgment in a case in which a claim concerning an invention of a product recites a manufacturing process of the product. Consultations focusing on searches were also enhanced. Furthermore, revisions were made onto the Examination Handbook, as to how judgment should be made in the cases in which a claim concerning an invention of a product recites a manufacturing process of the product, and the examples were added on Internet of Things (IoT)-related technology that has increasingly been applied in research/development and businesses. The objectives of each initiative were thus achieved. Accordingly, this item was evaluated as "Satisfactory." However, some members gave an evaluation of "Very Satisfactory" because the effects that would contribute to further improvement of quality were achieved through the above-described initiatives. Evaluation item (g) (Initiatives for quality verification) Verification of the validity of searches as well as the validity of identification and judgments through quality audit and verification of formality matters of notices through partial quality audit, which were both required for the verification of quality, were conducted as planned in terms of the number of cases, and the objectives of each initiative were achieved. Furthermore, improvement was made to the method of providing the results of quality audit to the managerial staff in the examination departments, and feedback was provided on the results of examination on the filed oppositions to grant of patent, and thus this item was evaluated as "Satisfactory." However, some members gave an evaluation of "Very Satisfactory" because initiatives necessary for quality validation such as sample checks and the user

28

satisfaction survey had been planned and implemented, their objectives had been achieved, and the effects that would contribute to further improvement of quality were achieved. Evaluation item (h) (Examination quality analysis and identification of issues) The identification of issues was conducted sufficiently by means of an analysis based on the evaluations obtained in the JPO such as approvals and quality audit, and an analysis based on the evaluations obtained from the outside of the JPO such as the user satisfaction survey. Furthermore, the issues to be addressed were narrowed down. Thus, this item was evaluated as "Satisfactory." However, some members gave an evaluation of "Very Satisfactory" because the issues were narrowed down from the perspective of improvement in the method of analyses and from the comprehensive perspective, and the issues to be addressed in the future on the examination process were specifically narrowed down. Evaluation item (i) (Status of improvement of policies, procedures, and structures to achieve high-quality examinations (evaluation items from (a) to (e))) The Quality Manual was revised without delay so as to reflect the implementation systems in FY2016; the basics of quality management were disseminated to the staff members more thoroughly than before, and then it was checked whether they understood them well; and the policies, procedures, and structures were sufficiently improved. Thus, this item was evaluated as "Satisfactory." Evaluation item (j) (Status of improvement of quality management initiatives (evaluation items from (f) to (h))) Improvement in quality management initiatives, such as improvement in the efficiency of searches and in the consistency of judgments, was sufficiently made in response to the issues identified through the analysis in FY2015. Thus, this item was evaluated as "Satisfactory." However, some members gave an evaluation of "Very Satisfactory" because improvement in quality management initiatives was actively made with various creative measures. Other members gave an evaluation of "Generally Achieved" because, though efforts were made to improve examination quality, further improvement from users' perspective was desirable.

29

Evaluation item (k) (Communication of information on initiatives for examination quality improvement) Information on examination quality improvement was communicated domestically and internationally, and cooperative relations was built with domestic and overseas institutions and organizations; however, building cooperative relations with emerging countries, strengthening of communication with overseas users, and collaborating with them to conduct initiatives for examination quality improvement were desirable in the future. Thus, this item was evaluated as "Satisfactory." However, some members gave an evaluation of "Very Satisfactory" because information on examination quality improvement was communicated to domestic and overseas users through exchanges of opinions as well as to overseas IP offices through international gatherings and dispatch/acceptance of examiners. Furthermore, continuous cooperative relations with other IP offices were built and maintained. Other members gave an evaluation of "Generally Achieved" because strengthening communication of information to users was still desirable.

2. Designs Evaluation item (a) (Status of creation of the Quality Policy, Quality Manual, and other documents) The Quality Policy, the Quality Manual, and other documents indicating specific procedures for quality management were created and appropriately managed, and thus this item was evaluated as "Very Satisfactory." Some members gave an evaluation of "Satisfactory" because it was unclear which activity was the basis for the revision of the Quality Manual; a quality assurance system chart or a document system chart was not clearly displayed in the Quality Manual, and thus the relation between the Quality Manual and the documents specifying the procedures for quality assurance was ambiguous. In addition, simpler and more comprehensible description was desirable for documents whose main purpose was external communication from the JPO. Evaluation item (b) (Clarity of procedures for examinations and quality management) It was recognized that the Examination Guidelines for Design stipulated what needed to be done in the examination process and the specific procedures. Meanwhile, the Quality Manual clearly described establishing and improving the quality

30

management system, its implementation, and who was responsible for quality management. It also described the procedures for quality management. Thus, this item was evaluated as "Very Satisfactory." Some members gave an evaluation of "Satisfactory" because the reason for the revision of the documents specifying processes was not clear, and the relation between the results or other findings of the processes and the respective procedures was not clear. It was thus difficult to judge whether the current procedures were clear enough for assuring examination quality and what the points to improve were. Evaluation item (c) (Publication of the fundamental principles of quality management, etc. to users of IP systems and dissemination of such information to staff) The Quality Policy and the Quality Manual were published so that users, including overseas users, could easily access to them. They were also disseminated through the web pages containing quality management information on the intranet of the JPO, in addition to conventional methods, to all the staff members who engaged in examination. Not only that, training exercises and seminars for staff were provided on a regular basis. In addition to these, training exercises and seminars on the importance of examination quality and the fundamental principles of quality management were continued to be provided from the previous fiscal year for all the design examiners, and their understanding was tested after each program. Thus, this item was evaluated as "Very Satisfactory." Other members gave an evaluation of "Satisfactory" or "Generally Achieved" because strengthening of communication of information to users was desirable. Evaluation item (d) (Examination implementation system) Regarding examinations processed, under the current organizational and staffing structure, it was recognized that the number of issuing first actions had been almost identical to that of filing applications over these past few years. Moreover, an average period from filing an application to sending a notice of first action reached as short as 6.1 months in 2015, remaining the same in the first half of 2016. However, the number of examinations per examiner was greater in Japan than in the U.S., and what is more, the limited number of design examiners conducted examinations of international applications for design registration., as well as worked on initiatives for quality improvement. So, it fell short of saying that the organizational and staffing structure for examinations was well established, compared with other IP offices where the

31

substantive examination was adopted. Thus, this item was evaluated as "Generally Achieved." Some members gave an evaluation of "Satisfactory" because the substantive examinations were performed on schedule with the limited number of examiners. Evaluation item (e) (Quality management system) The roll of a coordinator for planning (one officer) was established in the Design Division that plans and make proposals for quality management. The coordinator for planning served as an executive officer and analyzed and evaluated the quality audit. Examination quality was properly managed despite the limited human resources within the organization, and therefore, it could be regarded that a system for quality management equivalent to that of other IP offices where substantive examination is adopted was established at the JPO. Thus, this item was evaluated as "Satisfactory." Evaluation item (f) (Initiatives for quality improvement) Initiatives necessary for improvement of quality (e.g., consultations on national applications and their feedback, consultations for all international applications for design registration), which started in FY2015, continued to be implemented as planned in FY2016. As an initiative that would contribute to further improvement of quality, the Examination Guidelines for Design, the "Manual for description of an application for design registration and drawings" and other documents were revised, and draft documents of all cases of international applications for design registration were double-checked for formality flaws. Thus, this item was evaluated as "Satisfactory." Evaluation item (g) (Initiatives for quality verification) Verification of the validity of judgment and notices drafted by examiners through quality audit, which were required for the verification of quality, was conducted as planned. A system was also established to allow quality audit to be conducted on approved cases waiting for sending out. Regarding the quality audit for international applications for design registration, all applications continued to be subject to consultations in order to see whether verification items were appropriate. Regarding the user satisfaction survey, necessary items were added and the range of respondents surveyed was broadened. The response rate also increased from the previous fiscal year. Thus, this item was evaluated as "Satisfactory." Some members gave an evaluation of "Generally Achieved" because some of the objectives of verification had not been achieved.

32

Evaluation item (h) (Examination quality analysis and identification of issues) It was recognized that analyses of examination quality were conducted inside and outside the JPO, and identification of issues was conducted sufficiently. Thus, this item was evaluated as "Satisfactory." Some members gave an evaluation of "Generally Achieved" because issues and circumstances of the analyses were unclear. Evaluation item (i) (Status of improvement of policies, procedures, and structures to achieve high-quality examinations (evaluation items from (a) to (e))) The Quality Manual was revised so as to reflect the implementation system in FY2016, and the basics of quality management continued, from the previous fiscal year, to be sufficiently disseminated to staff members by providing lectures for all design examiners. In addition, the policies, procedures, and structure were sufficiently improved by, for example, assigning an officer dedicated to planning and making proposals for quality management as an executive officer who analyzed and evaluated the quality audit. Thus, this item was evaluated as "Satisfactory." Evaluation item (j) (Status of improvement of quality management initiatives (evaluation items from (f) to (h))) Initiatives for quality management were improved in response to the issues identified in FY2015. Specifically, efforts were made on improvements such as a review of the implementation system of the user satisfaction survey, establishment of a system for quality audit, and implementation of a double-check of draft documents for international applications for design registration. Thus, this item was evaluated as "Satisfactory." Evaluation item (k) (Communication of information on initiatives for examination quality improvement) Information on examination quality improvement was communicated to domestic users and opinions were exchanged regularly through meetings with users. In this way, continuous cooperative relations were built and maintained. Moreover, information continued to be provided to overseas IP offices at international gatherings such as the Industrial Design 5 Forum (ID5) and the Meeting of the Joint Experts Group for Design (Japan, China and Korea), and through dispatch/acceptance of examiners. In this way, cooperative relations with other countries including emerging countries were built and maintained. Thus, this item was evaluated as "Satisfactory."

33

Others evaluated as "Generally Achieved" in view of further strengthening of communication of information to users. 3. Trademarks Evaluation item (a) (Status of creation of the Quality Policy, Quality Manual, and other documents) The Quality Policy, the Quality Manual, and other documents indicating specific procedures for quality management were created and appropriately managed, and thus this item was evaluated as "Very Satisfactory." Some members gave an evaluation of "Satisfactory" because it was unclear which activity was the basis for the revision of the Quality Manual; a quality assurance system chart or a document system chart was not clearly displayed in the Quality Manual, and thus the relation between the Quality Manual and the documents specifying the procedures for quality assurance was ambiguous. In addition, simpler and more comprehensible description was desirable for documents whose main purpose was external communication from the JPO. Evaluation item (b) (Clarity of procedures for examinations and quality management) It was recognized that the Examination Manual for Trademarks and the Outline of Trademark Examination Procedure stipulated what needed to be done in the examination process and the specific procedures. Meanwhile, the Quality Manual clearly described establishing and improving the quality management system, its implementation and who was responsible for quality management. It also described the procedures for quality management. Therefore, this item was evaluated as "Very Satisfactory." Some members gave an evaluation of "Satisfactory" because the reason for the revision of the documents specifying processes was not clear, and the relation between the results or other findings of the processes and the respective procedures was not clear. It was thus difficult to judge whether the current procedures were clear enough for assuring examination quality and what the points to improve were. Evaluation item (c) (Publication of the fundamental principles of quality management, etc. to users of IP systems and dissemination of such information to staff)

34

The Quality Policy and the Quality Manual were published so that users, including overseas users, could easily access to them. They were also disseminated through multiple methods to all staff members who engaged in examination. In addition, training exercises and seminars for staff were provided on a regular basis. Furthermore, lectures on the importance of examination quality and the fundamental principles of quality management were newly provided for all trademark examiners, and their understanding on the content of the lectures was tested after each program. Thus, this item was evaluated as "Very Satisfactory." However, some members gave an evaluation of "Satisfactory" because further communication to overseas users was necessary. Other members gave an evaluation of "Generally Achieved" because strengthening of communication of information to users was desirable. Evaluation item (d) (Examination implementation system) A dedicated examination team for non-traditional trademarks was organized as part of the initiatives for enhancing the organizational structure for examinations. Under such new structure, examiners conducted examinations at a high-quality level and in an efficient manner, while fulfilling their quota. However, improvement efforts for quality were made with a limited number of examiners, and hence it could not quite be said that an internationally comparable level of organizational and staffing structure for examination had been established. Thus, this item was evaluated as "Generally Achieved." Some members gave an evaluation of "Satisfactory" because the organizational structure for appropriate and prompt examinations had been established in which, for example, examiners were fulfilling their quota efficiently, the JPO provided the results of examinations earlier than oversea IP offices did, and the range of applicable applications for the accelerated examination system was broadened so as to achieve early reposes to applicants who wish earlier grant of rights. Evaluation item (e) (Quality management system) The Quality Management Section was newly established in the Trademark Division for planning and making proposals for quality-related initiatives consistently, and the roles of persons in charge, persons conducting examinations, persons planning and making proposals for initiatives, and persons analyzing and evaluating the quality of examinations were clearly divided within the organization structure of examination quality management. In this way, the organizational and staffing structure that allows

35

more efficient and effective quality management was established. Thus, this item was evaluated as "Satisfactory." Some members gave an evaluation of "Very Satisfactory" because the Quality Management Section was newly established in the Trademark Division, which completed a system for implementing quality management with an appropriate staffing structure in cooperation with committees for quality management in the JPO. Other members gave an evaluation of "Generally Achieved" because deliberate registration was required since trademarks tend to draw public attention. Efforts needed to be made to further enhance the systems for approval and consultation. Evaluation item (f) (Initiatives for quality improvement) Consultation notes created by examiners were collected and accumulated to share information in view of increased efficiency of searches for examination, and a framework for accumulating information regarding cases returned to examiners at approval was established in order to improve the efficiency of approval. Such initiatives for improving quality of examination were implemented as planned, and the objectives of the respective initiatives were achieved. Thus, this item was evaluated as "Satisfactory." Some members gave an evaluation of "Very Satisfactory" because the initiatives for improving quality were implemented as planned, and the effects that would contribute to further improvement were obtained. In addition, information was shared among examiners and managerial staff, and appropriate training exercises and seminars were conducted to improve the quality of trademark examinations. Furthermore, it was recognized that a better understanding was achieved between users and examiners through interviews and telephone contact to improve the quality of examination. Other members gave an evaluation of "Generally Achieved" because efforts for improvement were recognized but they fell short of "Satisfactory." Evaluation item (g) (Initiatives for quality verification) Regarding the quality audit, operation was changed to conduct quality audit throughout the year, and the transparency and reliability of the results were increased. Regarding the user satisfaction survey, the method of the survey was reviewed to drastically increase the response rate of questionnaires. Thus, the initiatives for verifying examination quality were implemented as planned. In addition, information was shared on the results of analyses on the cases in which there was a discrepancy between

36

examination decisions and appeal/trial decisions. Thus, the objectives of the respective initiatives were achieved, and this item was evaluated as "Satisfactory." Some members gave an evaluation of "Very Satisfactory" because the results of audit were verified, and a system for reflecting the results on the examination practices was established. In addition, the objectives of the initiatives for quality verification had been achieved, and effects that would contribute to further improvement of quality were achieved. Evaluation item (h) (Examination quality analysis and identification of issues) Under the current quality management system, analysis of examination quality and identification of issues were conducted in each of initiatives. Thus, this item was evaluated as "Satisfactory." Some members gave an evaluation of "Very Satisfactory" because the issues were narrowed down from the perspective of improvement in the method of analyses and from the comprehensive perspective, and the issues to be addressed in the future on the examination process were specifically narrowed down. Evaluation item (i) (Status of improvement of policies, procedures, and structures to achieve high-quality examinations (evaluation items from (a) to (e))) The Quality Manual was revised without delay so as to reflect the implementation systems in FY2016; the basics of quality management were disseminated to staff members more thoroughly than before, and then it was confirmed that they understood them well; and the policies, procedures, and structures were sufficiently improved. Thus, this item was evaluated as "Satisfactory." Evaluation item (j) (Status of improvement of quality management initiatives (evaluation items from (f) to (h))) Improvement in quality management initiatives was fully made in response to the issues identified through the analysis in FY2015. Thus, this item was evaluated as "Satisfactory." Some members gave an evaluation of "Very Satisfactory" because improvement in quality management initiatives was fully made, efficiency was promoted in quality audit by systemization of quality audit, and the results of the analyses on the factors or other elements of the discrepancy between examination decisions and appeal/trial decisions were disseminated to examiners.

37

Evaluation item (k) (Communication of information on initiatives for examination quality improvement) Information on examination quality improvement was communicated to domestic and overseas users through exchanges of opinions as well as to overseas IP offices through international gatherings and dispatch/acceptance of examiners. Furthermore, continuous cooperative relations with other IP offices were built and maintained. Thus, this item was evaluated as "Satisfactory." Some members gave an evaluation of "Very Satisfactory" because information was communicated to domestic users through exchanges of opinions with companies and industry organizations and other measures, as well as to overseas IP offices through international gatherings and dispatch/acceptance of examiners. Furthermore, continuous cooperative relations with other IP offices were built and maintained. Other members gave an evaluation of "Generally Achieved" because strengthening of communication of information to users was desirable.

38

III. Recommendations for Improvement in Implementation System/Implementation Status of Quality Management In parallel with evaluations concerning the implementation system/implementation status in quality management, the Subcommittee also deliberated on the matters expected to be improved concerning the implementation system/implementation status of quality management, which had been revealed through the evaluation process. Improvement recommendations by the Subcommittee are summarized as follows. 1. Patents (1) Recommendations for Evaluation item (a): Status of creation of documents - For related documents such as the Quality Policy, the Quality Manual, the Examination

Guidelines, the Examination Handbook, and other guidelines, create a list or other materials that indicates positioning of each document in the whole documentation system and the correlation among the documents.

(2) Recommendations for Evaluation items (d) and (e): Examination implementation system and quality management system - Continue enhancement of the system and infrastructure for supporting the efficiency

and quality improvement of examinations. - Continue the initiatives for achieving examinations of the world's utmost quality,

results of which are utilized in examinations conducted by overseas IP offices, in an appropriate and timely manner.

- Involve all staff members engaged in examination work in the improvement and assurance of examination quality.

(3) Recommendations for Evaluation item (f): Initiatives for quality improvement - Enhance prior art searches including searches for foreign patent literature and non-

patent literature, through sharing knowledge on searches such as new classifications and improving search environment.

- Promote initiatives for assuring and verifying appropriateness of examinations, specifically of searches, in the course of approval and consultations.

39

(4) Recommendations for Evaluation item (g): Initiatives for quality verification - Broaden the range of respondents in the user satisfaction survey so as to understand

users' needs more correctly. (5) Recommendations for Evaluation item (h): Examination quality analysis and identification of issues - Conduct continuous improvement while ensuring that the PDCA cycle is effectively

functioning through evaluating the relation between the initiatives for quality management and the results obtained from the initiatives.

(6) Recommendations of Evaluation item (k): Communication of information on initiatives for examination quality improvement - Increase reliability of the quality of patent examinations at the JPO through actively

communicating, to domestic and overseas users and overseas IP offices including those in emerging countries, JPO's initiatives for maintaining and improving examination quality, and providing opportunities to listen to their opinions on the initiatives, for example.

2. Designs (1) Recommendations for Evaluation item (a): Status of creation of documents - For related documents such as the Quality Policy, the Quality Manual, the Examination

Guidelines, the Examination Manual, and other guidelines, create a list or other material that indicates positioning of each document in the whole documentation system and the correlation among the documents.

(2) Recommendations for Evaluation items (d) and (e): Examination implementation system and quality management system - Secure the necessary number of examiners for the examination implementation system

that allows efficient and appropriate examinations. - Continue improving the system and infrastructure for supporting the efficiency and quality improvement of examinations. - Involve all staff members engaged in examination work in the improvement and assurance of examination quality.

40

(3) Recommendations for Evaluation item (f): Initiatives for quality improvement - Strengthen information sharing among examiners for improving and guaranteeing examination quality. - Enhance communication with applicants through on-site interview examinations or other means. (4) Recommendations for Evaluation item (g): Initiatives for quality verification - Enhance initiatives for quality management such as quality audit corresponding to

examinations of international applications for design registration. - Discuss operation for efficient audit practices by means of operation of a quality audit

system. - Enhance the user satisfaction survey to understand users' needs correctly. (5) Recommendations for Evaluation item (h): Examination quality analysis and identification of issues - Conduct continuous improvement while evaluating the relation between the initiatives for quality management and the results obtained from the initiatives. (6) Recommendations of Evaluation item (k): Communication of information on initiatives for examination quality improvement - Increase reliability of the quality of design examinations at the JPO through, for example,

actively communicating to domestic and overseas users and overseas IP offices including those in emerging countries, regarding JPO's initiatives for maintaining and improving examination quality.

3. Trademarks (1) Recommendations for Evaluation item (a): Status of creation of documents - For related documents such as the Quality Policy, the Quality Manual, the Examination

Guidelines, the Examination Manual, and other guidelines, create a list or other material that indicates positioning of each document in the whole documentation system and the correlation among the documents.

41

(2) Recommendations for Evaluation item (b): Clarity of procedures - Review the Examination Guidelines for Trademarks on the basis of users' needs. (3) Recommendations for Evaluation items (d) and (e): Examination implementation system and quality management system - Employ an appropriate number of examiners and enhance development of examiners. - Enhance the examination implementation system and quality management system

regarding examination of non-traditional trademarks. - Steadily perform examination according to the Examination Guidelines for Trademarks. - Promote consultations on important cases and share information on rulings and external

opinions (such as those from the mass media). - Continue system development for supporting the efficiency and quality improvement of

examinations. - Involve all staff members engaged in examination works for the improvement and assurance of examination quality. (4) Recommendations for Evaluation item (g): Initiatives for quality verification - Discuss operation for efficient audit practices by means of operation of a quality audit

system. - Broaden the range of respondents surveyed in the user satisfaction survey so as to

understand users' needs correctly. (5) Recommendations for Evaluation item (h): Examination quality analysis and identification of issues - Conduct continuous improvement while ensuring that the PDCA cycle is effectively

functioning through evaluating the relation between the initiatives for quality management and the results obtained from the initiatives.

(6) Recommendations of Evaluation item (k): Communication of information on initiatives for examination quality improvement - Increase reliability of the quality of trademark examinations at the JPO through, for

example, actively communicating to domestic and overseas users and overseas IP offices including those in emerging countries, regarding the JPO's initiatives for maintaining and improving examination quality, and providing opportunities to listen to their opinions on the initiatives.

42

IV. Conclusion Through verifications and evaluations on the implementation system of the quality management and its status in FY2016, it was confirmed that evaluation results and improvement recommendations provided by the Subcommittee in FY2015 were reflected in the initiatives within the JPO. In addition, consultations among examiners were enhanced in order to steadily conduct examinations in accordance with the revised examination guidelines or other standards, and thus examinations were also conducted steadily in accordance with the new examination guidelines. Moreover, it was confirmed that examination quality at the JPO remained at an internationally high level, and the initiatives for building trust relations with overseas IP offices had been promoted. Furthermore, it was also confirmed that opportunities to communicate with users of the industrial property rights system had been increased. In light of these points, this Subcommittee expects that the JPO will continue its efforts to improve the quality of examination, through evaluation results and improvement recommendations concerning the implementation system of quality management and its status as outlined in this report being reflected in the initiatives to be implemented within the JPO, which would result in further enhancement of the implementation system of examination quality management, promoting cooperation between applicants serving as users and patent attorneys serving as representatives. In addition, the Subcommittee also expects that the JPO will contribute to global activities of users of the industrial property right system through actively communicating its high-quality examination results to overseas IP offices, and continuing to interact with them in the area of quality management.

1

Evaluation Items and Criteria Concerning Examination Quality Management Appendix

Items Objectives and Perspectives Examples for

evaluation materials

Examples of evaluation methods/ evaluation criteria

Very satisfactory Satisfactory Generally achieved Requiring

Improvements I. Have policies, procedures, and structures been established to achieve high-quality examination? (1) Have policies and procedures been established to achieve high-quality examination?

(a)

Status of creation of Quality Policies, Quality Manuals, and other documents

To evaluate whether the Quality Policies stipulating the fundamental principles of quality management, the Quality Manuals describing initiatives for improvement of examination quality management along with the roles of departments/divisions and the personnel, and other documents indicating specific procedures for the purpose of quality management have been properly created, and to confirm whether Code of Conduct for the improvement of examination quality has been documented.

The Quality Policies and the Quality Manuals, sample documents of specific procedures, etc.

The Quality Policies, the Quality Manuals, and documents indicating specific procedures have been created and have been appropriately managed.

The Quality Policies and the Quality Manuals have been created, and documents indicating specific procedures have also been created.

The Quality Policies and the Quality Manuals have been created.

Either the Quality Policies or the Quality Manual has been created.

(b) Clarity of procedures for examination and quality management

To evaluate whether it is clearly stipulated who is to do what, and when, regarding examination and quality management, and to confirm whether specific procedures for the improvement of examination quality have been defined.

The procedural method and the flow for examination, quality management, etc.

The procedures and responsible persons for examination and quality management have been made sufficiently clear.

The procedures and responsible persons for examination and quality management have been made clear.

The procedures and responsible persons for examination and quality management have been generally made clear.

The procedures and responsible persons for examination and quality management have not been made clear.

2

Evaluation Items and Criteria Concerning Examination Quality Management Appendix

Items Objectives and Perspectives Examples for

evaluation materials

Examples of evaluation methods/ evaluation criteria

Very satisfactory Satisfactory Generally achieved Requiring

Improvements

(c)

Publication of the fundamental principles of quality management, etc. to users of IP systems and dissemination of such information to staff

• To evaluate whether the fundamental principles of examination quality management that the JPO has formulated as a goal, and other relevant initiatives have been clearly shown to users of IP systems, including overseas users, and to confirm whether examination quality is allowed to be evaluated in relation to such fundamental principles. • To evaluate whether the fundamental principles of examination quality management that the JPO has formulated as a goal have been sufficiently disseminated to and understood by staff, and to confirm whether staff is allowed to conduct their works in accordance with them.

The status of publication, the methods of access, the status of dissemination to staff and their understanding, etc.

Policies and procedures on quality management have been published to the degree that users, including overseas users, can easily access, and have been disseminated through multiple methods to all staff members who engage in examination. Also, trainings have been provided regularly for staff, and the staff has well understood the content of the trainings.

Policies and procedures on quality management have been published to the degree that national users can easily access, and have been disseminated through multiple methods to all staff members who engage in examination.

Policies and procedures on quality management have been published and disseminated to all staff members who engage in examination.

Policies and procedures on quality management have not been published or disseminated to staff.

I. Have policies, procedures, and structures been established to achieve high-quality examination? (2) Have structures been established to achieve high-quality examination?

(d) Examination implementation system

To evaluate the form of organization that is in charge of examination, the number of examiners, etc., and to confirm whether or not to establish the world’s highest level of implementation system of examination, while efficiently conducting the required number of examination cases.

The implementation system and the implementation status of examination, a comparison with other countries, etc.

While efficiently conducting the required number of examination cases, the JPO has established the world’ highest level of organizational structure for examination and personnel deployment.

While efficiently conducting the required number of examination cases, the JPO has established internationally comparable level of organizational structure for examination and personnel deployment.

While efficiently conducting the required number of examination cases, the JPO has generally established internationally comparable level of organizational structure for examination and personnel deployment.

The JPO has not established internationally comparable level of organizational structure for examination and personnel deployment.

3

Evaluation Items and Criteria Concerning Examination Quality Management Appendix

Items Objectives and Perspectives Examples for

evaluation materials

Examples of evaluation methods/ evaluation criteria

Very satisfactory Satisfactory Generally achieved Requiring

Improvements

(e) Quality management system

To evaluate the form of organization that is in charge of quality management, the number of staff responsible for quality management, etc., and to confirm whether or not to establish the efficient and effective, as well as the world’s highest level of quality management system.

The quality management system, a comparison with other countries, etc.

At the world’s highest level, initiatives for the quality management system have been efficiently and effectively planned, as well as the organizational structure and personnel deployment to implement such initiatives have been established.

At the internationally comparable level, initiatives for the quality management system have been efficiently and effectively planned, as well as the organizational structure and personnel deployment to implement such initiatives have been established.

At the internationally comparable level, initiatives for the quality management system have been efficiently and effectively planned, as well as the organizational structure and personnel deployment to implement such initiatives have been generally established.

At the internationally comparable level, initiatives for the quality management system neither have been efficiently and effectively planned, nor have the organizational structure and personnel deployment to implement such initiatives been established.

4

Evaluation Items and Criteria Concerning Examination Quality Management Appendix

Items Objectives and Perspectives Examples for

evaluation materials

Examples of evaluation methods/ evaluation criteria

Very satisfactory Satisfactory Generally achieved Requiring

Improvements II. Has the quality management been implemented according to policies and procedures? (1) Has the quality management been appropriately implemented?

(f) Initiatives for quality improvement

To evaluate whether initiatives necessary for the improvement of examination quality have been planned, and specifically how and to what degree such initiatives have been implemented according to policies and procedures, and confirm whether the objectives of the initiatives have been achieved.

The status of checks of notices of reasons for refusal, etc. for quality assurance, the status of examiner consultations, quantitative data such as the number of interviews, etc.

Initiatives necessary for the improvement of quality have been planned and implemented as planned, and the objectives of the initiatives have been achieved, having effects that contribute to further improvement of quality.

Initiatives necessary for the improvement of quality have been planned and implemented as planned, and the objectives of the initiatives have been achieved.

Initiatives necessary for the improvement of quality have been planned and implemented mostly as planned.

Initiatives necessary for the improvement of quality have not been planned, or even if planned, they have not been implemented as planned.

(g) Initiatives for quality verification

To evaluate whether initiatives necessary for the verification of examination quality have been planned, and specifically how and to what degree such initiatives have been implemented according to policies and procedures, and to confirm whether the objectives of such initiatives have been achieved.

The status of initiatives, including quality audits (sampling checks), user satisfaction surveys, and confirming discrepancy in judgment between examination decision and appeal/trial decision, quantitative data obtained from the results of such initiatives, etc.

Initiatives necessary for the verification of quality have been planned and implemented as planned, and the objectives of the initiatives have been achieved, having effects that contribute to further improvement of quality.

Initiatives necessary for the verification of quality have been planned and implemented as planned, and the objectives of the initiatives have been achieved.

Initiatives necessary for the verification of quality have been planned and implemented mostly as planned.

Initiatives necessary for the verification of quality have not been planned, or even if planned, they have not been implemented as planned.

5

Evaluation Items and Criteria Concerning Examination Quality Management Appendix

Items Objectives and Perspectives Examples for

evaluation materials

Examples of evaluation methods/ evaluation criteria

Very satisfactory Satisfactory Generally achieved Requiring

Improvements

(h)

Examination quality analysis and identification of issues

To evaluate specifically how examination quality has been analyzed and what kind of issues have been identified based on the results of the analysis, and to confirm whether the methods of analysis and the identification of issues have been appropriate.

The methods and results of analysis, and identified issues, etc. concerning quality of searches, quality of judgements in examinations, quality of descriptive content in notices of reasons for refusal, etc.

Analysis of examination quality and identification of issues have been conducted sufficiently and from a comprehensive perspective.

Analysis of examination quality and identification of issues have been conducted sufficiently.

Analysis of examination quality and identification of issues have been generally conducted.

Analysis of examination quality and identification of issues have not been conducted.

6

Evaluation Items and Criteria Concerning Examination Quality Management Appendix

Items Objectives and Perspectives Examples for

evaluation materials

Examples of evaluation methods/ evaluation criteria

Very satisfactory Satisfactory Generally achieved Requiring

Improvements II. Has the quality management been implemented according to policies and procedures? (2) Has continuous improvement been appropriately implemented?

(i)

Status of improvement of policies, procedures, and structures to achieve high-quality examination (evaluation items from (a) to (e))

To evaluate whether improvement has been specifically made on evaluation items from (a) to (e), and to confirm whether the status of improvement has been appropriate.

The status of revising the Quality Manuals , the implementation system of examination, the quality management system, etc.

Improvement in policies, procedures, and structures has been sufficiently made at an excellent level.

Improvement in policies, procedures, and structures has been sufficiently made.

Improvements in policies, procedures, and systems have been generally made.

Improvement in policies, procedures, and structures has not been made.

(j)

Status of improvement of quality management initiatives (evaluation items from (f) to (h))

To evaluate whether improvement has been made on evaluation items from (f) to (h), and to confirm whether the status of improvement has been appropriate.

The correlative relationship between analysis of examination quality/ identification of issues, and the improvement status of quality management initiatives

Improvement in quality management initiatives has been sufficiently conducted at an excellent level.

Improvement in quality management initiatives has been sufficiently conducted.

Improvement in quality management initiatives has been generally conducted.

Improvement in quality management initiatives has not been conducted.

III. Has information on initiatives for examination quality improvement been communicated?

(k)

Communication of information on initiatives for examination quality improvement

To evaluate whether information on initiatives for examination quality improvement has been appropriately communicated, and to confirm whether the JPO’s quality management has been well understood inside and outside Japan, efforts have been made to increase the presence of the JPO in the field of quality management, and as a result the trust has been gained.

The status of communication of information on initiatives for examination quality improvement, the status of meetings with overseas IP offices, etc. and the dispatch and acceptance of examiners, the status of PPH usage, etc.

Information on initiatives for examination quality improvement has been ambitiously communicated inside and outside Japan, and continuous cooperative relations with organizations and bodies inside and outside Japan have been built up.

Information on initiatives for examination quality improvement has been communicated inside and outside Japan, and cooperative relations with organizations and bodies inside and outside Japan have been built up.

Information on initiatives for examination quality improvement has been communicated inside and outside Japan.

Information on initiatives for examination quality improvement has not been communicated outside Japan.

7