trademark infringement recognition assistance system based
TRANSCRIPT
Trademark Infringement Recognition AssistanceSystem based on Human Visual Gestalt Psychologyand Trademark DesignKuo-Ming Hung ( [email protected] )
Kainan University https://orcid.org/0000-0002-9174-5256Li-Ming Chen
Tamkang UniversityTing-Wen Chen
Tamkang University
Research
Keywords: trademark, Gestalt psychology, trademark infringement
Posted Date: February 8th, 2021
DOI: https://doi.org/10.21203/rs.3.rs-174352/v1
License: This work is licensed under a Creative Commons Attribution 4.0 International License. Read Full License
Version of Record: A version of this preprint was published at EURASIP Journal on Image and VideoProcessing on July 22nd, 2021. See the published version at https://doi.org/10.1186/s13640-021-00566-2.
Hung et al.
RESEARCH
Trademark infringement recognition assistancesystem based on human visual Gestalt psychologyand trademark designKuo-Ming Hung1*†, Li-Ming Chen2 and Ting-Wen Chen2
*Correspondence:
[email protected] of Information
Management, Kainan University,
No.1 Kainan Rd., Luzhu Dist.,
33857 Taoyuan, Taiwan (R.O.C.)
Full list of author information is
available at the end of the article†Equal contributor
Abstract
Trademarks are common graphic signs in human society. People used this kind of
graphic sign to distinguish the signs of representative significance such as
individuals, organizations, countries, and groups. Under effective use, these
graphic signs can bring maintenance and development resources and profits to
the owner. In addition to maintenance and development, organizations that have
obtained resources can further promote national and social progress. However,
the benefits of these resources have also attracted the attention of unfair
competitors. By imitating counterfeit trademarks that appear, unfair competitors
can steal the resources of the original trademark. In order to prevent such acts of
unfair competitors, the state has formulated laws to protect trademarks. In the
past, there have also been researches on similar trademark searches to assist in
trademark protection. Although the original trademark is protected by national
laws, unfair competitors have recently used psychological methods to counterfeit
the original trademark and steal its resources. Trademarks counterfeited through
psychology have the characteristics of confuse consumers and do not constitute
infringement under the law. Under the influence of such counterfeit trademarks,
the original trademark is still not well protected. In order to effectively prevent
such trademark counterfeiting through psychology, this article proposes new
features based on trademark design and Gestalt psychology to assist legal
judgments. These features correspond to a part of the process that is not fully
understood in the human visual system and quantify them. In the experimental
results, we used past cases to analyze the proposed assistance system.
Discussions based on past judgments proved that the quantitative results of the
proposed system are similar to the plaintiff or the judgment to determine the
reasons for plagiarism. This result shows that the assistance system proposed in
this article can provide visually effective quantitative data, assist the law to
prevent malicious plagiarism on images by unfair competitors, and reduce the
plagiarism caused by the similar design concepts of late trademark designers.
Keywords: trademark; Gestalt psychology; trademark infringement
Introduction
Trademarks are graphics that can be seen everywhere in human society. People use it
as a recognition mark to distinguish countries, groups, and other representative signs
that represent individuals or organizations [1]. These trademarks usually consist of
various well-known entities or abstract graphics and symbols. In addition, they
may also be hearing, smell, behavior, or graphics with distinctive features [2]. The
origin of the trademark is that ancient craftsmen signed or marked their artwork
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
Hung et al. Page 2 of 15
or practical products to recognition the producer or guarantee the quality of the
product. These signatures and marks have evolved to this day and become today’s
trademark registration and protection system [3]. Trademarks have the following
three common functions [4]:
• Recognition source or ownership
• Ensure that the goods or services have the same level of quality or character-
istics
• Advertising
After effectively performing the above functions, ordinary people can quickly un-
derstand the image and reputation of the organization or product through the
trademark, and the organization with the trademark can also obtain the resources
to maintain and develop the organization itself.
Under the condition of effective use of trademarks, it can obtain maintenance
and development resources for organizations that own trademarks. However, these
resources have also drew the attention of unfair competitors. Unfair competitors
imitate similar trademarks to confuse people by its similarity. When people are
confused, the resources that originally belong to the trademark owner will be stolen
by unfair competitors. Moreover, unfair competitors damage the reputation of the
trademark through inferior products or services. These conditions have led to the
effect that trademark owners are unable to obtain sufficient resources to maintain or
develop the organization. At the same time, it further prevents organizations from
developing new technologies that contribute to the country and society. In order
to advance the technology level of society and protect the right of the trademark
owner, many countries allow trademark owners to register their trademarks. This
method allows registered trademarks to be protected by national laws and prevents
unfair competitors from indirectly preventing national progress.
While unfair competitors intend to steal the resources of the trademark organiza-
tion, they will imitate similar trademarks to deceive people, steal the resources that
the organization deserves, and damage the image and reputation of the trademark.
Assuming that the behavior of unfair competitors is not prevented, the organization
that owns the trademark would lost sufficient resources to maintain and develop it-
self. Furthermore, such behavior prevents the organization from developing novel
technologies that contribute to the progress of the country and society. Therefore,
many countries allow trademark-owned organizations to register their trademarks.
After legitimate registration, trademark owners can claim their rights protected by
national laws and prevents unfair competitors from stealing resources and profits.
However, even though registered trademarks seem already to be protected by law,
there are still endless cases of trademark infringement in the world. In these cases, in
addition to malicious infringement by unfair competitors, there are also some late-
comers who violated the law since they are unfamiliar with local legislation. There-
fore, the way of related work in the past was to search for similar features from a
huge trademark database to check whether the trademark constitutes infringement
for research purposes. In related works of searching for trademarks using digital
images, a common method was to extract features from the high-frequency images,
and then find the nearest trademark through the distance between the features
[5, 6]. In addition, there are also methods to directly use the binarized image as a
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
Hung et al. Page 3 of 15
[width = 3.5cm]fig01a.pdf
(a)
[width = 3.5cm]fig01b.pdf
(b)
[width = 3.5cm]fig01c.pdf
(c)
Figure 1 Illustrate a hypothetical trademark case that uses visual psychology to confuse humanvision. (a) Hypothetical well-known trademark; (b) Hypothetical counterfeit trademark; (c)Examples of possible confusion in environmental impact;
feature to perform image matching search [7]. Meenalochini et al. [8] proposed the
use of perceptual hashing algorithm to search for similar images. Showkatramani et
al. [9] utilized the recently popular Convolutional Neural Network (CNN) to train
a variety of trademarks in different environments, and implemented the computer
vision trademark identification system of trademarks in different environments. Dif-
ferent from the use of images in trademark recognition, such as F. M. Anuar et al.
[10, 11] proposed to use Tversky’s theory of similarity to search for the similarity
between semantic meanings based on the semantic combination in trademarks. H.
H. N. Abadi et al. [12] used artificial intelligence to classify trademarks registered
by the US Trademark Office.
The related work described above has significant results in similar trademark in-
spections and classifications. Nevertheless, unfair competitors recently started to
produce similar trademarks through Gestalt psychology. This kind of similar trade-
mark not only confuses people with the original registered trademark, but also
avoids the similar characteristics of the trademark, which make it difficult to judge
whether they are plagiarism in related works. Related reports are as described in
[13, 14] literature, this article uses Figure 1 as an example. Suppose there is a well-
known trademark as shown in Figure 1(a), and an unfair competitor wants to use
the reputation of the well-known trademark to draw people’s attention. Consider-
ing the basis of legal judgment, unfair competitors refer to the existing cognition of
Gestalt psychology and combine their own trademarks with well-known trademarks
to produce infringing trademarks such as Figure 1(b). Under normal circumstances,
people can easily distinguish the difference between the two trademarks. However,
as shown in Figure 1(c), when the distance in the human visual system and the
influence of the surrounding environment are considered, people will confuse the
two trademarks due to the change of characteristics. Just when people are confused
due to their existing cognition, infringing trademarks have already robbed benefits
from well-known trademarks, and at the same time damaged the reputation and
image of well-known trademarks.
In the past actual cases [15], since there was no suitable image analysis system to
assist in legal judgments, most cases were judged based on the semantic meaning of
trademarks. Also take Figure 1 as an example. In semantic analysis, the trapezoid
is a general figure and cannot be used as a basis for recognition. And the symbols
appearing in the two trademarks cannot be regarded as similar trademarks since
they are “Gesture” and “Glasses” respectively. Although in this case we had an
objective and fair semantic judgment, we ignored that the judgement did not con-
sider two trademarks from aspect of human visual system. This result has enabled
unfair competitors in recent years to use Gestalt psychology to steal the reputa-
tion of the original trademark. Some consumers even misunderstood the counterfeit
trademarks of unfair competitors as original trademarks.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
Hung et al. Page 4 of 15
Considering the influence of Gestalt psychology on trademark reputation, this
paper refers to and attempts to implement a system that includes elements of
attracting people in trademark design [16, 17]. In trademark design, these attractive
elements are called “visual weight” [16, 17], and the details will be introduced in
section 2. Based on the elements mentioned in trademark design, this article realizes
and extracts corresponding features in trademark image processing. We discuss the
impact of these characteristics in the experimental results section, and use past
cases to discuss the correctness of the assistance system.
The main contributions of this article are as follows:
• According to trademark design and Gestalt psychology, a new seven-character
visual judgment aid system is proposed. These features correspond to a part
of the process that is not fully understood in the human visual system and
quantify them.
• Discuss the cases that have been finalized in the past based on the exper-
imental results, and discuss the differences before and after considering the
similarity of visual images.
• Provide a visual-based trademark judgment system to assist the law to prevent
malicious plagiarism on images by unfair competitors, and reduce plagiarism
due to similar design concepts by less advanced trademark designers.
In the next chapter, we introduce trademark design and visual psychology. Subse-
quently, the “visual weight” element in the implementation of the trademark design
is explained, and the system flowchart proposed in this article is explained. In the
experimental results, we use some cases to verify the system proposed in this arti-
cle. When verifying, we explain the relationship between existing trademark search
methods, trademark design and Gestalt psychology to explain why the existing
methods cannot assist legal judgments. Finally, we will add the actual judgments of
some cases to this system for discussion, explaining that visual image and semantic
analysis are equally important factors in judging trademark similarity. At the same
time, it explains that the system proposed in this article can help latecomers reduce
plagiarism caused by similar design concepts.
Method
Trademark Design and Gestalt psychology
In the trademark design, S. Bradley [16, 17] proposed an influential element named
“visual weight” based on human vision. The main purpose of ”visual weight” is
to make the designed trademark have characteristics that can draw human atten-
tion and represent the image of the organization or product. The original features
mentioned by S. Bradley in “visual weight” are as follows:
• Size:
For human attention, large elements are more attractive than small elements.
• Shape: Since the impression of irregular shapes is easily replaced by regu-
lar shapes, objects with regular shapes are more impressive than those with
irregular shapes.
• Color:
Warm-colored objects are easily recognized as the foreground, while cool-
colored objects are easy to blend with the background. In addition, red is
considered the darkest color, and yellow is considered the lightest color.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
Hung et al. Page 5 of 15
• Value:
Dark elements are easier to notice than light elements.
• Texture:
Texture makes the element look like a three-dimensional object, giving the
appearance of mass and physical weight. Therefore, textured elements draw
human attention more than untextured objects.
• Position:
The foreground object is easier to become the main area of the logo than the
background object. The farther an element is from the main area, the easier
it will be noticed.
• Orientation:
In the display direction of the object, diagonal display with diagonal elements
can give people the strongest impression, followed by vertical display, and
horizontal display gives the lowest impression.
Gestalt psychologist K. Koffka explained the theory of human visual perception
through the proposed perceptual organization [18]. K. Koffka believes that when
everyone recognizes the images they see, including children and uncivilized people,
they recognize the meaning of things in front of them based on organizational
experience. In perceptual organization, the following points describe human visual
perception:
• The stronger the difference between the foreground graphic object and the
background, the more it can become the focus of human vision.
• Close graphic objects can easily form a whole.
• Coordinated graphic objects are easy to form a whole, and uncoordinated
graphic objects are easy to be separated.
• When the distance between each part is equal, the graphic objects of the same
color will form a whole.
• According to people’s existing cognition, even if it changes drastically, it can
be recognized from its characteristics.
• If the graphic objects have directionality, graphic objects with the same di-
rectionality can easily form a whole.
Proposed Trademark Graphic Recognition Features and Trademark Similarity
Comparison System based on Visual Weight
From the description of the trademark design in the previous section, we can learn
that how to draw human attention through vision, and to associate its image and
reputation, has always been the purpose of a trademark. However, in the case of
trademark judgments, due to the lack of objective image judgment methods, the
judgments have been implemented through semantic meaning. This approach gives
unfair competitors an opportunity to constantly use similar graphics but different
semantics to counterfeit well-known trademarks as shown in Figure 1. Due to the
different semantics of counterfeit trademarks, trademarks cannot be protected by
national laws. Visually, counterfeit trademarks confuse consumers and further steal
the resources and profits of well-known trademarks. In order to more reliably pro-
tect the image and reputation of trademarks, there must be a system that can assist
in the judgment of trademarks on graphics. The graphic judgment system is not
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
Hung et al. Page 6 of 15
to replace semantic judgment, but to jointly deal with the problem of counterfeit
trademarks to achieve the effect of trademark protection. Therefore, based on the
visual perception theory and visual weight characteristics described in the previ-
ous section, this article proposes a system to assist in the judgment of trademark
graphics.
Proposed Trademark Graphic Recognition Features based on Visual Weight
The first feature mentioned by visual weight is size. Using human vision to differ-
entiate the size characteristics of the two trademarks is to compare the difference
in size between the two. In image processing, the simplest and similar processing
method is to take out the trademark that separates the background and calculate
the difference between the two pixels. Since trademarks have different scales in each
layout, this article uses all pixels in the region of interest (ROI) as the normaliza-
tion parameter NSIZ of the feature. Considering that most images nowadays use
lossy compressed images for storage, we use adaptive threshold [19] processing in
the work of separating foreground and background. After obtaining the best fore-
ground distance through basic image processing, the system would count its pixels
to obtain the trademark pixel TSIZ . Through the normalization of formula (1), the
normalized trademark size feature F1 is obtained.
F1 =TSIZ
NSIZ
(1)
The second feature is shape. Also considering the changes of trademark images in
various layouts, the shape feature does not use the perimeter or binarized area of
the trademark object. This article extracts the shape feature F2 using circularity for
quantification to reduce the impact of layout changes. When the Circularity formula
calculates a graphic object, it uses the area of the graphic object for quantitative
calculation, so that the circle and the hollow ring present different values. Due to
trademark objects, even if there is a gap between the graphic objects, each object is
designed to have a certain degree of coordination. In the description of perceptual
organization, these coordinated graphic objects make the trademark appear as a
whole in human vision. Therefore, when we use the circularity formula to quantify
the shape feature, we use the area of the trademark object as the variable for
the quantitative calculation according to the previous description. The circularity
calculation method is shown in formula (2), TP is the circumference of the trademark
object.
F2 =4πTSIZ
(TP )2(2)
The third feature is color. When extracting the characteristics of color, in addition
to the knowledge of visual perception theory and trademark design, the deactivation
reaction of human vision should still be considered [20]. The visual deactivation
reaction is that humans produce optical illusion such as complementary colors and
persistence of vision based on the density or color difference between the foreground
and background of the object. Considering the visual perception theory, trademark
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
Hung et al. Page 7 of 15
design and visual deactivation, the color feature F3 is extracted as formula (3) in
this article. Where Tr,c represent the pixels in the processing trademark, RTM and
CTM are the rows and columns of the trademark object, respectively, and ch means
the image channels.
F3 =
√
√
√
√
∑
ch
(
∑RTM
r
∑CTM
c (Tr,c)
RTM × CTM
)2
(3)
The fourth characteristic is value. In the trademark design, value pointed out that
darker elements are easier to be noticed than lighter elements. The main reason is
that lighter elements give humans a lower sense of oppression than dark-colored
elements, so that the elements in human society have more light-colored elements
[17]. Among the many lighter elements, embedding an object with a darker element
will make the object easy to be noticed. This kind of strongly discontinuous picture
is called contrast in image processing. In general trademark design, in order to
increase the chance of the trademark being noticed by people, the contrast between
the background and the foreground is quite significant. Therefore, the feature F4 of
the extracted value in this article focuses on the contrast of the foreground, and the
calculation method is shown in formulas (4) and (5). Where the ∗ symbol in formula
(4) is the convolution operation, and Tmsk is the coordinate of the trademark pixel.
Formula (5) is the common gray-scale processing, and Pg is the defined gray-scale
parameter [20].
F4 =maxr,c(Gr,c ∗ Tmsk)−minr,c(Gr,c ∗ Tmsk)
maxr,c(Gr,c ∗ Tmsk) + minr,c(Gr,c ∗ Tmsk)(4)
Gr,c =∑
ch
(Tr,c(ch)× Pg(ch)) , Pg = [0.299, 0.587, 0.144] (5)
The fifth feature is texture. In the description of visual weight, the texture fea-
ture combines trademarks with common things in human life to increase attrac-
tiveness. Common things include elements such as light sources, shadows, and
three-dimensional perspectives, which make the logo graphics on the plane feel like
three-dimensional objects. When the same figure is combined with different texture
features, visual perception can not only recognize the figure, but also the type of
combined texture features. From the perspective of the frequency domain of the sig-
nal system, the texture feature only changes the high-frequency information of the
graphics. The original meaning of graphics at low frequencies has not been changed
by texture features. In image processing, the basic feature of object recognition is
to extract the corresponding high-frequency features from the object image. This
feature is also considered in the related literature based on image processing in the
related works [5, 6]. Therefore, the extraction method of texture feature F5 in this
article uses the image high-frequency feature extraction method.
Position is the penultimate feature. S. Bradley described the position feature as a
foreground object that easily becomes the main area of a trademark. Moreover, he
also pointed out that when some elements are far away from the main area, these
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
Hung et al. Page 8 of 15
[width = 8.5cm]fig02.pdf
Figure 2 Proposed the implementation flow chart of proposed trademark similarity comparisonsystem
elements will become the visual focus. In this part of this article, we use formulas (6)
and (7) to find the centroid (C ′
xi, C′
yi) of each object. Then, combine the centroid
of each object through formula (8). Finally, the position feature F6 is normalized
and calculated by formula (9). Where, i is the index of objects; Sy(x) is the length
of the y-axis at coordinate x; Sx(y) is the length of the x-axis at coordinate y; and
OA is the area of the object.
C ′
xi =
∫
xSy(x)dx
OAi
(6)
C ′
yi =
∫
ySx(y)dy
OAi
(7)
(C ′
xi, C′
yi) =
(∑
C ′
xiOAi
TSIZ
,C ′
yiOAi
TSIZ
)
(8)
F6 =
√
√
√
√
∑
a∈(x,y)
(
C ′a
αTM
)
,a = x, α = C
a = y, α = R(9)
The last feature is orientation. Visual weight means that when a graphic is com-
posed of multiple objects, objects with the same orientation are easily regarded as
the same object by visual perception, and objects with different orientations are
regarded as different objects. In this article, we use an ellipse to approximate the
irregular shape of the trademark. Then, the angle between the main axis of the
ellipse and the horizontal line is taken as the original orientation feature. Finally,
the original orientation feature is normalized to [0, 1] as the trademark orientation
feature F7.
Trademark Similarity Comparison System based on Visual Weight
After introducing the recognition features proposed based on visual weight, we
introduce the proposed trademark similarity comparison system. Figure 2 shows
the flow chart of the proposed system. First, the original and counterfeit trademark
image data input based on actual cases. From the input image data, the seven
normalized features of size, shape, color, value, texture, position and orientation
are extracted according to the description of the previous visual weight. From the
features extracted from the case images, use formulas (10) and (11) to calculate the
standard deviation σk. Where k represents the characteristic index, oi is sample
index.
Dk = ‖Fk − F̄k‖2 (10)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
Hung et al. Page 9 of 15
σk =
√
√
√
√
1
M
M∑
oi=1
(Dk − µk)2 (11)
Next, we use formulas (12) and (13) to calculate the feature quantity threshold
Tσ that takes into account human visual errors. After obtaining the characteristic
quantity valve value Tσ, calculate the valve value T according to formulas (14)
to (16). In the formula, Mo(·) represents the mode; Q1(·) is the first interquartile
difference; wk is the weight of the feature. The finally obtained valve value T is the
valve value that assists the system to judge.
Tσ = Mo(Fk) (12)
Fk = |Dk < σk|0, ∀ k (13)
T = Q1(RN ) (14)
RN = 1− ω +∑
k
Dkwk (15)
ω = Fk ≥ Tσ (16)
Since in the human visual system [21], there are still biological and psychological
processes that are not yet fully understood. Whether counterfeit trademarks cause
human confusion is one of the parts we could discuss. At the same time, when hu-
mans recognize images through vision, if some of the features of the image are not
obvious, the human brain will ignore the features and turn to use other organiza-
tional experience to recognize available features for identification [18]. Taking these
factors into account, this article will further discuss the seven characteristics results
in the experimental results, and discuss it in conjunction with the previous known
cases.
Experimental Results and Discussion
This implementation uses Graphis public trademark database [22], Trademark
database of the Japan Platform for patent (JPP) [23] and Trademark database
of the United States Patent and Trademark Office (USPTO) [24]. The Graphis
public trademark database has 5906 trademark images. There are 5,450 JPEG im-
ages with a size of 670 × 670 pixels in the database, 254 images with a size less than
670 × 670 pixels, and 202 images with a size greater than 670 × 670 pixels. The
trademark database of USPTO is a trademark database containing all trademarks
registered in the United States.
Although there are still many cases related to trademark infringement in society
today, it is still difficult to collect trademark images related to the case. Taking the
difficulty of image search in actual cases into account, the counterfeit trademark
images used in this article are mostly counterfeit trademarks similar to the original
trademarks registered by the original trademark owners in advance. Such counterfeit
trademarks are the mark that original trademark owners believe that they have the
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
Hung et al. Page 10 of 15
opportunity to confuse customers and further damage the reputation of the original
trademark. Therefore, trademark owners first registered these numerous counterfeit
trademarks to prevent unfair competitors from using similar methods to damage
the reputation of the original trademarks.
Figure 3 is the statistics of the number of features that use pre-registered similar
trademarks and different trademarks at a distance less than the standard deviation.
There are 391 groups of pre-registered similar trademarks in the picture, and 363
groups of different trademarks. It can be seen from Figure 3 that among the seven
features in similar trademarks, most of them have four or five feature distances
that are less than the standard deviation. When the trademarks are not the same,
there are usually two to three features whose distances are less than the standard
deviation. Taking into account the error of the simulated human vision, the mode
Mo(·) is used to identify the feature quantity threshold Tσ of similar trademarks.
Figure 4 shows the statistics of each feature whose distance is less than the stan-
dard deviation. In the figure, we can notice that the characteristic of F5, whether it
is a similar trademark or not, are not below the standard deviation. This shows that
compared with the past research using F5 feature, the search and recognition of digi-
tal trademark features are quite effective. F5 feature can clearly distinguish different
trademarks, and accurately search for similar trademarks by feature distance. Since
the F5 feature is quite accurate, it cannot directly describe the confusion caused by
the human visual system seeing the trademark. On the standpoint that trademarks
are designed to attract consumers, this advantage has made it impossible for past
research to be applied to assist the law and the recognition of similar trademarks.
In addition, Figure 4 allows us to further analyze the results of Figure 3. From
Figure 4 we can notice the different trademark’s features in F3, F4 and F7, and the
feature distance is still easily smaller than the standard deviation. The F3 feature
is mainly attributed to color. If features of different shapes have the same color,
the feature distance may be lower than the standard deviation. The F4 feature
is mainly the contrast on the trademark. Most trademarks have the main goal of
attracting human attention, and the mention of strong contrast in “visual weight”
makes the trademark easy to attract attention. Therefore, the F4 feature is indeed
easily smaller than the standard deviation in terms of distance comparison. The
orientation of F7 features mainly depends on the ellipse encircled by the overall
scope of the trademark. When the ellipse circled by the trademark arrangement
position is similar, the calculated angle will be quite similar, resulting in the distance
being easily smaller than the standard deviation.
Although the features of F3, F4 and F7 are easily misjudgmental, when analyzing
trademark graphics in combination of features, they will become a key feature that
makes human vision confused due to trademark similarity. Next, we will explain
through actual case analysis.
In case [25], the proposed system only has a significant distance at F4 and F5,
while other features are less than σk. The results of Table 1 show that it is as seen
[width = 8.5cm]fig03.pdf
Figure 3 Statistics of the number of features whose distance is less than the standard deviation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
Hung et al. Page 11 of 15
[width = 8.5cm]fig04.pdf
Figure 4 Statistics where the distance of each feature is less than the standard deviation
Table 1 Past trademark litigation cases were evaluated using the proposed image similarity resolutionsystem
[25] [26] [27] [15] case 1 [15] case 2F1 0.0116 0.0908 0.2133 0.0830 0.1051
F2 0.0018 0.6443 0.1852 0.0103 0.0083F3 0.0769 0.0002 0.1816 0.0701 0.1057
F4 0.2532 0.3801 0.0000 0.0000 0.0000F5 0.0942 0.1089 0.1054 0.0824 0.0881F6 0.0096 0.0277 0.0120 0.0132 0.0040F7 0.0004 0.0073 0.00110 0.0005 0.0014
Verdict Infringement Dismiss Dismiss Dismiss DismissProposed output Similar Dissimilar Dissimilar Similar DissimilarItalic: distance < 2σk; Bolder: distance < σk;
by human vision, that is, the two can be seen differently by careful observation, but
other characteristics are quite similar, which is easy to cause confusion. The results
of past judgments also indicated that the defendant’s trademark was judged in favor
of the plaintiff due to it “may dilute the uniqueness of the plaintiff’s trademark.”
Case [26] is a recent case where the plaintiff was sentenced to lose. The results of
Table 1 show that the proposed system is similar in features of F3 and F6. However,
there is a huge gap in the characteristics of F2, which is like showing that the shape
of the plaintiff and the defendant’s trademark is not easy to confuse people. In the
actual judgment, the plaintiff in this case tended to file a complaint with similar
meaning. The plaintiff believed that the meaning of the trademark “caught from
the sea” was similar to the meaning of the defendant’s trademark “caught from the
river.” Nevertheless, the judgment found that “sea” and “river” have a specific con-
nection, but it will not confuse people. At the same time, after confirming that other
items such as merchandise items and trademark signs would not confuse people, the
final judgment indicated that the defendant did not constitute an infringement.
Case [27] is a case where the plaintiff was judged to lose the case many years
ago. In this case, the plaintiff used the overall shape and color of the trademark
design and imitation of some objects as the reasons for filing a lawsuit. The result
of the judgment shows that the design of double concentric circles is a universal
design figure and is less recognizable. As for the corrugated pattern in the trade-
mark, the overall graphic of the plaintiff’s trademark represents long hair, but the
overall graphic of the defendant’s trademark represents heat. Although the color
difference was not described, the overall appearance, pronunciation, and semantics
were completely different, which made the final judgment believe that there is no
doubt of confusion. In the analysis results of Table 1 in this article, based on the
similarities described by the plaintiff, the corresponding features are F2, F3 and F5.
Feature F2 calculating the distance shows that although the whole object is visually
double concentric circles, the pattern of the inner circle is different, and the area
of the whole pattern makes the shape calculation result obviously different. From
the analysis of feature F3, the plaintiff’s trademark is green and black, while the
defendant is pure green. Normal people’s vision can easily detect the differences,
just as the distance calculated from feature F3 is greater than 2σk. The calculation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
Hung et al. Page 12 of 15
result of feature F5 is also greater than 2σk, indicating that the overall F5 difference
between the two is far. The calculation results of the above three characteristics are
in line with the results of human visual perception observing the trademarks of the
two parties.
Case 1 and case 2 in [15] are also cases where the plaintiff lost. In case 1, the plain-
tiff believed that the defendant’s trademark was similar in shape, color and design
to the trademark owned by it, which violated the trademark rights. However, the
verdict at that time indicated that “star” was a general term, with weak discerni-
bility, and could not be used as a recognition criterion. “preya” and “bucks” have
no semantic meaning. Pronunciation, “Starpreya” is pronounced as “Star Freya”,
which is not similar in syllable to “Starbucks”. Moreover, the semantic meanings
of the patterns in the trademarks were “Sirens-mermaids” and “Goddesses”, so
that the final judgment was not considered to be similar trademarks. Similarly, the
shape F2, color matching F3 and design pattern texture feature F5 recognized by
the plaintiff were used to analyze the state of human attention to the trademark.
When only looking at the shape feature F2, the calculation result is less than σk.
However, it can be known from the judgment statement that “the design of double
concentric circles is a universal design figure”, so the similarity cannot be recognized
by this feature alone. Different from the previous case, in the F3 feature, due to the
outer circle green and the inner circle black have the same color, the calculation
result also less than σk. F5 texture feature is one of the few features larger than 2σk
in this case. Although the verdict of this case was that the plaintiff lost the case, the
characteristics described above and other characteristics of the system are less than
2σk. The combination of these results shows that the two trademarks are confused
in human visual perception. However, the verdict was based on the difference be-
tween hearing and semantics, which lacked visual judgment. This verdict is similar
to Figure 1 in the introduction section. Without assistance in judging the pattern
design standards, relying only on hearing, semantics and other methods may enable
unfair competitors to steal resources through similar methods.
In case 2, the plaintiff also filed a lawsuit against the defendant on the grounds of
the shape and color of the trademark. The verdict in this case indicated that on the
inner circle pattern, the trademark owned by the plaintiff was “Sirens-mermaids”,
while the trademarks of the defendant were “mountain shape” and “Mt. RAINIER.”
The semantics of the two trademarks are also completely different, so that the final
judgment is not considered to be similar trademarks. Based on the reasons for
reporting, the system proposed in this article also first examines the features F2,
F3, and F5. The calculated result of feature F2 is less than σk, indicating that the
two trademarks do have similar shapes visually. Feature F5 also shows the texture,
the similarity is greater than 2σk. The difference from the previous case is the color
scheme of feature F3. The F3 feature distance calculated in the proposed system
is between 2σk and σk. In this case and the previous case, the color scheme is the
same as green and black, but because of the different area of the color scheme, the
human visual perception has a different perception of the combination of F2 and
F3, which further recognizes the difference between the two.
In the past systems used to recognition objects, the corresponding literature will
explain the use of specified methods to combine features, and then perform object
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
Hung et al. Page 13 of 15
identification based on the combined results. This article considers the visual per-
ception theory of trademark design and Gestalt psychology. In trademark design
and visual perception theories, they describe that when human vision recognizes
objects or images, unobvious features will be ignored, and other organizations’ ex-
perience considers available features for identification [18]. Suppose, as in the cases
of Table I [27], [15] case 1 and 2 combine the features through a specified method.
Among the seven features, the F4 feature is the most similar feature by mathemat-
ical calculation. However, in actual human vision, the F4 feature of these cases is
not considered as a reference feature compared to the other two cases. In addition,
in the similarities raised by the plaintiff in the case, not all features were raised
to have similarities. Therefore, in the biological and psychological process that is
not yet fully understood [21], this article quantifies and lists the characteristics of
trademark design that meet the observation of human visual perception, and pro-
vides quantitative data that simulates vision for design and legal decision-making
reference.
Based on the visual perception theory of trademark design and Gestalt psychology,
this article proposes seven features to assist judgment. Since these features are the
defining elements of trademark design to attract human attention, the purpose
of proposing features is to provide a basis for new trademarks or legal decisions
when quantifying human visual perception through image processing, rather than
making existing digital trademark search systems better. These features, which are
combined with the design code search on the USPTO website, can help designers
determine whether their new designs are similar to existing trademarks, and further
prevent problems caused by misuse of others’ trademarks. At the same time, in the
face of counterfeit trademarks of unfair competitors, it can also give quantitative
and visual judgments similar to human visual perception, helping national laws to
protect the reputation of the original trademark owners.
Conclusion
As individuals, organizations, groups, and other representative signs, trademarks are
common graphics in human society. The resources and profits brought by trade-
marks can maintain and develop the organization and further promote national
progress. However, these resources have attracted the attention of unfair competi-
tors. They faked similar trademarks and defrauded the confused consumers of the
resources that the original trademark owners should obtain. These stolen resources
may prevent the organization from maintaining or developing itself, so that the
country cannot progress. In order to promote national progress, most countries
enact trademark laws to protect organizations that own trademarks. Even though
trademarks are protected by law, there are still many infringement cases. Apart from
some latecomers who caused infringements because of their unfamiliarity, there were
also malicious plagiarism by unfair competitors. Therefore, most of the research on
trademarks is to search for similar features from a huge trademark database to check
whether the trademark constitutes plagiarism for research purposes. These studies
have had significant results on similar trademark inspections and classifications.
However, in recent years, unfair competitors have begun to produce similar trade-
marks through psychological means. Such similar trademarks not only confuse peo-
ple with the original registered trademarks, but also avoid the legal judgment of
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
Hung et al. Page 14 of 15
similar characteristics of the trademarks, making it difficult for law enforcement
officers to judge whether they are plagiarism. Considering the influence of Gestalt
psychology on trademark reputation, this article proposes a new seven-character
visual judgment assistance system based on trademark design and Gestalt psychol-
ogy. These features correspond to a part of the process that is not fully understood
in the human visual system and quantify them. In the experiment, the trademark
information used by the valve of this article to build the assistance system is Graphis
public trademark database and USPTO database. During the verification test, we
use the data in the USPTO database that is independent of the built-in valve value
and the trademark images of past actual cases for testing. In the test results, the
analysis of past cases shows that the system proposed in this article to assist in
the judgment of visual perception is consistent to a certain extent with the reasons
for the plaintiff’s litigation. This shows that the assistance system proposed in this
article can effectively provide visual-based trademark judgment results. Therefore,
the system proposed in this article can assist the law to prevent malicious plagiarism
on images by unfair competitors, and reduce the plagiarism caused by the similar
design concepts of late trademark designers.
List of abbreviations
• ROI: Region of interest. ROI are samples within a data set identified for a particular purpose.
• TM: Abbreviation of trademark.
• USPTO: Trademark database of the United States Patent and Trademark Office. The database website of
the registered trademark of the storage manufacturer in the United States.
• JPP: Trademark database of the Japan Platform for Patent. The database website of the registered
trademark of the storage manufacturer in Japan.
Availability of data and materials
The trademark database is available online, the URL is as follows:
https://www.graphis.com/logos/
https://www.uspto.gov/trademarks/
https://www.j-platpat.inpit.go.jp/
Competing interests
The authors declare that they have no competing interests.
Funding
This research received no external funding.
Author’s contributions
Conceptualization, K.M.H.; Methodology, K.M.H.; Investigation, K.M.H., L.M.C. and T.W.C.; Resources, L.M.C.
and T.W.C.; Writing–Original Draft, K.M.H., L.M.C. and T.W.C.; Writing–Review & Editing, K.M.H.; Funding
Acquisition, K.M.H.; Supervision, K.M.H.;
Acknowledgements
The authors would like to thank several anonymous reviewers and the editor for their comments and suggestions.
Author details1Department of Information Management, Kainan University, No.1 Kainan Rd., Luzhu Dist., 33857 Taoyuan,
Taiwan (R.O.C.). 2Department of Electrical and Computer Engineering, Tamkang University, No.151, Yingzhuan
Rd., Tamsui Dist., 25173 New Taipei, Taiwan (R.O.C.).
References
1. Garner, B.A.: Black’s Law Dictionary (11th Ed.). West Group, ??? (2019)
2. Taiwan Intellectual Property Office: Review criteria: risk of confusion and misidentification. Taiwan Intellectual
Property Office (2017).
https://www.tipo.gov.tw/public/Attachment/690b8217-d7da-48f5-9187-a7b9284cdbc0.pdf
3. Richardson, G.: Brand names before the industrial revolution. Working Paper 13930, National Bureau of
Economic Research (April 2008). doi:10.3386/w13930. http://www.nber.org/papers/w13930
4. Callmann, R.: The Law of Unfair Competition, Trademarks and Monopolies (4th Ed.). Callaghan, ??? (1998)
5. Nguyen, T.D., Nguyen, H.H.H., Le, T.H.: Trademark image retrieval based on scale, rotation, translation
invariant features. In: The 2013 RIVF International Conference on Computing Communication Technologies -
Research, Innovation, and Vision for Future (RIVF), pp. 282–285 (2013). doi:10.1109/RIVF.2013.6719908
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
Hung et al. Page 15 of 15
6. Agrawal, D., Jalal, A., Tripathi, R.: Trademark image retrieval by integrating shape with texture feature. In:
Proceedings of the 2013 International Conference on Information Systems and Computer Networks, ISCON
2013, pp. 30–33 (2013). doi:10.1109/ICISCON.2013.6524168
7. Joshi, K.D., Bhavsar, S.N., Sanghvi, R.C.: Image retrieval system using intuitive descriptors. Procedia
Technology 14, 535–542 (2014). doi:10.1016/j.protcy.2014.08.068
8. Meenalochini, M., Saranya, K., Rajkumar, G.V., Mahto, A.: Perceptual hashing for content based image
retrieval. In: 2018 3rd International Conference on Communication and Electronics Systems (ICCES), pp.
235–238 (2018). doi:10.1109/CESYS.2018.8723888
9. Showkatramani, G., Khatri, N., Landicho, A., Layog, D.: Deep learning approach to trademark international
class identification. In: 2019 18th IEEE International Conference On Machine Learning And Applications
(ICMLA), pp. 608–612 (2019). doi:10.1109/ICMLA.2019.00112
10. Anuar, F.M., Setchi, R., Lai, Y.-K.: A conceptual model of trademark retrieval based on conceptual similarity.
Procedia Computer Science 22, 450–459 (2013). doi:10.1016/j.procs.2013.09.123
11. Anuar, F.M., Setchi, R., Lai, Y.: Semantic retrieval of trademarks based on conceptual similarity. IEEE
Transactions on Systems, Man, and Cybernetics: Systems 46(2), 220–233 (2016).
doi:10.1109/TSMC.2015.2421878
12. Habibollahi Najaf Abadi, H., Pecht, M.: Artificial intelligence trends based on the patents granted by the united
states patent and trademark office. IEEE Access 8, 81633–81643 (2020). doi:10.1109/ACCESS.2020.2988815
13. Wu, Y.: Cheated! This shop is not the FamilyMart, Customers almost ran in to buy drinks. written by
Traditional Chinese (2016). http://bepo.ctitv.com.tw/2016/09/128615/
14. Chen, Q.J., Xu, Y.X.: Appearance similarity is as high as 90%, pet shop involved plagiarism PX Mart. written
by Traditional Chinese (2019). https://news.tvbs.com.tw/local/1169189
15. Kim, T.: Starbucks, A fierce battle to defend the logo (2019).
https://www.darts-ip.com/starbucks-logo-battle/
16. Bradley, S.: Design Principles: Visual Weight And Direction (2014).
https://www.smashingmagazine.com/2014/12/design-principles-visual-weight-direction/
17. Bradley, S.: Design Fundamentals - Elements, Attributes, & Principles. Vanseo Design, Boulder, Colorado
(2018)
18. Koffka, K.: Principles of Gestalt Psychology. Mimesis International, ??? (2014)
19. Bradley, D., Roth, G.: Adaptive thresholding using the integral image. Journal of Graphics Tools 12(2), 13–21
(2007)
20. Eagleman, D.: Visual illusions and neurobiology. Nat Rev Neurosci 2, 920–926 (2001)
21. Gonzalez, R.C., Wood, R.E.: Digital Image Processing, (4rd Ed.). Pearson Education, ??? (2009)
22. Pedersen, B.M.: Logo Database - Graphis. Graphis Inc. (1986). http://www.graphis.com/logos/
23. Japan Platform for Patent Information — J-PlatPat (JPP),. https://www.j-platpat.inpit.go.jp/
24. USPTO United States Patent and Trademark Office. https://www.uspto.gov/
25. 7-Eleven Wins Trademark Infringement Suit vs. Super-7 Store. Convenience Store News (2015).
https://csnews.com/7-eleven-wins-trademark-infringement-suit-vs-super-7-store
26. Li, Z.W.: Learned the lesson of being defeated to ”Hodilao” in the lawsuit, Haidilao registered 200 trademarks
in 2 days. udn.com. written by Traditional Chinese (2020).
https://udn.com/news/story/7332/4991502?from=udn-ch1 breaknews-1-0-news
27. Li, Y., Peng, G.: Trademark battle! Starbucks defeated in lawsuit against e-coffee. TVBS News. written by
Traditional Chinese (2007). http://www.graphis.com/logos/
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
Input
Input original and
infringed image
from cases
Extract visual weight
features
Using cases to
calculate threshold
scores
Obtain thresholdVerify model by
testing casesEnd
Figures
Figure 1
Illustrate a hypothetical trademark case that uses visual psychology to confuse human vision. (a)Hypothetical well-known trademark; (b) Hypothetical counterfeit trademark; (c) Examples of possibleconfusion in environmental impact;
Figure 2
Proposed the implementation flow chart of proposed trademark similarity comparison system
Figure 3
Statistics of the number of features whose distance is less than the standard deviation
Figure 4
Statistics where the distance of each feature is less than the standard deviation