the national do mi to attua unitateabba2u.com/ovod_paper/1_a0_patent_us9770163b2_170926.pdfa61b 1 /...
TRANSCRIPT
THE NATIONAL DO MI TO ATTUA UNITATE US009770163B2
( 12 ) United States Patent Ovod
( 10 ) Patent No . : US 9 , 770 , 163 B2 ( 45 ) Date of Patent : Sep . 26 , 2017
( 56 ) References Cited ( 54 ) METHOD AND APPARATUS FOR CONTROLLING LIGHT OUTPUT INTENSITY AND PROTECTION FROM HIGH INTENSITY LIGHT
U . S . PATENT DOCUMENTS 3 , 670 , 722 A 4 , 356 , 534 A ( 71 ) Applicant : Karl Storz Imaging , Inc . , Goleta , CA
( US )
6 / 1972 Kosaka 10 / 1982 Hattori
( Continued )
FOREIGN PATENT DOCUMENTS ( 72 ) Inventor : Vladimir I . Ovod , Goleta , CA ( US ) ( 73 ) Assignee : Karl Storz Imaging , Inc . , Goleta , CA
( US )
1358838 B1 62155689 A
2004033755 A
11 / 2003 7 / 1987 2 / 2004
( * ) Notice : OTHER PUBLICATIONS Subject to any disclaimer , the term of this patent is extended or adjusted under 35 U . S . C . 154 ( b ) by 261 days . “ A Robust Multistage Algorithm for Camera Self - Calibration Deal
ing With Varying Intrinsic Parameters ” Adnane El - Attar , Moham med Karim , Hamid Tairi , Silviu Ionita ; E - ISSN : 1817 - 31 .
( Continued ) ( 21 ) Appl . No . : 14 / 505 , 130 ( 22 ) Filed : Oct . 2 , 2014 ( 65 ) Prior Publication Data
US 2015 / 0116470 A1 Apr . 30 , 2015 Primary Examiner — Dramos I Kalapodas ( 74 ) Attorney , Agent , or Firm — Whitmyer IP Group LLC
( 63 ) Related U . S . Application Data
Continuation of application No . 13 / 181 , 350 , filed on Jul . 12 , 2011 , now Pat . No . 8 , 878 , 920 .
( 51 ) Int . CI . A61B 1 / 06 ( 2006 . 01 ) H04N 7 / 18 ( 2006 . 01 )
( Continued ) ( 52 ) U . S . CI .
CPC . A61B 1 / 06 ( 2013 . 01 ) ; A61B 1 / 04 ( 2013 . 01 ) ; A61B 1 / 042 ( 2013 . 01 ) ; A61B
1 / 0669 ( 2013 . 01 ) ; ( Continued )
( 58 ) Field of Classification Search CPC . . . . . . . . . . . A61B 1 / 06 ; A61B 1 / 04 ; A61B 1 / 0669 ;
A61B 1 / 042 ; A61B 1 / 00006 ; ( Continued )
( 7 ) ABSTRACT
A method and apparatus where the output from a high intensity light source is controlled to produce well - exposed images / videos and to reduce automatically the intensity when an unsafe issue is detected in medical devices such as endoscopes and the like . The method and apparatus over come problems to control light sources that have high frequency noise , slow - response time , nonlinearity , and non monotonic response time and to protect the patients ' tissues from possible overheating / burning and the eyes of personnel and patients from possible direct exposure to high intensity light used in medical devices such as endoscopes and the like .
7 Claims , 8 Drawing Sheets
Completion of Correlation Method : I? correlation between “ Luma meas " and incrementing output intensity is
NOT detected , power will be set to a safe level
p _ max - !
D ) . Correlation Method is disabled ; B ) . EV Method is enabled
* . . . . . . . .
* * * 8 . - . . . . . . . .
- - - - . . .
* - -
* . . . . .
-
- - - - -
- Requested Output Intensity , P reci Output intensity - Range for Normal Operation of Light Source Control -
-
. . . . . . . . * -
- P scan min * -
-
. , * * * * - .
+ -
D ) . Correlation Method is enabled ; B ) . EV Method is disabled
p min ? ? ? ? ? ? ? ? . . ? ? . . - - 4 . - - - - - - - - - - - . . . - - - - - . . . Area of safe - levels for output intensity , when - or D failure is detected
. - - . . . - N Time . -
. .
Potential un - safety issue is detected
Scanning ( increment ) of output intensity and Correlation
Method are requested / enabled based on detection of potential
un - safety
US 9 , 770 , 163 B2 Page 2
NNNN ( 51 ) Int . Ci .
H04N 5 / 235 ( 2006 . 01 ) A61B 1 / 04 ( 2006 . 01 ) GO3B 15 / 03 ( 2006 . 01 ) A61B 1 / 00 ( 2006 . 01 ) GO3B 15 / 14 ( 2006 . 01 )
( 52 ) U . S . CI . CPC . . . . . . . . . H04N 5 / 2352 ( 2013 . 01 ) ; H04N 5 / 2354
( 2013 . 01 ) ; H04N 7 / 18 ( 2013 . 01 ) ; A61B 1 / 00006 ( 2013 . 01 ) ; AC1B 1 / 00009 ( 2013 . 01 ) ;
GO3B 15 / 03 ( 2013 . 01 ) ; G03B 15 / 14 ( 2013 . 01 ) ( 58 ) Field of Classification Search
??? . . . . . . . . . . . . . A61B 1 / 00009 ; A61B 5 / 0059 ; A61B 1 / 00057 ; A61B 1 / 07 ; A61B 1 / 00172 ;
A61B 5 / 0084 ; HO4N 7 / 18 ; HO4N 5 / 2354 ; HO4N 5 / 2352 ; HO4N 5 / 235 ; GO3B 15 / 14 ;
G03B 15 / 03 ; G01J 1 / 00 See application file for complete search history .
7 , 556 , 414 B2 * 7 / 2009 Hopkins . . . . . . . . . . . A61B 1 / 00059 362 / 231
7 , 563 , 010 B2 7 / 2009 Birnkrant et al . 7 , 585 , 276 B2 9 / 2009 Itoi 7 , 798 , 959 B2 9 / 2010 Negishi 7 , 828 , 726 B2 11 / 2010 Negishi 7 , 847 , 817 B2 12 / 2010 Negishi 7 , 853 , 106 B2 12 / 2010 Takahashi et al . 8 , 118 , 732 B2 * 2 / 2012 Banik . . . . . . . . . . . . . . . . A61B A61B 1 / 00059
600 / 117 8 , 403 , 835 B2 3 / 2013 Matsuura 8 , 454 , 500 B2 * 6 / 2013 Yabe . . . . . . . . . . . . . . . . . . A61B 1 / 00059
600 / 180 8 , 831 , 710 B2 * 9 / 2014 Kobayashi . . . . . . . A61B 1 / 00165
600 / 117 8 , 878 , 920 B2 * 11 / 2014 Ovod . . . . . . . . . . . . . . . . . . . . . . A61B 1 / 06
348 / 68 2003 / 0191368 A1 * 10 / 2003 Wang . . . . . . . . . . . A61B 1 / 00009
600 / 160 2004 / 0122291 Al * 6 / 2004 Takahashi . . . . . . . . . . . . A61B 1 / 0638
600 / 180 2004 / 0186382 A1 * 9 / 2004 Modell . . . . . . . . . . . . A61B 1 / 00059
600 / 473 2005 / 0020926 A1 * 1 / 2005 Wiklof . . . . . . . . . . . . . . A61B 1 / 00193
600 / 476 2007 / 0156021 A1 * 7 / 2007 Morse . . . . . . . . . . . . . . . . A61B 1 / 0019
600 / 167 2007 / 0183162 AL 8 / 2007 Higuchi 2008 / 0058602 A1 * 3 / 2008 Landry . . . . . . . . . . . . . . A61B 1 / 00055
600 / 180 2008 / 0287742 A1 * 11 / 2008 St . George . . . . . . . . . A61B 1 / 00036
600 / 160 2009 / 0187077 A1 * 7 / 2009 Hosoda . . . . . . . . . . . . . . A61B 1 / 00034
600 / 178 2009 / 0287051 A1 * 11 / 2009 Itoi . . . . . . . . . . . . . . . . . . . . . . A61B 1 / 0005
600 / 115 2010 / 0063355 Al * 3 / 2010 Matsuura . . . . . . . . . . . . . A61B 1 / 045
600 / 109 2010 / 0067002 A1 * 3 / 2010 Ishii . . . . . . . . . . . . . . . . . . . A61B 1 / 00009
356 / 317 2011 / 0237885 A1 * 9 / 2011 Matsubara . . . . . . . . . . A61B 1 / 0638
600 / 109 2011 / 0246109 Al 10 / 2011 Fafard
( 56 ) References Cited U . S . PATENT DOCUMENTS
4 , 366 , 529 A 12 / 1982 Takahashi et al . 4 , 415 , 952 A 11 / 1983 Hattori et al . 4 , 433 , 675 A 2 / 1984 Konoshima 4 , 509 , 508 A 4 / 1985 Tsukaya 4 , 527 , 552 A 7 / 1985 Hattori 4 , 561 , 429 A 12 / 1985 Sato et al . 4 , 963 , 960 A 10 / 1990 Takami 5 , 091 , 779 A 2 / 1992 Ams et al . 5 , 131 , 381 A 7 / 1992 Ams et al . 5 , 134 , 469 A 7 / 1992 Uchimura 5 , 159 , 380 A 10 / 1992 Furuya et al . 5 , 162 , 913 A 11 / 1992 Chatenever et al . 5 , 436 , 655 A * 7 / 1995 Hiyama . . . . . . . . . . . A61B 1 / 063
348 / 139 5 , 475 , 420 A * 12 / 1995 Buchin . . . . . . . . . . . . . . . A61B 1 / 0005
348 / 65 5 , 749 , 830 A 5 / 1998 Kaneko et al . 5 , 822 , 486 A * 10 / 1998 Svetkoff . . . . . . . . . GO2B 6 / 065
385 / 116 5 , 957 , 834 A 9 / 1999 Mochida 6 , 100 , 920 A * 8 / 2000 Miller . . . . . . . HO4N 7 / 18
348 / 68 6 , 320 , 331 B1 * 11 / 2001 Iida . . . . . . . . . . . . . . . . . . . . . . HO5B 37 / 03
315 / 129 6 , 464 , 633 B1 10 / 2002 Hosoda et al . 6 , 511 , 422 B1 * 1 / 2003 Chatenever . . . . . . . . . . A61B 1 / 0661
600 / 180 6 , 707 , 485 B1 3 / 2004 Higuchi et al . 6 , 734 , 894 B1 5 / 2004 Higuchi et al . 6 , 767 , 320 B27 / 2004 Farkas et al . 6 , 856 , 436 B2 * 2 / 2005 Brukilacchio G02B 26 / 10
257 / E33 . 071 7 , 070 , 560 B2 7 / 2006 Takahashi 7 , 077 , 804 B2 7 / 2006 Ota 7 , 140 , 893 B2 11 / 2006 Abe et al . 7 , 235 , 045 B2 6 / 2007 Wang et al . 7 , 247 , 135 B2 7 / 2007 Iriyama 7 , 336 , 894 B2 2 / 2008 Ito
OTHER PUBLICATIONS European Search Report Application No . EP 12 17 6218 Completed Oct . 19 , 2012 dated Nov . 6 , 2012 10 pages . Anonymous : “ Exposure ( photography ) ” Wikipedia , the free ency clopedia ; XP002685566 ; Retrieved From the Internet : URL : http / / en . wikipedia . org / wiki / Exposure _ % 28photography % 29 Retrieved on Oct . 18 , 2012 Anonymous : “ Exposure Value ” Wikipedia , the free encyclopedia ; XP002685565 ; Retrieved From the Internet : URL : http / / en . wikipedia . org / wiki / Exposure _ value Retrieved on Oct . 18 , 2012 . Anonymous : “ Flash ( photography ) ” Wikipedia , the free encyclo pedia ; XP002685567 , Retrieved From the Internet : URL : http / / en . wikipedia . org / wiki / Flash _ % 28photography % 29 Retrieved on Oct . 18 , 2012 . Anonymous : “ Light Meter ” Wikipedia , the free encyclopedia ; XP002685568 ; Retrieved From the Internet : URL : http / / en . wikipedia . org / wiki / Exposure _ meter Retrieved on Oct . 18 , 2012 .
* cited by examiner
16 16
. . . Figure 1
38
- 52
atent
amp
Output Control Circuitry
Bus Interface
20
Sep . 26 , 2017
. . . .
.
.
.
.
.
.
.
3
H
.
Imager
S
24
- 28
54 -
Auto Exposure Circuitry
Sheet 1 of 8
Bus Interface
Controller includes : A ) . Self - Recovery Method that includes : Adaptive Normalization of EV and Self - Calibration Methods B ) . Detection of Hot Distal - End Method C ) . Failure - Detection Method D ) . Exposure Value ( EV ) Method E ) . Power Scan ( Correlation Method
Video Display
F ) Set Delays Method : Set Delay Time before and after programming Light Source ( LS ) to compensate slow response of the light Source
36
US 9 , 770 , 163 B2
32
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
US 9 , 770 , 163 B2
( EV ) Method D ) . Exposure Value
Method C ) . Failure - Detection !
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . :
220
215
Sheet 2 of 8
( Correlation ) Method E ) . Power Scan
Distal - End Method B ) . Detection of Hot
225
210
Lara . . . .
Sep . 26 , 2017
- - - - - -
F ) . Set Delays Method
Methods Self Calibration Normalisation of EV &
A ) . Adaptive
- - - - - - - - - - - - - - - - - -
234 234
226
205
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD D DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD
U . S . Patent
Figure 1A
U . S . Patent S ep . 26 , 2017 Sheet 3 of 8 US 9 , 770 , 163 B2
240
Figure 1B If requested , compute Correlation
- 230 235
- I - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
" Scan Request " : Estimate safety - needs for Scan - Request to compute Correlation between Lume _ meas and Output Intensity " Power Scun " : If requested , increment output intensity from Power _ scan _ min to Power _ max
Figure 2
U . S . Patent
Completion of Correlation Method : If correlation between " Luma _ meas " and incrementing output intensity is NOT detected , power will be set to a safe level
P max
* * * .
.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- -
- - - - - -
- -
- - - -
. .
* * * * *
. * * *
D ) . Correlation Method is disabled ; B ) . EV Method is enabled
*
. . . . . .
Sep . 26 , 2017
:
on .
*
Wome
* .
* .
. .
uuuuuuu * * * * *
. * * * * *
"
ww www
w
i scan
* * * *
.
* *
-
* *
W
*
"
* * *
-
.
-
. . . . . * * * *
: . . .
-
Requested Output Intensity , P _ rec _ i
Output Intensity - Range for Normal Operation
of Light Source Control
* * * * * * * * * * * . . . . . .
* * * * * *
- - -
* * * * * * * * * * *
-
P scan min -
- -
Sheet 4 of 8
- -
. . .
- -
1 tel .
D ) . Correlation Method is enabled ; B ) . EV Method is disabled
p _ min
- - - -
- - -
-
- -
- - - - -
-
-
-
-
-
-
-
Area of safe - levels for output intensity , when B ) - or D ) - failure is detected
.
. .
Time
Potential un - safety issue is detected
Scanning ( increment ) of output intensity and Correlation Method are requested / enabled based on detection of potential un - safety
US 9 , 770 , 163 B2
et
Imager
30
“ _ tr " means treshold “ _ meas " - means measured
Auto Exposure Circuitry
Normal Operation
Camera / Head Type . - PAL / NTSC , - Frame Size / Resolution
Failure *
-
234
U . S . Patent
Light Source Controller
- Exposure Value ( EV )
- Perimeter Black ( PB )
LSC - CORE
- Luma ( Luma )
- Image Motion Metric ( IMM )
A ) . Self - Recovery Method that includes : Adaptive Normalization of EV and Self - Calibration
Methods ( based on Camera / Head Type , PAL / NTSC , Frame Size / Resolution ) .
Lower power to avoid high temperature at the F distal end :
F ( tissue is too close to
B ) . Detection of Hot Distal
EV meas < EV I - M
End Method
Decrease LS _ power to the safe value * the hot distal end )
Yes
Tight Guide / Source disconnected Decrease LS power to the safe value *
C ) . Failure - Detection Method Yes ( low - or no light detected )
Luma mcas < Luma _ tr
.
Camera disconnected : Decrease LS power to the safe value * | Decrease Ls power to the safe value *
Yes Yes
PBmeas PB ir
Sep . 26 , 2017
No
NO motion : Decrease LS _ power to the safe value *
- Yes ( motion is NOT detected
JIMM meas < IMM tr
- No
| NO power change K
no -
Tower Change allowed
Yes Scanning completed ?
Yes
D ) . Exposure Value ( EV ) Method
- No
220
EV is below the optimal range : Decrement LS Power
EV < EV _ opt _ mini
Sheet 5 of 8
Yes
FCEV > EV _ upl _ max
E ' is above the optimal range : Increment LS Power EV is in optimal range : Do not change LS Power
- No
225 - No
" Scan Request " :
E ) . Correlation Method
Estimate safety - needs for Scan - Request to compute Correlation between Lume meas and LS Power
" Power Scan " : Scan LS _ power ( increment ) from
Accept LS _ Power
suggested by EV _ Method No _ canning of Power
LS _ Power _ scan _ min to LS _ Power _ max
is requested ?
Compute Correlation
Scanning is completed &
No Correlation : Decrease
Correlation failed ?
LS power to the safe value *
NO
L
yes
-
226
F ) . Set Delays
Set Delay Time " dlt _ t _ pause " / sec ) to compensate slow - response of the light source ( IS )
Compute new optimal EV range : EV opt min ; EV opt max ( dNow opt )
US 9 , 770 , 163 B2
Power _ requested
Figure 3
Light Source ( LS )
16
402
Measure ?y
Figure 3A
210
W
wwwwwwww wwwwwwww
atent
Lower output / light intensity to avoid high temp : Set Output Intensity to Minimum Safe Level
Ev meas EV _ tr
Yes
Sep . 26 , 2017
108 .
406
No
Lower outputlight intensity to avoid high temp Set Outp : t intensity to Mininum Safe Level
Luma meas klumat Yes
416
$ 18 . .
Sheet 6 of 8
No
424
Camera Disconnecte Sei Output Intensity to Minirunn Safe Level
Pb meas PB Ir
Yes
426
428
No notion , Set Output ight intensity to Minimum Safe Level
IMM meas < IMM ir
Yes
US 9 , 770 , 163 B2
438
Figure 3B
220
atent
4449
EV < EV _ opt _ min
EV is below the optimal range ; Decrement Output Intensity
Sep . 26 , 2017
Yes
448
- 446
3 EV is above the
optimal range ; Increment Output Intensity
Sheet 7 of 8
Yes
EV > EV _ opt _ max
- 456
458
No EV is in the optimal
range ; Do not change Output Intensity
US 9 , 770 , 163 B2
Figure 4
EXPOSE VALUE ( EV , OR ANOW ?
?
?? … … …
atent
0
-
.
www
POWER : LOG - SE?ALOG SCALE - - - - UNOWrmisec - - - E?PRICANOM
N _ Ts ?
?
???????????1
s
MAMMA???????????? ? ?
New to
ther
in
iLTE
?? . ????
Mone
Sep . 26 , 2017
????????????ptt4 . M4AM #
www … … … 2
- EV Cow
M
NT SEE OCEAE ?5 % TO 100 %
* MMMMMMMM
?????????? ????
50 %
ore
PARAMETERS VS POWER
window _ 1
?? LIGHT SOURCE OUTPUT
Re : =
?? ?????
????
??? ;
Sheet 8 of 8
ty witterch L? + ? ?? , ????t - ?? . * . * * * ? : ??????????
( ??KANY 2 . RM RESPONSE 3 ) . NONLINEARITY AND ? MOMOMOTOME RESPONS THE ABOVE THREE FEATURES OF THE XILINX LIGHT SOURCES DOES NOT ALLOW TO IMPLEMENT APITTER FIRMANY
who Arrrr .
Y??????
4 S
5 ?
??????? ????
CONTROL XENON LIGHT SOURCES
3
2
2
?
4 18 18
? POWER
US 9 . 770 . 163 B2
TIME , SENDS
US 9 , 770 , 163 B2
METHOD AND APPARATUS FOR that the light source intensity is automatically reduced to a CONTROLLING LIGHT OUTPUT safe level when the light source output is not directed to a
INTENSITY AND PROTECTION FROM surface and / or the camera / imager or light - guide are discon HIGH INTENSITY LIGHT nected from each other . It is also desirable to protect tissues
5 operated on during a surgical procedure from overheating or FIELD OF THE INVENTION burning due to the intensity of the light source being set
higher than required to produce well - exposed images . It is This invention generally relates to a method and apparatus also desirable to protect the eyes of the operator of an
for automatically protecting personnel and patients from endoscope or persons in the surgical area from direct expo direct exposure to the output of a high intensity light source 10 sure to high intensity light used in medical devices such as used in medical devices , such as endoscopic imaging sys - endoscopes and the like . tems and the like . The invention also generally relates to a Techniques for controlling the output intensity from a method and apparatus for overcoming problems to control light source are known in the art . For example , a technique high intensity light sources that have high - frequency noise , for automatically controlling the light intensity from a light slow - response time , nonlinearity , and non - monotonic 15 source , on the basis of an image signal from an imaging unit response time , such as Xenon or Xenon - like based light associated with an endoscope , is described in Japanese sources . Unexamined Patent Publication No . 62 - 155689 as men
tioned at column 2 , lines 1 - 21 in U . S . Pat . No . 5 , 957 , 834 to BACKGROUND OF THE INVENTION A . Mochida . As recognized in the Mochida patent , when
20 light intensity control is made dependent upon a signal The imaging of body surfaces through an endoscope is derived from the image , then upon removal of the endoscope
well known within the medical and veterinarian fields . from the body , the control is likely to increase the output Typically , this involves inserting an endoscope into a body intensity level from the light source , when instead it should cavity and directing a high intensity light source output decrease the output to protect the operator ' s eyes from through the endoscope to illuminate body tissue . Light 25 inadvertent high intensity light exposure and prevent igni reflected by the body tissue then is guided along an optical tion of combustible material . In the Mochida patent , a switch path to an image sensor to generate both video and still is added to manually adjust and control the output of the images of the tissue . One such approach is described in U . S . light source when the endoscope is removed from a body . Pat . No . 5 , 162 , 913 to Chatenever et al . , and provides a As further described in the Mochida patent , the output technique for an automatic adjustment of the exposure of 30 intensity level of the light source is controlled by regulating video images detected with a CCD ( charge coupled device ) the position of a diaphragm with respect to the light source . image sensor . The control signal for doing this is derived from an image
The use of high intensity light sources involves potential sensor in the endoscope . hazards to medical personnel and patients . For example , In U . S . Pat . No . 4 , 527 , 552 a photoelectric element gen when a light guide cable , used to convey the high intensity - 35 erates a signal indicative of the intensity of light reflected light source output , is momentarily disconnected from the from an object illuminated by a light source associated with endoscope and placed on a sterile drape used to protect the the endoscope to control the light source output level . In patient , the high intensity of light output can be sufficient to U . S . Pat . No . 5 , 131 , 381 a light source associated with an ignite the drape and pose a fire hazard ; or , the user can endoscope is controlled by a signal that represents the inadvertently hold the disconnected light guide cable in such 40 density value of each line of a camera video image derived a way as to temporarily blind another person in the room . In through the endoscope . Other patents relevant to light inten some instances , when the endoscope is removed from ( i . e . , sity level controls for endoscopes are U . S . Pat . Nos . 3 , 670 , pulled out of ) a patient , there can be a risk of these same 722 ; 4 , 963 , 960 ; 4 , 561 , 429 ; 5 , 091 , 779 ; 5 , 134 , 469 ; 5 , 159 , hazards . When the light source is used with an endoscopic 380 ; 6 , 767 , 320 ; 7 , 070 , 560 ; 7 , 585 , 276 ; 7 , 798 , 959 ; 7 , 847 , video camera , which has an automatic exposure system , the 45 817 ; and 7 , 828 , 726 . light source output intensity may be turned up to a intensity Techniques have been proposed to reduce the risks asso level higher than required for the camera to produce well - ciated with high intensity light sources . One involves a exposed images . This increased light intensity level can special light guide cable with wires in it that are shorted desiccate body tissue and cause serious injury to the patient . together when the cable is attached to an endoscope . The Typically , endoscopic video camera automatic exposure 50 short is detected at the light source and light output intensity systems can produce well - exposed images with an electronic is reduced when the cable is disconnected and the short is shutter setting of approximately 1 / 125th to 1 / 500th of a second subsequently removed . A retractable mechanical shroud , If the distal end of an endoscope is placed within close which covers the light guide cable end when not connected proximity to tissue being imaged , typically , a relatively to an endoscope , has also been suggested . lower light intensity level will still enable an endoscopic 55 These safety solutions are not necessarily effective against video camera to produce well - exposed images . An undesir - all potential hazardous conditions that may arise ; such as able , and potentially dangerous , scenario can occur if the when the endoscope with the light guide cable still attached light source output is set to a high intensity level , and the is pulled out of a patient and inadvertently directed at a endoscope distal end is placed within close proximity to person or surgical drape , or when the light guide or source tissue being imaged . Typically , in such a case , automatic 60 initially is directed to treat openly accessible tissue and camera exposure systems will adjust the electronic shutter inadvertently misdirected during or after a procedure , or setting to approximately 1 / 10 , 000th of a second ( or faster ) to when a video camera head , attached to the endoscope light compensate for the high illumination reflections from the guide cable combination , is disconnected from its corre tissue . In such a situation , the risk of desiccating delicate sponding control unit . tissue is greatly increased . 65 One such approach to solve this problem is described in
Thus , it is desirable to solve these problems , specifically U . S . Pat . No . 6 , 511 , 422 to Chatenever ( hereinafter Chate to control the output from a high intensity light source so never ’ 422 ) . Chatenever ’ 422 , herein incorporated by refer
US 9 , 770 , 163 B2
ence , describes a method and apparatus where the output lems when working to control light sources that have from a high intensity light source is controlled so that slow - response , high - frequency noise , nonlinearity , and non whenever the output is not directed at tissue ( meaning that monotonic response times . the endoscope / video camera / light source combination is not It is an object of the invention to provide a method for currently being used to image body tissue ) , the light source 5 upgrading existing and newly - designed endoscopic imaging output intensity is automatically reduced to a safer level . systems with a light source control ( “ LSC ” ) feature that This is done by monitoring the reflected light from tissue and controls the intensity of the light source output . It is an when this reflection indicates that the light source is not object of the invention to reduce and / or eliminate costs directed at tissue , the light intensity is turned down to a safer associated with upgrading of existing and newly - designed level . This involves generating a modulation signal and and 10 endoscopic imaging systems with the LSC feature without
any hardware change in product lines of light sources , modulating the intensity of the light source output with the camera - heads / imagers , CCUs , videoscopes , and endo modulation signal . scopes . Chatenever ' 22 involves monitoring the light reflected It is an object of the invention to provide a cost - effective by a surface , detecting the modulation in the monitored 15 single LSC implementation that is suitable for existing and light , and reducing the intensity of the light source output newly - developed or in - development product lines of light when the detected modulation is below a reference level . sources , camera - heads / imagers , CCUs , videoscopes , and However , Chatenever ’ 422 , while effective as a safety endoscopes . solution , has problems controlling Xenon lights and other It is an object of the invention to provide a method and light sources because the amplitude or frequency modulation 20 apparatus that enables adaptive normalization and self methods described by Chatenever ’ 422 do not work well calibration , so as to simplify the adjustment of the light with light sources having these problems . Specifically , control feature to new camera heads / imagers , CCUs , endo Chatenever ' 422 does not work well with lights sources that scopes , videoscopes , and light sources . have slow - response , high - frequency noise , nonlinearity , and It is an object of the invention to provide a self - recovery non - monotonic response times , such as Xenon lights . It is 25 method that involves adaptive normalization and self - cali thus desirable to provide an improved method and apparatus bration , so as to recover and / or optimize the LSC feature by that works with light sources that have slow - response , adding new or replacement components to the LSC system . high - frequency noise , nonlinearity , and non - monotonic Adding such new or replacement components can be added
response times . during use and / or surgery . It is also desirable to improve upon the methods and 30 nd 30 It is an object of the present invention to control the output
from a high intensity light source so that the output intensity apparatus described in the Chatenever ' 422 patent to over is automatically set at a reasonable and safe level while come problems to control Xenon light sources , as Xenon producing well - exposed images . light sources have increased applicability in endoscope It is an object of the present invention to protect personnel technology . None of the other existing methods and appa 4 . 35 and patients from direct exposure to high intensity light used ratuses described in the prior art work effectively with light in medical devices such as endoscopes and videoscopes by sources that have slow - response , high - frequency noise , non automatically reducing the light source output intensity to a linearity , and non - monotonic response times . safe level when a safety - failure is detected . It is an object of
It is further desirable to provide a method and apparatus the invention to provide a failure - detection method that to upgrade existing and future endoscopic imaging systems 40 detects when the light source output is not directed to a with a light source control ( “ LSC ” ) feature that solves surface , when the camera / imager and / or light - guide are problems associated with light sources , such as Xenon disconnected , and when no motion is detected in the video lights . It is also desirable to do so in a cost effective way , and image . without any hardware change in product lines of light It is an object of the present invention to protect tissues sources and endoscopic imaging systems . 45 from overheating / burning when the distal end of an endo
It is also desirable to design a cost - effective single LSC scope is placed within close proximity to tissue by auto implementation based on the method that is suitable for m atically reducing the light source output to a safe level various existing and yet to be developed product lines of while still producing well - exposed images . light sources , camera heads , camera control units ( “ CCUs ” ) , These and other objects of the invention are achieved by videoscopes , and endoscopes . It is also desirable to provide 50 providing an apparatus for viewing a surface comprising : an software that executes upon hardware . examining instrument , the examining instrument having an
It is also desirable to provide a method and apparatus for imaging path through which the surface is observed ; a light LSC that enables adaptive normalization and self - calibra - source , the light source illuminating the surface ; an imager , tion ; so as to simplify the adjustment of the LSC feature to the imager detecting light reflected from the surface , and new endoscopic imaging systems and light sources and to 55 generating image signals ; a camera control unit ( CCU ) , the minimize manual adjustment . CCU processing the image signals ; and a controller associ
It is also desirable to provide a self - recovery method that ated with the imager and CCU , the controller decrementing involves the adaptive normalization and self - calibration or incrementing the intensity of the light source output via techniques , so as to recover and / or optimize the LSC feature at least two steps . to possible new vs . replacement of light source and / or the 60 The controller of the apparatus may decrement or incre type of the scope attached to the camera during surgery , ment the intensity of the light source output via a plurality and / or the camera type of CCU during use and / or surgery . of steps . The intensity of the at least two steps and / or
plurality of steps may be incremented or decremented in SUMMARY OF THE INVENTION intervals as small as 1 % of the total light source intensity
65 output . The power intervals incremented or decremented Accordingly , it is an object of the present invention to may be in steps that are greater than 1 % ( i . e . 3 % , 5 % , 15 % )
provide a method and apparatus to overcome known prob - or in steps less than 1 % ( i . e . 0 . 25 % , 0 . 5 % ) of the total light
US 9 , 770 , 163 B2
scan , min .
source output intensity ; depending upon the operational Pscanmin is less than Pmax , and Pmar is the maximum allowed characteristics of the light source , camera head / imager , light source output intensity . The output intensity may be CCU , and other devices being utilized . The magnitude of the incremented via steps from Pscan , min to Pmax . Optionally , the at least two steps may be incremented or decremented by a output intensity may be decremented via steps from Pmax to predetermined percentage of the maximum light source 5 P . . . output intensity . The power scan method may be enabled by a scan request The apparatus may further comprise a communication bus method , the scan request method being able to detect a coupled to a plurality of bus interfaces for communication potential safety issue . between the light source , CCU , and camera . Furthermore , the controller may compensate for the prior The light source of the apparatus may be a Xenon or 10 art shortcomings and disadvantages of detecting a modula Xenon - like based light source . The light source may have at tion within the output of a Xenon or a Xenon - like based light least one of a high - frequency noise , slow - response time , nonlinearity , and non - monotonic response time . source , due to high frequency noise , slow , nonlinear , and
The examining instrument of the apparatus may be non - monotonic response , by using the scan request method selected from a group consisting of an endoscope , laryngo - 15 that includes a correlation method allowing for detecting scope , bronchoscope , fiberscope , duodenoscope , gastro when the light source output is not directed at a surface , and scope , flexible endoscope , arthroscope , cystoscope , laparo for decrementing the output intensity with a specified step scope , anoscope , and sigmoidoscope . AP scan until a safe output level is reached . When the scan
The controller of the apparatus may process the image request method detects a potential safety issue , the correla signals representative of light reflected from the tissue / scene 20 tion method is enabled and the EV method is disabled , and as detected by the imager . wherein when a potential safety issue is not detected , the
The imager ( or image sensor ) may be located distally correlation method is disabled and the EV method is inside the examining instrument , proximally inside the enabled . examining instrument , or externally from the examining The apparatus may involve having the output intensity of instrument . The imager may be a CCD , CMOS , or imager 25 the light source be automatically reduced to a safe level types yet to be developed when the light source output is not directed to a surface . The
The apparatus may further comprise software executing intensity of the light source may be automatically reduced to on the controller . The software involves operating the con - a safe level when at least one of the camera head , image troller such that the software allows for the apparatus to sensor and light source and / or light guide are disconnected . operate . The software may involve various specific param - 30 The controller may include adaptive normalization and eters that allow for the apparatus and controller to operate . self - calibration . The adaptive normalization may include
In one example , the software may operate by including an normalizing EV and recomputing calibration curves based exposure value EV , ( i . e . , the shutter speed in seconds ) , EV ; on the type of light source and image sensor . The adaptive being used to control the output intensity of the light source . normalization and self - calibration may depend upon image The measured EV , is referred to as EV meas , and when 35 sensor integration time Time and also the value of EV min and EV meas is below a threshold EV # r , the light source output EV more intensity is reduced . EV , is referred to as the shutter speed . The controller may also include a self - recovery method It may be determined in seconds or other such units of time that involves adaptive normalization and self - calibration , so for measuring the shutter speed . as to recover and / or optimize the LSC feature . This involves
The optimal value of EV ; is referred to as EV ope and is 40 adaptive normalization and self - calibration when compo between EV opt , min and EV opt , max , wherein EV opt depends on nents in the system are changed or replaced during use EV meas , the monotonic and linearity characteristics of the and / or surgery . Components that are replaced and / or light source output . changed are the type of scope attached to the camera , the
If EV meas is less than EV opt . min , the output intensity of the light source , the camera , any videoscopes , endoscopes and / light source is incremented , if EV meas is greater than 45 or CCU devices . Other components in the system may also EV . pt . max , the output intensity of the light source is decre - be replaced . mented , and if EV meas is between EV pt min and EV opt mare The self - recovery method may protect from possible the output intensity of the light source is unchanged . failure of the LSC caused by the new and / or replacement
The criteria to form a well - exposed image requires that components during surgery . EV meas be between EV opt . min and EV . pt . mar . The software 50 The controller may detect when no motion or the lack of executing on the controller operates such that EV meas is motion is detected in the video images so as to decrease between EV opt min and EV . pt . max so that well - exposed output intensity to a safe level . The controller may compen images are taken . sate the slow response of a Xenon or a Xenon - like light
The apparatus of various embodiments of the invention source by inserting optimal delays before and / or after con minimizes the number of increments or decrements of light 55 trolling of light source . In certain embodiments , the con output intensity . It is advantageous to minimize the number troller may not be continuously enabled in order to minimize of increments or decrements of light output intensity and to flickering of the images . do so in a stepwise manner . Other objects of the invention are achieved by providing
The controller of the apparatus may include a scan request an apparatus for viewing a surface comprising : an examin method for detecting a potential safety issue when the light 60 ing instrument , the examining instrument having an imaging source output intensity ( “ P " ) is potentially unsafe . Once a path through which the surface is observed ; a light source , potential safety issue is detected via the scan request the light source illuminating the surface ; an imager , the method , the apparatus may enable the power scan and the imager detecting light reflected from the surface , and gen correlation methods , and disable the EV method . The power erating image signals ; a camera control unit ( CCU ) , the scan method being able to increment or decrement the 65 CCU processing the image signals ; and a controller associ output intensity of the light source with a specified step ated with the imager and CCU , the controller decrementing AP scan from Pscan min to Pmax , and vice - versa , where the intensity of the light source output via at least two steps .
meas
eus max
t . max "
meas
US 9 , 770 , 163 B2
opt opt
opt , min
meas
The controller may also have light source output intensity ing instrument , the examining instrument having an imaging incrementing ability in certain embodiments . path through which the surface is observed ; a light source ,
The examining instrument may be selected from a group the light source illuminating the surface ; a CMOS imager , consisting of an endoscope , laryngoscope , bronchoscope , the CMOS imager detecting light reflected from the surface , fiberscope , duodenoscope , gastroscope , flexible endoscope , 5 and generating image signals ; a camera control unit ( CCU ) , arthroscope , cystoscope , laparoscope , anoscope , and sig the CCU processing the image signals ; and a controller moidoscope . associated with the CMOS imager and CCU , the controller Other objects of the invention are achieved by providing decrementing or incrementing the intensity of the light a method for controlling the light output intensity of a light source output via predetermined steps . source comprising : measuring an exposure value EV1 , EV ; 10 Other objects of the invention are achieved by providing being used to control the output intensity of the light source ; a method for protection from a high intensity light source and incrementing or decrementing the output intensity of the output comprising : measuring an exposure value EV meas , light source based upon the exposure value EV ;
The method may have software executing , the software EV meas being used to control the intensity of the light source being able to execute upon hardware . output ; and incrementing or decrementing the output inten
The method may have a measured EV , be EV meas , and sity of the light source Pscan based upon the exposure value when EV mens is below the threshold EV , the light source EV meas ; determining an optimal value of EV meas , which is output intensity is immediately set to a safe level . EV opt , EV opt being between EVopt , min and EV opt , max , and
The method may have the optimal value of EV , be EV EVopt being dependent on the monotonic and linearity char which is between EV opt , min and EV opt . max , wherein EV pt 20 acteristics of the light source , wherein if EV meas is less than depends on EV meas , and the monotonic and linearity char EVopi . min the output intensity is incremented , wherein if acteristics of the light source . EV meas is greater than EV opt , max , the output intensity is
If EV , is less than EV . pt . min , the output intensity of the decremented , and wherein if ÉV meas is between EV opt , min light source is incremented , if EV ; is greater than EV . ptmare and EV opt , max , the output intensity is unchanged . the output intensity of the light source is decremented , and 25 Other objects of the invention are achieved by providing if EV , is between EV ont min and EV optimax , the output inten - an apparatus for protection from a high intensity endoscopic sity of the light source is unchanged . The criteria to form a light source output comprising : an endoscope having an well - exposed image requires that EV meas be between imaging path through which the surface at a distal end can EVopt . min and EV opt , max be observed ; a Xenon light source for illumination of the
The method may further comprise the step of increment - 30 surface ; an imager detecting light reflected from the surface ing the output intensity of the light source via a power and generating image signals ; a camera control unit ( CCU ) scanning step , the power scanning step allowing for incre - processing the image signals received from the imager , the menting the output intensity of the light source a few times camera control unit including a controller , the controller with a specified step AP scan , from Pscan . min to Pmax , where being able to increment or decrement the output intensity of Pscan , min is less than Pmax , and Pmax is the maximum allowed 35 the Xenon light source , such that if the controller detects a light source output intensity . The method may further com - potential safety issue , the Xenon light source output inten prise the step of decrementing the output intensity of the sity is adjusted via at least one discrete step AP scan , from light source via a power scanning step , the power scanning Pmin to Pmor , where Pscan min is less than Por , and P is the step allowing for decrementing the output intensity of the maximum allowed light source output intensity ; and a light source a few times with a specified step AP scan , from 40 communication bus coupled to a plurality of bus interfaces Pscan . max to Pmin , where Pscan . max is greater than Pmin , and for communication between the Xenon light source and the Pin is the minimum allowed light source output intensity camera control unit .
Other objects of the invention are achieved by providing Other objects of the invention are achieved by providing an apparatus for viewing a surface comprising : an examin - an apparatus for protection from a high intensity endoscopic ing instrument , the examining instrument having an imaging 45 light source output comprising : an endoscope having an path through which the surface is observed ; a light source , imaging path through which the surface at a distal end can the light source illuminating the surface ; an imager , the be observed ; a light source for illumination of the surface ; a imager detecting light reflected from the surface , and gen - camera head including an image sensor aligned to detect erating image signals ; a camera control unit ( CCU ) , the light reflected from the surface and for generating image CCU processing the image signals ; and a controller associ - 50 signals ; a camera control unit processing the image signals ated with the imager and CCU . received from the camera head , the camera control unit
The controller of the apparatus may decrement or incre including a controller associated with the camera head to ment the intensity of the light source output via a plurality process image signals representative of images detected by of steps . The intensity of the at least two steps and / or the image sensor , the controller being able to increment or plurality of steps may be incremented or decremented in 55 decrement the intensity of the light source output ; and a intervals as small as 1 % of the total light source intensity communication bus coupled to a plurality of bus interfaces output . The power intervals incremented or decremented for communication between the light source and the camera may be in steps that are greater than 1 % ( i . e . 3 % , 5 % , 15 % ) control unit . or in steps less than 1 % ( i . e . 0 . 25 % , 0 . 5 % ) of the total light The apparatus may have its controller include an exposure source output intensity ; depending upon the operational 60 value EV ; ( i . e . the shutter speed ) , EV , being used to control characteristics of the light source , camera head / imager , the output intensity of the light source . The apparatus may CCU , and other devices being utilized . The magnitude of the measure the value EVi , referred to as EV meas . When EV meas at least two steps may be incremented or decremented by a is below a threshold EV r , the light source output intensity predetermined percentage of the maximum light source Precy may be reduced . This is to prevent the overheating or output intensity . 65 burning of tissue due to a small distance between the distal
Other objects of the invention are achieved by providing end of the endoscope or videoscope apparatus and the an apparatus for viewing a surface comprising : an examin observed tissue .
ca
. 771 AX
min
meas
772 easy
scan ?
US 9 , 770 , 163 B2 10
The apparatus may have an optimal value of EV? , EV opt eter Black ( PB ) and Image Motion Metrics ( IMM ) with their which may be between EV . pt . min and EV opt . max , wherein corresponding thresholds , i . e . , Lumayr , PBtr , and IMM , r , EVopt depends on EV meas , the monotonic and linearity respectively . characteristics of the light source output . The criteria to form The apparatus may have its controller include the self well - exposed images requires that EV be between 5 recovery method . This involves adaptive normalization and EV opt , min and EV opt , max . Thus , it is advantageous to keep self - calibration when components in the system are changed EV meas within this range so that well - exposed images are during use and / or surgery . Components that are replaced created . and / or changed are the type of scope attached to the camera ,
If EV ; or EV meas less than EV . pt . min , the light source the light source , camera , videoscopes , endoscopes and / or output Preci is incremented , if EV ; or EV meas is greater than 10 CCU . Other components in the system may also be replaced .
The self - recovery method may protect from possible EV opt , max , the light source output Preci is decremented , and failure of LSC caused by the new and / or replacement if EV ; is between EV opt . min and EV opt . max , the light source components during surgery . output is unchanged . This controls the light source output The controller of the apparatus may or may not be via increments / decrements and helps keep the light source 15 continuously enabled , so as to minimize possible flickering output intensity at a safe level while minimizing number of of the video images . The apparatus may include a control changes of output intensity . feature that involves single code - implementation , without
The apparatus minimizes the number of increments or having hardware changes . The apparatus may be reconfig decrements of Precis so as to keep the light source output ured with this single code - implementation , as the apparatus intensity at a safe level . Minimizing the number of incre - 20 may utilize an existing endoscope . ments or decrements Preci allows the apparatus to operate Other objects of the invention are achieved by providing more efficiently and accurately . a method for protection from the output of high intensity
The apparatus may have a scan request method . The scan light sources comprising : measuring an exposure value EV ; , request method is able to detect a potential safety issue . The EV ; being used to control the output intensity of the light potential safety issue may involve having the light source 25 source ; and adjusting the output intensity of the light source output intensity exceed Pmor . Once the potential safety issue based upon the EV ; . is detected by the scan request method , power scan method The measured EV , may be EV meas , and when EV meas is and correlation method may be enabled , and EV method below the threshold EV r , the light source output intensity may be disabled . may be reduced . This is known as a step of the detection of
The power scan method may be able to increment the 30 the hot distal end of the endoscope , as when EV meas is less output intensity a few times with a specified step AP scan than EV , then the tissue is too close to the distal end of the from Pscan , min to Pmax , where Pscan . min is less than Pmax , and endoscope and the light source output intensity is lowered to P is the maximum allowed light source output intensity . avoid high temperature at the distal end that may cause the The power scan method may be able to decrement the output tissue to burn . intensity a few times with a specified step AP scan , from 35 The method may also have a failure detection method or Pscan , max to Pscan , min may have failure detection steps . This may involve decreas
The scan request method may further involve a correla - ing the light source output intensity to a safe level when the tion method running simultaneously with the scan method light source is disconnected , when the camera and / or light and allowing for computation of correlation ( i . e . , depen - guide is disconnected from the endoscope , and when there dence ) between the incrementing output intensity and the 40 is no motion detected in the video images . The failure light intensity reflected from tissue or observed surface detection methods may be based on comparison of achro Absence of such dependency , i . e . , absence of computed matic image brightness ( Luma ) , Perimeter Black ( PB ) and correlation may be used to detect when light source output Image Motion Metrics ( IMM ) with their corresponding is not directed to a surface and to lower the output intensity thresholds , i . e . , Lumar , PB 1r and IMMtr , respectively . of the light source to a safe level . 45 When Lumapear is less than Lumar , either low or no light
The apparatus may have the intensity of its light source is detected by the imager ; therefore , the light guide / source is output be automatically reduced to a safe level when the possibly either disconnected from the endoscope or endo light source output is not directed at a surface . scope / camera combination , the distal tip of the endoscope is
The apparatus may have its controller include a method to not within close enough proximity to a subject being imaged detect disconnection of the camera head , image sensor , and 50 to produce well exposed images ( i . e . , reflected light levels light source . The apparatus may have the intensity of its light are below the Luma threshold , indicative of the endoscope source be automatically reduced to a safe level when at least being too far from the imaged subject to be “ in use " ) , and the one of the camera head , image sensor , and light source power is decreased to a safe level . When PB meas is less than output are disconnected . PB tr , the camera may be disconnected from the endoscope
The apparatus may have its controller include adaptive 55 and the power is decreased to a safe level . When IMM mens normalization and self - calibration . Adaptive normalization less than IMM , , , motion is not detected within the video may include normalizing of exposure value EV . The self - images , indicative of the endoscope or endoscope / camera calibration may have the capability to recompute calibration combination being static ( i . e . , not being actively used for a curves based on the type of light source and / or the camera surgical procedure ) ; therefore , the light source output is head / imager being used . The adaptive normalization and 60 decreased to a safe level . self - calibration may depend on imager integration time Tinta The method may also have an exposure value step and / or EV min and EV max . Moreover , the apparatus may include method . The goal of the exposure value step and / or method methods in its controller to detect when the light guide is to minimize the number of power changes ( increment or source is disconnected , when the camera is disconnected , decrement ) taking into account known issues with Xenon and when no motion is detected in the video images . Such 65 light sources . a method ( failure - detection method ) may be based on a The exposure value step may involve having the optimal comparison of achromatic image brightness ( Luma ) , Perim - value of EV ¡ , EV opt , be between EV opt , min and EV opt , max?
SC0727
min
US 9 , 770 , 163 B2 12
max
wherein EVopt depends on the monotonic and linearity minimize the number of increments or decrements of light characteristics of the light source . If EV ; is less than intensity output taking into account known issues with EVont miw , the output intensity of the light source is incre - Xenon light sources . The exposure value denotes all com mented , if EV ; is greater than EV opt , max , the output intensity binations of a camera ' s shutter speed and relative optical of the light source is decremented , and if EV ; is between 5 aperture that result in the same exposure . EVopt , min and EV opt , max , the output intensity of the light Other objects of the invention allow for the controller to source is unchanged . The criteria to produce well - exposed be wirelessly controlled . Other objects of the invention images may require that EV meas be between EV opt , min and allow for the controller to incrementally change the intensity EVopt , max . The EV meas may be a continuously measured of the light source . Other objects of the invention allow for exposure value EV ; 10 the intensity of the light source to be changed so as to
To compensate for the ineffectiveness of using an output optimize the image quality while ensuring the light source modulation method with Xenon or Xenon - like light sources , output intensity remains at a safe level . the controller may compute a correlation metric between Other objects of the invention and its particular features illuminating and reflected light only when EV meas is too and advantages will become more apparent from consider high , i . e . , when Pi is requested to be around Pmax or higher 15 ation of the following drawings and accompanying detailed ( where Pmor is the maximum output intensity of the light description . It should be understood that the detailed source ) . While computing the correlation - metric , instead of description and specific examples , while indicating the light - modulation , one or a few sets of incrementing output preferred embodiment of the invention , are intended for intensity levels are requested : each set produces a ramp of purposes of illustration only and are not intended to limit the increasing light source output levels . This allows the present 20 scope of the invention . method to be effective for light sources that have slow response , high - frequency noise , nonlinearity , non - mono BRIEF DESCRIPTION OF THE DRAWINGS tonic response times , such as with Xenon based light sources . FIG . 1 is schematic block diagram view of an apparatus One goal of the exposure value method minimizes the 25 for controlling a light source in accordance with the inven
number of output intensity level changes and avoids loops in tion ; output level requests ( i . e . , avoids instability of the output FIG . 1A is schematic block diagram view of the controller control loop when the controller wrongfully requests peri - of FIG . 1 ; odical increment / decrement of output intensity ) . FIG . 1B is a schematic block diagram view of the
Another goal of the exposure value method is to increase 30 methods for controlling the controller of FIG . 1 ; the accuracy of computing correlation and minimize the FIG . 2 is a graph comparing requested output intensity of number of such computations , and as a result , minimize the light source against time that shows incrementing and dec number of changes in light source output intensity rementing the power in a stepwise manner ;
The method may also involve adaptive normalization and FIG . 3 is a schematic view of the steps of controlling the self - calibration method steps . The adaptive normalization 35 controller of FIG . 1 ; and self - calibration method steps take into account the type FIG . 3A is a schematic block diagram view of some of the of the imager ( i . e . , video endoscope or camera head ) con - steps of FIG . 3 ; nected to the CCU , and the type of light source . The method FIG . 3B is a schematic block diagram view of some of the may also involve a delay block to compensate for the slow steps of FIG . 3 ; and response of Xenon or Xenon - like light sources by using 40 FIG . 4 is a graph showing the high frequency noise , optimal delays before and / or after controlling of the light slow - response and nonlinearity of a Xenon or a Xenon - like SO source output . light source and comparing the output intensity of a Xenon
The adaptive normalization and self - calibration method or a light source against time . steps normalize EV to the imager being used in order to reuse the same control - equations , and re - computes calibra - 45 DETAILED DESCRIPTION OF THE tion curves based on the type of imager and light source INVENTION being used . The adaptive normalization and self - calibration method steps monitor the imager integration time Tint With reference to FIG . 1 , an endoscope 10 is illustrated minimum exposure value EV min and the maximum exposure having a camera head 12 mounted thereto at the proximal value EV max values of EV . 50 end to produce video images in a manner for example as
The method may also include steps of the self - recovery described in the aforementioned U . S . Pat . No . 5 , 162 , 913 to method . The self - recovery method may work , such that Chatenever , et al . The distal end of endoscope 10 is directed when equipment is changed , the previously used processing at tissue 14 to inspect the tissue with light from a high coefficients , equations , and / or calibration curves are no intensity light source 16 and passed to the distal end through longer valid . 55 a light guide cable 18 . Typically , light guide cable 18 can be
The self - recovery method adaptive normalization and disconnected from endoscope 10 at connector 20 , thus , self - calibration when components in the system are changed posing a safety hazard as described above . during use and / or surgery . Components that are replaced The light from light guide cable 20 is directed to illumi and / or changed are the type of scope attached to the camera , nate tissue 14 as suggested with path 22 and light reflected the light source , the camera , videoscopes , endoscopes and / or 60 by tissue 14 is passed along optical path 24 to imager 26 the CCU . Other components in the system may also be within camera head 12 . Imager 26 detects light reflected off replaced . tissue 14 by means of optical path 24 . Imager 26 may be any
The self - recovery method may protect from possible type commonly used within the art , such as but not limited failure of LSC caused by the new and / or replacement to CCD , CID or CMOS imagers . Camera head 12 produces components during surgery . 65 image signals 28 , which are received by auto exposure
Other objects of the invention are achieved by providing circuitry 30 , within camera control unit ( CCU ) 32 . Auto a method and apparatus that uses an exposure value ( EV ) to exposure circuitry 30 may consist of various types of
US 9 , 770 , 163 B2 13 14
methods for controlling the electronic shutter of imager 26 , being too far from the imaged subject to be “ in use " ) , Thus , as well as adjusting amplification gain in response to illu the output intensity is decreased to a safe level . If no 416 , mination levels received by imager 26 . Typically , within the then the method checks whether PB meas is less than PB1 , 424 . field of video endoscopy , auto exposure circuitry has high Perimeter Black ( PB ) is typical in endoscope images . Endo speed and wide dynamic range capabilities . Various methods 5 scopes generally provide a circular image of the tissue being may be utilized , that are well known within the art . Video examined in the middle of the overall image , surrounded by display 36 , receives signals from CCU 32 , where an image a black perimeter extending to the square or rectangular of tissue 14 is presented . edges of the overall image . The Perimeter Black ( PB ) being As shown in FIG . 1 , light source 16 is controlled by CCU absent in the image ( i . e . , PB meas is less than PBtr ) is
32 and controller 234 , by means of CCU bus interface 54 , 10 indicative of the camera being disconnected from the endo digital communication bus 50 , and light source bus interface scope . Therefore , if yes 428 , then the camera is disconnected 52 . Controller 234 , may be any type of device designed to and the output intensity is decreased to a safe level . If no receive and execute software programs , or which is designed 426 , then the system checks whether IMMmeas is less than to be modified in functionality by software programs , and IMM , 434 . If yes 438 , then motion is not detected within the preferably is from the group consisting of digital signal 15 video images and the output is decreased to a safe level . If processors , microcontrollers , and microprocessors , or the no , then the Controller checks to see if an output level group consisting of field programmable gate arrays , and change is allowed . If no output change is allowed ( no ) , then computer programmable logic devices . the light source output level is unchanged . If an output
Typically , high intensity light sources utilize an incandes - change is allowed ( yes ) , then the Controller checks if cent bulb 38 ( being a Xenon bulb , or other type ) , driven by 20 Scanning of Power ( light output intensity ) is complete . If an amplifier 40 , which in turn is controlled by output control Power Scan is completed , the Exposure Method EV is circuitry 42 , to set the light output intensity level of the light allowed / enabled . If Power Scan is not completed , EV source 16 . Other types of light source intensity output method is disabled while Power Scan and Correlation meth control are known within the art ; such as mechanical dia - ods are enabled . The next step involves the Exposure Value phragm or iris , liquid crystal shutter , rotary reed or slot 25 Method 220 if EV method is enabled . The exposure value devices , and the like . These various types of light source method involves having the optimal value of EV , EV opt , be output intensity control may be utilized within the scope of between EV opt . min and EV opt . mar , wherein EV opt depends on the present invention . In the present embodiment , output the monotonic and linearity characteristics of the light control circuitry 42 varies the intensity of bulb 38 in source . accordance with controller 234 . 30 The exposure value method 220 first checks to see if EV ;
Controller 234 is a modified controller that is used to is less than EVopt . min 444 . If yes 448 , then EV is below the achieve the various objects of the invention . As shown in optimal range and the output level of the light source is FIG . 1 , controller 234 involves various steps A - F that allow decremented . If no 446 , next the method checks to see if EV ; the apparatus to control light sources that have slow - re - is greater than EV opt . max 454 . If yes 458 , then EV is above sponse , high - frequency noise , nonlinearity , non - monotonic 35 the optimal range and the output level of the light source is response times . Controller 234 may have steps A ) Adaptive incremented . If no 456 then , EV is in the optimal range ( EV , Normalization of EV and Self Calibration 205 , B ) Detection is between EV opt , min and EV opt . max ) and the output level of of Hot Distal - End Method 210 , C ) Failure - Detection the light source is unchanged . The criteria to form a well Method 215 , D ) Exposure Value Method 220 , E ) Power exposed image typically requires that EV meas be between Scan and Correlation Methods 225 and F ) Set Delays 40 EV opt , min and EV opt , max . The EV meas is continuously mea Method 226 . Such steps are shown in FIG . 1A . FIG . 1B sured exposure value EV . The EVopt depends on EV meas , the shows steps for the Scan Request 230 , Power Scan 235 , and monotonic and linearity characteristics of the light source Correlation 240 . Scan Request 230 , Power Scan 235 , and output . Correlation 240 are part of the Power Scan and Correlation The absence of intensity is compensated by increasing of Method 225 . 45 EV in the CCU / imager that is measured ( EV meas ) for the
FIG . 3 shows a global schematic view of the steps of the LSC . The larger the EV meas , the worse quality of the image controller 234 . FIG . 3A - 3B show steps of FIG . 3 . because the image is under exposed . To improve quality of
The step 210 is the Detection of Hot Distal - End Method . the image , the intensity is increased . Thus , when EV meas is Here , the step 210 measures EV 402 and checks to see if greater than EV opt . max ( where EV . pt . max is the upper range of EV meas is less than the threshold EV 7 , 404 . If yes 408 , then 50 the optimal window for EV meas ) , LSC requests increasing the tissue is too close to the hot distal end and the light the light intensity . source intensity is reduced by lowering the output intensity When the end of the scope is touched by the patient to avoid high temperature at the distal end . The output tissue / surface , EV will be reduced by CCU to prevent from intensity is decreased to a safe value , as when EV en is less overexposure ( when image is one bright spot ) . As shown in than EV , then the tissue is too close to the distal end of the 55 FIGS . 3 , 3A and 3B , LSC helps the CCU / imager by decreas endo cope and the output intensity is lowered to avoid high ing light intensity when CCU reduces EV . temperature at the distal end that may cause the tissue to In other words , the LSC helps the CCU / imager keep burn . EV meas in the optimal range , i . e . , EV shall not be too big ( by Next , Failure - Detection Method step 215 is provided . incrementing the intensity ) and EV shall not be too small ( by
Here , the method checks to see if Luma meas is less than 60 decrementing the intensity ) . Luma , , 414 . If yes 418 , this indicates that low or no light is The goal of the exposure value step and / or method is to detected by the camera ; therefore , the light guide / source is minimize the number of light source output level changes possibly either disconnected from the endoscope or endo ( increment or decrement ) taking into account known issues scope / camera combination , the distal tip of the endoscope is with Xenon light sources . not within close enough proximity to a subject being imaged 65 The goal of the exposure value method 220 is to minimize to produce well exposed images ( i . e . , reflected light levels the number of output intensity level changes and avoid loops are below the Luma threshold , indicative of the endoscope in output intensity requests ( i . e . , avoids instability of the
771eas
15
meas
US 9 , 770 , 163 B2 16
output intensity control loop when the controller wrongfully software implementation and / or package for all imagers and requests periodical increment / decrement of output ) . Another re - computes calibration curves based on the type of imager goal of the exposure value method 220 is to increase the and light source . accuracy of computing correlation and minimize the number T he adaptive normalization and self - calibration methods of such computations , and as a result , to minimize the 5 may also include steps for the self - recovery method . The number of changes of light source output intensity . self - recovery method involves , such that when components
The next step is power scan step 225 , which involves scan of the system are changed , updating the previously used request step 230 , power scan step 235 and if requested , processing coefficients , equations , and / or calibration curves correlation step 240 . These steps allow for the incrementing associated with the previous components . These processing of the output intensity of the light source . Here , the output 10 coefficients , equations , and / or calibration curves associated
with the previous components are no longer valid or accu intensity of the light source may be incremented via the rate when components of the system are changed . Keeping power scan step 225 , power scan step 225 being able to these old values could lead to a wrong computation of a new increment the output intensity a few times with a specified safety level . step AP scan , from Pscan . min to Pmax , where Pscan is less than man 15 When components in the system are changed , the self Pmax , and Pmax is the maximum allowed light source output recovery method automatically fixes the above issue by intensity . The above mentioned step is typically not con re - computing the coefficients , equations and calibration tinuously enabled to minimize possible flickering of the curves taking into account new correct equations and / or video images . The next step is Set Delays method 226 to Look Up - Tables in order to compute the safe power - level compensate for the slow response of Xenon or Xenon - like 20 correctly . In other words , the self - recovery method works as light sources by using optimal delays before and / or after auto - adaptive method that allows for high - accuracy of com controlling of light source . putation of a safe power level or output intensity level during
The correlation method may compute a correlation metric the change of components and / or equipment in a system . when EV mear is too small , i . e . , when P , is requested to be Components that are replaced and / or changed are the type around Poor higher ( where P is the maximum output 25 of scope attached to the camera , the light source , the camera , level of the operating light source ) . While computing the any endoscopes , videoscopes and / or CCU . correlation metric , instead of light - modulation , one or a few As an example using the self - recovery method , the LSC sets of incrementing intensity output levels are requested : implemented from GI - CUU and Image - 1 HD CCU can each set producing a ramp of increasing light output inten self - recover when a type of the scope / light source is changed sity . 30 ( for example , when 10 mm scope is replaced with 5 mm
FIG . 2 is a graph of the power scan step 225 . Here , the scope ; or when Xenon - 300 is replaced with Power LED graph shows regions where the correlation method is light source during the surgery ) .
Furthermore , the self - recovery method may be incorpo enabled or disabled and where there is a safe intensity output rated into other operative methods of the system and / or level . 35 method of the invention . As shown in the graph , where the correlation method is The method and apparatus of the present invention has enabled and the EV method is disabled , the output intensity advantages over existing systems . As shown in FIG . 4 . the can be incremented . The output intensity can be incremented method overcomes slow - response , high - frequency noise , via steps from Pscan . min to Pmax , and vice - versa . Optionally , nonlinearity , and non - monotonic response features of Xenon the output intensity may be decremented via steps from P max 40 light sources and light sources similar to Xenon sources . Pscan , min . When a potentially unsafe condition is detected , FIG . 4 shows a time versus light source output graph the correlation method is enabled and the EV method is illustrating high - frequency noise , as well as slow - response disabled . On the right hand side of the graph , upon comple - ( more than a second to change light source output intensity tion of the correlation method one or more times , if the from 5 % to 100 % ) . On the right , FIG . 4 shows an exposure correlation between Luma , neas and incrementing output level 45 value EV versus output intensity graph on a semi - log scale is not detected , the output intensity may be set to a safe level . demonstrating the nonlinearlity and non - monotonic
The method may also involve adaptive normalization and response of Xenon based light sources . These features ( i . e . , self - calibration method steps . The adaptive normalization the slow - response , high - frequency noise , nonlinearity , and and self - calibration method steps take into account the type non - monotonic response features of Xenon light sources ) do of the imager ( i . e . , video endoscope or camera head ) , 50 not allow control methods involving any type of modulation connected to the CCU , and the type of light source . ( such as described in U . S . Pat . No . 6 , 511 , 422 to Chatenever )
The adaptive normalization and self - calibration method to satisfactorily control Xenon light source outputs . steps may normalize EV to the imager being used in order The present invention may also operate to prevent fire to utilize a single LSC software implementation , to reuse the hazard through light guide disconnection and to minimize same control - equations , and to re - compute calibration 55 light source intensity output in - vivo to minimize tissue burn . curves based on the type of imager and light source . The The present invention may also provide eye safety , and adaptive normalization and self - calibration method steps detection of camera and / or light guide disconnection . The may monitor the integration time of the imager , and the invention achieves a single software package that does not minimum and maximum values of EV ; . impact existing hardware .
The adaptive normalization and exposure value methods 60 The controller of the present apparatus takes into account may take into account the type of imager ( i . e . , camera and the exposure value ( i . e . , the shutter speed in seconds ) . The camera head ) connected to the CCU , including imager invention may also provide a correlator that is a part of the format ( i . e . PAL , NTSC , SECAM , etc . ) resolution , ( i . e . controller . frame size or field size for interlaced imagers ) , and type of The invention may also involve providing software pack light source . The method may adaptively normalize EV ; to 65 ages that do not impact existing algorithms for power the imager being used , in order to re - use the same control control . The invention also supports various imager formats equations , the same control - thresholds , and a single LSC ( i . e . NTSC , PAL , SECAM , etc . ) , as well as varying resolu
US 9 , 770 , 163 B2 18
tions , frame and field sizes as in interlaced imagers . The hot to increment or decrement an output intensity of the distal end method step may involve checking if measured Xenon light source via the software executing on the exposure value EV meas is below the threshold EV + , and if so controller , the software executing on the controller the light source intensity is immediately reduced to prevent incrementing the output intensity of the Xenon light overheating / burning of tissue due to the small distance 5 source with a specified step AP scan , from Pscan . min to between the distal end of the scope and the observed tissue . Pmax , where Pscan , min is less than Pmar , and Pmax is a
The failure - detection method may include methods to maximum allowed output intensity of the Xenon light detect the following failures to reduce the light source source ; and intensity to a safe level : light guide / source disconnect , wherein the controller is further adapted to compute , camera disconnect , and detection of no movement within the 10 during the incrementing , a correlation metric indicative video image . This may be based upon comparison of mea of a correlation between a measured achromatic image sured Luma , Perimeter Black ( PB ) , and Image motion metrics ( IMM ) with corresponding thresholds , i . e . , Lumar , brightness ( Lumaneas ) , on Image Motion Metrics or PB , and IMM . . Perimeter Black , and the output intensity of the Xenon
The design style / workflow for the method for LSC may 15 light source , within a specified step , and to set the involve an effective and fast - to - market style , such as the output intensity of the Xenon light source to a safe style provided in http : / / www . mil - embedded . com / articles / id / level , if a correlation between the measured achromatic 24881 , an article by Joy Lin , entitled “ Developing next - gen image brightness and the incrementing output intensity signal processing and communications systems : Engineer of the Xenon light source is not detected . ing tools and design flow advancements ” , which is herein 20 2 . The apparatus of claim 1 , further comprising a com incorporated by reference . As an example the Model - Based munication bus coupled to a plurality of bus interfaces for design workflow may involve various Matlab tools for communication between the Xenon light source and the design , test and verification of LSC , and for one - button push CCU . conversion of the proved Matlab LSC code into the corre 3 . The apparatus of claim 1 , wherein the Xenon light sponding C / C + + / VHDL / Verilog - code for hard / firm - ware 25 source has at least one of high - frequency noise , slow implementation . This allows fast - to - market of the reusable response time , nonlinearity , and non - monotonic response LSC design and prospective upgrades , changes , or modifi time . cation of LSC . 4 . The apparatus of claim 1 , wherein the examining While the invention has been specifically described in instrument is selected from a group consisting of an endo connection with certain specific embodiments thereof , it is 30 scope , laryngoscope , bronchoscope , fiberscope , duodeno to be understood that this is by way of illustration and not of scope , gastroscope , flexible endoscope , arthroscope , cysto limitation and that various changes and modifications in scope , laparoscope , anoscope , and sigmoidoscope . form and details may be made thereto , and the scope of the 5 . The apparatus of claim 1 , wherein the imager is either appended claims should be construed as broadly as the prior
art will permit . located distally inside the examining instrument , proxi The description of the invention is merely exemplary in arv in mately inside the examining instrument , or externally from
nature , and thus , variations that do not depart from the gist the examining instrument . 6 . The apparatus of claim 1 , wherein the imager is a CCD of the invention are intended to be within the scope of the
invention . Such variations are not to be regarded as a or CMOS imager . 7 . A method for controlling a Xenon light source in an departure from the spirit and scope of the invention . 40
What is claimed is : apparatus configured to view a surface , the method com 1 . An apparatus for viewing a surface , the apparatus Pro prising :
incrementing or decrementing an output intensity of the comprising : an examining instrument having an imaging path through Xenon light via at least two steps by a predetermined which the surface is observed ; percentage of a maximum output intensity of the Xenon
a Xenon light source illuminating the surface ; light source ; an imager detecting light reflected from the surface , and detecting a potential safety issue and , if a potential safety
generating image signals representative of light issue is detected , incrementing the output intensity of the Xenon light source with a specified step AP scan , reflected from the surface ;
a camera control unit ( CCU ) processing the image sig - 50 from Pscan , min to Pmax , where Pscan , min is less than P max . nals ; and and where Pmax is a maximum allowed output intensity
of the Xenon light source ; and a controller associated with the imager and the CCU , the controller processing the image signals , the controller computing , during the incrementing , a correlation - metric
indicative of a correlation between a measured achro including software executing on the controller , the matic image brightness ( Lumamead ) , on Image Motion software executing on the controller decrementing or 55 Metrics or Perimeter Black , and the output intensity of incrementing an output intensity of the Xenon light the Xenon light source , within a specified step , and source via at least two steps , wherein the intensity of the at least two steps is incremented or decremented by setting the output intensity to a safe level , if a corre a predetermined percentage of a maximum output lation between the measured achromatic image bright intensity of the Xenon light source ; ness and the incrementing output intensity of the Xenon
wherein the controller is adapted to detect a potential light source is not detected . safety issue and , if a potential safety issue is detected , * * * * *
Scan ?
zees
60