optimal robot scheduling for web search engines

23
HAL Id: inria-00073372 https://hal.inria.fr/inria-00073372 Submitted on 24 May 2006 HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés. Optimal Robot Scheduling for Web Search Engines E. G. Coffman, Zhen Liu, Richard R. Weber To cite this version: E. G. Coffman, Zhen Liu, Richard R. Weber. Optimal Robot Scheduling for Web Search Engines. RR-3317, INRIA. 1997. inria-00073372

Upload: others

Post on 19-Oct-2021

13 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Optimal Robot Scheduling for Web Search Engines

HAL Id: inria-00073372https://hal.inria.fr/inria-00073372

Submitted on 24 May 2006

HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.

Optimal Robot Scheduling for Web Search EnginesE. G. Coffman, Zhen Liu, Richard R. Weber

To cite this version:E. G. Coffman, Zhen Liu, Richard R. Weber. Optimal Robot Scheduling for Web Search Engines.RR-3317, INRIA. 1997. �inria-00073372�

Page 2: Optimal Robot Scheduling for Web Search Engines

ISS

N 0

249-

6399

ap por t de r ech er ch e

THÈME 1

INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET EN AUTOMATIQUE

Optimal Robot Scheduling for Web Search Engines

E. G. Coffman, Jr., Zhen Liu, Richard R. Weber

N˚ 3317

Décembre 1997

Page 3: Optimal Robot Scheduling for Web Search Engines
Page 4: Optimal Robot Scheduling for Web Search Engines

Unité de recherche INRIA Sophia Antipolis2004, route des Lucioles, B.P. 93, 06902 Sophia Antipolis Cedex (France)

Téléphone : 04 93 65 77 77 - International : +33 4 93 65 77 77 — Fax : 04 93 65 77 65 - International : +33 4 93 65 77 65à partir du 01/01/1998

Téléphone : 04 92 38 77 77 - International : +33 4 92 38 77 77 — Fax : 04 92 38 77 65 - International : +33 4 92 38 77 65

������������ ���������������������� ��!�"$#%��&(' ���)�*���&��+�-,.!�"�� !���/

0*132416587:9<;>=:?<@3A:BC1D@ E<F+GH?JI�KDLM@ N�KDO%F�=:B�PJNQ1�RSGCT6GCB

U�VXWZYQ[�\^]`_�a%bc[edCfhg4[Zi�bkjhbkicWZYQ[%b

l�mcnCop[Zi^qsrkt:Uu_�v8w

_8dCxyx�nzmki�{y[*mc[%| VX[Zmc|cVX[~}��M���h\��4]���a%|%[ZY��ymc[�\e������] \e��x�dC��[%b

�����e�e�H�y����� v�mcnz��nzi���b�{y[Zxy��n�j�[%{���j�d>� [Z�¡b [edCmc|cV¡[Z}y���¢}X[£�¢}�nzmc{y[Zm�icn¤Y�dC�¢}�i dC�¢}¡ikVX[£|Zfymkmc[Z}X|Zj�nC¥^�¢icb{XdCi d£��dzb [�nC¥¦� [Z�§x�dC��[%be¨�U�Vy��b*x�dCx+[Zm©bkikfX{h��[%b*mcnz�+nzi©bc|cVX[%{hfy�¢�¢}y�>x�nz�¢��|Z��[%b*ikV�dCi�Y��¢}y�¢Y��¢ªe[�ikVX[�¥«m dz|Zik��nz}Xb�¬%­nC¥®ik�¢YQ[�x�dC��[%b^bcx�[Z}X{snzfyip¯°nC¥±¯°{XdCic[�²�dzbcbkfyY��¢}y�³�¢}X{y[Zx+[Z}X{y[Z}�i*l�nz��b bcnz}sx�dC��[´¯°| V�dC}y��[�xymcnµ|%[%bcb [%be²�dC}X{¤d���[Z}X[Zm dC�{h��bkikmk�¢�yfyik��nz}�¥¶nzm�ikVX[³x�dC��[£dz|%|%[%b b�ik�¢YQ[³·¸¨§� [£bkVXne¹ºikV�dCie²®�»¥8·)��b�{y[%|Zmc[edzbc[%{��¢}JikVX[£�¢}X|Zmc[edzbk�¢}y�z¯°|%nz}�¼�[´gnzmc{y[Zmk�¢}y��bc[Z}Xbc[�²�ikVX[Z}�¬%­���b½{y[%|Zmc[edzbc[%{�¥¶nzm½dC�¢��¾3fy}X{y[Zm¿dC}�j�bc|cVX[%{hfy�¢�¢}y��x+nz�¢��|Zj�²µdC}X{�ikV�dCie²��¢}Qnzmc{y[Zm½icn*Y��¢}y�¢Y��¢ªe[[´ghx�[%|Zic[%{Àicnzi dC��nz�Xbcnz��[%bc|%[Z}X|%[©ik�¢YQ[�nC¥®dC}�j£x�dC��[�²XikVX[�dz|%|%[%b bc[%buicn�ikV�dCi�x�dC��[�bkVXnzfy��{4�+[©dzb�[Z¼�[Z}y�¢jÀbkx�dz|%[%{4�¢}ik�¢YQ[�dzb�x�n�bcbc�¢�y��[�¨

� [�ikVX[Z}Á�¢}�¼�[%bkik�¢��dCic[�ikVX[�xymcnz�y��[ZYÂnC¥½bc| VX[%{hfy�¢�¢}y�4icn�Y��¢}y�¢Y��¢ªe[�ikVX[�|%n�bki^¥«fy}X|Zik��nz}.Ã~Ä ­ ¬ ­ÆÅ ¹uVX[Zmc[�ikVX[�Ä ­dCmc[^���¢¼�[Z}£¹¦[Z�¢��V�icb�xymcnzx+nzmkik��nz}�dC�+icn�ikVX[^x�dC��[´¯°|cV�dC}y��[�m dCic[%b�Ç+­p¨<� [����¢¼�[�d©ik�¢��Vµi���nzfy}X{³nz}�ikVX[^x+[Zmp¥ÈnzmkY�dC}X|%[nC¥©bkfX| VÉd§x+nz�¢��|Zj¡dC}X{¡xymcn�¼�[>ikV�dCi�ikVX[>nzxyik�¢Y�dC��¥Èmc[%ʵfX[Z}X|ZjËdCiQ¹uVy��|cVÉikVX[Àmcnz�+nzi�bkVXnzfy��{Édz|%|%[%bcb�x�dC��[s¾©��bxymcnzx+nzmkik��nz}�dC�+icn�Ì�Í�ΰÏy­ÑÐkÒ�Óe²µ¹uVX[Zmc[�Ïy­MÔÖÕØ׿ÙzÒXÚHÛDÜ�ݵÞ8nzic[�ikV�dCi¿ikVy��b½mc[%{hfX|%[%b¦icn���[Z�¢}y��xymcnzx�nzmkik��nz}�dC��icn�Ç+­�¹uVX[Z}·ß��b�d�|%nz}Xbki dC}µie²X�yfyi�}Xnzie²Xdzb�nz}X[©Y��¢��V�i^[´ghx+[%|Zie²h¹uVX[Z}>·àV�dzb^d���[Z}X[Zm dC��{h��bkikmk�¢�yfyik��nz}3¨

Þ8[´ghie²�¹¿[^[Z¼�dC�¢f�dCic[�m dC}X{ynzY��¢ªe[%{4dz|%|%[%bcbk�¢}y��x�nz�¢��|Z��[%b¦¹uVX[Zmc[Z��j�ikVX[�| VXnz��|%[%b¿nC¥�x�dC��[�dz|%|%[%bcb�dCmc[�{y[Zic[ZmkY��¢}X[%{�µj³�¢}X{y[Zx+[Z}X{y[Z}�i^m dC}X{ynzYáb dCY�xy��[%bu¥«mcnzY�ikVX[*{h��bcikmk�¢�yfyik��nz}.â�ã ­Æä ¨�� [©bkVXn�¹(ikV�dCi�¹uVX[Z}ÀikVX[�¹¿[Z�¢��V�icb�Ä ­ �¢}ÀikVX[|%n�bki<¥Èfy}X|Zik��nz}QdCmc[¦xymcnzx+nzmkik��nz}�dC�yicn�Ç+­Æ²zikVX[¿Y��¢}y�¢Y�fyYå|%n�bki<��b®dz|cVy��[Z¼�[%{�¹uVX[Z}QãC­���b<xymcnzx+nzmkik��nz}�dC�yicn�ΰÏy­ÆÐkÒ�Óµæ�ç�¨è<�¢}�dC�¢�¢j�²h¹¦[�xymc[%b [Z}�i�dC}X{ÀdC}�dC�¢j:ªe[�d�VX[Zfymk��bkik��|�x+nz�¢��|Zj�ikV�dCi���bu[%bkx+[%|Z�DdC�¢�¢j�bcfy�¢ic[%{�icn�ikVX[©dzbkj:Y�xyicnzik��|8mc[Z���¢YQ[*nC¥�DdCmk��[©{XdCi d���dzbc[%be¨

é4ê�ë+ìcí�î �Hï<��� � [Z�³t:[edCmc| V�ð�}y���¢}X[%be²µ� [Z��_�nz�+nzicbe²µtµxy��{y[Zmcb�²:tµicn:|cV�dzbkik��|^t:| VX[%{hfy�¢�¢}y�X²Xtµicnµ| V�dzbkik��|^ñ^mc{y[Zmk�¢}y�X¨

ò½ózôcôcõ�ök÷yó�ø:ù:õHø:ú´õ�û�ü�ýHþpÿ������������ ��������þpÿ���� þ������Hý���� �6ÿ!�"�#�$�&%��#'(�*),+�+,- �"��./� þ10�þ('���.�23�#��% þ('(�5476 896;:&<��5+&=�:&+>)?���&�Cý/�@�BA��ÿ��"�#�$��%#�#'(�$C/�D� ÿE2Æþ&6 þ3AGF1�,��%¢û�ü�ýCþpÿ�6 ���#.$H�'��&�Cý����!6 � ÿ��"�@�!6 IJ�

Page 5: Optimal Robot Scheduling for Web Search Engines

��&�����!�!��!����®�Â�®!��(��� &������u�����!�/ �c��/(�Â�����®��&�/Ø��� &���������&������º��� ' ���

��� ����� � � w3[%b8YQnzic[Zfymcb*{y[�mc[%| VX[Zmc|cVX[�{hfÁ� [Z�Áfyik�¢�¢��b [Z}�i*{y[%b8mcnz��nzicb^x�nzfym8Y�dC�¢}�ic[Z}y�¢m��©oknzfym^��[Zfymcb8��dzbc[%b{y[©{ynz}y}Xa%[%b�¨®w6d�x+nz�¢�¢ik��Ê�fX[*{ nzmc{ynz}y}�dC}X|%[ZYQ[Z}µi*{y[%bumcnz��nzicb�{yaZic[ZmkY��¢}X[©�Dd�Y�dC}y��WZmc[�{ynz}µi���[%bux�dC��[%b�� [Z�Àbcnz}�iY���bc[%b��^opnzfym�{XdC}Xb^|%[%bu��dzbc[%b�{y[�{ynz}y}Xa%[%be¨ �¿[Zi�dCmkik��|Z��[�b� �¢}�icaZmc[%bcb [����� nzmc{ynz}y}�dC}X|%[ZYQ[Z}�i*nzxyik�¢Y�dC��{y[%b�mcnz��nzicbx+nzfym��Dd>Y��¢}y�¢Y���b�dCik��nz}¡{y[%b�¥«m dz|Zik��nz}Xb�{hf�ic[ZY�xXb�n�����[%b�x�dC��[%b�}X[£bcnz}�i�x�dzb��Qopnzfyme¨ Þ8nzfXb�bkfyxyx+n�bcnz}Xb�Ê�fX[��[%b�x�dC��[%b�bcnz}�i�YQnµ{h����a%[%b�bc[Z��nz}J��[%b�xymcn:|%[%bcbkfXb�x+nz��bcbcnz}y}y��[Z}Xb��¢}X{yaZx�[Z}X{XdC}µicb�[Zi�ʵfX[Q��[%b�{hfymca%[%b�{ dz|%|%W%b�{y[%bx�dC��[%b�bcnz}µi�{y[%b�¼zdCmk�DdC�y��[%b�dC��aedCicnz�¢mc[%b��¢}X{yaZx�[Z}X{XdC}µic[%b�[Zi���{y[Z}�ik��ʵfX[ZYQ[Z}�iQ{h��bcikmk�¢�yfXa%[%be²�{ynz}�i��Dd4��nz�¿[%bki�}Xnzica%[·¸¨^Þ8nzfXb8YQnz}�ikmcnz}Xb*ʵfX[�bk�<· {ya%|Zmcn���i*{XdC}Xb^��[�bc[Z}Xb8{y[��� nzmc{hmc[�|%nz}µ¼�[´gh[´¯°|Zmcnz��bcb�dC}�ie²6dC��nzmcb�¬%­p²��DdQ¥Èm dz|Zik��nz}¤{hfic[ZY�xXb�n��J�DdÀx�dC��[£¾8}� [%bki�x�dzb���oknzfyme²�{ya%|Zmcn���i�x�nzfym�icnzfyic[%b���[%b�x�dC��[%b�¾8b nzfXb�}� �¢Y�x+nzmkic[£Ê�fX[Z�¢��[³x+nz�¢�¢ik��Ê�fX[{ nzmc{ynz}y}�dC}X|%[ZYQ[Z}µie²¿[Zi�ʵfX[�²®d��X}�{y[�Y�dHgh�¢Y���bc[Zm��� [%bkx+aZm dC}X|%[³{y[��Dd>¥«m d��D| VX[Zfym�{ fy}X[�x�dC��[�²®�� dz|%|%W%b��>|%[Zikic[x�dC��[*x�dCm���[�mcnz��nzi�{ynz�¢i��Zikmc[*��[�xy�¢fXb�a%ʵfy�¢i dC�y��[ZYQ[Z}�i�[%bkx�dz|%a©[Z}4ic[ZY�xXbe¨

Þ8nzfXb�aZikfX{h��nz}Xb�[Z}Xbkfy�¢ic[���[Qxymcnz�y��WZYQ[�{ nzmc{ynz}y}�dC}X|%[ZYQ[Z}µi�x+nzfym©Y��¢}y�¢Y���bc[Zm��Dd³¥¶nz}X|Zik��nz}¸{y[Q|%n��yi�Ã~Ä�­¶¬%­ Ån��£��[%b�Ä�­�bcnz}µiu{y[%bu|%nµ[���|Z��[Z}µicbuxymcnzx�nzmkik��nz}y}X[Z��b^dCfhg�i dCfhg³{y[�| V�dC}y��[ZYQ[Z}�i�{y[%b�x�dC��[%buÇ+­p¨�Þ^nzfXb�{ynz}y}Xnz}Xb�fy}X[�+nzmk}X[*bkfym¿�Dd�x+[Zmp¥¶nzmkY�dC}X|%[*{y[��Dd�x+nz�¢�¢ik��ʵfX[8nzxyik�¢Y�dC��[*[Zi�YQnz}µikmcnz}Xb�ʵfX[��Dd�¥Èmca%Ê�fX[Z}X|%[�nzxyik�¢Y�dC��[©de¼�[%|��DdzÊ�fX[Z�¢��[��[³mcnz�+nzi�dz|%|%W%{y[��>�DdÀx�dC��[£¾�[%bki�xymcnzx�nzmkik��nz}y}X[Z�¢��[ �sÌ�Í�ΰÏh­ÑÐ Ò�Ó ²�n��¡Ïh­�ÔÖÕá×�Ù ÒXÚHÛ±Ü ÝMÞ8nzicnz}Xb�ʵfX[³|%[�{y[Zmk}y��[Zm{y[Z¼:��[Z}�i^xymcnzx�nzmkik��nz}y}X[Z�!��Ç ­ bc[Zfy��[ZYQ[Z}µi^ʵf�dC}X{À·à[%bciufy}X[©|%nz}Xbki dC}�ic[�¨

Þ8nzfXb�aZ¼zdC�¢fXnz}Xb�dCfXbcbc�¦��[%b�x+nz�¢�¢ik��Ê�fX[%b�{ dz|%|%W%b�dC��aedCicnz�¢mc[%b�n��¸��[³|cVXnz�»g�{ dz|%|%W%b�{y[%b�x�dC��[%b�[%bci�{yaZic[ZmkY��¢}Xabc[Z��nz}J��[%b�a%|cV�dC}�ik�¢�¢��nz}Xb�{y[Q�Dds{h��bkikmk�¢�yfyik��nz}Sâ�ãC­ ä ¨ÀÞ8nzfXb�YQnz}�ikmcnz}Xb�ʵfX[�²<ʵf�dC}X{¸��[%b�Ä�­�bcnz}µi�{y[%b�|%n:[���|Z��[Z}�icbxymcnzx+nzmkik��nz}y}X[Z��b®dCfhg©i dCfhg�{y[½| V�dC}y��[ZYQ[Z}�i®{y[%b�x�dC��[%b�Ç+­p²C��[½|%n��yi<Y��¢}y�¢Y�fyY [%bki<dCikic[Z�¢}µi<bc�:ãC­y[%bki6xymcnzx+nzmkik��nz}y}X[Z��>Î°Ï ­ Ð Ò�Ó æ�ç�¨uè<�¢}�dC��[ZYQ[Z}�ie²�}XnzfXb^xymca%bc[Z}�icnz}Xb�[Zi^dC}�dC�¢jhbcnz}Xb^fy}X[©VX[Zfymk��bkik��ʵfX[�Ê�fy�6[%bci�x�dCmkik��|Zfy�¢��WZmc[ZYQ[Z}�i*fyik�¢��[{XdC}Xb���[*|edzb�{ fy}À��m dC}X{4}XnzY��ymc[�{y[*x�dC��[%be¨

" î �e� ì �$# � �Ø� qsnzic[Zfym³{y[4mc[%|cVX[Zmc| VX[s{hf¡� [Z�3²¿mcnz�+nzicb�{hf¡� [Z�3²�nzmc{ynz}y}�dC}X|%[ZYQ[Z}µi£bcicnµ| V�dzbkik��Ê�fX[�²�nzmc{hmc[bkicn:|cV�dzbcik��Ê�fX[�¨

Page 6: Optimal Robot Scheduling for Web Search Engines

������������ ���������������������� �"!$#&%'��(*)+�,�-�����(����/.0!�#��"!��21 �

3 4 !���&��*�����6���c�^!

U�VX[8� [Z�� b¿mcnz��[8dzbud�Y�dHopnzm¿�¢}h¥¶nzmkY�dCik��nz}³xyfy�y�¢��bcVy�¢}y��dC}X{�mc[Zikmk��[Z¼:�¢}y��YQ[%| V�dC}y��bkY�nz}³ikVX[^rÑ}�ic[Zmk}X[Ziu|%nz}�ik�¢}µfX[%bicnÁ��mcne¹º[´ghikmc[ZYQ[Z�¢j ¥ dzbkie¨¤r°}X{y[%[%{�²®ikVX[£dCYQnzfy}µiQnC¥��¢}h¥¶nzmkY�dCik��nz}¡nz}JikVX[�� [Z��V�dzb���nz}y�Ábc�¢}X|%[��+[%|%nzYQ[³icnµn�DdCmk��[Q¥Ènzm©Y�dC}µf�dC�¢�¢j �ymcne¹�bc�¢}y�sikVymcnzfy��V�dC}�j bk�¢��}y����|edC}µi�x+nzmkik��nz}¸nC¥u�¢icb©Vµjµx+[Zmkic[´ghi�bkikmkfX|Zikfymc[�¨4rÑ}Xbkic[edz{�²6ikVX[� [Z�¤n65+[Zmcb�bc[edCmc| VÁ[Z}y���¢}X[%b^¥¶nzm�ikVy��b8xyfymkx+n�bc[87+v��¢i d9^��bki dh²�w�jh|%n�be²�r°}h¥¶n�bc[%[;:+²�q¤dC��[Z�¢�DdC}3²+ðMgh|Z�¢ic[�²�dC}X{=<¦dCVXnµndCmc[��yfyi�dÀ¥¶[Z¹~nC¥�ikVXn�bc[£d%¼�dC�¢�DdC�y��[�¨sU�VX[%b [�bkjhbkic[ZYQb�|%nz}Xbk��bki�nC¥��¢}X{y[´g:�¢}y�¤[Z}y���¢}X[%b�¥Ènzm�|%nz}XbkikmkfX|Zik�¢}y� dÀ{XdCi d��dzbc[³nC¥�� [Z�.x�dC��[%b�²®dC}X{.�¢}¸Y�dC}�j.|edzbc[%b � î � î �e� ¥¶nzm��ymk�¢}y���¢}y�Á�¢}h¥ÈnzmkY�dCik��nz}�icn>ikVX[��¢}X{y[´gh�¢}y�Á[Z}y���¢}X[�¨sU6nY�dC�¢}µi dC�¢}s|Zfymkmc[Z}X|ZjÀdC}X{À|%nzY�xy��[Zic[Z}X[%bcb^nC¥<ikVX[*{XdCi d���dzb [�²hmcnz��nzicbux+[Zmk��n:{h��|edC�¢�¢j�Y�d6:�[*mc[%|Zfymcbc�¢¼�[*ikm d%¼�[Zmcb dC��b^nC¥ikVX[�� [Z�� b®V�j:x+[Zmkic[´g:i¿bkikmkfX|Zikfymc[��µj�dz|%|%[%bcbk�¢}y�©x�dC��[%be²µikVX[Z}QikVX[�x�dC��[%b®mc[´¥¶[Zmc[Z}X|%[%{��µj�ikVX[%b [�x�dC��[%be²:dC}X{�bcn©nz}3¨U�VX[Zmc[8dCmc[�|Zfymkmc[Z}�ik�¢j�¹¿[Z�¢��ne¼�[Zm8\,>8>�mcnz��nzicb½nz}QikVX[u� [Z�3¨Mqsn�bci®nC¥+ikVX[ZYÀ²��¢}X|Z�¢fX{h�¢}y��ikVX[�nz}X[%b½nC¥+�¢}µic[Zmc[%bki®VX[Zmc[�²dCmc[£{y[%bk�¢��}X[%{¸¥¶nzm�mc[%bcnzfymc|%[£{h��bc|%n�¼�[Zmkj�²��yfyi�nzikVX[Zmcb�dCmc[�fXb [%{ ¥¶nzm�Y��¢mkmcnzmk�¢}y�X²½Y�dC�¢}�ic[Z}�dC}X|%[@?«�¢�¢}A:¸|cVX[%|B::�¢}y�$C´²bki dCik��bcik��|%be²X[Zic|�¨®èXnzm�dC}s[´g:ic[Z}Xbk�¢¼�[�{h��bc|ZfXbcbk��nz}ÀnC¥M� [Z�Àmcnz�+nzicbe²ybc[%[�ikVX[*¹¿nzm�:£nC¥�q¤dCmkik� oÆ}=D^n�bkic[Zm^dCi

E$F�F�G0HJIAKML$N�OQPSRAT�U�VMWAX6RAY$T�W0PBV�O6Z[I6Z[X�\AI�G$W�O�]�T�V^F[_8I8W�O8U�O�F`_8I8W�O�UAO�F`_aPSE$F8Z`Y

rÑ}>ikVX[��¢�¢ic[Zm dCikfymc[�nz}X[ �X}X{yb^YQnzmc[�|%nz��nzmp¥Èfy��ic[ZmkYQb�¥¶nzm�mcnz�+nzie²�bkfX| V¤dzb^¹¿dC}X{y[Zmc[Zme²+|Zm d%¹u��[Zm�nzm�bkxy��{y[Zme²�dC}X{ikVX[³}Xnzik��nz}�nC¥�d>mcnz��nzi��+[Z�¢}y�cb mcnzfyic[%{�icn Mnzm/b ¼µ��bc�¢ik�¢}y� Md>x�dC��[�¨ U�Vy��b�x�dCx+[Zmd:�[%[ZxXb�¹u�¢ikV�ikVX[eb mcnz�+nzi�<dC}X{b dz|%|%[%bcbc�¢}y� yic[ZmkY��¢}Xnz��nz��j4ikVymcnzfy��VXnzfyie¨

� [�bkikfX{hj£xymcnz�y��[ZYQb�nC¥®bc|cVX[%{hfy�¢�¢}y�Àd�mcnz�+nzi�ikV�dCi8Y�dC�¢}�i dC�¢}Xb8ikVX[�|Zfymkmc[Z}X|ZjsnC¥�[´gh��bkik�¢}y��x�dC��[%b��¢}Ábc[edCmc|cVh¯[Z}y���¢}X[©{XdCi d���dzbc[%be¨M� [©dzbcbkfyYQ[�ikV�dCi�ikVX[*bc[Zi�nC¥M{XdCi dH¯ ��dzb [�x�dC��[%b���b �ygy[%{�²y�yfyi�dzbu¹¦[*bcV�dC�¢�6bc[%[�²ynzfymumc[%bkfy�¢icb|edC}¸��[�xymcnzYQnzic[%{�dzb�VX[Zfymk��bkik��|%b©¥¶nzm�{XdCi dÀ��dzbc[%b©ikV�dCi�dz|%Ê�fy�¢mc[�}X[Z¹åx�dC��[%b�dC}X{.{hmcnzx¸nz��{.x�dC��[%b�n�¼�[Zm�ik�¢YQ[�¨vºbkx+[%|Z����|�nz�hok[%|Zik�¢¼�[Q¹u�¢�¢�¦��[�icn��X}X{.mcnz��nzi�bc|cVX[%{hfy��[%b�ikV�dCi�Y��¢}y�¢Y��¢ªe[�ikVX[�nz�Xb nz��[%bc|%[Z}X|%[�nC¥uikVX[�{XdCi dÀ��dzbc[�¢}§bcnzYQ[�fXbc[´¥Èfy�<b [Z}Xbc[�¨*èXnzm©[´gXdCY�xy��[�²3dzbcbkfyYQ[�ikVX[Zmc[QdCmc[df � [Z�Áx�dC��[%b�²+�DdC��[Z��[%{Jç Å2gµÅ Ý%Ý%Ý Å f¸²+¹uVy��| V.dCmc[�icn�+[�dz|%|%[%bcbc[%{ mc[Zx�[edCic[%{h�¢j¤��j d³mcnz�+nzie²6ikVX[�{hfym dCik��nz}.nC¥�[edz|cV¸dz|%|%[%bcb���[Z�¢}y�ÁdC} �¢}X{y[Zx�[Z}X{y[Z}µi�b dCY�xy��[�¥«mcnzYßd���¢¼�[Z}£{h��bkikmk�¢�yfyik��nz}3¨®v8bcbkfyYQ[8dC��bcn�ikV�dCi¦ikVX[^|%nz}µic[Z}�icbunC¥3x�dC��[8¾�dCmc[�YQn:{h����[%{£dCi¦ik�¢YQ[%b¿ikV�dCi®¥¶nz�¢��ne¹(d©l<nz��bcbcnz}xymcn:|%[%bcb�¹u�¢ikV£x�dCm dCYQ[Zic[Zm�Ç+­p¨�v�x�dC��[���b�|%nz}Xbk��{y[Zmc[%{4fyxh¯ icnC¯°{XdCic[���j�ikVX[��¢}X{y[´g:�¢}y��[Z}y���¢}X[8¥ÈmcnzY ikVX[�ik�¢YQ[^�¢i���bdz|%|%[%bcb [%{���j�ikVX[umcnz�+nzi®fy}µik�¢��ikVX[u}X[´ghi®ik�¢YQ[u�¢i®��b½YQnµ{h����[%{�²hdCi®¹uVy��| V�x+nz�¢}�i®�¢i®�+[%|%nzYQ[%b½nzfyip¯°nC¥«¯°{XdCic[^fy}�ik�¢��ikVX[mcnz�+nzi� b�}X[´g:i*dz|%|%[%bcbe¨*w6[Zi�¬e­®��[�ikVX[�¥«m dz|Zik��nz}§nC¥½ik�¢YQ[�x�dC��[�¾�bkx+[Z}X{yb8nzfyip¯°nC¥«¯°{XdCic[�¨©U�VX[�xymcnz�y��[ZY��b8icn��X}X{mc[Z�DdCik�¢¼�[�x�dC��[´¯Ñdz|%|%[%b b8¥«mc[%ʵfX[Z}X|Z��[%b�dC}X{§d³bc[%Ê�fX[Z}X|Z�¢}y�4x+nz�¢��|Zj>ikV�dCi*mc[edC�¢�¢ªe[%b©ikVX[%bc[�¥«mc[%ʵfX[Z}X|Z��[%b*bkfX|cV§ikV�dCi*ikVX[nz�hok[%|Zik�¢¼�[�¥«fy}X|Zik��nz}ih Õ Ã Ó�j ­ j�k Ä�­¶¬%­p²���b^Y��¢}y�¢Y��¢ªe[%{�²�¹uVX[Zmc[�ikVX[�Ä�­¦dCmc[����¢¼�[Z}s¹¿[Z�¢��V�icbe¨&l�}X{y[Zm�bk�¢Y�xy�¢�»¥Èjµ�¢}y��yfyiMxy�DdCfXbk�¢�y��[udzbcbkfyY�xyik��nz}XbMnz}�ikVX[¦¹¦[Z�¢��V�icb�²�x�dC��[�dz|%|%[%bcb<ik�¢YQ[%be²�dC}X{�ikVX[¦|Z�DdzbcbMnC¥�dC�¢��n�¹¦[%{�x+nz�¢��|Z��[%be²z¹¦[�nz�yi dC�¢}bcnz�¢fyik��nz}Xb�icn�ikVy��buxymcnz�y��[ZYÀ¨

èXmcnzYåd�ikVX[%nzmc[Zik��|edC�Xx+nz�¢}�i�nC¥�¼:��[Z¹©²�nzfymMxymcnz�y��[ZY ��bM|Z��n�bc[Z�¢j©mc[Z�DdCic[%{�icn^ikVXn�bc[¿Y�fy�¢ik�¢xy��[´¯°ÊµfX[ZfX[�bk�¢}y����[´¯°bc[Zmk¼�[Zmbkjhbkic[ZYQbufXbcf�dC�¢�¢j£|edC�¢��[%{ �������m��!$#n12o61��p�;�q1 �¢}ÀikVX[©ÊµfX[ZfX[Z�¢}y�Q�¢�¢ic[Zm dCikfymc[�¨¦rÑ}X{y[%[%{�²yikVX[*mcnz�+nzi�|edC}À�+[*|%nz}Xbk��{y[Zmc[%{dzb�ikVX[�bc[Zmk¼�[Zm©dC}X{ÁikVX[�x�dC��[%b©dzb8ikVX[�bki dCik��nz}Xb��¢}ÁikVX[�x�nz�¢�¢�¢}y�4bkjhbkic[ZYÀ¨*U�VX[�{hfym dCik��nz}Xb*nC¥¦|%nz}Xbc[%|Zfyik�¢¼�[�x�dC��[dz|%|%[%bcb [%b^|%nzmkmc[%bkx+nz}X{Àicn�bk¹u�¢ic|cVh¯°n�¼�[Zm*ik�¢YQ[%b8dC}X{>ikVX[©x�dC��[�YQnµ{h����|edCik��nz}Xb�|%nzmkmc[%bkx+nz}X{>icn�|ZfXbkicnzYQ[Zm*dCmkmk�¢¼zdC��be¨�7�áÿ8r9<&<MsBt

Page 7: Optimal Robot Scheduling for Web Search Engines

� .��������Q�����+8!���( ��������Q�"��a ��� ��&)+�,�B��(

U�VX[Qb [Zmk¼µ��|%[�ik�¢YQ[%b*�¢}.ikVy��b*x�nz�¢�¢�¢}y�Àbcj:bkic[ZYàdCmc[�ªe[Zmcny¨�v�}§[´ghic[Z}Xbk�¢¼�[��¢�¢ic[Zm dCikfymc[�[´gh��bkicb*nz}§ikVX[�dC}�dC�¢jhbk��b©dC}X{|%nz}µikmcnz�<nC¥�x+nz�¢�¢�¢}y��bkjhbkic[ZYQbe¨¦U�VX[©�¢}�ic[Zmc[%bcic[%{>mc[edz{y[Zm^��b�mc[´¥¶[Zmkmc[%{Àicn�ikVX[���n:n�:³nC¥®U<d6:zdC�����¢\ �y²3\����3¥¶nzm���[Z}X[Zm dC�mc[´¥¶[Zmc[Z}X|%[%b,7MikVX[Qbkx+[%|Z�DdC�®��bcbcfX[QnC¥¿ikVX[©opnzfymk}�dC��� �[���[����!$# �`o61��p���d1 ² 9®nz� ¨Á\�\=?p\e������C*nz} x+nz�¢�¢�¢}y�ÀYQn:{y[Z��b�dC}X{ikVX[8mc[%|%[Z}�i�ikVX[%bk��b�nC¥���nzmcbki � �!��|edC}³�+[8|%nz}Xbkfy�¢ic[%{�¥Ènzm¿YQnzmc[8mc[%|%[Z}�iu{y[Z¼�[Z��nzxyYQ[Z}�icbe¨¦r°}³x�dCmkik��|Zfy�DdCme²hikVX[^x�nz�¢�¢�¢}y�bkjhbkic[ZYQb�¹u�¢ikV�ªe[Zmcn�bc[Zmk¼:��|%[�ik�¢YQ[%b3¹¿[Zmc[½YQnzik�¢¼�dCic[%{*�µj©|%nzY�Y�fy}y��|edCik��nz}�}X[Zi°¹¦nzm�:hbMbkfX|cV�dzb �p��� �;�p��"8� dC}X{$# � �������p��"¹uVX[Zmc[�x�dC��[%b�nC¥��¢}h¥ÈnzmkY�dCik��nz}�dCmc[¿icn8��[¿�ymcn�dz{y|edzbki�icn8ic[ZmkY��¢}�dC��b®|%nz}y}X[%|Zic[%{�icn*d8|%nzY�xyfyic[Zm®}X[Ziѹ¦nzm�:%�¢\�²'&X²µ\e�'�°¨

(8ne¹¿[Z¼�[Zme²3ikVX[�xymcnz�y��[ZYVX[Zmc[�V�dzb�}Xnzi��+[%[Z}§dC}�dC�¢j:ªe[%{�¨�r°}X{y[%[%{�²+�¢}§ikVX[�fXbkf�dC�Mx+nz�¢�¢�¢}y�>dC}�dC�¢j:bk��b�²+�¢}�ic[Zmc[%bki|%[Z}µic[Zmcb�nz}�YQ[edC}¡¹�dC�¢ik�¢}y�.ik�¢YQ[%b�dC}X{¡YQ[edC}ËÊ�fX[ZfX[À��[Z}y��ikVXbe²½¹uVX[Zmc[edzb��¢}Ënzfym�xymcnz�y��[ZYÀ²¿ikVX[4x+[Zmp¥ÈnzmkY�dC}X|%[YQ[edzbkfymc[¦nC¥y�¢}�ic[Zmc[%bkie²C¼:�¢ª�¨¢²HikVX[½nz�Xbcnz��[%bc|%[Z}X|%[®ik�¢YQ[�²z|%nzmkmc[%bkx+nz}X{yb�icnuikVX[®Y�dHgh�¢Y�fyY�¹¿dC�¢ik�¢}y�8ik�¢YQ[½nC¥�du|ZfXbkicnzYQ[Zm{hfymk�¢}y�£d�¼µ��bk�¢i�|Zjh|Z��[*nC¥MikVX[©bc[Zmk¼�[Zme¨�)^f�dC}µik�¢ik��[%b^nC¥MikVy��b�::�¢}X{£V�d%¼�[©}Xnzi���[%[Z}>dC}�dC�¢j:ªe[%{4�¢}ÀikVX[�x�dzbkie¨

U�VX[³}X[´ghi�b [%|Zik��nz}J��b�{y[Z¼�nzic[%{Jicn§dÀxymc[%|Z��b [Q¥¶nzmkY�fy�DdCik��nz}¡nC¥^nzfym�YQn:{y[Z� ²®dC}X{�dÀmc[Z¼:��[Z¹ºnC¥^bcnzYQ[�fXbc[´¥«fy�|%nz}X|%[Zxyicbu�¢}4bkicn:|cV�dzbcik��|*nzmc{y[Zmk�¢}y��ikVX[%nzmkj�¨¦t:[%|Zik��nz}>����[Z���¢}Xb��µj�xymcne¼:�¢}y��i°¹¿n�xymcnzx+[Zmkik��[%bunC¥�mcnz�+nzi�b |cVX[%{hfy�¢�¢}y�x+nz�¢��|Z��[%b *,+ �.- [´ghx�[%|Zic[%{Ánz�Xbcnz��[%bc|%[Z}X|%[�ik�¢YQ[%b��¢}X|Zmc[edzbc[�dzb^ikVX[�x�dC��[´¯Ñdz|%|%[%bcb�ik�¢YQ[��¢}X|Zmc[edzb [%b��¢}sikVX[��¢}X|Zmc[edzbk�¢}y�z¯|%nz}µ¼�[´gµ¯°nzmc{y[Zmk�¢}y� bc[Z}Xb [�²<dC}X{/+ ���.- ²3��j¸t:| V�fymp¯°|%nz}µ¼�[´g:�¢iÑj¸mc[%bkfy�¢icb�²<dz|%|%[%bcb [%b©icnsdC}µj§���¢¼�[Z}¸x�dC��[�bkVXnzfy��{ �+[�dzb[Z¼�[Z}y�¢j>bcx�dz|%[%{Ádzb^x�n�b bk�¢�y��[�¨�� [�ikVX[Z}Á{y[Zmk�¢¼�[�d�ik�¢��V�i8��ne¹¦[Zm*�+nzfy}X{>nz}ÁikVX[�|%n�bki�¥Èfy}X|Zik��nz}@hådzbcbcfyY��¢}y��ikV�dCiikVX[�¹¿[Z�¢��V�icb^Ä�­�dCmc[�xymcnzx�nzmkik��nz}�dC�6icn�ikVX[*Ç+­p¨�U�VX[%bc[�mc[%bkfy�¢icb�jµ��[Z��{4d�¥¶nzmkY�fy�Dd�¥¶nzm�nzxyik�¢Y�dC�6dz|%|%[%bcb�¥Èmc[%Ê�fX[Z}X|Z��[%be¨U�VX[�}X[´ghi�i°¹¦n�bc[%|Zik��nz}Xbe²�t:[%|Zik��nz}Xb0��dC}X{21h²y{y[edC�+¹u�¢ikV4ikVX[�xymcnz�y��[ZY�nC¥<bc[%ʵfX[Z}X|Z�¢}y��x�dC��[*dz|%|%[%bcbc[%bunzxyik�¢Y�dC�¢�¢j�²nzm�}X[edCm�nzxyik�¢Y�dC�¢�¢j�²½bcn¤dzb�icnÁmc[edC�¢�¢ªe[£ds���¢¼�[Z}�bc[Zi�nC¥8dz|%|%[%bcb�¥Èmc[%Ê�fX[Z}X|Z��[%b�¨§U�Vy��b�Y�dCic[Zmk�DdC�u��b�xymc[´¥ dz|%[%{J��jJd{h��bc|ZfXb bk��nz}sdCi�ikVX[©[Z}X{>nC¥½t:[%|Zik��nz}¤��¹uVy��|cVÁmc[Z�DdCic[%b�nzfym�b |cVX[%{hfy�¢�¢}y��xymcnz�y��[ZYáicn�ikVXn�bc[*ikV�dCi8|%nzYQ[*fy}X{y[Zm�ikVX[VX[edz{h�¢}y�³nC¥<��[Z}X[Zm dC�¢�¢ªe[%{>mcnzfy}X{:¯ mcnz�y�¢}snzm�ic[ZY�xy�DdCic[´¯°{hmk�¢¼�[Z}Ábc|cVX[%{hfy�¢�¢}y�X¨

rÑ}>t:[%|Zik��nz}2�X²:¹¦[��¢}�ikmcn:{hfX|%[�m dC}X{ynzY��¢ªe[%{£x�dC��[©dz|%|%[%bcbk�¢}y�X²:¹uVX[Zmc[*[edz| V>dz|%|%[%bcb¿��bu{y[Zic[ZmkY��¢}X[%{£��j³dC}£� ¨ � ¨ {�¨b dCY�xy��[u¥«mcnzYåd*{h��bcikmk�¢�yfyik��nz}Àâ�ãC­ ä ¨<� [�bcVXne¹¡VXn�¹¡icn��X}X{�ikV�dCi½| VXnz��|%[u¥¶nzm�ikVy��b®{h��bkikmk�¢�yfyik��nz}Q¹uVy��| V�Y��¢}y�¢Y��¢ªe[%bhQ¨�rÑ}�t:[%|Zik��nz}31h²�¹¿[�{y[Z¼�[Z��nzx�d8x+nz�¢��|Zj�ikV�dCi®x+[Zmp¥¶nzmkYQb�¹¦[Z�¢�X¹uVX[Z}fá��b��DdCmk��[�¨MrÑi���b���dzbc[%{�nz}�¹¦nzm�:�nC¥�r°i dC�XdC}X{_^n�bk��[Zmk�4�¢\�\5� ?«�¢}ÁdC}À[Z}�ik�¢mc[Z�¢j4{h� 5+[Zmc[Z}µi^bc[Zikik�¢}y�$C�dC}X{£jµ��[Z��{yb�d�|%n�bciu¹u�¢ikVy�¢}6187 nC¥�nzxyik�¢Y�dC� ¨¦t:nzYQ[©{h�¢mc[%|Zik��nz}Xb¥¶nzm�¥«fymkikVX[Zm�mc[%bc[edCmc|cV¤dCmc[*YQ[Z}�ik��nz}X[%{>�¢}st:[%|Zik��nz}9�h²h¹uVy��|cVÁ|%nz}X|Z�¢fX{y[%b�ikVX[*x�dCx+[Zme¨

: ; &��®��� �Â��!��&�� ��/

w3[Zi¤âe·=< ä ��[sikVX[sb [%Ê�fX[Z}X|%[ÁnC¥�{hfym dCik��nz}Xb4nC¥�|%nz}Xbc[%|Zfyik�¢¼�[Áx�dC��[§dz|%|%[%bcbc[%b���jSikVX[>mcnz�+nzie²�[edz| V�·=<¤��[Z�¢}y�{h��bkikmk�¢�yfyic[%{s�¢}X{y[Zx+[Z}X{y[Z}�ik�¢jsdzb�d�m dC}X{ynzYá¼�dCmk�DdC�y��[©·.¨¿èXnzm8bc|cVX[%{hfy�¢�¢}y�³x�nz�¢��|Zj2>�²y��[Zi?>A@%B â�ç Å2gµÅ Ý%Ý%Ý Å f ä ��[ikVX[�bc| VX[%{hfy�¢�¢}y�Q{y[%|Z��bk��nz}£¥Ènzm¿ikVX[ C�¯ ikV£dz|%|%[%b be²:� ¨ [�¨¢²µikVX[8�¢}X{y[´g£nC¥6ikVX[ C�¯ ikV�x�dC��[8icn���[�dz|%|%[%bcbc[%{³�µj�ikVX[8mcnz��nzify}X{y[ZmD>�¨���[��X}X[8ikVX[ �"!`�p�;( �� � �21 1q�8� 1;�S�!�� �0E ­F ÎG><ÐMÕHC ­F ÎG><Ð+æ6C ­F Ò�Ó ÎG><Ð ²h¹uVX[Zmc[ C ­F ÎG><Ю��b¿ikVX[^�¢}X{y[´g£nC¥6ikVX[0I�¯ ikVdz|%|%[%bcb*nC¥¿x�dC��[�¾c²+� ¨ [�¨¢²JC ­F ÎG><Ð�ÕLK�ÍMZâ CON/C ­F Ò�Ó ÎG><ÐQPR>A@ÀÕ ¾ ä ²�dC}X{s¹uVX[Zmc[$C ­ S ÎG><ÐDTVUh¨�w6[Zi©· ­F Õ · ­F ÎG><Ðu��[ikVX[QI�¯ ikV �"!��p��( � ��� ��� �'������! �����n� nC¥�x�dC��[�¾c²<� ¨ [�¨¢²MikVX[Qik�¢YQ[���[Ziѹ¦[%[Z}JikVX[sÎWI�æ�çzж¯°bki�dC}X{6I�¯ ikV¸x�dC��[´¯°¾*dz|%|%[%bcb

��7�7���

Page 8: Optimal Robot Scheduling for Web Search Engines

������������ ���������������������� �"!$#&%'��(*)+�,�-�����(����/.0!�#��"!��21 1

|%nzY�xy��[Zik��nz}�ik�¢YQ[%be¨�� [®V�de¼�[¿· ­F Õ Ã@ Û�<�� @ Û������� Ó ·=<�²zbcn�ikVX[½m dC}X{ynzY ¼zdCmk�DdC�y��[%b<· ­F dCmc[®Y�fyikf�dC�¢�¢j��¢}X{y[Zx+[Z}X{y[Z}�ie¨

Þ8nzic[�ikV�dCie²+�»¥®x�dC��[�dz|%|%[%bcb^ik�¢YQ[%b�·=<QdCmc[�[´g:x+nz}X[Z}�ik�DdC�¢�¢j¤{h��bkikmk�¢�yfyic[%{�²�ikVX[Z}§· ­F V�dzb*dC}¤ð�mk�DdC}y�À{h��bkikmk�¢�yfyik��nz}nC¥ E ­F bki dC��[%be¨

��(B�� %;�p��( q��"�� �p���&��! �����![����� ��!A1 ����� ����� ���'o > � �����a!���(2����� oe�B�/��� ��� �p�B� %�(���� ����(+!��8�S8��� ��!��&�"!@12�A����B�1'�21 0����� ����� ���'o � ���"� �� � �o61q�B� �;� ���(-��! �B��!`�p��"�� �

w6[Zi�� ­F Õ � ­F ÎG><Ð+��[¦ikVX[¿ik�¢YQ[¿ikV�dCi�x�dC��[�¾���bMnzfyip¯°nC¥±¯°{XdCic[�{hfymk�¢}y�*ikVX[�I�¯ ikV��¢}�ic[Zmc|%nzY�xy��[Zik��nz}Qik�¢YQ[�nC¥�x�dC��[�¾c¨w3[Zi�� ­@ Õ�� ­@ ÎG><Ð��+[®ikVX[¿}�fyY���[Zm®nC¥�dz|%|%[%bcb [%b<nC¥�x�dC��[¦¾3dCYQnz}y��ikVX[ �Xmcbki�C£dz|%|%[%bcbc[%b *�� ­@ Õ Ã @ <�� Ó�� â >R<�Õؾ ä ²¹uVX[Zmc[ � â�� ä ��buikVX[��¢}X{h��|edCicnzm�¥Èfy}X|Zik��nz}3¨�(8[Zmc[edH¥Èic[Zme²y¹¦[�|%nz}Xbk��{y[Zm^nz}y�¢j4bki dCik��nz}�dCmkj4bc| VX[%{hfy�¢�¢}y��x+nz�¢��|Z��[%b��¢}ÀikVX[bc[Z}Xb [�ikV�dCie²h¥Ènzm^[edz|cV>bkfX| VÀx+nz�¢��|Zj�²yikVX[��¢�¢Y��¢i

ãC­+Õ(ãC­ÆÎG><ÐMÕ�ÌWK��@���� � ­@C ?p\MC

[´gh��bkicb�dC}X{���b�bkikmk��|Zik�¢j.x+n�bk�¢ik�¢¼�[�¥ÈnzmQdC�¢�u¾c²uç! º¾" f¸¨¤� [4|edC�¢��ãC­�ikVX[4dz|%|%[%bcb�¥Èmc[%Ê�fX[Z}X|ZjJnC¥�x�dC��[4¾c¨§� [dC��bcn�mc[%ʵfy�¢mc[�ikV�dCi8ikVX[��¢�¢Y��¢icb8ÌWK��=@��#��à @F � Ó � ­F%$ C�dC}X{sÌWK�� @�����à @F � Ó'&)( � ­F+*,$ CJ[´g:��bci8dC}X{Á��[�[%Ê�f�dC� ¨�U�VX[%bc[�Ddzbki�dzbcbcfyY�xyik��nz}XbuVXnz��{£fy}X{y[Zm¿¥¶dC�¢mk�¢j�Y��¢��{4|%nz}X{h�¢ik��nz}Xbe²X[�¨ �X¨¢²:¹uVX[Z}4ikVX[*bc[%ʵfX[Z}X|%[�â E ­F ÎG><Ð ä F ��bubki dCik��nz}�dCmkj³dC}X{[Zmk��n:{h��|n?È|´¥k¨ D��¢}y��Y�dC} �¢\ �'��C´¨

U�VX[ ����1;��� �21;�2��!�� �q(�8�p� ¬ ­ Õج ­ ÎG><Ð�nC¥6x�dC��[�¾M��b½ikVX[8�¢�¢Y��¢ik�¢}y��¥«m dz|Zik��nz}4nC¥6ik�¢YQ[8ikV�dCi�x�dC��[�¾<��b¿nzfyi�nC¥�{XdCic[87xymc[%|Z��bc[Z�¢j�²��¢iu��b�{y[��X}X[%{>dzb

¬e­3Õ`ÌWK��@����Ã.- Û/F � Ó � ­Fà - Û/F � Ó · ­F

ÕÌWK��@����

Ã0- Û/F � Ó � ­FC

ÌWK��@��#�Ã.- Û/F � Ó · ­FC

Õ ç&)( · *

�*ÌWK��@����Ã!- Û/F � Ó &)( � ­F+*

C ?G��C

rÑ}Sx�dCmkik��|Zfy�DdCme²^¹uVX[Z}Éx�nz�¢��|Zj >���b³|Zj:|Z�¢��|s¹u�¢ikV�|Zj:|Z��[s��[Z}y��ikV21�²u� ¨ [�¨¢²�¹uVX[Z} > @�3 � <�Õ >54 @ Ò�Ó�6 3 � < ¥¶nzm£dC�¢�ç# 27) 81 dC}X{>dC�¢� CÀÕ ç Å2gµÅ Ý%Ý%ÝȲhikVX[Z}

¬ ­ Õ ç1 &)( · *

- Û 9:F � Ó &;(

� ­F * Å ?¶��C

¹uVX[Zmc[<� ­ 3 ��b^ikVX[�}µfyY��+[Zm©nC¥®x�dC��[´¯°¾�dz|%|%[%bcbc[%b8{hfymk�¢}y�Àd�|Zjh|Z��[�¨^U�VX[�|%n�bki^¥«fy}X|Zik��nz}Áicn���[�Y��¢}y�¢Y��¢ªe[%{Á��b^ikVX[¹¿[Z�¢��V�ic[%{ÁbkfyY�nC¥<ikVX[�nz�Xbcnz��[%bc|%[Z}X|%[*m dCic[%b *

h Õ h�ÎG><Ð<Õ k:­ � Ó

Ä�­�¬%­ Å ? �$C

¹uVX[Zmc[�Ä�­½dCmc[����¢¼�[Z}Áx+n�bk�¢ik�¢¼�[�mc[edC�<}�fyY���[Zmcb*dC}X{sikVX[�Y��¢}y�¢Y��¢ª�dCik��nz}¤��b^icn��+[�n�¼�[Zm©dC�¢��bki dCik��nz}�dCmkj>b |cVX[%{hfy�¢�¢}y�x+nz�¢��|Z��[%be¨�7�áÿ8r9<&<MsBt

Page 9: Optimal Robot Scheduling for Web Search Engines

� .��������Q�����+8!���( ��������Q�"��a ��� ��&)+�,�B��(

vS¥¶[Z¹���dzbk��|%b®�¢}Qbcicnµ| V�dzbkik��|�nzmc{y[Zmk�¢}y��|%nz}X|Z�¢fX{y[�ikVy��b½bc[%|Zik��nz}3¨MèXnzm¦i°¹¿n#�³¯°{h�¢YQ[Z}Xbk��nz}�dC��mc[edC�X¼�[%|Zicnzmcb��>dC}X{� ²��>Y�dHoknzmk�¢ªe[%b � ²�¹umk�¢ikic[Z}���� � ²µ�»¥ à <­ � Ó�� ­��� à <­ � Ó��� ­�� ²�¥¶nzm 7QÕ ç Å Ý%Ý%Ý Å � æsç8dC}X{ à -­ � Ó�� ­�� Õ Ã -­ � Ó��� ­�� ²¹uVX[Zmc[ � ­�� ��b©ikVX[�¾����£�DdCmk��[%bki�|%nzY�x�nz}X[Z}µi�nC¥��®¨£r°}µikfy�¢ik�¢¼�[Z�¢j�² � ��b���[Zikic[Zm���dC�DdC}X|%[%{ ikV�dC}��®¨£v~¥Èfy}X|Zik��nz}�Ï��b©b�dC��{sicn£�+[ �����$�$(��J�B�8! # ��" �»¥uÏ3Î��MÐ Ï3Î � Ð^¹uVX[Z}X[Z¼�[Zm���� � ¨Qt:[%[2�¢\ ���M¥Ènzm*YQnzmc[�{y[Zi dC�¢��b�dC��nzfyi*ikVy��b©dC}X{mc[Z�DdCic[%{>xymcnzx�[Zmkik��[%be¨

v~m dC}X{ynzY¼�dCmk�DdC�y��[�� Ó ��b*b dC��{¤icn³�+[�}Xn£��mc[edCic[Zm*ikV�dC}¸d�m dC}X{ynzY¼�dCmk�DdC�y��[ �"!��¢}§ikVX[�|%nz}�¼�[´g nzmc{y[Zmk�¢}y�bc[Z}Xb [�²y{y[Z}Xnzic[%{#� Ó %$'&(�)!C²h�»¥ &;( Ï�Î�� Ó Ð * &;( Ï�Î��)!HÐ * ¥Ènzm�dC�¢�3|%nz}µ¼�[´g³¥Èfy}X|Zik��nz}Xb^Ï3²hxymcne¼:��{y[%{4ikVX[*[´g:x+[%|Zi dCik��nz}Xb[´gh��bkie¨�r°¥��¢} ikVy��b©{y[��X}y�¢ik��nz} b |%nz}µ¼�[´g 6��b©mc[Zxy�Ddz|%[%{¸[Z¼�[Zmkjµ¹uVX[Zmc[���j b �¢}X|Zmc[edzbk�¢}y�ÁdC}X{.|%nz}�¼�[´g�² 6ikVX[Z}¸¹¦[�¹umk�¢ic[� Ó u­*$�&��"!C¨�v8b���b©[edzbk�¢�¢j>¼�[Zmk����[%{�²+� Ó %$�&,�"!��¢Y�xy�¢��[%b*ikV�dCi-� Ó V�dzb�ikVX[Qb dCYQ[�YQ[edC}§�yfyi�bcY�dC�¢��[Zm*¼zdCmk�DdC}X|%[ikV�dC}.�)!H¨MrÑi½��budC��bcn�[edzbkj�icn�b [%[�ikV�dCi�� Ó %$'&/�)!u�¢Y�xy�¢��[%b0� Ó u­*$�&/�)!C¨½t:[%[ �¢\ 1���¥¶nzm¦[%ʵfy�¢¼zdC��[Z}µi�{y[��X}y�¢ik��nz}XbudC}X{¥ÈfymkikVX[Zm�xymcnzx+[Zmkik��[%be¨

1 ��������&32Ë��!54��+6��p�87 �!�� :9³��;S��&=<Ë����!��

_^[%|edC�¢��ikV�dCi4d§x�dC��[À��b³|%nz}Xbk��{y[Zmc[%{Énzfyip¯°nC¥±¯°{XdCic[>¥ÈmcnzY-ikVX[Àik�¢YQ[>�¢i���b�YQnµ{h����[%{Sfy}�ik�¢�8ikVX[À}X[´ghi�ik�¢YQ[>�¢i���bdz|%|%[%bcb [%{§��j¤ikVX[�mcnz�+nzie¨QU�V�fXb�²3�»¥�x�dC��[Q¾���b*}Xnzi�YQnµ{h����[%{ {hfymk�¢}y�>�¢icbDI�¯ ikV¸�¢}�ic[Zmc|%nzY�xy��[Zik��nz}J�¢}�ic[Zmk¼�dC� ²6ikVX[Z}ikVX[Qnz�Xb nz��[%bc|%[Z}X|%[�ik�¢YQ[���b#� ­F Õ Uh¨Qñ^ikVX[Zmk¹u��bc[�² � ­F ��b�ikVX[�ik�¢YQ[�ikV�dCi©[Z�DdCxXb [%b�¥«mcnzYikVX[ �Xmcbci*YQnzYQ[Z}�i©x�dC��[¾���b©YQn:{h����[%{.{hfymk�¢}y�>�¢icb�I�¯ ikV.�¢}µic[Zmc|%nzY�xy��[Zik��nz}��¢}�ic[Zmk¼�dC�¦fy}µik�¢�½ikVX[Q[Z}X{ nC¥uikV�dCi��¢}�ic[Zmk¼�dC� ¨�_^[%|edC�¢�¦dC��bcn4ikV�dCiikVX[�YQn:{h����|edCik��nz} ?Ènzm�Y�fyi dCik��nz}�C�[Zx+n:|cVXb*nC¥¦x�dC��[�¾¿¥¶nz�¢��ne¹åd£l<nz��bcbcnz}§xymcn:|%[%bcb8¹u�¢ikV§x�dCm dCYQ[Zic[Zm�Ç ­ ¨��¿j>ikVX[YQ[ZYQnzmkj:��[%bcb³xymcnzx�[ZmkiÑjSnC¥©ikVX[Ál�nz��b bcnz}Sxymcn:|%[%bcbe²�ikVX[Àik�¢YQ[ÁikV�dCi³[Z�DdCxXb [%bQ¥ÈmcnzY ikVX[s�+[Z���¢}y}y�¢}y��nC¥©x�dC��[Á¾ bI�¯ ikV¡�¢}�ic[Zmc|%nzY�xy��[Zik��nz}S�¢}�ic[Zmk¼�dC��icn¤ikVX[��Xmcbki�bkfy�Xbc[%ʵfX[Z}�iQY�fyi dCik��nz}¡V�dzb�dC}�[´ghx�nz}X[Z}µik�DdC��{h��bkikmk�¢�yfyik��nz}¡¹u�¢ikVx�dCm dCYQ[Zic[Zm8Ç+­Æ¨�w3[Zi5> ­

Ó Å > ­ ! Å Ý%Ý%ÝX�+[©dC}À� ¨ � ¨ {�¨®b [%Ê�fX[Z}X|%[©nC¥�bkfX| VÀm dC}X{ynzY�¼�dCmk�DdC�y��[%be²Xbcn�ikV�dCi

� ­F@?ÕBA�· ­F æ�> ­FDC � Å ?G1�C

¹uVX[Zmc[ � � {y[Z}Xnzic[%b �,EGF�Î � Å UµÐ�dC}X{ ?Õ�{y[Z}Xnzic[%b�[%ʵf�dC�¢�¢i°j£�¢}>{h��bcikmk�¢�yfyik��nz}3¨

v8b�dC}À�¢Y�YQ[%{h�DdCic[�|%nz}Xbc[%ʵfX[Z}X|%[�²y¹¦[©nz�yi dC�¢}

HJI êµî � ê �LKM�NPO %+����� ��,#�� ��B� �2121 ���"�n�� 1�����'(����1'���i�"! ������"!��'(����12��!$# �B��! # � "i��(�����(��"!$#=1,��!A1'�5 ������! ���������1;��� �21;�2��!�� �+( 8�p� �"1 �����;(B��61,��� %'��( �������,#���1 �$!�����(q�!`o+1;���A�������m��!$# ����� � �;o �

Q � î�î�R M w3[Zi�âe·#S< ä ��[�dQbc[%ʵfX[Z}X|%[�nC¥®dz|%|%[%b b�ik�¢YQ[%b�{h��bkikmk�¢�yfyic[%{s�¢}X{y[Zx�[Z}X{y[Z}µik�¢jÁdzb8·#SȲ�dC}X{s{y[��X}X[�âe·TS ­F ä FdC}X{§â%� S ­F ä F dzb�¥Ènzm�·=<µ¨�v8bcbkfyYQ[�ikV�dCi�· S �­U$'&�·¸¨�U�VX[Z}3²

� S ­FV?Õ A · S ­F æ�> ­F C � �­U$'& A · ­F æ�> ­F C � ?Õ � ­F Å��7�7���

Page 10: Optimal Robot Scheduling for Web Search Engines

������������ ���������������������� �"!$#&%'��(*)+�,�-�����(����/.0!�#��"!��21 �

dC}X{Àb n &)( � S ­F * &;( � ­F+* ¨�U�VµfXbe²

¬ S­ Õ ç&)( · S * �©ÌWK��@����

à - Û/F � Ó &;( � S ­F *C ç&)( · *

�*ÌWK��@����à - Û/F � Ó &)( � ­F *

C Õج%­ Å

dzb�{y[%bc�¢mc[%{�¨

� [³}Xn�¹�bkVXne¹ ikV�dCi�ikVX[£nz�Xbcnz��[%bc|%[Z}X|%[³m dCic[³��b�d¤t:|cVµfym�|%nz}�¼�[´g¸¥«fy}X|Zik��nz}�nC¥�ikVX[³¼�[%|ZicnzmQnC¥��¢}µic[Zm dz|%|%[%bcb{h��bki dC}X|%[%b�¨®èXnzm�ikVy��be²:¹¦[�}X[%[%{³ikVX[^¥Ènz�¢��n�¹u�¢}y�Q|edC��|Zfy�DdCik��nz}À¹uVy��|cV4¹u�¢�¢�3dC��bcn��+[^fXb [´¥«fy��¥Ènzm��DdCic[Zm�mc[%bkfy�¢icbe¨���[��X}X[Ïy­3Õ &)( ÙCÒXÚCÛ±Ü * ²yikVX[*w6dCxy�Ddz|%[�ikm dC}Xbp¥ÈnzmkYánC¥M·¸¨

�½ê � �.��K�M�N�� ��(q�!`o ��,#�� ¾

&;( �­F * Õ E ­F &)( · * æ

çÇ+­� ç�æ¸Ï ? Û�­�� Ý

Q � î�î�R M w3[Zi�� ­F ��[�ikVX[�xymcnz��dC�y�¢�¢�¢iÑjÀ{h��bkikmk�¢�yfyik��nz}snC¥�· ­F ¨M� [©V�d%¼�[�¥ÈmcnzY ?G1�C¦ikV�dCi

&;( �­F * Õ �S Î � ­F N��:Ð E �

Õ �S ζ· ­F æ�> ­F N��:Ð E �Õ �S � � ­F Î E � Ð � AHç�æ¸Ù ÒXÚCÛ 4 & Ò 6 C E �Õ �S &S A�ç�æ¸Ù ÒXÚCÛ 4 & Ò 6 C E ��� ­F Î E � ÐÕ �S � � æ ç�æ¸Ù ÒXÚHÛ &

Ç+­ � � ­F Î E � Ð

¹uVy��| V>j:��[Z��{ybuikVX[*��[ZY�Y�dh¨

� [u|edC}�|%nz}X|Z�¢fX{y[¿¥ÈmcnzY�ikVX[�dC�+ne¼�[�xymcn:nC¥�ikV�dCi�ikVX[¿mc[%bkfy�¢i�nC¥�U�VX[%nzmc[ZYå�h¨¢\�bkik�¢�¢�hVXnz��{ybM¹uVX[Z}�ikVX[¿�¢}X|Zmc[edzbk�¢}y�|%nz}µ¼�[´g�nzmc{y[Zmk�¢}y�*��b<mc[Zxy�Ddz|%[%{���j�ikVX[¿¹¦[ed6:�[Zm®w6dCxy�Ddz|%[´¯ ikm dC}Xbp¥¶nzmkYånzmc{y[Zmk�¢}y�?Èbc[%[ �¢\ 1'��C´¨<r°i�¥¶nz�¢��ne¹�b<¥ÈmcnzY d^mc[%bkfy�¢inC¥½t:|cVµfymd?È|´¥k¨D�¢\ �X²yl�mcnzx+n�bk�¢ik��nz}Á�h¨ �8¨¢\�²yx�dC��[ �'�!��CudC}X{Àw3[ZY�Y�dQ�h¨¢\©ikV�dCi

HJI êµî � ê �LKM���� �8(q�![o �"����/!`�$�n�B��(+� % ��,#���� ¾ ��B�2�21 1,�21 ������ "2�A���'�p�B�=�S�8�S8�a���21'��� �21;� �;!�� �n���"�n�d� %��A,#�� ¾�"1 ��������(q�B��! # ��"&%2��!��'��� ��! � %-�����d���"1��S�!�� �21 E ­F �

�7�áÿ8r9<&<MsBt

Page 11: Optimal Robot Scheduling for Web Search Engines

& .��������Q�����+8!���( ��������Q�"��a ��� ��&)+�,�B��(

U�VµfXbe²®�¢}�nzmc{y[Zm�icnÁY��¢}y�¢Y��¢ªe[£ikVX[£[´g:x+[%|Zic[%{�nz�Xbcnz��[%bc|%[Z}X|%[£ik�¢YQ[�²�ikVX[4dz|%|%[%bcb [%b�icn¤dC}µj.x�dCmkik��|Zfy�DdCm�x�dC��[bkVXnzfy��{À��[©dzb�[Z¼�[Z}y�¢j4bkx�dz|%[%{sdzbux�n�b bk�¢�y��[�¨

v^}�dC�¢��nzmk�¢ikVyY�ikV�dCi�|%nzY�xyfyic[%bMd�bc|cVX[%{hfy��[¿nC¥:ikVX[®mcnz��nzi3ikV�dCi��¢Y�xy��[ZYQ[Z}�icbMd����¢¼�[Z}�b [Zi3nC¥Xdz|%|%[%b b�¥Èmc[%Ê�fX[Z}X|Z��[%b�¢}³ikVX[^b [Z}Xbc[�nC¥ ?p\MC���b¦|edC�¢��[%{4dC} ��B� �21212��!$# x�nz�¢��|Zj�¨�rÑ}�ikVX[%b [�ic[ZmkYQbe²µikVX[8bc|cVX[%{hfy�¢�¢}y��x+nz�¢��|Z��[%b¦xymcnzx+n�bc[%{��¢}�ikVy��bx�dCx+[Zm�|%nz}Xbk��bki�nC¥�iѹ¦nsbki dC��[%b,7<ikVX[ �Xmcbki�|%nzY�xyfyic[%b�d4bc[Zi�nC¥�dz|%|%[%bcb©¥«mc[%ʵfX[Z}X|Z��[%bQâ�ã ­Æä dC}X{ ikVX[�bc[%|%nz}X{§��b�dC}dz|%|%[%bcbc�¢}y��x+nz�¢��|Zj³ikV�dCi^�¢Y�xy��[ZYQ[Z}�icb�â�ã ­Æä ¨¦U�VX[�[Z¼�[Z}h¯°bkx�dz|Z�¢}y�³nz�hok[%|Zik�¢¼�[�nC¥½dz|%|%[%bcbc�¢}y��x+nz�¢��|Z��[%b�j:��[Z��{yb^d���n�¹¦[Zm�+nzfy}X{�²�dzb�¥¶nz�¢��ne¹�be¨

HJI êµî � ê �LKM*K�� ���n���21;�8� �21'� ��!��2�/( ��p�+��!�����(n8![o=��B� ��1 12��!$#����� � �;o*��� ��� ���n��!����"!$#=����� ��B� �2121 %2(B���;�[��!��'� �21â�ã ­Æä 1'8���"1,0�21Q%'��(q����;� ¾

¬e­ ç&)( · *

�&)( · * æ

ãC­Ç ­�� ãC­

Ç ­ Ï Ó����ÆÛ­ � Ý

Q � î�î�R M¬%­ Õ ç

&)( · *�*ÌWK��@��#�

Ã0- Û/F � Ó &)( � ­F *C

Õ ç&)( · *

�*ÌWK��@��#� çC- Û/:F � Ó

� E ­F &;( · * æ çÇ+­� ç�æJÏ ? Û�­ � �

Õ ç&)( · *

�*ÌWK��@��#� çCC &;( · * æ

� ­@Ç ­ � ç

Ç ­- Û/:F � Ó

Ï ? Û�­�� Õ ç

&)( · *&;( · * æ

ãC­Ç+­�� ç

Ç+­ ÌWK��@���� çC- Û/:F � Ó

Ï ? Û�­�� ç

&)( · *� &)( · * æ ãC­

Ç ­ � çÇ ­ ÌWK��@���� � ­@

C Ï @ � - Û/­ �Õ ç

&)( · *�&)( · * æ

ãC­Ç+­�� ãC­

Ç+­ Ï Ó����ÆÛ­ � Å

¹uVX[Zmc[*ikVX[©�¢}X[%Ê�f�dC�¢�¢iÑjÀ|%nzYQ[%b�¥«mcnzY ikVX[�t:|cV�fym8|%nz}�¼�[´gh�¢i°jÀnC¥�U�VX[%nzmc[ZY �h¨ �h¨

U�VX[�dC�+ne¼�[���ne¹¿[Zm*��nzfy}X{Á|edC}s�+[�dz| Vy��[Z¼�[%{§nz}y�¢jÀ�¢}¤bkx�[%|Z�DdC�<|edzbc[%be¨8èXnzm8�¢}Xbki dC}X|%[�²��»¥®ikVX[�¥Èmc[%ʵfX[Z}X|Z��[%b�dCmc[dC�¢��[%ʵf�dC� ²®ikVX[£x+nz�¢��|Zj.ikV�dCi�dz|%|%[%bcb [%b�x�dC��[%b4ç Å2gµÅ Ý%Ý%Ý Å f`|Zj:|Z�¢��|edC�¢�¢jJj:��[Z��{yb�ikVy��b�nzxyik�¢Y�dC��nz�Xbcnz��[%bc|%[Z}X|%[£m dCic[�¨v^}XnzikVX[Zm�[´gXdCY�xy��[*¹uVX[Zmc[*¹¦[�|edC}��X}X{>d�¥¶[edzbk�¢�y��[©dz|%|%[%bcbk�¢}y�Qx+nz�¢��|Zj4dz|cVy��[Z¼:�¢}y��ikVX[���ne¹¿[Zm��+nzfy}X{4��bu¹uVX[Z}ÀikVX[¥Èmc[%Ê�fX[Z}X|Z��[%budCmc[8nC¥6ikVX[�¥ÈnzmkY ãC­�Õ ç $ g < Û ²µ¹uVX[Zmc[ 7�­3��budC}��¢}µic[Z��[Zm¿¥Ènzm¿[Z¼�[Zmkj�¾c¨M� [^mc[Zikfymk}�icn�ikVX[���[Z}X[Zm dC��|edzbc[dH¥Èic[Zm�xymcne¼:�¢}y�QikVX[�¥¶nz�¢��ne¹u�¢}y�³|%n�bkip¯ Y��¢}y�¢Y��¢ª�dCik��nz}>ikVX[%nzmc[ZYÀ¨

��7�7���

Page 12: Optimal Robot Scheduling for Web Search Engines

������������ ���������������������� �"!$#&%'��(*)+�,�-�����(����/.0!�#��"!��21 �

HJI êµî � ê �LKM���� 1212���n�d����8�0����� � ��� #6�$��1 ��! ����� �B�61;��%���!��'��� ��!i�(�� ��( �2�A��(���� ��!�8�a�S�/�����q� ���S8�����8!e(�8�p�21 � %��������M#��21 0�G��� �W Ä ­6Õ�Ä S Ç+­ %'��( ���� ¾<Õ�ç Å2gµÅ Ý%Ý%Ý Å f � � �A��!q%'��( �!`o+1;���A�������m��!$# ����� � �;o'

h�Õ�Ä S � k:­ � Ó

Ç+­«¬%­ Ä S � Ç4æ�� � �k�­ � Ó

Ïh­ � N U Å ?G��C

� ���;(�� Ç>Õ Ã k­ � Ó Ç ­ �!�� � ����(B� �QÕ�Î &)( · * Ð Ò�Ó � 1-�����-( ��p� ���(���n�;�p��(d��%-�����d��B� �2121 �J���"�n� ���"1���(�� �;����� ��! �

Q � î�î�R M èXnzm�ikVX[©YQnzYQ[Z}�i���[Zi�ikVX[�Ä�­<�+[*��[Z}X[Zm dC� ¨¦èXnz�¢��ne¹u�¢}y�£U�VX[%nzmc[ZYá�h¨ �h²X¹¿[*V�d%¼�[ h h��C²y¹uVX[Zmc[qh�^��bikVX[©b nz�¢fyik��nz}Àicn�ikVX[8¥¶nz�¢��ne¹u�¢}y��nzxyik�¢Y��¢ª�dCik��nz}sxymcnz�y��[ZY4*

h � Õ �=K�Í k:­ � Ó

Ä�­ � ç�æ �Ç ­ � ­ � �

Ç ­ � ­ Ï Ó�� & Û­ � ? �8C

bkfy�hok[%|Zi�icn � ­ U�dC}X{k:­ � Ó � ­�Õ ç�Ý

U�nQbcnz�¢¼�[*ikVX[�dC�+ne¼�[©xymcnz�y��[ZYÀ²y¹¦[*fXb [©w6dC��m dC}y��[©Y�fy�¢ik�¢xy�¢��[Zmcb8dC}X{4{y[��X}X[

Î � Ó Å Ý%Ý%Ý Å � k Å�� ÐMÕ k:­ � Ó

Ä ­ � ç�æ �Ç+­ � ­ � �

Ç+­ � ­ Ï Ó�� & Û­ � � � � k:­ � Ó � ­ æ¡ç � Ý

�¿j�ikVX[u|%nz}�¼�[´gh�¢i°j�nC¥�ikVX[¿¥Èfy}X|Zik��nz} à k­ � Ó Ä�­�AHç�æ� ÚHÛ � ­ � ÚHÛ � ­ Ï Ó�� & Û­ C �¢}�ikVX[u¼�[%|Zicnzm�Î � Ó Å Ý%Ý%Ý Å � k Ð ²�ikVX[ubcnz�¢fyik��nz}b dCik��b ��[%buikVX[*}X[%|%[%bcb dCmkj4dC}X{Àbkf ��|Z��[Z}µi^|%nz}X{h�¢ik��nz}

� � � ­ Õ æ Ä�­��

Ç ­� ç�æ¸Ï Ó�� & Û­ � Ì�Í�Ïh­

� ­ Ï Ó�� & Û­ � � � Õ U Å ?G&�C

� � � Õ k:

­ � Ó � ­+æ�ç8Õ U:Ý ?¶��C

ñ^�Xb [Zmk¼�[�ikV�dCi�Ïh­���ç³bcn4ikV�dCi�Ï Ó�� & Û­ �ºç�¨>ñ^}X[�|edC}¸[edzbk�¢�¢j | VX[%|B:§ikV�dCi�ikVX[�¥Èfy}X|Zik��nz}Sç©æ � � � Ì�Í � ��bbkikmk��|Zik�¢j�{y[%|Zmc[edzbk�¢}y�*�¢} � ¥¶nzm � ��ç�¨MU�V�fXb�²�fy}X{y[ZmMikVX[�dzb bkfyY�xyik��nz}�ikV�dCi®Ä ­���bMxymcnzx�nzmkik��nz}�dC�yicn�Ç+­p²z¹¦[u|%nz}X|Z�¢fX{y[¥ÈmcnzY ?G&�C¦ikV�dCi^dC�¢�<Ï Ó�� & Û­ dCmc[*��{y[Z}µik��|edC�<dC}X{4ikV�dCi�ikVX[�Y��¢}y�¢Y�fyYá��b^dz|cVy��[Z¼�[%{�²X�µj ?¶��C´²X¹uVX[Z}

� ­ Õ Ì�Í�Ïy­Ã k­ � Ó Ì�Í�Ïy­ Õ Ì�Í�ΰÏh­ÆÐkÒ�Ó

à k­ � Ó Ì�Í�ΰÏh­ÆÐ Ò�ÓÝ ?p\,>�C

U�Vy��b^bcnz�¢fyik��nz}À��bux+n�bk�¢ik�¢¼�[*bcn��¢i���b^dC��bcn�ikVX[©bcnz�¢fyik��nz}Àicn�ikVX[©Y��¢}y�¢Y��¢ª�dCik��nz}>xymcnz�y��[ZY �¢}@? �8C´¨�(^[Z}X|%[�²

h � Õ k:­ � Ó

Ä S Ç+­+æ.Ä S � k:­ � Ó � ­ � Ä S � k:

­ � Ó � ­ ÏÓ�� & Û­ Õ�Ä S � Ç4æ�� � �XÙ Ã��Û�� ����� � Û � ÕØÄ S � Ç4æ�� � �

k�­ � Ó

Ïy­ � Ý�7�áÿ8r9<&<MsBt

Page 13: Optimal Robot Scheduling for Web Search Engines

\,> .��������Q�����+8!���( ��������Q�"��a ��� ��&)+�,�B��(

Þ8nzic[*ikV�dCik�­ � Ó

Ï ­ Õ &)( Ù Ò Ã �Û�� � ÚHÛDÜ<Û * N�ç^æ &)(k:­ � Ó

Ç ­ · ­ * Õ ç�æ Ç� Å

bcn

Ç4æ�� � �k�­ � Ó

Ïh­�N U Å

dC}X{4ikVX[©xymcnµnC¥M��b�|%nzY�xy��[Zic[�¨

ñ^�Xb [Zmk¼�[�ikV�dCi8¹uVX[Z}ÁikVX[�¹¿[Z�¢��V�icb��¢}ÁikVX[�|%n�bki�¥«fy}X|Zik��nz} dCmc[�}Xnzi�xymcnzx+nzmkik��nz}�dC�Micn�ikVX[�Y�fyi dCik��nz}§m dCic[%b8nC¥x�dC��[%be²�nz}X[�|edC}§bkik�¢�¢�MfXbc[�w6dC��m dC}y��[�Y�fy�¢ik�¢xy�¢��[Zmcb�icn�b nz�¢¼�[�ikVX[�nzxyik�¢Y��¢ª�dCik��nz} xymcnz�y��[ZYÀ¨�(^n�¹¦[Z¼�[Zme²�¹¿[�{yn�}XnziV�de¼�[�|Z��n�bc[%{:¯¶¥¶nzmkY bcnz�¢fyik��nz}Xb��¢}4��[Z}X[Zm dC� ¨

rÑi���b�dC��bcn¤¹¿nzmkikV�¹uVy�¢��[s}Xnzik��|Z�¢}y� ikV�dCiQikVX[4nzxyik�¢Y�dC�^dz|%|%[%b b�¥«mc[%ʵfX[Z}X|Z��[%b=?È|´¥k¨ ?p\,>�C�C��¢}�ikVX[ÀdC�+ne¼�[4��n�¹¦[Zm�+nzfy}X{§dCmc[�}Xnzi8}X[%|%[%bcb dCmk�¢�¢jsxymcnzx�nzmkik��nz}�dC�Micn³ikVX[�x�dC��[�Y�fyi dCik��nz} m dCic[%b�Ç ­ ¨�_8dCikVX[Zme²�ikVX[Zj¤dCmc[�xymcnzx+nzmkik��nz}�dC�

icn�Ì�Í�ΰÏy­ÆÐ Ò�Ó ÕØÌ�Í�A &)( Ù ÒXÚHÛ«Ü * C Ò�Ó ÝXl�mcnzx+nzmkik��nz}�dC�¢�¢i°j4icn�ikVX[*Ç+­�n:|%|Zfymcb�nz}y�¢j�¹uVX[Z}À·à��b�d�|%nz}Xbki dC}�ie¨½Þ^nzic[©dC��bcn

ikV�dCi^ikVX[©Y�dC��}y�¢ikfX{y[�nC¥MikVX[�{h� 5+[Zmc[Z}X|%[���[Ziѹ¦[%[Z}¤Ç+­ &;( · * dC}X{ÀÌ�Í A &;( Ù ÒXÚHÛ±Ü * C Ò�Ó ��b��DdCmk��[©�»¥�����¬hζ·§Ð¿��b��DdCmk��[?Ènzm�·ß��bu�DdCmk��[*�¢}4ikVX[�|%nz}�¼�[´gÀnzmc{y[Zmk�¢}y��bc[Z}Xb [MC´¨

U�nQbkfyY�Y�dCmk�¢ªe[�²yikVX[�mc[%bcfy�¢icbunC¥<ikVy��bubc[%|Zik��nz}ÀbcVXne¹(ikV�dCie²h�»¥<ikVX[�¹¦[Z�¢��V�icb^�¢}£ikVX[*|%n�bki�¥Èfy}X|Zik��nz}>dCmc[�xymcnzx�nzmp¯ik��nz}�dC��icn©ikVX[�Y�fyi dCik��nz}£m dCic[%b¦nC¥�ikVX[�x�dC��[%be²�ikVX[Z}4dC}³dz|%|%[%b bk�¢}y��x+nz�¢��|Zj�ikV�dCi¿|%nzYQ[%b¿|Z��n�bc[�icn�ikVX[���n�¹¦[Zm���nzfy}X{�¢}>U�VX[%nzmc[ZY �h¨ ��¹u�¢ikVÀikVX[�ã ­ }X[edCmk�¢j�xymcnzx+nzmkik��nz}�dC�6icnQikVX[*Ì�Í�Î°Ï ­ Ð Ò�Ó ¹u�¢�¢�6|%nzYQ[©|Z��n�b [�icn�Y��¢}y�¢Y��¢ª%�¢}y� hQ¨

è<�¢}X{h�¢}y�4��n:nµ{§dz|%|%[%bcbc�¢}y�4x�nz�¢��|Z��[%b�ikV�dCi�mc[edC�¢�¢ªe[Qd����¢¼�[Z} bc[Zi*nC¥�dz|%|%[%bcb^¥Èmc[%Ê�fX[Z}X|Z��[%b©��b^ikVX[�bcfy�hop[%|Zi*nC¥¦ikVX[}X[´ghi©i°¹¿nsb [%|Zik��nz}Xbe¨�rÑ}¸t:[%|Zik��nz} 1h²6¹¦[�{y[Z¼�[Z��nzx�dC}.nzxyik�¢Y�dC�½m dC}X{ynzY��¢ªe[%{�dz|%|%[%bcbk�¢}y�>x�nz�¢��|Zj�²MdC}X{§�¢}Jt:[%|Zik��nz}�h²�¹¿[Àdz{XdCxyi�ikVX[³¹¦[Z�¢�»¯°bkikfX{h��[%{���nz��{y[Z}h¯ m dCik��n x�nz�¢��|Zj icn¤nzfym�xymcnz�y��[ZYÀ²½xymk�¢Y�dCmk�¢�¢j�dzb�dÁ|edC}X{h��{XdCic[£¥¶nzm���n:nµ{dzbkj:Y�xyicnzik��|�x+[Zmp¥ÈnzmkY�dC}X|%[87h¹¿[^¹u�¢�¢��bc[%[8ikV�dCi�ikVy��b¿x�nz�¢��|Zj����¢¼�[%budC}£nz�Xb nz��[%bc|%[Z}X|%[�m dCic[8¹u�¢ikVy�¢}2187~nC¥6ikVX[8��n�¹¦[Zm�+nzfy}X{�²y�¢}ÀikVX[��¢�¢Y��¢i�nC¥M�DdCmk��[-fJ¨

� [ mc[ZY�dCm�:SikV�dCi4ikVy��b4xymcnz�y��[ZY ��b4|Z��n�bc[Z�¢jSmc[Z�DdCic[%{(icn�ikVX[§{y[%bk�¢��} dC}X{(dC}�dC�¢j:bc��bÀnC¥�x+nz�¢�¢�¢}y���Cbkxy�¢�¢ikik�¢}y�bc[%ʵfX[Z}X|%[%b��¢}4ikVX[*|%nz}µic[´g:i8nC¥MʵfX[ZfX[Z�¢}y� ?¶dC}X{£�¢}Àx�dCmkik��|Zfy�DdCme²�|%nzY�Y�fy}y��|edCik��nz}�C�bkjhbkic[ZYQb � �y²X�h²A1h²A�h²�� �°²h¹uVX[Zmc[dC�¢��nzmk�¢ikVyYQbÁdCmc[§{y[%bc|Zmk�¢��[%{�dzb£ic[ZY�xy�DdCic[ {hmk�¢¼�[Z} nzm4��[Z}X[Zm dC�¢�¢ªe[%{�mcnzfy}X{�mcnz�y�¢}3¨ �É�¢ikVØ¥Èfyikfymc[¤mc[%bc[edCmc|cV(�¢}Y��¢}X{�²�¹¦[©}Xnzic[©ikV�dCi�ikVX[%bc[©bcikfX{h��[%b�bkfy����[%bki^nzikVX[Zm^dCxyxymcn�dz|cVX[%b8¹¦nzmkikVs�¢}�¼�[%bkik�¢��dCik�¢}y�X²�[�¨ �X¨¢²�[´ghic[Z}Xbk��nz}Xb^nC¥MikVX[Y�dCikVX[ZY�dCik��|edC�+xymcnz��m dCY�Y��¢}y��ic[%|cVy}y��ʵfX[%b¿�¢}6� 1��+dC}X{�ikVX[^dC�¢��nzmk�¢ikVyYQb � �'��{y[Zmk�¢¼�[%{�¥ÈmcnzY (�dHop[;: b�� ����mc[%bkfy�¢icb¦nz}mc[Z��fy�DdCm^�y�¢}�dCmkj³bc[%ʵfX[Z}X|%[%be¨®v^�¢ikVXnzfy��VÀikVX[��DdCikic[Zm��Ddz| :£ikVX[*[%bki dC�y�¢��bkVX[%{4x+[Zmp¥ÈnzmkY�dC}X|%[©��nzfy}X{yb�nC¥MikVX[*��nz��{y[Z}h¯m dCik��n�x+nz�¢��|Zj�²ybk�¢Y�fy�DdCik��nz}Xb�bkVXne¹ ikV�dCiuikVX[Zj³dCmc[*bkfyx+[Zmk��nzm�dC�¢��nzmk�¢ikVyYQb��¢}£ikVX[*[edCmk�¢��[Zm�ʵfX[ZfX[Z�¢}y��YQnµ{y[Z��b�¨�U�V�fXbe²ikVX[Zj£Y�d6:�[*xymcnzY���bk�¢}y��|edC}X{h��{XdCic[%b�¥¶nzm�nzfymux�dC��[´¯Ñdz|%|%[%bcbk�¢}y��YQnµ{y[Z� ¨

��7�7���

Page 14: Optimal Robot Scheduling for Web Search Engines

������������ ���������������������� �"!$#&%'��(*)+�,�-�����(����/.0!�#��"!��21 \�\

� �^!������Â�����������<����/�/�� !�"�!�� 4 ��/��(����� �^�4�*��� �©��� ��!

w3[Zi�ã Ó Å ã ! Å Ý%Ý%Ý Å ã k ��[8���¢¼�[Z}>dz|%|%[%bcbu¥«mc[%ʵfX[Z}X|Z��[%be¨�vu|%|%nzmc{h�¢}y�Qicn�ikVX[�m dC}X{ynzY��¢ªe[%{Àb |cVX[%{hfy�¢�¢}y�Qx+nz�¢��|Zj�²XdCi�[edz|cV{y[%|Z��bk��nz}�x�nz�¢}µie²CikVX[®mcnz�+nzi6|cVXn:n�bc[%b�icn�dz|%|%[%bcb3x�dC��[¦¾�¹u�¢ikV�xymcnz��dC�y�¢�¢�¢i°j�ãC­J7zikVX[®{y[%|Z��bk��nz}���b3Y�dz{y[®�¢}X{y[Zx+[Z}X{y[Z}�ik�¢jnC¥¦dC�¢��xymc[Z¼µ��nzfXb8{y[%|Z��bk��nz}Xbe¨�ñ^}X[�|edC}Á[edzbk�¢�¢j4bc[%[�ikV�dCi�â E ­F ä F ²3âe· ­F ä F dC}X{.â%� ­F ä F dCmc[�ikVymc[%[�bc[%ʵfX[Z}X|%[%b^nC¥�� ¨ � ¨ {�¨m dC}X{ynzY ¼�dCmk�DdC�y��[%b�¥Ènzm�dC�¢��¾c¨§qsnzmc[%n�¼�[Zme² E ­F V�dzb�ds��[%nzYQ[Zikmk��|4{h��bkikmk�¢�yfyik��nz}J* Î E ­F Õ C<Ð�Õ ãC­pÎpç�æSãC­°Ð @ Ò�Ó�¨U�VµfXbe²y¹¦[�V�d%¼�[

�½ê � �.� �M�N�� ��( #�� # ��! %�(�� �;�A�;!��;�S�21 ã Ó Å ã !HÅ Ý%Ý%Ý Å ã k

¬e­3Õ ç&)( · *

�&)( · * æ

ãC­Ç+­ � ãC­

Ç+­ �ãC­ Ïh­

ç�æ¸Ïh­ � ãC­ Ïh­ � Ý

Q � î�î�R M v^b E ­F V�dzb�d���[%nzYQ[Zikmk��|©{h��bkikmk�¢�yfyik��nz}3²X¹¿[©nz�yi dC�¢}

&;( · ­F * Õ�:@ � Ó

ãC­ÆÎpç�æ.ãC­ÑÐ @ Ò�Ó C &)( · * Õ &;( · *ãC­ Å

dC}X{4�µj4w3[ZY�Y�dQ�h¨¢\*¹¿[*V�d%¼�[�²

&;( �­F * Õ

�:@ � Ó

ãC­pÎpç�æ ãC­ÑÐ @ Ò�Ó � C &)( · * æ çÇ ­ � ç

Ç ­ Ï@­ � Õ &;( · *ã ­ æ ç

Ç ­ � çÇ ­

ãC­ÈÏy­ç�æ¸Ï ­ � ã ­ Ï ­ Å

bcn�[Z��[ZYQ[Z}µi dCmkj4mc[Z}X[Z¹¿dC��ikVX[%nzmkj4dC}X{c?G��C¿�¢Y�xy�¢j

¬e­ÆÎ��XÐMÕ &;( � ­F Î��yÐ *&)( · ­F Î��yÐ *

Õ ç&)( · *

�&)( · * æ

ãC­Ç ­ � ãC­

Ç ­ �ãC­¶Ïy­

ç�æ¸Ï ­ � ã ­ Ï ­ � Ý

rÑiQ��b��¢}�ic[Zmc[%bkik�¢}y�Jicn |%nzY�x�dCmc[sikVX[4��ne¹¿[Zm��+nzfy}X{ËnC¥©U�VX[%nzmc[ZY �h¨ � ¹u�¢ikV¡ikVX[>nz�Xb nz��[%bc|%[Z}X|%[Àm dCic[snC¥�ikVX[m dC}X{ynzY��¢ªe[%{Áx+nz�¢��|Zj�¨^ñ^}X[�|edC}¤bc[%[�ikV�dCi�¹uVX[Z}§ãC­���b�bkY�dC�¢��?È|Z��n�bc[©icn/>�C�nzm^�DdCmk��[/?È|Z��n�b [©icnÁ\MC´²XikVX[�{h� 5+[Zmc[Z}X|%[�+[Zi°¹¿[%[Z}s¬ ­ dC}X{£ikVX[*��n�¹¦[Zm��+nzfy}X{4ic[Z}X{ybuicn >h¨�qsnzmc[*xymc[%|Z��b [Z�¢j�²yikVy��b�{h� 5�[Zmc[Z}X|%[*��b

��Ïh­Ç ­ ã !­ ��� ÎkÎ ãC­ÆÐ ! Ð

¹uVX[Z}sãC­<��n:[%buicnn>h²XdC}X{4��b��Ïh­Ç+­ Îpç � Ì�Í�Ïy­°Ð Î ãC­�æ�çzÐ �� Îpç^æ.ãC­ Ð

�7�áÿ8r9<&<MsBt

Page 15: Optimal Robot Scheduling for Web Search Engines

\ � .��������Q�����+8!���( ��������Q�"��a ��� ��&)+�,�B��(

¹uVX[Z}sãC­<��n:[%buicnÀ\�¨

� [¿}Xn�¹�|%nz}Xbc��{y[Zm<ikVX[¦xymcnz�y��[ZY nC¥ �X}X{h�¢}y��ikVX[¦nzxyik�¢Y�dC�Xdz|%|%[%bcb�¥Èmc[%Ê�fX[Z}X|Z��[%b<fy}X{y[ZmMikVX[¿m dC}X{ynzY��¢ªe[%{�x�nz�¢��|Zj�¨è<�¢mcbkie²y¹¿[*V�d%¼�[©ikVX[�¥Ènz�¢��n�¹u�¢}y����n�¹¦[Zm��+nzfy}X{4ne¼�[Zm�dC�¢�+¥«mc[%ʵfX[Z}X|Z��[%be¨

HJI êµî � ê � � M�N � 1212���n�d����8�0����� � ��� #6�$��1 ��! ����� �B�61;��%���!��'��� ��!i�(�� ��( �2�A��(���� ��!�8�a�S�/�����q� ���S8�����8!e(�8�p�21 � %��������M#��21 0�G��� �W Ä ­ Õ�Ä S Ç ­ %'��( ���� ¾<Õ�ç Å2gµÅ Ý%Ý%Ý Å f � � �A��!

h�Õ�Ä S � k:­ � Ó

Ç ­ ¬ ­ Ä S � Ç4æ � à k­ � Ó Î°Ï Ò�Ó­ æ�çzÐç � à k­ � Ó Î°Ï Ò�Ó­ æ�çzÐ � Ý ?p\�\MC

� �8(���� # ��( �����"1 � � � ��(q������!��/�"1-������S� # ��� � ���;! ���A�q��B� �2121�%2(B���;�[��!��'� �21+�(B� ��( �2�A��(���� ��!�8� �S� Ï Ò�Ó­ æ�ç �

Q � î�î�R M èXnz�¢��ne¹u�¢}y�£w3[ZY�Y�d$�X¨¢\�²h¹¿[*V�d%¼�[

¬ ­ Õ ç&)( · *

�&)( · * æ

ãC­Ç+­ � ãC­

Ç+­ �ãC­ Ïh­

ç�æ¸Ïh­ � ãC­ Ïh­ � Å

bcn�ikV�dCi h h � ¹uVX[Zmc[dh � ��buikVX[©bcnz�¢fyik��nz}>icn�ikVX[�¥Ènz�¢��n�¹u�¢}y��nzxyik�¢Y��¢ª�dCik��nz}sxymcnz�y��[ZY4*

h � Õ �=K�Í k:­ � Ó

Ä�­�æ�� k:­ � Ó

Ä�­ � ­Ç ­ � �k:­ � Ó

Ä�­Ç ­ � � !­ Ïh­ç�æ¸Ï ­ � � ­ Ï ­ Å ?p\ ��C

bkfy�hok[%|Zi�icn � ­ U�dC}X{k:­ � Ó � ­�Õ ç�Ý

v^��dC�¢}3²y¹¦[©fXbc[*w6dC��m dC}y��[©Y�fy�¢ik�¢xy�¢��[Zmcb�icn�bcnz�¢¼�[�ikVX[�dC�+ne¼�[*xymcnz�y��[ZYÀ¨¦�8[��X}X[

Î � Ó Å Ý%Ý%Ý Å � k Å�� ÐMÕ k:­ � Ó

Ä�­�æ�� k:­ � Ó

Ä�­ � ­Ç+­ � �k:­ � Ó

Ä�­Ç+­ � � !­ Ïh­ç�æ¸Ïy­ � � ­ Ïh­�� � � k:

­ � Ó � ­+æ�ç � Ý

�¿j³ikVX[©|%nz}�¼�[´gh�¢i°jÀnC¥MikVX[8¥«fy}X|Zik��nz}

k:­ � Ó

Ä ­ æ�� k:­ � Ó

Ä ­ � ­Ç+­ � �k:­ � Ó

Ä�­Ç+­ � � !­ Ïy­ç�æJÏh­ � � ­ Ïh­

�¢}.Î � Ó Å Ý%Ý%Ý Å � k Ð ²hikVX[©bcnz�¢fyik��nz}sb dCik��b ��[%buikVX[*}X[%|%[%b b dCmkj£dC}X{>bkf ��|Z��[Z}�i8|%nz}X{h�¢ik��nz}

� � � ­ Õ æ Ä�­��

Ç+­ � Ä ­ �Ç+­ �

g Ïy­pÎpç^æ¸Ïh­ÆÐ � ­ � Ï !­ � !­Îpç�æJÏh­ � Ïh­ � ­ÑÐ ! � � Õ U Å ?p\e��C

� � � Õ k:

­ � Ó � ­+æ�ç8Õ U:Ý ?p\ �$C��7�7���

Page 16: Optimal Robot Scheduling for Web Search Engines

������������ ���������������������� �"!$#&%'��(*)+�,�-�����(����/.0!�#��"!��21 \e�

ð½Ê�f�dCik��nz} ?p\e��C�|edC}À�+[�mc[Z¹umk�¢ikic[Z}sdzb

� Ç+­ÆÎpç^æ¸Ïh­ � Ïh­ � ­°Ð ! Õ �XÄ ­pÎpç�æ¸Ïy­ÆÐ ! Ýtµ�¢}X|%[�Ïy­ ��ç�²y¹¦[©V�d%¼�[

ç^æ¸Ïh­ � Ïh­ � ­ç�æ¸Ïh­ Õ�� �yÄ�­� Ç+­ Å

bcn

� ­�Õ � � �yÄ�­� Ç+­ æ�ç � ç�æ¸Ïy­

Ïh­ Ý ?p\ 1�C

Þ8ne¹(fXb [�ikVX[�dzbcbkfyY�xyik��nz}ÀikV�dCi8Ä ­6Õ�Ä S Ç+­�dC}X{4nz�yi dC�¢}À¥ÈmcnzY ?p\ �$CudC}X{@?p\ 1�C¿ikV�dCi

� ­ Õ Ï Ò�Ó­ æ�çà k­ � Ó Î°Ï Ò�Ó­ æ�çzÐ Ý ?p\ ��C

U�Vy��b^bcnz�¢fyik��nz}À��bux+n�bk�¢ik�¢¼�[*bcn��¢i���b^dC��bcn�ikVX[©bcnz�¢fyik��nz}Àicn�ikVX[©Y��¢}y�¢Y��¢ª�dCik��nz}>xymcnz�y��[ZYánC¥ ?p\ ��C´¨ (8[Z}X|%[�²

h � Õ�Ä S Ç4æ.Ä S � k:­ � Ó � ­ �

ç�æ¸Ïh­ç^æ¸Ïh­ � � ­¶Ïy­ Õ�Ä S Ç4æ.Ä S � à k­ � Ó Î°Ï Ò�Ó­ æ�çzÐ

ç � à k­ � Ó Î°Ï Ò�Ó­ æ�çzÐ Å

¹uVy��| V>��buikVX[©{y[%bc�¢mc[%{£��ne¹¦[Zm8�+nzfy}X{�¨

Þ8nzic[£ikV�dCiQ�»¥8ikVX[£¹¦[Z�¢��V�icb��¢}�ikVX[4|%n�bki�¥Èfy}X|Zik��nz}ËdCmc[4}Xnzi�xymcnzx+nzmkik��nz}�dC��icn¤ikVX[£Y�fyi dCik��nz}Sm dCic[%bQnC¥8ikVX[x�dC��[%be²+nz}X[�|edC}Ábcik�¢�¢��fXbc[/?p\ �$C�dC}X{ ?p\ 1�C�icn�bcnz�¢¼�[�ikVX[�xymcnz�y��[ZYÀ¨?(8ne¹¿[Z¼�[Zme²+ikVX[�b nz�¢fyik��nz}s��b8¼�dC�¢��{snz}y�¢j4¹uVX[Z}ikVX[*mk�¢��Vµip¯ V�dC}X{:¯°bk��{y[�nC¥�?p\ 1�C¿��b�x�n�bk�¢ik�¢¼�[�¨��¿j£bcnz�¢¼:�¢}y��¥Ènzm � fXbk�¢}y�=?p\ �$CudC}X{c?p\ 1�C¿¹¦[©nz�yi dC�¢}

� � Õ� � à k­ � Ó�� $ ÛÚHÛ A´Ï Ò�Ó­ æ�ç C

ç � à k­ � Ó AZÏ Ò�Ó­ æ�ç Cbcn�ikVX[�bcnz�¢fyik��nz}c?p\ 1�C¿��bu¼�dC�¢��{£�»¥

�=K�ÍÓ�j ­ j�k

� Ä ­Ç+­

à k­ � Ó�� $ ÛÚHÛ AZÏ Ò�Ó­ æ�ç Cç � à k­ � Ó A Ï Ò�Ó­ æ�ç C Ý

� ��/�7©���������� � ����������� �k��7à^!�� ������� ���c���®! ���� � ; �^� �c� 7

rÑ}�ikVy��b¦bc[%|Zik��nz}�¹¿[�|%nz}Xbk��{y[Zm¿ikVX[^dzbkj:Y�xyicnzik��|��DdCmk��[´¯°ÞØ�+[ZV�d%¼:��nzm¦nC¥3bc| VX[%{hfy�¢�¢}y��x�nz�¢��|Z��[%be¨MvØbk�¢Y��¢�DdCm¿bkikfX{hj�¹¿dzb|edCmkmk��[%{¤nzfyi8��j>rÑi dC�<dC}X{Á_�n�bk�+[Zmk�6�¢\�\ ���¢}ÁikVX[�|%nz}µic[´g:i*nC¥®ikVX[�|%nz}�ikmcnz��nC¥¦dQY�fy�¢ik�¢xy��[´¯Ñdz|%|%[%bcb*|cV�dC}y}X[Z� ¨©t:nzYQ[nC¥�nzfym�mc[%bkfy�¢icbuVX[Zmc[�dCmc[�dC}�dC��nz��nzfXb�icn�ikVX[Z�¢mcb�¨�7�áÿ8r9<&<MsBt

Page 17: Optimal Robot Scheduling for Web Search Engines

\ � .��������Q�����+8!���( ��������Q�"��a ��� ��&)+�,�B��(

� [�{y[��X}X[*dzbkj:Y�xyicnzik��|edC�¢�¢j�nzxyik�¢Y�dC�+x+nz�¢��|Z��[%b¦¹u�¢ikV£mc[%bkx+[%|Zi¿icn�ikVX[^��ne¹¿[Zmu��nzfy}X{³�¢}£U�VX[%nzmc[ZY��h¨ �X¨�(^[Z}X|%[�²¹¿[¦dzbcbkfyYQ[®ikVymcnzfy��VXnzfyi�ikVy��b3b [%|Zik��nz}�ikV�dCi3ikVX[½¹¦[Z�¢��V�icb��¢}©ikVX[®|%n�bci�¥«fy}X|Zik��nz}�dCmc[�xymcnzx+nzmkik��nz}�dC�µicnuikVX[®Y�fyi dCik��nz}m dCic[%bMnC¥�ikVX[¦x�dC��[%be²z� ¨ [�¨¢²�Ä�­3Õ(Ä S Ç+­h¥¶nzm�dC�¢�:¾MÕ ç Å2gµÅ Ý%Ý%Ý Å fJ¨<� [¦b dej*ikV�dCi®dux+nz�¢��|Zj�>���bMdzbkj:Y�xyicnzik��|edC�¢�¢j�nzxyik�¢Y�dC��»¥

ÌWK��k �#� h�ÎG><Ð�æch � Õ U:Ý

Þ8nzic[��Xmcbki¿ikV�dCi¦�»¥6ikVX[8icnzi dC��Y�fyi dCik��nz}4m dCic[�Çsic[Z}X{yb¦icn�ªe[Zmcny²:ikVX[Z}ÀdC�¢� �;o��;� � � x�nz�¢��|Z��[%b�dCmc[�dzbkj:Y�xyicnzik��|edC�¢�¢jnzxyik�¢Y�dC� ¨Ér°}X{y[%[%{�²�|%nz}Xbk��{y[Zm³dC}SdCmk�y�¢ikm dCmkj¡|Zj:|Z�¢��|Àx�nz�¢��|ZjJ¹u�¢ikVS|Zjh|Z��[£��[Z}y��ikV 1�¨¡rÑi�¥Ènz�¢��n�¹�bQ¥ÈmcnzY ?¶��C�dC}X{w3[ZY�Y�d��h¨¢\*ikV�dCi

h Õ Ä S � k:­ � Ó

Ç+­È¬%­

Õ Ä S �1 � k:

­ � ÓÇ+­ -

Û 9:F � Ó�� E ­F &)( · * æ ç

Ç+­� ç�æ¸Ï ? Û�­ ���

Õ Ä S Ç4æ Ä S �1 � k:

­ � Ó- Û 9:F � Ó

AHç�æ¡Îpç�æ E ­F Ç ­ &)( · * ��� 릂 !­ ÐkÐ ? Û� C

Õ Ä S Ç4æ Ä S �1 � k:

­ � Ó- Û 9:F � Ó

A E ­F Ç+­ &)( · * ��� 릂 !­ Ð CÕ � 릂 ! Ð Å

bcn��»¥�Ç�� Uh²yikVX[Z} h�� Uh¨

U�VµfXbe²�¹¦[sdzbcbkfyYQ[4ikV�dCi�¹uVX[Z} f � ز¦ikVX[4icnzi dC��Y�fyi dCik��nz}Sm dCic[ÀÇ�²�dzbQ¹¿[Z�¢�^dzbQikVX[sdz|%|%[%bcbk�¢}y� m dCic[x�dCm dCYQ[Zic[Zm �+²3��b �ygh[%{�¨ (^n�¹¦[Z¼�[Zme²�¥¶nzm�dC}�j ¾c²½ç ¾� fJ²3¹¦[QV�d%¼�[�Ç+­� U4¹uVX[Z}cf���ب l�}X{y[Zm�bkfX|cVdzbcbcfyY�xyik��nz}Xbe²yikVX[*��n�¹¦[Zm��+nzfy}X{ h����¢}ÀU�VX[%nzmc[ZY$�h¨ �Q�+[%|%nzYQ[%b

ÌWK��k ��� h � Õ�Ä S Ç4æ.Ä S � � ç�æ ÌWK��

k ���k�­ � Ó

Ï ­ � Õ�Ä S A Ç4æ � � �XÙ ÒXÚ � C Å ?p\��8C

¹uVX[Zmc[*¹¿[©fXbc[%{£ikVX[*¥¶dz|ZicbuikV�dCi &)( ÙCÒXÚCÛDÜ * Õ(ÙCÒXÚCÛ� Ü � � � ζÇ��­ ЦdC}X{4ikV�dCi à k­ � Ó Ç��­ � U�¥¶nzm�dC�¢�®ç������ g ¨

Þ8ne¹ |%nz}Xbk��{y[Zm�ikVX[�¥¶nz�¢��ne¹u�¢}y�¤|Zj:|Z�¢��|³bc| VX[%{hfy�¢�¢}y�sx+nz�¢��|Zj�²M|edC�¢��[%{.ikVX[���nz��{y[Z}J_8dCik��nÀx+nz�¢��|Zj�²�dC}X{.bkikfX{h��[%{�¢} �¢\�\ �½¥¶nzm©ikVX[³|%nz}�ikmcnz��nC¥^d4Y�fy�¢ik�¢xy��[´¯Ñdz|%|%[%b b�|cV�dC}y}X[Z� ¨ÀU�VX[�x�nz�¢��|Zj§��b�{y[��X}X[%{.�¢}¸ic[ZmkYQb�nC¥�ikVX[�è<�¢��nz}�dz|%|Z�}µfyY��+[Zmcb �

<�Õ�� < æ¡Îpç^æ � Ð <� � Å 7�Õ U Å ç Å Ý%Ý%Ý Å

¹uVX[Zmc[ � Õ Î � � � çzÐ $ g ²�dC}X{£¹uVX[Zmc[ � Ò�Ó¿Õ�Î � � æ�çzÐ $ g�� U:Ý��:ç���U � � ! � �#"���buikVX[*��nz��{y[Z}>m dCik��ny¨��7�7���

Page 18: Optimal Robot Scheduling for Web Search Engines

������������ ���������������������� �"!$#&%'��(*)+�,�-�����(����/.0!�#��"!��21 \ 1

w6[Zi���Î 7 Å f.Ð^{y[Z}Xnzic[�ikVX[���nz��{y[Z} m dCik��nÀx+nz�¢��|ZjÁ¹u�¢ikV

�<�ikVX[Q|Zjh|Z��[���[Z}y��ikV3²MdC}X{if ikVX[�icnzi dC�®}�fyY���[Zm�nC¥

x�dC��[%be¨½w3[Zi�� ­<�� k ��[�ikVX[�}µfyY��+[Zm^nC¥Mx�dC��[´¯°¾¿dz|%|%[%b bc[%bu�¢}À[edz|cVÁ|Zj:|Z��[*nC¥���Î 7 Å f.Ð 7XikVX[%b [�}�fyY��+[Zmcb8b dCik��bp¥Èj

� � ­ � k�<� � ­<�� k � � ­ � k

�<��

dC}X{ à k­ � Ó � ­<�� k Õ

�< Å ¹uVX[Zmc[ � ­ � k dCmc[�ikVX[©nzxyik�¢Y�dC��dz|%|%[%bcb�¥«mc[%ʵfX[Z}X|Z��[%bu���¢¼�[Z}À�µji?p\,>�C5*

� ­3Õ Ì�Í�Ï ­Ã k­ � Ó Ì�Í^Ïh­ Ý

U�VµfXbe²

ÌWK��< ���� ­<�� k�< Õ � ­ � k Ý

w6[Zi*ã+¬ �:ÄCÎ � Ð�Õ � æ � � �Q��[�ikVX[�¥«m dz|Zik��nz}�dC�Mx�dCmki*nC¥ � ²+dC}X{�� <�Õ â�ã+¬ �:ÄCÎWI � Ò�Ó%Ð P8I4ÕLU Å ç Å Ý%Ý%Ý Å�<©æËç ä ¨

U�VX[��%¯ ikV4dz|%|%[%b b¿nC¥�ikVX[^mcnz�+nzi���b¿��{y[Z}�ik����[%{4¹u�¢ikV£ikVX[��Z¯ ikV£bkY�dC�¢��[%bki�x+nz�¢}�iunC¥�� <µ¨<rÑ}³ikVX[^��nz��{y[Z}4m dCik��n�x+nz�¢��|Zj�<Î 7 Å f Ð ²�ikVX[*x�nz�¢}µicb �ã+¬ �:ÄCÎWI � Ò�Ó Ð P

­ Ò�Ó:- � Ó

� -<�� k I �­:

- � � -<�� k��

|%nzmkmc[%bkx+nz}X{4icn�ikVX[�dz|%|%[%bcbc[%b�nC¥Mx�dC��[*¾c¨®U�VµfXbe²

hQÎ���Î 7 Å f.ÐkÐ�Õ�Ä S Ç4æ Ä S ��< �

k:­ � Ó��

Û��� �:- � Ó

AHç�æ¸Ï ? Û�­ C Å

¹uVX[Zmc[�ikVX[¿�¢}�ic[Zm dz|%|%[%b b®{h��bki dC}X|%[ E ­- ��bMdCYQnz}y�*ikVX[¿ikVymc[%[�}µfyY��+[Zmcb

�F Û ²

�F Û � Ó nzm

�F Û � !z²z¹uVX[Zmc[�Ie­3Õ��DÌ�Í! � ­ � k �

?È|´¥k¨D�¢\�\5��C´¨®qÁnzmc[%ne¼�[Zme²��¢i�|edC}À��[*bcVXne¹u}À�µj�Y��¢Y���| :µ�¢}y��ikVX[*xymcnµnC¥¶bu�¢} �¢\�\ �3ikV�dCi

hQÎ���Î f.ÐkÐ)ÔÖÕ ÌWK��< ��� hQÎ���Î 7 Å f.ÐkÐ

Õ Ä S Ç4æ.Ä S � � � ç�æ k:­ � Ó#" A � ­ � k æ � Ò F Û C Ï!$ � Û­

� A � ­ � k æ � Ò F ÛÈÒ�Ó C Ï $ � Û&% �­ æ A � ­ � k æ � Ò F Û � Ó C Ï $ � Û'%)(­ * � Ý

HJI êµî � ê �,+�M�N � 1212���n�Q%'��(q���� ¾ ����8� Ç+­�� U 61 f � �!�� ����8� à k­ � Ó Ç+­�ÕØÇ N U � � ����!

ÌWK��.-�/10k ���

hQÎ���Î f.ÐkÐ ËÄ S � Ç4æ � � ��A�ç�æ � Ò�Ó C Ù Ò3254687 9 � � � Ò�Ó Ù Ò:254 (6;7 9 � Ý ?p\ &�C

�7�áÿ8r9<&<MsBt

Page 19: Optimal Robot Scheduling for Web Search Engines

\ � .��������Q�����+8!���( ��������Q�"��a ��� ��&)+�,�B��(

Q � î�î�R M �¦j³Y��¢Y���|B::�¢}y��ikVX[©xymcnµnC¥�nC¥MU�VX[%nzmc[ZY 1h¨ ���¢} �¢\�\5�°²y¹¿[©|edC}ÀbkVXne¹ ikV�dCi

h�Î���Î f.ÐkÐ ËÄ S Ç4æ.Ä S � � � ç�æ k:­ � Ó � ­ � k " A ç^æ � Ò�Ó C�� ­ � k � � Ò�Ó �

(­ � k * � Å

¹uVX[Zmc[ � ­ � k ÕºÏ Ó�� 4 & Û � ��� � 6­ ¨QÞ8nzic[�ikV�dCi©¹uVX[Z}.Ç+­ � Uh²<Ïh­uÕºÙCÒXÚCÛ � � � ζÇ+­ÆÐ�bcn4ikV�dCi � ­ � k Õ Ç+­ $ Ç � � ζÇ+­ÑÐ ¨U�VX[%bc[©�¢Y�xy�¢j³ikV�dCi � ­ � k � ÙCÒXÚ � � � ¹uVX[Z}=f � ب (8[Z}X|%[�²h��j£}Xnzik�¢}y�QikV�dCi à k­ � Ó�� ­ � k Õ�ç�²y¹¦[©nz�yi dC�¢}

ÌWK��.-5/�0k ���

h�Î���Î f.ÐkÐ ËÄ S Ç4æ.Ä S � � � ç�æ A ç�æ � Ò�Ó C Ù Ò32546 7 9 æ � Ò�Ó Ù Ò#2�4 (6 7 9 � Ý

è<�¢}�dC�¢�¢j£¹¦[�|%nzY�x�dCmc[©ikVX[�mk�¢��V�ip¯ V�dC}X{Ábk��{y[*nC¥ ?p\ &�C¿¹u�¢ikVc?p\��8C´¨

� î � î # #«�y� ë + M�N � 1 12���n��%;��( 8�"� ¾ ����8� Ç+­�� U 61 f � �!�� ����8� à k­ � Ó Ç+­6ÕØÇ NOU � � ����!

ÌWK��-�/�0k �#�

hQÎ���Î f.ÐkÐh � g � !� Õ � � � �� ��ç�Ý.U � Ý ?p\e��C

Q � î�î�R M w3[Zi��©ÕÉÇ $ �+¨®��[��X}X[

�JÎ��ЮÔÖÕ�ÌWK��-�/�0k ���

h�Î��<Î f ÐkÐh � Õ �^æ¡ç �� ç�æ � Ò�Ó � Ù Ò 47 9�� � � Ò�Ó Ù Ò 4 (7 9���8æ�ç � Ù Ò �

Ý

� [�¹u�¢�¢�¦bkVXne¹ �+[Z��ne¹~ikV�dCi��JÎ��Ð^��b©{y[%|Zmc[edzbk�¢}y�>�¢}��6NVUh¨QU�VµfXbe²6�µj dCxyxy�¢jµ�¢}y�À�� (8n�bkxy�¢i dC�� b�mkfy��[�i°¹u��|%[�²�¹¦[nz�yi dC�¢}

-�/10���S �¸Î��Ð Õ ÌWK��

� �S �¸Î��Ð

Õ ÌWK��� �S ç^æ Ò�Ó

� �Ù Ò 47 9�� æ

� �Ù Ò 4 (7 9��

ç�æ.Ù Ò �

Õ ÌWK��� �S ( Ò � Ù Ò 47 9�� � ��

� Ù Ò 4 (7 9��Ù Ò �

Õ � ! æ � � ���� Õ g � !�(8[Z}X|%[�² ?p\e��C¿¹u�¢�¢�3VXnz��{À�»¥��@S«Î��Ð U�¥Ènzm�dC�¢���QN Uh¨

��7�7���

Page 20: Optimal Robot Scheduling for Web Search Engines

������������ ���������������������� �"!$#&%'��(*)+�,�-�����(����/.0!�#��"!��21 \��

rÑiu��b�|Z��[edCm�ikV�dCi��TSÈÎ��Ð U��»¥®dC}X{4nz}y�¢j£�»¥ >�Î��Ð Uh²y¹uVX[Zmc[

>�Î��ÐMÕ � ç�æ � æ�ç� � Ù Ò�47 9�� æ �� � Ù Ò 4 (7 9 � � �8æ�ç � Ù Ò � � æ � �^æ¡ç � A ç�æ � Ò�Ó C Ù Ò�47 9�� � � Ò�Ó Ù Ò 4 (7 9�� � ç�æ¸Ù Ò � � Ý

v�¥«ic[Zm�b nzYQ[©bk�¢Y�xy��[�dC�¢��[Z�ym dh²y¹¿[©nz�yi dC�¢}

æ >�Î��Ð� � Ù Ò � Õ � � ç�æ g� � � ç�æ¸Ù Ò �� � � � æ�ç

g � � � ��Ù Ò 47 9 � � � æ � ç�æ g� � � ç�æ¸Ù Ò �� � � � � çg � � � ��Ù Ò 4 (7 9 � Ý

w3[Zi

� Ó Õ � ç�æ g� � � ç�æ¸Ù Ò �� � � � æ¡çg � �

� !�Õ æ � ç�æ g� � � ç�æ¸Ù Ò �� � � � � çg � �

rÑi¦��b�[edzbkj�icn�bc[%[^ikV�dCi � Ó � � !�Õ�ç*dC}X{�ikV�dCiDU � � Ó �ç+?«fXbk�¢}y��ikVX[��¢}X[%ʵf�dC�¢�¢i°jÁç½æÁÙzÒ & � C´¨�U�VµfXbe²yne¹u�¢}y�icn�ikVX[�|%nz}�¼�[´gh�¢i°j4nC¥<ikVX[�[´g:x+nz}X[Z}�ik�DdC�6¥«fy}X|Zik��nz}ÁÙzÒ � & ²h¹¿[*V�d%¼�[

� � ç^æ g� � � ç�æJÙ Ò �� � � � æ�çg � � � �HÙ Ò 47 9 � � � æ � ç^æ g� � � ç�æJÙ Ò �� � � � � ç

g � � � �HÙ Ò 4 (7 9 � Ù Ò � � Å

¹uVX[Zmc[

� Õ � Ó � �� � � � ! � � !� � Ý

rÑiu��b�bk�¢Y�xy��[*icn�| VX[%|B:4ikV�dCi � ��ç�b n�ikV�dCi

� � ç�æ g� � � ç�æ¸ÙCÒ �� � � � æ¡çg � � � �HÙ Ò 47 9�� � � æ � ç�æ g� � � ç�æ¸ÙCÒ �� � � � � ç

g � � � �HÙ Ò 4 (7 9 � Ù Ò � � NSÙ Ò � Ý

U�VX[Zmc[´¥¶nzmc[�²6æ%>�Î��Ð $ �QN U�¹uVy��| VÀ�¢Y�xy�¢��[%b�ikV�dCi��TSÈÎ��Ð � Uh¨

� 2Ë��!��<������� !�"á �®��&���/

q¤dC}�j¤nC¥¦ikVX[�}µfyYQ[ZmcnzfXb�ʵfX[%bkik��nz}Xb���[´¥«i*fy}�dz{y{hmc[%b bc[%{¤��j¤nzfym*mcnz��nzip¯°b |cVX[%{hfy�¢�¢}y�Àmc[%bkfy�¢icb©��[edz{¤icn��¢}µic[Zmc[%bkik�¢}y�de¼�[Z}�fX[%b¦¥Ènzm®¥«fyikfymc[umc[%b [edCmc|cV3¨®èXnzm¿[´gydCY�xy��[�²�ikVX[Zmc[^dCmc[�ikVX[uYQnzmc[���[Z}X[Zm dC��Ê�fX[%bkik��nz}Xb½x�n�b [%{���j�bc[edCmc| V�[Z}y���¢}X[%bbc[Zmk¼�[%{³�µj�Y�fy�¢ik�¢xy��[�mcnz�+nzicbe²:¹uVy��|cV£Y�dej��+[8{h��bcikmk�¢�yfyic[%{�¨®v���b ny²��¢}Àd�|%nzY�xymc[ZVX[Z}Xbk�¢¼�[*{XdCi d���dzbc[�bkfX|cVÀdzb�v��¢i d98��bki dh²�ikVX[u}µfyY��+[Zm¦nC¥+x�dC��[%b�¼�dCmk��[%b�¹u�¢ikVQik�¢YQ[87µikVX[�|Zmc[edCik��nz}�nC¥+}X[Z¹¡� [Z�Qx�dC��[%b®|%nz}µik�¢}�fX[%b�dCx�dz|%[�²µdC}X{�Y�dC}µjx�dC��[%b�¹u�¢�¢�®��[%|%nzYQ[�{y[´¥«fy}X|Zie¨£ñ^}X[Q|edC}.dz{XdCxyi�icn£ikVy��b�{hjµ}�dCY���|�¼�[Zmcbc��nz} ��j§{y[Z�Dd%j:�¢}y�>ikVX[ �Xmcbci�dz|%|%[%bcb*icn>d�7�áÿ8r9<&<MsBt

Page 21: Optimal Robot Scheduling for Web Search Engines

\ & .��������Q�����+8!���( ��������Q�"��a ��� ��&)+�,�B��(

}X[Z¹u�¢j§dCmkmk�¢¼µ�¢}y�4x�dC��[�fy}µik�¢��ikVX[���[Z���¢}y}y�¢}y�ÀnC¥¿ikVX[�}X[´ghi*|Zj:|Z��[�¥¶nz�¢��ne¹u�¢}y�4�¢icb©dCmkmk�¢¼�dC��ik�¢YQ[�²3dCi*¹uVy��|cV§x+nz�¢}�i�d}X[Z¹(ic[ZY�xy�DdCic[���bu|%nzY�xyfyic[%{�¨ ?Èv�x�dC��[©{hmcnzxyx+[%{£¥«mcnzYºikVX[�{XdCi d���dzbc[���b�{y[Z��[Zic[%{£¥ÈmcnzYºikVX[*ic[ZY�xy�DdCic[�dzb�bcnµnz}dzb½ikV�dCi®¥ dz|Zi½�+[%|%nzYQ[%b ::}Xne¹u}³icn*ikVX[�mcnz�+nzi,7:dC��dC�¢}3²hd�}X[Z¹Sic[ZY�xy�DdCic[�Y�fXbki½�+[�|%nzY�xyfyic[%{�¥Ènzm®ikVX[�}X[´g:i¦|Zj:|Z��[�¨ C(8[Zfymk��bkik��|%b���dzbc[%{¸nz}¸ikVy��b�ic[%| Vy}y��Ê�fX[£bkVXnzfy��{¸x+[Zmp¥ÈnzmkYß¹¦[Z�¢�¿¹uVX[Z}¸ikVX[ (���� 8��� # � ¼�dCmk�DdCik��nz}JnC¥�ikVX[�{XdCi dÀ��dzbc[n�¼�[Zm8d�|Zjh|Z��[*��bubkY�dC�¢� ²Xnzm^dCiu��[edzbci�YQnµ{y[Zm dCic[Z�¢j4bcny¨

� [�V�de¼�[*dC�¢mc[edz{hj�x+nz�¢}�ic[%{4nzfyi��¢}>t:[%|Zik��nz}À��ikVX[�nzx+[Z}£xymcnz�y��[ZYQb�|%nz}y}X[%|Zic[%{£¹u�¢ikVÀbc[%ʵfX[Z}X|Z�¢}y��ikVymcnzfy��V>dbc[Zi�nC¥®x�dC��[%b^icn³dz| Vy��[Z¼�[�dQ���¢¼�[Z}Áb [Zi8nC¥¦dz|%|%[%bcb�¥Èmc[%Ê�fX[Z}X|Z��[%b*dC}X{sdCxyxymcn�dz| V.dzb^|Z��n�b [Z�¢j>dzb�x+n�bcbk�¢�y��[�ikVX[���n�¹¦[Zm�+nzfy}X{*�¢}�U�VX[%nzmc[ZY��h¨ �h¨<l�mcnz��mc[%bcb�nz}©ikVy��b�iÑjµx+[�nC¥yic[ZY�xy�DdCic[®{hmk�¢¼�[Z}�bc| VX[%{hfy�¢�¢}y��xymcnz�y��[ZY(V�dzb6��[%[Z}�¥ÈmkfXbkikm dCik�¢}y��¢}¡nz}X[4�¢Y�x+nzmki dC}�i�mc[%bkx�[%|Zi *4v^�¢ikVXnzfy��V¡ikVX[Zmc[>dCmc[£¼�[Zmkj�bk�¢Y�xy��[�²¿[�¨ �X¨¢²½��mc[%[%{hjJi°j:x+[�²½x+nz�¢��|Z��[%b�ikV�dCi�bc[%[ZY icnx+[Zmp¥ÈnzmkYámc[ZY�dCm�:zdC�y�¢j4¹¿[Z�¢� ²XikVX[Zmc[�V�dzb��+[%[Z}À�¢�¢ikik��[�²X�»¥½dC}µj�²�bkfX|%|%[%b bu�¢}s[%bki dC�y�¢��bcVy�¢}y��ikVy��bu¥ dz|Zi8dC}�dC�¢j:ik��|edC�¢�¢j�¨�r°}��[Z}X[Zm dC� ²�nz}X[©Y�fXbki8[´ghx+[%|Zi�ikVX[%bc[�{y[%bc�¢��}ÁdC}X{sdC}�dC�¢jhbk��b�xymcnz�y��[ZYQb^icnQ��[%|%nzYQ[�[Z¼�[Z}sYQnzmc[�|%nzY�xy�¢��|edCic[%{>¹uVX[Z}ikVX[�de¼�nz��{XdC}X|%[�nC¥��´m dCxy��{:¯ �Xmc[�� ?¶dz|%|%[%b bk�¢}y��d�� [Z�>bk�¢ic[�icn:n�¥Èmc[%Ê�fX[Z}µik�¢jAC¿��b���bui d6:�[Z}À�¢}�icn�dz|%|%nzfy}�ie¨

��#%��&��®!��<��/

�¢\5��qJ¨J(�¨�v�Y�Y�dCm�dC}X{��X¨���¨�� nz}y�X²��pñ^} ikVX[Qnzxyik�¢Y�dC�¢�¢i°j nC¥�|Zjh|Z�¢��|�ikm dC}XbkY���bcbc��nz} �¢} ic[Z��[Zic[´ghi�bkjhbkic[ZYQbe² �O .�. . � (��!A1 ���Q�8�d�,� ²:¼�nz� ¨ �uñ8qÀ¯Ñ��1h²X}Xny¨3\�²yxyx3¨A��&H¯Æ�z�h²���dC}3¨�\e��&��:¨

� ����qJ¨yv^}X{hmc[Z¹�be²yv�¨hèX[Zmk}�dC}X{y[Zª�²�q¸¨ (�dCmc|cVXnz�»¯ �udC�¢ic[Zme²�U*¨yw6[Z�¢��V�icnz}3²�dC}X{4w½¨�V�dC}y�X²��ÑU6[ZY�xy�DdCic[�v��¢��nzmk�¢ikVyY¥¶nzm¦ñ^}X[´¯ (8nzx�l<dz| :�[Zi¦_^nzfyik�¢}y�X² �uw�dC��nzm dCicnzmkj�¥¶nzm �¿nzY�xyfyic[Zm�t:|Z��[Z}X|%[�²�q¤dzbcb dz| V�fXbc[Zikicb¦r°}Xbcik�¢ikfyic[unC¥+U�[%|cVh¯}Xnz��nz��j�² ��dCY��ymk��{h��[�²�qsv >��h\e���h²3\e����� ?«xymc[Zxymk�¢}µi2C´¨

� ��� <�¨�v�mk�DdC}3²�<�¨�w6[Z¼µj�²�Ñv^�¢��nzmk�¢ikVyYQb8¥Ènzm ��[Z}X[Zm dC�¢�¢ªe[%{§_�nzfy}X{¤_^nz�y�¢}Á_^nzfyik�¢}y��� ² � �[��( �a �21 � � �;� �p��(21 ²�9®nz� ¨\ �h²Xxyx3¨��h\e�H¯Ñ�h\e�h²3\e�����h¨

� �!��t�¨ �¿nzmcbcie² �����m��!$#�`o61��p���d1 ²yl®V3¨ ��¨�U�VX[%bk��be² �¦�Ør�²yU�VX[©Þ8[ZikVX[Zmk�DdC}X{ybe²�\e���'�X¨� 1���t�¨ ��nzmcbkie²½ñ�¨��X¨ ��n%ghY�dh²��X¨�(�¨<v�¨�(�dCmk�¢}A:�² ��¨ ��¨�(�fy�¢ic[ZY�dh²��pñ^xyik�¢Y��¢ª�dCik��nz}¡nC¥8è��»gy[%{JU��¢YQ[³l<nz�¢�¢�¢}y�

t:| VX[ZYQ[%b�� ² � ��� ���B��� � ��![��� 8�����8!A1-�`o61��p�;�q1 ²�9®nz� ¨X�h²yxyx3¨��h\´¯ 1��h²3\e���'�X¨� ����ñ�¨��X¨ ��neg:Y�dh² (�¨�w3[Z¼:j�²��X¨µv�¨�� [%bkicbkikm dCic[�²��Ñð!��|Z��[Z}�i 98��bk�¢i½èymc[%ʵfX[Z}X|Z��[%b¦¥Ènzm¿l�nz�¢�¢�¢}y��U�dC�y��[%b *�q>�¢}y�¢Y��¢ª�dH¯

ik��nz}snC¥<�.dC�¢ik�¢}y���¿n�bci�� ²�� �A���$12��!$# �`o61;�p���d1 ²�9®nz� ¨��h²yxyx3¨�\e���H¯ \ ���h²�\e���h\�¨

� � ��ñ�¨�X¨A��n%ghY�dh²A(�¨Xw3[Z¼:j�²��X¨Xv�¨h� [%bkicbkikm dCic[�²��Ñð!��|Z��[Z}�i 9^��bk�¢i^ñ^mc{y[Zmcb�¥¶nzm�l�nz�¢�¢�¢}y�4tµj:bkic[ZYQb�� ²� ��(�% � . # �� � ²9®nz� ¨6\ &h²yxyx3¨�\,>��H¯ \ ���h²6\e�����h¨

� &���(�¨6��¨3�8j�:�[ZY�dC}3²�q¸¨ (�¨�v�Y�Y�dCme²�dC}X{���¨3��¨3� nz}y�X²��pt:|cVX[%{hfy�¢�¢}y� dC�¢��nzmk�¢ikVyYQb�¥Ènzm�¼µ��{y[%nzic[´g.bcj:bkic[ZYQbfy}X{y[Zm��ymcn�dz{y|edzbki^{y[Z�¢�¢¼�[Zmkj�² �� (��,� �Q��% O � ��� ��� ²hxyx3¨�\ &'�µ��¯ \ &�1h\�²6\e��&��h¨

� ������¨(�dHop[;:+²��ÑðMghikmc[ZY�dC�Mtµxy�¢�¢ikik�¢}y��b^nC¥Ml<nz�¢}�i8l�nzmc|%[%bcb [%b�� ² � ���� � ���A��( � �21 � ²�9®nz� ¨3\,>h²Xxyx3¨A1'���H¯ 1�1��h²6\e��&�1h¨��7�7���

Page 22: Optimal Robot Scheduling for Web Search Engines

������������ ���������������������� �"!$#&%'��(*)+�,�-�����(����/.0!�#��"!��21 \e�

�¢\,>���qJ¨8(8nC¥«mk� ² <¨�_�n�bc��[Zmk�X² �ÑlMdz|B:�[Ziu��[Z�Dd%j�fy}X{y[Zm¦ikVX[ ��nz��{y[Z}�_8dCik��n*� [Z�¢��V�ic[%{³Uu��q$l�nz�¢��|Zj��¢}�d©q>fy�¢ik�¢xy��[´¯vu|%|%[%b �¦V�dC}y}X[Z� � ² O . .�. � ( 8!A1 � ��! O !,%'��(��=� � ���B��(2o ²A9®nz� ¨X���h²Xxyx3¨��'�X\´¯Ñ�'���h²3\e��&��:¨

�¢\�\5��v�¨XrÑi dC� ² <¨X_^n�bk�+[Zmk�X²��Ñv ��nz��{y[Z}>m dCik��n�|%nz}µikmcnz�6x+nz�¢��|Zj£¥Ènzm8d�Y�fy�¢ik�¢xy��[´¯Ñdz|%|%[%bcb�| V�dC}y}X[Z� � ² O . .�. � (��!A1;�� ���� ��!A1 ��! � ���S���8����� �Q��!`��(���� � � ì � � ²yxyx3¨��:\ �H¯Æ�:\ &h²6\e��&'�X¨

�¢\ ������¨�è�¨ �8¨8D��¢}y��Y�dC}3² �ÑU�VX[�ð®mk��nµ{h��|�U�VX[%nzmkj�nC¥6tµfy��dz{y{h�¢ik�¢¼�[8tµicn:|cV�dzbkik��|�l®mcnµ|%[%bcb [%b�� ² � �` ���o �`���S8���"1�� �`���M� �� ��( ��� ² K�� ²hxyx3¨�����H¯ 1h\,>h²3\e����&h¨

�¢\e��� <¨�w��¢f3²�l<¨�Þ�dC�¢}3²��pñ^xyik�¢Y�dC�Mt:| VX[%{hfy�¢�¢}y���¢}¤t:nzYQ[�qsfy�¢ik�»¯ )^fX[ZfX[�tµ�¢}y����[´¯Æt:[Zmk¼�[Zm©tµj:bkic[ZYQb�� ² O . . . � ( �![1 ���'�����8!A1 ��! � ���S���8��� �%�Q��!`��(���� ²�9®nz� ¨����:²XÞ^ny¨A�h²hxyx3¨R�'�µ��� ��1��h²6\e�����h¨

�¢\ �!��v�¨:� ¨hq¤dCmcbkV�dC�¢�3dC}X{4r�¨yñ^� ::�¢}3² O !����;�A�� �����S�21� � ���B��(2o/� % � �'��(2� �,�������!e�!��+����1 � ����� ��� 8�����8!A1 vu|edz{y[ZY���|l®mc[%bcbe²XÞ8[Z¹ <½nzm�:�²�\e���z�h¨

�¢\ 1�����¨utµicnej�dC}3² �Q��� ���(�� 1'��! � �;�����,�61q%;��( � �A���[�21 �!�� ��������(*���S�,����61;����� � �,�����m1 ¨¿ð�}y���¢��bkVØikm dC}Xbk�DdCik��nz}?È��¨ �X¨���dC��[Zj4[%{h�¢icnzm2C´²���¨y�S�¢��[Zj�� t:nz}Xbe²XÞ8[Z¹ <½nzm�:+²3\e��&��h¨

�¢\ ��� (�¨yU�d6:�dC��� ² � !���mo61�� 1-� % ����� �"!$#�`o61��p�;�q1�� qsrÆUØl�mc[%bcbe²6\e��&��h¨�¢\�� � (�¨�U<d6:zdC��� ²�� )^fX[ZfX[Z�¢}y�4dC}�dC�¢j:bk��b8nC¥�x�nz�¢�¢�¢}y�³YQn:{y[Z��be² � ���S�,����61���� �+�!��� o612�"1q� %d� ��� �����p�;( �!�� �B��� � �$![� �

� 8�����8! ��o^1;�p���d1 ²(�¨yU<d6:zdC��� ?Èð½{�¨ C´²�Þ8nzmkikV4(^nz�¢�DdC}X{�²�\e���8>h¨

�7�áÿ8r9<&<MsBt

Page 23: Optimal Robot Scheduling for Web Search Engines

Unité de recherche INRIA Sophia Antipolis

2004, route des Lucioles - B.P. 93 - 06902 Sophia Antipolis Cedex (France)

Unité de recherche INRIA Lorraine : Technopôle de Nancy-Brabois - Campus scientifique

615, rue du Jardin Botanique - B.P. 101 - 54602 Villers lès Nancy Cedex (France)

Unité de recherche INRIA Rennes : IRISA, Campus universitaire de Beaulieu - 35042 Rennes Cedex (France)

Unité de recherche INRIA Rhône-Alpes : 655, avenue de l’Europe - 38330 Montbonnot St Martin (France)

Unité de recherche INRIA Rocquencourt : Domaine de Voluceau - Rocquencourt - B.P. 105 - 78153 Le Chesnay Cedex (France)

Éditeur

INRIA - Domaine de Voluceau - Rocquencourt, B.P. 105 - 78153 Le Chesnay Cedex (France)��������� ���� ���������� ��� ���

ISSN 0249-6399