playing the search enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… ·...

44
Frédéric Philipp Thiele / Prof. Hendrik Speck Playing The Search Engines or Hacking The Box Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V. Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele Prof. Hendrik Speck

Upload: others

Post on 12-Jun-2020

16 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Playing The Search Enginesor Hacking The Box

Frédéric Philipp Thiele Prof. Hendrik Speck

Page 2: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Google Hacking

Page 3: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Google Hacking

Search engines make it easier for everyone to gain information, hackers

included. “ ”Danny Sullivan editor of searchenginewatch.com

Page 4: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Google Hacking

Database with more than9 Billion documents

• Personal data• Passwords• User accounts• Webcams• Databases• Password Secured Areas• Customer Data• Security Bugs• …

Concept

Page 5: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Google Hacking

Database with over 9 Billion Documents

++ ??powerful

Search Functionality

==

Page 6: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Google Hacking

Google Hacking is a special search technique used to find hidden information, break into sites, and access

supposedly secure information. Using search engines such as Google, "search engine hackers" can easily find

exploitable targets and sensitive data.

Definition

Page 7: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Search Functionality

Page 8: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Google Advanced SearchOverview

You can do a lot more with Google search than just typing in search terms. With Advanced Search, you can search for pages:

• that contain ALL the search terms you type in

• that contain the exact phrase you type in

• that contain at least one of the words you type in

• that do NOT contain any of the words you type in

• written in a certain language

• created in a certain file format

• that have been updated within a certain period of time

• that contain numbers within a certain range

• within a certain domain, or website

• that don't contain "adult" material

Source: http://www.google.com/help/refinesearch.html

Page 9: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Advanced Search Functionalitylink:wikipedia.org

Page 10: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Advanced Search FunctionalityThe Beatles - Octopus's Garden

"I'd ask my friends to come and see"

relevant searchresultirrelevant searchresult

Page 11: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Advanced Search Functionalitylecture search engines filetype:ppt

Page 12: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

User Account Hacking Getting your Virtual Identity

Page 13: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

HIT JAMMER 1.0 - CMSSearch String: POWERED BY HIT JAMMER 1.0!

Many administrators fail to protect the admin panel (/admin/admin.php) with the .htaccess logon procedure. In such cases, customer information like email and passwords are in clear view of the attacker.

Page 14: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Password SecurityDon't use the same passwords for different services.

Page 15: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Password SecuritySearch String: ronboy51

Page 16: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Password SecuritySearch String: ronboy51

A search for ronboy51 offers about 100 potential targets.

Page 17: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Password SecurityDon't use the same passwords for different services.

Username orRealname or

E-MailPassword++

Page 18: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Password SecurityDon't use the same passwords for different services.

Username orRealname or

E-MailPassword++

??????

Page 19: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Application Hacking

Page 20: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

WS FTP – FTP PasswordsSearch String: intitle:index.of ws_ftp.ini

WS_FTP.ini is a configuration file for a popular win32 FTP client that stores usernames and weakly encoded passwords.

Page 21: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

WS FTP – FTP PasswordsSearch String: intitle:index.of ws_ftp.ini

[GAUSS]HOST=gauss.xxx.eduUID=ssaperstP W D =V B C 76D E 5F7E 25A 74422C 7E 21… 9CPASVMODE=0TIMEOFFSET=0LOCDIR=H:\webDIR="/export/home/fac/ssaperst"TYPE=6010DOUPDATE=1HASH=1FORCLOW=1CONVEXT=0rdir0="/export/home/fac/ssaperst"ldir0=H:\webldir1=H:\sites\math 413ldir2=H:\sitesldir3=H:\

ws_ftp.ini

WS_FTP.ini is a configuration file for a popular win32 FTP client that stores usernames and weakly encoded passwords.

Page 22: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

WS FTP – FTP PasswordsSearch String: intitle:index.of ws_ftp.ini

Page 23: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Other Examples

Source: http://johnny.ihackstuff.com

profiles!Host=*.* intext:enc_UserPassword=* ext:pcf

VPN Profiles

ext:ini eudora.ini Eudora Mailclient Configuration Files

"parent directory" +proftpdpasswd ProFTPd. Password

ext:ini Version=4.0.0.4 password servU FTP Server Configuration File

intitle:index.of trillian.ini Trillian Messenger Password File

filetype:inf sysprep MS Windows Configuration FileAdministrator's passwords, IP addresses and product IDs

Page 24: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Database Hacking

Page 25: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

phpMyAdminSearch String: "phpMyAdmin" "running on" inurl:"main.php"

phpMyAdmin is a tool written in PHP intended to handle the administration of MySQL over the WWW.

Onlineshop Database with customer details and credit card informations.

Page 26: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

phpMyAdminSearch String: "phpMyAdmin" "running on" inurl:"main.php"

Page 27: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Digital Device Hackingand Default Passwords

Page 28: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Linksys WebcamSearch String: vr intitle:"Linksys wireless-G Internet Video Camera"

Default Username: admin Default Password: admin

There are a lot of digital devices with HTTP configuration interface.Most users forget to change the default passwords of their digital devices.

Page 29: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Search Engine Market AnalysisUsing Google as a Research Tool

Page 30: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

5.9%

42.7 %

Search Engine MarketcomScore. March 2006 (USA)

OTHERS13.2 %

28.0 %

2.6 %

7.6 %

Source: comScore. Search Engine Rankings. March 2006, Available: http://www.comscore.com/press/release.asp?press=802

Page 31: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

84.0 %

Search Engine MarketWebHits. March 2006 (Germany)

OTHERS

4.4 %

4.4 %4.8 %

2.4 %

Lycos 0.6%Altavista 0.5% T-Online 0.5% WEB.DE 0.4% suche.freenet.de 0.4%Meta.Ger 0.3% AllesKlar 0.3% arcor.de 0.3%Web.de directory0.2%search.com 0.2%fireball.de 0.2%DMOZ 0.1%

Source: WebHits.de. Web Barometer. March 2006, Available: http://www.webhits.de/deutsch/webstats.html

Page 32: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Statistics - Search Engine MarketIntroduction

Different technologies and methodologies have resulted in different measurement standards and indicators.

Statistical problems, including active and latent errors, different target groups, samples, sample sizes and objectives lead to different results.

Different results support misuse, bias, and intentional errors.

Page 33: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Statistics - Search Engine Market

1 Miuse, Bias, and Intentional Errors

2 Environment and Conditions

3 Methods, Procedures, and Processing

4 Indicators and Understanding Data

5 Data, Sample, and Sample Size

Problem Sources

Page 34: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Statistics - Search Engine Market

Search Engines Search Engines (Bias and Business Secrets)

Examples: Google, MSN, Yahoo, Ask Jeeves)

User Surveys/ProxiesUser Surveys/Proxies(Procedure, Reliability, Market, Sample, Sample Size)

Example. Nielsen.

Logfile AnalyzerLogfile Analyzer(Real Time, Source, Completeness, Focus, Sample, Sample Size)

Examples. Analog, AWStats

Problem Sources

Page 35: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Statistics - Search Engine Market

Web Tracker Web Tracker -- Tracking Software/Counter Tracking Software/Counter installed on every Pageinstalled on every Page(Real Time, Source, Completeness, Market, Focus, Sample, Sample Size)

Examples. Google Analytics, Webposition

Web Analytics Software Web Analytics Software -- Logfile AnalyzerLogfile Analyzer(Based on Logfiles, Source, Focus, Sample Size, Focus, Resources)

Examples. Sawmill

Problem Sources

Page 36: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Searching for Logfilesintitle:"Usage Statistics for" "Generated by Webalizer"

Page 37: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Searching for Logfilesintitle:"statistics of" "advanced web statistics"

Page 38: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Searching for Logfilesintitle:"statistics of" "advanced web statistics"

Page 39: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Regular Expression

A regular expression (abbreviated as regexp or regex, with plural forms regexps, regexes, or regexen) is a string that describes or matches a set of strings, according to certain syntax rules. Regular expressions are used by many text editors and utilities to search and manipulate bodies of text based on certain patterns.

Definition

Source: Wikipedia. Regular Expression. Available: http://en.wikipedia.org/wiki/Regular_expression

Page 40: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Searching for LogfilesDifferent Search Queries

in title:"statistics of" "ad van ced w eb statistics“

allinurl:awstats.pl?month=

allinurl:?output=refererse

80.000294.000

11.500

Page 41: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Searching for LogfilesDifferent Search Queries

"Estatísticas de acesso ao servidor WEB" site:.br "Estatísticas de acesso ao servidor WEB"

Portuguese Language Brazil

Page 42: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Searching for LogfilesDifferent Search Queries

site:.cn ....

Chinese Taiwanese British

site:.co.uk

site:.tw .... site:.co.uk ....

Page 43: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Searching for LogfilesFunctional Overview

1. search engine query2. parsing search engine result

pages (regex)

3. identifying domain and cc4. crawling search engine results

5. parsing data sets (regex)6. data computing, consolidation

and analysis (ip2dns)

7. database8. filtering and dataset creation

Collecting Analysing Selecting

Page 44: Playing The Search Enginesthe-eye.eu/.../presentation2006searchengineworkshophackingthebox.… · Playing The Search Engines or Hacking The Box Frédéric Philipp Thiele / Prof. Hendrik

Frédéric Philipp Thiele / Prof. Hendrik SpeckPlaying The Search Engines or Hacking The Box

Suma e.V. - Gemeinnütziger Verein zur Förderung der Suchmaschinen-Technologie und des freien Wissenszugangs e.V.

Questions?Frédéric Philipp Thiele Prof. Hendrik Speck