1 networks the internet may be described as part technology and part human interaction. to describe...

1

Networks

The Internet may be described as part technology and part human interaction. To describe it as one or the other is not quite accurate. Unlike other technologies, it does not do anything in the absence of the human mind – in fact, the human mind is the sole source of its viability. Accordingly, the destiny of the Net will be determined by the interaction of two adaptive agents:

• The systems and software of the Net

• Its human users

Network Theory… and other stuff

Source: Valovic, “Digital Mythologies”

2

Networks

The Web is a Network … Not only that, the Web is complex network… so says Sir Tim Berners-Lee (and just about every other scientist in the world who is doing research on networks or complexity theory).

So let’s take this as a given. The Web is a Complex Network


Graphic view of the Web (by tracing links)

Each color on this Opte map represents a region; North America, blue; Europe/Middle East/Central Asia/Africa, green; Latin America, yellow; Asia Pacific, red; Unknown, white. (Image: Opte.org)

Source: NewScientist.com

4


Elements of complex systems •They are dynamic in the sense they are constantly changing.•They are adaptive, which is to say they evolve to benefit themselves and to insure their survival.•The adaptations are controlled to some extent by the interactions of the entities that comprise the system.•The control is typically highly dispersed. •They exhibit many levels of organisation.•They are comprised of many niches.•They are self-organising.•New elements or entities emerge from complex systems.•The emergent elements are not necessarily predictable from analysis of the individual parts of the system.•Complex systems are defined by relationships between components more than by describing its constituent parts.

5


Research published by scientists at Notre Dame in 1999 indicated that there were fundamental attributes of most networks, including the Internet and the Web, in that they:

•Exhibited rapid and/or consistent growth.

•Exhibited a power law distribution.

•Exhibited forms of preferential attachment.

6


Rapid and/or Consistent Growth

As the chart shows, the number of Websites has experienced three growth stages:

•1991-1997: Explosive growth, at a rate of 850% per year. •1998-2001: Rapid growth, at a rate of 150% per year. •2002-2006: Maturing growth, at a rate of 25% per year.

Source: Jakob Nielson

http://www.useit.com/alertbox/web-growth.html

Netcraft's latest Web survey found 101,435,253 websites in November 2006. Not all of these sites are live: some are "parked" domains, while others are abandoned weblogs that haven't been updated in ages. But even if only half the sites are maintained, there are still more than 100 M sites that people pay to keep running.

Total sites across all domains

7

Total sites across all domains

The 100 million site milestone caps an extraordinary year in which the Internet has already added 27.4 million sites, easily topping the previous full-year growth record of 17 million from 2005. The Internet has doubled in size since May 2004, when the survey hit 50 million.

Blogs and small business web sites have driven the explosive growth this year, with huge increases at free blogging services at Google and Microsoft. Domain industry juggernauts Go Daddy (U.S.) and 1&1 Internet (Germany) have also seen strong demand for low-priced domain names and shared hosting accounts.

Rapid and/or Consistent Growth


8


Power Law Distribution

What is a “Normal Distribution”?

9



What is a “Normal Distribution”?

A normal distribution of data means that most of the examples in a set of data are close to the "average," while relatively few examples tend to one extreme or the other.

10



Until relatively recently (the mid 1990’s) it was generally assumed that many (most?) networks exhibited a normal distribution of nodes and edges.

A normal distribution of data means that most of the examples in a set of data are close to the "average," while relatively few examples tend to one extreme or the other.

11



By 1999 several scientist’s had published papers indicating the nodes of Web did NOT form a normal distribution when measured by the edges (or [inbound] links), but instead formed a ‘Power Law’ distribution.

Adamic and Huberman – “The Webs Hidden Order”, Communications of the ACM, 2001Barabasi and Albert – “Emergence of Scaling in Random Networks”, Science, Vol 256, Oct, 1999

Power laws as related to websites may be verbally represented as:

•a very few sites that rank very high in the number of inbound links;

•a larger number of sites with close to median numbers of inbound links;

•a great number of sites with very few inbound links.

12




•a very few sites that rank very high in the number of inbound links;

•a larger number of sites with close to median numbers of inbound links;

•a great number of sites with very few inbound links.

13




•a very few sites that rank very high in the number of inbound links;•a larger number of sites with close to median numbers of inbound links;•a great number of sites with very few inbound links.

806040200

count

1200

1000

800

600

400

200

0

in-b

nd

lin

ks

Linear representation of distribution of inbound links to websites categorized as ‘Business’ in the dot-com zone.

1001010.1

1E-5

count

10000

1000

100

10

1

-0.1

in-b

nd

lin

ks

R Sq Linear = 0.61

Logarithmic scale of distribution of inbound links to

websites categorised as ‘Business’ in the dot-com zone

14


806040200

count

1200

1000

800

600

400

200

0

in-b

nd

lin

ks

This is the Long Tail of the Power Law distribution

Shallow Web

Deep Web


15



16



So the question is – WHY?...

Why is there a power law distribution?

Why is it important?

1 networks the internet may be described as part technology and part human interaction. to describe...

Documents

network theory

rapid growth

explosive growth

year growth record

growth stages

maturing growth

stufftotal sites

normal distribution