NetCache 5.3.1Overview

ObjectivesWhen you have completed this module you will be able to describe the following concepts:

• NetCache architecture• NetCache models• Managing objects• Request distribution and

load balancing

• Deployment strategies• Internet Content

Adaptation Protocol (ICAP)• Content Delivery Networks• Global Request Manager

Web Without Caching

Clients make requests directly to web servers and web servers respond directly

Internetor

WAN

Remote Web Server

Browsers(Clients)

Page 1

Web With Caching

• Web Caching Local replication (storage) of frequently used web objects to reduce network latency and bandwidth consumption.

• Benefits– Lower bandwidth costs– Faster response time / better quality of service– Enhanced security

Page 1

Browsers

Internetor

WAN

Remote Web Server

Copy of Page 1

NetCache

Architecture

Client

Server

Router

Firewall

Internet

NetCache

NetCache

NetCache Hardware Series

PERFORMANCE

C2100

C3100/C6100

C1100/C1105

C1200

Protocols• HTTP• FTP• Gopher• SSL• SSH• DNS• SNMP

• ICAP• NTP• WCCPv2.0• NNTP• RTSP• MMS• ICP

Software Architecture

Filtering and Client Side Access Controls

TCP Transparency API

SSLTunneling

ExtensibleProtocols

UserAuthentication DB

User Filtering DB

TCP/IP Stack

TCP/UDP Socket API

Protocol Fetch ModulesHTTP / FTP / Gopher / ICP / SSL Tunnel / Ext. Fetch

WAFL

MemoryManager

File LayoutManager

Storage ManagerObjectMap

Disk I/O Buffer Pool

HTTP 1.1Persistent

Connections

Client Side Protocol Processing

Types of Proxy Servers

• Proxy Server (NetCache)

• Accelerator (Reverse proxy)

Client

Server

Internet

NetCache

ClientServer

Internet

NetCache

What a Web Cache CachesNetCache appliance caches objects that use

– HTTP (Web content– FTP (file transfer)– Gopher (document search and retrieval)– MMS– RTSP

• NetCache caches only cacheable objects• Proxies noncacheable objects (CGI scripts or

private pages) without storing them

Which Objects Are Cached?• An object's HTTP MIME header contains one of the

following:– Cache-Control: no-cache – Cache-Control: no-store – Cache-Control: private– Pragma: no-cache

• An object's HTTP MIME header contains a Set-Cookieline

• NetCache can define other types of objects as either cacheable or noncacheable

Time To Live (TTL)

• Each object has a TTL• TTL is determined in the following ways:

– NetCache uses the object’s Cache Control Header

– Settings entered in the NetCache– NetCache calculates a default TTL

• Once NetCache has these numbers, it takes the smallest

Data Validity

• Refresh rate– During an object’s lifetime, NetCache periodically issues

an If Modified Since (IMS) request to the web server– Based on the response from the web server, NetCache

decides whether an object is still valid

• For example:– An ISP guarantees that their objects are no older than

20 minutes, so they set the refresh rate to 20 minutes – After 20 minutes has expired an IMS request will be sent

to the origin server the next time the object is requested

Request Distribution

• Transparent caching• Automatic Proxy Configuration File• Round robin DNS

Request Distribution - Switch

Client

L4 or L7Switch

World Wide Web

Client

Client

IP Address A

IP Address B

IP Address C

HTTPRequests

For cache misses, objectsfetched from web

NetCache

NetCache

NetCache

Request Distribution With Automatic-Proxy Configuration File

• Can distribute requests based on IP address

• No intercache communication required• Browser-based traffic portioning not as

efficient as switch• All browser protocols

Request Distribution with Round Robin DNS

• Multiple IP addresses assigned a single DNS host name

• Distributes requests in a cyclical pattern• Same objects can be cached in more

than one cache• Cannot detect a failed NetCache• Inexpensive method of load balancing

Failover Strategies

• Failover as part of a transparent deployment

• Automatic Proxy Configuration File• NetCache Appliance takeover pairs

Failover With L4 Switch (Transparent Deployment)

Switch fails:

Switches can be deployed in pairs for fail over protection

NetCache fails:• Switch detects the

failure • Switch fails over to

the web• Lose URL blocking

or logging • Transparent to

client

NetCache

Failover With Multiple NetCaches (Transparent Deployment)

ClientL4 Switch

Client

Client

Router

Internet

NetCache NetCache

Failover With Automatic Proxy File

• JavaScript file installed on client• Fail over to another web cache or

the web• Not all client browser versions fail

over correctly

Failover With NetCache Takeover Pair

CrossoverCable

NetCacheTakeoverPartners

NetCache NetCache

Firewall Deployments

• Parallel to the firewall• Inside the firewall

Parallel to the Firewall

Inside the Firewall

Web Accelerators

• Single Web Accelerator• Accelerator Advantages• Distributed Web Accelerators• Multiple Accelerators• Accelerating Multiple Web Servers

Accelerator Advantages

• Shields the web server from the outside world

• Offloads the work from the web server and firewall

Single Web Accelerator

Distributed Web-Site Accelerators

Multiple Accelerators

Single Accelerator Multiple Web Servers

Internet Content Adaptation Protocol (ICAP)

Virus Checker

Content Filter

LanguageTranslator

Ad insertion

Legend: ICAP servers for Compute-Intensive Operations

NetCache

Distributed Caching

• Supports Inter Cache Protocol (ICP)– Peer-to-peer proxy protocol– Compatible with Harvest-based caches

• Increases availability• Improves scalability through hierarchies

and clusters

DNS Caching

DNSServer

L4 Switch NetCache

• Host → IP lookup• Proxy other requests

Data Center Remote PoPs

Internet

Filer NetCache

Servers

Reporting and MonitoringContentReporterAppliance Watch

FilerView

Distribution & ManagementContentDirector

Snap Mirror

NetCache

NetCache

FilerPeering Partner

Commercial CDN

Global Request Manager (GRM) Concepts and Assumptions

• A Content Delivery Network is intended to move content as close as possible to the users

• A Global Request Manager manages the network of caches in a CDN, and is responsible for directing client requests to the closest cache

• Requests for content need to be directed to nearby edge caches without requiring reconfiguration of the customer’s environment

GRM Server• DNS used to redirect clients• NetCache assumes the identity of the

authoritative DNS • Client queries are resolved by DNS

servers authoritative for the domain• IP addresses returned to the client will

be the closest ones available

GRM Agent

• Reports cache load to the GRM server• Makes proximity measurements to

addresses provided by the GRM server• Reports results and subsequent

changes to GRM server

LocalName Server

Top-LevelName Server

(1)

(2)(3 & 4)

GRMName Server

(5)

(5)(6)

OriginServer

Agent

Agent

Agent(s)

(5)

(5)

Client

NetCache

NetCache

NetCache NetCache

NetCache

L7 Redirection

(1)

GRML7 Server

OriginServer

Agent

Agent(s)(2)

(3)

(4) www.cdn.netapp.com

Topic Review• What is web caching?• What is the difference between a forward and

a reverse proxy?• What is transparent caching?• How are cache contents kept current?• What is fail over and how is it accomplished?• What are some methods to load balance

multiple caches?• What does the Global Request Manager do?