world wide web: introduction · 2016. 11. 7. · networking: simpliﬁed the internet routes ip...

World Wide Web: IntroductionDan S. Wallach and Mack Joyner, Rice University

Copyright © 2016 Dan Wallach, All Rights Reserved

What, exactly, is the Web?

The Web is a whole stack of thingsOSI “7-Layer Model” https://en.wikipedia.org/wiki/OSI_model

Layer Data Unit Function Examples

Host layers

7. Application

Data

High-level APIs HTTP, FTP, SMTP, SSH

6. Presentation Specific file formats, data formats HTML, JPEG, JSON

5. Session Managing communications SSL/TLS

4. Transport Datagram Reliable transmission between any two nodes on a network TCP, QUIC (concurrent HTTP)

Media Layers

3. Network Packet Addressing, routing, traffic control IPv4, IPv6, old stuff like AppleTalk

2. Data link Frame Reliable transmission of data frames between two nodes PPP, Ethernet MAC

1. Physical Bit Transmit/receive raw bits over a physical medium Ethernet, DSL, WiFi

https://en.wikipedia.org/wiki/OSI_model

Networking: simplified

The Internet routes IP packets.

Every packet has a source and destination address. Routers don’t need to do anything else. Just pass on the packet. (Yes, it’s more complicated than that. For now we don’t care.)

The Internet makes no promises. Packets might not arrive. It’s up to the transport layer (TCP) to notice failures and retransmit. Subtle details matter, especially congestion control / fairness.

If you want to understand “net neutrality”You’ll want to learn a lot more. For example, whose fault is it if your connection to Netflix doesn’t work? Netflix blames Verizon/AT&T; they disagree. The details matter.

blog.level3.com/open-internet/verizons-accidental-mea-culpa/

http://blog.level3.com/open-internet/verizons-accidental-mea-culpa/

If you want to understand “net neutrality”You’ll want to learn a lot more. For example, whose fault is it if your connection to Netflix doesn’t work? Netflix blames Verizon/AT&T; they disagree. The details matter.

blog.level3.com/open-internet/verizons-accidental-mea-culpa/

Take C

omp42

9, learn

lots

more a

bout th

is and o

ther

related

topics!

http://blog.level3.com/open-internet/verizons-accidental-mea-culpa/

The web happens on topSeveral technologies combined to make it happen:

URLs (uniform resource locators) scheme:[//[user:password@]host[:port]][/]path[?query][#fragment] https://en.wikipedia.org/wiki/Uniform_Resource_Locator

HTML (hypertext markup language)

HTTP (hypertext transfer protocol) Also, HTTPS (HTTP running over TLS - Transport Layer Security)

JavaScript, Cascading Style Sheets (CSS), image/media types

Example: Web advertisementsSome web sites host their own ads

If the user clicks the ad

User


User GET /adx/bin/adx_click.html? ...nytimes.com


User GET /adx/bin/adx_click.html? ...

HTTP/1.1 302 Found Location: http://www.marcjacobs.com/?utm_source=nyt ...User

nytimes.com


marcjacobs.com

User GET /adx/bin/adx_click.html? ...

HTTP/1.1 302 Found Location: http://www.marcjacobs.com/?utm_source=nyt ...User

GET /?utm_source=nyt ...

Main web site tells the user’s browser to redirect to the advertiser, give it credit for the click.

nytimes.com

Typical 3rd party display adsSimilar operation to 1st party ads

If the user clicks the adUser GET /click;h=v8/3d41/4/a2/ ...

ad.doubleclick.net


HTTP/1.1 302 Moved Temporarily Location: http://at.atwola.com/adlink...

User GET /adlink...

ad.doubleclick.net

at.atwola.com


HTTP/1.1 302 Moved Temporarily Location: http://at.atwola.com/adlink...

User GET /adlink...

DoubleClick redirects through Atwola then to destination.

HTTP/1.1 302 Redirect Location: http://www.choosenissan.com/...

GET /...

Userchoosenissan.com

ad.doubleclick.net

at.atwola.com

Tracking Cookies & Mobile Geolocation

Here’s a paper example

Here’s a paper example

What happens if you scan this QRcode with your phone?

http://bqt.co/936647AE1AE6C73

http://bqt.co/936647AE1AE6C73

HTTP/1.1 200 OK Date: Sun, 02 Dec 2012 20:47:28 GMT Server: Apache/2.2.9 (Fedora) X-Powered-By: PHP/5.2.6 Set-Cookie: PHPSESSID=psm5oqtkff8p1gvnbf716dpqm0; path=/ Expires: Thu, 19 Nov 1981 08:52:00 GMT Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Pragma: no-cache Set-Cookie: _unbq=50bbbe60cb7419.09871084.50bbbe60cb7512.99318963; expires=Fri, 02-Dec-2022 20:47:28 GMT Content-Length: 976 Connection: close Content-Type: text/html;charset=UTF-8

QRCodeTracking function tagAction(result) { window.location="/936647AE1AE6C73/r"; //show("content"); document.body.innerHTML = res; }

var res = null; Waiting ... tracking('2957203_db7b71b43a3ad916d08ac54b0b2c86a8');

HTTP Headers

Body (HTML + JavaScript)




Cookies

Tracking Code




Cookies

Key/value pairs, stored in the browser

Transmitted back to the site that set them

Set-Cookie: _unbq=50bbbe60cb7419.09871084.50bbbe60cb7512.99318963;

expires=Fri, 02-Dec-2022 20:47:28 GMT

JavaScript behaviors

Arbitrary code, running inside your browser

tracking('2957203_db7b71b43a3ad916d08ac54b0b2c86a8');

Geolocation code

function tracking(vid) { if(vid== -1) { hide('user_action'); tagAction(); return; } g_vid = vid; if (navigator.geolocation){ setTimeout('errorCallback({code: 4});', 15000); //15sec for user decision navigator.geolocation.getCurrentPosition(function(pos){ if(send) return; send = true; var a = (pos.address) ? pos.address : {}; var c = (pos.coords) ? pos.coords : {}; var params = { vid: vid, postalCode: ((a.postalCode) ? a.postalCode : ''), countryCode: ((a.countryCode ) ? a.countryCode : ''), country: ((a.country ) ? a.country : ''), region: ((a.region) ? a.region : ''), county: ((a.county) ? a.county : ''), city: ((a.city) ? a.city : ''), street: ((a.street) ? a.street : ''), streetNumber: ((a.streetNumber) ? a.streetNumber : ''), latitude: ((c.latitude) ? c.latitude : ''), longitude: ((c.longitude) ? c.longitude : ''), altitude: ((c.altitude) ? c.altitude : ''), accuracy: ((c.accuracy) ? c.accuracy : ''), altitudeAccuracy: ((c.altitudeAccuracy) ? c.altitudeAccuracy : ''), heading: ((c.heading) ? c.heading : ''), speed: ((c.speed) ? c.speed : '') };

Web server programming

Two main kinds of requests

HTTP “GET” requests Used to request static resources Ideally, should make no changes to the server (functional!)

HTTP “PUT” requests Used to respond to form requests Ideally, should be the only thing that changes state (mutation!)

So far, you’ve only written code for “GET” requests And your adventure game is returning a different value every time!

“Please don’t hit the back button”“Because we’re not doing functional programming!” You’ll do better with next week’s project

Web requests & SparkJavaIf the user types “2 5 +”, the URL looks like this. http://localhost:4567/rpnserver/?input=2%205%20%2B get("/rpnserver/", (request, response) -> {  logSparkRequest(TAG, request);   String commandLine = request.queryParams("input");    if (commandLine != null) {  response.status(200); // okay!  response.header("cache-control", "no-cache"); // because we're regenerating it every time  return jobject(jpair("response", rpnCalculator.calc(commandLine))).toString();  }   // if we got here, the command line we wanted was absent  Log.i(TAG, "empty command line");   response.status(400); // bad request  return jobject().toString(); // empty JSON object });


Key/value pairs, escaped  (%20 = space, %2B = +)


Extracts and unescapes the “input” request parameter

Patterns in the URL

Spark lets you encode, match requests in the URL path Let’s say you’re returning an image of a given size: get("/color-rectangle/imgsize/:width/:height/color/:rgb", (request, response) -> {  final int widthParam = stringToTryInteger(request.params().get(":width")).getOrElse(1);  final int heightParam = stringToTryInteger(request.params().get(":height")).getOrElse(1);   final int rgb = stringToTryInteger(request.params().get(":rgb"), 16).getOrElse(0); … });

This is structural pattern matching! You’re guaranteed there will be a width, height, and rgb in the URL or else Spark won’t call your lambda.

• No guarantee that they’re integers, though!

“Parameter” vs. “query parameters”scheme:[//[user:password@]host[:port]][/]path[?query][#fragment]

Query parameters: http://localhost:4567/rpnserver/?input=2%205%20%2B

• Arbitrary key/value strings in the “query” portion of the URL • Spark passes these along, doesn’t check for required elements

Parameters: http://localhost:4567/color-rectangle/imgsize/500/100/rgb/ff0000

• Specific arguments in the URL’s path • Spark will only call your lambda if the URL matches the pattern

Web responsesEach lambda takes two inputs (request,response) and returns String So what’s the purpose of the response object?

• Set cookies • Issue “redirect” commands • Label the response “type” (image/jpeg, text/html, etc.)

• MIME types: www.iana.org/assignments/media-types/media-types.xhtml Ultimately the lambda returns the “body” of the request

• We’re mostly returning JSON to respond to a JavaScript client in the browser • Spark has a huge “plugin” architecture to “transform” things (which you

basically don’t care about)

http://www.iana.org/assignments/media-types/media-types.xhtml

Returning vanilla HTML

Option 1: just stick a .html file in the “WebPublic” folder

Option 2: compute it yourself /**  * j2html helper used for making web page bodies that implement a command-line interface.  */ @NotNull @Contract(pure = true)  static ContainerTag muicssCommandLineBody(@NotNull String pageTitle) {  return body().with(  header()  .withId("header").with(  nav().withId("appbar").withClass("mui-container-fluid").with(h1(pageTitle))),  div().withId("textOutput"),   div().withId("goButton").with(  button(">").withClass("mui-btn mui-btn--fab mui-btn--primary")),  div().withId("footer").with(Utils.textEntryForm("commandLine", "type your commands here"))); }




j2html “fluent” HTML builder




j2html “fluent” HTML builder“helper” function that we wrote

CSS “styles” for the “MUI” library that we’re using




j2html “fluent” HTML builder“helper” function that we wrote

Why j2html?We want to have “correct” HTML no matter what Just like we want to have “correct” JSON no matter what Even if the strings we’re building from have weird codes in them

Solution? edu.rice.json.Builders (for JSON) and j2html (for HTML)

Why return JSON data?Make it easy to return many things (JSON is general purpose) get("/rpnserver/", (request, response) -> {  logSparkRequest(TAG, request);   String commandLine = request.queryParams("input");   if (commandLine != null) {  response.status(200); // okay!  response.header("cache-control", "no-cache"); // because we're regenerating every time  return jobject(jpair("response", rpnCalculator.calc(commandLine))).toString();  }   // if we got here, the command line we wanted was absent  Log.i(TAG, "empty command line");   response.status(400); // bad request  return jobject().toString(); // empty JSON object });


hey browser, don’t cache this (“because mutation!”)

HTTP response status codes


hey browser, don’t cache this (“because mutation!”)

Response status codes?2xx: Okay

3xx: Redirect / moved

4xx: Client error 404: Not found

• Yahoo! once bought this advertisement in AT&T Park (SF Giants)

5xx: Server error https://en.wikipedia.org/wiki/List_of_HTTP_status_codes

https://en.wikipedia.org/wiki/List_of_HTTP_status_codeshttps://en.wikipedia.org/wiki/List_of_HTTP_status_codes

Live demo: Chrome’s dev console

Further readingj2html (fluent HTML builder) http://j2html.com/

SparkJava web server http://sparkjava.com/documentation.html

Apache Commons Imaging https://commons.apache.org/proper/commons-imaging/apidocs/

RESTful web development (i.e., functional programming for the web) https://en.wikipedia.org/wiki/Representational_state_transfer
http://j2html.com/http://sparkjava.com/documentation.htmlhttps://commons.apache.org/proper/commons-imaging/apidocs/https://en.wikipedia.org/wiki/Representational_state_transfer

world wide web: introduction · 2016. 11. 7. · networking: simpliﬁed the internet routes ip...

Documents