Network Programming and Applications
The Application Layer
Socket Programming
Application Protocols: DNS, FTP
Application Protocols: HTTP, FTP…
Remote Procedure Calls
RPC Applications: NFS, NIS
Web Applications: Search, Active Contents
Distributed Objects: CORBA
Domain Name System
Map between host names and IP addresses
– People: many identifiers: name, Passport #, …
– Internet hosts:
• IP address (32 bit) - used for addressing datagrams
• “name”, e.g., www.iitb.ernet.in - used by humans
Provides logical hierarchical view of the Internet
– globally distributed database implemented in hierarchy of many
name servers
Domain Name System
Functioning:
– application-layer protocol to communicate to resolve names
(address/name translation)
– client/server interaction
– clients: query servers to resolve names; nslookup
– servers: name server daemons, reply to queries; BIND, named
– gethostbyname: resolver library call that can be invoked from an
application program.
Lazily validated caches for performance
DNS Name Servers
Centralized DNS? no server has all name-to-IP
address mappings
single point of failure
traffic volume Local Name Servers:
distant centralized database – each organization/ISP has local
(default) name server
maintenance
– host DNS query first goes to
doesn’t scale!
local name server
Authoritative Name Server:
– for a host: stores that host’s IP
address, name
– can perform name/address
translation for that host’s name
DNS: Root Name Servers
Contacted by local name server that cannot resolve name
Root Name Server:
– contacts authoritative name server if name mapping not known
– gets mapping
– returns mapping to local name server
Several root name servers worldwide
DNS: Example root name server
host xyz.iitb.ernet.in wants IP address
of www.ibm.com 2 4
5 3
1. Contacts its local DNS server,
dns.iitb.ernet.in
2. dns.iitb.ernet.in contacts root name
server, if necessary
local name server authoritative name server
3. root name server contacts dns.iitb.ernet.in dns.ibm.com
authoritative name server,
dns.ibm.com, if necessary 1 6
xyz.iitb.ernet.in www.ibm.com
DNS: Name Resolution
Root name server:
– may not know authoritative name server
– may know intermediate name server to contact to find authoritative
name server
Recursive queries:
– puts burden of name resolution on contacted name server
– not scalable under heavy load
Iterated queries:
– contacted server replies with name of server to contact
DNS: Caching and Updating Records
A name server caches the mappings learnt
– cache entries have a time-to-live period after which they become
invalid
– update/notify mechanisms: RFC 2136
DNS Record:
– Resource Record (RR) format: (name, value, type, ttl)
– Type=A: name is hostname; value is IP address
– Type=NS: name is domain (e.g. ibm.com); value is IP address of
authoritative name server for this domain
– Type=CNAME: name is an alias name for some “cannonical” (the
real) name; value is cannonical name
– Type=MX: value is hostname of mailserver associated with name
DNS: Protocol and Messages
DNS protocol:
– client-server interaction
– query and reply messages, both
with same msg format
Message header
– identification: 16 bit # for
query, reply uses same #
– flags:
• query or reply
• recursion desired
• recursion available
• reply is authoritative
FTP: File Transfer Protocol
Transfer file to/from remote host [ftp: RFC 959]
client/server model
– client: side that initiates transfer (either to/from remote)
– server: remote host
– Uses TCP as transport protocol
Functioning:
– client contacts ftp server at port 21
– two separate, parallel TCP connections opened
– control: port 21; exchange commands, responses between
client, server.
– data: port 20; file data to/from server
– server maintains “state”: current directory, earlier authentication
FTP: commands, responses
Sample commands: Sample return codes
sent as ASCII text over control status code and phrase
channel 331 Username OK, password
required
OPEN: servername
125 data connection already
USER: username open; transfer starting
PASSWORD: password 425 Can’t open data connection
ls: return list of file in current 452 Error writing file
directory
get filename: retrieve file
put filename: store file onto
remote host
Network Programming and Applications
The Application Layer
Socket Programming
Application Protocols: DNS, FTP
Application Protocols: SMTP, HTTP…
Remote Procedure Calls
RPC Applications: NFS, NIS
Web Applications: Search, Active Contents
Distributed Objects: CORBA
SMTP: Electronic Mail
Three Components:
– user agents: “mail readers”
• composing, editing, reading mail messages
• e.g., Eudora, pine, elm, Netscape Messenger
• outgoing, incoming messages stored on server
– mail servers:
• mailbox contains incoming messages (yet to be read) for user
• message queue of outgoing (to be sent) mail messages
– smtp: simple mail transfer protocol
• protocol between mail servers
• client: sending mail server
• server: receiving mail sever
SMTP [RFC 821]
Uses TCP to transfer message from client to server, port 25
Three phases of transfer:
– handshaking: Connection establishment
– transfer: direct transfer from sending server (client) to receiving
server (server); push-based: client sends data instead of server
– closure: Connection termination
Command/Response interaction:
– commands: ASCI text
– response: status code and phrase
SMTP Interaction: Example
C: telnet mailServer.iitb.edu 25
S: 220 mailServer.iitb.edu
C: Helo myServer.edu
S: 250 Hello myServer.edu, pleased to meet you
C: MAIL FROM: <[email protected]>
S: 250 [email protected]... Sender ok
C: RCPT TO: <[email protected]>
S: 250 [email protected] ... Recipient ok
• C: DATA
S: 354 Enter mail, end with "." on a line by itself
• C: Hi abc,
C: This is uvw pretending to be xyz.
C: .
S: 250 Message accepted for delivery
• C: QUIT
S: 221 mailServer.iitb.edu closing connection
Mail Message Format
smtp: protocol for exchanging email messages
RFC 822: standard for text message format:
– header lines: - To: - From: - Subject:
• different from smtp commands!
– body: the “message”, ASCII characters only
– line containing only `.’
MIME: Multimedia Extension Types:
– Text: subtypes: plain, html
– Image: subtypes: jpeg, gif
– Audio: subtypes: basic (8-bit mu-law encoded), 32kadpcm (32 kbps coding)
– Video: example subtypes: mpeg, quicktime
– Application: other data that must be processed by reader before “viewable”;
example subtypes: msword, octet-stream
Message Format: Multimedia Extensions
MIME: multimedia mail extension, RFC 2045, 2056
additional lines in message header declare MIME content type
From: [email protected]
MIME version To: [email protected]
Subject: Picture.
method used MIME-Version: 1.0
to encode data Content-Transfer-Encoding: base64
Content-Type: image/jpeg
multimedia data
type, subtype, base64 encoded data .....
parameter declaration .........................
......base64 encoded data
encoded data .
Mail Access: Post Office Protocol
SMTP: delivery to receiver’s server S: +OK POP3 server ready
Mail access protocol: retrieval from C: user alice
server S: +OK
– POP: Post Office Protocol [RFC C: pass hungry
1939] S: +OK user successfully logged on
– IMAP: Internet Mail Access C: list
Protocol [RFC 1730] S: 1 498
S: 2 912
POP3: authorization phase S: .
C: retr 1
– client commands: user; password
S: <message 1 contents>
– server responses: +OK; -ERR S: .
C: dele 1
POP3: transaction phase, C: retr 2
– Client commands: list, retrieve, S: <message 2 contents>
delete, quit S: .
C: dele 2
C: quit
S: +OK POP3 server signing off
HyperText Transfer Protocol (HTTP)
World Wide Web’s application layer protocol
Client/Server model:
– client: browser that requests, receives, “displays” WWW objects
– server: WWW server daemon, sends objects in response to client requests
Functionality:
– client initiates TCP connection (creates socket) to server, port 80
– server accepts TCP connection from client
– http messages (application-layer protocol messages) exchanged between
browser (http client) and WWW server (http server)
– TCP connection closed
http is “stateless”: server maintains no information about past client requests
– http1.0: RFC 1945
– http1.1: RFC 2068
HTTP: Example
User enters URL www.iitb.edu/someDept/index.html (contains refs to 10 jpeg images)
1a. http client initiates TCP connection
to http server at www.iitb.edu; Port
80. 1b. http server at host www.iitb.edu
waiting for TCP connection at port
80. “accepts” connection,
notifying client
2. http client sends http request
message (containing URL) into
TCP connection socket 3. http server receives request message,
forms response message containing
requested object
(someDept/index.html), sends
message into socket
time
HTTP Example (contd)
4. http server closes TCP connection.
5. http client receives response
message containing html file,
displays html. Parsing html file,
finds 10 referenced jpeg objects
time 6. Steps 1-5 repeated for each of 10
jpeg objects
non-persistent connection: one object in each TCP connection
– some browsers create multiple TCP connections simultaneously -
one per object
persistent connection: multiple objects transferred within one TCP
connection
HTTP: Trying it yourself
$ telnet www.cse.iitb.ernet.in 80
Stateless client-server Trying 144.16.111.14...
interaction Connected to surya.cse.iitb.ernet.in.
Escape character is '^]'.
One request per GET / HTTP/1.0
connection
HTTP/1.1 200 OK
Date: Fri, 11 Jun 1999 06:26:16 GMT
Server: Apache/1.3.0 (Unix) PHP/3.0.4
Last-Modified: Tue, 04 May 1999 11:47:42 GMT
ETag: "2f70a-1645-372ede5e"
Accept-Ranges: bytes
Content-Length: 5701
Connection: close
Content-Type: text/html
X-Pad: avoid browser bug
<HTML>
<HEAD><TITLE>IIT Bombay
CSE Department Home Page</TITLE></HEAD>
<BODY> … </BODY> </HTML>
Connection closed by foreign host.
HTTP: Requests and Headers
Request types:
– GET: fetches the specified resource
– HEAD: requests meta-information about specified resource
– PUT: places a client document on the server
– POST: sends user-specified form data to a server script and returns results
– DELETE: erases a document from the server
Request headers:
– User-agent: Mozilla (Netscape)
– Pragma: no-cache
– If-modified-since: conditional-get
– Content-type: E.g. application/x-www-form-urlencoded
– Content-length: number of bytes in POST
– Authorization: user name and password
– Accept, Accept-encoding, Accept-language
HTTP Message Format: request
two types of http messages: request, response
http request message:
– ASCII (human-readable format)
request line
(GET, POST, GET /somedir/page.html HTTP/1.1
HEAD commands) Connection: close
User-agent: Mozilla/4.0
header Accept: text/html, image/gif,image/jpeg
lines Accept-language:en
Carriage return,
(extra carriage return, line feed)
line feed
indicates end
of message
HTTP Message Format: reply
status line
(protocol
status code HTTP/1.1 200 OK
status phrase) Connection: close
Date: Thu, 06 Aug 1998 12:00:15 GMT
Server: Apache/1.3.0 (Unix)
header
Last-Modified: Mon, 22 Jun 1998 …...
lines
Content-Length: 6821
Content-Type: text/html
data data data data data ...
data, e.g.,
requested
html file
HTTP Reply Status Codes
First line in server->client response message.
Success codes (200-299)
– 200=OK; request succeeded, requested object later in this message
Redirection (300-399)
– 301=Moved Permanently; new location later in this message
– 302=Found, 304=Not modified
Client errors (400-499)
– 400=Bad Request; request message not understood by server
– 401=Unauthorized, 403=Forbidden, 404=Not Found
Server errors (500-599)
– 500=Internal Error, 503=Server Overloaded
– 505 HTTP Version Not Supported
HTTP Response Headers
Header name Meaning
Date The current date
Last-Modified The last time the requested resource was
modified
Expires The time at which the requested resource expires
URI/Location The new location of a redirected resource
(301/302)
MIME-Version Multipurpose Internet Mail Extensions version
number
Content-Type The MIME type of the returned resource, used by
browser to render resource
Content-Length Resource size in bytes; if unspecified, client
reads until server disconnects
HTTP Interaction: conditional GET
Goal: don’t send object if client has client server
up-to-date stored (cached) version
client: specify date of cached copy http request msg
in http request
If-modified-since: object
If-modified-since: <date>
<date>
not
server: response contains no object
http response modified
HTTP/1.0
if cached copy up-to-date: 304 Not Modified
HTTP/1.0 304 Not Modified
http request msg
object
If-modified-since:
<date>
modified
http response
HTTP/1.1 200 OK
…
<data>
Web Server Design
All but the smallest sites use a database
Reliability of content storage, versions, backups, and
updates
Providing dynamic views of content, e.g., products by
price, vendor, store location
Perform transactions on-line, e.g. air ticket purchase, fund
transfers
Present electronic ‘store’ customized by each visitor’s
behavior
Web Server Interaction: authentication
Authentication: control access to client server
server documents
usual http request msg
stateless: client must present
authorization in each request 401: authorization req.
WWW authenticate:
authorization: typically name,
password
– authorization: header line in usual http request msg
+ Authorization:line
request
– if no authorization presented, usual http response msg
server refuses access, sends
401 WWW authenticate: usual http request msg
header line in response + Authorization:line
usual http response msg time
Web Server Interaction: cookies
server sends “cookie” (text client server
string) to client in response usual http request msg
Set-cookie: #
usual http response +
client presents cookie in later
Set-cookie: #
requests
cookie: #
server matches presented- usual http request msg
cookie: # cookie-
cookie with server-stored
cookies spectific
usual http response msg action
– authentication
– remembering user
preferences, previous usual http request msg
cookie-
choices cookie: #
spectific
usual http response msg action
Web: Active Contents
Embedding calls to server-side programs
– FORM, CGI, Servlet
Embedding client-side programs
– SCRIPT, PARAM, APPLET
Active Contents: Forms and CGI
User fills out HTML form
Server
Browser passes info to Common Gateway
Interface script Form Subprocess
info runs script
Script processes info and sends answer to
STDOUT HTTP Program
Server relays output back to browser server output
GET
– URL of CGI file is extended with arguments Form Program
– http://amazon.com/search.cgi?author=gates info output
POST
– Server relays browser form information to the
standard input (STDIN=‘keyboard’) of process Web User fills
running script
browser in info
– Script process reads a series of NAME=VALUE
strings on STDIN
Client
GET: Example (trial.cgi)
Program that
runs the script Disable character Specify
buffering in response
#!/usr/bin/perl MIME type
Web server
$|=1;
print "Content-type: text/html\r\n\r\n";
print "<html><body><table>\n";
for (sort(keys %ENV)) {
print "<tr> <td> $_ </td> <td> $ENV{$_} </td> </tr> \n";
}
print "</table></body></html>\n";
For each NAME
available in the …print the …and the
environment... NAME… VALUE
Active Contents: Java Applets
The Java Language:
– object oriented, strongly typed (Similar to C++, stricter rules)
– interpreted by a virtual machine, portable
– memory management by virtual machine
• no malloc, no free/delete, no pointers
– dynamic loading of classes
Applets:
– particular Java class and derived subclasses
– no main(), restricted API and behavior
– compiled binary (‘bytecode’) transmitted from server to browser client
– browser client includes a Java virtual machine (JVM)
– JVM loads and interprets bytecode
Applet API:
– init: initializations, class construction from params
– getParameter: read parameters from HTML PARAM tags
– start, paint: start the applet, refresh the applet panel
– stop, destroy: suspension and termination
Applet Example
hello.html
<html><body bgcolor=“blue”>
<applet code=hello.class
width=100 height=100>
<param name=“text” value=“Mumbai”>
</applet></body></html>
hello.java
import java.applet.*;
import java.awt.*;
public class hello extends Applet {
public void paint(Graphics g) {
g.drawString(getParameter(“text”), 20, 20);
}
}
Web: Caching
User sets browser to access WWW via web cache (proxy server)
Client sends all http requests to web cache
– if object at web cache, web cache immediately returns object in http
response
– else requests object from origin server, then returns http response to
client
Satisfy client request without involving origin server
– smaller response time: cache “closer” to client
– decrease traffic to distant servers
• link out of institution/local ISP network often bottleneck
