System Security

Topics
• Hardening Network Components: router, switch, hub, wireless devices
security
• Intrusion detection IPS/IDS, Firewall, VPN
• Operating system security
• Malicious software and countermeasures
• Distributed Denial of service attacks (DDOS)
Hardening Network Components
Hardening Network Components

• Hardening network devices reduces the risk of unauthorized access into a

network’s infrastructure
• All networking devices, including routers and switches, come equipped
with services turned on when they are received from the manufacturer
• Enable
• SSHv3 or TLS: Both of these protocols are used to securely communicate to
remote network devices.
• Disable
• Echo Protocol, Chargen Protocol, Discard Protocol, Daytime Protocols, FTP
Protocol, Telnet, BootP Service, HTTP Server, SNMP Protocol, Discovery Protocols,
IP Source Routing, IP Unreachable (ICMP), IP Mask Reply, Zero Touch Provisioning

https://media.defense.gov/2020/Aug/18/2002479461/-1/-1/0/HARDENING_NETWORK_DEVICES.PDF
Hardening Network Components

• Interface and Switch Port Recommendations

• Enable port security.
• Shut down unused interfaces and switch ports.
• Place unused switch ports in a VLAN that is not routed and closely
monitored. Reassign the native VLAN.
• Disable
• Unused interfaces and routing protocols
• IP direct-broadcast
• IP proxy-arp
Hardening Network Components

• Secure Access Recommendations

• Use multi-factor authentication using hardware tokens and passwords.
• Use out-of-band management to separate network administration traffic from normal
user traffic.
• Implement the manufacturer’s configuration guidance to restrict access to the console
port.  Limit the number of simultaneous management connections.
• Enable the strongest password encryption supported by the equipment.
• Follow “Digital Identity Guidelines – Authentication and Lifecycle Management” (NIST
SP 800-63B2 ).
• Reduce the risk of exposing administrative interfaces to user traffic by applying IP
address access control lists.
• Restrict physical access to routers/switches and apply access control lists for remote
access.
• Monitor and log all attempts to access network devices.
Intrusion
detection
IPS/IDS
Intruders

• significant issue for networked systems is

hostile or unwanted access
• either via network or local
• can identify classes of intruders:
• masquerader
• misfeasor
• clandestine user
• varying levels of competence
Intruders

• range
• benign: explore, still costs resources
• serious: access/modify data, disrupt system
• led to the development of CERTs
• intruder techniques & behavior patterns constantly
shifting, have common features
Examples of Intrusion

remote root compromise

web server defacement
guessing / cracking passwords
copying viewing sensitive data / databases
running a packet sniffer
distributing pirated software
using an unsecured modem to access net
impersonating a user to reset password
using an unattended workstation
Hackers

• motivated by thrill of access and status

• hacking community a strong meritocracy
• status is determined by level of competence
• benign intruders might be tolerable
• do consume resources and may slow performance
• can’t know in advance whether benign or malign
• IDS / IPS / VPNs can help counter
• awareness led to establishment of CERTs
• collect / disseminate vulnerability info / responses
Hacker Behavior Example

1. select target using IP lookup tools

2. map network for accessible services
3. identify potentially vulnerable services
4. brute force (guess) passwords
5. install remote administration tool
6. wait for admin to log on and capture password
7. use password to access remainder of network
Criminal Enterprise

• organized groups of hackers now a threat

• corporation / government / loosely affiliated gangs
• typically young
• often Eastern European or Russian hackers
• often target credit cards on e-commerce server
• criminal hackers usually have specific targets
• once penetrated act quickly and get out
• IDS / IPS help but less effective
• sensitive data needs strong protection
Criminal Enterprise Behavior

1. act quickly and precisely to make their activities

harder to detect
2. exploit perimeter via vulnerable ports
3. use trojan horses (hidden software) to leave
back doors for re-entry
4. use sniffers to capture passwords
5. do not stick around until noticed
6. make few or no mistakes.
Insider Attacks

among most difficult to detect and prevent

employees have access & systems knowledge
may be motivated by revenge / entitlement
when employment terminated
taking customer data when move to competitor
IDS / IPS may help but also need:
least privilege, monitor logs, strong authentication,
termination process to block access & mirror data
Insider Behavior Example

1. create network accounts for themselves and their friends

2. access accounts and applications they wouldn't normally
use for their daily jobs
3. e-mail former and prospective employers
4. conduct furtive instant-messaging chats
5. visit web sites that cater to disgruntled employees, such
as f'dcompany.com
6. perform large downloads and file copying
7. access the network during off hours.
Intrusion Techniques

• aim to gain access and/or increase privileges on a system

• often use system / software vulnerabilities
• key goal often is to acquire passwords
• so then exercise access rights of owner
• basic attack methodology
• target acquisition and information gathering
• initial access
• privilege escalation
• covering tracks
 Password Guessing

one of the most common attacks

attacker knows a login (from email/web page etc)
then attempts to guess password for it
defaults, short passwords, common word searches
user info (variations on names, birthday, phone, common words/interests)
exhaustively searching all possible passwords
check by login or against stolen password file
success depends on password chosen by user
surveys show many users choose poorly
Password Capture

another attack involves password capture

watching over shoulder as password is entered
using a trojan horse program to collect
monitoring an insecure network login
• eg. telnet, FTP, web, email
extracting recorded info after successful login (web
history/cache, last number dialed etc)
using valid login/password can impersonate user
users need to be educated to use suitable
precautions/countermeasures
Intrusion Detection

• inevitably will have security failures

• so need also to detect intrusions so can
• block if detected quickly
• act as deterrent
• collect info to improve security
• assume intruder will behave differently to a
legitimate user
• but will have imperfect distinction between
Intrusion Detection
Approaches to Intrusion Detection

• statistical anomaly detection

• attempts to define normal/expected behavior
• threshold
• profile based
• rule-based detection
• attempts to define proper behavior
• anomaly
• penetration identification
Audit Records

• fundamental tool for intrusion detection

• native audit records
• part of all common multi-user O/S
• already present for use
• may not have info wanted in desired form
• detection-specific audit records
• created specifically to collect wanted info
• at cost of additional overhead on system
Statistical Anomaly Detection

• threshold detection
• count occurrences of specific event over time
• if exceed reasonable value assume intrusion
• alone is a crude & ineffective detector
• profile based
• characterize past behavior of users
• detect significant deviations from this
• profile usually multi-parameter
Audit Record Analysis
• foundation of statistical approaches
• analyze records to get metrics over time
• counter, gauge, interval timer, resource use
• use various tests on these to determine if current behavior is acceptable
• mean & standard deviation, multivariate, markov process, time series,
operational
• key advantage is no prior knowledge used
Rule-Based Intrusion Detection

• observe events on system & apply rules to decide

if activity is suspicious or not
• rule-based anomaly detection
• analyze historical audit records to identify
usage patterns & auto-generate rules for them
• then observe current behavior & match
against rules to see if conforms
• like statistical anomaly detection does not
require prior knowledge of security flaws
Rule-Based Intrusion
Detection
rule-based penetration identification
uses expert systems technology
with rules identifying known penetration,
weakness patterns, or suspicious behavior
compare audit records or states against rules
rules usually machine & O/S specific
rules are generated by experts who interview &
codify knowledge of security admins
quality depends on how well this is done
Base-Rate Fallacy

• practically an intrusion detection system needs to

detect a substantial percentage of intrusions with
few false alarms
• if too few intrusions detected -> false security
• if too many false alarms -> ignore / waste
time
• this is very hard to do
• existing systems seem not to have a good record
Distributed Intrusion Detection

• traditional focus is on single systems

• but typically have networked systems
• more effective defense has these working
together to detect intrusions
• issues
• dealing with varying audit record formats
• integrity & confidentiality of networked data
• centralized or decentralized architecture
Distributed Intrusion Detection -
Architecture
Distributed Intrusion Detection – Agent
Implementation
Honeypots

decoy systems to lure attackers

away from accessing critical systems
to collect information of their activities
to encourage attacker to stay on system so
administrator can respond
are filled with fabricated information
instrumented to collect detailed information on
attackers activities
single or multiple networked systems
cf IETF Intrusion Detection WG standards
Firewall
What is a Firewall?

• a choke point of control and monitoring

• interconnects networks with differing trust
• imposes restrictions on network services
• only authorized traffic is allowed
• auditing and controlling access
• can implement alarms for abnormal behavior
• provide NAT & usage monitoring
• implement VPNs using IPSec
• must be immune to penetration
What is a Firewall?
Firewall Limitations

• cannot protect from attacks bypassing it

• eg sneaker net, utility modems, trusted organisations,
trusted services (eg SSL/SSH)
• cannot protect against internal threats
• eg disgruntled or colluding employees
• cannot protect against access via WLAN
• if improperly secured against external use
• cannot protect against malware imported via laptop, storage
infected outside
Firewalls – Packet Filters

simplest, fastest firewall component

foundation of any firewall system
examine each IP packet (no context) and permit
or deny according to rules
hence restrict access to services (ports)
possible default policies
that not expressly permitted is prohibited
that not expressly prohibited is permitted
Firewalls – Packet Filters
Firewalls – Packet Filters
Attacks on Packet Filters

• IP address spoofing
• fake source address to be trusted
• add filters on router to block
• source routing attacks
• attacker sets a route other than default
• block source routed packets
• tiny fragment attacks
• split header info over several tiny packets
• either discard or reassemble before check
Firewalls – Stateful Packet Filters

• traditional packet filters do not examine higher layer context

• ie matching return packets with outgoing flow
• stateful packet filters address this need
• they examine each IP packet in context
• keep track of client-server sessions
• check each packet validly belongs to one
• hence are better able to detect bogus packets out of context
• may even inspect limited application data
Firewalls - Application Level
Gateway (or Proxy)
have application specific gateway / proxy
has full access to protocol
user requests service from proxy
proxy validates request as legal
then actions request and returns result to user
can log / audit traffic at application level
need separate proxies for each service
some services naturally support proxying
others are more problematic
Firewalls - Application Level Gateway (or Proxy)
Firewalls - Circuit Level Gateway
• relays two TCP connections
• imposes security by limiting which such connections are allowed
• once created usually relays traffic without examining contents
• typically used when trust internal users by allowing general outbound
connections
• SOCKS is commonly used
Firewalls - Circuit Level Gateway
 Bastion Host
highly secure host system
runs circuit / application level gateways
or provides externally accessible services
potentially exposed to "hostile" elements
hence is secured to withstand this
hardened O/S, essential services, extra auth
proxies small, secure, independent, non-privileged
may support 2 or more net connections
may be trusted to enforce policy of trusted
separation between these net connections
 Host-Based Firewalls

• s/w module used to secure individual host

• available in many operating systems
• or can be provided as an add-on package
• often used on servers
• advantages:
• can tailor filtering rules to host environment
• protection is provided independent of topology
• provides an additional layer of protection
Personal Firewalls

• controls traffic between PC/workstation and Internet or

enterprise network
• a software module on personal computer
• or in home/office DSL/cable/ISP router
• typically much less complex than other firewall types
• primary role to deny unauthorized remote access to the
computer
• and monitor outgoing activity for malware
Firewall Configurations
Firewall Configurations
Firewall Configurations
DMZ
Networks
Distributed
Firewalls
Operating System Security
Designing Trusted Operating Systems

• What makes an operating system “secure”? Or “trustworthy?

• How are trusted systems designed, and which of those design
principles carry over naturally to other program development tasks?
• How do we develop “assurance” of the correctness of a trusted
operating systems?
Designing Trusted Operating Systems
• Primitive security services
• Memory protection
• File protection
• General object access control
• User authentication
• OS is trusted if we have confidence that it provides these four services in a
consistent and effective way.
What is a trusted system?
Secure Trusted
Either-or: something Graded: There are
either is or is not secure degrees of
“trustworthiness
Property of presenter Property of receiver
Asserted based on Judged based on
product characteristics evidence and analysis
Absolute: not qualified as Relative: viewed in
to how, where, when, or context of use
by whom used
A goal A characteristic
What is a trusted system?

• Trusted process – process that can affect system security

• Trusted product – evaluated and approved product
• Trusted software- software portion of system that can be relied upon
to enforce security policy
• Trusted computing base – set of all protection mechanisms within a
computing system that enforce a unified security policy
• Trusted system – system that employs sufficient hardware and
software integrity measures to allow its use for processing sensitive
information
Trusted System Design Elements
• Least privilege
• Economy of mechanism
• Open design
• Complete mediation
• Permission based
• Separation of privilege
• Least common mechanism
• Ease of use
 Security Features of
Ordinary Operating Systems
• Authentication of users
• Protection of memory
• File and I/O device access control
• Allocation and access control to general objects
• Enforcement of sharing
• Guarantee of fair service
• Interprocess communications and synchronization
• Protection of operating system protection data
 Security Features of Trusted
Operating Systems
• Trusted systems incorporate technology to address both features and
assurance

• Objects are accompanied (surrounded) by an access control

mechanism

• Memory is separated by user, and data and program libraries have

controlled sharing and separation
Security Features of Trusted
Operating Systems

• Identification and Authentication

• Require secure id of individuals, each individual must
be uniquely identified
• Mandatory and Discretionary Access Control
• MAC – access control policy decisions are made
beyond the control of the individual owner of the
object
• DAC – leaves access control to the discretion of the
object’s owner
• MAC has precedence over DAC
 Security Features of Trusted
Operating Systems
• Object Reuse Protection
• Prevent object reuse leakage
• OS clears (overwrites) all space to be reassigned
• Problem of magnetic remanence
• Complete Mediation
• All accesses must be controled
• Trusted Path
• For critical operations (setting password, etc.), users want unmistakable
communications
 Security Features of Trusted
Operating Systems
• Accountability and Audit
• Maintain a log of security relevant events
• Audit log must be protected from outsiders
• Audit Log Reduction
• Audit only open and close of files/objects
• Intrusion detection
• Build patterns of normal system usage, triggering an alarm any time usage
seems abnormal
• Intrusion prevention
Kernelized Design
• Kernel – part of OS that performs lowest-level functions
• Synchronization, interprocess communications, message passing, interrupt
handling
• Security kernel – responsible for enforcing security mechanism for entire OS;
provides interface among the hardware, OS, and other parts of computer
system
 Malicious software
and countermeasures
Software Flaws and Malware

If automobiles had followed the same development cycle as the computer,

a Rolls-Royce would today cost $100, get a million miles per gallon,
and explode once a year, killing everyone inside.
 Robert X. Cringely

My software never has bugs. It just develops random features.

 Anonymous
 Software Issues

Alice and Bob Trudy

 Find bugs and flaws by accident
• Actively looks for bugs and flaws
 Hate bad software…
• Likes bad software…
 …but they learn to live with it
• …and tries to make it misbehave
 Must make bad software work
• Attacks systems via bad software
System Lines of Code (LOC)
 “Complexity is the enemy of security”, Netscape 17 million
Paul Kocher, Cryptography Research, Inc. Space Shuttle 10 million
Linux kernel 2.6.0 5 million
Windows XP 40 million
 Nowadays, a new car contains more LOC than
Mac OS X 10.4 86 million
was required to land the Apollo astronauts on
the moon Boeing 777 7 million
 Lines of Code and Bugs

• Conservative estimate: 5 bugs/10,000 LOC

• Do the math
• Typical computer: 3000 exe’s of 100,000 LOC each
• Conservative estimate: 50 bugs/exe
• Implies about 150,000 bugs per computer
• So, 30,000-node network has 4.5 billion bugs
• Maybe only 10% of bugs security-critical and only 10% of those
remotely exploitable
• Then “only” 45 million critical security flaws!
Software Security Topics

 Program flaws (unintentional)

o Buffer overflow
o Incomplete mediation
o Race conditions
 Malicious software (intentional)
o Viruses
o Worms
o Other breeds of malware
 Program Flaws

• An error is a programming mistake

• Made by human/programmer
• An error may lead to incorrect state: fault
• A fault is internal to the program
• A fault may lead to a failure, where a system departs from its expected behavior
• A failure is externally observable

error fault failure

 Program Flaws - Example

 This program has an error Example

 This error might cause a fault char array[10];
o Incorrect internal state for(i = 0; i < 10; ++i)
 If a fault occurs, it might lead to a failure array[i] = ‘A’;
o Program behaves incorrectly (external) array[10] = ‘B’;
 We use the term flaw for all of the above
 Secure Software

 Program Flaws
• In software engineering, try to ensure that a program
does what is intended o Program flaws are unintentional

• Secure software engineering requires that  But can still create security risks
software does what is intended……and o We’ll consider 3 types of flaws
nothing more 1. Buffer overflow (smashing the stack)
• Absolutely secure software? Dream on…  2. Incomplete mediation
• Absolute security anywhere is impossible 3. Race conditions
o These are the most common flaws
• How can we manage software risks?
1. Buffer Overflow
1. Buffer Overflow: Attack Scenario
• Users enter data into a Web form
• Web form is sent to server
• Server writes data to array called buffer, without checking length of input data
• Data “overflows” buffer
• Such overflow might enable an attack
• If so, attack could be carried out by anyone with Internet access
 Q: What happens when code is executed? Buffer Overflow
 A: Depending on what resides in memory int main(){
at location “buffer[20]” int buffer[10];
o Might overwrite user data or code buffer[20] = 37;
o Might overwrite system data or code }
o Or program could work just fine
 Simple Buffer Overflow
• Consider boolean flag for authentication
Boolean flag
• Buffer overflow could overwrite flag allowing
anyone to authenticate buffer
F O U R S C … TF

 In some cases, Trudy need not be so lucky as in the above example

 Memory Organization
o Text  code low 
address text
o Data  static variables
o Heap  dynamic data data
o Stack  “scratch paper” heap
 Dynamic local variables 
 Parameters to functions
 Return address ¬ stack
 pointer (SP)
high  stack
address
 Memory Layout of a C Program
low 
text #include <stdio.h>
address #include <stdlib.h>

Initialized data int x;

int y = 15;
uninitialized data int main(int argc, char* argv[])
{
heap int *value;
 int i;

value = (int*) malloc(sizeof(int)*5);

for(i=0; i < 5; i++)

value[i] = i;

stack   return 0;
}
pointer (SP) stack

argc, argv
high 
address
 Simplified Stack Example
low 

void func(int a, int b){ :

:
char buffer[10];
}
void main(){
func(1,2); ¬ SP
buffer
}
ret ¬¬SP
return
address
a ¬ SP

high  b ¬ SP
 Smashing the Stack
low 

 What happens if buffer overflows?

:
??? :

 Program “returns” to wrong location

buffer ¬ SP

 A crash is likely overflow

ret ¬ SP
ret… NOT!
overflow
a ¬ SP

high  b ¬ SP
Smashing the Stack
low 

 Trudy has a better idea… :

:
 Code injection
 Trudy can run code of her choosing…
o …on your machine
¬ SP
evil code

ret
ret ¬ SP
a ¬ SP

high  b ¬ SP
 Smashing the Stack
:
:

NOP
 Trudy may not know… :
1) Address of evil code NOP
2) Location of ret on stack
evil code
 Solutions
1) Precede evil code with NOP “landing ret
pad”
ret ¬ ret
2) Insert ret many times :

ret
:
:
Summary: Stack Smashing

• A buffer overflow must exist in the code

• Not all buffer overflows are exploitable
• Things must align properly
• If exploitable, attacker can inject code
• Trial and error is likely required
• Fear not, lots of help is available online
• Smashing the Stack for Fun and Profit, Aleph One
• Stack smashing is “attack of the decade”…
• …for many recent decades
• Also heap & integer overflows, format strings, etc.
 Example: Stack Smashing
• Suppose program asks for a serial number that Trudy does not know
• Also, Trudy does not have source code
• Trudy only has the executable (exe)

 Program quits on incorrect serial number

 By trial and error, Trudy discovers apparent buffer overflow

 Note that 0x41 is ASCII for “A” (6510=0100 00012=4116)

 Looks like ret overwritten by 2 bytes!
 Disassemble Code

• Next, disassemble bo.exe to find

 The goal is to exploit buffer overflow to jump to address 0x401034

 Buffer Overflow Attack
• Find that, in ASCII, 0x401034 is “@^P4”

 Byte order is reversed? What the …

 X86 processors are “little-endian”
 Reverse the byte order to “4^P@” and…

 Success! We’ve bypassed serial number check by exploiting a buffer overflow

 What just happened?
o Overwrote return address on the stack
Buffer Overflow

• Trudy did not require access to the source code

• Only tool used was a disassembler to determine address to jump to
• Find desired address by trial and error?
• Necessary if attacker does not have exe
• For example, a remote attack
 Buffer Overflow - Example

 Source code for buffer overflow example

 Flaw easily exploited by attacker……without access to source code!
#include<stdio.h>
#include<string.h>
void main()
{
char in[75];
printf("\nEnter Serial Number\n");
scanf("%s", in);
if(!strncmp(in,"S123N456", 8))
{
printf("Serial number is correct.\n");
}
}
 Stack Smashing Defenses

• Employ non-executable stack

• “No execute” NX bit (if available)
• Seems like the logical thing to do, but some real code executes on the stack
(Java, for example)
• Use a canary
• Address Space Layout Randomization (ASLR)
• Use safe languages (Java, C#)
• Use safer C functions
• For unsafe functions, safer versions exist
• For example, strncpy instead of strcpy
Stack Smashing Defenses
low 
:
:

 Canary
o Run-time stack check buffer
o Push canary onto stack overflow
canary ¬
o Canary value: overflow
ret
 Constant 0x000aff0d
a
 Or, may depends on ret
high  b
 Microsoft’s Canary

• Microsoft added buffer security check feature to C++ with /GS compiler flag
• Based on canary (or “security cookie”)
Q: What to do when canary dies?
A: Check for user-supplied “handler”
• Handler shown to be subject to attack
• Claimed that attacker can specify handler code
• If so, formerly “safe” buffer overflows become exploitable when /GS is used!
 Address Space Layout Randomization (ASLR)

 Address Space Layout Randomization (ASLR)

o Randomize place where code loaded in memory
o Makes most buffer overflow attacks probabilistic
 E.g. Windows Vista uses 256 random layouts
o So about 1/256 chance buffer overflow works
 Similar thing in Mac OS X and other OSs
 Attacks against Microsoft’s ASLR do exist
o Possible to “de-randomize”
 Summary: Buffer Overflow

• A major security threat yesterday, today, and tomorrow

• The good news?
• It is possible to reduce overflow attacks
• safe languages
• NX bit
• ASLR
• education
• etc.
• The bad news?
• Buffer overflows will exist for a long time
• Why?
• Legacy code,
• bad development practices,
• clever attacks,
2. Incomplete Mediation
 Input Validation

• Consider: strcpy(buffer, argv[1])

• A buffer overflow occurs if
len(buffer) < len(argv[1])
• Software must validate the input by checking the length of argv[1]
• Failure to do so is an example of a more general problem: incomplete mediation
Input Validation - Example

 Consider web form data

 Suppose input is validated on client. For example, the following is valid

http://www.things.com/orders/final&custID=112&num=55
&qty=20&price=10&shipping=5&total=205
 Suppose input is not checked on server. Why bother since input checked on client?
o Then attacker could send http message

http://www.things.com/orders/final&custID=112&num=55
&qty=20&price=10&shipping=5&total=25
Incomplete Mediation (ex. SQL Injection)

Ali

SELECT * from CUSTOMERS

WHERE name = ‘Ali’
Incomplete Mediation (ex. SQL Injection)

Ali' or '1'='1'

SELECT * from CUSTOMERS

WHERE name = ‘Ali’ or '1'='1'
 3. Race Conditions

 Security processes should be atomic

o Occur “all at once”
 Race conditions can arise when security-critical process occurs in stages
o The term race condition refers to a "race" between the attacker and the next stage of the process
 Attacker makes change between stages
o Often, between stage that gives authorization, but before stage that transfers ownership
 Example: Unix mkdir
o The outdated version of the Unix command mkdir, which creates a new directory
o Thus, the directory is created in stages—
1. there is a stage that determines authorization
2. followed by a stage that transfers ownership
 mkdir Race Condition
 mkdir Race Condition
 mkdir creates new directory
 How mkdir is supposed to work

mkdir
1. Allocate
space
2. Transfer ownership

mkdir
 mkdir Attack 1. Allocate
 The mkdir race condition space
3. Transfer ownership

2. Create link to
 Not really a “race” password file
o But attacker’s timing is critical
Race Conditions

• Race conditions are common

• Race conditions may be more prevalent than buffer overflows
• But race conditions harder to exploit
• Buffer overflow is “low hanging fruit” today

• To prevent race conditions, make security-critical processes atomic

• Occur all at once, not in stages
• Not always easy to accomplish in practice
 Malware

 Malicious Software (Malware) is not new…

 Fred Cohen’s initial virus work in 1980’s
 Cohen used viruses to break MLS systems
 Types of malware (no standard definition)
 Virus  passive propagation
 Worm  active propagation
 Trojan horse  unexpected functionality
 Trapdoor/backdoor  unauthorized access
 Rabbit  exhaust system resources
 Spyware  steals info, such as passwords
Where do Viruses Live?

o They live just about anywhere, such as…

o Boot sector
 Take control before anything else
o Memory resident
 Stays in memory
o Applications, macros, data, ……. .. etc.
o Library routines
o Compilers, debuggers, virus checker, ….. .. etc.
 Malware Detection

Three common detection methods

1. Signature detection
2. Change detection
3. Anomaly detection
 Signature Detection
• A signature may be a string of bits in exe; might also use wildcards, hash values, etc.
• For example, W32/Beast virus has signature
“83EB 0274 EB0E 740A 81EB 0301 0000”
• That is, this string of bits appears in virus
• We can search for this signature in all files, if string found, have we found W32/Beast?
• Not necessarily  string could be in normal code. But software is not random…

 Advantages
o Effective on “ordinary” malware
o Minimal burden for users/administrators
 Disadvantages
o Signature file can be large (10s of thousands)…making scanning slow
o Signature files must be kept up to date
o Cannot detect unknown viruses
o Cannot detect some advanced types of malware
 Change Detection
• Viruses must live somewhere
• If you detect a file has changed, it might have been infected
• How to detect changes?
• Hash files and (securely) store hash values
• Periodically re-compute hashes and compare
• If hash changes, file might be infected

 Advantages
o Virtually no false negatives
o Can even detect previously unknown malware
 Disadvantages
o Many files change  and often
o Many false alarms (false positives)
o Heavy burden on users/administrators
o If suspicious change detected, then what? Might fall back on signature detection
 Anomaly Detection
• Monitor system for anything “unusual” or “virus-like” or “potentially malicious”
• Examples of anomalous things
• Files change in some unexpected way,
• System misbehaves in some way
• Unexpected network activity
• Unexpected file access, etc., etc., etc., etc.
• But, we must first define “normal” and normal can (and must) change over time

 Advantages
o Chance of detecting unknown malware
 Disadvantages
o No proven track record
o Trudy can make abnormal look normal (go slow)
o Must be combined with another method (e.g., signature detection)
 Also popular in intrusion detection (IDS)
 Miscellaneous Software based Attacks

• Numerous attacks involve software

• We’ll discuss a few issues that do not fit into
previous categories
• Salami attack
• Linearization attack
• Time bomb
• Can you ever trust software?
 Salami Attack

• What is Salami attack?

• Programmer “slices off” small amounts of money
• Slices are hard for victim to detect
• Example
• Bank calculates interest on accounts. Programmer “slices off” any fraction of a cent and puts it in his own
account. No customer notices missing partial cent. Bank may not notice any problem. Over time, programmer
makes lots of money!
 Salami Attack - Examples

 Such attacks are possible for insiders

 Do salami attacks actually occur?
o Or is it just Office Space folklore?
 Programmer added a few cents to every employee payroll tax withholding
o But money credited to programmer’s tax
o Programmer got a big tax refund!
 Rent-a-car franchise in Florida inflated gas tank capacity to overcharge customers

 In LA, four men installed computer chip that overstated amount of gas pumped
o Customers complained when they had to pay for more gas than tank could hold
o Hard to detect since chip programmed to give correct amount when 5 or 10 gallons purchased
o Inspector usually asked for 5 or 10 gallons
 Linearization Attack

• Program checks for serial number S123N456

• For efficiency, check made one character at a time
• Can attacker take advantage of this?  #include <stdio.h>

int main(int argc, const char *argv[])

{
int i;
int serial[9] ="S123N456\n";

for(i=0; i < 8; i++){

if(argv[1][i] != serial[i]) break;
}
if(i == 8){
printf("\nSerial number is correct!\n\n");
}
return 0;
}
 Linearization Attack
• Correct number takes longer than incorrect
• Trudy tries all 1st characters; Find that S takes longest
• Then she guesses all 2nd characters: S; Finds S1 takes longest; and so on…
• Trudy can recover one character at a time!
• Same principle as used in lock picking

 A real-world linearization attack: TENEX (an ancient timeshare system)

o Passwords checked one character at a time
o Careful timing was not necessary, instead……could arrange for a “page
fault” when next unknown character guessed correctly
o Page fault register was user accessible
 Attack was very easy in practice
 Time Bomb
• In 1986 Donald Gene Burleson told employer to stop withholding taxes from his
paycheck
• His company refused
• He planned to sue his company
• He used company time to prepare legal docs
• Company found out and fired him
• Burleson had been working on malware…
• After being fired, his software “time bomb” deleted important company data

 Company was reluctant to pursue the case; So Burleson sued company for back
pay!
o Then company finally sued Burleson
 In 1988 Burleson fined $11,800
o Case took years to prosecute…Cost company thousands of dollars…
o Resulted in a slap on the wrist for attacker
 One of the first computer crime cases
 Many cases since follow a similar pattern
o Companies reluctant to prosecute
 Trusting Software
• Can you ever trust software?
• See Reflections on Trusting Trust
• Consider the following thought experiment
• Suppose C compiler has a virus
• When compiling login program, virus creates backdoor (account with known password)
• When recompiling the C compiler, virus incorporates itself into new C compiler
• Difficult to get rid of this virus!

 Suppose you notice something is wrong

o So you start over from scratch
o First, you recompile the C compiler
o Then you recompile the OS
 Including login program…
 You have not gotten rid of the problem!
 In the real world
o Attackers try to hide viruses in virus scanner
o Imagine damage that would be done by attack on virus signature updates