Memory Hierarchy

A memory unit is an essential component in any digital computer since it is needed for storing programs and
data.

Typically, a memory unit can be classified into two categories:

1. The memory unit that establishes direct communication with the CPU is called Main Memory. The
main memory is often referred to as RAM (Random Access Memory).

2. The memory units that provide backup storage are called Auxiliary Memory. For instance, magnetic
disks and magnetic tapes are the most commonly used auxiliary memories.

Apart from the basic classifications of a memory unit, the memory hierarchy consists all of the storage devices
available in a computer system ranging from the slow but high-capacity auxiliary memory to relatively faster
main memory.

The following image illustrates the components in a typical memory hierarchy.

Auxiliary Memory
Auxiliary memory is known as the lowest-cost, highest-capacity and slowest-access storage in a computer
system. Auxiliary memory provides storage for programs and data that are kept for long-term storage or when
not in immediate use. The most common examples of auxiliary memories are magnetic tapes and magnetic
disks.

A magnetic disk is a digital computer memory that uses a magnetization process to write, rewrite and access
data. For example, hard drives, zip disks, and floppy disks.

Magnetic tape is a storage medium that allows for data archiving, collection, and backup for different kinds of
data.

Main Memory
The main memory in a computer system is often referred to as Random Access Memory (RAM). This
memory unit communicates directly with the CPU and with auxiliary memory devices through an I/O
processor.

The programs that are not currently required in the main memory are transferred into auxiliary memory to
provide space for currently used programs and data.

I/O Processor
The primary function of an I/O Processor is to manage the data transfers between auxiliary memories and the
main memory.

Cache Memory
The data or contents of the main memory that are used frequently by CPU are stored in the cache memory so
that the processor can easily access that data in a shorter time. Whenever the CPU requires accessing memory,
it first checks the required data into the cache memory. If the data is found in the cache memory, it is read from
the fast memory. Otherwise, the CPU moves onto the main memory for the required data.

We will discuss each component of the memory hierarchy in more detail later in this chapter.
Main Memory
The main memory acts as the central storage unit in a computer system. It is a relatively large and
fast memory which is used to store programs and data during the run time operations.

The primary technology used for the main memory is based on semiconductor integrated circuits.
The integrated circuits for the main memory are classified into two major units.

1. RAM (Random Access Memory) integrated circuit chips

2. ROM (Read Only Memory) integrated circuit chips

RAM integrated circuit chips

The RAM integrated circuit chips are further classified into two possible operating
modes, static and dynamic.

The primary compositions of a static RAM are flip-flops that store the binary information. The nature
of the stored information is volatile, i.e. it remains valid as long as power is applied to the system.
The static RAM is easy to use and takes less time performing read and write operations as compared
to dynamic RAM.

The dynamic RAM exhibits the binary information in the form of electric charges that are applied to
capacitors. The capacitors are integrated inside the chip by MOS transistors. The dynamic RAM
consumes less power and provides large storage capacity in a single memory chip.

RAM chips are available in a variety of sizes and are used as per the system requirement. The
following block diagram demonstrates the chip interconnection in a 128 * 8 RAM chip.
 A 128 * 8 RAM chip has a memory capacity of 128 words of eight bits (one byte) per word.
This requires a 7-bit address and an 8-bit bidirectional data bus.

The 8-bit bidirectional data bus allows the transfer of data either from memory to CPU during
a read operation or from CPU to memory during a write operation.

The read and write inputs specify the memory operation, and the two chip select (CS) control
inputs are for enabling the chip only when the microprocessor selects it.

The bidirectional data bus is constructed using three-state buffers.

The output generated by three-state buffers can be placed in one of the three possible states
which include a signal equivalent to logic 1, a signal equal to logic 0, or a high-impedance
state.

Note: The logic 1 and 0 are standard digital signals whereas the high-impedance state behaves
like an open circuit, which means that the output does not carry a signal and has no logic
significance.

The following function table specifies the operations of a 128 * 8 RAM chip.
From the functional table, we can conclude that the unit is in operation only when CS1 = 1
and CS2 = 0. The bar on top of the second select variable indicates that this input is enabled when it
is equal to 0.

ROM integrated circuit

The primary component of the main memory is RAM integrated circuit chips, but a portion of
memory may be constructed with ROM chips.

A ROM memory is used for keeping programs and data that are permanently resident in the
computer.

Apart from the permanent storage of data, the ROM portion of main memory is needed for storing
an initial program called a bootstrap loader. The primary function of the bootstrap loader program
is to start the computer software operating when power is turned on.

ROM chips are also available in a variety of sizes and are also used as per the system requirement.
The following block diagram demonstrates the chip interconnection in a 512 * 8 ROM chip.

A ROM chip has a similar organization as a RAM chip. However, a ROM can only perform read
operation; the data bus can only operate in an output mode.

The 9-bit address lines in the ROM chip specify any one of the 512 bytes stored in it.
The value for chip select 1 and chip select 2 must be 1 and 0 for the unit to operate.
Otherwise, the data bus is said to be in a high-impedance state.
Auxiliary Memory
An Auxiliary memory is known as the lowest-cost, highest-capacity and slowest-access storage in a
computer system. It is where programs and data are kept for long-term storage or when not in
immediate use. The most common examples of auxiliary memories are magnetic tapes and magnetic
disks.

Magnetic Disks
A magnetic disk is a type of memory constructed using a circular plate of metal or plastic coated
with magnetized materials. Usually, both sides of the disks are used to carry out read/write
operations. However, several disks may be stacked on one spindle with read/write head available on
each surface.

The following image shows the structural representation for a magnetic disk.

The memory bits are stored in the magnetized surface in spots along the concentric circles
called tracks.

The concentric circles (tracks) are commonly divided into sections called sectors.

Magnetic Tape
Magnetic tape is a storage medium that allows data archiving, collection, and backup for different
kinds of data. The magnetic tape is constructed using a plastic strip coated with a magnetic
recording medium.

The bits are recorded as magnetic spots on the tape along several tracks. Usually, seven or nine bits
are recorded simultaneously to form a character together with a parity bit.

Magnetic tape units can be halted, started to move forward or in reverse, or can be rewound.
However, they cannot be started or stopped fast enough between individual characters. For this
reason, information is recorded in blocks referred to as records.
Associative Memory
An associative memory can be considered as a memory unit whose stored data can be identified for
access by the content of the data itself rather than by an address or memory location.

Associative memory is often referred to as Content Addressable Memory (CAM).

When a write operation is performed on associative memory, no address or memory location is given
to the word. The memory itself is capable of finding an empty unused location to store the word.

On the other hand, when the word is to be read from an associative memory, the content of the word,
or part of the word, is specified. The words which match the specified content are located by the
memory and are marked for reading.

The following diagram shows the block representation of an Associative memory.

From the block diagram, we can say that an associative memory consists of a memory array and logic
for 'm' words with 'n' bits per word.

The functional registers like the argument register A and key register K each have n bits, one for each
bit of a word. The match register M consists of m bits, one for each memory word.

The words which are kept in the memory are compared in parallel with the content of the argument
register.

The key register (K) provides a mask for choosing a particular field or key in the argument word. If the
key register contains a binary value of all 1's, then the entire argument is compared with each memory
word. Otherwise, only those bits in the argument that have 1's in their corresponding position of the
key register are compared. Thus, the key provides a mask for identifying a piece of information which
specifies how the reference to memory is made.

The following diagram can represent the relation between the memory array and the external
registers in an associative memory.
The cells present inside the memory array are marked by the letter C with two subscripts. The first
subscript gives the word number and the second specifies the bit position in the word. For instance,
the cell Cij is the cell for bit j in word i.

A bit Aj in the argument register is compared with all the bits in column j of the array provided that
Kj = 1. This process is done for all columns j = 1, 2, 3......, n.

If a match occurs between all the unmasked bits of the argument and the bits in word i, the
corresponding bit Mi in the match register is set to 1. If one or more unmasked bits of the argument
and the word do not match, Mi is cleared to 0.
Cache Memory
The data or contents of the main memory that are used frequently by CPU are stored in the cache
memory so that the processor can easily access that data in a shorter time. Whenever the CPU
needs to access memory, it first checks the cache memory. If the data is not found in cache
memory, then the CPU moves into the main memory.

Cache memory is placed between the CPU and the main memory. The block diagram for a cache
memory can be represented as:

The cache is the fastest component in the memory hierarchy and approaches the speed of CPU
components.

The basic operation of a cache memory is as follows:

When the CPU needs to access memory, the cache is examined. If the word is found in the
cache, it is read from the fast memory.

If the word addressed by the CPU is not found in the cache, the main memory is accessed to
read the word.

A block of words one just accessed is then transferred from main memory to cache memory.
The block size may vary from one word (the one just accessed) to about 16 words adjacent
to the one just accessed.
The performance of the cache memory is frequently measured in terms of a quantity
called hit ratio.

When the CPU refers to memory and finds the word in cache, it is said to produce a hit.

If the word is not found in the cache, it is in main memory and it counts as a miss.

The ratio of the number of hits divided by the total CPU references to memory (hits plus
misses) is the hit ratio.