Azure Storage F5
In Azure Essential course, we briefly learned about various storage services,
replication methods, and finally about the hybrid cloud storage solution
StorSimple.
In this course, you will learn:
Performance tiers of Storage Accounts
Features of Premium Storage
Virtual Machine Storage
Azure Files storage
Azure Blob or Unstructured Storage
Storage Tools
To use any of the Azure Storage services like Blob storage, File storage, and Queue
storage, you will first create a storage account, and then you can transfer data
to/from a specific service in that storage account.
Once created, Azure Storage resources can be accessed by any language that can make
HTTP/HTTPS requests. Additionally, Azure Storage offers programming libraries for
several popular languages to simplify many aspects of working with Azure Storage.
We will learn about all of this in next few cards!
Before we get started with creating Azure Storage, let us understand various Azure
Storage Areas and Accounts.
Azure storage is broadly grouped into 3 categories:
Storage for Virtual Machines
Unstructured Data storage
Structured Data storage
Azure Storage Areas Contd...
Storage for Virtual Machines:
Disks - Persistent block storage for Azure VMs.
Files - Fully managed file share on the cloud.
Unstructured Data storage:
Blobs - Highly scalable, REST based Cloud Object Storage.
Data Lake Store - Hadoop Distributed File System (HDFS) as a Storage.
Structured Data storage:
Azure SQL DB - Fully managed Database-as-a-service built on SQL.
Tables - Key Value, high scale, auto-scaling NoSQL store.
DocumentDB - NoSQL document database service.
You will learn more about Disk and File storage in next couple of cards. Structured
and Unstructured data storages are covered in separate topics.
Virtual machines in Azure use disks as a place to store Operating system,
Applications, and Data.
VM's also can have one or more data disks
Standard Storage Account uses Hard Disk Drive(HDD) as VM disk.
Premium Storage Account uses Solid State Drive(SSD) as VM disk.
Temporary Disk is a Non Persistent storage and uses SSD storage.
All disks are stored as Virtual Hard Disk's (VHD), and the maximum capacity of the
VHD is limited to 1023 GB.
It is not recommended to store any data on Temporary Disk.
File storage offers shared storage using the standard SMB 3.0 protocol.
It can be accessed as a mounted drive or Map network Drive as typical SMB share in
Desktops.
On-premises applications can access file data in a share via the File storage API.
Common uses of File storage:
Applications that rely on file shares.
Files like Configuration files that need to be accessed from multiple VMs.
Diagnostic data like logs that can be written to a file share and processed later.
Tools and utilities used by multiple developers.
Storage Accounts
Types of General Purpose Storage Account
Standard storage - most widely used storage accounts that can be used for all types
of data (tables, queues, files, blobs and VM disks).
Premium storage - high-performance storage for page blobs, which are primarily used
for VHD files.
Performance tiers of Blob Storage Account
Hot access - for files that are accessed frequently. You pay a higher cost for
storage, but the cost of accessing the files is much lower. Example: File Share.
Cool access - to store large amounts of rarely accessed data for lower cost.
Example: Backup Data.
Storage Account Conversion
Standard storage accounts are backed by magnetic drives (HDD) and provide the
lowest cost per GB.
Premium storage accounts are backed by solid state drives (SSD) and offer
consistent low-latency performance.
Hence it is not possible to convert standard storage account to Premium Storage
account or vice versa.
Microsoft recommends using Premium Storage for all VMs.
Premium storage has high bandwidth with extremely low latency and it offers less
than 1ms read latency(cache). Also, premium storage disks for virtual machines
support up to 64 TB of storage, 80,000 IOPS per VM and 50,000 IOPS per disk.
To improve total IOPS throughput we recommend striping across multiple disks and
using SSD premium disks.
Premium Storage is only supported on Azure GS and DS series of virtual machines.
Premium Storage supports only Locally Redundant Storage (LRS) Replication.
In Premium (SSD), the size of the VM disk is restricted to 128, 512, and 1023 GB.
Azure Storage tools, make the life of a storage administrator much easier. Here are
few of the most commonly used tools.
Azure Portal and Azure PowerShell
Azure Storage Explorer - a useful GUI tool to inspect and alter data in Azure
Storage. It can be used to upload, download, and manage blobs, files, queues, and
tables from any platform, anywhere.
AZ Copy - a command-line utility to copy blob, and file data within a storage
account or across accounts.
Azure Import/Export service - to import or export large amounts of blob data to or
from a storage account.
In this course we will learn about Azure Storage Explorer.
Securing Storage
Azure Storage provides a comprehensive set of security capabilities which together
enable developers to build secure applications.
Data Security:
Data in transit can be secured using client-side encryption, HTTPS or SMB 3.0
protocol.
Data at rest can be secured using Storage Service Encryption.
OS and Data disks used by VMs can be encrypted using Azure Disk Encryption.
Security Management:
Storage Access Policy: define policies to grant and revoke access at a granular
level, with a time limit.
Role-Based access control: use default and custom defined roles to control access
to the storage accounts
Audit and monitor authorization: using request information available in storage
analytics logs.
Storage Account Access keys and Shared Access Signatures (SAS) can be used to
secure data access.
Azure Blob storage is a service that stores Unstructured data in the cloud as
blobs/objects. All blobs must be in a container and a container can store an
unlimited number of blobs.
Common uses of Blob storage:
Serving images or documents directly to a browser.
Storing files for distributed access.
Streaming videos and audios.
Storing data for backup & restore, disaster recovery, and archiving.
Storing data for analysis by an on-premises or Azure-hosted service.
Blob storage is also referred to as Object storage.
Azure Storage offers three types of blobs: Block blobs, Page blobs, and Append
blobs.
Block blobs
Suitable for Sequential Read\Write operations.
Ideal for storing text or binary files, such as documents and media files.
Append blobs
Optimized for Append operations.
Can be used for logging scenarios.
Page blobs
Optimized for Random read/write operations.
Can be used for storing VHD files of Azure VM as OS and Data disks.
Managing Blob Storage
Blob storage typically requires transacting Huge amounts of data from On-Premises
to Azure and vice-versa.
Examples of Huge Data:
Large Virtual Hard Disks (VHDs) - Using Upload and Download Commands
TBs of Backup Data - Using Export and Import services
Generally uploading and downloading VHD files is done through Azure PowerShell or
Storage Explorer. Azure PowerShell provides a very efficient way for moving these
large files through following cmdlets:
Add-AzureRmVHD - Uploads a VHD from an on-premises virtual machine to a blob
storage in Azure.
Save-AzureRmVHD - Saves downloaded VHD images locally.
Import and Export is another service that is suitable for scenarios when several
TBs of data needs to be transferred. Transferring such volume of data over the
network is not feasible due to limited bandwidth and high network costs.
Example: Backup data to or from Azure.
Import Service - Securely transfers large amounts of data to Azure blob storage by
shipping hard disk drives to an Azure DC.
Export Service - Transfers data from Azure blob storage to hard disk drives and
ship to the on-premises site.
Usage Scenarios:
Migrating data to the cloud.
Content distribution to a different Datacenter.
Backup and Recovery data.
