© Copyright Microsoft Corporation. All rights reserved.

FOR USE ONLY AS PART OF MICROSOFT VIRTUAL TRAINING DAYS PROGRAM. THESE MATERIALS ARE NOT AUTHORIZED
FOR DISTRIBUTION, REPRODUCTION OR OTHER USE BY NON-MICROSOFT PARTIES.

Classified as Microsoft Confidential

Microsoft Azure Virtual Training
Day: AI Fundamentals
Fundamentals of
Artificial Intelligence
 • Fundamental AI concepts
• Fundamentals of machine learning
Learning Objectives
• Fundamentals of Azure AI services
Learning Objective:
Fundamental AI concepts
What is Artificial Intelligence?
Software that imitates human
capabilities
 Predicting outcomes and recognizing
patterns based on historic data.
 Recognizing abnormal events and
making decisions.
 Interpreting visual input.
 Understanding language and engaging
in conversations.
 Extracting information from sources to
gain knowledge.
Common AI workloads

Machine Learning Predictive models based on data and statistics – the foundation for AI.

Capabilities within AI to interpret the world visually through cameras,

Computer Vision video, and images.

Natural Language Capabilities within AI for a computer to interpret written or spoken

Processing language and respond appropriately.

Capabilities within AI that deal with managing, processing, and using

Document Intelligence high volumes of data found in forms and documents.

Capabilities within AI to extract information from large volumes of

Knowledge Mining often unstructured data to create a searchable knowledge store.

Capabilities within AI that create original content in a variety of

Generative AI formats including natural language, image, code, and more.
Principles of responsible AI
Challenge or Risk Example
A loan-approval model discriminates by gender due to bias
Fairness Bias can affect results. in the data with which it was trained.

An autonomous vehicle experiences a system failure and

Reliability & safety Errors may cause harm.
causes a collision.
A medical diagnostic bot is trained using sensitive patient
Privacy & security Private data could be exposed. data, which is stored insecurely.

A predictive app provides no audio output for visually

Inclusiveness Solutions may not work for everyone.
impaired users.
An AI-based financial tool makes investment
Transparency Users must trust a complex system.
recommendations – what are they based on?
An innocent person is convicted of a crime based on
Accountability Who’s liable for AI-driven decisions? evidence from facial recognition – who’s responsible?
Learning Objective:
Fundamentals of machine learning
What is machine learning?
 Creating predictive models by finding relationships in data
1. Training data 2. Algorithm 3. Model
(past observations) (Generalizes the relationship (encapsulates the function)
[x1,x2,x3], y between x and y as a function)
Training [x1,x2,x3], y
[x1,x2,x3], y
[x1,x2,x3], y f
y = (x)
[x1,x2,x3], y

Features Label
(x) (y)

Inferencing [x1,x2,x3] ŷ
4. Inferencing data Prediction
(unlabeled features) (inferred label)
Types of machine learning

5
Model training and evaluation
Multiple training iterations with different algorithms and parameters

1. Training data 2. Algorithm 3. Model

(past observations) (Generalizes the relationship (encapsulates the function)
[x1,x2,x3], y between x and y as a function)
[x1,x2,x3], y
[x1,x2,x3], y
[x1,x2,x3], y f
y = (x)
[x1,x2,x3], y
5. Evaluate model
[x1,x2,x3], y (compare predicted
[x1,x2,x3], y labels to actual labels)
[x1,x2,x3], y 4. Predictions
(from the validation data)
Validation data (with known labels)
Deep learning
Human neural network Artificial neural network

w
x

∫(f (x,w) )
• Neurons fire in response to • Each neuron is a function that operates on an
electrochemical stimuli input value (x) and a weight (w)
• When fired, the signal is passed to • The function is wrapped in an activation function
connected neurons that determines whether to pass the output on
Deep learning: neural network example - multiclass

f (x,w) f (x,w)
x1 f (x,w) 0

• x1: 37.3
f (x,w) f (x,w)
f (x,w) y1
• x2: 16.8
x2 f (x,w) • P(y=0 | x): 0.2 1
f (x,w) f (x,w) f (x,w) y2 • P(y=1 | x): 0.7
• x3: 19.2
x3 f (x,w) • P(y=2 | x): 0.1
• x4: 30.0 f (x,w) f (x,w) f (x,w) y3 2
x4 f (x,w)
f (x,w) f (x,w)
What is Azure Machine Learning?
• Azure Machine Learning is a
cloud-based platform for
machine learning.
• Azure Machine Learning Studio
is a user interface for accessing
Azure Machine Learning
capabilities.
• Machine learning models
trained with Azure Machine
Learning can be published as
services.
Demo • Explore Automated Machine Learning in
Azure Machine Learning Studio
Learning Objective:
Fundamentals of Azure AI services
Azure basics

Microsoft’s Azure cloud

platform provides scalable and
reliable:
 Data storage
Subscription
 Compute

 Services Resource Group

AI services in Microsoft Azure
 Azure Machine A platform for training, deploying, and managing machine learning
Learning models

A suite of services covering Vision, Speech, Language, Decision, and

 Azure AI services
Generative AI

Data extraction, enrichment, and indexing for intelligent search and

 Azure AI Search
knowledge mining
Azure AI services
AI application resources in an Azure
subscription:
 Standalone resources for specific services
 General Azure AI services resource for
multiple services

Consumed by applications via: https://eastus/...

• A REST endpoint (https://address)
• An authentication key or authorization token abc-1234-efg-5678…
Demo • Explore Azure AI services
Fundamentals of
Computer Vision
Learning Objectives
Learning Objective:
Computer vision concepts
Images and image processing

An image is an array of pixel values Filters are applied to change images

0 0 0 0 0 0 0 0
-1 0
-1 0
-1 0 0 0 0

0 0 0 0 0 0 0 0
-1 80 0
-1 0 0 0 0

25 25 25 25 25 25
0 0 0 0 0
-1 0
-1 -1 0 0
5 5 5 5 5 5

25 25 25 25 25
0 0 0 0 0 0 0 0 0
5 5 5 5 5

25 25 25 25 25 25
0 0 0 0 0 0 0 0
5 5 5 5 5 5

0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0
Laplace filter applied
 Convolutional Neural Networks

1. Labeled mages are used to train the

model ? ? ? 255

2. Filter layers extract feature maps from ?

?
?
?
?
?
255

each image 255

0
3. The feature maps are flattened
? ? ? 255

? ? ? 255 1
4. The feature values are fed into a fully ? ? ?
255 2

connected neural network ?

?
?
?
?
?
255

5. The output layer produces a probability

255
? ? ?
255

value for each possible class label

1 2 3 4 5
Multi-modal models
• Orange
Computer Vision tasks • Fruit
An apple • Healthy

Adaptive models Classification

Object
Captioning Tagging …
Detection

Foundation model Language Encoder Image Encoder

• A newer approach to modeling involves combining language and vision models that encode
image and text data
• The model encapsulates semantic relationships between features extracted from the images
and text extracted from related captions.
• A multi-modal model can be used as a foundation model for more specialized adaptive models.
Learning Objective:
Computer vision capabilities in Azure
Computer vision services in Azure

Vision Face

• Image Analysis: • Face detection

• Image tagging, captions, • Face recognition
model customization, and
more.
• Optical Character Recognition
(OCR)
• Spatial analysis
Image analysis 4.0 with the AI Vision Service

Capabilities include:
Caption: A group of
• Model customization people walking on a person

sidewalk
• Read text from images Tags: Building, jeans,
person
male (34)
street, outdoor,
• Detect people in images jacket, city, person
• Generate image captions jeans

• Detect objects footwear

• Tag visual features

• Smart crop
Detecting faces with the Face Service
Everyone can use the Face service to detect:
 Blur
 Exposure
 Glasses
 Head pose
 Noise
 Occlusion
Only Managed Microsoft customers can access
facial recognition capabilities:
 Similarity matching
 Identity verification
Reading text with Optical Character Recognition (OCR)

 Detect the location and characters

of printed and handwritten text

 Options for quick text extraction

from images, or asynchronous
analysis of larger scanned
documents Shopping List
Non-fat milk
Bread
Eggs
Demos
Fundamentals of
Natural Language
Processing
Learning Objectives
Learning Objective:
Natural language processing concepts
What is natural language processing (NLP)?

Text analysis
Tokenization Opinion mining
{
1: a
2: boat
Preprocessing Training
3: canal
4: dangled Machine
n: … Language model translation
Raw text
}

Summarization
Conversational AI
NLP and conversational AI in Azure

Language Speech Translator

• Language detection • Text to speech • Text translation

• Key phrase extraction • Speech to text • Document translation
• Named entity detection • Speech translation • Custom translation
• Sentiment analysis and opinion mining • Speaker identification • …
• Personal information detection • Language identification
• Summarization
• Question answering
• Conversational language understanding
Learning Objective: Natural language
processing capabilities in Azure
Analyzing text
I had a wonderful vacation
in France.

• Predominant Language: English

• Sentiment: .88 (positive)
• Key Phrases: “wonderful
vacation”
• Entities: France
Demo
Question answering
Define a knowledge base of question and
answer pairs:

 By entering questions and answers

?
 From an existing FAQ document !
 By using built-in chit-chat

Consume the knowledge base from client

apps, including bots.
Azure bot service

 Cloud-based platform for

developing and managing
bots
 Integration with AI Bot
Language and other services
 Connectivity through
multiple channels
 • Use Question Answering model with
Demo Language Studio
Conversational language understanding

Switch the light on.

Entity
Intent

Utterance
 • Use Conversational Language Understanding
Demo with Language Studio
Speech recognition and synthesis

Use the speech-to-text capabilities of the Speech

service to transcribe audible speech to text

Use the text-to-speech capabilities of the Speech

service to generate audible speech from text
Demo
Fundamentals of Document
Intelligence and Knowledge
Mining with Azure AI Search
Learning Objectives
Learning Objective: Fundamentals of Azure
AI Document Intelligence
What is Azure AI Document Intelligence?

Document analysis Prebuilt models Custom models

 Returns structured data  Trained to recognize and extract  Allows you to train models
representations. key-value pairs on forms such as with at least five sample
invoices, receipts, IDs documents.
Analyzing forms with Document Intelligence service

 Extract information from scanned

forms in image or PDF format.
 Use the pre-trained models for Detected Text:
common document types, such as
Northwind
invoices, receipts, IDs, etc. Traders
 Train a custom model using your 123 Main Street
own forms. 555-123-4567
 Models perform semantic ….
recognition of form fields – not just
text extraction.
Getting Started with Document Intelligence Studio

 Using a no code approach you can explore

the functionality using samples and your
own documents.

 First - create a resource

 Document Intelligence resource
 AI services resource
 Then Enable the resource in Document
Intelligence Studio.

 Getting Started page: select a model to

try.
 • Extract form data in Document Intelligence
Demo
Studio
Learning Objective:
Fundamentals of Knowledge Mining and
Azure AI Search
What is knowledge mining?
Organizations have a lot of content:
 Data is locked away in documents, PDFs, hand-written
notes, etc.
 Time consuming and labour intensive to find data.
 Knowledge mining finds insights – at scale.

Azure AI Search is Azure’s AI-powered knowledge mining

platform.
 Azure AI Search results contain only your data, and can
include new insights powered by AI
 Offered as Platform as a Service (PaaS) solution, meaning
Microsoft manages the infrastructure and availability
What is Azure AI Search?

Azure AI
Search

Data ingestion AI enrichment & indexing Exploration

 Azure Blob Storage containers  AI enables deeper understanding  Search performed on indexes
 Azure SQL Database  Extract information & patterns  Results used:
 Documents in Cosmos DB  Azure AI services  Within applications
 Azure Data Lake Storage Gen2  Vision, Natural Language Processing, etc.  Create data visualizations
 Azure Table Storage  Indexing makes content searchable
Elements of a search solution
Create a search index by moving data through the
following indexing pipeline:
1. Start with a data source of original documents
2. Automate data movement with an Indexer.
3. Document cracking: the indexer opens files and
extracts content.
4. Enrichment: the indexer moves data through AI
enrichment by adding skills in a skillset. AI enriched
content can be sent to a knowledge store
5. Push to index: the serialized JSON data populates
the search index.
6. The result is a populated search index which can be
explored through queries.
Demo • Explore an Azure AI Search index (UI)
Fundamentals of
Generative AI
  Fundamentals of generative AI
Learning Objectives  Introduction to Azure AI Studio
 Responsible generative AI in AI Studio
Learning Objective:
Fundamentals of generative AI
What is generative AI?

Language model

Natural language
Image generation Code generation
generation

Write a cover letter for a Create a logo for a florist Write Python code to add
job application business two numbers

Dear <recipient>, def add(n1, n2):

Please find enclosed my result = n1 +
application for the role of n2;
… return result;
 A high-level overview of language models

Transformer Model
Training Encode text
text tokens as vectors dog [10,3,2]
(embeddings) cat [10,3,1]
puppy [5,2,1]
Training
skateboard [-3, 3, 2]

Input Embedding vectors

When my dog was… encapsulate semantic
(prompt) attributes in multiple
Generate
Output dimensions
… a puppy
natural
(completion) language (related words have
sequences similar orientation)
Inferencing
Language models – tokenization
Step one: tokenization
The first step in training a transformer model is to decompose the training text into tokens.

Example sentence: I heard a dog bark loudly at a cat.

"I"=1 "heard"=2 "a"=3 "dog"=4 "bark"=5 "loudly"=6 "at"=7 "cat"=8

• The sentence is now represented with the tokens: [1 2 3 4 5 6 7 3 8].

• Note "a" is tokenized as 3 only once.
• Similarly, the sentence "I heard a cat" could be represented with the tokens [1 2 3 8].
Language models – embeddings
Step two: embeddings
The semantic relationship between tokens is encoded in vectors, known as embeddings.

Token Word Embedding

4 dog [10,3,2]

8 cat [10,3,1]

9 puppy [5,2,1]

10 skateboard [-3, 3, 2]
Language models – attention
Step three: attention
Capture the strength of relationships between
tokens using the attention technique.

Example:
• Goal: predict token after “dog.”
• Represent “I heard a dog” as vectors.
• Assign “heard” and “dog” more weight.
• Several possible tokens can come after dog.
• The most probable token is added to the
sequence, in this case “bark.”
Language models
Foundation models
• You can train your own language model from scratch, but..
• Most generative AI solutions use a foundation model (optionally fine-tuned with your own data)

Azure OpenAI Model Catalog

Azure AI Studio
Azure Machine Learning Studio
OpenAI models hosted in Azure:
• GPT models for natural language
• Azure OpenAI models
and code generation • Open-source models
• Embeddings models to vectorize from:
text tokens • Microsoft
• DALL-E models for image • OpenAI
generation • HuggingFace
• Whisper models for speech
• Mistral
recognition
• Meta
• Databricks
• others…
Large and small language models
Large Language Models (LLMs) Small Language Models (SLMs)
 Trained with large volumes of general text data  Trained with focused text data
 Many billions of parameters  Fewer parameters
 Comprehensive language generation capabilities in  Focused language generation capabilities in
multiple contexts specialized contexts
 Large size can impact performance and portability  Fast and portable
 Time-consuming (and expensive) to fine-tune with  Faster (and less expensive) to fine-tune with
your own training data your own training data
 Examples include:  Examples include:
 OpenAI GPT 4  Microsoft Phi 2
 Mistral 7B  Microsoft Orca 2
 Meta Llama 3  OpenAI GPT Neo
What are copilots?
Generative AI powered assistants integrated into
applications, often as chat interfaces
• Copilots provide contextual assistance with everyday
tasks; typically using a language model to interpret
and generate natural language
• Business users can use copilots to boost their
productivity and creativity with AI-generated content
and automation of tasks
• Developers can:
• Extend copilots by creating plug-ins that
integrate them into business processes and data
• Create custom copilots to build generative AI
capabilities into apps and services
Levels of copilot adoption
1. Unlock productivity across your business with Microsoft Copilot
• Use copilot to optimize the way you work in the tools you use every day
• Empower employees to be more creative
• Spend less time on mundane tasks and more on high impact activities

2. Extend Microsoft Copilot for your business with plug-ins

• Integrate business-specific tasks and processes into Microsoft 365
• Get actionable insights from your own data and systems in productivity apps
• Build 3rd-party plug-ins so your customers can use your services in Microsoft 365

3. Build custom copilots of your own

• Integrate generative AI into custom workloads and apps
• Create compelling customer experiences and commercial products
• Have full control over design and development
Microsoft Copilot Examples
Generative AI to support information communication

AI on the web – accessible to all: AI in Microsoft 365 – accessible with subscription:

Use Copilot to answer questions, Write, edit, and analyze documents in Word, create and enhance presentations in
create content, and search the web at PowerPoint, get support with communication on Outlook and Teams, and more.
https://copilot.microsoft.com
Microsoft Copilot Examples
Generative AI to support business processes

AI in Microsoft Dynamics 365:

For instance, use Copilot in Microsoft Dynamics 365 Sales to find relevant AI in Microsoft Fabric:
customer information, qualify leads, and prepare proposals. In another instance, Use Copilot in Microsoft Fabric to write the
use Copilot in Dynamics 365 Customer Service to quickly research and resolve code needed to analyze, manipulate, and
customer issues. visualize your data.
Microsoft Copilot Examples
Generative AI to support IT infrastructure, security, and software development

AI in Microsoft Azure: AI in Microsoft Copilot for Security: AI in GitHub Copilot:

Use Copilot in the Azure portal to Use Microsoft Copilot for Security as a Use GitHub Copilot to maximize developer
design, operate, optimize, and standalone security portal or embedded in productivity by analyzing and explaining
troubleshoot cloud infrastructure. security tools to assess and respond to code, adding code documentation,
security threats. generating new code based on natural
language prompts.
Considerations for copilot prompts

Example: System Message:

Summarize the key You are a helpful assistant…
considerations for adopting +
Conversation history5:
Copilot1 described in this User: Hello
document2 for a corporate Assistant: Hi. How can I help?…
Language
executive3. Format the summary +
as no more than six bullet Copilot Current prompt: Model
User: Summarize the key …
points with a professional tone4. (May be further optimized by Copilot)

1. Start with a specific goal for what you want the copilot to do
2. Provide a source to ground the response in a specific scope of information
3. Add context to maximize response appropriateness and relevance
4. Set clear expectations for the response
5. Iterate based on previous prompts and responses to refine the result
Extending and developing copilots
Choosing a copilot development tool

Copilot Studio Azure AI Studio

• Low-code development of copilots and plug-ins • Pro-code development with full catalog of models and
using Azure OpenAI models for generative AI fine-tuning capabilities
• Fully managed, hosted SaaS service • PaaS services with full control over cloud infrastructure
• Dialog and conversational orchestration • Prompt and model orchestration
• Built-in analytics with security and • Evaluations engine to test performance, reliability,
governance controls scalability, and responsible AI safety
• Deploy to common chat channels like web, apps, • Deploy as an endpoint in Azure for use in custom apps
social channels, and Teams and services
Demo
Learning Objective:
Introduction to Azure AI Studio
What is Azure AI Studio?
• A Platform as a Service
(PaaS) for developers
• Provides a collaborative
pro-code environment
with enterprise-grade
security for AI innovation
• Use to explore, build,
test, and deploy AI
solutions that are
reliable, scalable, and
adhere to responsible AI
safety
What solutions does Azure AI Studio bring together?

Azure OpenAI service: Azure Machine Azure AI services encompass

Microsoft's cloud solution Learning: Microsoft's Microsoft’s cloud solutions for
for deploying, cloud solution for creating cutting-edge, market-
customizing, and hosting training and deploying ready, and responsible
language models. models and managing applications with prebuilt and
machine learning customizable APIs and models.
operations (MLOps).
How does Azure AI Studio work?
AI hub resources allow you to collaborate with others as a team. AI projects are contained
within a hub and allow for AI customization and orchestration (process automation).

AI hub resource
AI projects
Deploy and test

Augment and build

Evaluate and manage

Use Azure AI Studio to:
 Create and manage AI projects: provides a centralized hub for all your AI projects
 Develop generative AI applications: create applications that can generate content or
build your own prompt flow
 Explore available AI models: Experiment with various AI models in Azure AI Studio's
model catalog
 Leverage Retrieval Augmented Generation (RAG): enhance the quality and relevance
of generated content.
 Monitor and evaluate AI models: ensure AI models meet performance metrics
 Integrate with Azure services: work seamlessly with other Azure services
 Build responsibly: ensure that your applications adhere to ethical standards and best
practices
What are prompt flows?

Prompt: input sent to a model, consisting of

the user input, system message, and any Initial prompt from user
examples.
Add conversation history

Flow: an executable instruction set that can Add relevant data from data source
implement AI logic such as data retrieval,
calling language models, executing code. Combined data and history for grounding
context with initial prompt

Prompt flow: a feature that can be used to

generate, customize, or run a flow. Copilot response
What capabilities does the model catalog support?
Azure AI Studio provides an extensive model catalog that allows you to discover, evaluate, fine-tune, and
deploy models provided by Microsoft and Non-Microsoft companies.

Use these models to build generative AI applications that have:

Natural language generation capabilities

Code generation capabilities

Image generation capabilities, and more.

Demo
Learning Objective:
Responsible generative AI in
AI Studio
Plan a responsible generative AI solution
Four stage process to develop and implement a plan for responsible AI are:

Identify Measure Mitigate Operate

• Potential harms • The presence of • The harms at • The solution

that are relevant these harms in multiple layers in responsibly by
to your planned the outputs your solution to defining and
solution. generated by minimize their following a
your solution. presence and deployment and
impact. operational
readiness plan.
Azure AI Content Safety
Feature Functionality
Scans for the risk of user input attacks on language
Prompt shields
models
Detects if text responses are grounded in a user's
Groundedness detection
source content

Protected material detection Scans for known copyrighted content

Define custom categories for any new or emerging

Custom categories
patterns
Demo