“MODERNISING VIDEOS USING AI”
ABV-IIITM GWALIOR
B.TECH MINI
PROJECT
Presented By:
● Piyush Raikwar (2017IMT-062)
● Harshit Patel (2017IMT-039)
● Aryanshu Verma (2017IMT-019)
● Dhananjai Kumar (2017IMT-033)
Under the Supervision of :
Dr. Somesh Kumar
2
Outline
❑ Introduction
❑ Objectives
❑ Components Used / Type of Coding
❑ Methodology/Circuit Diagram/Code
❑ Demonstration
❑ Importance for Society
❑ Novelty and Expected Outcomes of the Idea
❑ References
3
Introduction
● Ever wanted to bring your good old days of watching classic black and white shows like
‘Charlie Chaplin’ and not so old colored shows like ‘Malgudi Days’ but which are not
upto the mark according to today's display technology.
● Nowadays, people are accustomed of watching movies, TV shows and other video
content in high definition. As a result of which they don’t enjoy watching old movies and
TV shows in poor quality, in black and white format, inconsistent playback etc. All these
problems have reduced the interest of modern life viewers.
● What if we told you that you could bring those days back and re-experience those shows
in COLOURED FORM, HIGHER RESOLUTION and FREE FROM ALL THE NOISE AND
FLICKERING? All these things can be done with the help of Artificial Intelligence and that
too very fast without any human intervention. The idea behind this project is to develop
a fully automatic approach for restoration and enhancement of such videos.
4
Objectives
We'll provide with an easy to use and interactive web app for video-to-video synthesis. It
will allow you to either apply any one or two or all of the following filters:
● Video Colorization: Bring new life to old black and white videos by automatically colorizing
them. Given a grayscale video as input, this filter attacks the problem of hallucinating a
plausible color version of the video.
● Super Resolution: Super resolution is the process of upscaling and or improving the details
within an image i.e, recovering a High Resolution (HR) image from a given Low Resolution (LR)
image.
● Denoising : Image or Video denoising refers to the recovery of a digital image that has been
contaminated by noise. The presence of noise in videos is unavoidable. It may be introduced
during video formation, recording or transmission phase.
5
Components Used
6
Methodology
➢ Video Colorization:
● Given a grayscale photograph(frames of a video) as input, we implemented the solution to the
problem of hallucinating a plausible color version of the photograph.
● Hallucinating the colors seems a pretty hard task at first, since so much of the information (two out
of the three dimensions) has been lost.
● However, in many cases, the semantics of the scene and its surface texture provide ample cues
for many regions in each image: the grass is typically green, the sky is typically blue, etc. Of
course, these kinds of semantic priors do not work for everything, e.g., the ball on the grass might
not, in reality, be red, yellow, and purple.
● Our approach is not necessarily to recover the actual ground truth color, but rather to produce a
plausible colorization that could potentially fool a human observer.
7
Methodology
➢ Video Colorization:
● First, frames are extracted from video and then our model is applied over each frame.
● The system is implemented as a feed-forward pass in a Convolutional Neural Network at test time
and is trained on over a million color images.
● Given the lightness channel L, our system predicts the corresponding a and b color channels of
the image in the Lab colorspace.
8
Methodology
● Image Colorization Example:
* Notice how model learns that Coca-Cola logo is supposed to be Red.
9
Methodology
➢ Super Resolution:
● Super resolution is a task of estimating a high resolution (HR) image from its low resolution (LR)
counterpart. This estimated image is called as super resolved (SR) image.
● First, frames are extracted from video and then our model is applied over each frame.
● The system will try to replicate SRGAN, a generative adversarial network (GAN) for image
super-resolution (SR).
● Super-resolution GAN applies a deep network in combination with an adversary network to produce
higher resolution images.
● During the training, A high-resolution image (HR) is downsampled to a low-resolution image (LR). A GAN
generator upsamples LR images to super-resolution images (SR). We use a discriminator to distinguish
the HR images and backpropagate the GAN loss to train the discriminator and the generator.
● We have designed a framework that inputs a low resolution video, super resolves it by 4X.
10
11
SRGAN Architecture
12
Results
● Super Resolution Example:
13
Methodology
● Video Denoising Methodology:
● So our methodology includes working with an autoencoder basically autoencoders are Neural Networks
which are commonly used for feature selection and extraction.
● Denoising Autoencoders solve this problem by corrupting the data on purpose by randomly turning some of
the input values to zero. In general, the percentage of input nodes which are being set to zero is about 50%.
Other sources suggest a lower count, such as 30%. It depends on the amount of data and input nodes you
have.
14
Architecture
● An autoencoder is a neural network that learns to copy its input to its output. It has an internal (hidden)
layer that describes a code used to represent the input, and it is constituted by two main parts: an
encoder that maps the input into the code, and a decoder that maps the code to a reconstruction of the
original input.
● The encoder part of the network will be a typical convolutional pyramid. Each convolutional layer will be
followed by a max-pooling layer to reduce the dimensions of the layers.
Architecture
1. The Convolution Layer
The convolution step creates many small pieces called the feature maps or features like the green, red or
navy blue squares in Figure (E). These squares preserve the relationship between pixels in the input image. Let
each feature scan through the original image like what’s shown in Figure (F). This process in producing the
scores is called filtering.
16
Architecture
2. Decovolution layer
It explains what's going on with the decoder
17
Results
● Testing on MNIST Data
To test the model, the first step was to test it on a domain specific dataset. We used the MNIST dataset
which comprised of handwritten grayscale images of numerals.
18
Results
● Testing on images
I'm adding noise to the test images and passing them through the autoencoder. It does a surprisingly
great job of removing the noise, even though it's sometimes difficult to tell what the original image was
19
Importance for Society & Policy-making
● Conserving old videos.
● Medical Imaging (Super-Resolution and Colorization)
○ Increase resolution.
○ Auto colorization according to labels e.g, tissue type.
● Security & Surveillance (Super-Resolution)
○ Increase clarity of CCTV footage.
● Compression (Super-Resolution and Video Frame Interpolation)
○ Video streaming could benefit greatly from this technique, as it could allow for
streaming of lower frame rate video that could then be interpolated at the end node
to generate more frames and produce smoother high-frame rate video.
○ Reduce network bandwidth required by sending low resolution videos and converting
them to higher resolution at the end node.
20
Novelty and Expected Outcomes
● All three topics involving Video-to-Video synthesis i.e, Image Colorization, Super Resolution
and Video Frame Interpolation using Deep Learning is still very new and is under research.
● There are no commercial apps, softwares or platform available where all these three
techniques can be applied to ‘Modernize your Old’ videos.
● We expect that this project will not be only limited to enhancement of old videos, but can
also be used in various other domains like healthcare and surveillance.
● Most importantly these techniques can be used for efficient compression.
21
Demo of Web Application
Home Page of Webapp
22
Step 1
Click on Browse Button
to select video or
image from your disk
23
Step 2
Select video or image to be processed
24
Step 3
Press Green ‘HERE’ button
under UPLOAD VIDEO.
Video name will be displayed
under “Video Added”
section.Check if your video is
succesfully uploaded.
25
Step 4
Select Filter of your
choice,multi-filter can
be selected as well.
26
Step 5
Click on ‘Here’ under
‘Press To Convert’ to
apply filter.
Progress can be
monitored on the right
bar.
27
Final Step
Download the succesfully
processed video by pressing
the green download button
under the progress-bar
28
THANK YOU
29
