January 2024

AI BEHIND THE SCREEN

A r tific ial Inte lligenc e in the Audiovisual Indus tr y:
b es t tools , top companies , main trends

J oa n Ro s é s

S P E C IA
L
E D IT IO
N
 A I B E H I N D T H E S C R E E N 2

Technology, It’s no secret that technology has been the great technology are some of the new technologies that
a transforming catalyst of the audiovisual industry in recent years. are already challenging the entire value chain of the
In successive waves of innovation, audiovisual tech- audiovisual industry, from creation, through production,
element in the
nology has shaped the current audiovisual landscape. business processes, and the dissemination and use
audiovisual
Advances in digitalization, telecommunications, internet of content and experiences.
industry technologies, screens, and personal devices such
as smartphones have mapped a much larger, global, Artificial Intelligence will have a very relevant impact
and diversified audiovisual sector. Today, we have on the business models of our sector and will affect
screens and content in the fields of health, education, the competitiveness of our companies that will find
culture, tourism, events and corporate training and new business opportunities. Even though there’s
communications, which add to the traditional sectors controversy regarding this technology, we firmly believe
of cinema, television, and advertising. A few years ago, in never stopping learning, testing, and implementing
the Audiovisual Cluster already identified technology it in our products and business processes. Regulation
as the element around which we needed to reinvent will come, and we’ll welcome it, but our companies can’t
the Catalan audiovisual industry and made it a core afford to limit themselves beforehand. Our recipe is
element of its strategies. simple: apply and learn. That’s why the Catalan Audio-
visual Cluster wants to support the institutions and
We are now entering a new technological revolution at companies of our industry on this new journey through
the hands of what we have called augmented audio- the transmission of knowledge and the promotion of
visual. Artificial intelligence, immersive technologies, strategic policies and investments. This report by Joan
Big Data, blockchain, and the deployment of 5G Rosés, gives us a quick starting guide to this new world.

_ Miquel Rutllant
Catalan Audiovisual Cluster Chairman
 A I B E H I N D T H E S C R E E N 3

AI Behind AI Behind the Screen reviews the main innovations in We highlight cases that offer new business perspectives
the Screen artificial intelligence applied to the audiovisual industry. or innovative ways of working, those using technology
not only as a tool, but exploring new paths.
In this report we identify over 150 companies, products
or relevant experiences arranged in twelve thematic We also incorporate a critical eye. We want to move
areas, along with examples of the transformation that away from the enthusiasm that characterises this type
this new technological vector is generating in the sector. of report. The impacts of technology are not always
positive. There are uncertainties and risks that need
It is not intended to be an exhaustive collection, but to be taken into consideration and that challenge the
rather an indicative sample of transforming trends, responsibility of its agents.
leading companies, research centres and exemplifying
cases. Since the beginning, the audiovisual industry has been
assimilating technologies that have facilitated the
We propose an international perspective that follows omnipresence it boasts today. This new technological
relevant experiences of research centres, large compa- leap has further increased the prevalence of screens.
nies and start-ups from all over the world.
Welcome to AI Behind the Screen!
You will find a still image, that of the beginning of 2024,
of a technology in permanent acceleration and change,
which is difficult to fully comprehend if we do not take a _ Joan Rosés
moment to try and understand its impact.
 A I B E H I N D T H E S C R E E N 4

CONTENTS
Artificial Intelligence _p.5

Generative artificial intelligence _p.6

Generative AI applications and capabilities _p.7

01 Text generation _p.8

02 Image generation _p.14

03 Music generation _p.19

04 Video generation _p.23

Generative AI. Risks, uncertainties and sustainability _p.28

Generative AI. Trends _p.33

05 Production, post-production, VFX _p.36

06 Synthetic hyperrealism _p.43

07 Audio and voice _p.50

08 Data everywhere _p.54

More _p.61
 A I B E H I N D T H E S C R E E N 5

Index

Artificial Artificial intelligence is permeating all sectors of economy and society. In the
audiovisual and cultural spheres, it affects the agents that make up the entire
Intelligence value chain. From creation to production, to distribution and public relations;
no area is beyond its reach.

We are talking about automatic generation of content, efficiency and quality in

the production processes, post-production and creation of visual effects, the
processing of images, voice and audio, the datification of content and users,
the analysis of large amounts of data and personalisation and recommendation
strategies.
 A I B E H I N D T H E S C R E E N 6

Index

Generative Generative artificial intelligence has become all the rage in technology. It affects all
areas of life; everyone is trying it, everyone is talking about it.
artificial From the processing of millions of data, AI is capable of generating texts, images and
intelligence newly created computer code. The versions that appeared in the first few months of
2022 have achieved a high level of verisimilitude, similar to what a person could do.

But it is also a transformative tool for artistic and cultural creation. The sectors where
generative AI has the greatest impact are those linked to art and culture: the various
existing programmes can generate fictional texts, scripts, dialogues, drawings, illustra-
tions, music, voices... And also videos, short at the moment. Production processes will
change, professional skills will need to be updated, creative possibilities will expand, but
the risks are no less great.

Society has received them with enthusiasm but also with concern. Groups of artists,
scriptwriters, media and image marketing companies have already made their first
judicial demands. Will they replace the work of creators or will they be an aid to human
creativity?

In the following pages we analyse representative cases and trends in the area with the
biggest impact on the audiovisual industry.
 A I B E H I N D T H E S C R E E N 7

Index

Generative AI applications and capabilities

TEXT VIDEO IMAGE CODE MUSIC SPEECH 3D

Marketing Video Art & design Code Music Text-to-speech 3D object

content generation generation composition modeling
Marketing Virtual
Emails Video editing illustrations Code Sound effect assistance Architectural
completion design visualization
Creative Video game Photo editing Voice cloning
writing development Refactoring & Arrangement & Animations for
Product design optimization orchestration Voice synthesis characters
Translation Video & prototyping
summarization Bud detection Remixing & Audiobook Industrial
Legal & Fashion & & fixing mashups production design
technical texts VR & AR apparel
Testing Virtual Personalized Gaming
News articles Data instruments voice interfaces environments
& summaries visualization Code
formatting

Source: Pixelplex
 A I B E H I N D T H E S C R E E N 8

Index

Artificial Intelligence

01 Text generation
Text generators, such as ChatGPT, the flagship generative AI appli- Based on these models, an infinite number of services are being im-
cation that revolutionised the technological landscape at the end of plemented that seek to refine text generation in specific areas and
2022, can be used as assistants to write all kinds of texts in any lan- specialised tasks: marketing, legal contracts, etc. Writesonic, Jasper,
guage: Snazzy, Writer, Copysmith, Headlime review and Ryrt are some of the
most outstanding.
emails, school and academic papers, search engine searches, ad-
vertising copy, posts, tweets, formulae for spreadsheets, program- There are some that specialise in grammatical correction or style
ming code... and even poems. improvement. It’s the case with Grammarly, Hemingway app, and
also Quillbot, which, in addition to perfecting the writing style,
Other generators have reached a similar level. The alliance between integrates all kinds of functionalities linked to text generation.
OpenAI and Microsoft, which immediately integrated ChatGPT into
the Bing search engine and other software, led to a reaction from
Google, which now has its own text generator Bard; Anthropic, which
launched Claude; and Meta with the LLama 2 model. China presented
its own, Ernie, and banned ChatGPT.
 Tex t generation A I B E H I N D T H E S C R E E N 9

Index

THE DATA

_ CAN AI WRITE (GOOD) SCRIPTS? By mid-February 2023 the Amazon Kindle store
was already offering more than 200 books in
Several automated solutions facilitate the writing of scripts. For exam-
which ChatGPT was listed as author or co-authorde
ple, NovelAI, Scriptshaper, Brain Pod AI and Scalenut are offered as
support tools for creation.

The (human) scriptwriter plans a story, defines the characters, writes _ WHERE CHATGPT IS GOOD AND WHERE IT FAILS
the synopsis or has it written in a text generation programme and,
The evolution of AI is exponential. What doesn't work well today may
once revised, incorporates it into a script creation programme that
have improved tomorrow. Thousands of tests and evaluations have
will make suggestions that will need to be revised in order to continue
been carried out on the version we know today. Those who have
progressing.
analysed the impact on creative writing agree that ChatGPT can be
Dramatron is a tool from DeepMind, Google's AI lab, trained to co- useful for:
create stories and scripts with humans. From a first line of text,
Suggesting ideas for setting an action or describing spaces.
Dramatron interactively generates character descriptions, plot
points, location descriptions and dialogues. Adapting the tone or nuances of a text according to a given con-
text.
Character.AI is a very popular artificial intelligence app: from a
Experimenting with the creation of characters and checking how
brief description, it generates interactive characters with different
they would react according to certain situations.
looks, voices, personalities and identities. It is not designed for
scriptwriters, but it is a potentially useful tool for suggesting or Suggesting names or characteristics of characters.
generating dialogues. Generating dialogues.

Can a machine write the whole script? One could try, but currently it Revising a story to check if it is coherent given certain parameters.
does not seem advisable. Generating alternative versions of a text.
 Tex t generation A I B E H I N D T H E S C R E E N 10

Index

Errors and limitations: Hallucinations: Chatbots often invent facts. Using a common euphe-
mism in technology, experts say that they suffer from "hallucinations".
Inaccurate or meaningless answers.
In reality they are blatant errors that misrepresent verifiable facts or
Very generic texts, with little empathy and emotional power. directly invent them.
Does not understand sarcasm, humour or subtlety.
Difficulty in creating coherent long texts. _ MASTERING THE LANGUAGE OF PROMPTS
Does not prioritise well between different options. To profit from text generation apps, you need to make the most of
Responses that are sometimes sketchy and discriminatory. their capabilities. The prompts or indications that are transferred
Grammatical constructions that are not always correct. to the machine so that it interprets what we want need to make the
following very clear.
Does not know how to generalise.
The context in which the action takes place.
Give as many details as possible of the places, era, relevant social
conflicts... that can guide the text generation.
Be concrete and specific.
Be clear about the genre, the format, the style.
Specify the tone in which each character speaks and their
personality.

DALL-E
 Tex t generation A I B E H I N D T H E S C R E E N 11

Index

_ THE PRECEDENT OF THE NORTH AMERICAN

SCREENWRITERS' STRIKE

Artificial intelligence was one of the reasons that prompted North The most ambiguous part of the agreement refers to whether the
American screenwriters to stage a strike last year that lasted 148 texts created by the script writers can be used to train AI models. The
days. union reserves the right to prevent this, but so far it does not do so.

The unions and studios agreed that this technology could be used as An OpenAI study on the "potential impact of large language models
long as certain limitations were respected. on the labour market" published in March 2023 says that multiple ca-
tegories of writers are almost 100% exposed to the impact of AI. This
Studios will not be able to use AI to write or rewrite literary material does not necessarily mean that AI will take their jobs (although in
provided by screenwriters. some cases it could), but it is almost certain that it will have some
AI generated texts will not be considered source material and, the- impact on the way they work.
refore, may not be used to limit the credits or rights of screenwri-
ters.

A screenwriter may use AI for writing if the company gives its con-
sent and as long as the writer follows company policies.

A company may not require a writer to use AI software.

Studios must inform screenwriters if any material delivered has

been generated by IA or incorporates IA-generated material.

AI protest WGA strike 2023

Source: Wikimedia Commons
 Tex t generation A I B E H I N D T H E S C R E E N 12

Index

_ IS IT POSSIBLE TO DETECT TEXTS CREATED BY AI?

Overall accuracy for each tool calculated as an average of all approaches discussed
There are initiatives aimed at creating detectors for ChatGPT and
similar automatisms. The reliability of these applications is still low. Check For AI

They can be useful for confirming suspicions but not for attributing Compilatio
Content at Scale
with certainty the authorship of a text to a machine. Crossplag
DetectGPT
Researchers from several European universities have published a Go Winston

study that analyses the results of the main specialised detection GPT Zero
GPT-2 Output Detector Demo
tools. The most accurate is Turnitin, which has a detection rate of
OpenAI Text Classifier
less than 80%. Other well-known ones such as GPTZero, DetectGPT, PlagiarismCheck
Compilatio, Writer and ZeroGPT obtain lower percentages. TurnItIn
Writeful GPT Detector
Writer
Zero GPT

0 10% 20% 30% 40% 50% 60% 70% 80%

“The danger of tools like ChatGPT is that Hines, Kristi. «Should You Trust An AI Detector?» Search Engine Journal, 18th July 2023

they convince us that intelligence is no more than the

accumulation of knowledge or that creativity comes
down to the most probable answer”

Daniel Innerarity, philosopher

 Tex t generation A I B E H I N D T H E S C R E E N 13

Index

For support tasks, AI is becoming increasingly common.

Audio transcription: Transkriptor, Otter.ai

_ USE OF AI IN JOURNALISM
Selection of statements: Avista (Reuters)
Artificial intelligence has made its way into newsrooms to help
Verification of data and facts: ClaimBuster, Google Fact Checker,
in the research and verification of information, to write texts, to
Fact Mata, ClaimDetection, ClaimHunter
facilitate the updating and personalisation of news, to convert
texts to video, to transcribe audio or video recordings... Analysis of large volumes of data: Tableau, Florecer, Power BI

The media have begun to establish internal rules for the use of Analysis of archives: Google Pinpoint, Brandwatch, Talkwalter

AI. In most cases they forbid the publication of automatically

generated news without supervision, except in specific cases.

One of the most widely implemented automatic news gene- Most important uses of AI by news organisations in 2024
ration solutions is Narrativa, based in Madrid. Very important Somewhat important Not/not very important Don't know

Radar is a British company that offers an automated Back end automation (e.g. tagging/
transcription/copyediting, etc) 56% 36% 7%

system that analyses news, social networks and other Distribution and recommendations
37% 39% 19% 5%
(e.g. personalised home pages, alerts, etc)
sources to identify potential stories. From the selection,
Content creation albeit with human
the algorithm generates a draft that has to be reviewed by oversight (e.g. summaries, headlines, etc)
28% 44% 25%

a team of human editors. Commercial uses

(e.g. better propensity to pay models, etc)
27% 31% 27% 14%

Coding and product development 25% 37% 20% 17%

News gathering (e.g. help identifying

stories/interrogate data) 22% 49% 27%

To what extent will the following uses of artificial intelligence (AI) and generative AI be important to your company in 2024?
Base: 296. Reuters Institute.
 A I B E H I N D T H E S C R E E N 14

Index

Artificial Intelligence

02 Image generation
The emergence in 2021 of DALL-E, the application developed by _ QUALITY LEAP
OpenAI that allows images to be generated automatically from texts,
In little over a year the quality leap in the generation of images has
revolutionised the creative scene and the world of graphic design.
been spectacular. To achieve this, they have been fed with millions of
Since then, new and more powerful versions have appeared, such images circulating on the internet, which has called into question the
as DALL-E 3 and other similar systems. The most consolidated basis of their legality.
and most used are Midjourney and Stable Diffusion.
Compare the resolution and detail of the two images generated with
At the beginning the images were a bit sloppy, but over time they Midjourney. Only 18 months have gone by between one and the other
have been improved and now achieve high levels of quality by either (see Hyperrealism section).
generating realistic compositions or imitating the style of well-known
artists.

THE DATA

By August 2023, 15 billion images

had been generated with AI, as many as
photographs have been taken in 150 years

_ Source: Everypixel

Source: Alphasignal.ai
 Image generation A I B E H I N D T H E S C R E E N 15

Index

_ WHAT CAN BE DONE?

All sorts of things: drawings, illus- In addition to generating images, the most recent versions automate
trations, character design, book all kinds of editing effects: changing the background, deleting objects,
and magazine covers, posters… embedding plain text or text with effects, modifying colours, conver-
ting images into 3D, copying the style of another image, modifying the
One of the first AI-generated de- point of view, vectorising images, upscaling, enhancing...
signs (2022) that had the greatest
impact was the cover of Cosmo- The editing functionalities that have been emerging through specific
politan, designed in, they say, in apps are being integrated into the traditional editing tools.
20 seconds by illustrator Karen
X. Cheng. Adobe Firefly has been chosen by Time magazine as one of the
best innovations of 2023. Canva also incorporates automated
generation and editing functionalities.

Google's AI Magic Editor has gone further and offers to change the
facial expressions of the people photographed. It has not avoided
The cover of the novel House of Earth criticism.
and Blood, by Sarah J. Maas, a New
York Times bestseller published by Amazon has incorporated a tool that makes it possible to retouch
Bloomsbury in 2023, made with the the images of the products that are sold on its platform.
help of IA was criticised by many ar-
tists and designers.
 Image generation A I B E H I N D T H E S C R E E N 16

Index

_ AWARD-WINNING WORKS

The verisimilitude of machine-generated creations can be confusing The winning entry in the Colorado State Fair art contest in the
for the public if the origin of the work is not made explicit. It can con- summer of 2022 was generated by video game creator Jason
fuse even juries. Allen using Midjourney.

The company Absolutely AI intentionally sent an image created

using artificial intelligence to an Australian photography competi-
tion. It fooled the jury and won a prize, but then they gave it back.
Their idea, they say, "was to show that we are at a turning point
with artificial intelligence technology".

Space opera theatre, winner of the Colorado State Fair

"Photography" by Absolute AI
 Image generation A I B E H I N D T H E S C R E E N 17

Index

_ STORYBOARDS _ GRAPHIC STORIES

One of the most interesting features for audiovisual creators is that AI generators have entered the world of comics and graphic novels in
they can draw the storyboards for a sequence. Once the characters a big way.
have been defined with or without the help of the AI, the scenes can
be visually designed from the description of the sets, costumes, types Neuralcanvas is an artificial intelligence machine specialised in
of shots, etc. with more detail than usual and, most of all, saving time. comic book design. From three dollars it helps you to create the
story and illustrations of a graphic story.
Arter.xyz is an artificial intelligence application specialised in
storyboard design that has won several international awards. Japanese manga and anime is undergoing a real transformation
due to the large number of specialised solutions in this field. ZMO.
AI, Fotor, Comicai and Getimg.ai are some of the tools that facilita-
te this style. The book Cyberpunk Momotarō has had remarkable
How
success in Japan; it was directed by artist Rootport but entirely
Arter.xyz works
written and drawn by an AI.

Cover of Cyberpunk Motomaro

 Image generation A I B E H I N D T H E S C R E E N 18

Index

_ TOOLS OR SUBSTITUTES? _ EXPLORATION FIELD

The public emergence of generative artificial intelligence has shaken There are also artists who see in these systems a creative opportunity
the creative community, especially artists and graphic designers. or, at the very least, a terrain to explore.
There are those who fear for the future of their work, the art market
is concerned about the arrival of a flood of artificial creations, many In the first months of 2023, the MoMA in New York exhibited
based on the imitation of the style of recognised authors, jurists the work Unsupervised, created by Refik Anadol. It consisted of
are uncertain about how to treat the copyright of these images and a mosaic of 380,000 moving images generated by an automatic
society in general fears that the door is being opened to the mass learning system from other works of art and pieces from MoMA's
creation of forgeries and deceptive images, which are increasingly own collection. Last summer, the museum decided to acquire it
easier to produce and more difficult to identify. and incorporate it into its art collection.

The digital art platform MakersPlace has opened a marketplace

for AI-generated artworks.

NightCafe is an AI digital art generator, combining artificial intelli-

gence with art to create unique images.
 A I B E H I N D T H E S C R E E N 19

Index

Artificial Intelligence

03 Music generation
Long before texts or images began to be generated, artificial
intelligence systems were already being applied to music: imitating
styles, composing, orchestrating, generating sounds, detecting
successful patterns... In fact, the first piece of music created with
a computer dates back to 1957, Illiac Suite.

_ AIDS FOR CREATION AND CO-CREATION

There are many available AI tools that facilitate the creation of

music or create it directly. They are basically professional tools to
help production or artistic composition, or generators of functional Aiva.ai lets you select from eleven styles or you can indicate a
music for corporate videos or ambient audio. To name a few: melody already created and it will create another one following in
that style.
Amper Music is one of the most liked because of its user-friend-
liness and for the possibilities it offers for composition. A more sophisticated tool is Jukebox, from the company OpenAI,
the same company that created ChatGPT. Jukebox can create the
Songmastr masters your songs by mixing them with reference whole song from the lyrics, the melody, the voices and the instru-
ones so that they sound like, for example, a Beyoncé song. mental sounds.

On Beatoven.ai you upload a video or a podcast, you choose from Lalal.ai and Moises.ai are useful tools for separating tracks or
twenty different genres, select from seventeen "moods", press play extracting the voices from a recorded track.
and it suggests a melody.
 Music generation A I B E H I N D T H E S C R E E N 20

Index

_ TEXT-BASED GENERATION _ TEXT-BASED GENERATION

Text-based generators have also made their way into music. Remastering, recovering sound archives damaged over time, editing
the musical production of old songs, versioning... These are new
One of the most advanced is Lyria, from the Google DeepMind features made possible by the use of artificial intelligence.
lab. It is an evolution of the MusicLM model that generates music
from short text indications and with very high quality. You write Now and Then is the title of an unpublished song that John Lennon
a sentence about the type of music you want and the program composed in the late 1970s. He recorded it on a home cassette
composes it. It also recognises melodies that you hum or whistle, tape and it was left in a box until McCartney, Ringo, Yoko Ono
or song sketches. and Harrison's widow were able to recover it thanks to artificial
intelligence techniques that pulled out John Lennon’s voice
It's not the only one. Riffusion is an open-access website that and isolated it from the badly-recorded original sound. It was
offers similar features. Mubert generates music to accompany released last November.
videos, podcasts... MusicGen is Meta's tool.

The Music ControlNet model adds editing possibilities so that the _ A FERTILE GROUND FOR EXPERIMENTATION
user has more facilities for sound production and control of the
resulting music. There are several international centres dedicated to resear-
ching the relationship between music and artificial intelligence.
Among the most prominent is the Sony Computer Science
Laboratories in Paris, which in 2016 used neural networks to
Some examples of music create DeepBach, a music generator that imitates the style of
generated with MusicLM Johann Sebastian Bach.
 Music generation A I B E H I N D T H E S C R E E N 21

Index

In Catalonia, there is the MTG (Music and Technology Group) of _ AI SONGS MOVE MILLIONS
the Universitat Pompeu Fabra, which develops all kinds of tech-
The automatic generation of music seems to have stimulated the
nologies: synthesis, audio analysis, pattern detection, etc., and
desire to do business.
the IIIA (Artificial Intelligence Research Institute from its initials in
Catalan) of the CSIC, which carries out various projects related to The large Chinese technology corporation Tencent has launched
music and AI. more than 1,000 songs that imitate human voices. Some have sur-
passed 100 million views.

Hybe, a South-Korean entertainment company listed on the Seoul

Stock Exchange and which manages, among others, the band
BTS, has bought Supertone, a start-up specialising in the creation
of artificial voices, for $32 million.

Google and Universal Music have agreed to develop an engine

that will allow users to generate music with the look and style of
the record company's singers. Google will pay for the right of use
and the artists will have to give their consent. With the collaboration
of Universal musicians, YouTube has set up an AI Music Incubator
to jointly design the business strategy.
 Music generation A I B E H I N D T H E S C R E E N 22

Index

_ UNLOCKING THE SECRET OF SUCCESS _ FIGURING OUT RIGHTS

Beyond generation, artificial intelligence is also being applied to Data analytics has helped to bring order to the management of
analytics, pattern detection and recommendation. musical works reproduction rights.

Stanford University psychologist Justin M. Berg has analysed the BMAT is a Barcelona-based company specialised in tracking and
patterns that make up hit songs from more than 3 million songs indexing the reproduction of musical works around the world and
produced between 1959 and 2010. He has used an algorithm on all channels. With the Vericast tool, they monitor around 6,000
from the company EchoNest to measure sound characteristics TV and radio channels all worldwide, which they compare with a
such as tonality, tempo, "danceability”... Result: imitating gives database of over 60 million musical works. This allows them to
good results. identify the songs and generate playback reports that are distri-
buted to authors' societies and music publishers.
The Claremont Graduate University (California) has specialised in
the analysis of success in music. CGU researchers have develo-
ped AI tools that analyse the factors that influence the commercial _ AI TO CREATE FILM SCORES?
success of music, such as distribution, promotion and advertising, Although most artificial musical productions are short pieces, concerts
those that influence expert critics and public reception, and those generated and orchestrated by AI have already been programmed.
that have the greatest influence on popular culture. It's not hard to imagine that software could create the soundtrack for
a long audiovisual production.
Spotify has made audience taste prediction and hyper-
personalised recommendations the key to its success. It has Gareth Edwards, director of The Creator, tried to produce the film
its own development department and in the last five years score with AI, imitating the style of composer Hans Zimmer, who
has bought six AI start-ups that cover everything from user won two Oscars for the film score of Dune and The Lion King. The
behaviour analysis to audio recommendation and detection. They result was acceptable, but in the end Zimmer himself was hired.
are not the only ones. Recommendation superpowers such as
One of the most valued tools is Soundful, although it hasn’t rea-
TikTok and YouTube achieve huge audiences with personalised
ched the quality of traditional music catalogues yet. Stable Audio
broadcasting of music clips.
or the aforementioned Google tools are also popular options.
 A I B E H I N D T H E S C R E E N 23

Index

Artificial Intelligence

04 Video Generation “This is pretty amazing progress. It's much

harder to generate video than photos because beyond
correctly generating each pixel, the system also has to
If you can generate static images, why not generate them in motion? predict how they'll change over time”

_ STATIC IMAGE ANIMATORS Mark Zuckerberg, CEO of Meta

The first steps have been taken by applications that generate _ THE CHALLENGE OF CONSISTENT GENERATION
movement from static images. Nowadays there are hundreds such
as Midjourney V6, MotionGPT, Blender, AnimateZero, Live Photo or Technology already makes it possible to generate video clips, now
DreamVideo that, based on a text indication, generate slight move- there are advances regarding the tools to join them together and build
ments of the characters or the backgrounds (landscapes, clouds…) stable and consistent sequences.

More sophisticated are Luma's flythrough functionalities that One of the most popular generators is Pika, widely used for short
allow you, for instance, to explore a three-dimensional image with videos that are spread on TikTok and other social networks, but
virtual drone flights. also with useful functionalities in the professional field. In recent
months it has secured 55 million dollars of new investment.

Some reference models are CogVideo, created by Chinese

researchers, and Imagen Video or the announced Videopoet,
Flythroughs by Luma.ai from Google. Meta has presented EMU Video which integrates the
generation of text to video with the animation of static images, a
methodology that could exceed the quality of the other options.
Stable Video Diffusion seems to be following the same path.
 Video Generation A I B E H I N D T H E S C R E E N 24

Index

Researchers from the company Sense Time (Hong Kong) have The updates presented in mid-2023 represent a remarkable quality
presented Story-to-Motion, a video-generation model capable of leap that can transform the way professional audiovisuals are pro-
chaining sequences from text prompts. duced. Directors and art directors have a tool in their hands that can
change the way they characterize characters, as well as the way they
design scenes and shoot them.
_ FROM VIDEO TO VIDEO

Among the main novelties that have emerged in recent months are
systems that allow you to generate a video from another video.

RunwayML's Gen-1 and Gen-2 are the most prominent applications

in this area. Their emergence opened up new possibilities for the
generation of home video as well as professional video by also
combining text, image or video sources.

Any realistic video can be turned into an animation or the other

way around: a poorly drawn fish becomes a sea monster; an Gen-1, RunwayML

innocuous scene recorded with a mobile phone is transformed into

a fantastic adventure. A home video can be the starting point of a
professional sequence, without any need for lighting, construction
or set design, only by having a few text instructions or images that
serve as a model. Image-to-video comparison:
Pika vs Runway (August 2023)
Source: AI Video School
 Video Generation A I B E H I N D T H E S C R E E N 25

Index

_ FROM VIDEO TO ANYTHING

Multimodality is gaining ground. Any input, whether text, image, audio

or video can serve as a starting point for artificial generation. The
most spectacular step has been taken by Google with the presentation
of Gemini, a next-generation model capable, among other functions,
of decomposing and interpreting video images. According to their
own analysis, the most advanced version (Ultra) still in the testing
phase exceeds almost all the functionalities of the other models,
including GPT-4. Trailer: AI Star Wars teaser
Created and edited by Dave Villalva
A few weeks before Gemini was introduced, OpenAI released GPT-
4 Turbo with Vision, which analyzes images, describes them, and
generates subtitles.

_ CO-GENERATION, CO-CREATION, COMBINATION

The combination of artificial generation tools with conventional tools

offers film and media producers and artists new creative possibilities.
Advertising:
In the following QRs you can find some examples resulting from the Coca Cola (2023).
combination of tools and artists’ work. Stable Diffusion
 Video Generation A I B E H I N D T H E S C R E E N 26

Index

“At the rate that research in this area is progressing,

it is likely that in a couple of years we will be able to see a
television program created entirely with similar techniques”

Alonso Martínez,
Principal AI Researcher at Google

Advertising: Cruzcampo (2021).

Created by Metropolitana _ VIDEO EDITORS
(Barcelona)
Video editors that integrate generative AI functions are gaining
ground: from the generation itself to the removal or incorporation
of visual objects, colour or lighting correction, resolution improve-
ment, etc. CapCut and VEED.IO stand out among the most used. But
also: Clipchamp (from Microsoft), FlexClip, Magisto (from Vimeo) and
Cutout.Pro. Also Palette to colour black and white images.

Almost all traditional editing tools incorporate AI functionali-

ty. Adobe began by incorporating them into After Effects and
Premiere Pro and offering comprehensive generation software
VFX: generated by called Adobe Firefly. But now they are also in DaVinci, Final Cut
Martin Haerling Pro, Avid Media Composer, Vimeo and Filmora.
with Pika
 Video Generation A I B E H I N D T H E S C R E E N 27

Index

_ AI AS A GENRE _ GENERATION OF VIDEO GAMES

The +RAIN Film Fest (Barcelona) is the first European festival of Generative AI is rapidly entering video game design and production.
films generated with artificial intelligence. The festival is interes-
ted in the use of creative methodologies with AI at the service Roblox has integrated automatic generation facilities for its
of storytelling. It came to life as a collaboration between Pom- users. They allow users to create 3D characters, landscapes and
peu Fabra University, Sónar and the Catalan Audiovisual Cluster. situations from simple text instructions or use natural language
Next edition: June. to interact with virtual assistants who guide and resolve doubts
for the most professional programmers.
Runway AI Film Festival is the North American version of a similar
festival. Videos up to 10 minutes long are accepted and there are The main video game factories such as Blizzard (World of War-
prizes worth $60,000 in cash and vouchers to use the company's craft), Ubisoft (Assassin's Creed) and Square Enix (Final Fantasy)
tools. Next edition: May. are incorporating AI functionalities to help their programmers save
time in the most tedious processes.
Los Angeles hosts AI Cinema Screenings in November.

_ GENERATION OF VR

Artificial intelligence is also incorporated into Extended Reality devi-

ces and applications to accelerate the rendering of virtual worlds or
the ultra-fast generation of frames and thus improve the feeling of
immersion. The possibilities of real-time generation open the door to
previously unthinkable interactions such as those offered, for exam-
ple, by The Grid Factory for virtual sports broadcasts.
 A I B E H I N D T H E S C R E E N 28

Index

Artificial Intelligence

Generative AI.
Risks, uncertainties and sustainability
Rarely before has technology so directly affected an area that we _ WHO IS THE AUTHOR?
considered exclusive to the human condition: artistic creativity.
There are already a number of lawsuits in the courts against gene-
With the arrival of generative AI, the boundaries between original and rative artificial intelligence platforms for infringement of intellectual
copy are blurred. All artists are influenced by other artists, but can it property rights.
be considered a simple influence when artificial systems take frag-
The artificial generation of music, text or images is based on the
ments of original works or copy the style outright?
analysis of previously created works, which have an author or, in any
Generative AI is causing contradictory feelings and reactions. On the case, a person or entity that decides what use they want to make of
one hand, there are those who see it as an opportunity for creativity: them. Artificial generators track databases, especially everything that
has been uploaded online, establish patterns and make it possible to
New tools for creative support.
generate huge quantities of combinations and proposals.
Reducing mechanical and repetitive processes.
New materials for experimentation, new artistic terrain. Aside from very specific cases, no one asks the original creators for
permission to use their works as a training source for these systems.
On the other hand, those who see disadvantages and risks: This is where the complaints are coming from.
Job insecurity in the creative sector.
Cultural overproduction and loss of value.
Cultural homogenisation and devaluation of diversity.
Confusion with copyright.
 Generative AI. Risks, uncer tainties and sus tainabilit y A I B E H I N D T H E S C R E E N 29

Index

_ COURT CLAIMS

The Recording Industry Association of America (RIAA) published a Those who have done it are the American Writers' Association
report condemning these practices and announcing a legal battle (Authors Guild), led by authors such as George R.R. Martin,
against the platforms that use them. Jonathan Franzen and John Grisham. With more than 13,000
affiliates, it has started the most important legal battle that OpenAI
The North American Recording Academy has stipulated that has had to face. If found to be in infringement, the company could
in order to qualify for awards such as the Grammys, the main have to pay hundreds of millions of dollars in damages and rebuild
elements of a piece of music, such as the voice, must be human or the models trained with copyrighted works.
created by humans.
Getty Images, one of the largest photography platforms, has sued
Several collective lawsuits by artists and writers have been filed in the company that owns Stable Diffusion before courts in the
North American courts for copyright infringement and data theft United States and London for having illegally processed millions
through scraping techniques. The New York Times has blocked of images protected by copyright. In 2023, it launched its own
the trackers that feed the generative systems databases and generation model with images that claim to respect the rights of
announced a lawsuit against OpenAI. authors who have agreements with the platform.

The large generative AI platforms trust that the judges will rule
in their favour, and the first rulings confirm this. But to prevent
panic from spreading and users being blamed for misuse of
protected material, Microsoft has announced that they will take
responsibility. OpenAI has said it will defend its users.

Image that reproduces the style of the painter Jackson Pollock. Has anyone
asked his heirs for permission? Image generated with Stable Diffusion
Source: OpenArt.ai
 Generative AI. Risks, uncer tainties and sus tainabilit y A I B E H I N D T H E S C R E E N 30

Index

_ AI TO PROTECT AGAINST AI _ LABELLING AI

Creators are feeling uneasy. Legislation protecting authorship is open Legislators and ethics specialists are calling for AI-generated work to
to interpretation and it is not clear that generative AI systems violate be labelled. The executive order that the White House issued in late
it. Just in case, there are those who use technology to protect them- October urged the Commerce Department to establish a “watermar-
selves from technology. king” plan that would identify them. The US Congress itself is already
promoting a labelling law.
Glaze is a tool developed at the University of Chicago that aims to
prevent AI models from recognizing the style of a graphic artist. Some private corporations have launched initiatives in this direction.
How do they do it? A computer program introduces changes to Google has three tools that allow you to identify the origin of an ima-
the work, imperceptible to the human eye but which mislead com- ge. OpenAI also has a tool to check if an image has been AI-genera-
puter vision systems. A new option from the same tool, dubbed ted. And Leica has launched a new range of cameras that certify the
Nightshade, goes a bit further: it makes AI models learn the wrong authenticity of the images.
names for the objects and scenarios they're looking at.
The effectiveness of these measures remains to be seen. It
may serve to identify works generated mostly by AI, but when
machine-generated creations complement human creations
and the end product is a mix of contributions, labelling will be of
“We should focus on building an AI that can little use.
represent the values we aspire to in the future,
instead of simply perpetuating data from the past”

Gary Marcus,
psychologist and neuroscientist
 Generative AI. Risks, uncer tainties and sus tainabilit y A I B E H I N D T H E S C R E E N 31

Index

_ ETHICAL QUESTIONS

Beyond the possible violation of copyright, generative AI presents Waking the dead. AI makes it possible to recover the voice or
other questions and uncertainties. image of dead artists and incorporate them into new productions.
Despite the consent that the heirs may give, it is a practice that
Disinformation.Text generators make it easier to manufacture raises ethical questions.
manipulations and disinformation, due to the speed of production
and the verisimilitude of the texts they generate. Who is the author? Determining who the true author of a creation
is will not be easy. What percentage of an AI's contribution will
Hallucinations. The language models on which generative AI are we consider admissible? Should works created with the help of AI
based make mistakes and invent answers. An excess of confiden- tools be identified?
ce in these systems could cause "hallucinations" to spread.
Identification. Can the texts or images generated by an AI be
Biases. Generative models have been trained with data that res- detected? There are already programs that are trying. It won't
pond to dominant race and gender stereotypes in Western society. be easy.
Almost two years after the appearance of the most popular gene-
rative systems, the biases are still there

Falsifying visible reality. Image manipulation is reaching levels of

realism that make them indistinguishable from the real thing. Video
and audio deep fakes that simulate people are becoming easier
to make. The temptation is great even for solvent companies.
Adobe Stock sold fake photographs of the war in Israel. The CEO
of Sports Illustrated magazine resigned after admitting to publi- This girl doesn't exist
Image generated at
shing AI-generated articles without saying it. Thispersondoesnotexist
 Generative AI. Risks, uncer tainties and sus tainabilit y A I B E H I N D T H E S C R E E N 32

Index

_ ECONOMIC SUSTAINABILITY _ THE ENERGY FOOTPRINT

By the end of 2023 ChatGPT had 100 million weekly active users; an All these automated generation models require a huge amount of
impressive figure. Generative AI revenues could exceed $1 billion by data to process.
2023, but even so, the profitability of these systems is unclear. Busi-
nesses and professionals are trying paid premium versions, but free Independent researchers have estimated that just training GPT-
and open source options are becoming more comprehensive. On the 3, the model that has supported ChatGPT so far, generated more
other hand, the production and maintenance costs of the language than 550 tons of carbon dioxide emissions, the equivalent of about
models that support them are very high. 120 cars running on gasoline for a year.

Is generative AI economically sustainable? It’s still a question without Generating a single image using a powerful AI model requires as
an answer. much energy as fully charging a smartphone.

If the emission costs of the necessary infrastructure and those of

daily operation are taken into account, the figures multiply. Is it an
THE FORECAST
acceptable cost?

By 2030, the energy demand from data centres

will have increased 15-fold
_ Kate Crawford in Atlas of IA
 A I B E H I N D T H E S C R E E N 33

Index

Artificial Intelligence

Generative AI. Trends

TRENDS

_ Integrations. After Microsoft integrated OpenAI products _ More accuracy, less bias. Today's text generators still make
into its software many companies did the same. Search noticeable mistakes. On some topics they are vague or give
engines, design platforms and all kinds of traditional tools wrong data. Debugging the systems through manual verification
are incorporating them into their portfolios. New start-ups processes and the training involved in using them daily by
specializing in specific AI solutions are also expanding their millions of users will gradually reduce the errors and biases
catalogue by integrating third-party tools to offer increasingly detected. Smaller and more specialized language models will
more complete services. also be built, aimed at being precise in specific thematic areas.

_ Multimodality. New capabilities of AI generators and new _ Realism. Increased precision and verisimilitude. Debugging
developments will facilitate multimodality, that is, solutions that automated texts will bring them closer to the desired style,
allow the generation of any product from any input. Text prompts with the possibility of incorporating expressive characteristics
are no longer the only entry point. It can be an image, a video, a linked to emotions. This phase could be interesting for fiction
piece of music... Same applies to the results generated. generation. The images are also getting more faithful to the
realistic models of faces, objects or spaces.
_ Control and editing. Until recently, generators presented closed
results based on text prompts. If you were not convinced by the _ Independent agents. The next phase for chat bots could
result, you had to generate new proposals. Control and editing be autonomous agents, bots capable of commissioning and
features of the final product are now being incorporated. coordinating tasks with other robots. Many experts position
them as the next big evolution in AI.
 Generative AI. Trends A I B E H I N D T H E S C R E E N 34

Index

TRENDS

_ Video progresses appropriately. We will continue to see _ Consistency. Gen AI needs consistency: the ability to
remarkable progress in video generation. From the short and create content that is coherent and uniform across different
shaky animations of the first few months, things have moved on generations. This includes maintaining consistency in
and now we’re seeing decent quality sequences being created. characters, objects, styles, and other elements within the
But there is still a long way to go. generated content. Achieving consistency is considered a major
challenge in the field of generative AI.
_ Pending in court. Further rulings on copyright infringement
lawsuits will be divulged. Those decisions could decide the _ Business concentration. The large language models that
future of these systems. Legislators will have to be vigilant support generative systems are expensive to implement and
in case it is necessary to modify the current regulations on maintain. Large technology corporations will concentrate
copyright protection. services and dominate the market. Many of the start-ups
emerging in 2023 will disappear or be absorbed.
_ Improved detectors. Text or image detectors generated by
AI will continue to be perfected. The ease with which they can _ Earn loyalty. There are so many novelties emerging every week
multiply manipulations and generate disinformation means in all kinds of generative AI services and benefits, that user
mechanisms of control and identification will need to be loyalty is very low. Everyone is trying everything. Companies that
established. want to consolidate will have to design strategies to retain users.
 Generative AI. Trends A I B E H I N D T H E S C R E E N 35

Index

TRENDS THE FORECAST

_ AI in professional productions. AI is gradually making its way into Spending in Generative AI

professional audiovisual productions. The ability to generate synthetic software and hardware or IT/
images in real time combined with the possibilities offered by virtual business-related services
production and the incorporation of video game engines will transform is expected to reach $143
the way we produce. billion in 2027. This
represents a compound annual
growth rate (CAGR) of 73.3%
MID-TERM TRENDS
over the 2023-2027 period
- More time spent on pre-production and less on post-production.
_ IDC
- Incorporation of new professional profiles, especially those involved in video games.
- Regular use of text generators to support scriptwriting.
- Automated personalization of marketing and promotion.
- Replacement of specialists and extras by digital doubles. LONG-TERM TRENDS
- Growth of dubbing with artificial voices.
- Automated multilingual subtitling. By 2030, a major blockbuster film
will be released with 90% of the film
- Regular use of AI-generated videos for transitions and B-roll footage.
generated by AI (from text to video),
- Growing impact of AI on animation productions and those that require visual effects. from 0% of such in 2022.
- Increasing impact on the production of advertising and videos for social networks. (Gartner, January 2023).
 A I B E H I N D T H E S C R E E N 36

Index

Artificial Intelligence

05 Production, post-production, VFX

Artificial intelligence is starting to be a common resource for many Speeding up rotoscoping processes.
tasks in production, post-production and visual effects creation. Improving motion capture (mocap) of a real character to transfer
Better quality and spectacular graphics, real-time generation, them to an animation.
reduction of work processes, and file reuse are some of the benefits Facilitating frame interpolation.
that automatic learning technologies, neural networks and computer
vision bring to the creation of audiovisual products.
_ TRANSFORMING REALITY

Systems that allow changing faces, moving lips or generating facial

_ REDUCING TIMES
expressions are becoming a recurring tool in film and television
AI makes it possible to substantially reduce the repetitive tasks in productions. They make it possible to replace actors and actresses,
animation and VFX processes. It frees animators from monotonous rejuvenate or age characters, incorporate actors who have not
tasks and helps them focus on the most creative aspects and reduce participated in the filming, modify settings...
production times by:
Re-aging. Modifying the age of the characters without having to go
Automatically removing characters, objects or imperfections from through long make-up sessions or manual post-production can be
the video. done in an increasingly automated and believable way. It is not an
Recreating repetitive movements in an animation. easy task.
 Produc tion, pos t-produc tion and visual ef fec t s A I B E H I N D T H E S C R E E N 37

Index

Metaphysic also provides automated face swapping and reju-

Examples of re-aging in the Disney venation technology (it's the company that produced an AI video
Research labs series called Deep Tom Cruise).

_ REAL TIME

The acceleration of processes that is a feature of artificial intelligence

makes it easier for animation and effects creators to work in real time
and in graphically complex productions with a high level of detail.

Video game engines have been incorporated into film and television
production processes. Their features make the spectacular 3D
effects or graphic quality until now reserved for big film productions
Disney Research presented a few months ago a solution that accessible to all types of productions.
improves the resolution and prevents instability in the movements
Unreal Engine, from Epic Games, is the most widely implemented
and loss of quality from these techniques.
real-time 3D graphics generation tool. Designed for the develop-
Vanity AI, created by the Marz VFX studio, allows the automation ment of video games, it is also applied to content creation for
of make-up, aging and digital rejuvenation. They claim it's 300 television and cinema, production of live events, and creation and
times faster than traditional VFX workflows. simulation of spaces, machines or complex virtual environments,
among other uses.
Perfection42 offers a style transfer tool that allows an artist to
work on a few key frames and then apply that style to subsequent Unity is also one of the market’s most powerful virtual creation
frames. and real-time production engines for animations and effects.
 Produc tion, pos t-produc tion and visual ef fec t s A I B E H I N D T H E S C R E E N 38

Index

One of the most spectacular tests of real-time production was The technique known as ICVFX (In Camera Visual Effects) radically
presented by Sony a few months ago in preparation for a new episode transforms production processes by generating the visual effects at
of Ghostbusters. An entire sequence was shot without cameras and the time of filming without having to wait for post-production.
with hyper-realistic images generated by computer and animated in
real time with Unreal. VFX specialists join the film shoots.

Directors and producers can visualize the final look of the scene.

New professional profiles such as specialists in video game

engines are brought into the production.

Images can be generated while filming. Cuebric is one of the main

Ghostbusters sequence generated in tools specialized in the generation of settings with AI, created by
real time without cameras the Seyhan Lee collective.

What benefits does it bring?

Quality. Physical and virtual elements are merged creating more

_ VIRTUAL PRODUCTION realistic and better quality results compared to the chroma key
used until now.
Film sets and studios are rapidly implementing audiovisual production
Savings. They reduce post-production tasks and time.
systems that integrate large walls of LED screens, virtual image
generation engines and camera tracking mechanisms and dynamic Real time. The generation of visual effects and complex graphics
lighting. It’s known as virtual production. in real time focuses the work and decision-making on the film set.

Dimension. The virtualization of the space increases the size of

the sets.
 Produc tion, pos t-produc tion and visual ef fec t s A I B E H I N D T H E S C R E E N 39

Index

The Mandalorian (Disney, 2019) marked the start of the expansion

_ SCREENS
of virtual production systems in film and television. A team of 80
specialized professionals coordinated by Lux Machina, from the NEP LEDs. The LED screens incorporated in virtual production systems in
group, participated in this production. For more modest productions, studios and film sets are often more than 100m2 and 4K definition. This
the crew is significantly reduced. allows them to reproduce real or virtual spaces with a level of detail,
dynamic lighting and graphic quality that far exceeds the performance
Can an entire film be shot with virtual production?
of the chroma green walls, which have been commonplace until now.
At the end of 2023, Parenostre was the first European film shot
In the outdoors, the modularity of LED screens allows for large
entirely through virtual production at the Mediapro Studio and
installations such as the 6,500 m2 video scoreboard The Infinite
produced by Last Minute, Minoría Absoluta and Lastor Media.
Screen that Samsung installed at the 2022 Super Bowl.
Barcelona is one of the leading European cities in incorporating
virtual production systems. In addition to the one mentioned above,
it’s also worth mentioning the facilities of Gestmusic (Banijay) and
also Plató Nou from Lavinia Group with Disguise technology.
Features of Infiled's virtual
production system
_ INFiLED

Plató Nou. Virtual Production Studios (Barcelona)

 Produc tion, pos t-produc tion and visual ef fec t s A I B E H I N D T H E S C R E E N 40

Index

The Sphere in Las Vegas is an impressive ball measuring 112 Ultra High Definition (UHD). Television manufacturers are committed
meters high and 157 meters wide, equipped with a spherical screen to constant quality improvement. The current milestone is Ultra
of 1.2 million LED points on the outside. A U2 concert opened the High Definition (UHD) which includes technologies that increase
venue at the end of September in front of an audience of 18,000. the resolution of screens to 4K (4,000 pixels horizontally),
8K (8,000 pixels) and other higher formats.
LED screens can have all kinds of shapes and be used for all kinds of
purposes. High-quality animated displays replace static advertising Screen manufacturers are filling the market with 4K devices, but their
panels and are incorporated into shop windows and inside stores. aim must be in line with the will of producers and broadcasters. For
Commercial pedestrian streets are filled with screens platforms and OTTs that broadcast through high-speed networks
(5G, 6G) it will not be a problem. On the contrary, it can be a
competitive advantage against the linear television channels that
broadcast via DTT, a space limited by the bandwidths required by
UHD. To compensate, some manufacturers incorporate artificial
intelligence systems that multiply the resolution of the contents
inside the television.

“AI will reduce the cost

of animation by 90%”

Jeff Katzenberg,
co-founder of DreamWorks

Sphere, the largest LED screen in the world

Source: Wikimedia Commons
 Produc tion, pos t-produc tion and visual ef fec t s A I B E H I N D T H E S C R E E N 41

Index

_ DIGITAL PRODUCT PLACEMENT _ AI ENTERS MOVIE THEATRES

Techniques for removing, adding, or replacing 3D objects are AI is not limited to production. There is a whole range of possibilities
increasingly used in advertising. The possibilities of product offered by data analytics (see chapter 8) but it is also reaching the
placement increase and become personalized. It is now possible cinemas.
to insert objects and 3D products depending on the scene and the
audience. HyperCinema is being promoted as the first movie theatre to use AI
to bring viewers into the productions being screened. It has opened
Ryff is a Californian company specializing in dynamic advertising in Auckland, New Zealand. When users enter the room, their image
insertion. It analyzes any moving image content, identifies is captured and from there they become part of the adventures or
opportunities to place a product depending on the context, stories that appear on the screen, like another character.
incorporates the 3D representation of the product in the scene and
can customize it depending on the country, type of audience, etc. Bristol will have a cinema that will analyze audience reactions.
A company with a similar product is TripleLift. Instrumented Auditorium is the name of the new cinema with
capacity for 36 people that will be launched in 2024. The novelty is
that it will record the biometric responses of the viewers, including
their heart rate, eye movement, brain activity and skin reactions.

Personalized cinema based on emotions. Emotional Films is a

project born in Galicia to generate films in real time depending on
the emotional reactions of the public. 40 researchers from various
Ryff's Spanish universities are taking part.
dynamic
product
placement
 Produc tion, pos t-produc tion and visual ef fec t s A I B E H I N D T H E S C R E E N 42

Index

TRENDS

_ Virtual production. The combination of LED screens and video _ New life to the archives. The complexity of handling archive
game generation engines is prevailing on film and television images is simplified: identification, viewing and even restoration
sets. Large facilities will be equipped but also small spaces that and editing can be delegated, in whole or in part, to automated
will virtually multiply their dimensions. Generative AI will play an processes.
important role in this.
_ The battle of the definition. The increasing quality of screens
_ Real time. The ability to generate three-dimensional graphics will not be an innocuous topic. It can lead to winners and losers
in real time reduces post-production processes and jobs. between platforms that use ultra-fast networks to distribute UHD
The incorporation of artificial intelligence systems means post- content and television broadcasters that have limitations to do
production processes and tasks will need to be reassessed. the same in their natural realm, DTT.

_ Fewer outdoor shoots. Video game generation engines _ Production goes to the cloud. The high performance offered
combined with the quality of LED screens open up the possibility by centralized computing and the improvement of network
of replacing shooting in physical spaces with hyper-realistic, connections allows for many jobs that had to be done locally to
high-quality virtual recreations. be diverted and shared in the cloud. VFX creators can now work
remotely on the same project and share animations or graphic
_ Digital immortality. Voice cloning technologies combined with effects.
re-aging techniques and hyper realistic generation of characters
opens the door for actors' lives to be as long as the producers'
interest in keeping them "alive".
 A I B E H I N D T H E S C R E E N 43

Index

Artificial Intelligence

06 Synthetic hyperrealism
Ziva

Of all the advances that artificial intelligence brings to the produc-

tion, post-production and creation of visual effects processes, what
stands out is the ability to generate characters and virtual spaces a library of data and algorithms with more than 72,000 applica-
that seem real. ble facial expression movements. The results are “digital humans”
with surprising expressiveness.
The quality of the textures, details and movements of the virtual
characters makes them increasingly indistinguishable from the
real ones and they can replace actors and actresses without the _ DIGITAL DOUBLES
viewers perceiving the swap or perform assistance functions and
Virtual humans have been used in film production for several years.
interaction with users with complete normality. The recreation of
Since the appearance in 2001 of Aki Ross, the virtual protagonist of
three-dimensional spaces and objects is also increasingly detailed
Final Fantasy, the incorporation of digital doubles into film productions
and believable.
has been non-stop.

In 2020 the actress Malika Alaoui had to leave the filming of the
_ HUMAN EXPRESSIVENESS
French soap opera Plus Belle La Vie due to covid. A hyper-realistic
Human replicas created with digital techniques are beginning to synthetic clone replaced her in order to continue with production.
incorporate the expressive nuances that generate emotions.
They are also used as virtual assistants for applications or web pages
Ziva Dynamics is an American company specializing in the crea- and in social media campaigns. An example was DeepTomCruise,
tion of hyper-realistic synthetic characters, now owned by Unity. a production for TikTok of "fake" Tom Cruises that multiplied in a
One of the most spectacular products is the face "trainer" that has multitude of scenes created for this platform.
 Synthetic hyperrealism A I B E H I N D T H E S C R E E N 44

Index

The agreement that ended the Hollywood actors' strike ensures

that studios cannot create digital replicas of actors without their
permission and payment. It does not prevent studios from using Chinese Influencers
AI to create synthetic characters by blending the characteristics of
different actors. This possibility could affect the work of suppor-
ting or stunt actors. Another controversial issue is the possibility
for studios to require body scans as a condition of employment Synthesia is an English company with 85 avatars available for any
and to use past performances to train AI tools. company or individual to use as an assistant, presenter... They can
speak 120 languages and complete speeches with a catalogue of
Companies worth mentioning are Animatic Media, Eisko, CAA images also available. They are activated by simple text prompts
Vault and the catalan Low Key Moves. and anyone can use them with a $30 monthly subscription.

_ HOSTS, PRESENTERS AND DIGITAL INFLUENCERS

Virtual assistants that have a face and a voice are gaining in expres-
sive quality thanks to the hyper-realism that can be achieved through
increasingly precise and inexpensive technologies.

Lil Miquela, created in 2016 by the transmedia studio Brud, is

the pioneering virtual influencer and one of the most followed:
2.8 million followers on Instagram. It paved the way for virtual
humans living in networks. Amouranth, a real influencer who has
over 6 million followers on Twitch created AI Ammouranth to be
available 24/7. Zae-In, Korean virtual presenter
 Synthetic hyperrealism A I B E H I N D T H E S C R E E N 45

Index

Another reference company is the Israeli D-ID that offers both _ AVATARS
cheaper options and also others for professionals that allow the
One of the essential elements of virtual worlds are the avatars that
generation of exclusive characters. One of the latest features
represent the users digital identities.
introduced is the creation of videos from photos. It is also worth
mentioning the Korean DeepBrain AI, the North American HeyGen The vast majority of platforms offer the possibility of customizing
and the Hungarian/British Collosyan. fairly simple pre-designed avatars, but new trends point to realistic
three-dimensional creations created from the real image of the user.
The Chinese e-commerce platform Taobao is full of virtual
influencers. They are clones of real streamers who sell products At last September's Meta Connect conference, Meta presented a
24 hours a day. They cost about $1,000 to make. new generation of hyper-realistic avatars that replicate the faces
of real people and interact in real time.
Virtual news presenters started in the East but are spreading all
over the world: Zae-In in Korea, Sana and Lisa in India, Hermes in
Greece, Fedha in Kuwait, Ni Zhen in Taiwan...

One of the most complete hyper-realistic avatars is Alta B,

designed by Simon Fuller, the creator of the American Idol show.
Alta B has become a pop star. Her first song achieved 55 million
hits on YouTube. But in addition, Alta B interacts live with fans and
she can share choreography with real dancers. Behind it is the
company Hyperreal, based in the United States and in Barcelona.
Hyper realistic avatar of
Mark Zuckerberg
Alta B dances with
real dancers
 Synthetic hyperrealism A I B E H I N D T H E S C R E E N 46

Index

_ HOLOGRAMS "moonwalk" in 2014 at the Billboard Music Awards. Roy Orbison

and Buddy Holly even toured together in 2019.
Projections simulating holographic images at concerts and shows
have regained the prominence they seemed to have lost after their You don't even have to revive anyone to do a show with "holograms".
boom four or five years ago.
The Swede Eric Prydz has produced HOLO, a super show with
Holograms are projected images that can be observed from diffe- house music and "holograms".
rent viewing angles. Popularly the term is used for any projected
three-dimensional image.

One of the most impressive "holographic" shows produced to date Eric Prydz
was ABBA starring in a concert premiere in London in May 2022.
Hyper realistic avatars created by George Lucas' company Industrial
Light & Magic made it possible to produce a concert 40 years after
ABBA left the stages. The same company is responsible for creating
the avatars of the band Kiss presented at their farewell concert last
November and which from now on will tour around the world.

The temptation to revive artists is controversial. But holograms aren't limited to shows. Museums and exhibition cen-
tres also incorporate holography to show the public works that are
Prince hated the idea and stopped anyone from making a located too far away.
"holographic" version of him after his death. Relatives of Amy
Winehouse and Whitney Houston also prevented such projects. Dalí's Venus de Milo with drawers, property of the Art Institute of
Chicago, could be seen in full size at the Dalí Museum in Figueres
Resurrections: Rapper Tupac, murdered in 1996, "performed" at thanks to a holographic reproduction made from 72 photographs
the 2012 Coachella festival in California. Michael Jackson did the by the Girona studio Tururut Art Infogràfic.
 Synthetic hyperrealism A I B E H I N D T H E S C R E E N 47

Index

One of the latest novelties is holographic zoos. _ AUTOMATED AND BELIEVABLE DEEP FAKES

The Australian company Axiom Holographics is responsible for Deep fakes (fake videos of people who appear to be real) are a deriva-
one of the most spectacular creations of 2023: the Hologram Zoo tive of real life character simulation.
in Brisbane with 50 virtual animals. This year it will expand to Ja-
Used in comedy programs and in creations for social networks, they
pan, the USA and Europe. Time magazine considered it one of the
already have automated tools that ease their production and make
best innovations of 2023.
them increasingly believable.
Already a few years ago, the German Roncalli circus included
There are free face swap tools for every taste. DeepFaceLab,
shows with virtual animals projected onto the stage.
Reface.ai, Zao (Chinese), Wombo and Faceapp are some of the
applications available to anyone who wants to manipulate videos
and upload them to the Internet.
Hologram Zoo
The previously mentioned Synthesia company was affected by
controversy after several of its avatars were used by an alleged
news network (Wolf News) to spread false information.

If the professional audiovisual industry is benefiting from the

improvements in efficiency and detail of virtual creations, so are
those who use them to pervert and manipulate reality. We’re seeing
a great increase in the number of cases of image misappropriation
of popular characters to use them, for example, in pornographic
videos or for manipulation driven by political agendas.
 Synthetic hyperrealism A I B E H I N D T H E S C R E E N 48

Index

Researchers from large companies and research centres are wor- _ SPACES AND DIGITAL TWINS
king to refine the tools to detect manipulation of texts, audios and
What is also increasing is the level of synthetic realism in the creation
videos. In Spain, RTVE, the Universitat Politècnica de Catalunya,
of virtual spaces. Hyper-realistic replicas are increasingly detailed
the Universitat Carlos III (Madrid), the University of Granada and
and allow for near-perfect simulation.
the Universitat Autònoma de Barcelona are developing the IVERES
project to advance the detection of multilingual disinformation. Digital twins are starting to become common in architectural and
urban planning or engineering.
Large technology companies have tools that analyse signs of
image manipulation, difficult to identify at a glance but which can NVIDIA_Omniverse is a reference tool for the generation and
be traced with the help of computer vision systems. There are also management of digital twins in professional areas. BMW, Siemens,
specialized independent organizations such as First Draft, Witness Ericsson and Mercedes Benz use it to virtually monitor their
and the Center for Information Technology Policy at Princeton production plants. Digital replicas can be updated in real time
University that have detection technology and processes. MIT based on dynamic data.
researchers have developed a tool that prevents falsifications:
Photoguard. Industrial Metaverse. The virtual replica of machines, factories,
cities, transport networks and other highly complex systems is
gaining momentum and is one of the growth vectors of immersive
and virtual reality simulation technologies.

THE DATA

By 2030, the industrial metaverse will be

driving a global business figure that will
reach 100 trillion dollars
_ Source: ABI Research
 Synthetic hyperrealism A I B E H I N D T H E S C R E E N 49

Index

Images generated with Midjourney:

versions V5.2 (left) and V6 (right)

_ GENERATIVE HYPERREALISM

Generative AI is evolving rapidly. From the first imperfect sketches Midjourney has led a spectacular leap in image generation: detailed
and portraits offered by DALL-E in early 2022, it has moved on to faces, photo-level lighting and transparencies and well-resolved
high-quality realistic images that can be created with just a few text hands, one of the outstanding challenges of generative AI. Users
prompts. can now enlarge images to resolutions of more than 4,000 pixels.

The styles can be very diverse, from fantastic representations to the New startup Magnific offers a tool to upscale and enhance blurry
most realistic ones. The features in the new versions of the image or low-resolution images and convert them to high quality.
generator can reach a truly amazing level of photo emulation.
 A I B E H I N D T H E S C R E E N 50

Index

Artificial Intelligence

07 Audio and voice Apple already has a catalogue of audiobooks narrated by AI-
generated voices. It moved on to Audible, Amazon's audiobook
company.
The fascination caused by generative artificial intelligence in regards
to text and images is tarnishing the importance of innovations in the Audiobox from Meta can generate voices and sound effects using
field of audio in general and voice generation in particular. a combination of voice inputs and natural language text prompts.

What progress is being made?

_ CHANGING THE VOICE

_ SYNTHETIC VOICE GENERATION If you don't like your voice or want to try a different one, change it.
It can now be done in real time.
Automatic audio generation allows an artificial character to recreate
in real time a particular voice with the typical speech intonations, Voicemod is a company from Valencia that develops a technology
style and flaws. which allows you to change your voice in real time. It is used for
free by users on platforms such as Roblox, VRChat, Discord, etc.
The quality gap between synthetic voice and human voice is getting
as a complement to their digital identity. At the beginning of 2023,
smaller. It is increasingly difficult to distinguish them. There is still
a hurdle to overcome, although a lot of work is being done in that
regard: expressing emotions.
Singer Holly Heldon shows how she
ASome companies and research laboratories have made can transfer her voice with Voctrolabs
spectacular progress. For example, the hyper realistic voices technology
of Sonantic (at the end of 2022 it was bought by Spotify). _ Voicemod

There’s also Resemble.ai, Murf.ai, Synthesys.io and Lovo.ai,

among others.
 Audio and voice A I B E H I N D T H E S C R E E N 51

Index

the company announced the acquisition of Voctrolabs, a spin-off James Earl Jones, the actor who voiced the character of Darth
from the Universitat Pompeu Fabra de Barcelona specializing in Vader in the Star Wars films for almost 50 years, has given away
music technology. the rights to clone his voice for future Lucasfilm projects. The saga
can continue.
So-vits-svc is an open-source software that allows you to train
a neural network with a singer's voice and then make it sing any Warner plans to produce a biopic of French singer Édith Piaf by
music. In early 2023 the song Heart on my Sleeve that appeared recreating her voice and image with AI. They have the consent of
to be performed by Drake and the Weeknd was generated with this the agent who controls the rights of the artist who died in 1963.
software. James Stewart's voice has also been regenerated for a collection
of audios based on the film It's a Wonderful Life.

_ CLONING In Top Gun: Maverick one of the challenges was to recover the
voice of Val Kilmer who in the original films played the role of Tom
Voice cloning is also becoming increasingly accurate. This feature
"Iceman" Kazansky. Val Kilmer suffers from cancer which has left
is highly appreciated by producers and actors who want to preserve
him without a voice. Job accomplished.
their voices.
ElevenLabs is one of the most popular cloning and voice generator
tools. At the end of 2023, it introduced the AI dubbing tool that
allows users to automatically translate any speech into a different
language while maintaining the speaker’s original voice.

In this area, it’s worth following Rask.ai and PlayHT too.

The North American actors' association SAG-AFTRA has signed

an agreement with Replica Studios for union protections, terms
Darth Vader will be able to continue
with the voice of James Earl Jones and conditions for the use of digital voice replication in the video
game industry.
 Audio and voice A I B E H I N D T H E S C R E E N 52

Index

_ INVISIBLE DUBBING

The technologies that enable automatic dubbing are also advancing

“Soon dubbing will be imperceptible.
rapidly. The audience will not know in which original
language the film was shot”
They adapt the lip movement of the actors to the text with exact
precision. With techniques similar to those used to create deep
fakes, they have managed to make Robert De Niro speak impeccable Scott Mann,
German; Tom Hanks, Spanish, and Tom Cruise and Jack Nicholson, director of Flawless

French.

Some British companies stand out: Other similar companies are the Israeli Deepdub and the
Ukrainian Respeecher. If consolidated, automatic dubbing may
Flawless, the company behind many of the automatic dubbing of
end up taking over subtitling, which is common in markets
Hollywood figures, has patented a technology called TrueSync,
where international productions are not usually dubbed. In Latin
which, based on a neural network system, generates almost
America, dubbing done with AI is commonplace. Voice actors are
perfect synchronization.
worried, and rightly so.
Papercup is now growing with the help of Sky, which in 2022
The latest version of Meta's Seamless model allows you to modify
contributed $20 million to expand its automatic dubbing system.
the tone and emotional expressiveness of automated dubbing.
Many of Sky and Discovery's productions for the Latin American
market are now automatically dubbed with this tool, including live
Sky News broadcasts.
 Audio and voice A I B E H I N D T H E S C R E E N 53

Index

Audios generated from

_ TEXT TO AUDIO GENERATION text with AudioGen
From the realm of generators of anything using artificial intelligen-
ce, there is no shortage of text-to-audio generators, either to create
synthetic voices or as a generator of sound effects.

Text to speech techniques have been perfected for many years but _ AUTOMATIC TRANSCRIPTION
have always encountered the challenge of recreating the nuances, Transcribing interviews, conversations or podcast audio is one of the
intonations and imperfections of human speech. most tiresome jobs in journalism and in the production of reports and
New systems based on artificial intelligence are managing to over- documentaries.
come these difficulties and go one step further: generate sound Whisper is a tool from the Open AI factory that allows you to
effects, phrases or music from short text instructions. For exam- transcribe and translate audios in 98 languages with very accurate
ple AudioLM, from the Google research laboratory and AudioLDM, precision.
created by researchers from several British universities. Meta has
brought together several audio and music generation models into Nuance Dragon is one of the most highly rated speech-to-text
a tool called AudioCraft. tools. Other options are Trint, TranscribeKit, Moises.ai and AIrite.

AudioShake integrates several AI solutions for audio generation DoblAI, from the Catalan company Ugiat, is one of the most
and editing. Time magazine considered it one of the best innova- complete multilingual tools for transcribing, translating and
tions of 2023. dubbing videos.
 A I B E H I N D T H E S C R E E N 54

Index

Artificial Intelligence

08 Data everywhere
The trail that users voluntarily or involuntarily leave offers an Largo.ai is a Swiss data management company specialized
opportunity to fine-tune the business for the whole audiovisual value in cinema. It helps producers and distributors fine-tune the
chain; from creation to content distribution and advertising. commercial success of a film and it does so by analysing the
correlation between market data and the type of production,
Managed by artificial intelligence systems, the data allows intervention content, script structure, casting, etc.
in two areas:
Cinelytic is a similar North American company. It was launched
Establishing patterns around the taste and habits of viewers/users
in 2018 and claims to get near 85% accuracy on the future
and predict their behaviour.
performance of a film's box office or a series' audience from
Personalizing content offer. script analysis, casting and production characteristics. In 2022
it bought RightsTrade, a marketplace that connects sellers and
_ PREDICTING SUCCESS buyers of audiovisual production rights. With the addition of
technologies, the prediction of success is not only limited to
Is it possible to predict the success of a movie, series or TV show? guiding production but also to quantifying the expected audience
and suggesting the right price when it comes to selling rights
Not quite, but we're getting close. The secret is to have enough data
and formats. Vault AI also stands out in this area.
and know how to cross-reference them to define behavioural patterns.
Success is not guaranteed but the margin of error is reduced. StoryFit and ScriptBook, also North American, have specialized
in script analysis using artificial intelligence technology. Avail
is a tool that summarizes scripts to help producers who receive
hundreds of proposals.
 Data ever y where A I B E H I N D T H E S C R E E N 55

Index

The Film Financing Market held as part of both the Sitges Parrot Analytics claims it can measure the demand for certain
International Film Festival and the Málaga Film Festival uses the content in any territory and on any platform. Its products are
business intelligence technology of Inorbis Analytics to select aimed at global producers who have to customize the distribution
and present the projects with the best expected commercial of their content. They claim to capture behavioural data from
performance to investors. 2 billion people in over 100 markets.

_ PERSONALIZING CONTENT _ PERSONALIZING ADVERTISING

Bringing the right content to each viewer is the dream goal of any The massive gathering of user behaviour data allows algorithmic
platform. Some algorithms that perform this function, such as those treatment methods to be applied to the advertising offer based on
of Netflix and TikTok, are well known. hyper-segmentation of audiences, personalization of messages and
automation of insertions.
The market is also starting to see companies that offer tools to analyse
what the appropriate channels for the distribution of certain content Konodrac is a Catalan company specialized in the management
are, their value in each market and its ability to attract advertising of online audience data for television channels and OTT platforms.
revenue. They individualize user consumption data, segment audiences
based on real behavioural patterns, customize channels' adver-
tising offerings and automate campaign planning.
Largo AI analysis panel
Once the audience is segmented, the work of producing personalized
messages multiplies and it can become unachievable unless
generative artificial intelligence tools are used. With text there
are enough tools to do it, but in video the quality generators still
need to be consolidated.
 Data ever y where A I B E H I N D T H E S C R E E N 56

Index

_ FROM DATA TO CONTEXT

_ EFFICIENCY OR INVASION OF PRIVACY?
Another growing trend is context-based advertising communication.
Viewer monitoring and tracking techniques are not without Contextual advertising does not need viewer data. It is based on
controversy due to the risk of invading privacy and violating content analysis without requiring user data.
the protection of personal data.
Computer vision technologies applied to contextual segmentation
Proponents of predictive advertising defend the model. They make it possible to analyse the content of videos and decide which
argue that they are not interested in knowing the behaviour ads are best to insert and at what times.
of a particular person, but in setting consumption patterns
The QUVA Lab from the multinational Qualcomm and the
outside of each viewer’s particular identity. They claim to
University of Amsterdam segment YouTube videos based on
be able to put an end to the indiscriminate bombardment of
image analysis at pixel level. In the United States, Netra offers
advertising messages so advertising ceases to annoy and
several types of services based on contextual video segmentation.
becomes interesting. Other voices allege that this is another
step in the invasion of privacy. Techsoulogy, a technology startup applied to advertising,
offers the Contextualize-it solution, capable of analysing the
Is it possible to personalize content ethically? The BBC is
images and audio from videos, identifying objects, places and
trying it. Its research and development laboratory has a
dialogues, distinguishing the different scenes in the video and
content personalization system that does not track user
defining the best moments to insert the most appropriate ad
data. It uses the Solid model created by web founder
based on the context.
Tim Berners-Lee. This model allows users to know what
data has been collected, decide which data they want to YouTube offers a package for advertisers called Spotlight
hand over to the BBC and have the ability to delete them Moments. Using AI, it automatically identifies the most popular
at any time. YouTube videos related to a specific cultural moment, such as
Halloween or a sporting event, and the advertiser can place ads on
videos related to the topic or event.
 Data ever y where A I B E H I N D T H E S C R E E N 57

Index

_ VIDEO AS A DATA SOURCE

Computer vision offers the ability to turn video into a data source. It is The Computer Vision Center, located on the campus of the
no longer just the tracking and segmentation of viewer behaviour that Autonomous University of Barcelona, is one of the most advanced
provides data, but also the content. European research centres in image analysis.

Camaleonic Analytics is a Catalan company created in 2021 Ugiat is a company born out of the Universitat Politècnica de
that analyses in real time all the advertising impacts of sports Catalunya -UPC, a benchmark in video, audio and text analysis for
event sponsorships using automatic identification and labelling media and audiovisual companies..
techniques. It can also detect unused placements and assess the
impacts that could be generated.
_ DATA AS A SOURCE OF INFORMATION

The increasingly large amount of data available to the media makes

it necessary to use artificial intelligence systems to manage large
amounts of data, verify and view them. Automated management of
large volumes of information opens the door to data-driven journalism
and more effective detection of misinformation.

The International Consortium of Journalists used machine-

learning tools to uncover the thousands of injuries caused by
faulty implants and prostheses every year. Data from 340,000
people was analysed for detection.

Real-time analysis of the impact of sponsorships

Image: Camaleonic Analytics
 Data ever y where A I B E H I N D T H E S C R E E N 58

Index

_ CONTENT ANALYSIS Companies such as North American Affectiva claim to be able to

capture the user's subconscious intuitive reactions and determine
Another possibility offered by AI is the analysis of films or long
their disposition towards an advertisement or the purchase of a
audiovisual products in detail to extract and compare information.
product.
A few months ago The Washington Post published a graphic
Cogito is a company created by alumni of Boston's MIT Media
analysis of the amount of scare jumps in horror films. Conclusion:
Lab that helps call centre agents identify the mood of callers and
Movies include more and more scares, and remakes of genre
adjust how they direct the conversation in real time.
classics add more jump scares.
Bristol will have a cinema this year that will analyse audience
reactions. The 36-seat Instrumented Auditorium will record
_ ANALYSIS OF EMOTIONS
viewers' biometric responses, including their heart rate, eye
One of the trends in artificial intelligence systems and, specifically, movement, brain activity and skin reactions.
biometric analysis through facial or voice recognition, is to detect
Emogg is a Catalan startup that provides streamers and media
people's emotions and feelings.
with an application to share emotions with the audience, live
and visually. imotion Analytics offers several solutions based on
artificial vision and behaviour and emotion analysis.

In the audiovisual field and in video games, emotional AI aims to

analyse whether the contents arouse the expected reactions in the
viewers and to select the advertising offers that are most suitable
depending on the users’ mood.
 Data ever y where A I B E H I N D T H E S C R E E N 59

Index

_ BIOMETRIC’S

When emotional detection aims to be done through the biometric analysis of users,
things get complicated.

From a scientific point of view, whether that is possible is called into question.
The UK Information Commissioner warned a few months ago that there is no
scientific evidence that the emotional analysis of biometric systems shows any
accuracy and credibility and warned that they would pursue companies that
market such services.

From an ethical point of view, the analysis of biometric data raises a lot of
questions. The new European law (AI Act) prohibits some uses such as:
biometric categorization (by political, religious, philosophical beliefs or by
sexual orientation or race); systems to expand or create facial databases by
capturing data indiscriminately through the internet or audiovisual recordings;
the identification of emotions in the workplace and in educational institutions;
social scoring (systems that score people based on their social behaviour or
personal characteristics); systems that manipulate human behaviour, and AI
used to exploit people's vulnerabilities (for example, due to their age or social or
economic situation).

Image Ars Electronica

Source: Visualhunt
 Data ever y where A I B E H I N D T H E S C R E E N 60

Index

TRENDS

_ Predicting success. Considering that artificial intelligence is _ Personalizing everything. So much data can be collected that
a good tool for finding patterns, it can be useful to guide film with good management and accurate algorithms, it’s possible
and media producers when incorporating and analysing in more to fulfil the aspiration of media and brands to offer the content
detail some of the elements of a production that contribute to that each viewer/user wants at the most appropriate time.
its success (casting, script, effects...). AI is not used, however, to The trend towards personalization is growing. Along with the
produce surprises or make ground-breaking proposals. statistical data left by each user, there is also the possibility of
personalizing the contents based on the narrative context of the
_ Data for creation. The efficient management of large audiovisual product.
volumes of data becomes a determining element not only for
management but also for creation. From the preparation of _ Preserving privacy. Finding the balance between the exhaustive
journalistic information to the viewing of data in new graphic and analysis of user data and the preservation of privacy is an
audiovisual formats.
unsolved challenge that must be faced by administrations and
industry. Now in addition there’s the challenge raised by the
analysis of emotions and the use of biometric technologies such
as facial recognition.
 A I B E H I N D T H E S C R E E N 61

Index

Artificial Intelligence

More
_ Futurepedia. https://www.futurepedia.io

_ AI tools. https://aitoolreport.com/

_ Runway Akademy. https://academy.runwayml.com/

_ Generative AI Market Map: From History and State to Trends and Applications [With Infographic] https://pixelplex.io/blog/generative-ai-market-map/

_ Journalism, Media and Technology Trends and Predictions 2024. Reuters Institute. Download

_ AI use cases in the film industry. https://dougshapiro.medium.com/ai-use-cases-in-hollywood-362707e899f1

_ EmotionAI, explained. https://mitsloan.mit.edu/ideas-made-to-matter/emotion-ai-explained

_ Hollywood agreements:
· Writers. https://www.wgacontract2023.org/the-campaign/summary-of-the-2023-wga-mba
· Actors. https://www.sagaftra.org/files/sa_documents/AI%20TVTH.pdf

_ Impact of AI on the media. Study by the European Broadcasting Union (EBU)

https://www.ebu.ch/news/2023/10/navigating-the-digital-frontier--the-impact-of-ai-on-media-literacy

_ AI will not kill Hollywood. https://www.runningtowards.xyz/p/ai-will-not-kill-hollywood

_ Data: analyzing scary movies. https://www.washingtonpost.com/business/interactive/2023/jump-scare-horror-movies/

_ Courses:
· https://curiousrefuge.com/ai-filmmaking
· https://www.filmschool.ai/
 A I B E H I N D T H E S C R E E N 62

CREDITS

The Switch Observatory is a project from the Written and edited by Joan Rosés
Catalan Audiovisual Cluster. Design and layout: Marta Aguiló
Translation: Maria Gasol
Visit:
www.clusteraudiovisual.cat/en/switch-observatory
Documentation: Collateral Bits

Production, Coordination, Communication (Catalan Audiovisual Cluster):

Eduard Gil - Cluster Manager
Helena Alabart - Program Manager
Sílvia González - Project Manager and Switch Report Coordinator
Ester Villacampa - Project Manager
Núria Grinyó - Communication

Attribution-NonCommercial-NoDerivs 4.0 International

CC BY-NC-ND 4.0 Deed
Supported by: