Understanding Module Federation: A Deep Dive

13 min readAug 29, 2023

Introduction

Module Federation is an advanced feature in Webpack (and soon Rspack) that provides a way for a JavaScript application to dynamically load code from another application. This feature allows for efficient code sharing and dependency management. In this article, we will explore the architecture, prerequisites, and underlying implementation of Module Federation.

Federation is powerful, but the lack of meta-framework support leads to challenges. ByteDance is one of the largest users of Module Federation, ModernJS is their Meta-framework, if module federation is a key part of your engineering operational improvements. ModernJS is the only way to go

The Future of Module Federation

scriptedalchemy.medium.com

Architecture Blocks
Prerequisites
Factory Object
Dependency Object
Factory Object Resolution
Example Context
Product Structure
Execution Flow
Source Code Analysis

This content was originally written by @2hea1

Architecture Blocks

Module Federation comprises three main components:

Exposed Module (Remote)
Consumption Module (Host Remote Import)
Shared Module/Dependency

// Exposed Module (Producer)
export const exposedFunction = () => {
    console.log("I am an exposed function");
};
// Consumption Module
import { exposedFunction } from 'exposedModule';
exposedFunction();

// Shared Module/Dependency
// shared.js
export const sharedFunction = () => {
    console.log("I am a shared function");
};

The sections that follow will first present the overall operational flow and then delve into the specific code implementations within each module.

Prerequisites

Before diving deeper, it’s essential to understand the following Webpack fundamentals:

Webpack Product Execution Flow
Immediate-Invoked Function Expression (IIFE) in Webpack
Modules are stored in __webpack_modules__
Webpack uses a global array (webpackChunk) to cache loaded resources

Webpack exposes a webpackChunk array to the global (globalObject). This array is used to store loaded chunk resources. If the resource is loaded, the cache will be read, and if it is not loaded, the content will be synchronously added to the __webpack_modules__ by calling the overridden push method (aka webpackJsonpCallback ) for module integration.

Factory Object

During the Webpack compilation process, code is initially written into a custom Webpack module. Then multiple modules are aggregated into a chunk based on their file reference relationships.

// Webpack custom module
class CustomModule {
    constructor(code) {
        this.code = code;
    }
}

Dependency Object

A Dependency object is essentially an unresolved module instance. For example, the entry module ('./src/index.js') or other modules that a module relies upon are transformed into Dependency objects.

// Dependency Object
class Dependency {
    constructor(module) {
        this.module = module;
    }
}

Factory Object Resolution

Each derived class of Dependency pre-determines its corresponding factory object, and this information is stored in the dependencyFactories property of the Compilation object.

class EntryPlugin {
    apply(compiler) {
        compiler.hooks.compilation.tap(
            "EntryPlugin",
            (compilation, { normalModuleFactory }) => {
                compilation.dependencyFactories.set(
                    EntryDependency,
                    normalModuleFactory
                );
            }
        );

        const { entry, options, context } = this;
        const dep = EntryPlugin.createDependency(entry, options);

        compiler.hooks.make.tapAsync("EntryPlugin", (compilation, callback) => {
            compilation.addEntry(context, dep, options, err => {
                callback(err);
            });
        });
    }
}
class Compilation extends Tapable {
    addModuleTree({ context, dependency, contextInfo }, callback) {
        // ...
        const Dep = /** @type {DepConstructor} */ (dependency.constructor);
        const moduleFactory = this.dependencyFactories.get(Dep);
        if (!moduleFactory) {
            return callback(
                new WebpackError(
                    `No dependency factory available for this dependency type: ${dependency.constructor.name}`
                )
            );
        }
        this.handleModuleCreation(
            {
                factory: moduleFactory,
                dependencies: [dependency],
                originModule: null,
                contextInfo,
                context
            },
            (err, result) => {
                if (err && this.bail) {
                    callback(err);
                    this.buildQueue.stop();
                    this.rebuildQueue.stop();
                    this.processDependenciesQueue.stop();
                    this.factorizeQueue.stop();
                } else if (!err && result) {
                    callback(null, result);
                } else {
                    callback();
                }
            }
        );
    }
}

Example Context

Consider two projects: App1 and App2.

App1 exposes a Button component and sets a shared React dependency:

new ModuleFederationPlugin({
    name: 'component_app',
    filename: 'remoteEntry.js',
    exposes: {
        '.': './src/Button.jsx',
    },
    shared: {
        'react': {
            version: '2.3.2'
        }
    }
})

App2 consumes the components and libraries provided by App1:

new ModuleFederationPlugin({
    library: { type: 'module' },
    remotes: {
        'component-app': 'component_app@http://localhost:3001/remoteEntry.js',
    },
    shared: ['react'],
})

Build Flow

https://miro.medium.com/v2/resize:fit:13190/1*Ke2k8KfVr7Id_nXyEqpx_w.png

Product Structure

Two types of products are generated after the build process:

Entry file
Module chunk

The entry file exposes get and init methods along with a moduleMap.

var moduleMap = {
    "./src/Button.jsx": () => {
        return __webpack_require__.e(507).then(() => (() => ((__webpack_require__(507)))));
    }
};
var get = (module, getScope) => {
    __webpack_require__.R = getScope;
    getScope = (
        __webpack_require__.o(moduleMap, module)
            ? moduleMap[module]()
            : Promise.resolve().then(() => {
                throw new Error('Module "' + module + '" does not exist in container.');
            })
    );
    __webpack_require__.R = undefined;
    return getScope;
};
var init = (shareScope, initScope) => {
    if (!__webpack_require__.S) return;
    var oldScope = __webpack_require__.S["default"];
    var name = "default"
    if(oldScope && oldScope !== shareScope) throw new Error("Container initialization failed as it has already been initialized with a different share scope");
    __webpack_require__.S[name] = shareScope;
    return __webpack_require__.I(name, initScope);
};

__webpack_require__.d(exports, {
    get: () => (get),
    init: () => (init)
});

Execution Flow

The execution sequence involves the following steps:

Load shared dependencies (import react from 'react')
Load the entry file asynchronously
Consume remote modules (import Button from 'component-app')

Reference shared dependencies ( `import react from 'react '`)

Load the entry file, since the entry is processed asynchronously at this time, you can see that __webpack_require__ will be executed (ensureChunk: load asynchronous chunk content)

__webpack_require__ will traverse ensureChunkHandlers ( __webpack_require__ .f.xxxx ), which contains __webpack_require__ .f.consumes

Here you can see the react configured in shared , this time will traverse chunkMapping ['bootstrap_js - webpack_sharing_consume_default_react_react'], and get the corresponding loadSingletonVersionCheckFallback callback function through moduleToHandlerMapping mapping table, the execution returns a promise, where resolve returns a factory function.
loadSingletonVersionCheckFallback will call __webpack_require. I ( RuntimeGlobals.initializeSharing ) before execution to initialize the sharedScope of the current project.

When __webpack_require__. I is executed, the shared dependencies of the current project ( react ) are registered, and then the init method exposed by the remote module's entry file (if it exists) is called

Execute the init method exposed by app1 , which will pass in the sharedScope of app2 , synchronizing the shared dependency information set by app2 to app1

If the shared dependency versions are different, there will be multiple version information. The shareScope information is as follows

Similarly, calling __webpack_require__. I in app1 will also take the same method as in app2 . However, unlike app1 , there is no remotes field, so there is no initExternal method. Here is an explanation of the register function used to initialize shareScope .

var register = (name, version, factory, eager) => {
 // Get all versions of react that have been registered
  var versions = (scope[name] = scope[name] || {});

  // Find out if 17.0.1 has been initialized
  var activeVersion = versions[version];

 // If one of the following conditions is met (i.e. the if statement below is true),
   // Mount app1's shared dependencies (such as react), otherwise reuse app2's shared dependencies (such as react):

   // 1. There is no corresponding module version in app2, that is, activeVersion is empty
   // 2. The old version is not loaded, and eager is not configured (forced loading)
   // 3. The old version is not loaded, and uniqueName of app1 > uniqueName of source module, see #12124 for details
   //(uniqueName is actually the main field of packagejson)

  if (
    !activeVersion ||
    (!activeVersion.loaded &&
      (!eager != !activeVersion.eager
        ? eager
 : uniqueName > activeVersion.from))
  )
    versions[version] = {
      get: factory,
      from: uniqueName,
      eager: !!eager,
    };
};

Back to app2, the __webpack_require__. Iexecution is complete, and the getSingletonVersion method is executed. This method is mainly used to obtain dependent versions that meet the requirements.

After getting the version, execute the get method to get the shared dependencies.

Reference remote module ( `import Button from 'component-app '` )

In app1, there is also a get method exposed. When referencing the corresponding component, the basic process and reference sharing dependency are consistent, the difference is that the final initialization will call app1's getmethod

Source Code Analysis

The underlying source code can be divided into three main parts:

Expose Module: Exposes specific modules for other projects
Consumption Module: Consumes remote modules
Shared Dependency: Shares dependencies across modules

Exposing Modules: The Role of `ContainerPlugin`

The Expose module is principally concerned with making specific modules in a project available for use by other projects. The primary code for this functionality resides in webpack/lib/container/ContainerPlugin.

Why is it Named `ContainerPlugin`?

The name ContainerPlugin is derived from its function. Module Federation itself establishes a container and furnishes the modules within this created container. These containers can then be connected across different projects, enabling the modules exposed from one project to be utilized in another.

Core Functions of `ContainerPlugin`

ContainerPlugin performs two key operations:

Adding Specified Module to Compile Entry: It utilizes compilation.addEntry to add the designated module that needs to be exposed.
Setting the Factory Object: It sets the factory object using compilation.dependencyFactories.set.

Expose Module Workflow

The overall process of exposing a module comprises the following steps:

Add Entry File (remoteEntry): An entry file named remoteEntry is added to the project.
Set Exposed Module to Async Chunk: The exposed module is set as an asynchronous chunk, allowing for dynamic imports.
Inject Runtime Code: Additional runtime code is appended to the entry file, typically including methods like get and init that manage the module's behavior.

Add entry file

compiler.hooks.make.tapAsync(PLUGIN_NAME, (compilation, callback) => {
    const dep = new ContainerEntryDependency(name, exposes, shareScope);
    dep.loc = { name };
    compilation.addEntry(
        compilation.options.context,
        dep,
        {
            name,
            filename,
            runtime,
            library
        },
        error => {
            if (error) return callback(error);
            callback();
        }
    );
});

Set Factory Object

compiler.hooks.thisCompilation.tap(
    PLUGIN_NAME,
    (compilation, { normalModuleFactory }) => {
        compilation.dependencyFactories.set(
            ContainerEntryDependency,
            new ContainerEntryModuleFactory()
        );

        compilation.dependencyFactories.set(
            ContainerExposedDependency,
            normalModuleFactory
        );
    }
);

Setting up the expose module

In the Factory Object you can see that the ContainerEntryDependency factory object is ContainerEntryModuleFactory .

ContainerEntryModuleFactory will provide a create method. This will get the ContainerEntryDependency instance ( dependency ) added in the previous step, which contains the name , exposes , and shareScope information set by the user.

Then create ContainerEntryModule .

When building the ContainerEntryModule , you can see that the modules exposed by exposes are added to the block as asynchronous code. And the ContainerExposedDependency will be created based on the actual file path, which explains why the ContainerExposedDependency was added in the previous addEntry step. After adding dependency and block , the module build process will be called recursion.

Add specific runtime code to the entry file

CodeGeneration in the ContainerEntryModule will get the module information specified after compile and generate the final exposed module entry:

codeGeneration({ moduleGraph, chunkGraph, runtimeTemplate }) {
    const source = Template.asString([
        `var moduleMap = {`,
        Template.indent(getters.join(",\n")),
        "};",
        `var get = ${runtimeTemplate.basicFunction("module, getScope", [
            `${RuntimeGlobals.currentRemoteGetScope} = getScope;`,
            // reusing the getScope variable to avoid creating a new var (and module is also used later)
            "getScope = (",
            Template.indent([
                `${RuntimeGlobals.hasOwnProperty}(moduleMap, module)`,
                Template.indent([
                    "? moduleMap[module]()",
                    `: Promise.resolve().then(${runtimeTemplate.basicFunction(
                        "",
                        "throw new Error('Module \"' + module + '\" does not exist in container.');"
                    )})`
                ])
            ]),
            ");",
            `${RuntimeGlobals.currentRemoteGetScope} = undefined;`,
            "return getScope;"
        ])};`,
        `var init = ${runtimeTemplate.basicFunction("shareScope, initScope", [
            `if (!${RuntimeGlobals.shareScopeMap}) return;`,
            `var oldScope = ${RuntimeGlobals.shareScopeMap}[${JSON.stringify(
                this._shareScope
            )}];`,
            `var name = ${JSON.stringify(this._shareScope)}`,
            `if(oldScope && oldScope !== shareScope) throw new Error("Container initialization failed as it has already been initialized with a different share scope");`,
            `${RuntimeGlobals.shareScopeMap}[name] = shareScope;`,
            `return ${RuntimeGlobals.initializeSharing}(name, initScope);`
        ])};`,
        "",
        "// This exports getters to disallow modifications",
        `${RuntimeGlobals.definePropertyGetters}(exports, {`,
        Template.indent([
            `get: ${runtimeTemplate.returningFunction("get")},`,
            `init: ${runtimeTemplate.returningFunction("init")}`
        ]),
        "});"
    ]);
}

Summarized Flow:

RemoteModule Consumption

The ContainerReference module is mainly used to consume remote modules.

Its main implementation is in webpack/lib/container/ContainerReferencePlugin .

ContainerReferencePlugin It is simple and understandable to do four pieces of content:

Add remote module to external
Set the factory object (compilation.dependencyFactories.set)
Intercepts request parsing for remote modules ( normalModuleFactory.hooks.factorize ) and returns to generate RemoteModule
Add a runtime module RemoteRuntimeModule

Add remote module to external

After parsing the passed parameters, the remote module is added to external:

This also explains why the remotes parameter and the externals parameter feel so similar.

Set Factory Object

Then add the corresponding factory object for the module to be used later.

Note: fallbackis set here, this module will only trigger if multiple external settings are set

Intercept request resolution for remote modules normalModuleFactory.hooks.factorize

Next, block the module request, which returns a custom RemoteModule:

The RemoteModuleis mainly used to collect the corresponding request dependencies and collect the remote modules that need to be initialized and their chunkIDs, and place the results in the codeGenerationResultsfor use when initializing shared dependencies (if you set shared dependencies, you need to initialize shared dependencies first, and then initialize remote modules).

Add the `RemoteRuntimeModule`

This module will collect the compiled chunkid of all remote modules and place it in chunkMaping. Set the RuntimeGlobals.ensureChunkHandlers method accordingly. This method will be called when referencing an asynchronous chunk, and when referencing a remote module, the corresponding get method will be called to obtain the corresponding remote module.

Corresponding code address: RemoteRuntimeModule

Shared Dependency

Shared dependencies are mainly used to share the same dependency across modules.

Its main implementation is in webpack/lib/sharing/SharePlugin

Shared dependencies are divided into two parts: consuming shared dependencies ( ConsumeSharedPlugin) and providing shared dependencies ( ProvideSharedPlugin).

And SharePluginonly does parameter parsing and applies these two plugins. Therefore, it will split the two parts for parsing.

Consumption Sharing Dependency

ConsumeSharedPluginsimply understands four things:

Set Factory Object
Intercepts request resolution for shared dependencies ( normalModuleFactory.hooks.factorize ) and returns a custom ConsumeSharedModule
Intercepting Shared Dependency Request Parsing for Absolute Paths ( normalModuleFactory.hooks.createModule )
Add runtime module

Set Factory Object

Set the module factory object to be used later

Intercept request resolution for shared dependencies normalModuleFactory.hooks.factorize

Mainly intercepts module requests (import react from react ) and suffix requests (import react from 'react/’).

After interception, it will be resolved according to the locally installed dependencies/modules, and recorded to the resolveContext, createConsumeSharedModule return a custom ConsumeSharedModule

The ConsumeSharedModulecollects the shared dependencies and their chunkIDs and places the results in sourcesfor consumption by the shared dependencies #runtime module ConsumeSharedRuntimeModule .

Press enter or click to view image in full size

ConsumeSharedModulealso adds shared dependencies to the AsyncDependenciesBlockof asynchronous modules

Intercepting Shared Dependency Request Resolution for Absolute Paths normalModuleFactory.hooks.createModule

Intercept requests with absolute paths and return a custom ConsumeSharedModule

Add runtime module ConsumeSharedRuntimeModule

The module consumes the data provided by the ConsumeSharedModuleand generates the initialization shared dependency runtime code:

After setting, the final production code is as follows:

Asynchronous entry:

When referencing a remote module, shared dependencies are registered first, and then the remote module is loaded.

Sync entry:

After loading the shared dependency, hang it on the shareScope for use by other modules

Provide shared dependencies

The ProvideSharedPlugin does three things:

Intercept request resolution of shared dependencies ( normalModuleFactory.hooks.module ) to collect information about all shared dependencies
Add Include
Set Factory Object

Intercept request resolution for shared dependencies normalModuleFactory.hooks.module

Intercept the request resolution of shared dependencies and collect all information.

Add Include

Add Include to finishMake

The whole is very similar to addEntry, the difference may be that this module has no other dependencies?

Note: This hook does not appear in the official doc, and is only used by the module federation function

Set Factory Object

Set the module factory object to be used later

Where ProvideSharedDependencycreates a ProvideSharedModulemodule.

The ProvideSharedModulewill sharedinformation and set the register function based on intercepting requests for shared dependencies. This data will be used when initializing shared dependencies.

Press enter or click to view image in full size

Congrats You Made it!

And thats how Module Federation works.

If you like deep dives:

Bundler Design Trade-offs: The Rationale Behind Creating Rspack

Before embarking on the development of Rspack, we explored various build tools and frameworks, including extensive use…

scriptedalchemy.medium.com

Understanding Module Federation: A Deep Dive

Introduction

The Future of Module Federation

Table of Contents

Architecture Blocks

Prerequisites

Factory Object

Dependency Object

Factory Object Resolution

Example Context

Build Flow

Product Structure

Execution Flow

Reference shared dependencies ( `import react from 'react '`)

Reference remote module ( `import Button from 'component-app '` )

Source Code Analysis

Exposing Modules: The Role of `ContainerPlugin`

Why is it Named `ContainerPlugin`?

Core Functions of `ContainerPlugin`

Expose Module Workflow

Setting up the expose module

Add specific runtime code to the entry file

RemoteModule Consumption

Add remote module to external

Set Factory Object

Intercept request resolution for remote modules normalModuleFactory.hooks.factorize

Add the `RemoteRuntimeModule`

Shared Dependency

Consumption Sharing Dependency

Set Factory Object

Asynchronous entry:

Sync entry:

Provide shared dependencies

Add Include

Set Factory Object

Congrats You Made it!

Bundler Design Trade-offs: The Rationale Behind Creating Rspack

Before embarking on the development of Rspack, we explored various build tools and frameworks, including extensive use…

Written by Zack Jackson

Responses (4)

Understanding Module Federation: A Deep Dive

Introduction

The Future of Module Federation

Table of Contents

Architecture Blocks

Prerequisites

Factory Object

Dependency Object

Factory Object Resolution

Example Context

Build Flow

Product Structure

Execution Flow

Reference shared dependencies ( import react from 'react ')

Reference remote module ( import Button from 'component-app ' )

Source Code Analysis

Exposing Modules: The Role of ContainerPlugin

Why is it Named ContainerPlugin?

Core Functions of ContainerPlugin

Expose Module Workflow

Setting up the expose module

Add specific runtime code to the entry file

RemoteModule Consumption

Add remote module to external

Set Factory Object

Intercept request resolution for remote modules normalModuleFactory.hooks.factorize

Add the RemoteRuntimeModule

Shared Dependency

Consumption Sharing Dependency

Set Factory Object

Asynchronous entry:

Sync entry:

Provide shared dependencies

Add Include

Set Factory Object

Congrats You Made it!

Bundler Design Trade-offs: The Rationale Behind Creating Rspack

Before embarking on the development of Rspack, we explored various build tools and frameworks, including extensive use…

Written by Zack Jackson

Responses (4)

Reference shared dependencies ( `import react from 'react '`)

Reference remote module ( `import Button from 'component-app '` )

Exposing Modules: The Role of `ContainerPlugin`

Why is it Named `ContainerPlugin`?

Core Functions of `ContainerPlugin`

Add the `RemoteRuntimeModule`