rtdip · GBBBAS · Aug 11, 2023 · Aug 10, 2023
diff --git a/docs/domains/smart_meter/data_model.md b/docs/domains/smart_meter/data_model.md
@@ -0,0 +1,7 @@
+# Meters Data Model
+
+Base Raw To Meters Data Model: [here](/rtdip/core/sdk/code-reference/pipelines/transformers/spark/base_raw_to_mdm/)
+
+* ISO:
+    * MISO To Meters Data Model: [here](/rtdip/core/sdk/code-reference/pipelines/transformers/spark/iso/miso_to_mdm/)
+    * PJM  To Meters Data Model: [here](/rtdip/core/sdk/code-reference/pipelines/transformers/spark/iso/pjm_to_mdm/)
diff --git a/docs/domains/smart_meter/overview.md b/docs/domains/smart_meter/overview.md
@@ -0,0 +1,46 @@
+# Smart Meter / Energy Domain
+## Overview
+Meter data is central to accelerating the electrification and decarbonisation of the energy grid. RTDIP provides the ability to consume meter data from exemplar sources, transform it and store it in an appropriate open-source format to enable domain-specific energy services, such as:
+
+* Energy Load Forecasting
+* Energy Generation Forecasting
+* Other behind-the-meter services and insights
+
+At a high level, the electricity system (US example) works as follows:
+
+* **Generators**, of various types (coal, oil, natural gas, nuclear, wind turbines & PV, etc.) produce electricity 
+* **Utilities**, distribute and transmit the electricity from the Generators through the grid to the  point of consumption i.e. buildings and homes
+* **Suppliers**, wholesale purchase the electricity and sell it as retail contracts to Buyers
+* **Buyers**, consume electricity, via buildings, homes, and electric vehicles, etc.
+* **Consultants**, facilitate these transactions and/or offer data insights e.g. load forecasting to tailor purchasing, targeting reduced risk, profit, and competitive costs for Buyers
+
+An **Independent System Operator (ISO)** sometimes called the Regional Transmission Organisation (RTO) is an organisation that is in charge of the entire process. They coordinate, control, and monitor the electric grid in a specific region, typically a multi-state area.
+
+## Meter Data Pipelines
+Load forecasting is a technique used by ISO's, and energy-providing companies to predict the power/energy needed to meet the demand and supply equilibrium of the energy grid. RTDIP defines and provides example pipelines for the two primary inputs to energy services like load forecasting, namely [weather](/rtdip/core/domains/weather/overview/) and meter data.
+
+Specifically, with respect to meter data RTDIP defines and provides two exemplar ISO's:
+
+* the Midcontinent Independent System Operator, [MISO](https://www.misoenergy.org/about/)   
+* the PJM Interconnection LLC Independent System Operator, [PJM](https://www.pjm.com/about-pjm)
+
+## Architecture
+
+The overall ETL flow of the pipeline is outlined below:
+
+``` mermaid
+graph LR
+  A(External Meter Source e.g. MISO, PJM) --> B(RTDIP Source/Connector);
+  B --> C(RTDIP Transformer);
+  C --> D(RTDIP Source/Connector);
+  D --> E[(RTDIP Destination)];
+```
+
+1. Source: The specific source/connector acquires data from a specific external endpoint (MISO or PJM) and persists the raw data into Deltalake 
+2. Transformer:  An RTDIP transformer translates this raw data into a meter specific Delta schema.  
+3. Destination: Essentially the function of loading is abstracted from the user and is handled by Deltalake. 
+
+Indicative schema are available [here](data_model.md). 
+
+# Real Time Data Ingestion Platform
+For more information about the Real Time Data Platform and its components to connect to data sources and destinations, please refer to this [link](../../sdk/overview.md).
diff --git a/docs/domains/weather/overview.md b/docs/domains/weather/overview.md
@@ -1,6 +1,6 @@
 
-# Weather Services Overview
-
+# Weather Services 
+## Overview
 Many organizations need weather data for day-to-day operations. RTDIP provides the ability to consume data from examplar weather sources, transform it and store the data in an appropiate open source format to enable generic functions such as:
 
 * Data Science, ML and AI applications to consume the data
@@ -13,17 +13,17 @@ A primary aim for RTDIP in 2023 is to demonstrate how the platform can be utilis
 * Energy Generation Forecasting
 * Other behind the meter services and insights
 
-Load forecasting is a technique used by power or energy-providing companies to predict the power/energy needed to meet the demand and supply equilibrium. Weather data is a primary driver of variance in load forecasting and energy generation forecasting in renewable energy sources. 
+Weather data is a primary driver,  together with [meter](/rtdip/core/domains/smart_meter/overview/) data, of variance in load & generation forecasting in the energy domain. 
 
-## Weather Data in Energy Domain
+## Weather Data in the Energy Domain
 
 One of the most widely used weather data standards is the combined METAR (Meteorological Aerodrome Report) and ICAO (International Civil Aviation Organization) standard. This standard is used by meteorological agencies and aviation organizations around the world to report weather conditions at airports and other aviation facilities. This standard is broadly utilised beyond the aviation industry including the energy domain. 
 
 The METAR ICAO standard includes a set of codes and abbreviations that describe weather conditions in a standardized format. These codes include information such as temperature, wind speed and direction, visibility, cloud cover, and precipitation. The standard also includes codes for reporting special weather phenomena, such as thunderstorms or volcanic ash.
 
 Many actors in the energy domain utilise Historical, Forecast and near real-time METAR data as part of their services. Such data can be used to calculate average weather data by date and interval spanning multiple years, eg Historical Weather Data is often used to calculate an average or typical value for each weather variable eg. temperature, humidity over a given timeframe, which can be used for long range forecasting etc. 
 
-## Weather Data Pipeline Architecture
+## Architecture
 
 An exemplar pipeline is defined and provided within RTDIP. The overall approach and weather data in general is agnostic but the exemplar utilises a specific external source. The overall ETL flow of the pipeline is outlined below:
 

diff --git a/mkdocs.yml b/mkdocs.yml
@@ -111,6 +111,9 @@ nav:
       - Process Control: 
           - Overview: domains/process_control/overview.md
           - Data Model: domains/process_control/data_model.md
+      - Smart Meter Data: 
+          - Overview: domains/smart_meter/overview.md
+          - Data Model: domains/smart_meter/data_model.md
       - Weather Data: 
           - Overview: domains/weather/overview.md
           - Data Model: domains/weather/data_model.md