ExadataX11MHardware Overview
ExadataX11MHardware Overview
1. What changed?
2. Exadata X11M overview
- Database server changes
- Storage server changes
- Memory options
- Networking options
- Configurations
3. OASG X9 vs E6-2L (Oracle Advanced Support Gateway)
4. Additional resources:
- Exadata Expansion Options presentation available in database.us.oracle.com
X11M Servers:
• Powerful new X11M Database and Storage Servers
X11M-Z Servers:
• X11M-Z Database Servers introduced
• X11M-Z Storage Servers introduced
Mix-and-Match*:
• Any combination (including –Z servers) of DB Servers and Storage Servers can be in the same Rack
Database Server compute performance is up to 25% faster than X10M due to faster chip, faster
memory, zero overhead for security, and faster virtualization
New memory DIMMs have 33% higher throughput: X10M @ 4800MT/s X11M @ 6400MT/s
• Local memory throughput per DB server: X10M 684GB/s X11M 1TB/sec
• Bi-directional memory throughput to remote socket stays the same as X10M: 568GB/s
RoCE network unchanged from X10M: 2 ports of 100Gbps using ConnectX-7 NIC
Faster Flash Card used for flash cache for up to 2X faster flash scan throughput
• Faster PCIe interface used in new flash card: PCIe Gen4 45GB/sec PCIe Gen5 100GB/sec
• Flash capacity unchanged
Exadata RDMA Memory (XRMEM) Cache uses faster memory DIMMs for up to 21% improved read latency
• Ultra-low latency OLTP IO 17us 14us
• Ultra-high throughput in-memory smart scan for analytics Up to 500GB/s
• Memory capacity unchanged: 1.25TB of XRMEM Cache per Storage Server
• OLTP IOPs unchanged: up to 2.8M database OLTP 8K block RDMA read IOPS
Hard drive raw capacity per High-Capacity Storage Server 22TB – same as X10M
RoCE network unchanged from X10M: 2 ports of 100Gbps using ConnectX-7 NIC
Network
• Faster 100Gb/s RoCE Network Cards; PCIe Gen5 from CX5-100G PCIe Gen4
Storage Server
• Faster 5th Generation AMD EPYC™ CPU – 32-core, 2.95 GHz (4.4 GHz boost) from 24-core 2.0 GHz (2.8 GHz
Boost) 8 more cores and 34% faster CPU!
• Faster Exadata RDMA Memory Accelerator (XRMEM) – 1.25 TB DDR5 6400 MT/s vs 1.5TB PMEM
• Faster memory – 256 GB DDR5 6400 MT/s from 256GB DDR4-3200 MT/s Data rate is doubled
• New performance-optimized F680 v2 6.8 TB NVMe PCIe Gen5 flash vs F640 PCIe Gen4 6.4TB
• New and increased storage capacity – 22 TB Hard Drives (up from 18 TB) 22% increase in storage capacity
- 264 TB per HC Storage Server, up from 216 TB
• Exadata X11M Database Machine uses x86 2-socket and 1-socket Database Servers, available in
two configurations:
• X11M Database Server
• Both CPU sockets populated with 96 cores
• Well suited for all workloads - OLTP, Analytics, Consolidation, Memory Intensive, horizontally and vertically
scalable database workloads and all multi-rack data workloads
• X11M-Z Database Server
• A single CPU socket populated with 32 cores
• Perfect for moderate workloads which do not require scalability or maximum performance
• 2x 96-core AMD EPYC™ 9J25 Processors • 1x 32-core AMD EPYC™ 9J15 Processors
• 512 GB (16x 32 GB) / 1.5 TB (24x 64 GB) / 2.25 TB (24x 96 GB) /
• 768 GB (12x 64 GB) / 1152 GB (12x 96 GB) 6400MT/s DDR5 DIMMs
3 TB (24x 128 GB) 6400MT/s DDR5 DIMMs (factory installable
(factory installable options, field upgradeable)
options, field upgradeable)
• 2x 3.84 TB NVMe Drives – expandable to 4x 3.84 TB (hot
• 2x 3.84 TB NVMe Drives – expandable to 4x 3.84 TB (hot swappable)
swappable)
• One dual-port QSFP28 100G RDMA Ethernet NIC (PCIe5.0) – both • One dual-port QSFP28 100G RDMA Ethernet NIC (PCIe5.0) – both
ports active – RoCE Network ports active – RoCE Network
• Two dual-port SFP28 10/25G ethernet NIC – Client Network • Two dual-port SFP28 10/25G ethernet NIC – Client Network
Up to 3 of the following (any combination): One of the following:
• Dual port QSFP28 100G ethernet NIC (field install) • Dual port QSFP28 100G ethernet NIC (field install)
• Dual-port SFP28 10/25G ethernet NIC (field install) • Dual-port SFP28 10/25G ethernet NIC (field install)
• Quad-port RJ45 10G ethernet NIC (field install) • Quad-port RJ45 10G ethernet NIC (field install)
• 1x ethernet port (ILOM) • 1x ethernet port (ILOM)
• 1x ethernet port (eth0) • 1x ethernet port (eth0)
• Redundant hot swappable power supplies and fans • Redundant hot swappable power supplies and fans
• 2x 96-core AMD EPYC™ 9J25 Processors • 1x 32-core AMD EPYC™ 9J15 Processors
• 1.5 TB (24x 64 GB) / 2.25 TB (24x 96 GB) / 3 TB (24x 128 GB)
• 768 GB (12x 64 GB) 6400MT/s DDR5 DIMMs
6400MT/s DDR5 DIMMs (factory installable options, field
(factory installable options, field upgradeable)
upgradeable)
• 4x 3.84 TB NVMe Drives • 2x 3.84 TB NVMe Drives – expandable to 4x 3.84 TB (hot swappable)
• One dual-port QSFP28 100G RDMA Ethernet NIC (PCIe5.0) – both • One dual-port QSFP28 100G RDMA Ethernet NIC (PCIe5.0) – both
ports active – RoCE Network ports active – RoCE Network
• Two dual-port SFP28 10/25G ethernet NIC – Client Network • Two dual-port SFP28 10/25G ethernet NIC – Client Network
Up to 1 of the following (any combination):
Up to 1 of the following:
• Dual-port SFP28 10/25G ethernet NIC (field install)
• Quad-port RJ45 10G ethernet NIC (field install)
• Quad-port RJ45 10G ethernet NIC (field install)
• Redundant hot swappable power supplies and fans • Redundant hot swappable power supplies and fans
• 1.25 TB Exadata RDMA Memory • 1.25 TB Exadata RDMA Memory • 576 GB Exadata RDMA Memory
• 2x 480 GB M.2 SSD NMVe • 2x 480 GB M.2 SSD NMVe • 2x 480 GB M.2 SSD NMVe
• 12x 22 TB 7200rpm hard disk drives • 4x 30.72 TB capacity-optimized NVMe Flash • 6x 22 TB 7200rpm hard disk drives
• 4x 6.8 TB F680 NVMe PCIe5.0 Flash Cards • 4x 6.8 TB F680 NVMe PCIe5.0 Flash Cards • 2x 6.8 TB F680 NVMe PCIe5.0 Flash Cards
• 1x dual-port QSFP28 100Gb/s RDMA Network • 1x dual-port QSFP28 100Gb/s RDMA Network • 1x dual-port QSFP28 100Gb/s RDMA Network
Fabric Adapter Fabric Adapter Fabric Adapter
• 2x 32-core 5th Generation AMD EPYC™ processors • 1x 32-core 5th Generation AMD EPYC™ processors
• 4x 6.8 TB F680 NVMe PCIe5.0 Flash Cards • 2x 6.8 TB F680 NVMe PCIe5.0 Flash Cards
• 1x dual-port QSFP28 100Gb/s RDMA Network Fabric Adapter • 1x dual-port QSFP28 100Gb/s RDMA Network Fabric Adapter
• Notes:
- No option to expand memory on Exadata X11M Storage Servers
- Previous generation memory is not compatible with X11M (and vice versa)
The Database Server X11M-Z is a single-socket server that ships with either 768 GB (12x 64 GB) or 1.125 TB (12x 96
GB). Only the 768 GB server can be expanded.
• Expand the memory configuration of the Database Server X11M-Z
- From 768 GB to 1.125 TB per Database Server using 96 GB DIMMs (1x kit for two servers) – replace existing
- Qty 1: 7627327 - Exadata X11M Memory Expansion Kit - Twenty four 96 GB DIMMs
• Notes
- No option to expand memory on Exadata X11M-Z Storage Servers
- Previous generation memory is not compatible with X11M (and vice versa)
• Exadata X11M Database Servers come with Dual 25G RoCE Network Dual 25G
• Exadata X11M-Z Database Servers come with RoCE Network Dual 100G / Dual 25G / Quad 10G
Dual 25G
Note: If additional Quad 10G Network Cards are required, existing Dual 25G Cards can be replaced
CPU and Memory upgrade kit to convert Database Server X11M-Z to X11M Database Server
• Replace the existing 32-core processor with 2x 96-core processors per server. Order the following parts:
- Qty 1: 7627323 - CPU upgrade kit for Exadata Database Machine X11M-Z to X11M DB Server Rack Upgrade
- Qty 1: 7627324 - Heat Sink Upgrade Kit for Exadata Database Machine X11M-Z to X11M DB Server Rack Upgrade
- For Memory:
- Qty 1: 7627326 Exadata X11M Memory Expansion Kit - Twenty four 64 GB DIMMs. Install 12 per server.
- Or Qty: 2 of the following upgrade kits:
- 7627327 Exadata X11M Memory Expansion Kit - Twenty four 96 GB DIMMs
- 7627328 Exadata X11M Memory Expansion Kit - Twenty four 128 GB DDR5 DIMMs
• Notes:
- Both database servers need to be upgraded at the same time. The CPU and heat sink upgrade kits include four components, for two DB
servers. The memory kit includes 24 DIMMs; each X11M-Z server
- The X11M-Z database server ships with a minimum of 768GB Memory (12x64). If the customer desires to upgrade from 768 to 1.5TB,
purchase Qty:1 of the 64 GB Kit and add 12 DIMMs to each server (there are already 12 DIMMs present). Once done, each server will have
24x64 GB = 1.5 TB. If higher capacity is desired, purchase the 96 GB or 128 GB DIMM upgrade kits, one per server.
Exadata Database and Storage Servers X11M and X11M-Z are available for X11M-Z capacity expansion
•
24
Maximum 15x X11M-Z Database Servers (768 GB and 1.125 TB Memory) 23
22
•
14
Up to 15 additional HC, HC-Z or EF Storage Servers 13
12
11
10
09
08
07
06
05
Use OECA (Oracle Exadata Configuration Assistant) to create your desired Exadata configuration. Maximum configurations will vary 04
depending on server memory capacity, power requirements, PDU capacity, etc. OECA will ensure that your configurations are valid. 03
02
https://www.oracle.com/database/technologies/oeca-download.html 01
Q: Why should I use OECA to model my Exadata configuration? Q: Can I mix 2.25 or 3TB memory database servers with 512GB or
- In addition to facilitating the scoping and analyzing of elastic 1.5TB memory servers?
configurations, OECA also validates your Exadata configuration on power - Yes, it is possible. Months or years after the Exadata purchase, customer
requirements, PDU size, weight distribution, etc. requirements may change, and some servers need to be upgraded to 2.25TB
or 3TB memory. Use OECA to model your desired configuration and ensure
Q: Should I model the Exadata configuration I want to purchase in that it is OK with your current rack.
OECA?
- Yes, you must. It will provide you with detailed information on weight, Q: Are the X11M PDU Cabling Tables the same as Exadata prior
usable capacity, PDU size to order, power and cooling requirements, etc. generations?
With this information, the quoting and booking in CPQ/Configurator will go - No, the X11M PDU tables are updated to reflect the greater power demands
smoothly. of the X11M database and storage servers.
Q: Why are there four rack options in OECA for Exadata X10M &
X11M? Prior generations only had one rack option.
- To accommodate the database server with different and larger memory
configurations which have different power requirements.
CPU 2 x Intel 2.3GHz 16C IceLake CPU 1 x AMD 3.4Ghz 32C Turin CPU
Memory 16 x 16GB DDR4 32000 (256GB total) 4 x 32GB DDR5 6400 (128GB)