Oracle Exadata Database Machine
🎯Components of Database Machine:
✅Exadata Software Architecture
✅Compute Node/ Database Server
✅Network
✅Cell Node/ Storage Server
🎯Oracle Exadata Architecture:
Exadata Database Machine provides a highly available, high-performance platform for Oracle Database based on the clustered architecture with Oracle RAC support. Exadata Database Machine is also available in specific configurations, such as a full rack, half rack, quarter rack or eighth rack.
Key Components Include:
✔️Powerful and well-balanced Database Servers joined in a cluster.
✔️Powerful and intelligent Shared Storage provided by Exadata Cell Server
✔️ Storage mirroring provided by ASM
✔️High bandwidth and low latency Cluster Interconnect and storage networking provided using ultra-fast RoCE network.
🎯The Exadata Database Machine Software Architecture includes components on the Database Servers and on the Exadata Cells. The overall architecture is illustrated in the above diagram.
🎯Software components that reside in each Database Server:
🔵 Oracle Exadata Database Machine X8-M-2 comes with Oracle Linux KVM is the virtualization technology for systems that use RoCE Network Fabric. The ability to run Oracle Linux without OVM remains, and is the only option for 8-socket Database Servers.
🔵Exadata X8M Database Servers: can run Oracle Databae 11g Release 2, 12c, 18c, and 19c. The database servers can be configured with all supported Oracle Database software releases; however, Oracle Grid Infrastructure 19c is required to support Oracle Database 19c. The precise patch release of Oracle software must be compatible with the Exadata Storage Server software and other Exadata software components. My Oracle Support note 888828.1 contains an up-to-date list of supported versions for the Exadata software components.
🔵 Automatic Storage Management (ASM): is required and provides a file system and volume manager optimized for Oracle Database.
🔵LIBCELL: Oracle Database communicates with Exadata cells by using a special library called LIBCELL. In combination with the database kernel and ASM, LIBCELL transparently maps database I/O operations to Exadata Storage Server enhanced operations. LIBCELL communicates with Exadata cells by using the Intelligent Database protocol (iDB).
🔵iDB Protocol: The iDB protocol is a unique Oracle data transfer protocol that serves as the communications protocol among Oracle ASM, database instances, and storage cells.
🔵DBRM & IORM: The Database Resource Manager (DBRM) is integrated with Exadata Storage Server I/O Resource Management (IORM). DBRM and IORM work together to ensure that I/O resources are allocated based on administrator-defined priorities.
🔵Diskmon: It checks the storage network interface state and cell liveness. It also performs DBRM plan propagation to Exadata cells. Diskmon uses a node-wide master process (diskmon.bin) and one slave process (DSKM) for each RDBMS or ASM instance. The master performs the monitoring and propagates state information to the slaves. The slaves use the SGA to communicate with the RDBMS or ASM processes. If there is a failure in the cluster, Diskmon performs I/O fencing to protect data integrity. Cluster Synchronization Services (CSS) decides what to fence. The master Diskmon starts with the clusterware processes. The slave Diskmon processes are background processes that are started and stopped in conjunction with the associated RDBMS or ASM instance.
🔵The Management Server (MS) on a database server provides a set of management and configuration functions. It works in cooperation with the DBMCLI command-line interface. Each database server is individually managed with DBMCLI. DBMCLI can be used only from within a database server to manage that server. However, you can run the same DBMCLI command remotely on multiple nodes with the dcli utility. In addition, MS is responsible for sending alerts, and collects some statistics.
🔵 The Restart Server (RS) is used to start up or shut down MS and monitors MS to automatically restart it if required.
🎯Software components that reside in each Exadata Cell:
Oracle Linux x86_64 provides the Exadata Storage Server operating system.
🔵 Cell Server (CELLSRV) is the primary Exadata Storage Server software component and provides the majority of Exadata storage services. CELLSRV is a multithreaded server. It serves simple block requests, such as database buffer cache reads, and Smart Scan requests, such as table scans with projections and filters. CELLSRV also implements IORM, which works in conjunction with DBRM to meter out I/O bandwidth to the various databases and consumer groups that are issuing I/Os. Finally, it collects numerous statistics relating to its operations. Oracle Database and ASM processes use LIBCELL to communicate with CELLSRV, and LIBCELL converts I/O requests into messages that are sent to CELLSRV by using the iDB protocol.
Get Cloud Evolving Data’s stories in your inbox
Join Medium for free to get updates from this writer.
🔵 The Management Server (MS) provides Exadata cell management and configuration functions. It works in cooperation with the Exadata cell command-line interface (CellCLI). Each cell is individually managed with CellCLI. CellCLI can be used only from within a cell to manage that cell.
However, you can run the same CellCLI command remotely on multiple cells with the dcli utility.
In addition, MS is responsible for sending alerts, and collects some statistics in addition to those collected by CELLSRV.
🔵The Restart Server (RS) is used to start up or shut down the CELLSRV and MS services, and monitors these services to automatically restart them if required.
🎯Exadata Database Machine:
It provides a resilient, high-performance platform for clustered and non-clustered implementations of Oracle Database. The fundamental architecture underpinning. Exadata is the same core architecture that underpins Oracle Real Application Clusters (RAC) software.
Following are the key elements of the Exadata architecture.
🎯Shared Storage:
Exadata provides intelligent, high-performance shared storage to both single-instance and RAC implementations of Oracle Database using Exadata Storage Server technology. Storage supplied by Exadata Storage Servers is made available to Oracle databases using the Automatic Storage Management (ASM) feature of Oracle Database. ASM adds resilience to Exadata storage by providing a mirroring scheme that can be used to maintain redundant copies of data on separate Exadata Storage Servers. This protects against data loss if a storage server is lost. Normal protection ensures that two copies of data are maintained on
separate Exadata Storage Servers while high level protection maintains three copies of data on three separate Exadata Storage Servers.
🎯Storage Network:
Exadata X8-M contains a storage network based on Remote Direct Memory Access over Converged Ethernet (RoCE). This provides high bandwidth and low latency access to the Exadata Storage Servers. Fault tolerance is built into the network architecture through the
use of multiple redundant network switches and network interface bonding•Server cluster: The database servers in Exadata are designed to be powerful and well balanced so that there are no bottlenecks within the server architecture. They are equipped with all of the components required for Oracle RAC, enabling customers to easily deploy Oracle RAC across a
single Exadata Database Machine. Where processing requirements exceed the capacity of a single Database Machine, customers can join multiple Database Machines together to create a single unified server cluster.
🎯Cluster Interconnect:
The high bandwidth and low latency characteristics of the ultra-fast RoCE network are ideally suited to the requirements of the cluster interconnect. Because of this, Exadata X8M is configured by default to also use the RDMA network fabric as both the storage network and the cluster interconnect.
🎯Shared Cache:
In a RAC environment, the instance buffer caches are shared. If one instance has an item of data in its cache that is required by another instance, that data will be shipped to the required node using the cluster interconnect. This key attribute of the RAC architecture
significantly aids performance because the memory-to-memory transfer of information via the interconnect is significantly quicker than writing and reading the information using disk. With Exadata, the shared cache facility uses the InfiniBand-based high-performance cluster interconnect.
🎯Remote Direct Memory Access (RDMA):
It has the ability for one computer to access data from a remote computer without OS or CPU involvement. Remote Direct Memory Access (RDMA) is the ability for one computer to access data from a remote computer without OS or CPU involvement: the network card directly reads/writes memory with no extra copying or buffering resulting in very low latency. RDMA was introduced to Exadata with InfiniBand and is a foundational part of Exadata’s high-performance architecture. RDMA enables several unique Exadata features, such as Direct-to-Wire Protocol and Smart Fusion Block Transfer.
🎯RDMA over Converged Ethernet (RoCE):
RoCE is a set of protocols defined by an open consortium, developed in open source, and maintained in upstream Linux. RoCE’s protocols enable InfiniBand RDMA software to run on top of Ethernet. This allows the same software to be used at the upper levels of network protocol stack, while transporting the InfiniBand packets across Ethernet as UDP over IP at the lower level. Because the API infrastructure is shared between InfiniBand and Ethernet, all existing InfiniBand RDMA benefits are also available on RoCE, including over a decade’s worth of performance engineering on Exadata.
Adopters of RoCE can benefit from RDMA’s capabilities without changing their network infrastructure.
• All Ethernet NICs require RoCE network adapter cards.
• Exadata takes care of all performance and tuning work under the RoCE configuration and nothing is required by the user
As its name shows, RoCE is a network protocol defined in the “InfiniBand Trade Association(IBTA)” standard, allowing RDMA over converged Ethernet network. Shortly, it can be regarded as the application of RDMA technology in hyper-converged data centers, cloud, storage, and virtualized
environments. It possesses all the benefits of RDMA technology and the familiarity of Ethernet.
Exadata takes care of all performance and tuning work under the RoCE configuration and nothing is required by the user.
🎯Exadata Network Architecture:
Starting with Oracle Exadata System Software release 19.3 and Oracle Exadata Database Machine X8M, RDMA over Converged Ethernet (RoCE) is used for the RDMA Network Fabric instead of InfiniBand. RDMA over RoCE Network Fabric uses different switches and cables from those used
by RDMA over InfiniBand Network Fabric.
To deploy Oracle Exadata Database Machine, ensure that you meet the minimum network requirements. Oracle Exadata Database Machine requires a minimum of three networks, and there are interfaces available for additional networks. Each network must be on a distinct and separate subnet from the others. Oracle recommends one common additional network for backup, which you can configure with OEDA.
Management Network: This required network connects to your existing management network, and is used for administrative work for all components of Oracle Exadata Database Machine. It connects the servers, Integrated Lights Out Manager (ILOM), and switches connected to the Ethernet switch in the rack.
Each Database Server and Storage Server has two network interfaces for management. One provides management access to the operating system through the embedded NET0 Ethernet port on Oracle Exadata Database Machine Two-Socket systems, or the NEM0/NET1 Ethernet port on Oracle Exadata Database Machine Eight-Socket systems. The other network interface provides access to the ILOM through the NET MGT Ethernet port. Oracle Exadata Database Machines are delivered with the NET MGT and NET0 or NEM0/NET0 ports connected to the Ethernet switch in the rack. The NET0 or NEM0/NET0 port on the database servers should not be used for client or application network traffic. Cabling or configuration changes to these interfaces on Oracle Exadata Storage Servers is not permitted.
Please visit my previous posts for more details about the Exadata Features.
⚫⚫⚫
