🪟Oracle Exadata — “Flash Cache in Nutshell”
Exadata High Capacity Storage Server contains 12 hard disk drives. Exadata cell software automatically senses the “Physical Disks” in each storage server. As a cell administrator you can only view a predefined set of physical disk attributes. Each physical disk is mapped to a logical abstraction called a “Logical Unit” (LUN). A LUN exposes additional predefined metadata attributes to a cell administrator. You cannot create or remove a LUN; they are automatically created.
A “Cell Disk” is a higher level abstraction that represents the data storage area on each LUN. After a Cell Disk is created, it can be subdivided into one or more “Grid Disks”, which are directly exposed to “ASM”.
Placing multiple grid disks on a cell disk allows the administrator to segregate the storage into pools with different performance characteristics.
Exadata X8M-2 High Capacity Storage Server contains 25.6 TB of high performance flash memory distributed across four PCI flash memory cards. Therefore, each flash card has a capacity of 6.4 TB.
Essentially, each flash device is like a physical disk in the storage hierarchy. Each flash device is visible to the Exadata cell software as a LUN and the initial cell configuration process creates flash-based cell disks on all the flash devices. By default, nearly all the available space on each flash-based cell disk is allocated to Exadata Smart Flash Cache and only a small portion is allocated to Exadata Smart Flash Log.
✅Flash Storage Entities:
Exadata X8M-2 Extreme Flash Storage Servers contain eight high-performance flash drives. Each flash drive has a capacity of 6.4 TB. Like Exadata High Capacity Storage Server disks and flash devices, each flash drive is visible to the Exadata cell software as a LUN, and each LUN can contain a cell disk. By default, 95% of the available space on each Extreme Flash cell disk is used for data storage. This space is allocated to grid disks that are exposed to ASM in the same way as the disk space from a high capacity storage server. Nearly all of the remaining 5% is allocated to Exadata Smart Flash Cache, and a small amount is allocated to Exadata Smart Flash Log.
✅Exadata Smart Flash Cache:
✔️ High-performance cache for frequently accessed objects.
✔️ Write-through and write-back modes available.
✔️ Write-through mode is excellent for absorbing repeated random reads.
✔️ Write-back mode is best for write intensive workloads.
✔️ Allows optimization by application table.
Get Cloud Evolving Data’s stories in your inbox
Join Medium for free to get updates from this writer.
For many years, a constraining factor for storage performance has been the number of random I/Os per second (IOPS) that a disk can deliver. To compensate for the fact that even a high-performance disk can deliver only a few hundred IOPS, large storage arrays with hundreds of disks are required to deliver in excess of 100,000 IOPS.
Exadata Storage Server provides Exadata Smart Flash Cache, a caching mechanism for frequently accessed data. Using Exadata Smart Flash Cache, an Exadata cell can support approximately 1,500,000 read IOPS, two cells can support approximately 3,000,000 read IOPS, and so on. Exadata Smart Flash Cache focuses on caching frequently accessed data and index blocks, along with performance critical information such as control files and file headers. In addition, DBAs can influence caching priorities by using the CELL_FLASH_CACHE storage attribute for specific database objects.
🎯Exadata Smart Flash Cache Intelligent Caching:
Exadata Smart Flash Cache understands different types of database I/O:
⚫With Cache:
Frequently accessed data and index blocks are cached.
✔️ Control file reads and writes are cached.
✔️ File header reads and writes are cached.
✔️ DBA can influence caching priorities.
⚫ Without Cache:
✔️ Backup-related I/O is not cached.
✔️ Data Pump I/O is not cached.
✔️️ Table scans do not thrash the cache.
✔️ I/Os to mirror copies are managed intelligently
🎯How Intelligent Cachng works ??
Caching is automatically governed by:
A cache hint, based on the reason for the I/O:
✔️ CACHE indicates that the I/O should be cached.
✔️ NOCACHE indicates that the I/O should not be cached.
✔️ EVICT calls for data to be removed from the cache.
✔️ Object size and frequency of access, versus the frequency of access for other cached objects.
✔️ The most useful data is prioritized and the least useful data is evicted.
✔️ Automatic caching priorities can be manually adjusted by setting the
CELL_FLASH_CACHE storage attribute
✔️ Exadata Smart Flash Cache automatically works in conjunction with Oracle Database to intelligently optimize the efficiency of the cache. Each database I/O is tagged with a cache hint, which is assigned by the database based on the reason for the I/O:
• CACHE indicates that the I/O should be cached. For example, the I/O is for an index lookup.
• NOCACHE indicates that the I/O should not be cached. For example, the I/O is for a mirrored block of data or is associated with a backup.
• EVICT indicates that data should be removed from the cache. For example, when an ASM rebalance operation moves data between different disks, the cached copies that correspond to the original location are removed from the cache.
✔️ Exadata Smart Flash Cache also uses internal statistics and other measures to determine whether or not an object (table or index) should be cached. It takes into account the size of the object and the frequency of access to the object, and weighs that against the frequency of access to other cached objects to determine which objects to prioritize. Objects can be fully or partially cached, depending on the object size, flash cache size, and the other concurrent workloads. This behaviour is fully automatic and designed to intelligently maximize the benefits associated with Exadata Smart Flash Cache.
✔️ In addition to the automatic caching algorithms, administrators can manually influence caching priorities by setting the CELL_FLASH_CACHE storage attribute for specific tables and indexes. The options are:
✔️ DEFAULT specifies that the object is cached normally.
✔️ KEEP specifies that the object has a higher than default caching priority, making it more likely that the object will be cached.
✔️ NONE ensures that the object is never cached in Exadata Smart Flash Cache. This allows flash cache space to be reserved for other objects.
🎯Write-through mode Exadata Smart Flash Cache works:
✔️ For read operations, CELLSRV must first determine if the request should use the cache. This decision is based on various factors including the reason for the read, the CELL_FLASH_CACHE setting for the associated object, and the current load on the cell. If it is determined that the cache should be used, CELLSRV uses an in-memory hash table, to quickly determine if the data resides in Exadata Smart Flash Cache. If the requested data is cached, a cache lookup is used to satisfy the I/O request.
✔️ For read operations that cannot be satisfied using Exadata Smart Flash Cache, a disk read is performed and the requested information is sent to the database. Then if the data is suitable for caching, it is written to Exadata Smart Flash Cache.
✔️ For write operations, CELLSRV writes data to disk and sends an acknowledgement to the database so it can continue without any interruption. Then, if the data is suitable for caching, it is written to Exadata Smart Flash Cache. Write performance is not improved or diminished using this method. However, if a subsequent read operation needs the same data, it is likely to benefit from the cache. When data is inserted into a full cache, a prioritized least recently used (LRU)
algorithm determines which data to replace.
Note that in write-through mode, multiple copies of each data block are written to disk to maintain ASM redundancy. As a result, there is usually no need to cache the secondary block copy because ASM will read the primary copy if it is available.
By default, Smart Flash Cache operates in write-through mode on high capacity storage servers.
🎯How Conventional Write: Write-Back Cache Works ??
◾ Suitable writes go to flash only.
◾ Data is automatically written to disk as it ages out of the cache.
◾ Active data blocks can reside in flash indefinitely.
◾ Reads are handled the same way as in write-through mode.
🔷Characteristics:
◾Ideal for write-intensive applications.
◾ For many applications, most I/O is serviced by flash.
◾If a problem is detected, I/O operations transparently fail over to mirrored copies of data also on flash.
🎯Exadata Smart Flash Cache can Operate in Write-Back Mode:
✔️ CELLSRV receives the write operation and uses intelligent caching algorithms to determine if the data is suitable for caching.
✔️ If the data is suitable for caching, it is written to Exadata Smart Flash Cache only. If the cache is full, CELLSRV determines which data to replace using the same prioritized least recently used (LRU) algorithm as in write-though mode.
✔️ After the data is written to flash, an acknowledgment is sent back to the database.
✔️ Data is only written back to disk when it is aged out of the cache.
Note the following regarding write-back flash cache:
• Write-back flash cache is ideal for write-intensive applications that would otherwise saturate the disk controller write cache.
• The large flash capacity of Exadata Storage Server means that for many applications a high proportion of all I/O can be serviced by flash.
• An active data block can remain in write back flash cache for months or years. Also, Exadata Smart Flash Cache is persistent through power outages, shutdown operations, cell restarts, and so on.
With write-back flash cache, data redundancy is maintained by writing primary and secondary data copies to cache on separate storage servers.
✔️ Secondary block copies are aged out of the cache (and written to disk) more quickly than primary copies. Therefore, blocks that have not been used recently only keep the primary copy in cache, which optimizes the utilization of flash space.
✔️ If there is a problem with the flash cache on one storage server, then operations transparently fail over to the mirrored copies (on flash or disk) on other storage servers. No user intervention is required. The unit for mirroring is the ASM allocation unit. This means that the amount of data
affected is proportional to the lost cache size, not the disk size.
✔️ With write-back flash cache, read operations are handled the same as with the write-through flash cache.
🎯Conventional Log Write:
Exadata Smart Flash Log provides a high-performance, low-latency, reliable temporary store for redo log writes:
◾ Log writes are directed to disk and Exadata Smart Flash Log.
◾ Processing continues after fastest acknowledgment.
◾ Conceptually similar to multiplexed redo logs.
◾ Exadata Storage Server automatically manages Smart Flash Log and ensures all log entries are persisted to disk.
⚫Exadata Smart Flash Log: It provides a mechanism for improving the latency of database redo log write operations. Exadata Smart Flash Log uses a small portion of the high-performance flash memory on Exadata Storage Server as temporary storage to provide low latency redo log writes.
Using Exadata Smart Flash Log, log records are written simultaneously to the disk-based online redo log files and Exadata Smart Flash Log, and processing continues upon acknowledgment of the faster write operation, whether it is to hard disk or Exadata Smart Flash Log. When the hard disks containing the online redo log files experience temporary performance issues, such as heavy I/O loads, the write operations to Exadata Smart Flash Log will typically complete faster and provide consistently good response times.
Exadata Smart Flash Log is conceptually similar to multiplexed online redo log files, which have been available in Oracle Database for many years. The main difference is that Exadata Smart Flash Log uses flash memory only for temporary storage of redo log data. By default, Exadata Smart Flash Log uses 512 MB of flash storage on each Exadata Storage Server.
🪟Exadata Storage Server Statistics:
Numerous cell-specific statistics are recorded in V$SYSSTAT and V$SESSTAT. These statistics can be used to monitor Exadata Storage Server operations at both the system and session level. The statistics can be used to monitor the effectiveness of Smart Scan. There are also statistics relating to Exadata Smart Flash Cache, Hybrid Columnar Compression, storage index, fast file creation, and optimized incremental backups. In addition, other statistics provide the total volume of I/O exchanged over the interconnect and the total volume of physical disk reads and writes. The slide lists a selection of the available statistics.
The query in the slide shows key cell statistics for the current session. Examples of the output from this query are shown on the following pages.
In addition, V$SQL lists statistics on shared SQL areas. It contains statement-level statistics for the volume of physical I/O (reads and writes), the volume of I/O exchanged over the interconnect, along with information relating to the effectiveness of Smart Scan and other Exadata Storage Server features.
⚫⚫⚫
