[Paper Review] Reinforcement Learning-Based SLC Cache Technique for Enhancing SSD Write Performance

Paper Info

Titile: Reinforcement Learning-Based SLC Cache Technique for Enhancing SSD Write Performance Publisher: HotStorage’20 Initial release: July, 2020

읽은 이유

Reading papers in my field of interest.

Scopes of Work

QLC.
SLC Cache size.
Hot/Cold separation.

Summary

Propose a reinforcement learning-based SLC cache managment technique.
By observing workload pattern and internal status of hybrid SSD, it determines the optimal SLC cache parameters maximizing the efficiency of hybrid SSD.
Improves write throughput and write amplification factor.

Motivation Problem

Quad-level-cell (QLC)-based SSDs adopt a hybrid SSD architecture with fixed design factors.

QLC has a higher dendity, and lower cost compared to SLC or TLC.
However, QLC has slower performance and lower endurance.
Most recent QLC-based SSDs adopt a hybrid SSD architecture which ahs a partitioned SLC region for hideing the slow performance.
Two important factors for a hybrid SSD.
- The size of the SLC region It must be determined considering the trade-off between capacity loss and SLC-to-QLC block migration overhead.
Large SLC region -> the total capacioty of SSD is reduced.
Small SLC region -> high migration costs while incrasing the latency of write requests and the write amplification ratio.
- the hot/cold separation threshold.
Considering the SLC-to-QLC migration cost, it is better to write only frequently-updated data at the SLC region.
To discriminate between hot data and cold data, simple approach is to use the data size since small data tend to be frequently-updated.
Lower threshold -> SLC traffic decreased, migration cost decreases, write performance degradation.
Must be determined by considering the wokload patterns and the internal behavoir.

However, the existing techniques use heuristically determined fixed design factors and do not adjust the factors at run time.

The performacne of current dynamic hybrid SSDs can be degraded under unexamined or variable workloads.

Need a more intelligent algorithm to adjust the SLC cache configureation considering the dynamically changin the system states.

However, it is hard to design and tune an alggorithm which can effectively handle the various system states of hybrid SSD under dynamically changing user wokloads.

Key Idea

Propose a reinforcement learning-based SLC cache management technique for hybrid SSDs.

Without any prior knowledge about user workload or storage chracteristics, the RL-based approach can learn to find the optimal management policy.

RL-based Dynamic SLC cache

Action: +/- SLC cache size, +/- hot-cold threshold.
Every time step: predetermined cumulative amount of host write requests.The size of eight SLC blocks.
State (#bin): SLC cache size(9), Space utilization(4), Previous action(9), Demand for SLC writes(2), Update write frequency in SLC cache(2).
Reward:alogrithm ahtat gets several write-related costs and space utilization as inputs.

Note for Remembering

The latency of a write request will be differenct depending on the program mode (SLC or QLC).
The host write request to the SLC region can be delayed by SLC-to-QLC migration operations and QLC garbage collection operations.
The host write request to the QLC region can be delayed by QLC garbage collection.

The CPU and DRAM overheads of RL are much smaller compared to the cost of flash memory operations.
Therefore, ignored the overhead of the agent in the simulation.